Dr Moncrieff (Reference Moncrieff2003) has suggested that the advantage of clozapine in treatment-resistant schizophrenia, when compared with conventional antipsychotics, may not be substantial. This appears to be discordant with an earlier meta-analysis (Reference Wahlbeck, Cheine and EssaliWahlbeck et al, 2000). As clozapine's advantage in treatment-resistant schizophrenia is well accepted in psychiatry and is reflected in most practice guidelines, any questions about its validity need careful scrutiny. Clues to the disagreement between meta-analyses on the same topic can often be found in the studies that are included or excluded, the ways in which the data are abstracted and in the interpretation of the results (Reference Jadad, Cook and BrowmanJadad et al, 1997).

Dr Moncrieff included two studies in her analysis that were not in the earlier meta-analysis: Essock et al (Reference Essock, Hargreaves and Covell1996) and Kane et al (Reference Kane, Marder and Schooler2001).

The Essock et al (Reference Essock, Hargreaves and Covell1996) study was a naturalistic study with serious methodological deficiencies from the perspective of determining efficacy of clozapine treatment. The randomisation was imperfect. The study was not blinded. The study population was poorly defined in terms of diagnosis. Later application of the Structured Clinical Interview for DSM–III–R Personality Disorders to a subgroup of the study population picked up diagnoses including bipolar disorder, organic mood disorder and one case of ‘no disorder’. ‘Crossovers’ were allowed, with nearly 66% of the control group receiving clozapine at some time. There was no restriction on the prescription of other medications, with patients in both groups receiving other psychotropic medications, including other antipsychotics. An intention-to-treat analysis would be meaningless given the number of crossovers. Also, analysis of data with crossovers excluded is unlikely to be informative as it would end up comparing a small subgroup of responders in either arm of the study. The validity of including this study in the meta-analysis is questionable. This is particularly relevant as the ‘forest plot’ in Moncrieff's analysis reveals that this is the only study where the effect size is in the opposite direction (i.e. unfavourable to clozapine). Thus, inclusion of this study would dilute the effect size of clozapine and vice versa.

Moncrieff's handling of the data from the Kane et al (Reference Kane, Marder and Schooler2001) study also raises questions. In this longer-duration study, patients in both the control and experimental groups were allowed to drop out if they were not responding to the given treatment. A nonintention-to-treat analysis, as Dr Moncrieff has done, would end up comparing a small subgroup of responders in either group. An intention-to-treat analysis would have captured clozapine's strength; that is, showing that more patients on clozapine responded in comparison with the control group.

Despite these observations, Moncrieff's analysis produced an effect size of 0.38 (0.44 using a random effects model). In my opinion, this is not unimpressive given that clozapine is being compared with other medications with proven efficacy and not placebo.