Overview of individual studies

Table 1 shows the characteristics of the nine trials of clozapine v. conventional neuroleptics for treatment-resistant schizophrenia. Definitions of treatment resistance varied. All trials, except one that did not provide a definition (Reference Klieser, Schonell, Möller and PelzerKlieser & Schönell, 1990), required participants to have failed to respond to adequate trials of at least two conventional antipsychotic agents. Failure of response was defined by evidence of long-lasting serious difficulties in functioning in three trials, and by symptom levels of moderate or above on BPRS positive symptom items in four trials. One trial used both, and three trials provided no definition of failed response. Initial severity as measured by BPRS score varied between 38 and 84 (Table 1).

Table 2 shows the results of individual studies according to the outcome measures used in each. It is clear that there is great variation between study results. The study by Kane et al (Reference Kane, Honigfeld and Singer1988) is the only sizeable trial to have found a substantial and unequivocal difference between clozapine and standard neuroleptics. The largest trial conducted found only small differences in symptom scores and improvement rates in the intention-to-treat analysis, although the former are reported to be statistically significant because of the large sample size (Reference Rosenheck, Cramer and XuRosenheck et al, 1997). The large naturalistic study by Essock et al (Reference Essock, Hargreaves and Covell1996) is difficult to interpret because 66% of patients in the control group were given clozapine at some time during the course of the trial. There were no differences on symptom scores at the end of the trial, both in intention-to-treat analysis and analysis excluding crossovers. Survival analysis of improvement rates in the two groups showed almost identical survival curves, according to the authors. Slight differences favouring clozapine at the end of the study were based on small numbers and were not felt to represent real differences (N. Covell, personal communication, 2002).

Table 2

Efficacy of clozapine v. comparator drugs

Study	Clozapine	Comparator drug	Difference between groups (P)	Difference between groups1 (%)
Kane et al (Reference Kane, Honigfeld and Singer1988)
BPRS score reduction	15	5	<0.001	24.4
Patients improved	30%	4%	<0.001
Patients withdrawn	12%	13%
Klieser & Schönell (Reference Klieser, Schonell, Möller and Pelzer1990)
BPRS score reduction	35	19	Not given	44.4
Patients withdrawn	6%	6%	NS
Essock et al (Reference Essock, Hargreaves and Covell1996)
BPRS score reduction	1	3	NS	−8.7% (favours control group)
Reduction with crossovers excluded	1	2	NS
Patients withdrawn	34%	Crossovers 66%2
Kumra et al (Reference Kumra, Frazier and Jacobsen1996)
BPRS score reduction	31.2	20	<0.04	23.2
Patients withdrawn	30%	9%
Rosenheck et al (Reference Rosenheck, Cramer and Xu1997)
PANSS score reduction	12	8.6	<0.02	5.7
Patients improved	37%	32%	NS
Patients withdrawn	43%	72%	<0.001
Hong et al (Reference Hong, Chen and Chiu1997)
BPRS score reduction	8	1	0.007	15.6
PANSS score reduction	10	2	0.03
Patients improved	29%	0%	0.02
Patients withdrawn	10%	11%	NS
Buchanan et al (Reference Buchanan, Breier and Kirkpatrick1998)
BPRS score reduction	1.8	1.3	NS	3.9
Patients withdrawn	21%	8%	NS
Howanitz et al (Reference Howanitz, Pardo and Smelson1999)
PANSS score reduction	19.9	14.5	NS	3.7
Patients withdrawn	Not reported	Not reported
Kane et al (Reference Kane, Marder and Schooler2001)
BPRS score reduction	7.4	2.53	<0.01	21
Patients withdrawn	35%	65%	0.03

Only two studies reported data on rates of re-hospitalisation or discharge. Essock et al (Reference Essock, Hargreaves and Covell1996) reported that there was no difference in discharge rates between patients assigned to clozapine and those assigned to ‘usual care’, but patients in the clozapine group who were discharged were less likely to be readmitted. Rosenheck et al (Reference Rosenheck, Cramer and Xu1997) found no difference in the proportion of patients who were readmitted during the study but found that overall the group of patients assigned to clozapine spent 24 fewer days in psychiatric in-patient beds. Neither of the out-patient trials presented data on admission rates.

Table 2 also shows the difference in improvement ratings on the main outcome measure expressed as a percentage of the post-treatment score for the control group. This difference was greatest in the two earliest trials, but falls far short of the 20% clinically relevant difference in the more recent trials. Exceptions are the small trial in children (Reference Kumra, Frazier and JacobsenKumra et al, 1996), the study conducted in Taiwan (Reference Hong, Chen and ChiuHong et al, 1997) and the latest study of out-patients (Reference Kane, Marder and SchoolerKane et al, 2001). The last is the only long-term study to find that clozapine was substantially superior to a typical anti-psychotic. It is worth noting therefore that 50% of patients in the haloperidol group were withdrawn because of ‘lack of efficacy’. It is difficult to understand this finding in a group of patients who were considered stable enough to live in the community. The fact that they had to be withdrawn from the trial suggests that their condition had deteriorated more than would be expected in the normal course of events. Information on how these patients fared subsequently is necessary to address this possibility. It is also unusual that the rate of withdrawal because of the lack of efficacy was chosen as the main measure of outcome. Considering only patients who remained in the trial, final total BPRS scores were similar for groups allocated to haloperidol (n=11, mean 40.2, s.d.=12.2) and to clozapine (n=23, mean 37.5, s.d.=9.0). Rates of withdrawal were also very variable across trials, ranging from 9.5% to 43% in the clozapine groups and from 8 to 72% in the control groups.

Meta-analysis

Figure 1 shows that heterogeneity between study results was substantial and the statistical test for heterogeneity was highly significant (Q=38.2, d.f.=8, P < 0.001). Overall meta-analysis using intention-to-treat data and a fixed effects model produced an overall effect of 0.38 standard deviations (95% CI 0.27–0.50) in favour of clozapine over the standard antipsychotic. A random effects model yielded a similar result of 0.44 standard deviations (95% CI 0.15–0.73). The between-study variance was 0.14. Even when trials were grouped according to their duration, heterogeneity was still substantial. For the three long-term trials the heterogeneity statistic was 12.9 (P=0.002) and for the six short-term studies it was 11.1 (P=0.05).

Fig. 1

Forest plot using intention-to-treat data and fixed effect estimate for studies listed in Table 1.

Table 3 shows the results of univariate meta-regression analysis. Using the intention-to-treat data-set, it was found that duration, initial BPRS score and financial support from the pharmaceutical industry predicted outcome. Studies that found larger differences in favour of clozapine were of shorter duration and the participants had higher initial BPRS scores. Trials where there was information that some financial support had been provided by the pharmaceutical company manufacturing clozapine also showed a greater benefit of clozapine compared with the conventional neuroleptic. Study duration had the strongest effect on outcome and reduced the between-study variance by the greatest amount.

Table 3

Meta-regression analysis

Variable1	Regression coefficient (s.e.), P	Tau-squared2
Intention-to-treat data-set
Initial BPRS score	0.03 (0.01), P=0.02	0.05
Duration of trial (weeks)	0.009 (0.002), P<0.001	0.02
Sponsor (n=8)3	0.63 (0.21), P=0.003	0.03
Dosage ratio (mg of clozapine v. CPZeq)	0.40 (0.23), P=0.08	0.07
Data-set with crossovers or withdrawals excluded, from Essock et al (Reference Essock, Hargreaves and Covell1996) and Kane et al (Reference Kane, Marder and Schooler2001)
Initial BPRS score	0.03 (0.009), P=0.001	0.01
Duration of trial (weeks)	0.006 (0.003), P=0.04	0.03
Sponsor (n=8)3	0.54 (0.14), P<0.001	<0.001
Year of publication	0.06 (0.02), P<0.001	<0.001

Using the data-set with non-intention-to-treat data for Essock et al (Reference Essock, Hargreaves and Covell1996) and Kane et al (Reference Kane, Marder and Schooler2001) showed that the same variables predicted outcome, but also showed a significant and strong effect for year of publication with between-study variance reduced to zero. Source of financial support had a stronger effect in this analysis and also reduced between-study variance to zero.

Methodological limitations

There are several caveats to these findings. All reviews — including meta-analyses — involve subjective judgements, for example in the selection of outcome measures and data with which to conduct the analysis. Sensitivity analyses have been performed where the author felt that there were grounds for debate about which set of statistics to use in any given study and, in particular, where arguments could be made not to include crossovers or withdrawals. In addition, results of individual studies are summarised qualitatively as well as quantitatively.

The effect of treatment on symptom levels was chosen for the purposes of the meta-analysis and exploration of heterogeneity. This was felt to be the best reflection of the objectives of clozapine treatment, and as such was present in all trials. Other main published outcomes are summarised, but were not used in the meta-analysis. Continuous data have been used, since these were available for all studies, whereas categorical data were not. Moreover, categorical data may be more subject to bias, if the basis of categorisation has not been specified a priori.

Meta-regression analysis has severe limitations in a small set of nine trials. The small number of studies limits its power, but despite this some consistently significant effects were found. The results for year of publication were only apparent within the data with withdrawals excluded, which demonstrates the sensitivity of the analysis to decisions about which data to use. It is also important to note that meta-regression is an uncontrolled analysis, which lacks the protection of randomisation. It must therefore be viewed cautiously as an exploratory procedure. In addition, since multiple regression was not performed, the possibility of correlation or confounding between explanatory variables could not be explored.

Overall findings

Despite clozapine's reputation, there is substantial heterogeneity among trials comparing clozapine with conventional antipsychotics for treatment-resistant schizophrenia. Most recent trials have not replicated the dramatic superiority shown by clozapine in early trials. In particular, they fail to demonstrate that the differences between clozapine and conventional anti-psychotics are clinically relevant in terms of the degree of difference in reduction of BPRS or other symptom scores. It is interesting to note that so far no differences have been found between the effects of clozapine and other atypical antipsychotics in patients with treatment-resistant schizophrenia (Reference Tuunainen, Wahlbeck and GilbodyTuunainen et al, 2002).

This analysis demonstrates the potential danger inherent in combining results from different studies while overlooking important variations between them. The influential Cochrane review (Reference Wahlbeck, Cheine and EssaliWahlbeck et al, 2000a ) might also have overestimated the effects of clozapine by using non-intention-to-treat data in the largest study, by Rosenheck et al (Reference Rosenheck, Cramer and Xu1997), and by excluding the large study by Essock et al (Reference Essock, Hargreaves and Covell1996) from the analysis of effects on mental state. The reasons for excluding the findings of Essock et al (Reference Essock, Hargreaves and Covell1996) from this analysis were not given but it may be because not all the data required were available in the published paper. Other results from the study were included in other analyses.

Sources of heterogeneity

In the Cochrane review including all clozapine trials, no association was found between various measures of trial quality and outcome, but whether a study had received commercial sponsorship did predict some outcomes (Wahlbeck et al, Reference Wahlbeck, Cheine and Essali1999, Reference Wahlbeck, Tuunainen and Gilbody2000b). In the current analysis it appeared that shorter duration of trial, higher levels of baseline symptoms, commercial support and possibly earlier year of publication predicted greater superiority of clozapine over conventional anti-psychotics. The influence of the pharmaceutical industry has probably been understated in some studies, since published reports declared only whether the study had received direct funding from a pharmaceutical company. Financial interests of individual authors were not declared. The duration of the study also emerged as a strong predictor of outcome, suggesting that initial beneficial effects may not be maintained over the long term. Only one small long-term study suggested that clozapine might have substantial benefits (Reference Kane, Marder and SchoolerKane et al, 2001). However, the high withdrawal rate from the haloperidol group in this trial requires explanation and may have some bearing on the results obtained.

All trials in the current analysis used relatively high daily doses of conventional antipsychotics, including the most recent study (Reference Kane, Marder and SchoolerKane et al, 2001). There was probably not enough variation in comparative dosage levels therefore to examine the impact on outcome adequately. This may explain why the results do not confirm those of Geddes et al (Reference Geddes, Freemantle and Harrison2000), who found that the advantages of atypical anti-psychotics were apparent only when doses greater than 12 mg of haloperidol or equivalent were used.

Initial severity and effects of clozapine

Although all patients in the trials examined were classified as having treatment-resistant disease, there was considerable variation between trials in average severity of baseline symptoms. The current analysis suggests that among patients with treatment-resistant disease, the benefits of clozapine may be most marked in those with higher levels of initial symptoms. Other data on the relationship between response to clozapine and severity of illness are inconsistent. In the large study of US veterans included here, ‘high hospital users’ showed a smaller advantage for clozapine over haloperidol in terms of symptom reduction than ‘low hospital users’ (Reference Rosenheck, Cramer and AllanRosenheck et al, 1999). In addition, differences between clozapine and haloperidol were not significant in the ‘high user’ group. In the later out-patient study, patients who were functioning at a lower level at trial entry as measured by Clinical Global Impression scores (Reference GuyGuy, 1976) were less likely to show an enhanced response rate to clozapine compared with haloperidol (Reference Umbricht, Wirshing and WirshingUmbricht et al, 2002). However, curiously, when this was controlled for, higher baseline rates of symptoms as measured by the BPRS predicted better relative response rates to clozapine. The relationship between severity and efficacy found in this analysis may therefore be an ecological effect that may not necessarily translate to the individual level. It is also not possible to know from the current analysis whether the greater benefit of clozapine in patients with higher baseline symptom levels would be maintained in the long term, since all long-term studies were conducted with patients who had lower levels of baseline symptoms.

CLINICAL IMPLICATIONS

1. ▪ The superiority of clozapine over conventional antipsychotic drugs has not been consistently and conclusively demonstrated.

2. ▪ There is substantial variation between results of different studies, which appears to be accounted for by study duration and funding and level of initial symptoms.

3. ▪ Clozapine's greatest advantage relative to standard medication may be seen in patients with very high levels of initial symptoms.

LIMITATIONS

1. ▪ Statistical analyses were based on results relating to symptom reduction only.

2. ▪ Meta-regression analysis was sensitive to use of different data from some trials.

3. ▪ Only one study compared clozapine with a moderate rather than a large dose of conventional drugs, meaning that the effects of dose could not be adequately explored.