Since the publication of the landmark study by Kane et al (Reference Kane, Honigfeld and Singer1988), clozapine has been regarded as an effective treatment for people with treatment-resistant schizophrenia, defined as failing to respond to standard antipsychotic drug treatment. A Cochrane review of comparative randomised trials concluded that clozapine is more effective than conventional antipsychotics for all patients with schizophrenia and that the comparative advantage of clozapine is greater in patients whose condition is classified as treatment-resistant (Wahlbeck et al, Reference Wahlbeck, Cheine and Essali1999, Reference Wahlbeck, Cheine and Essali2000a). However, despite this consensus, subsequent large-scale trials of clozapine in treatment-resistant schizophrenia have failed to replicate the dramatic effects achieved by Kane et al, finding at best only small differences (Reference Essock, Hargreaves and CovellEssock et al, 1996; Reference Rosenheck, Cramer and XuRosenheck et al, 1997). Another recent Cochrane review found no difference between the efficacy of other ‘atypical’ agents and clozapine in patients with treatment-resistant schizophrenia (Reference Tuunainen, Wahlbeck and GilbodyTuunainen et al, 2002). In the light of these findings, the trials comparing clozapine and conventional antipsychotics for treatment-resistant conditions were re-examined. In particular, the clinical relevance of the results of later studies was examined and possible causes of heterogeneity between studies were investigated.
Trials identified in the Cochrane review comparing clozapine and conventional antipsychotics for treatment-resistant schizophrenia (Reference Wahlbeck, Cheine and EssaliWahlbeck et al, 2000a ) were re-examined. In addition, an electronic search was performed using Medline and EMBASE from 1998 to April 2003 to identify trials conducted subsequent to the review. Keywords CLOZAPINE and RANDOMISED CONTROL TRIAL or CONTROLLED TRIAL or RANDOMISED TRIAL were used. This located one additional trial that had been conducted on patients with treatment-resistant disease. An additional trial was identified from the Cochrane review that had not been included in the treatment-resistant analysis, but was found to involve participants with treatment-resistant schizophrenia. In total nine studies were identified (Table 1).
|Study (main publication)||Participants||Study size and duration||Mean daily dosage (mg)|
|Kane et al (Reference Kane, Honigfeld and Singer1988)||VA in-patients, hospitalised 2 years||n=268||Clozapine 600|
|Initial BPRS 61||6 weeks||Chlorpromazine 1200|
|Klieser & Schönell (Reference Klieser, Schonell, Möller and Pelzer1990)||In-patients, duration of hospitalisation not given (mean duration of illness 17 years)||n=32||Clozapine 400|
|Initial BPRS 67||6 weeks||Haloperidol 20|
|Essock et al (Reference Essock, Hargreaves and Covell1996)||State hospital in-patients, hospitalised 8 years||n=227||Clozapine 496|
|Initial BPRS 43||2 years||‘Usual care’ CPZeq 1386|
|Kumra et al (Reference Kumra, Frazier and Jacobsen1996)||Children, in-patients, hospitalised 9 months||n=21||Clozapine 176|
|Initial BPRS 84||6 weeks||Haloperidol 16|
|Rosenheck et al (Reference Rosenheck, Cramer and Xu1997)||VA in-patients, hospitalised 1-12 months||n=423||Clozapine 552|
|BPRS 52||1 year||Haloperidol 28|
|Hong et al (Reference Hong, Chen and Chiu1997)||VA in-patients, hospitalised 10 years||n=40||Clozapine 543|
|BPRS 53||12 weeks||Chlorpromazine 1163|
|Buchanan et al (Reference Buchanan, Breier and Kirkpatrick1998)||Out-patients||n=75||Clozapine 413|
|Initial BPRS 38||10 weeks||Haloperidol 26|
|Howanitz et al (Reference Howanitz, Pardo and Smelson1999)||Elderly in-patients (age > 55 years), hospitalised 18 years||n=42||Clozapine 300|
|Initial BPRS 52||12 weeks||Chlorpromazine 600|
|Kane et al (Reference Kane, Marder and Schooler2001)||Out-patients||n=71||At 17 weeks:|
|Initial BPRS 46||29 weeks||clozapine 506|
Individual study results were tabulated and examined. The difference in symptom scores between the clozapine group and the comparison group at the end of treatment as a percentage of the post-treatment score in the control group was calculated. This was done in order to compare results with the 20% difference that is commonly said to represent clinically significant improvement in individuals in terms of symptom ratings in treatment-resistant cases (Reference Rosenheck, Cramer and XuRosenheck et al, 1997; Reference Wahlbeck, Cheine and EssaliWahlbeck et al, 1999).
Meta-analysis was conducted to examine heterogeneity between studies, which is assessed by testing the weighted variation of individual study results about the mean effect. The outcomes of individual trials were converted to standardised mean differences (SMDs) to allow the results of studies using different outcome measures to be combined. The SMD is usually calculated as the difference between the mean of the experimental group and the mean of the control group divided by the combined standard deviation (Reference Hedges and OlkinHedges & Olkin, 1985). The level of symptoms at the end of the study or change in symptoms over the course of the study was defined as the main outcome of interest, since this is the principal objective of clozapine therapy in treatment resistance. All studies used either the Brief Psychiatric Rating Scale (BPRS; Reference Overall and GorhamOverall & Gorham, 1962) or the Positive and Negative Syndrome Scale (PANSS; Reference Kay, Fiszbein and OplerKay et al, 1987) to rate symptoms. Intention-to-treat data were used if possible. In one study (Reference Rosenheck, Cramer and XuRosenheck et al, 1997) standard deviations were not available for the intention-to-treat data and so the standard deviations obtained from the analysis of treatment completers were used instead.
In the study by Essock et al (Reference Essock, Hargreaves and Covell1996) it was difficult to decide which data to use as the basis for calculating SMD, since this was a naturalistic study and a large proportion of people in the control group were prescribed clozapine at some point during the study. Intention-to-treat data were available for change in BPRS score at the end of the study from a later publication by the same authors, who noted that the results were similar when the analysis was performed with crossovers excluded (Reference Essock, Frisman and CovellEssock et al, 2000). The intention-to-treat data were therefore used as the primary basis for calculating SMDs, but sensitivity analysis was conducted using post-treatment scores with crossovers excluded. In this case standard deviation was calculated from the t-value provided (Reference Essock, Hargreaves and CovellEssock et al, 1996) and the number of patients in each group was provided on request by the authors (N. Covell, personal communication, 2002). Sensitivity analysis was also conducted using non-intention-to-treat data from the study by Kane et al (Reference Kane, Marder and Schooler2001), for reasons explained below.
Meta-regression analysis was then conducted to investigate possible sources of heterogeneity. This consists of a weighted regression analysis using the individual study SMDs as the data points. The following trial characteristics were investigated to see whether they predicted outcome in terms of the SMD:
(a) duration of study;
(b) size of study;
(c) year of publication;
(d) severity of patients' condition at trial entry in terms of BPRS score;
(e) whether financial support from the pharmaceutical industry was received;
(f) whether a pre-trial high-dose treatment period was employed;
(g) the ratio of dosage of clozapine to comparator drug (in chlorpromazine equivalents).
These factors were considered a priori to be potential predictors of outcome. Duration was considered to be a source of heterogeneity in the Cochrane review (Reference Wahlbeck, Cheine and EssaliWahlbeck et al, 2000a ). There has also been some suggestion that initial severity (Reference Umbricht, Wirshing and WirshingUmbricht et al, 2002) and financial support (Reference Wahlbeck, Cheine and EssaliWahlbeck et al, 2000a ) might predict outcome. Size of study was examined as a proxy for study quality and because it may indicate publication bias (Reference Sterne, Egger, Smith, Egger, Smith and AltmanSterne et al, 2001). Year of publication was examined to assess whether there was an effect of the initial enthusiasm for a novel treatment. The effect of a pre-trial treatment period was examined because of the possibility that this could introduce selection bias by excluding participants who respond to a new trial of a standard antipsychotic. Dose ratio was examined because of suggestions that the use of overly high dosages of comparator drugs have contributed to clozapine's apparent superiority.
A univariate analysis was conducted to explore the associations between outcome and the individual hypothesised explanatory variables. The reduction of the tau-squared value (that is, the residual variance between study results) was noted for each analysis. The analysis was repeated using the data-set with non-intention-to-treat data for the studies by Essock et al (Reference Essock, Hargreaves and Covell1996) and Kane et al (Reference Kane, Marder and Schooler2001). Multivariate analysis was considered but it was decided that there were too few trials for it to produce reliable results.
All meta-analysis was conducted using the STATA version 7.0 statistical package (Stata, 2001).
Overview of individual studies
Table 1 shows the characteristics of the nine trials of clozapine v. conventional neuroleptics for treatment-resistant schizophrenia. Definitions of treatment resistance varied. All trials, except one that did not provide a definition (Reference Klieser, Schonell, Möller and PelzerKlieser & Schönell, 1990), required participants to have failed to respond to adequate trials of at least two conventional antipsychotic agents. Failure of response was defined by evidence of long-lasting serious difficulties in functioning in three trials, and by symptom levels of moderate or above on BPRS positive symptom items in four trials. One trial used both, and three trials provided no definition of failed response. Initial severity as measured by BPRS score varied between 38 and 84 (Table 1).
Table 2 shows the results of individual studies according to the outcome measures used in each. It is clear that there is great variation between study results. The study by Kane et al (Reference Kane, Honigfeld and Singer1988) is the only sizeable trial to have found a substantial and unequivocal difference between clozapine and standard neuroleptics. The largest trial conducted found only small differences in symptom scores and improvement rates in the intention-to-treat analysis, although the former are reported to be statistically significant because of the large sample size (Reference Rosenheck, Cramer and XuRosenheck et al, 1997). The large naturalistic study by Essock et al (Reference Essock, Hargreaves and Covell1996) is difficult to interpret because 66% of patients in the control group were given clozapine at some time during the course of the trial. There were no differences on symptom scores at the end of the trial, both in intention-to-treat analysis and analysis excluding crossovers. Survival analysis of improvement rates in the two groups showed almost identical survival curves, according to the authors. Slight differences favouring clozapine at the end of the study were based on small numbers and were not felt to represent real differences (N. Covell, personal communication, 2002).
|Study||Clozapine||Comparator drug||Difference between groups (P)||Difference between groups1 (%)|
|Kane et al (Reference Kane, Honigfeld and Singer1988)|
|BPRS score reduction||15||5||<0.001||24.4|
|Klieser & Schönell (Reference Klieser, Schonell, Möller and Pelzer1990)|
|BPRS score reduction||35||19||Not given||44.4|
|Essock et al (Reference Essock, Hargreaves and Covell1996)|
|BPRS score reduction||1||3||NS||−8.7% (favours control group)|
|Reduction with crossovers excluded||1||2||NS|
|Patients withdrawn||34%||Crossovers 66%2|
|Kumra et al (Reference Kumra, Frazier and Jacobsen1996)|
|BPRS score reduction||31.2||20||<0.04||23.2|
|Rosenheck et al (Reference Rosenheck, Cramer and Xu1997)|
|PANSS score reduction||12||8.6||<0.02||5.7|
|Hong et al (Reference Hong, Chen and Chiu1997)|
|BPRS score reduction||8||1||0.007||15.6|
|PANSS score reduction||10||2||0.03|
|Buchanan et al (Reference Buchanan, Breier and Kirkpatrick1998)|
|BPRS score reduction||1.8||1.3||NS||3.9|
|Howanitz et al (Reference Howanitz, Pardo and Smelson1999)|
|PANSS score reduction||19.9||14.5||NS||3.7|
|Patients withdrawn||Not reported||Not reported|
|Kane et al (Reference Kane, Marder and Schooler2001)|
|BPRS score reduction||7.4||2.53||<0.01||21|
Only two studies reported data on rates of re-hospitalisation or discharge. Essock et al (Reference Essock, Hargreaves and Covell1996) reported that there was no difference in discharge rates between patients assigned to clozapine and those assigned to ‘usual care’, but patients in the clozapine group who were discharged were less likely to be readmitted. Rosenheck et al (Reference Rosenheck, Cramer and Xu1997) found no difference in the proportion of patients who were readmitted during the study but found that overall the group of patients assigned to clozapine spent 24 fewer days in psychiatric in-patient beds. Neither of the out-patient trials presented data on admission rates.
Table 2 also shows the difference in improvement ratings on the main outcome measure expressed as a percentage of the post-treatment score for the control group. This difference was greatest in the two earliest trials, but falls far short of the 20% clinically relevant difference in the more recent trials. Exceptions are the small trial in children (Reference Kumra, Frazier and JacobsenKumra et al, 1996), the study conducted in Taiwan (Reference Hong, Chen and ChiuHong et al, 1997) and the latest study of out-patients (Reference Kane, Marder and SchoolerKane et al, 2001). The last is the only long-term study to find that clozapine was substantially superior to a typical anti-psychotic. It is worth noting therefore that 50% of patients in the haloperidol group were withdrawn because of ‘lack of efficacy’. It is difficult to understand this finding in a group of patients who were considered stable enough to live in the community. The fact that they had to be withdrawn from the trial suggests that their condition had deteriorated more than would be expected in the normal course of events. Information on how these patients fared subsequently is necessary to address this possibility. It is also unusual that the rate of withdrawal because of the lack of efficacy was chosen as the main measure of outcome. Considering only patients who remained in the trial, final total BPRS scores were similar for groups allocated to haloperidol (n=11, mean 40.2, s.d.=12.2) and to clozapine (n=23, mean 37.5, s.d.=9.0). Rates of withdrawal were also very variable across trials, ranging from 9.5% to 43% in the clozapine groups and from 8 to 72% in the control groups.
Figure 1 shows that heterogeneity between study results was substantial and the statistical test for heterogeneity was highly significant (Q=38.2, d.f.=8, P < 0.001). Overall meta-analysis using intention-to-treat data and a fixed effects model produced an overall effect of 0.38 standard deviations (95% CI 0.27–0.50) in favour of clozapine over the standard antipsychotic. A random effects model yielded a similar result of 0.44 standard deviations (95% CI 0.15–0.73). The between-study variance was 0.14. Even when trials were grouped according to their duration, heterogeneity was still substantial. For the three long-term trials the heterogeneity statistic was 12.9 (P=0.002) and for the six short-term studies it was 11.1 (P=0.05).
Table 3 shows the results of univariate meta-regression analysis. Using the intention-to-treat data-set, it was found that duration, initial BPRS score and financial support from the pharmaceutical industry predicted outcome. Studies that found larger differences in favour of clozapine were of shorter duration and the participants had higher initial BPRS scores. Trials where there was information that some financial support had been provided by the pharmaceutical company manufacturing clozapine also showed a greater benefit of clozapine compared with the conventional neuroleptic. Study duration had the strongest effect on outcome and reduced the between-study variance by the greatest amount.
|Variable1||Regression coefficient (s.e.), P||Tau-squared2|
|Initial BPRS score||0.03 (0.01), P=0.02||0.05|
|Duration of trial (weeks)||0.009 (0.002), P<0.001||0.02|
|Sponsor (n=8)3||0.63 (0.21), P=0.003||0.03|
|Dosage ratio (mg of clozapine v. CPZeq)||0.40 (0.23), P=0.08||0.07|
|Data-set with crossovers or withdrawals excluded, from Essock et al (Reference Essock, Hargreaves and Covell1996) and Kane et al (Reference Kane, Marder and Schooler2001)|
|Initial BPRS score||0.03 (0.009), P=0.001||0.01|
|Duration of trial (weeks)||0.006 (0.003), P=0.04||0.03|
|Sponsor (n=8)3||0.54 (0.14), P<0.001||<0.001|
|Year of publication||0.06 (0.02), P<0.001||<0.001|
Using the data-set with non-intention-to-treat data for Essock et al (Reference Essock, Hargreaves and Covell1996) and Kane et al (Reference Kane, Marder and Schooler2001) showed that the same variables predicted outcome, but also showed a significant and strong effect for year of publication with between-study variance reduced to zero. Source of financial support had a stronger effect in this analysis and also reduced between-study variance to zero.
There are several caveats to these findings. All reviews — including meta-analyses — involve subjective judgements, for example in the selection of outcome measures and data with which to conduct the analysis. Sensitivity analyses have been performed where the author felt that there were grounds for debate about which set of statistics to use in any given study and, in particular, where arguments could be made not to include crossovers or withdrawals. In addition, results of individual studies are summarised qualitatively as well as quantitatively.
The effect of treatment on symptom levels was chosen for the purposes of the meta-analysis and exploration of heterogeneity. This was felt to be the best reflection of the objectives of clozapine treatment, and as such was present in all trials. Other main published outcomes are summarised, but were not used in the meta-analysis. Continuous data have been used, since these were available for all studies, whereas categorical data were not. Moreover, categorical data may be more subject to bias, if the basis of categorisation has not been specified a priori.
Meta-regression analysis has severe limitations in a small set of nine trials. The small number of studies limits its power, but despite this some consistently significant effects were found. The results for year of publication were only apparent within the data with withdrawals excluded, which demonstrates the sensitivity of the analysis to decisions about which data to use. It is also important to note that meta-regression is an uncontrolled analysis, which lacks the protection of randomisation. It must therefore be viewed cautiously as an exploratory procedure. In addition, since multiple regression was not performed, the possibility of correlation or confounding between explanatory variables could not be explored.
Despite clozapine's reputation, there is substantial heterogeneity among trials comparing clozapine with conventional antipsychotics for treatment-resistant schizophrenia. Most recent trials have not replicated the dramatic superiority shown by clozapine in early trials. In particular, they fail to demonstrate that the differences between clozapine and conventional anti-psychotics are clinically relevant in terms of the degree of difference in reduction of BPRS or other symptom scores. It is interesting to note that so far no differences have been found between the effects of clozapine and other atypical antipsychotics in patients with treatment-resistant schizophrenia (Reference Tuunainen, Wahlbeck and GilbodyTuunainen et al, 2002).
This analysis demonstrates the potential danger inherent in combining results from different studies while overlooking important variations between them. The influential Cochrane review (Reference Wahlbeck, Cheine and EssaliWahlbeck et al, 2000a ) might also have overestimated the effects of clozapine by using non-intention-to-treat data in the largest study, by Rosenheck et al (Reference Rosenheck, Cramer and Xu1997), and by excluding the large study by Essock et al (Reference Essock, Hargreaves and Covell1996) from the analysis of effects on mental state. The reasons for excluding the findings of Essock et al (Reference Essock, Hargreaves and Covell1996) from this analysis were not given but it may be because not all the data required were available in the published paper. Other results from the study were included in other analyses.
Sources of heterogeneity
In the Cochrane review including all clozapine trials, no association was found between various measures of trial quality and outcome, but whether a study had received commercial sponsorship did predict some outcomes (Wahlbeck et al, Reference Wahlbeck, Cheine and Essali1999, Reference Wahlbeck, Tuunainen and Gilbody2000b). In the current analysis it appeared that shorter duration of trial, higher levels of baseline symptoms, commercial support and possibly earlier year of publication predicted greater superiority of clozapine over conventional anti-psychotics. The influence of the pharmaceutical industry has probably been understated in some studies, since published reports declared only whether the study had received direct funding from a pharmaceutical company. Financial interests of individual authors were not declared. The duration of the study also emerged as a strong predictor of outcome, suggesting that initial beneficial effects may not be maintained over the long term. Only one small long-term study suggested that clozapine might have substantial benefits (Reference Kane, Marder and SchoolerKane et al, 2001). However, the high withdrawal rate from the haloperidol group in this trial requires explanation and may have some bearing on the results obtained.
All trials in the current analysis used relatively high daily doses of conventional antipsychotics, including the most recent study (Reference Kane, Marder and SchoolerKane et al, 2001). There was probably not enough variation in comparative dosage levels therefore to examine the impact on outcome adequately. This may explain why the results do not confirm those of Geddes et al (Reference Geddes, Freemantle and Harrison2000), who found that the advantages of atypical anti-psychotics were apparent only when doses greater than 12 mg of haloperidol or equivalent were used.
Initial severity and effects of clozapine
Although all patients in the trials examined were classified as having treatment-resistant disease, there was considerable variation between trials in average severity of baseline symptoms. The current analysis suggests that among patients with treatment-resistant disease, the benefits of clozapine may be most marked in those with higher levels of initial symptoms. Other data on the relationship between response to clozapine and severity of illness are inconsistent. In the large study of US veterans included here, ‘high hospital users’ showed a smaller advantage for clozapine over haloperidol in terms of symptom reduction than ‘low hospital users’ (Reference Rosenheck, Cramer and AllanRosenheck et al, 1999). In addition, differences between clozapine and haloperidol were not significant in the ‘high user’ group. In the later out-patient study, patients who were functioning at a lower level at trial entry as measured by Clinical Global Impression scores (Reference GuyGuy, 1976) were less likely to show an enhanced response rate to clozapine compared with haloperidol (Reference Umbricht, Wirshing and WirshingUmbricht et al, 2002). However, curiously, when this was controlled for, higher baseline rates of symptoms as measured by the BPRS predicted better relative response rates to clozapine. The relationship between severity and efficacy found in this analysis may therefore be an ecological effect that may not necessarily translate to the individual level. It is also not possible to know from the current analysis whether the greater benefit of clozapine in patients with higher baseline symptom levels would be maintained in the long term, since all long-term studies were conducted with patients who had lower levels of baseline symptoms.
Clinical Implications and Limitations
▪ The superiority of clozapine over conventional antipsychotic drugs has not been consistently and conclusively demonstrated.
▪ There is substantial variation between results of different studies, which appears to be accounted for by study duration and funding and level of initial symptoms.
▪ Clozapine's greatest advantage relative to standard medication may be seen in patients with very high levels of initial symptoms.
▪ Statistical analyses were based on results relating to symptom reduction only.
▪ Meta-regression analysis was sensitive to use of different data from some trials.
▪ Only one study compared clozapine with a moderate rather than a large dose of conventional drugs, meaning that the effects of dose could not be adequately explored.
I thank Professor Paul Bebbington of University College London (UCL) for comments on the manuscript, and Rebecca Hardy, also of UCL, for statistical advice.