Aceituno et al ^{Reference Aceituno, Vera, Prina and McCrone1} review research on the cost-effectiveness of early intervention in psychosis, a therapeutic movement that has generated great hope for improved outcomes. Two contexts shape the review. The first involves the challenges of cost-effectiveness analysis (CEA) which, on the one hand, is a technical academic/scientific discipline and, on the other hand, a blunt but often urgent tool for defending funding requests. The second context is that of psychiatric care of people with schizophrenia, a branch of medicine that has, regrettably, shown limited improvement in outcomes, but is ardently seeking to do better.

First, it should be recognised that CEA represents more than just the addition of cost as another outcome measure in longitudinal research. Often begrudgingly accepted as a necessary tool for persuading funders to pay for care, CEA serves a larger purpose as the penultimate step in the evaluation of medical innovation as it brings together in one analysis (a) the evaluation of clinical benefits and side effects for patients; (b) spillover consequences for families and society (e.g. criminal justice involvement); and (c) the costs and benefits of interventions from the diverse perspectives of patients, families, insurance companies, governments, taxpayers and ultimately society as a whole. When we speak of costs, dollars and pounds are mere proxy measures for what is really at stake: the allocation and consumption of scarce resources. These scarce resources (human labour and material ‘stuff’) could be used for other valued purposes such as education, poverty reduction, combatting global warming, or national defence, as well as many wasteful ones. Societies are continually making ponderous decisions about what do to with scarce resources, but rarely do so on the basis of comprehensive information. CEA is thus not just a matter of creating arguments for ‘coughing up dough’ but of establishing societal priorities on the grounds of providing more benefit more efficiently. CEA is a hard thing to do both theoretically and practically – but is so important that the effort must be made.

Second, there is a need to acknowledge some basic facts about psychiatric treatment of psychosis. The suffering of people and families living with psychosis cries out for relief. But psychosis remains poorly understood. We do not yet know what schizophrenia is, and experts suggest it may be neither a single disease nor distinct from other mental illnesses. Our aetiological models are largely unproven or unprovable and our treatments, specifically Coordinated Specialty Care (CSC) as it has come to be called in the USA, is neither logically nor empirically tightly linked to either a specific diagnosis, an aetiological model or even a specific phase of psychotic illness. In this challenging context our colleagues have sought to review a considerably heterogeneous set of studies to determine whether programme benefits exceed their costs.

The combination of the ambitious goals of CEA and our imperfect understanding of psychosis helps account for the fact that Aceituno et al’s conclusion, after conducting a comprehensive review of the literature, is that – although they find some consistency – early intervention in psychosis only ‘might’ be worth the cost of the resources it requires, largely because, as they explain, ‘the evidence is heterogeneous and sometimes methodologically flawed’. But the question is so important that it has to be addressed.

Only 13 of the 16 reviewed studies involved what has increasingly been called CSC in the USA. Of the 16 studies, 2 involved a quite different intervention and target population: 1 used cognitive–behavioural therapy for ultra-high-risk patients who were pre-psychotic and the other evaluated an intervention to facilitate entry into treatment rather than clinical effectiveness. Of the 13 remaining studies only 4 were randomised clinical trials, the other 9 were based on historical controls or other potentially relevant comparison groups. Such studies are subject to unmeasured or unknown biases that weaken their validity, although they offer useful information on the feasibility of interventions and evaluation methods. Because of the acknowledged heterogeneity of the studies, this editorial required a review of each study to see what information they actually contained since a quantitative meta-analysis was not possible. In the four most relevant randomised controlled trials, data were not always available to support a full CEA.

Although these limitations are clearly noted in the review and would be familiar to many readers, we can ask what would ideally be required for CEA of early intervention in psychosis?

First, if a study is to be generalisable for policy purposes, both the intervention arm and the control need to be well justified and their fidelity to an articulated model should be documented empirically. Historical control services are convenient but have high variance and are poorly specified. Operational definition and fidelity measurement of CSC is uncommon but needed.^{Reference Mueser, Meyer-Kalos, Glynn, Lynde, Robinson and Gingerich2}

Second, CEA studies require larger than usual samples due to the non-normality of cost data and long time frames are required, e.g. 18–24 months, because costs accrue slowly. However, large-N studies are expensive and long-term studies typically suffer from extensive attrition. Only 7 studies in the current review included more than 75 patients per group.

Third, effectiveness measures should be selected for the specific purpose of cost–utility analyses, comprehensively reflecting both health benefits, side effects and risks. Few measures are available for this purpose for studies of psychosis, although a measure of quality-adjusted life-years, developed by Lenert et al,^{Reference Lenert, Sturley, Rapaport, Chavez, Mohr and Rupnow3} approximates what is needed. Non-disease-specific measures such as the well-known EQ-5D or the Quality of Well-Being Scale are imperfect, but among the better measures available. Use of measures such as employment or housing status are too narrow and Global Assessment of Functioning ratings are not adequately standardised for the purpose of determining whether benefits are worth their cost.

Fourth, although cost and outcome measures (the resources entailed in achieving a standard outcome) are sufficient for CEA in that they allow determination of the incremental cost-effectiveness ratio, a joint analysis of cost and effectiveness is needed, not merely separate analyses demonstrating that the intervention was both less costly and more effective. Bootstrap methods are well described for this purpose but were used in only a few studies covered in this review.^{Reference Briggs4} The ‘dominant-choice’ situation, in which the intervention is both less costly and more effective, is increasingly unlikely in rigorous randomised trials. Why? Because the major source of cost in psychosis is hospitalisation. With significant reductions in overall hospital bed supply, large reductions in hospital use with CSC are less and less likely. Although one meta-analysis has found evidence of reduction in hospital use with CSC,^{Reference Correll, Galling, Pawar, Krivko, Bonetto and Ruggeri5} another did not.^{Reference Fusar-Poli, McGorry and Kane6}

Finally, in the absence of a robust dominant choice, CEA will be inadequate without additional cost–benefit analysis. The difference is that although CEA tells us the cost of certain health benefits, unless it converts those health benefits into monetised value it cannot determine whether the health benefits are ‘worth’ the additional costs. It is only when health benefits are monetised (difficult as that is) that it is possible to determine whether the cost of a treatment is justified by the benefits The estimation of the monetary value of health benefits is controversial, and many are reluctant or even opposed to putting monetary value on health benefits, in part because they consider health ‘priceless’. Nevertheless, such methods are available. As with many of the issues discussed in this editorial, some formidable challenges are so important that we have to take them on anyway.

This rough sketch briefly identifies a demanding set of standards, more fully detailed in standard texts. The task of providing policymakers and advocates with rigorous CEA and cost–benefit analysis results is not easy or cheap. Until such data are available, advocates and conscientious policymakers will have to muddle through, putting reviews like this one to good purpose whatever their limitations.