Schizophrenia is among the top ten disorders in terms of burden of illness, disability, and societal and health costs worldwide.1 In England, there are approximately 8256 new cases of schizophrenia each yearReference Kirkbride, Errazuriz, Croudace, Morgan, Jackson and Boydell2 and the total annual cost of schizophrenia to society is £11.8 billion.Reference Andrew, Knapp, McCrone, Parsonage and Trachtenberg3 In addition, life expectancy is 15 years shorter for people with these problems.Reference Andrew, Knapp, McCrone, Parsonage and Trachtenberg3, Reference Hjorthoj, Sturup, McGrath and Nordentoft4 Sleep problems are pervasive and complex in patients with schizophrenia and adversely affect physical and mental health.Reference Freeman, Pugh, Vorontsova and Southgate5 These problems typically comprise a mixture of insomnia, hypersomnia, circadian rhythm disorders and nightmares, and their evidence-based treatment in schizophrenia has been overlooked.Reference Chouinard, Poulin, Stip and Godbout6 However, how to treat such sleep disturbance in the context of schizophrenia is not the topic of any guideline. The effectiveness of cognitive–behavioural therapy (CBT) in treating sleep problems in patients with schizophrenia was only recently assessed in the Better Sleep Trial (BEST).Reference Freeman, Waite, Startup, Myers, Lister and McInerney7 The findings from this pilot randomised controlled trial suggest that CBT for insomnia, suitably modified for this population, is likely to be highly effective for improving sleep in patients with schizophrenia. However, given that healthcare resources in mental health are particularly scarce,Reference Docherty and Thornicroft8 it is important that any new intervention, particularly one delivered face-to-face to patients, provides good value for money. As a result, the aim of this study was to use data from BEST to assess the potential cost-effectiveness of CBT in treating sleep disorders in patients with schizophrenia.
Method
Study design and patients
We carried out an economic evaluation alongside BEST to establish the cost-effectiveness of CBT in addition to usual care to treat sleeping disorders in patients with schizophrenia. BEST was a prospective, assessor-blind, randomised controlled pilot study in Oxfordshire, Buckinghamshire and Northamptonshire, England. The study recruited 50 patients with persistent, distressing delusions or hallucinations in the context of schizophrenia adult mental health services. Details of the trial and the clinical findings have been reported elsewhere.Reference Freeman, Waite, Startup, Myers, Lister and McInerney7 The economic evaluation was conducted from National Health Service (NHS) and societal perspectives, complying with guidance for health technology appraisals issued by the National Institute for Health and Care Excellence (NICE) in England.9 Outcomes and costs were assessed at baseline, and at 12 weeks and 24 weeks post-treatment. The trial flowchart is presented in Appendix 1.
Intervention and comparator
The CBT intervention included approximately eight sessions over 12 weeks provided one-to-one by clinical psychologists either in NHS clinics or at patients' homes under weekly clinical supervision by a consultant clinical psychologist. Telephone calls and text messages between sessions were used to maintain treatment momentum. The intervention was written in a manual to guide the work, which was shared with the patient. The principal therapeutic techniques included stimulus control (i.e. learning to associate bed with sleep) and improvement of daytime activity levels. Adaptations needed for the particular problems of delusions and hallucinations interfering with sleep, attempts to sleep being overused by patients as an escape from voices, extensive disruption of circadian rhythms, insufficient daytime activity and fear of the bed based on past adverse experiences were incorporated. Standard care was delivered according to national and local service protocols and guidelines and mainly consisted of antipsychotic medication and contact with the local clinical team.
Outcomes and costs
Health-related quality of life was measured during the follow-up period using the EuroQol 5 Dimensions 5 Levels (EQ-5D-5L) questionnaire. The five-level version of the EQ-5D-5L questionnaire was preferred over the three-level version, because it is considered to be more sensitive for capturing the health-related quality of life of a relatively young population with mental conditions. EQ-5D-5L responses were then valued using population preferencesReference Devlin, Shah, Feng, Mulhern and Van Hout10 and combined with survival information to calculate quality-adjusted life years (QALYs).
Costs were measured with a modified version of the Client Service Receipt Inventory.Reference Beecham, Knapp and Thornicroft11 The recall period in this questionnaire was 6 months at baseline, and 3 months in each of the two follow-up measurements (hence covering the 6 months after baseline). The health and social care costs included were: admissions to hospital; psychiatrist and other consultant visits; general practitioner and other primary healthcare visits, either at home or at the general practice; visits to and by community psychiatric nurses; visits to any counsellor or therapist; social worker visits; visits to any day care centre; and antipsychotic medicines. The intervention costs were collected for each patient included in the treatment group and included the time taken for the psychologist to deliver the sessions, travelling time to home visits, additional contact time via phone and emails, time for administration and travelling costs. The broader societal perspective also included the costs of informal caregiving (i.e. hours of unpaid care delivered to patients by their family and social environment) and justice services (i.e. contacts with police, nights spent in a police cell or prison, and criminal or civil court appearances). All costs were valued based on unit cost prices derived from different resources and reported in 2014/2015 British pounds (£). The costs of informal caregiving were calculated based on the national minimum hourly wage. A list of the unit costs and their sources is presented in Appendix Tables 2.1 and 2.3.
Statistical analysis
In a descriptive analysis, we used t-tests to obtain mean differences (and their 95% confidence intervals) in age, EQ-5D-5L utilities and costs between patients in the CBT group and patients in the standard care alone group. We performed the same descriptive analysis using Mann–Whitney tests to cross-check statistical certainty. Five patients in the CBT group and two patients in the standard care alone group had missing observations in EQ-5D-5L utilities and/or costs on one or more measurement points. The pattern of missing observations is presented in Appendix Table 2.2. The main analysis included only complete cases (i.e. it excluded seven patients with incomplete information on costs and/or EQ-5D-5L utilities).
To address uncertainty in the results due to the small sample size of the pilot trial, we performed bootstrapping with replacement. For each of the 10 000 bootstrapped samples, we estimated the mean and mean differences in costs and QALYs. The statistical analysis was performed in STATA13.
Cost-utility and value of information analysis
An incremental analysis was performed, assessing the differences in costs between the two interventions and dividing by the difference in QALYs gained in order to generate an incremental cost-effectiveness ratio (ICER). ICERs were estimated using both an NHS and a societal perspective. The 10 000 bootstrapped pairs of incremental costs and incremental QALYs were plotted on cost-effectiveness planes to display the uncertainty in the estimated ICER. Cost-effectiveness acceptability curves (CEACs) were also derived to display the probability of the CBT treatment being cost-effective, as the ceiling ratio for the maximum acceptable cost-effectiveness ratio varies from £0 to £85 000 per QALY gained.
BEST was designed to inform the design of a larger trial in order to gain more information about the effectiveness and cost-effectiveness of the CBT treatment for sleeping disorders. Therefore, we followed NICE guidance and estimated the expected value of perfect information (EVPI), which represents the monetary value of eliminating the uncertainty in the cost-utility results.Reference Barton, Briggs and Fenwick12 In other words, it provides decision makers with the value of acquiring further information on costs and outcomes for a number of people who may benefit from the additional research. EVPI can potentially be used to set research priorities. If the EVPI exceeds the cost of collecting more information about the effects of an intervention on costs and outcomes, then a further study for this purpose is considered to be worthwhile. In this analysis, we assumed that 10 000 patients would be affected by the treatment per year, based on a report about schizophrenia.Reference Kirkbride, Errazuriz, Croudace, Morgan, Jackson and Boydell2 We also assumed that the time frame over which the additional information could be expected to retain its usefulness (i.e. before the intervention becomes obsolete from newer interventions) would be 10 years. This time frame is frequently used in EVPI studies.Reference Thorn, Coast and Andronis13 Using a discount rate of 3.5%, as suggested in NICE guidance, the effective (discounted) population over a 10-year period was calculated to be 86 077 patients. This population was used in the value of information analysis.
Sensitivity analyses
Four sensitivity analyses were performed to address the effects of missing data and the difference in costs between the treatment arms at baseline on the cost-utility results. In the first, multiple imputation with predictive mean matching (with three closest observations) was performed to impute missing observations in costs and QALYs during the follow-up period based on costs and EQ-5D-5L utilities at baseline. The imputation process was partitioned by treatment arm and generated 500 imputed data-sets for each missing observation. The data-set with the imputed values was then used in the bootstrapping process, similar to the statistical analysis performed in the main analysis. In the second sensitivity analysis, we used generalised linear models to estimate 6-monthly mean costs (using gamma distribution and log link) and QALYs (using normal distribution and identity link) in the two groups and their differences, adjusted for costs or EQ-5D-5L utilities at baseline. The choice of family distribution and link function in the statistical models were based on the modified Park test and Link test, respectively.
Propensity score matching (PSM) was used in the third sensitivity analysis to match patients in the treatment arms based on their costs and EQ-5D-5L utility at baseline. Local linear regression matching was chosen as it outperformed other PSM techniques (including Mahalanobis, one-to-one, k-to-one, kernel, spline and inverse probability weighting) based on percentage standardised bias and variance ratio. In this analysis, all complete cases were included and PSM was performed in each of the 10 000 bootstrapped samples before estimating incremental costs and QALYs. In the last sensitivity analysis, antipsychotic medication was excluded from the costs. This is because other types of mental health medicines were poorly recorded in the questionnaires, and the inclusion of only antipsychotic medicines could have introduced biases in the results.
Results
Descriptive analysis
There were 19 patients in the CBT group and 24 patients in the standard care alone group with complete information on costs and EQ-5D-5L utilities. As Table 1 shows, patients in the CBT group were on average 40 years old and the majority were male (63%) and unemployed (84%). Their mean EQ-5D-5L utility was 0.58, which is relatively low compared with the 0.81 national average EQ-5D-5L utility of 40-year-olds in England. There were no statistically significant differences between the two groups at baseline. The mean EQ-5D-5L utility was higher in the CBT group at 12- and 24-week follow-up, but these differences were not statistically significant. More details on patient characteristics at baseline are provided eslewhere.Reference Freeman, Waite, Startup, Myers, Lister and McInerney7
Baseline demographics and EQ-5D-5L utilities
Mann–Whitney tests showed no statistically significant differences in mean scores.
CBT, cognitive–behavioural therapy
a. Pearson χ2(1) = 0.0575, Pr = 0.811.
b. Pearson χ2(5) = 4.1504, Pr = 0.528.
For the provision of the CBT, a psychologist spent on average 23 h per patient for treatment, administration tasks and travelling to home visits. The intervention costs were on average £490 per patient. For the detailed cost calculation, please see Appendix Table 2.4.
Table 2 presents the 6-month costs before and after randomisation and the differences between the two groups. Compared with the standard care alone group, patients in the CBT group had on average £1462 higher healthcare costs in the 6 months before randomisation, mainly owing to higher day hospital costs (i.e. £851) and higher hospital admission costs (i.e. £414). In the 6 months after randomisation, patients in the CBT group had on average £1532 lower healthcare costs than patients in the standard care alone group. The mean differences in total healthcare costs before and after randomisation were mainly driven by the mean differences in day hospital costs, from £851 higher in the CBT group to £2241 higher in the standard care alone group. It should be noted that the −£2241 difference in mean NHS costs after randomisation incorporates the £490 intervention costs in the treatment group. The difference in total costs from the societal perspective between the two groups decreased from £1055 to −£1236.
Observed costs over 24 weeks before and after randomisation
Non-parametric tests also showed no statistically significant differences between the two groups; the recall period used to calculate costs at baseline was 24 weeks.
CBT, cognitive–behavioural therapy; NHS, National Health Service
a. Including intervention costs.
Results of the cost-utility analysis
The results of the economic evaluation of the pilot study showed that the treatment leads on average to a 0.035 QALY gain (95% CI −0.016 to 0.084), as well as £1524 (95% CI −10 529 to 4736) and £1227 (95% CI −10 395 to 5361) lower costs from the NHS and societal perspectives, respectively (Table 3). Thus, these results indicate that CBT is likely to have higher benefits at lower costs.
Estimated costs and QALYs
These are the mean estimates from 10 000 bootstrapped samples adjusted for baseline costs or EQ-5D-5L utilities. For this reason, the costs reported here do not correspond with the observed unadjusted costs reported in Table 3.
CBT, cognitive behavioural therapy; NHS, National Health Service; QALY, quality-adjusted life year
However, there was large uncertainty in the ICERs, led primarily by uncertainty in the costs. This is illustrated in the cost-effectiveness plane (Fig. 1), which displays the point estimate as a black bold dot and the uncertainty around the mean estimate as the surrounding ‘cloud’ of ICERs. This plane shows QALY gains at 90% certainty (i.e. 90% of the 10 000 bootstrapped ICERs were in the right half of the CE plane) and high uncertainty about the costs. The cost-effectiveness plane with costs from the societal perspective was very similar (Appendix 3).
Cost-effectiveness plane (NHS perspective).
The EVPI analysis showed that the population EVPI was approximately £87 million at a £0 ceiling ratio and decreased to approximately £33 million at an £85 000 ceiling ratio. This negative slope is explained by the negative ICERs resulted by the main analysis. The results of the EVPI analysis are presented in Appendix 4.
Results of the sensitivity analyses
The results of the sensitivity analyses when using multiple imputation, propensity score matching or excluding medication costs were very similar to the results of the main analysis (Table 4). However, using regression analysis to adjust incremental costs and QALYs for their baseline values resulted in a 0.040 (95% CI 0.005 to 0.074) QALY gain, as well as £1319 (95% CI −412 to 4319) and £1579 (95% CI −410 to 5094) higher costs from the NHS and societal perspectives, respectively.
Results of the sensitivity analyses
The results of each sensitivity analysis are based on 10 000 bootstrapped pairs of costs and QALYs; the sample size in all sensitivity analyses is the same as in the main analysis (i.e. 19 patients in the CBT group and 24 patients in the standard care alone group) except for the imputed data-set sensitivity analyses that included 24 patients in the CBT group and 26 patients in the standard care alone group.
CBT, cognitive behavioural therapy; NHS, National Health Service; QALY, quality-adjusted life year
For comparison purposes, the CEAC of the main analysis is presented alongside those of the sensitivity analyses in Fig. 2. According to the CEAC based on the main analysis, the probability of the CBT being cost-effective was 59% at a £0 ceiling ratio (i.e. the sum of 54 and 5% of 10 000 bootstrapped ICERs in the south-east and north-west quadrants of the cost-effectiveness plane, respectively) and increased to 83% at a £85 000 ceiling ratio. NICE suggests that the maximum acceptable ICER (i.e. ceiling ratio) is between £20 000 and £30 000 per QALY gained. Considering this range, the probability of the CBT treatment being cost-effective was 66% at a £20 000 ceiling ratio and increased to 70% at £30 000. The CEACs based on the sensitivity analyses were similar to those for the main analysis, except for the CEAC of the regression-based sensitivity analysis, which displayed less favourable results for the CBT at ceiling ratios lower than £50 000.
Cost-effectiveness acceptability curve and population EVPI. SA, sensitivity analysis; MI, multiple imputation.
Discussion
Summary of main findings
This is the first economic evaluation of CBT for treating insomnia in people with schizophrenia. The results from the economic evaluation showed that the CBT delivered in the pilot BEST is likely to be more effective at lower costs in the short term compared with standard care alone. The results are subject to high uncertainty, driven primarily by large uncertainty in the costs. This might be explained by the small sample size and the large variation in costs among the patients included in the trial, as well as the study's short time horizon (i.e. 6 months post-randomisation).
Our results provide several indications that the CBT has the potential to be cost-effective (i.e. having an ICER between £20 000 and £30 000 or below). Although not statistically significant, the differences in costs between the two groups at baseline were considerable (i.e. £1462 NHS costs), favouring the standard care alone group. Even with this unequal staring point, patients in the CBT group had on average £1532 lower health and social care costs than patients in the standard care alone group during the follow-up period, indicating that CBT has the potential to be cost-effective. This potential is enhanced by the 90% certainty in the results for improvement in patient quality of life, which is expected to negatively affect costs in the longer term. In addition, we used very conservative assumptions when assessing the costs of the CBT intervention. For example, we used the upper range of additional contact time per treatment session (i.e. 20 min) and administration time per treatment session (i.e. 30 min). Relaxing these assumptions to be more realistic could also increase the likelihood of the CBT being cost-effective. The costs of productivity loss were not included in the analysis because there was only one person in employment in each group. However, the employment rate of people with schizophrenia in England is estimated to be between 5 and 15%.Reference Marwaha and Johnson14 If the costs of productivity loss were included in the analysis and the sample was more representative in terms of employment, the likelihood of the CBT being cost-effective might potentially be improved, as positive changes in quality of life could yield higher participation in the labour market. Provision of sleep treatment also needs to be considered in the wider context of psychosis services. Patients want help for sleep problems, appreciate these concerns being taken seriously and value the provision of highly effective treatment.Reference Waite, Evans, Myers, Startup, Lister and Harvey15 Positive experiences of the provision of sleep treatment are likely to enhance the take-up of effective treatments for other difficulties that the patients may experience.
The considerable uncertainty in the evidence available, however, warrants caution about the pilot results. Additional empirical information is needed to reduce further decision uncertainty (i.e. whether to provide CBT via the NHS). The results of the EVPI analysis suggest that the value of this additional information is high (approximately £87 million at a £0 ceiling ratio), given the large scale of the population affected by the decision (i.e. 100 000 patients with schizophrenia with sleeping disorders in England). A recent review concluded that economic evaluations with estimated EVPI values over £2 million were very unlikely to receive recommendations against further research.Reference Thorn, Coast and Andronis13 Hence, our results suggest that undertaking additional major commissioned research work, especially to collect more cost information, is necessary and worthwhile to further reduce the decision uncertainty.
Challenges, strengths, and limitations
Similar to many pilot trials, the economic evaluation faced several challenges because of the small sample size of BEST. The five patients with missing observations comprised 21% of the patients in the CBT group, while the remaining 19 patients with complete observations in this treatment arm were too few to perform reliable imputation methods. Thus, a complete cases analysis was preferred in the main analysis, although this was valid and efficient only under the assumption that missing observations in the outcome variables (i.e. costs and EQ-5D-5L utilities in our study) are missing at random (i.e. the probability that data are missing does not depend on the values of the missing data). However, this might not be the case for the pilot BEST data, considering that non-completers were more severely affected patients than completers in the CBT group (see Appendix Table 2.4). Moreover, the small sample size may have contributed to the relatively large differences in the mean costs between the two trial arms at baseline. We performed PSM and regression analysis to reduce these differences, but their results should also be interpreted with caution owing to the limited statistical power. The economic evaluation had a short time horizon (6-month follow-up), which precluded assessment of the effects of CBT on costs and outcomes in the patient's lifetime. Poor-quality information on medication use was another limitation of the study.
The major strength of this study was the performance of several analyses that addressed the uncertainty in the results of the economic evaluation. Another strength was the value of information analysis, which showed substantial monetary benefits of further study of the cost-effectiveness of the CBT. To overcome the limitations of this study, future studies could recruit a larger patient population that is more representative, especially in terms of employment, and which is likely to have better-balanced costs at baseline. Future economic studies could also adopt a long(er) time horizon in the analysis and obtain good-quality medication data either by incorporating suitable questions in a survey or accessing administrative data. The influence of the CBT on productivity and state benefits could also be investigated in future studies.
Implications for patients, clinicians and policy makers
Patients with schizophrenia have multiple complex health needs, as well as high rates of depression, suicidal ideation and poor physical health. The results of this study showed that treating pervasive sleep problems in this patient group with CBT is very likely to improve their quality of life in the short term. In the long term, such improvements could potentially reduce depression, psychotic experiences, irritability, exhaustion, drowsiness, reduced attention, poorer decision-making, cardiovascular disease, metabolic, abnormalities, weight gain, risk of type II diabetes and reduced immunity. Clinicians often use hypnotic medication to treat sleeping disorders. Clinical guidelines recommend CBT as the treatment of choice for chronic sleep difficulties. The results of this study indicate that CBT may be an effective and cost-effective intervention in this patient group. This alternative would also be aligned with patient preferences for psychological and behavioural-type therapies.Reference Waters, Chiu, Janca, Atkinson and Ree16 If proven to be cost-effective, the CBT and associated training could be implemented in the NHS via the Improving Access to Psychological Therapies (IAPT) programme, which has been expanded for patients with severe mental illnesses such as schizophrenia. Furthermore, the need for greater access to psychological treatments for patients with schizophrenia is well recognised, and substantial investment has been suggested for this purpose.17 Efficiency is a crucial factor for the NHS in making wise investments while facing limited budgets, especially in mental healthcare. Our results indicate that CBT as provided in BEST is likely to be cost-effective, but further evaluation is necessary before stating whether it is good value for money or not.
Conclusions
This study found that CBT for treating insomnia in people with schizophrenia is likely to be effective and cost-effective. A large trial designed to facilitate a thorough economic evaluation is needed to provide further evidence about its cost-effectiveness. This information is valuable and necessary to support decision makers in improving efficiency in mental healthcare in England.
Funding
The trial was funded by a grant from the NHS National Institute for Health Research (NIHR) Research for Patient Benefit Programme (reference PB-PG-0211-10007). The research was also supported by a Wellcome Trust Strategic Award (098461/Z/12/Z) for the University of Oxford Sleep and Circadian Neurosciences Institute. D.F. is supported by an NIHR Research Professorship.
The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the Research for Patient Benefit Programme, NIHR, NHS or the Department of Health.
Appendix 1 Trial flowchart
Appendix 2
List of the unit costs and their sources
Pattern of missing observations
Intervention costs
Differences at baseline between completers and non-completers
Appendix 3 Cost-effectiveness plane with costs from the societal perspective
Appendix 4 Population EVPI
Open access
eLetters
No eLetters have been published for this article.