Skip to main content Accessibility help
×
Home
Hostname: page-component-5c569c448b-6g96d Total loading time: 0.88 Render date: 2022-07-05T00:45:56.180Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "useRatesEcommerce": false, "useNewApi": true } hasContentIssue true

Appraisal of patient-level health economic models of severe mental illness: systematic review

Published online by Cambridge University Press:  19 August 2021

James Altunkaya
Affiliation:
Health Economics Research Centre, Nuffield Department of Population Health, University of Oxford, UK
Jung-Seok Lee
Affiliation:
Health Economics Research Centre, Nuffield Department of Population Health, University of Oxford, UK
Apostolos Tsiachristas
Affiliation:
Health Economics Research Centre, Nuffield Department of Population Health, University of Oxford, UK
Felicity Waite
Affiliation:
Department of Psychiatry, University of Oxford; and Oxford Health NHS Foundation Trust, Oxford, UK
Daniel Freeman
Affiliation:
Department of Psychiatry, University of Oxford; and Oxford Health NHS Foundation Trust, Oxford, UK
José Leal*
Affiliation:
Health Economics Research Centre, Nuffield Department of Population Health, University of Oxford, UK
*
Correspondence: José Leal. Email: jose.leal@dph.ox.ac.uk
Rights & Permissions[Opens in a new window]

Abstract

Background

Healthcare decision makers require accurate long-term economic models to evaluate the cost-effectiveness of new mental health interventions.

Aims

To assess the suitability of current patient-level economic models to estimate long-term economic outcomes in severe mental illness.

Method

We undertook pre-specified systematic searches in MEDLINE, Embase and PsycINFO to identify reviews and stand-alone publications of economic models of interventions for schizophrenia, bipolar disorder and major depressive disorder (PROSPERO: CRD42020158243). We screened paper titles and abstracts to identify unique patient-level economic models. We conducted a structured extraction of identified models, recording the presence of key predefined model features. Model quality and validation were appraised using the 2014 ISPOR and 2016 AdViSHE model checklists.

Results

We identified 15 unique patient-level models for psychosis and major depressive disorder from 1481 non-duplicate records. Models addressed schizophrenia (n = 6), bipolar disorder (n = 2) and major depressive disorder (n = 7). The predominant model type was discrete event simulation (n = 9). Model complexity and incorporation of patient heterogeneity varied considerably, and only five models extrapolated costs and outcomes over a lifetime horizon. Key model parameters were often based on low-quality evidence, and checklist quality assessment revealed weak model verification procedures.

Conclusions

Existing patient-level economic models of interventions for severe mental illness have considerable limitations. New modelling efforts must be supplemented by the generation of good-quality, contemporary evidence suitable for model building. Combined effort across the research community is required to build and validate economic extrapolation models suitable for accurately assessing the long-term value of new interventions from short-term clinical trial data.

Type
Review
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
Copyright © The Author(s), 2021. Published by Cambridge University Press on behalf of the Royal College of Psychiatrists

Severe mental illness typically causes significant functional impairment and consequent poor physical health.1,Reference Reilly, Olier, Planner, Doran, Reeves and Ashcroft2 Excess mortality among people with severe mental illness is as much as 2–3 times higher than in the general population, with multiple interacting causes.Reference Hert, Correll, Bobes, Cetkovich-Bakmas, Cohen and Asai3 These include much higher rates of preventable chronic disease, such as diabetes and cardiovascular disease.Reference Hert, Correll, Bobes, Cetkovich-Bakmas, Cohen and Asai3Reference Hjorthøj, Stürup, McGrath and Nordentoft5 Premature death from non-communicable disease is up to 60% more likely in people with severe mental illness.Reference Vigo, Thornicroft and Atun6 Life expectancy with severe mental illness is 10–20 years shorter in high-income countries and 30 years shorter in low-income countries.Reference Walker, McGee and Druss7Reference Fekadu, Medhin, Kebede, Alem, Cleare and Prince9 Recent discussions have suggested that people with severe mental illness should be prioritised for COVID-19 vaccination, given their increased risk of severe infection and COVID-19-related morbidity and mortality.Reference Mazereel, Van Assche, Detraux and De Hert10

Schizophrenia, bipolar disorder and major depressive disorder are all major contributors to the global burden of disease.Reference James, Abate, Abate, Abay, Abbafati and Abbasi11 These three conditions are identified as severe mental illness in this review. All three conditions are associated with the pattern of excess mortality described above.Reference De Hert, Dekker, Wood, Kahl, Holt and Möller1214 Quality of life is also severely diminished in individuals affected by each of these conditionsReference Evans, Banerjee, Leese and Huxley15 and each is associated with substantial functional impairment.Reference Harvey, Heaton, Carpenter, Green, Gold and Schoenbaum16Reference Rosa, Reinares, Michalak, Bonnin, Sole and Franco19 From an economic perspective, each of these mental disorders also carries substantial lifetime costs, borne by both individuals and health systems.Reference Seabury, Axeen, Pauley, Tysinger, Schlosser and Hernandez20 Although major depressive disorder is not always classified as a severe mental illness,Reference Evans, Banerjee, Leese and Huxley15 we include it in this review to capture severe depression that leads to psychiatric hospital admission.

Clinical trials of new interventions for these conditions are generally short term, and therefore do not measure the full scale of lifetime patient outcomes. Long-term evidence is necessary to inform decisions of which interventions should be implemented within healthcare systems such as the National Health Service (NHS).21 Economic models that estimate lifetime health and cost outcomes for individual patients are vital to understanding the long-term value of new interventions for severe mental illness.Reference Knapp and Wong22 We therefore examine patient-level economic models for the three conditions described.

Challenges in economic modelling in mental health are well described,Reference Shearer and Byford23 in particular those due to the short time horizon of clinical trials in mental healthReference Ginnelly and Manca24 and the wide scope of potential economic effects of mental disorders – including productivity losses and greater lifetime use of healthcare resources for the individual directly affected, as well as spill-over effects on economic outcomes for a patient's family and their wider community.Reference Knapp and Wong22 In severe mental illness, a recent systematic review of economic models assessing antipsychotic medication for schizophrenia found 90% of models to have ‘very serious’ quality limitations based on National Institute for Health and Care Excellence (NICE) checklist appraisal.Reference Jin, Tappenden, Robinson, Achilla, Aceituno and Byford25,26 There is concern that poor-quality economic evidence may be similarly widespread in bipolar disorder and major depressive disorder.Reference Abdul Pari, Simon, Wolstenholme, Geddes and Goodwin27,Reference Kolovos, Bosmans, Riper, Chevreul, Coupé and van Tulder28 Poor-quality economic modelling may lead to inefficient allocation of healthcare resources, misestimating the health and cost effects of alternative interventions.Reference Tappenden and Chilcott29 With increasing financial commitment and focus on improving outcomes in mental health,30 clinical commissioners urgently need accurate evidence on cost-effective care in severe mental illness.

Informing national treatment guidelines

In the UK, NICE has separate guidelines for psychosis, encompassing schizophrenia and bipolar disorder,31 and for major depressive disorder.32 In psychosis, guidelines recommend the use of psychological therapies, such as cognitive–behavioural therapy (CBT) and family intervention, alongside pharmacological treatment.31 Further treatments are under development, such as the use of virtual reality therapy to help patients overcome anxious avoidance of everyday social situations.Reference Freeman, Yu, Kabir, Martin, Craven and Leal33 However, the long-term effectiveness and cost-effectiveness of new and existing treatments remain poorly understood, especially in the context of varying real-world adherence to treatment.Reference Nosé, Barbui and Tansella34,Reference David35

Studies available to inform NICE guidelines for psychosis are largely characterised by short follow-up periods (up to 6 months) and small samples (an average of 79 participants per study).Reference Gastaldon, Mosler, Toner, Tedeschi, Bird and Barbui36 NICE explicitly identifies this as a key limitation in their psychosis guidance – as data for several parameters, including relapse and treatment discontinuation probabilities, require extrapolation to a lifetime horizon to capture the long-term impact of treatment on patient outcomes and costs.31 Psychosis is a severe and often enduring mental health problem, with most patients experiencing multiple episodes or persistent symptoms.31,Reference Knapp, Chisholm, Leese, Amaddeo, Tansella and Schene37,Reference Lehman38 However, in the absence of robust long-term evidence, it is not possible to confirm whether any extrapolation of short-term data either over- or underestimates the cost-effectiveness of different psychosis treatments.31

Short follow-up in clinical studies similarly affects NICE guidelines for major depressive disorder. The bespoke economic model constructed to inform current guidelines for pharmacological interventions in depression had a time horizon of only 14 months, limited by short study follow-up.32 The NICE guidance explicitly noted the variable methodological quality of the economic evaluation studies that were available to inform its policy-making.32 As in psychosis, economic models of major depressive disorder must have capacity to estimate the impact of new interventions over the expected duration of the long-term disease course. Major depressive disorder can be a chronic condition with a high risk of recurrence over a patient's lifetime.Reference Melartin, Rytsala, Leskela, Lestela-Mielonen, Sokero and Isometsa39 In a large prospective observational study in The Netherlands, nearly 20% of patients had a single major depressive episode lasting longer than 24 months.Reference Spijker, De Graaf, Bijl, Beekman, Ormel and Nolen40 Economic modelling over a short time horizon does not give a fair reflection of a new intervention's value to the health system during each patient's lifetime.

The present review

The aim of this systematic review is to summarise health economic models of schizophrenia, bipolar disorder and major depressive disorder and their potential to extrapolate short-term studies informing the long-term value of interventions for severe mental illness. We undertook this review to inform the extrapolation of the gameChange trial,Reference Freeman, Yu, Kabir, Martin, Craven and Leal33 to provide recommendations for the broader research community to help identify patient-level models suitable for extrapolating short-term trials in severe mental illness, and to help improve the quality of future patient-level models in this area.

We focus on models designed to simulate individual patients (patient-level models) as they capture variation in presentation that leads to highly individualised patient experiences and outcomes in severe mental illness, which cannot readily be subgrouped.Reference Russo, Levine, Demjaha, Di Forti, Bonaccorso and Fearon41,Reference Harrow, Grossman, Jobe and Herbener42 Patient-level models are distinct from cohort models in explicitly calculating the expected costs and benefits for each individual patient, rather than estimating average outcomes across a patient group.Reference Davis, Stevenson, Tappenden and Wailoo43 Compared with cohort model approaches, patient-level model structures are better able to represent complex interactions between patient characteristics and an evolving disease historyReference Karnon, Stahl, Brennan, Caro, Mar and Möller44 and to capture non-linear relationships between individual risk factors and modelled outcomes.Reference Davis, Stevenson, Tappenden and Wailoo43 By more closely representing variation in disease course driven by individual patient histories and characteristics as seen in severe mental illness, patient-level models can generate more accurate estimates of health and cost outcomes in the overall population.Reference Brennan, Chick and Davies45

Method

The protocol for the literature review was registered in the PROSPERO international prospective register of systematic reviews (registration number CRD42020158243). Using OVID, we searched MEDLINE, Embase and PsycINFO for health economic models of psychosis, schizophrenia, bipolar disorder and major depressive disorder published between 1986 and 26 August 2020 (the date of extraction). Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA 2009) guidelinesReference Moher, Liberati, Tetzlaff and Altman46 were followed and the checklist is reported in supplementary Appendix 1, available at https://doi.org/10.1192/bjp.2021.121. Search terms are reported in supplementary Appendix 2.

We used a two-stage approach to identify patient-level models in the review. First, we identified previous reviews of economic models for psychosis, schizophrenia, bipolar disorder and major depressive disorder. To achieve this, two reviewers screened titles and abstracts of identified records for reviews of economic models (both patient-level and non-patient-level models). Full-text records were requested for the reviews identified. Two reviewers extracted details of patient-level models reported in each review, alongside the databases searched and time periods covered by each review. Second, we updated the identified reviews by searching for all economic models published since the last date covered by the reviews.

The inclusion criteria used to identify relevant studies were as follows:

  1. (a) studies with decision models of disease progression (models estimating risk factor progression) that reported health economic outcomes such as costs, (quality-adjusted) life expectancy and disease-related complications (such as psychotic or depressive episodes and treatment side-effects);

  2. (b) studies with a model-based economic evaluation of intervention(s) in severe mental illness, such as cost–consequence, cost–utility and cost-effectiveness studies.

Searches were restricted to English language studies owing to challenges in locating and assessing non-English studies, given limited resources available to the research team, but no geographical restrictions were applied. Reference lists of identified economic models were also searched to identify any additional patient-level models missed by systematic searching. Abstracts and conference presentations reporting decision models were not included, as these did not provide sufficient information to allow critical appraisal of the models. For economic models identified across all conditions, patient-level economic models were extracted by reviewing titles and abstracts using keywords such as: ‘microsimulation’, ‘first-order Monte Carlo simulation’, ‘(Markov) patient-level’, ‘individual-level’ and ‘discrete-event simulation’.Reference Brennan, Chick and Davies45 References were managed using ENDNOTE X9.

There are several types of patient-level model.Reference Brennan, Chick and Davies45 A patient-level decision tree estimates each patient's expected health outcomes and costs without accounting for the timing of each modelled event (such as an in-patient stay or medication switch), other than the sequence in which each event occurs. However, most patient-level models do account for the timing as well as the sequence of modelled events. Patient-level Markov models simulate individual patients flowing between several health states, with transitions between states at fixed time cycles (such as each day, month or year). In discrete-event simulation models, the timing of each event is predicted precisely for each patient, so the timing of changes in each patient's health status is completely flexible, rather than occurring at fixed intervals. We include all types of patient-level model in our review.

A detailed extraction form was completed for each unique model to assess the suitability of current patient-level economic models to estimate long-term economic outcomes in severe mental illness, which is reported in supplementary Appendix 3. Clinical and health economic experts within the authorship group tailored the questions in the structured extraction to capture the key economic drivers within the disease course. If a decision model was found to be associated with multiple publications, data were extracted from the study that described the model in greatest detail, supported by other publications and relevant online documentation. Two reviewers each extracted all identified studies, with disagreements resolved by consensus.

The main outcomes analysed were:

  1. (a) the objective of the model

  2. (b) the model structure (modelling method, modelled states and links between states)

  3. (c) the model's inputs and corresponding assumptions for costs and quality/length of life

  4. (d) whether internal or external model validation/calibration was undertaken and documented.

A standardised checklist ranking a hierarchy of evidence quality was completed for each model, in which the data source used to inform a certain aspect of the model is awarded a score of 1 (highest quality) to 6 (lowest quality).Reference Cooper, Coyle, Abrams, Mugford and Sutton47 This provided a structured assessment of the quality of input data for key model parameters. Full ranking criteria for the grading of evidence sources are presented in supplementary Appendix 4.

Two reviewers completed quality checklists for each patient-level model identified. To assess model quality they used the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) checklist, as published in a 2014 Good Practice Task Force Report by ISPOR, the Academy of Managed Care Pharmacy and the National Pharmaceutical Council (ISPOR-AMCP-NPC).Reference Caro, Eddy, Kan, Kaltz, Patel and Eldessouki48 This checklist aims to establish a model's credibility and relevance for decision-making, indicating any ‘fatal flaws’ that could render the model's results inaccurate or incomplete. To assess model validation processes and reporting they used the 2016 Assessment of the Validation Status of Health-Economic Decision Models (AdViSHE) checklist.Reference Vemer, Corro Ramos, van Voorn, Al and Feenstra49 The AdViSHE checklist supports structured reporting of model validation and aims to increase model transparency.

Findings from the review were synthesised narratively. This systematic review was exempt from ethics approval and consent of participants, as this study was based on previously published work.

Results

Literature search

In total, 2479 papers were identified from the three databases, of which 1481 were non-duplicates (Fig. 1); 39 review papers were identified, with the 3 most recent reviews in each condition covering patient-level models published up to December 2015.Reference Jin, Tappenden, Robinson, Achilla, Aceituno and Byford25,Reference Kolovos, Bosmans, Riper, Chevreul, Coupé and van Tulder28,Reference Mavranezouli and Lokkerbol50 Inclusion criteria for these reviews closely match this study and are detailed in full in supplementary Appendix 5. To update previous systematic reviews, 572 papers published between 1 January 2016 and 26 August 2020 were identified from the 1481 non-duplicate papers. An additional 5 patient-level economic models not covered by previous systematic reviews were identified from the 572 papers. Hence, we identified 15 unique patient-level economic models from a total of 28 studies.Reference Vera-Llonch, Delea, Richardson, Rupnow, Grogg and Oster5165 Full detail of records assessed is provided in supplementary Appendix 6.

Fig. 1 Literature search – PRISMA diagram.

Description of health economic models

An overview of the 15 models is provided in Table 1. Seven patient-level models (47%) targeted major depressive disorder,Reference Sobocki, Ekman, Ågren, Jönsson and Rehnberg5965 six (40%) targeted schizophreniaReference Vera-Llonch, Delea, Richardson, Rupnow, Grogg and Oster51Reference Jin, Tappenden, MacCabe, Robinson and Byford56 and two models (13%) targeted bipolar I or bipolar II disorder.Reference Klok, Al Hadithy, van Schayk, Antonisse, Caro and Brouwers57,Reference Ekman, Lindgren, Miltenburger, Meier, Locklear and Chatterton58 Five models (33%) examined a UK setting,Reference Heeg, Buskens, Knapp, van Aalst, Dries and de Haan52,Reference Jin, Tappenden, MacCabe, Robinson and Byford56,Reference Ekman, Lindgren, Miltenburger, Meier, Locklear and Chatterton58,Reference Tosh, Kearns, Brennan, Parry, Ricketts and Saxon62,Reference Vataire, Aballéa, Antonanzas, Roijen, Lam and McCrone63 four models (27%) were set in North America,Reference Heeg, Buskens, Knapp, van Aalst, Dries and de Haan52,Reference Furiak, Ascher-Svanum, Klein, Smolen, Lawson and Conley53,55,65 three models (20%) had a European settingReference Dilla, Möller, O'Donohoe, Álvarez, Sacristán and Happich54,Reference Klok, Al Hadithy, van Schayk, Antonisse, Caro and Brouwers57,Reference Sobocki, Ekman, Ågren, Jönsson and Rehnberg59 and the remaining three models were set in Asia/Oceania.Reference Prukkanone, Vos, Bertram and Lim60,Reference Saylan, Treur, Postema, Dilbaz, Savas and Heeg61,Reference Nguyen and Gordon64

Table 1 Description of the 15 included studies

CBT, cognitive–behavioural therapy; CCA, cost–consequence analysis; CEA, Cost-effectiveness analysis; CUA, cost–utility analysis; DES, discrete-event simulation; PLDT, patient-level decision tree; PLMM, patient-level Markov model; QALY, quality-adjusted life-years; rTMS, repetitive transcranial magnetic stimulation.

a. Incidence of relapse of symptoms and side-effects, therapy discontinuation, expected costs of antipsychotic therapy, psychiatric and non-psychiatric services.

b. Cost per relapse avoided, total time in psychosis, total costs of psychosis.

c. Time to response/remission, side-effects, non-adherence, length of stay, total costs.

The majority of models (n = 11, 73%) evaluated pharmacological interventions,Reference Vera-Llonch, Delea, Richardson, Rupnow, Grogg and Oster51Reference Dilla, Möller, O'Donohoe, Álvarez, Sacristán and Happich54,Reference Jin, Tappenden, MacCabe, Robinson and Byford56Reference Saylan, Treur, Postema, Dilbaz, Savas and Heeg61,Reference Vataire, Aballéa, Antonanzas, Roijen, Lam and McCrone63 five models (33%) evaluated non-pharmacological individual interventions55,Reference Jin, Tappenden, MacCabe, Robinson and Byford56,Reference Prukkanone, Vos, Bertram and Lim60,Reference Nguyen and Gordon64,65 and one model (7%) examined the effect of system-level reorganisation.Reference Tosh, Kearns, Brennan, Parry, Ricketts and Saxon62 Models were generally structured as discrete-event simulations (n = 9, 60%)Reference Heeg, Buskens, Knapp, van Aalst, Dries and de Haan52,Reference Dilla, Möller, O'Donohoe, Álvarez, Sacristán and Happich54,Reference Jin, Tappenden, MacCabe, Robinson and Byford56Reference Ekman, Lindgren, Miltenburger, Meier, Locklear and Chatterton58,Reference Prukkanone, Vos, Bertram and Lim60Reference Vataire, Aballéa, Antonanzas, Roijen, Lam and McCrone63 or individual-level Markov models (n = 5, 33%),Reference Vera-Llonch, Delea, Richardson, Rupnow, Grogg and Oster51,Reference Furiak, Ascher-Svanum, Klein, Smolen, Lawson and Conley53,55,Reference Sobocki, Ekman, Ågren, Jönsson and Rehnberg59,65 although there was also an individual-level decision tree (n = 1, 7%).Reference Nguyen and Gordon64 The majority of models (n = 12, 80%) took a health system or payer perspectiveReference Vera-Llonch, Delea, Richardson, Rupnow, Grogg and Oster51Reference Ekman, Lindgren, Miltenburger, Meier, Locklear and Chatterton58,Reference Saylan, Treur, Postema, Dilbaz, Savas and Heeg61,Reference Tosh, Kearns, Brennan, Parry, Ricketts and Saxon62,Reference Nguyen and Gordon64,65 and three models (20%) considered a societal perspective.Reference Sobocki, Ekman, Ågren, Jönsson and Rehnberg59,Reference Prukkanone, Vos, Bertram and Lim60,Reference Vataire, Aballéa, Antonanzas, Roijen, Lam and McCrone63

Five models (33%) considered a lifetime perspective for their analysis,55,Reference Jin, Tappenden, MacCabe, Robinson and Byford56,Reference Saylan, Treur, Postema, Dilbaz, Savas and Heeg61,Reference Tosh, Kearns, Brennan, Parry, Ricketts and Saxon62,65 with the most common time horizon considered being 5 years (n = 7, 47%).Reference Heeg, Buskens, Knapp, van Aalst, Dries and de Haan52,Reference Dilla, Möller, O'Donohoe, Álvarez, Sacristán and Happich54,Reference Ekman, Lindgren, Miltenburger, Meier, Locklear and Chatterton58Reference Prukkanone, Vos, Bertram and Lim60,Reference Vataire, Aballéa, Antonanzas, Roijen, Lam and McCrone63,Reference Nguyen and Gordon64 Two models (13%) considered a 1-year time horizonReference Vera-Llonch, Delea, Richardson, Rupnow, Grogg and Oster51,Reference Furiak, Ascher-Svanum, Klein, Smolen, Lawson and Conley53 and one model (7%) used a time horizon of only 100 days.Reference Klok, Al Hadithy, van Schayk, Antonisse, Caro and Brouwers57

Twelve models (80%) informed cost–utility analysis (CUA) and estimated health outcomes in terms of quality-adjusted (QALY) or disability-adjusted life-years (DALY).Reference Furiak, Ascher-Svanum, Klein, Smolen, Lawson and Conley53Reference Jin, Tappenden, MacCabe, Robinson and Byford56,Reference Ekman, Lindgren, Miltenburger, Meier, Locklear and Chatterton5865 The remaining three models (20%) estimated health outcomes such as relapse,Reference Vera-Llonch, Delea, Richardson, Rupnow, Grogg and Oster51,Reference Heeg, Buskens, Knapp, van Aalst, Dries and de Haan52,Reference Klok, Al Hadithy, van Schayk, Antonisse, Caro and Brouwers57 side-effects,Reference Vera-Llonch, Delea, Richardson, Rupnow, Grogg and Oster51,Reference Klok, Al Hadithy, van Schayk, Antonisse, Caro and Brouwers57 treatment discontinuationReference Vera-Llonch, Delea, Richardson, Rupnow, Grogg and Oster51,Reference Klok, Al Hadithy, van Schayk, Antonisse, Caro and Brouwers57 and time in a psychotic state.Reference Heeg, Buskens, Knapp, van Aalst, Dries and de Haan52,Reference Klok, Al Hadithy, van Schayk, Antonisse, Caro and Brouwers57 These three models estimated the total disease costs and presented disaggregated results in the form of cost–consequence analyses.Reference Vera-Llonch, Delea, Richardson, Rupnow, Grogg and Oster51,Reference Heeg, Buskens, Knapp, van Aalst, Dries and de Haan52,Reference Klok, Al Hadithy, van Schayk, Antonisse, Caro and Brouwers57

Structural complexity

Table 2 summarises the features of the individual models. Simplified model structures for each study are presented in supplementary Appendix 7. All studies modelled an episodic/relapse state and a separate health state representing time not in episode. All models allowed participants to experience multiple relapses, except for the single model (7%) structured as a decision tree, which allowed only one relapse.Reference Nguyen and Gordon64

Table 2 Complexity of the 15 models

Two models (13%) explicitly modelled incident disease in otherwise healthy populations,Reference Jin, Tappenden, MacCabe, Robinson and Byford56,Reference Tosh, Kearns, Brennan, Parry, Ricketts and Saxon62 whereas the remaining models considered populations with established psychosis or major depressive disorder. All but one model (7%)Reference Tosh, Kearns, Brennan, Parry, Ricketts and Saxon62 incorporated an excess mortality risk to reflect the increased risk of death by suicide in populations with psychosis or major depressive disorder, compared with the general population. Eleven models (73%) incorporated parameter uncertainty in their modelled results, conducting a probabilistic sensitivity analysis informed by non-arbitrary parameter distributions.Reference Heeg, Buskens, Knapp, van Aalst, Dries and de Haan52,Reference Dilla, Möller, O'Donohoe, Álvarez, Sacristán and Happich54Reference Jin, Tappenden, MacCabe, Robinson and Byford56,Reference Ekman, Lindgren, Miltenburger, Meier, Locklear and Chatterton58Reference Tosh, Kearns, Brennan, Parry, Ricketts and Saxon62,Reference Nguyen and Gordon64,65

In six models (40%) relapse severity was modelled explicitly.Reference Heeg, Buskens, Knapp, van Aalst, Dries and de Haan52,Reference Furiak, Ascher-Svanum, Klein, Smolen, Lawson and Conley53,Reference Ekman, Lindgren, Miltenburger, Meier, Locklear and Chatterton58,Reference Sobocki, Ekman, Ågren, Jönsson and Rehnberg59,Reference Nguyen and Gordon64,65 Of these, three models (20%) categorised two types of severity: a more severe relapse requiring hospital admission and a less severe relapse with the patient remaining in the community.Reference Furiak, Ascher-Svanum, Klein, Smolen, Lawson and Conley53,Reference Nguyen and Gordon64,65 The remaining three models allowed a more continuous distribution of episode severity based on patient characteristicsReference Heeg, Buskens, Knapp, van Aalst, Dries and de Haan52,Reference Sobocki, Ekman, Ågren, Jönsson and Rehnberg59 or simply modelled relapse severity based on random chance.Reference Ekman, Lindgren, Miltenburger, Meier, Locklear and Chatterton58 The latter three models varied time to remission from relapse, so specific patients could experience a shorter or longer relapse than the cohort average. One studyReference Heeg, Buskens, Knapp, van Aalst, Dries and de Haan52 modelled time-updated disease severity scores for individual patients, which jointly determined the frequency, length and symptom severity of each episode.

Nine models (60%) distinguished between those experiencing temporary relief of symptoms (remission between episodes) and those experiencing longer-term relief of symptoms (recovery), with these states being characterised by different costs and/or health outcomes.Reference Heeg, Buskens, Knapp, van Aalst, Dries and de Haan52,Reference Dilla, Möller, O'Donohoe, Álvarez, Sacristán and Happich54,Reference Jin, Tappenden, MacCabe, Robinson and Byford56,Reference Sobocki, Ekman, Ågren, Jönsson and Rehnberg59,Reference Saylan, Treur, Postema, Dilbaz, Savas and Heeg6165 In six models (40%), recovery was simulated by transition to a discrete ‘recovery state’.Reference Jin, Tappenden, MacCabe, Robinson and Byford56,Reference Sobocki, Ekman, Ågren, Jönsson and Rehnberg59,Reference Tosh, Kearns, Brennan, Parry, Ricketts and Saxon6265 However, two models (13%) used a gradient of recovery (via a continuous variable or categorical variable with several levels) based on the time since last relapse and category of disease severity as determined by baseline patient characteristics.Reference Heeg, Buskens, Knapp, van Aalst, Dries and de Haan52,Reference Dilla, Möller, O'Donohoe, Álvarez, Sacristán and Happich54

Thirteen models (87%) explicitly simulated changes in treatment status by incorporating treatment switches and/or periods of non-adherence.Reference Vera-Llonch, Delea, Richardson, Rupnow, Grogg and Oster51Reference Ekman, Lindgren, Miltenburger, Meier, Locklear and Chatterton58,Reference Prukkanone, Vos, Bertram and Lim60,Reference Tosh, Kearns, Brennan, Parry, Ricketts and Saxon6265 Seven models (47%) considered both treatment switching and treatment discontinuation,Reference Vera-Llonch, Delea, Richardson, Rupnow, Grogg and Oster51Reference Dilla, Möller, O'Donohoe, Álvarez, Sacristán and Happich54,Reference Jin, Tappenden, MacCabe, Robinson and Byford56Reference Ekman, Lindgren, Miltenburger, Meier, Locklear and Chatterton58 for example low treatment adherence driving treatment switching, whereas six models (40%) incorporated only treatment switching.55,Reference Saylan, Treur, Postema, Dilbaz, Savas and Heeg6165

Nine models (60%) predicted the impact of one or more side-effects resulting from antipsychotic treatments on patient health outcomes and costs.Reference Vera-Llonch, Delea, Richardson, Rupnow, Grogg and Oster51Reference Ekman, Lindgren, Miltenburger, Meier, Locklear and Chatterton58,Reference Vataire, Aballéa, Antonanzas, Roijen, Lam and McCrone63,Reference Nguyen and Gordon64 However, the type of side-effects incorporated varied considerably between models. Eight models (53%) incorporated the impact of extrapyramidal symptomsReference Vera-Llonch, Delea, Richardson, Rupnow, Grogg and Oster51Reference Ekman, Lindgren, Miltenburger, Meier, Locklear and Chatterton58 and seven models (47%) incorporated weight gain.Reference Vera-Llonch, Delea, Richardson, Rupnow, Grogg and Oster51Reference Dilla, Möller, O'Donohoe, Álvarez, Sacristán and Happich54,Reference Jin, Tappenden, MacCabe, Robinson and Byford56Reference Ekman, Lindgren, Miltenburger, Meier, Locklear and Chatterton58 Prolactin-related disorders,Reference Vera-Llonch, Delea, Richardson, Rupnow, Grogg and Oster51,Reference Heeg, Buskens, Knapp, van Aalst, Dries and de Haan52,55 neutropeniaReference Heeg, Buskens, Knapp, van Aalst, Dries and de Haan52,55,Reference Jin, Tappenden, MacCabe, Robinson and Byford56 and drowsinessReference Heeg, Buskens, Knapp, van Aalst, Dries and de Haan52,Reference Dilla, Möller, O'Donohoe, Álvarez, Sacristán and Happich54,Reference Vataire, Aballéa, Antonanzas, Roijen, Lam and McCrone63 were each considered in three models (20%), and tardive dyskinesiaReference Heeg, Buskens, Knapp, van Aalst, Dries and de Haan52,Reference Dilla, Möller, O'Donohoe, Álvarez, Sacristán and Happich54 and sexual dysfunctionReference Dilla, Möller, O'Donohoe, Álvarez, Sacristán and Happich54,Reference Vataire, Aballéa, Antonanzas, Roijen, Lam and McCrone63 were each considered in two models (13%). Four models (27%) considered a disparate range of other side-effects,Reference Furiak, Ascher-Svanum, Klein, Smolen, Lawson and Conley53,Reference Dilla, Möller, O'Donohoe, Álvarez, Sacristán and Happich54,Reference Vataire, Aballéa, Antonanzas, Roijen, Lam and McCrone63,Reference Nguyen and Gordon64 including insomnia, diarrhoea and post-injection syndrome.

Three models (20%) extended their modelling of medication-related side-effects,Reference Furiak, Ascher-Svanum, Klein, Smolen, Lawson and Conley53,55,Reference Jin, Tappenden, MacCabe, Robinson and Byford56 explicitly simulating a pathway from short-term transient side-effects into long-term comorbidities: diabetesReference Furiak, Ascher-Svanum, Klein, Smolen, Lawson and Conley53,55,Reference Jin, Tappenden, MacCabe, Robinson and Byford56 and cardiovascular disease.Reference Furiak, Ascher-Svanum, Klein, Smolen, Lawson and Conley53,55 The risk of developing long-term comorbidities was conditional on side-effects such as weight gain, via mediating pathways such as hyperlipidaemia and impaired glucose tolerance. In contrast, one model accounted for the long-term impact of comorbidities implicitly,Reference Prukkanone, Vos, Bertram and Lim60 with its authors applying a utility adjustment to the whole patient cohort, taking into account the incidence rate of medication-related comorbidities and their average health effect across all patients.

The medication-related side-effects incorporated by each model, and further detail of the precise modelling approach for long-term comorbidities, are presented in supplementary Appendix 8.

Incorporation of individual heterogeneity and patient history

Six patient-level models (40%) allowed the risk of a relapse to vary conditional on patient characteristicsReference Heeg, Buskens, Knapp, van Aalst, Dries and de Haan52,Reference Jin, Tappenden, MacCabe, Robinson and Byford56,Reference Sobocki, Ekman, Ågren, Jönsson and Rehnberg59,Reference Tosh, Kearns, Brennan, Parry, Ricketts and Saxon62,Reference Vataire, Aballéa, Antonanzas, Roijen, Lam and McCrone63,65 – mainly disease severity,Reference Heeg, Buskens, Knapp, van Aalst, Dries and de Haan52,Reference Jin, Tappenden, MacCabe, Robinson and Byford56,Reference Sobocki, Ekman, Ågren, Jönsson and Rehnberg59,Reference Tosh, Kearns, Brennan, Parry, Ricketts and Saxon62,Reference Vataire, Aballéa, Antonanzas, Roijen, Lam and McCrone63,65 ageReference Heeg, Buskens, Knapp, van Aalst, Dries and de Haan52,Reference Jin, Tappenden, MacCabe, Robinson and Byford56,Reference Sobocki, Ekman, Ågren, Jönsson and Rehnberg59,Reference Tosh, Kearns, Brennan, Parry, Ricketts and Saxon62,Reference Vataire, Aballéa, Antonanzas, Roijen, Lam and McCrone63 and gender.Reference Heeg, Buskens, Knapp, van Aalst, Dries and de Haan52,Reference Jin, Tappenden, MacCabe, Robinson and Byford56,Reference Sobocki, Ekman, Ågren, Jönsson and Rehnberg59,Reference Vataire, Aballéa, Antonanzas, Roijen, Lam and McCrone63 Full details of all patient characteristics considered in each model are provided in supplementary Appendix 8. The remaining nine models (60%) assumed the risk of first relapse to be equal across all individuals.Reference Vera-Llonch, Delea, Richardson, Rupnow, Grogg and Oster51,Reference Furiak, Ascher-Svanum, Klein, Smolen, Lawson and Conley5355,Reference Klok, Al Hadithy, van Schayk, Antonisse, Caro and Brouwers57,Reference Ekman, Lindgren, Miltenburger, Meier, Locklear and Chatterton58,Reference Prukkanone, Vos, Bertram and Lim60,Reference Saylan, Treur, Postema, Dilbaz, Savas and Heeg61,Reference Nguyen and Gordon64 In eight models (53%), including five models that varied relapse risk on the basis of patient characteristics, the risk of subsequent relapse was conditional on the number of previous relapses modelled.Reference Heeg, Buskens, Knapp, van Aalst, Dries and de Haan5255,Reference Sobocki, Ekman, Ågren, Jönsson and Rehnberg59,Reference Tosh, Kearns, Brennan, Parry, Ricketts and Saxon62,Reference Vataire, Aballéa, Antonanzas, Roijen, Lam and McCrone63,65 The risk of subsequent relapse varied in complexity. The simplest approach applied a single hazard ratio adjustment if a patient had any previous relapseReference Sobocki, Ekman, Ågren, Jönsson and Rehnberg59 and the most complex approach modelled future relapse risk as a continuous function driven by the duration of and time between previous relapses.Reference Heeg, Buskens, Knapp, van Aalst, Dries and de Haan52 In the remaining seven models (47%), the relapse risk was independent of the number of previous relapses modelled.Reference Vera-Llonch, Delea, Richardson, Rupnow, Grogg and Oster51,Reference Jin, Tappenden, MacCabe, Robinson and Byford56Reference Ekman, Lindgren, Miltenburger, Meier, Locklear and Chatterton58,Reference Prukkanone, Vos, Bertram and Lim60,Reference Saylan, Treur, Postema, Dilbaz, Savas and Heeg61,Reference Nguyen and Gordon64

Hierarchy of evidence informing the models

The hierarchy of evidence used in the models is summarised in Fig. 2, ranging from high-quality evidence (ranked 1) to the lowest rank of evidence (rank 6). Full ranking criteria for evidence sources across all categories are presented in supplementary Appendix 4.

Fig. 2 Hierarchy of evidence quality. NA, not applicable.

Quality of evidence informing the models was mixed, and few models used high-quality evidence across all model elements. Evidence was particularly poor for treatment effect extrapolation, with seven models (47%) relying on expert opinion to inform treatment effect extrapolation beyond observed data to model efficacy over the whole simulated time.Reference Heeg, Buskens, Knapp, van Aalst, Dries and de Haan52Reference Dilla, Möller, O'Donohoe, Álvarez, Sacristán and Happich54,Reference Jin, Tappenden, MacCabe, Robinson and Byford56,Reference Ekman, Lindgren, Miltenburger, Meier, Locklear and Chatterton58,Reference Prukkanone, Vos, Bertram and Lim60,Reference Nguyen and Gordon64 Often, there was no report of external expert consultation, so the opinion on the durability of the treatment effect over the time horizon was tacitly assumed by the model authors.Reference Dilla, Möller, O'Donohoe, Álvarez, Sacristán and Happich54,Reference Ekman, Lindgren, Miltenburger, Meier, Locklear and Chatterton58,Reference Prukkanone, Vos, Bertram and Lim60,Reference Nguyen and Gordon64 Two models (13%) did not attempt to extrapolate beyond their observed evidence sourcesReference Vera-Llonch, Delea, Richardson, Rupnow, Grogg and Oster51,Reference Klok, Al Hadithy, van Schayk, Antonisse, Caro and Brouwers57 and two models (13%) evaluated hypothetical treatments,Reference Tosh, Kearns, Brennan, Parry, Ricketts and Saxon62,Reference Vataire, Aballéa, Antonanzas, Roijen, Lam and McCrone63 meaning that this evidence category was not applicable to these four studies. Side-effect data were also often poorly evidenced, with studies often reporting several sources of input data with poor reporting of how each evidence source was used or how studies were combined to inform model parameters.

Model quality and validity

Models generally performed reasonably against ISPOR 2014 checklist criteria, which provide a broad overview of model quality and relevance (Fig. 3). Model design and analysis were generally adequate, with model credibility weakest in terms of data and reporting. Although the ISPOR checklist provides a high-level perspective over many diverse model attributes, only one checklist item directly scrutinises the model structure. Individual checklist items encompass multiple diverse topic areas. For example, short model time horizons are only assessed within a broad checklist item assessing the applicability of the model ‘context’. In terms of model credibility, the models performed poorly against ISPOR validation criteria, as corroborated by poor performance on the AdVISHE validation checklist (Fig. 3).

Fig. 3 Validation and quality checklists. ISPOR, International Society for Pharmacoeconomics and Outcomes; AdViSHE, Assessment of the Validation Status of Health-Economic Decision Models; NA, not applicable.

Of the 15 studies, 13 (87%) performed at least 1 of the 12 validation checks listed on the AdVISHE checklist.Reference Vera-Llonch, Delea, Richardson, Rupnow, Grogg and Oster51Reference Jin, Tappenden, MacCabe, Robinson and Byford56,Reference Ekman, Lindgren, Miltenburger, Meier, Locklear and Chatterton58,Reference Sobocki, Ekman, Ågren, Jönsson and Rehnberg59,Reference Saylan, Treur, Postema, Dilbaz, Savas and Heeg6165 However, model validation was generally restricted to face validity checks of model structure and the suitability of model inputs, and checks of model structure and inputs against published literature. Importantly, data and output validation was reported by only a handful of models. Two models (13%) reported validation of regression model fit,Reference Sobocki, Ekman, Ågren, Jönsson and Rehnberg59,Reference Nguyen and Gordon64 one model (7%) reported testing with alternative input dataReference Furiak, Ascher-Svanum, Klein, Smolen, Lawson and Conley53 and three models (20%) reported validation checks against empirical data.Reference Vera-Llonch, Delea, Richardson, Rupnow, Grogg and Oster51,Reference Jin, Tappenden, MacCabe, Robinson and Byford56,Reference Ekman, Lindgren, Miltenburger, Meier, Locklear and Chatterton58 Validation of the model as implemented in software (i.e. the computerised model) was particularly infrequent. Model authors rarely sought external expert model appraisal and rarely reported that basic model checks had been undertaken, such as extreme value testing or patient tracking through the computerised model. Jin et al (2020)Reference Jin, Tappenden, MacCabe, Robinson and Byford56 was an exception – providing full details of their validation procedures in supplementary material, documenting 10 of the 12 AdVISHE validation checks.

Full data for both quality checklists are presented in supplementary Appendix 9.

Discussion

Our review identified 15 unique patient-level economic models that simulated the natural history of schizophrenia, bipolar disorder and major depressive disorder. Our broad definition of severe mental illness allowed us to capture additional models that included severe depression leading to hospital admission. Antipsychotic medications can be used to treat all the conditions considered,Reference Wang and Si66,67 with common long-term comorbidities arising from specific medication-related side-effects. This is the first systematic review of economic models comparing patient-level model structures across schizophrenia, bipolar disorder and major depressive disorder, thus examining common economic modelling considerations across these patient groups.

We found considerable limitations in the quality and validity of current models. Outdated input data, lack of structural complexity and limited incorporation of patient heterogeneity are major concerns limiting models’ applicability to contemporary populations with severe mental illness. Only five models adopted the lifetime time horizon required by health technology reimbursement agencies.21 Current models therefore have limited potential to reliably extrapolate the results of short-term studies and thus inadequately inform decision makers’ assessment of the long-term value of existing and future interventions for severe mental illness.

Data quality

The data used to inform the models were generally of poor quality or were published more than 10 years ago. In psychosis, NICE treatment guidelines in England were last published in 201431 and more recent changes to the service pathway, including new waiting-time targets, have also substantially altered baseline care.68Reference Reichert and Jacobs70 Outdated model data are a significant concern, as projections derived from models are unlikely to be relevant to current decision-making owing to shifts in best practice care over time. In several models, related model parameters such as baseline relapse/remission rates and treatment effectiveness were obtained from different patient populations, with little or no adjustment to account for varying patient characteristics between evidence sources. Several models were informed by population data from regions and countries outside the setting of the model's decision problem, raising concerns about data transferability. Furthermore, despite the majority of models estimating QALYs or DALYs, the data informing quality-of-life weights were generally of medium/poor quality.

Decision models should be based on high-quality and contemporary evidence to ensure that estimates of the scale and severity of disease burden and economic benefit of new interventions are sufficiently reliable. If good economic evidence is not available to support further investment in severe mental illness, healthcare decision makers will choose to prioritise the allocation of scarce healthcare resources in other disease areas where there is better evidence to support potential economic benefit from any additional investment. The poor quality and outdated data used in many of the identified decision models suggests that new large high-quality studies in severe mental illness are needed to construct economic models. New evidence is needed in almost all model aspects – both clinical and economic – with particular need to obtain contemporary quality-of-life data in severe mental illness using modern methods.

Model complexity

Short-term side-effects and long-term comorbidities associated with treatment

There was considerable variation in model complexity. Although most models allowed for treatment switching, there was substantial divergence in the simulation of treatment-related side-effects. Side-effects are a primary driver of antipsychotic treatment switching.Reference Roussidis, Kalkavoura, Dimelis, Theodorou, Ioannidou and Mellos71 In clinical practice, a systematic approach to medication selection may be complemented by an element of ‘trial and error’ of several medications, in response to variation in the occurrence and acceptability of specific side-effects in different patients.Reference Arango, Kapur and Kahn72,Reference Buckley and Miller73 Models therefore should incorporate all relevant side-effects resulting from multiple possible treatment choices. This is particularly important early in the disease course, when a patient's individual presentation is emerging and treatment switches are more frequent.Reference Derks, Fleischhacker, Boter, Peuskens and Kahn74 Given evidence of side-effects that are common to both first-generation ‘conventional’ and second-generation ‘atypical’ antipsychotic drug classes,Reference Leucht, Corves, Arbter, Engel, Li and Davis75 it is therefore advisable for models to consider a wide range of potential side-effects.

Most models failed to incorporate the link between short-term side-effects and long-term medication-related comorbidities. Diabetes and cardiovascular disease are common comorbidities in populations with severe mental illness, and both have distinct well-studied effects on length and quality of life, and related healthcare costs.Reference Ward and Druss76,Reference Correll, Solmi, Veronese, Bortolato, Rosson and Santonastaso77 Omitting comorbidities arising from medication side-effects may bias a comparative assessment of the long-term value of different treatments for severe mental illness. Bias is particularly likely when an intervention and comparator induce side-effects of differing magnitude. By modelling distinct pathways for patients who develop comorbidities, the health effects of medication-related comorbidities can be distinguished from health effects and costs arising directly from a mental health condition. This means that prevention and treatment strategies for comorbidities can be easily scrutinised and updated with changes to current practice, without affecting the calculation of health and cost outcomes in patients without medication-related comorbidities. However, only two models55,Reference Jin, Tappenden, MacCabe, Robinson and Byford56 identified in our review considered the impact of these long-term comorbidities over a patient's lifetime.

Patient heterogeneity

Few models incorporated all relevant aspects of patient heterogeneity and comprehensively estimated the impact of patient heterogeneity on health outcomes and costs. Only three models estimated individualised severity and frequency of relapse, based on each patient's baseline characteristics and modelled relapse history.Reference Heeg, Buskens, Knapp, van Aalst, Dries and de Haan52,Reference Sobocki, Ekman, Ågren, Jönsson and Rehnberg59,65 Failure to model interactions between patient characteristics and evolving disease history will produce inaccurate estimates of population-level health outcomes and costs.Reference Karnon, Stahl, Brennan, Caro, Mar and Möller44 For example, in psychosis a significant proportion of health service costs are driven by a subset of patients who have both a significant history of relapse and high degree of dependency on statutory social services.Reference Byford, Barber, Fiander, Marshall and Green78 By incorporating all relevant aspects of patient heterogeneity, including patient heterogeneity in excess mortality risk, models can more accurately estimate population-level outcomes. Quantification of expected outcomes for individual patients is also necessary to establish the overall value of stratified patient care.

Model validity

Beyond structural inadequacies, most models did not report rigorous validation checks to establish confidence in their results. Through the appraisal checklists, we found that authors were most likely to report cross-comparability or face validity of their model structure or results, based either on existing literature or on consultation with a clinical expert. However, few models documented more extensive validation efforts. Few models compared model predictions against empirical data, tested the robustness of model outputs using alternative input data or subjected the software underlying the model to thorough checking procedures or an external expert review. Only one model thoroughly documented most validation procedures from the 2016 AdViSHE checklist.Reference Jin, Tappenden, MacCabe, Robinson and Byford56

Nevertheless, decision makers need validated economic models to provide a foundation for credible long-term policy-making and treatment reimbursement decisions. Researchers also need evidence that economic models are sufficiently valid to extrapolate data from clinical trials. Validation efforts provide insight into model accuracy and establish the credibility of evidence generated to inform healthcare investment decisions.Reference Kent, Becker, Feenstra, Tran-Duy, Schlackow and Tew79 Economic models generate predictions regarding the stream of future costs and benefits from changes to healthcare policy. Without internal and external validation, decision makers have limited knowledge of whether an economic model produces sufficiently accurate predictions for their own setting. In this regard, additional external validation of economic models is essential to enable decision makers to be confident in models’ ability to discriminate between cost-effective and cost-ineffective health policies. Without validated models, healthcare decision makers may judge the uncertainty involved with a new investment in severe mental illness to be too great. The lack of high-quality and validated models may result in patients being denied good-value care, as depending on the model used, a range of long-term cost-effectiveness estimates could be obtained indicating different policy decisions.Reference Kim and Thompson80

Limitations

This review has some limitations. First, we included only studies published in English, and therefore models developed for non-Anglophone decision contexts, where decision makers may have different evaluation requirements are probably underrepresented. Second, we did not include cohort-level models that could be used to extrapolate short-term studies. However, given that cohort models cannot fully capture the effect of patient heterogeneity in severe mental illness, these models were of limited interest. Third, our assessment of model quality is non-context specific – we did not address model suitability for each particular decision problem being addressed, instead providing an overview of each model's performance against a general benchmark standard. Finally, our assessment of model quality is contingent on the level of detail provided in publications about the model – with publication detail potentially constrained by word-count limits.

Implications for future research and policy makers

The deficiencies of current models documented in this review across multiple dimensions can be used to inform the design of future models. In bipolar disorder and major depression, no structurally complex lifetime models with well-documented validation were identified. In schizophrenia, although Jin et al's modelReference Jin, Tappenden, MacCabe, Robinson and Byford56 was shown to be of a higher quality and more structurally complex than its peers with well-documented internal validity, additional research is needed to demonstrate its external validity and accuracy to extrapolate clinical trial data for informing healthcare decision-making.

Poor-quality economic models hinder policy makers’ ability to allocate healthcare budgets appropriately. In turn, this reduces the ability of clinicians to improve performance against NHS mental health targets while simultaneously providing cost-effective care. There is a clear need for further development of contemporary and comprehensive patient-level decision models that capture the full structural complexity of severe mental illness, in particular its relation to long-term comorbidities. High-quality contemporary evidence is needed on health-related quality of life and costs (collected in line with current best practice), as well as on long-term disease progression to inform the development of robust economic models. Following extensive internal and external validation exercises, new economic models could significantly reduce the time needed to make health policy decisions by reliably extrapolating short-term clinical trials to inform the cost-effectiveness of interventions for severe mental illness. Public research resources in severe mental illness should be coordinated to prioritise these objectives.

Supplementary material

Supplementary material is available online at https://doi.org/10.1192/bjp.2021.121.

Data availability

Data availability is not applicable to this article as no new data were created or analysed in this study.

Acknowledgements

We thank our information specialists Eli Bastin and Nia Roberts, University of Oxford, for their help in developing the search strategy and for selecting databases.

Author contributions

J.A. and J.L. designed the study. J.A. and J.-S.L conducted the literature search. J.A., J.-S.L and J.L conducted data extraction. J.A. and J.L. performed the analysis and A.T., F.W. and D.F. critically commented on the analysis results. J.A., A.T., F.W. and J.L. wrote the manuscript.

Funding

D.F. is a National Institute for Health Research (NIHR) senior investigator. F.W. is funded by a Wellcome Trust Clinical Doctoral Fellowship (102176/B/13/Z). This research was funded by the National Health Service (NHS) National Institute for Health Research (NIHR) invention for innovation (i4i) programme (Project II-C7-0117-20001). This work was also supported by NIHR Oxford Health Biomedical Research Centre funding (BRC-1215-20005). The views expressed are those of the authors and not necessarily those of the NHS, NIHR or Department of Health.

Declaration of interest

None.

References

Public Health England. Severe Mental Illness (SMI) and Physical Health Inequalities: Briefing (PHE Publications Gateway Number: 2018395). PHE, 2018.Google Scholar
Reilly, S, Olier, I, Planner, C, Doran, T, Reeves, D, Ashcroft, DM, et al. Inequalities in physical comorbidity: a longitudinal comparative cohort study of people with severe mental illness in the UK. BMJ Open 2015; 5(12): e009010.CrossRefGoogle ScholarPubMed
Hert, MDE, Correll, CU, Bobes, J, Cetkovich-Bakmas, M, Cohen, D, Asai, I, et al. Physical illness in patients with severe mental disorders. I. Prevalence, impact of medications and disparities in health care. World Psychiatry 2011; 10: 5277.10.1002/j.2051-5545.2011.tb00014.xCrossRefGoogle Scholar
Robson, D, Gray, R. Serious mental illness and physical health problems: a discussion paper. Int J Nurs Stud 2007; 44: 457–66.CrossRefGoogle ScholarPubMed
Hjorthøj, C, Stürup, AE, McGrath, JJ, Nordentoft, M. Years of potential life lost and life expectancy in schizophrenia: a systematic review and meta-analysis. Lancet Psychiatry 2017; 4: 295301.10.1016/S2215-0366(17)30078-0CrossRefGoogle ScholarPubMed
Vigo, D, Thornicroft, G, Atun, R. Estimating the true global burden of mental illness. Lancet Psychiatry 2016; 3: 171–8.10.1016/S2215-0366(15)00505-2CrossRefGoogle ScholarPubMed
Walker, ER, McGee, RE, Druss, BG. Mortality in mental disorders and global disease burden implications a systematic review and meta-analysis. JAMA Psychiatry 2015; 72: 334–41.CrossRefGoogle ScholarPubMed
Lawrence, D, Hancock, KJ, Kisely, S. The gap in life expectancy from preventable physical illness in psychiatric patients in Western Australia: retrospective analysis of population based registers. BMJ 2013; 346: f2539.CrossRefGoogle ScholarPubMed
Fekadu, A, Medhin, G, Kebede, D, Alem, A, Cleare, AJ, Prince, M, et al. Excess mortality in severe mental illness: 10-year population-based cohort study rural Ethiopia. Br J Psychiatry 2015; 206: 289–96.10.1192/bjp.bp.114.149112CrossRefGoogle ScholarPubMed
Mazereel, V, Van Assche, K, Detraux, J, De Hert, M. COVID-19 vaccination for people with severe mental illness: why, what, and how? Lancet Psychiatry 2021; 8: 444–50.10.1016/S2215-0366(20)30564-2CrossRefGoogle ScholarPubMed
James, SL, Abate, D, Abate, KH, Abay, SM, Abbafati, C, Abbasi, N, et al. Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet 2018; 392: 1789–858.CrossRefGoogle Scholar
De Hert, M, Dekker, JM, Wood, D, Kahl, KG, Holt, RIG, Möller, HJ. Cardiovascular disease and diabetes in people with severe mental illness position statement from the European Psychiatric Association (EPA), supported by the European Association for the Study of Diabetes (EASD) and the European Society of Cardiology (ESC). Eur Psychiatry 2009; 24: 412–24.10.1016/j.eurpsy.2009.01.005CrossRefGoogle Scholar
Laursen, TM, Munk-Olsen, T, Nordentoft, M, Mortensen, PB. Increased mortality among patients admitted with major psychiatric disorders: a register-based study comparing mortality in unipolar depressive disorder, bipolar affective disorder, schizoaffective disorder, and schizophrenia. J Clin Psychiatry 2007; 68: 899907.CrossRefGoogle Scholar
World Health Organization. Information Sheet: Premature Death among People with Severe Mental Disorders. WHO, no date (https://www.who.int/mental_health/management/info_sheet.pdf).Google Scholar
Evans, S, Banerjee, S, Leese, M, Huxley, P. The impact of mental illness on quality of life: a comparison of severe mental illness, common mental disorder and healthy population samples. Qual Life Res 2007; 16: 1729.CrossRefGoogle ScholarPubMed
Harvey, PD, Heaton, RK, Carpenter, WT Jr, Green, MF, Gold, JM, Schoenbaum, M. Functional impairment in people with schizophrenia: focus on employability and eligibility for disability compensation. Schizophr Res 2012; 140: 18.10.1016/j.schres.2012.03.025CrossRefGoogle ScholarPubMed
Rapaport, MH, Clary, C, Fayyad, R, Endicott, J. Quality-of-life impairment in depressive and anxiety disorders. Am J Psychiatry 2005; 162: 1171–8.CrossRefGoogle ScholarPubMed
Hammer-Helmich, L, Haro, JM, Jönsson, B, Tanguy Melac, A, Di Nicola, S, Chollet, J, et al. Functional impairment in patients with major depressive disorder: the 2-year PERFORM study. Neuropsychiatr Dis Treat 2018; 14: 239–49.10.2147/NDT.S146098CrossRefGoogle ScholarPubMed
Rosa, AR, Reinares, M, Michalak, EE, Bonnin, CM, Sole, B, Franco, C, et al. Functional impairment and disability across mood states in bipolar disorder. Value Health 2010; 13: 984–8.10.1111/j.1524-4733.2010.00768.xCrossRefGoogle ScholarPubMed
Seabury, SA, Axeen, S, Pauley, G, Tysinger, B, Schlosser, D, Hernandez, JB, et al. Measuring the lifetime costs of serious mental illness and the mitigating effects of educational attainment. Health Aff (Millwood) 2019; 38: 652–9.CrossRefGoogle ScholarPubMed
National Institute for Health and Care Excellence. Guide to the Methods of Technology Appraisal 2013 (Process and Methods Guides PMG9). NICE, 2013.Google Scholar
Knapp, M, Wong, G. Economics and mental health: the current scenario. World Psychiatry 2020; 19: 314.CrossRefGoogle ScholarPubMed
Shearer, J, Byford, S. The basics of economic evaluation in mental healthcare. BJPsych Advances 2015; 21: 345–53.CrossRefGoogle Scholar
Ginnelly, L, Manca, A. The use of decision models in mental health economic evaluation: challenges and opportunities. Appl Health Econ Health Policy 2003; 2: 157–64.Google ScholarPubMed
Jin, H, Tappenden, P, Robinson, S, Achilla, E, Aceituno, D, Byford, S. Systematic review of the methods of health economic models assessing antipsychotic medication for schizophrenia. PLoS One 2020; 15(7): e0234996.CrossRefGoogle Scholar
National Institute for Health and Care Excellence. Developing NICE Guidelines: The Manual. NICE, 2014.Google Scholar
Abdul Pari, AA, Simon, J, Wolstenholme, J, Geddes, JR, Goodwin, GM. Economic evaluations in bipolar disorder: a systematic review and critical appraisal. Bipolar Disord 2014; 16: 557–82.CrossRefGoogle ScholarPubMed
Kolovos, S, Bosmans, JE, Riper, H, Chevreul, K, Coupé, VMH, van Tulder, MW. Model-based economic evaluation of treatments for depression: a systematic literature review. Pharmacoecon Open 2017; 1: 149–65.CrossRefGoogle ScholarPubMed
Tappenden, P, Chilcott, JB. Avoiding and identifying errors and other threats to the credibility of health economic models. Pharmacoeconomics 2014; 32: 967–79.CrossRefGoogle Scholar
NHS England. The NHS Long Term Plan. NHS, 2019.Google Scholar
National Institute for Health and Care Excellence. Psychosis and Schizophrenia in Adults: Prevention and Management (Clinical Guideline CG178). NICE, 2014.Google Scholar
National Institute for Health and Care Excellence. Depression in Adults: Recognition and Management (Clinical Guideline CG90). NICE, 2009.Google Scholar
Freeman, D, Yu, L-M, Kabir, T, Martin, J, Craven, M, Leal, J, et al. Automated virtual reality (VR) cognitive therapy for patients with psychosis: study protocol for a single-blind parallel group randomised controlled trial (gameChange). BMJ Open 2019; 9(8): e031606.CrossRefGoogle Scholar
Nosé, M, Barbui, C, Tansella, M. How often do patients with psychosis fail to adhere to treatment programmes? A systematic review. Psychol Med 2003; 33: 1149–60.CrossRefGoogle ScholarPubMed
David, AS. Treatment adherence in psychoses. Br J Psychiatry 2010; 197: 431–2.10.1192/bjp.bp.110.083022CrossRefGoogle ScholarPubMed
Gastaldon, C, Mosler, F, Toner, S, Tedeschi, F, Bird, VJ, Barbui, C, et al. Are trials of psychological and psychosocial interventions for schizophrenia and psychosis included in the NICE guidelines pragmatic? A systematic review. PLoS One 2019; 14(9): e0222891.CrossRefGoogle ScholarPubMed
Knapp, M, Chisholm, D, Leese, M, Amaddeo, F, Tansella, M, Schene, A, et al. Comparing patterns and costs of schizophrenia care in five European countries: the EPSILON study. European Psychiatric Services: inputs linked to outcome domains and needs. Acta Psychiatr Scand 2002; 105: 4254.10.1034/j.1600-0447.2002._10310.xCrossRefGoogle ScholarPubMed
Lehman, AF. Quality of care in mental health: the case of schizophrenia. Health Affairs 1999; 18: 5265.CrossRefGoogle ScholarPubMed
Melartin, TK, Rytsala, HJ, Leskela, US, Lestela-Mielonen, PS, Sokero, TP, Isometsa, ET. Severity and comorbidity predict episode duration and recurrence of DSM-IV major depressive disorder. J Clin Psychiatry 2004; 65: 810–9.CrossRefGoogle ScholarPubMed
Spijker, J, De Graaf, R, Bijl, RV, Beekman, ATF, Ormel, J, Nolen, WA. Duration of major depressive episodes in the general population: results from the Netherlands Mental Health Survey and Incidence Study (NEMESIS). Br J Psychiatry 2002; 181: 208–13.CrossRefGoogle Scholar
Russo, M, Levine, SZ, Demjaha, A, Di Forti, M, Bonaccorso, S, Fearon, P, et al. Association between symptom dimensions and categorical diagnoses of psychosis: a cross-sectional and longitudinal investigation. Schizophr Bull 2014; 40: 111–9.CrossRefGoogle ScholarPubMed
Harrow, M, Grossman, LS, Jobe, TH, Herbener, ES. Do patients with schizophrenia ever show periods of recovery? A 15-year multi-follow-up study. Schizophr Bull 2005; 31: 723–34.CrossRefGoogle ScholarPubMed
Davis, S, Stevenson, M, Tappenden, P, Wailoo, A. NICE DSU Technical Support Document 15: Cost-Effectiveness Modelling Using Patient-Level Simulation. National Institute for Health and Care Excellence, 2014.Google ScholarPubMed
Karnon, J, Stahl, J, Brennan, A, Caro, JJ, Mar, J, Möller, J. Modeling using discrete event simulation: a report of the ISPOR-SMDM modeling good research practices task force-4. Value Health 2012; 15: 821–7.CrossRefGoogle ScholarPubMed
Brennan, A, Chick, SE, Davies, R. A taxonomy of model structures for economic evaluation of health technologies. Health Econ 2006; 15: 1295–310.CrossRefGoogle ScholarPubMed
Moher, D, Liberati, A, Tetzlaff, J, Altman, DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009; 6(7): e1000097.CrossRefGoogle ScholarPubMed
Cooper, N, Coyle, D, Abrams, K, Mugford, M, Sutton, A. Use of evidence in decision models: an appraisal of health technology assessments in the UK since 1997. J Health Serv Res Policy 2005; 10: 245–50.CrossRefGoogle ScholarPubMed
Caro, JJ, Eddy, DM, Kan, H, Kaltz, C, Patel, B, Eldessouki, R, et al. Questionnaire to assess relevance and credibility of modeling studies for informing health care decision making: an ISPOR-AMCP-NPC good practice task force report. Value Health 2014; 17: 174–82.CrossRefGoogle Scholar
Vemer, P, Corro Ramos, I, van Voorn, GA, Al, MJ, Feenstra, TL. AdViSHE: a validation-assessment tool of health-economic models for decision makers and model users. Pharmacoeconomics 2016; 34: 349–61.CrossRefGoogle ScholarPubMed
Mavranezouli, I, Lokkerbol, J. A systematic review and critical appraisal of economic evaluations of pharmacological interventions for people with bipolar disorder. PharmacoEconomics 2017; 35: 271–96.10.1007/s40273-016-0473-1CrossRefGoogle ScholarPubMed
Vera-Llonch, M, Delea, TE, Richardson, E, Rupnow, M, Grogg, A, Oster, G. Outcomes and costs of risperidone versus olanzapine in patients with chronic schizophrenia or schizoaffective disorders: a Markov model. Value Health 2004; 7: 569–84.CrossRefGoogle ScholarPubMed
Heeg, BMS, Buskens, E, Knapp, M, van Aalst, G, Dries, PJT, de Haan, L, et al. Modelling the treated course of schizophrenia: development of a discrete event simulation model. PharmacoEconomics 2005; 23: 1733.CrossRefGoogle ScholarPubMed
Furiak, NM, Ascher-Svanum, H, Klein, RW, Smolen, LJ, Lawson, AH, Conley, RR, et al. Cost-effectiveness model comparing olanzapine and other oral atypical antipsychotics in the treatment of schizophrenia in the United States. Cost Eff Resour Alloc 2009; 7: 4.CrossRefGoogle ScholarPubMed
Dilla, T, Möller, J, O'Donohoe, P, Álvarez, M, Sacristán, JA, Happich, M, et al. Long-acting olanzapine versus long-acting risperidone for schizophrenia in Spain – a cost-effectiveness comparison. BMC Psychiatry 2014; 14: 298.CrossRefGoogle Scholar
Health Quality Ontario. Cognitive behavioural therapy for psychosis: a health technology assessment. Ont Health Technol Assess Ser 2018; 18(5): 1141.Google Scholar
Jin, H, Tappenden, P, MacCabe, JH, Robinson, S, Byford, S. Evaluation of the cost-effectiveness of services for schizophrenia in the UK across the entire care pathway in a single whole-disease model. JAMA Netw Open 2020; 3(5): e205888.CrossRefGoogle Scholar
Klok, RM, Al Hadithy, AFY, van Schayk, NPJT, Antonisse, AJJ, Caro, JJ, Brouwers, JRBJ, et al. Pharmacoeconomics of quetiapine for the management of acute mania in bipolar I disorder. Expert Rev Pharmacoecon Outcomes Res 2007; 7: 459–67.CrossRefGoogle ScholarPubMed
Ekman, M, Lindgren, P, Miltenburger, C, Meier, G, Locklear, JC, Chatterton, ML. Cost effectiveness of quetiapine in patients with acute bipolar depression and in maintenance treatment after an acute depressive episode. PharmacoEconomics 2012; 30: 513–30.CrossRefGoogle ScholarPubMed
Sobocki, P, Ekman, M, Ågren, H, Jönsson, B, Rehnberg, C. Model to assess the cost-effectiveness of new treatments for depression. Int J Technol Assess Health Care 2006; 22: 469–77.CrossRefGoogle ScholarPubMed
Prukkanone, B, Vos, T, Bertram, M, Lim, S. Cost-effectiveness analysis for antidepressants and cognitive behavioral therapy for major depression in Thailand. Value Health 2012; 15(suppl 1): S38.CrossRefGoogle ScholarPubMed
Saylan, M, Treur, MJ, Postema, R, Dilbaz, N, Savas, H, Heeg, BM, et al. Cost-effectiveness analysis of aripiprazole augmentation treatment of patients with major depressive disorder compared to olanzapine and quetiapine augmentation in Turkey: a microsimulation approach. Value Health Reg Issues 2013; 2: 171–80.CrossRefGoogle ScholarPubMed
Tosh, J, Kearns, B, Brennan, A, Parry, G, Ricketts, T, Saxon, D, et al. Innovation in health economic modelling of service improvements for longer-term depression: demonstration in a local health community. BMC Health Serv Res 2013; 13(1): 150.CrossRefGoogle Scholar
Vataire, A-L, Aballéa, S, Antonanzas, F, Roijen, L-v, Lam, RW, McCrone, P, et al. Core discrete event simulation model for the evaluation of health care technologies in major depressive disorder. Value Health 2014; 17: 183–95.CrossRefGoogle ScholarPubMed
Nguyen, K-H, Gordon, LG. Cost-effectiveness of repetitive transcranial magnetic stimulation versus antidepressant therapy for treatment-resistant depression. Value Health 2015; 18: 597604.CrossRefGoogle ScholarPubMed
Health Quality Ontario. Psychotherapy for major depressive disorder and generalized anxiety disorder: a health technology assessment. Ont Health Technol Assess Ser 2017; 17(15): 1167.Google Scholar
Wang, P, Si, T. Use of antipsychotics in the treatment of depressive disorders. Shanghai Arch Psychiatry 2013; 25: 134–40.Google ScholarPubMed
Royal College of Psychiatrists. Antipsychotics. RCPsych, 2021 (https://www.rcpsych.ac.uk/mental-health/treatments-and-wellbeing/antipsychotics (refd 15 Jan 2021)).Google Scholar
NHS England, National Collaborating Centre for Mental Health, National Institute for Health and Care Excellence. Implementing the Early Intervention in Psychosis Access and Waiting Time Standard: Guidance. National Collaborating Centre for Mental Health, 2016.Google Scholar
National Institute for Health and Care Excellence. Psychosis and Schizophrenia in Adults (Quality Standard QS80). NICE, 2015.Google Scholar
Reichert, A, Jacobs, R. The impact of waiting time on patient outcomes: evidence from early intervention in psychosis services in England. Health Econ 2018; 27: 1772–87.CrossRefGoogle ScholarPubMed
Roussidis, A, Kalkavoura, C, Dimelis, D, Theodorou, A, Ioannidou, I, Mellos, E, et al. Reasons and clinical outcomes of antipsychotic treatment switch in outpatients with schizophrenia in real-life clinical settings: the ETOS observational study. Ann Gen Psychiatry 2013; 12: 42.CrossRefGoogle ScholarPubMed
Arango, C, Kapur, S, Kahn, RS. Going beyond “trial-and-error” in psychiatric treatments: OPTiMiSE-ing the treatment of first episode of schizophrenia. Schizophr Bull 2015; 41: 546–8.CrossRefGoogle ScholarPubMed
Buckley, PF, Miller, BJ. Personalized medicine for schizophrenia. NPJ Schizophrenia 2017; 3: 2.CrossRefGoogle Scholar
Derks, EM, Fleischhacker, WW, Boter, H, Peuskens, J, Kahn, RS. Antipsychotic drug treatment in first-episode psychosis: should patients be switched to a different antipsychotic drug after 2, 4, or 6 weeks of nonresponse? J Clin Psychopharmacol 2010; 30: 176–80.CrossRefGoogle ScholarPubMed
Leucht, S, Corves, C, Arbter, D, Engel, RR, Li, C, Davis, JM. Second-generation versus first-generation antipsychotic drugs for schizophrenia: a meta-analysis. Lancet 2009; 373: 3141.CrossRefGoogle ScholarPubMed
Ward, M, Druss, B. The epidemiology of diabetes in psychotic disorders. Lancet Psychiatry 2015; 2: 431–51.CrossRefGoogle ScholarPubMed
Correll, CU, Solmi, M, Veronese, N, Bortolato, B, Rosson, S, Santonastaso, P, et al. Prevalence, incidence and mortality from cardiovascular disease in patients with pooled and specific severe mental illness: a large-scale meta-analysis of 3,211,768 patients and 113,383,368 controls. World Psychiatry 2017; 16: 163–80.CrossRefGoogle ScholarPubMed
Byford, S, Barber, JA, Fiander, M, Marshall, S, Green, J. Factors that influence the cost of caring for patients with severe psychotic illness: report from the UK700 trial. Br J Psychiatry 2001; 178: 441–7.CrossRefGoogle Scholar
Kent, S, Becker, F, Feenstra, T, Tran-Duy, A, Schlackow, I, Tew, M, et al. The challenge of transparency and validation in health economic decision modelling: a view from mount hood. PharmacoEconomics 2019; 37: 1305–12.CrossRefGoogle ScholarPubMed
Kim, LG, Thompson, SG. Uncertainty and validation of health economic decision models. Health Econ 2010; 19: 4355.Google ScholarPubMed
Figure 0

Fig. 1 Literature search – PRISMA diagram.

Figure 1

Table 1 Description of the 15 included studies

Figure 2

Table 2 Complexity of the 15 models

Figure 3

Fig. 2 Hierarchy of evidence quality. NA, not applicable.

Figure 4

Fig. 3 Validation and quality checklists. ISPOR, International Society for Pharmacoeconomics and Outcomes; AdViSHE, Assessment of the Validation Status of Health-Economic Decision Models; NA, not applicable.

Supplementary material: File

Altunkaya et al. supplementary material

Altunkaya et al. supplementary material

Download Altunkaya et al. supplementary material(File)
File 332 KB
Submit a response

eLetters

No eLetters have been published for this article.
You have Access Open access

Save article to Kindle

To save this article to your Kindle, first ensure coreplatform@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Appraisal of patient-level health economic models of severe mental illness: systematic review
Available formats
×

Save article to Dropbox

To save this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Dropbox account. Find out more about saving content to Dropbox.

Appraisal of patient-level health economic models of severe mental illness: systematic review
Available formats
×

Save article to Google Drive

To save this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Google Drive account. Find out more about saving content to Google Drive.

Appraisal of patient-level health economic models of severe mental illness: systematic review
Available formats
×
×

Reply to: Submit a response

Please enter your response.

Your details

Please enter a valid email address.

Conflicting interests

Do you have any conflicting interests? *