Background
The past decade has witnessed an unprecedented transformation in public perception of mental health,Reference Henderson, Robinson, Evans-Lacko, Corker, Rebollo-Mesa and Rose1 with political commitment2,3 to address substantial disparities between physical and mental healthcare.4 This has begun to have an impact on mental healthcare provision, which is undergoing substantial reconfiguration in many regions, including the USA,Reference Dixon, Goldman, Srihari and Kane5 Canada,Reference Malla, Shah, Iyer, Boksa, Joober and Andersson6 AustraliaReference McGorry, Trethowan and Rickwood7 and northern Europe.2,Reference Ronis, Slaunwhite and Malcom8,Reference Csillag, Nordentoft, Mizuno, McDaid, Arango and Smith9 One such example is early intervention in psychosis (EIP) services, which seek to provide multidisciplinary care for people with an emerging psychosis, informed by an evidence base that suggests that a longer duration of untreated psychosis is associated with less favourable health and social outcomes.Reference Marshall, Lewis, Lockwood, Drake, Jones and Croudace10 These services have rapidly gained traction in several countries, including England, based on clinical evidence for their efficacy,Reference Correll, Galling, Pawar, Krivko, Bonetto and Ruggeri11 effectivenessReference Correll, Galling, Pawar, Krivko, Bonetto and Ruggeri11,Reference Randall, Vokey, Loewen, Martens, Brownell and Katz12 and potential cost-effectivenessReference Aceituno, Vera, Prina and McCrone13 over the short to medium term. Effects on longer-term outcomes, particularly in the absence of sustained intervention, are less apparent,Reference Chan, Chan, Devlin, Bastiampillai, Mohan and Hui14 and are a challenge to providing sustainable care models for people experiencing psychosis.
Evidence-based public mental health
Optimising this care requires a broader evidence-based, pragmatic approach,Reference McGrath, Mortensen and Whiteford15 incorporating epidemiological knowledge to aid decision-making, to effectively allocate resources to EIP services based on anticipated local need in different populations. For example, age, gender and urbanicity have been consistently associated with the incidence of psychotic disorders in the UK, Europe and potentially beyond,Reference Oliver, Reilly, Baccaredda Boy, Petros, Davies and Borgwardt16 and their local distributions could therefore inform population-level need for treatment each year. Such population-based approaches are increasingly recognised as vital to effective public mental health, but few examples are available to inform local and national mental health policy.Reference McGrath, Mortensen and Whiteford15,Reference Malla and McGorry17
One example previously applied to population-level need for EIP care in England is known as the PsyMaptic model (Psychiatric Mapping Translated into Innovations for Care).Reference Kirkbride, Jackson, Perez, Fowler, Winton and Coid18,Reference Kirkbride19 Based on empirical epidemiological estimates of the incidence of first-episode psychosis (FEP) by age, gender, ethnicity and selected area-level characteristics such as deprivation, this model predicted the incidence (number and rate of new cases) in different population groups and regions based on the population structure of the 2011 Census of Great Britain. This model has subsequently informed national policy-making in the UK,20,21 including implementation guidelines for the landmark ‘Access and Waiting Time’ standard for EIP care in 2016,20 as part of the National Health Service and Department of Health's ‘Five-Year Forward View for Mental Health’,2 which legislated commitment to parity of esteem between physical and mental health.22 This standard mandated that at least 60% of all people aged 14–65 years who are referred to EIP services for ‘suspected psychosis’ must have been assessed, and where suitable, commenced a National Institute of Health and Care Excellence (NICE)-concordant,22,23 evidence-basedReference Bird, Premkumar, Kendall, Whittington, Mitchell and Kuipers24,Reference Stafford, Jackson, Mayo-Wilson, Morrison and Kendall25 package of EIP care within 14 days. However, although there is emerging evidence that most EIP services in England are already exceeding the waiting time target,26 achieving fidelity to other targets in the standard, including offering cognitive–behavioural therapy for psychosis, family interventions and supported employment and education programmes, has proved challenging,26 and may erode potential beneficial outcomes expected for patients available when full-fidelity EIP care is offered.Reference Malla and McGorry17 A prerequisite to achieving such fidelity is that mental health service policy makers, commissioners and providers have access to timely and accurate information about future local need for EIP care, so that services can be appropriately resourced to deliver effective, optimised evidence-based care.
Nonetheless, the previously published PsyMaptic toolReference Kirkbride, Jackson, Perez, Fowler, Winton and Coid18 had limited generalisability to forecast future need for EIP care. For example, it was restricted to (a) a static denominator population, using the 2011 Census; (b) incidence data ascertained before the national introduction of EIP services in 2002; (c) social determinants of health data at a local authority level (serving large, heterogenous areas; median population in 2018: 126 678 people27) and (d) ICD-10 diagnostic criteria for psychotic disorders, and not the broader need for assessment and treatment generated by referrals to EIP services for suspected psychosis.20
Aim
To overcome these limitations and facilitate generalisability of our methodology to other settings, we developed a new, population-based prediction model applied to psychosis care in England, validated against observed national routine data. Our methodology forecasted small-area need for EIP provision up to 2025 to inform the ‘NHS Long-Term Plan’28 and ‘Mental Health Implementation Plan in England’,29 which have recently reaffirmed and strengthened their commitment to EIP care, taking into account projected changes in the population at risk in England over the next 5 years.
Method
We followed the transparent reporting of multivariable prediction modell for individual prognosis or diagnosis reporting guidelines for prediction modelling (Supplementary Box 1 available at https://doi.org/10.1192/bjp.2021.18).
Model development
Seed data
Empirical incidence data were pooled from the three largest epidemiological catchment area studies of psychotic disorders in England in the past 25 years, which used similar methodologies, as described in detail elsewhereReference Kirkbride, Fearon, Morgan, Dazzan, Morgan and Tarrant30–Reference Coid, Kirkbride, Barker, Cowden, Stamps and Yang33 (Supplementary Section 1): the Aetiology and Ethnicity in Schizophrenia and Other Psychoses (ÆSOP),Reference Kirkbride, Fearon, Morgan, Dazzan, Morgan and Tarrant30 the East London First Episode Psychoses (ELFEP)Reference Kirkbride, Barker, Cowden, Stamps, Yang and Jones32,Reference Coid, Kirkbride, Barker, Cowden, Stamps and Yang33 and Social Epidemiology of Psychoses in East Anglia (SEPEA)Reference Kirkbride, Hameed, Ankireddypalli, Ioannidis, Crane and Nasir31 studies.
Outcome
These studies included incidence data on all people aged between 16–64 years who presented to services (including EIPReference Kirkbride, Hameed, Ankireddypalli, Ioannidis, Crane and Nasir31) in each catchment area for suspected psychosis, and who later met research-based criteria for a diagnosis of for non-affective psychotic disorder (ICD-10 codes F20–29), affective psychotic disorder (ICD-10 codes F30–33) or substance-induced psychotic disorder (ICD-10 code F1X.5). Individuals with an organic basis to their disorder or profound intellectual disability were excluded.
Predictors
Participants were linked to their small-area neighbourhood of residence at first presentation (ÆSOP/ELFEP: 2001 Census Area Statistics wards; SEPEA: 2011 Census merged wards). For each ward, we obtained measures of relevant socioenvironmental data, guided by previous models and the wider literature,Reference Kirkbride, Jackson, Perez, Fowler, Winton and Coid18,Reference Kirkbride, Errazuriz, Croudace, Morgan, Jackson and Boydell34,Reference Moore, Zammit, Lingford-Hughes, Barnes, Jones and Burke35 including population density; social fragmentation; own-group ethnic density; multiple deprivation; inequality and the gender-specific, age-standardised prevalence of self-reported past-year cannabis use in the population (Supplementary Section 2). We stratified our seed data-set by age group (16–17, 18–19, 20–24, 25–29, 30–34, 35–39, 40–44, 45–49, 50–54, 55–59 and 60–64 years), gender and self-assigned ethnicity (based on a priori evidence about risk:Reference Kirkbride, Errazuriz, Croudace, Morgan, Jackson and Boydell34,Reference Bourque, Van Der Ven and Malla36 White British, Irish, Gypsy and Traveller; White other; Black Caribbean and Black other; Black African; Indian; Pakistani; Bangladeshi; mixed; and all other ethnicities) at ward level. We recorded the count of FEP cases and person-years at risk in each strata, estimated from the nearest decennial census to the case ascertainment period (Supplementary Section 3). Ward- and regional-level covariates were appended to this data-set.
Model building
We modelled the count of new (incidence) cases of psychotic disorders by using Poisson regression in a Bayesian framework. A Poisson distribution was preferred to a negative binomial distribution as there was no evidence of overdispersion within the seed data-set (Supplementary Table 1). The Bayesian approach allowed us to combine this data-set with a priori knowledge about the epidemiology of psychotic disorders, specified as prior probability distributions on the covariates to be estimated, informed by published evidenceReference Kirkbride, Errazuriz, Croudace, Morgan, Jackson and Boydell34–Reference Hardoon, Hayes, Blackburn, Petersen, Walters and Nazareth37 (Supplementary Section 4, Figs 1–3 and Tables 2–5). We fitted our models by using integrated nested Laplace approximationReference Rue, Martino and Chopin38 to estimate the posterior distribution for the relevant parameters. Then, to fully characterise the uncertainty in the estimates and the predictions, we used Monte Carlo simulation to randomly sample 1000 values from the joint posterior distributions of each model parameter, simultaneously accounting for any correlation present between parameters in the model. This allowed us to capture the full uncertainty around the effect of each potential covariate on FEP incidence rates, which we expressed as posterior relative risks with their 95% credible interval estimates.
Initial modelling suggested that ethnic density (relative risk 1.001, 95% CI 0.998–1.005) and inequality (relative risk 0.90, 95% CI 0.50–1.62) were not associated with FEP incidence in our seed data-set, and were removed from further consideration. From the remaining covariates, we considered six a priori candidate models. We included age group, gender, their interaction and ethnic group in all models, with different permutations of deprivation, population density, social fragmentation and cannabis use fitted in each model. Person-years at risk were treated as an offset in the models.
Out-of-sample prediction
Denominator and socioenvironmental data
We estimated the projected population at risk, aged 16–64 years, for all Census merged wards in England (N = 7678) between 2017–2025, stratified by age group, gender and ethnicity (as before). Projected population estimates up to 2025 were obtained from demographic forecasts with an established methodology,Reference Rees, Wohland, Norman, Lomax, Clark and Swanson39,Reference Lomax, Wohland, Rees and Norman40 using 2011 Census data as a baseline, accounting for year-on-year changes in birth and death rates, immigration and emigration flows, and including assumptions about the impact of Brexit beyond 2020Reference Lomax, Wohland, Rees and Norman40 (Supplementary Section 5). Population projections for the City of London (four wards) and Isles of Scilly (one ward) were excluded because of substantial overestimation of the ward population, when compared with Office for National Statistics 2017 mid-year population estimates.41 For socioenvironmental covariates, we used the latest available routine data at the time of model development for deprivation (2015),Reference Smith, Noble, Noble, Wright, McLennan and Plunkett42 social fragmentation (2011), population density (re-estimated for each year between 2017–2025, based on ward-level projected populations) and regional cannabis prevalence (2014)Reference Mcmanus, Bebbington, Jenkins and Brugha43 (Supplementary Section 2).
FEP prediction
Posterior relative risk coefficients from each model were applied to the projected denominator data in a given year (2017–2025) at ward level, to estimate the predicted number of incident FEP cases in each age-gender-ethnic stratum in each ward. Within our Bayesian framework, we performed 1000 model simulations, sampling from the posterior distribution to yield the mean predicted FEP cases per stratum, and corresponding 95% confidence interval, derived from the 2.5% and 97.5% quantiles. This allowed us to aggregate prediction data to Clinical Commissioning Group (CCG) level and the national level by gender, broad age group (16–64, 16–35 and 36–64 years; consistent with those most relevant to EIP care in England) and ethnicity for any given year (2017–2025) (Supplementary Section 6). CCGs are autonomous, clinically led, statutory National Health Services (NHS) bodies responsible for the planning and commissioning of health services in local areas.Reference NHS Clinical Commissioners.44
Model validation
Apparent validation
Apparent validity of the six candidate models was evaluated with the deviance information criterion (DIC), with a smaller value indicating better fit.Reference Kirkbride, Errazuriz, Croudace, Morgan, Jackson and Boydell34,Reference Jongsma, Turner, Kirkbride and Jones45
External validation
To externally validate our models, we compared predicted FEP data based on the 2017 denominator population, with observed national data available from NHS Digital's Mental Health Services Dataset (MHSDS) in the closest (financial) year available (April 2017 to March 2018; henceforth ‘2017’). Comparisons were matched for the EIP age range served by each CCG in 2017 (Supplementary Section 7).46 The MHSDS records data from all health records for people in contact with mental health services in England. From this data-set, we extracted data for each CCG in England (N = 207) on the total number of new people (a) referred to mental health services for suspected psychosis; (b) accepted for assessment by EIP services; (c) who commenced EIP treatment for FEP and (d) who were probable FEP cases, based on empirical evidence that approximately 14% of those who commenced treatment for FEP in EIP services do not go onto to fulfil operationalised criteria for ICD-10 psychotic disorder.Reference Kirkbride, Hameed, Ankireddypalli, Ioannidis, Crane and Nasir31 Those not meeting criteria for probable FEP in categories a–c implicitly included people meeting the clinical high risk for psychosis (CHR-P) criteria, who are not currently separately disaggregated within the MHSDS data-set. Formal definitions of each of these nested levels, henceforth referred to as referred/assessed/treated/probable FEP case-loads, are given in Supplementary Section 7.
We inspected external validity at the CCG level by estimating correlation coefficients (r) between observed and predicted probable FEP counts, the r 2 statistic and root-mean-square error (RMSE), with higher r 2 and lower RMSE indicating better model fit. We inspected calibration plots of predicted versus observed FEP counts and compared the difference in predicted and observed FEP counts, expressed as a rate per 100 000 person-years at the CCG level. At the national level, we compared the difference between predicted and observed case-loads, expressed as a percentage, for the total count and by broad age group, gender and ethnic group. Choice of our best-fitting model was determined by consensus agreement between our authorship group, after evaluation of all apparent and external validation metrics.
Missing data
We excluded two CCGs from external validation (NHS Birmingham & CrossCity CCG and NHS Sandwell and West Birmingham CCG) because of problems with data validity in the MHSDS during this reporting period (C. Money, personal communication, 2019). These CCGs were reinstated for prediction forecasting (see below) between 2019 and 2025. MHSDS data were complete with respect to national- and CCG-level totals and by broad age group (16–35 and 36–64 years). Ten (0.1%) observed probable FEP cases were missing information on gender, and were excluded from gender-specific validation. Ethnicity data were missing for 820 (10.2%) observed probable FEP cases, which would have affected model validity by ethnicity without imputation, which we handled via imputation in sensitivity analyses (Supplementary Section 8).
Forecasting
Our best-fitting model was used to forecast predicted probable FEP case-loads in England between 2019–2025, using the relevant denominator projections for each year. Predictions and their 95% confidence intervals, stratified by broad age group, gender and ethnicity, were aggregated to 2019 CCG and national levels for dissemination. We also forecasted new referred, assessed and treated case-loads (implicitly including CHR-P individuals) per annum by multiplying predicted probable FEP case-loads by the ratios derived from the observed data in the 2017 MHSDS (Supplementary Section 9). All prediction forecasts were expressed as incidence counts, and rates per 100 000 persons-years.
Software
Data-set generation and validation were performed in Stata version 15 for Windows,47 prediction modelling was conducted with the integrated nested Laplace approximation package in R version 3.5 for Windows (R Foundation, Austria; see https://www.r-project.org/),Reference Rue, Martino and Chopin38 and out-of-sample prediction using matrix multiplication was performed in R.48
Ethics
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation, and with the Helsinki Declaration of 1975, as revised in 2008. All procedures involving human patients were approved by NHS Health Research Authority (approval reference 19/HRA/0145). Consent was not required for this study, which was based on secondary use of aggregated, anonymised data from the ÆSOP, ELFEP and SEPEA studies and similarly aggregated NHS MHSDS data.
Results
Seed data sample characteristics
From our seed data-set, we included 1638 cases of FEP from 4 515 379 person-years at risk (Table 1). As typical of FEP incidence samples,Reference Kirkbride, Errazuriz, Croudace, Morgan, Jackson and Boydell34 most cases were male (62%), younger than 30 years old at first contact (ÆSOP 54%, ELFEP 57%) and with an overrepresentation of participants from Black and minority ethnic backgrounds relative to the local at-risk population (varying from 24% in SEPEAReference Kirkbride, Hameed, Ioannidis, Ankireddypalli, Crane and Nasir49 to 77% in ELFEPReference Coid, Kirkbride, Barker, Cowden, Stamps and Yang33).
Summary of FEP data from the ÆSOP, ELFEP and SEPEA studies
FEP, first-episode psychosis; ÆSOP, Aetiology and Ethnicity in Schizophrenia and Other Psychoses study; ELFEP, East London First Episode Psychoses study; SEPEA, Social Epidemiology of Psychoses in East Anglia study.
a. Study age ranges: ÆSOP, 16–64 years; ELFEP, 18–64 years; SEPEA, 16–35 years.
Apparent validation
Apparent validation of our models suggested that inclusion of area-level predictors (Table 2, models 2–6, DIC 11 531.88–11 539.75) improved model fit over a model fitted solely with individual-level covariates (model 1: DIC 11 614.04); DIC values were equivocal for models 2 and 4–6 (Table 2). Full model parameter specifications (Supplementary Table 6) showed elevated posterior relative risk for young men and several ethnic minority groups, particularly people of Black Caribbean (relative riskmodel 4 4.80, 95% CI 4.08–5.63) and Black African (relative riskmodel 4 3.33, 95% CI 2.78–3.97) ethnicity, consistent with the available literature.Reference Kirkbride, Errazuriz, Croudace, Morgan, Jackson and Boydell34 Population density was weakly associated with FEP incidence (relative riskmodel 5 1.06, 95% CI 0.99–1.12). Deprivation showed a non-linear relationship with FEP incidence, with exponentially higher rates at increasingly levels of deprivation (Supplementary Fig. 3). In contrast to expectations, a 1-s.d. increase in social fragmentation was associated with lower incidence (relative riskmodel 4 0.92, 95% CI 0.87–0.97). As expected, and driven by prior information,Reference Moore, Zammit, Lingford-Hughes, Barnes, Jones and Burke35 lifetime self-reported cannabis use was strongly associated with psychosis incidence (relative riskmodel 4 1.41, 95% CI 1.16–1.72).
Apparent and external validity of six candidate Bayesian Poisson regression models
DIC, deviance information criterion; RMSE, root-mean-square error; MHSDS, Mental Health Services Dataset; CCG, Clinical Commissioning Group.
a. Observed data from NHS Digital MHSDS for the financial year April 2016 – March 2017.
b. Predicted data were age-matched to observed MHSDS data, according to the corresponding age range served by each CCG as reported in the Early Intervention in Psychosis Network national audit for the same period (see Supplementary Section 7 for further details). Not all CCGs served the entire population aged 16–64 years, so the figures in this table should not be interpreted as total predicted cases in England for a given stratum. For total predicted case-load sizes in England, for 2019–2025, see Supplementary Table 10.
c. Out of 207 CCGs, 205 were included in validation statistics; two CCGs were excluded because of reliability concerns over the observed MHSDS data.
d. Number and percentage of CCGs where difference between the observed and predicted incidence rate (per 100 000 person-years) fell within the category range shown.
External validation
National level
In 2017, 22 803 new referrals for suspected psychosis were recorded in the MHSDS across the 205 CCG included for validation; 89.9% (n = 20 492) were subsequently accepted for EIP assessment, 35.2% (n = 9346) commenced EIP treatment and 35.2% (n = 8038; 86% of the treated case-load) met our definition of probable FEP. For the same period, model 4 (individual covariates, deprivation, social fragmentation and cannabis prevalence) predicted 8112 FEP cases per annum (95% CI 7623–8597), giving the lowest error of any model at the national level (n = +74, +0.92%), compared with the observed probable FEP case-load (Table 2).
All models performed similarly against national observed probable FEP case-loads stratified by major age group, gender and ethnic group (Supplementary Tables 7–9). Model 4 (Table 3), for example, showed good accuracy by gender (Table 3) and ethnicity after imputation for missing observed data (Supplementary Section 8 and Table 9). Nonetheless, all models tended to underpredict FEP case-loads aged 16–35 years (e.g. model 4: n = −541, −9.2%), and, correspondingly, overpredicted case-loads aged 36–64 years (e.g. model 4: n = +615, +29.0%) compared with observed probable FEP case-loads in the MHSDS data-set.
Predicted counts and incidence rates of new FEP (model 4) and corresponding observed data in the MHSDS in 2017a
Rate shows the incidence rate per 100 000 person-years. FEP, first-episode psychosis; MHSD, Mental Health Services Dataset; CCG, Clinical Commissioning Group.
a. Predicted data were age-matched to observed MHSDS data, according to the corresponding age range served by each CCG as reported in the Early Intervention in Psychosis Network national audit for the same period (see Supplementary Section 7 for further details). Not all CCGs served the entire population aged 16–64 years, so the figures in this table should not be interpreted as total predicted cases in England for a given stratum. For total predicted case-load sizes in England, for 2019–2025, see Supplementary Table 10.
b. Reflects the total population according to age ranges accepted by individual CCGs.
c. Includes unknown, indeterminate and invalid values.
CCG level
At the CCG level, fit indices (Table 2) suggested that models 2–6 (with area-level covariates) provided a better fit to the data than model 1 (individual-level covariates only). Differences in fit indices between models for models 2–6 were small, although model 4 was associated with lowest RMSE and joint-highest r2 correlation coefficients (with models 2 and 3). For all models, calibration slopes were positive (e.g. model 4: β = 0.64; s.e. 0.04; Table 2, Supplementary Fig. 4), indicating good agreement between observed and predicted probable FEP case-loads. For model 4, 60% (123 of 205) of predicted probable FEP case-loads at the CCG level were within ±10 cases per 100 000 person-years of the observed case-loads, with 87% (179 of 205) within ±20 cases per 100 000 person-years (Table 2).
Candidate model selection
Models 2–6 generally yielded similar apparent and external validation indices, but we interpreted model 4 as performing most consistently across all parameters (Tables 2 and 3). Visual inspection of maps of observed and predicted FEP case-loads for CCGs in 2017 indicated good apparent validity (Supplementary Fig. 5), although some spatial variation in differences remained evident, particularly in South England (Supplementary Fig. 6c).
Forecasting
Using model 4, we forecasted referred, assessed, treated and probable FEP case-loads in England between 2019 and 2025, based on population projections for each year (Supplementary Fig. 7 and Table 10). For 2020, our model predicted a new referred case-load of 24 499 (95% CI 22 930–25 991), an assessed case-load of 22 033 (95% CI 20 622–23 375), a treated case-load of 10 516 (95% CI 9843–11 157) and a probable FEP case-load of 9066 (95% CI 8485–9618) aged 16–64 years (predicted probable FEP incidence rate of 25.2 cases per 100 000 person-years, 95% CI 23.6–26.8). Between 2019 and 2025, predicted treated case-loads increased by 6.2%, on average, but we could not exclude the possibility of no change or a small decline in case-loads over time (95% CI −3.2% to 15.6%; Supplementary Table 10).
Forecasted FEP incidence varied geographically (for example, in 2020, see Fig. 1), with higher rates in CCGs serving most major conurbations, including Greater London, Greater Manchester, Bradford and Birmingham and the Midlands (Supplementary Table 11). We also predicted that some CCGs with relatively low predicted incidence rates would have large predicted counts of cases (i.e. new case-load sizes), given the large at-risk populations and geographical areas they served (Supplementary Table 12). Complete forecast prediction data at CCG and national levels for 2019–2025 is available at www.psymaptic.org via an open-access web visualisation platform and downloadable data-sets.
Visualisation of predicted incidence rates of FEP per 100 000 person-years by age and geographical level in England, 2020. Predicted incidence rate per 100 000 person-years at the CCG level for people aged (A) 16–35 years, (B) 36–64 years, (C) 16–64 years and (D) at ward level for people aged 16–64 years. Predictions were not produced for five Census merged wards (Isles of Scilly [one ward] and all four wards within City of London) because of inaccurate population estimates, and values are not shown in (D). CCC, Clinical Commissioning Group; FEP, first-episode psychosis.
Discussion
Principal findings
We developed and validated a population-level prediction model to forecast need for mental healthcare, here applied to referral to, assessment by and treatment for suspected psychosis in EIP services in England, using robust epidemiological data and small-area population projections. Our best-fitting model yielded acceptable validity against observed routine data on probable FEP at the CCG level in England for 2017, from which we extrapolated predictions up to 2025 based on small-area population forecasts. Our model also forecasted additional levels of need generated by referred, assessed and treated case-loads to provide a comprehensive, evidence-based tool for EIP planning, consistent with the timeframe of the ‘NHS Long-Term Plan’28 and ‘Mental Health Implementation Plan’.29
Strengths and weaknesses
Our model has a number of strengths. We synthesised robust epidemiological data on psychosis risk, prior knowledge and small-area variation in social and environmental factors, to validate an a priori set of candidate models against national routine data at the CCG level. Using Bayesian inference to model uncertainty around these predictions allowed us to forecast expected need for psychosis care in England up to 2025, within tolerable parameter estimates.
Prediction models are influenced by their underlying assumptions and limitations. Four issues are of particular note. First, our seed data included incident cases from approximately 10% of 2011 Census merged wards (n = 751 of 7678), covering 14.5% of CCGs in England. Predictions could be less accurate in areas that have very different sociodemographic or socioeconomic characteristics from those in our seed data-set regions. However, our best-fitting model displayed acceptable external validity to a national data-set at the CCG level, for total, gender- and ethnicity-specific case-loads in people aged 16–64 years. Second, our models under- and overestimated probable FEP case-loads in people aged 16–35 and 36–64 years, respectively, compared with observed data. This may have arisen from inclusion of epidemiological data in our seed data-set collected before the introduction of EIP services in England;Reference Kirkbride, Fearon, Morgan, Dazzan, Morgan and Tarrant30,Reference Coid, Kirkbride, Barker, Cowden, Stamps and Yang33 there is some evidence that age at first contact with mental health services may have fallen since their introduction.Reference Kirkbride, Hameed, Ankireddypalli, Ioannidis, Crane and Nasir31 Third, external validity of our models was reliant on the reliability and validity of observed FEP data from the MHSDS. Sensitivity analyses allowed us to validate our models in the presence of missing observed data on ethnicity. Uncertainty in either our predictions or the observed data could explain deviance in our calibration plots at the CCG level. Since the MHSDS only provides information on who commenced EIP treatment, and not the subset of individuals who met diagnostic criteria for FEP, we revised observed treated case-loads downward by 14% to obtain a comparable FEP sample, in line with empirical evidence.Reference Kirkbride, Hameed, Ankireddypalli, Ioannidis, Crane and Nasir31 This highlights the importance of recording reliable symptomatic and likely diagnostic outcomes in routine practice. Future models should include explicit predictions of people who meet CHR-P criteria for psychosis, implicitly included in our forecasts of new referred, assessed and treated case-loads per annum, but who could not be disaggregated because there is currently no provision to identify these groups separately in the MHSDS, upon which our forecasts were based. Fourth, our models did not account for predicted need in people aged <16 years (although absolute FEP incidence is rareReference Okkels, Vernal, Jensen, McGrath and Nielsen50), or homeless or prison populations.
Implications
Our Bayesian population-level prediction model could be readily adapted and updated to forecast need beyond 2025, in other countries or for other disorders, where sufficiently reliable risk estimates and denominator data are available. Application to other countries, particularly the devolved nations in the UK, is both vital for public mental health and also offers the opportunity to establish the robustness of our models to populations with potentially different distributions of underlying risk factors for psychosis.Reference Oliver, Reilly, Baccaredda Boy, Petros, Davies and Borgwardt16 Applied to FEP in England, our model provides healthcare commissioners with a decision-making tool to support the allocation of finite resources for EIP services, based on forecasts of need likely to arise in local populations, enhancing an earlier, more limited tool.Reference Kirkbride, Jackson, Perez, Fowler, Winton and Coid18 Providing commissioners with access to evidence-based forecasts of such need should help optimise and fully implement NICE-concordant care for psychosis.20,22 This would provide the foundation for realising the demonstrable, evidence-based improvements in clinical and social outcomes for individuals experiencing FEP, as well as the cost-effectiveness benefits on which they were established.Reference Bird, Premkumar, Kendall, Whittington, Mitchell and Kuipers24,Reference Stafford, Jackson, Mayo-Wilson, Morrison and Kendall25,Reference Birchwood, Connor, Lester, Patterson, Freemantle and Marshall51,Reference Csillag, Nordentoft, Mizuno, Jones, Killackey and Taylor52 Moreover, optimal resource allocation in one mental health domain should propagate benefits to other areas of care for which mental health commissioners and planners are responsible. Our predictions also allow care providers to understand the likely sociodemographic and environmental characteristics of their case-loads, which may be useful for providing locally and regionally sensitive modes of care delivery across diverse communities.
Overall, we have developed and validated a generalisable, translational tool that used epidemiological data to forecast accurate estimates of future incidence of first-episode mental health disorders in different communities. The tool currently informs recently-updated NICE guidelines in England for EIP service provision.53 Our prediction methodology could be readily generalised to other settings or mental health problems underpinned by a precise, reliable epidemiology. Priorities here would include extension to predict people at clinical high risk of psychosis, ongoing recalibration in England, and extension and validation to other countries. Applied to psychosis care in England, our platform provides commissioners and providers with an open-access, evidence-based toolkit to optimise need for EIP care as and when it arises.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1192/bjp.2021.18.
Data availability
All PsyMaptic-A prediction data are freely available at www.psymaptic.org. We will share the online platform and the results from this study with NHS England, Clinical Commissioning Groups and national, regional and local EIP service leads for patient benefit. Because of the aggregated nature of the data included in this study, it is not possible to directly contact participants from the original studies included in our seed data-set.
Acknowledgements
We are very grateful to Jay Nairn, Amy Clark, Carl Money, Thomas Bardsley and Alison Brabban of NHS England, for their expertise and advice. We thank Carl Money and Thomas Bardsley for running our validation scripts on a non-supressed version of the MHSDS data at NHS England. We thank all EIP regional leads for their input into the initial results of the modelling process, which allowed us to further develop and refine our models. We are grateful to Gareth Simons from Cityseer.io, for development of the ‘PsyMaptic-A’ web visualisation tool. We thank those who participated in the original AESOP, ELFEP and SEPEA studies, whose aggregated data has allowed us to develop a tool to inform future mental health service provision for psychosis in England. Written consent to cite personal communication was obtained from Carl Money, a copy of which is available on our preprint server (https://osf.io/pqrj9/).
Author contributions
K.M. and T.D. contributed equally as co-first authors. T.D. and G.B. were responsible for development of the statistical prediction models. K.M. and J.B.K. were responsible for model validation. J.B.K. and P.B.J. conceived and designed the original inception of the work. T.R.D. assisted with data collection. H.K. assisted with data visualisation and preparation of figures. J.B.K., P.B.J. and J.W.C. provided access to empirical psychosis incidence data for the study. P.W. developed and provided the denominator forecasts. D.O., P.F. and P.B.J. provided clinical and service level expertise to inform model development. All authors contributed to the interpretation of results and critical review of the manuscript. J.B.K. had full access to all the data in the study and had final responsibility for the decision to submit for publication.
Funding
This study was funded by an Enhancement Award from Wellcome Trust (101272/Z/13/A). This work was also supported by the National Institute for Health Research (NIHR), University College London Hospital, Biomedical Research Centre. D.O. is in part supported by the NIHR Applied Research Collaboration (ARC) North Thames. P.B.J. is supported by the Wellcome Trust (095844/Z/11/Z), the NIHR ARC East of England and the NIHR (RPPG-0616-20003). This report is independent research supported by the NIHR ARC North Thames. The views expressed in this publication are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care.
Declaration of interest
P.B.J. has been a member of scientific advisory boards for Janssen and Ricordati. All other authors have no conflicts of interest to declare.
Open access
eLetters
No eLetters have been published for this article.