There is growing concern in the UK regarding the burden of common mental health problems, which include different types of depression and anxiety. The proportion of individuals aged 16–64 experiencing these common mental health problems in England was approximately 17% in 2000 (period prevalence: based on present episode and episodes in the week before the interview) and this remained unchanged in 2007. Reference Singleton, Bumpstead, O'Brien, Lee and Meltzer1,Reference Bebbington, Brugha, Coid, Crawford, Deverill and D'Souza2 The cost of common mental health problems alongside other mental illness has been estimated at £77 billion, which includes health and social care costs (16%), reduced health-related quality of life costs (54%) and lost productivity (30%). 3 Despite the burden of disease, a large proportion of individuals, 76% in 2000, with common mental health problems were not receiving any form of treatment. Of the 34% receiving treatment, 15% were on medication whereas the rest were receiving therapy with or without medication. Reference Singleton, Bumpstead, O'Brien, Lee and Meltzer1 Evidence shows that psychological therapies such as cognitive–behavioural therapy (CBT) and combined therapies (for example CBT and medication together) are effective both for depressive and anxiety disorders. 4 Patients have shown a preference for these psychological therapies Reference Warner, Mariathasan, Lawton-Smith and Samele5 but provision in primary care has been low and in secondary care has been characterised by long waiting lists. In light of this, Lord Layard proposed a change in the delivery of treatment in the National Health Service (NHS) in order to improve access to psychological therapies. Reference Layard6 National Institute for Health and Clinical Excellence (NICE) guidance for depression and anxiety recommends that psychological therapies should be made available in a stepped approach within a collaborative care model. 7,8 Doncaster was one of two NHS primary care trusts selected in 2006 as ‘demonstration sites’ to deliver stepped care in order to improve access to psychological therapies. We carried out an economic evaluation of the Improving Access to Psychological Therapies (IAPT) service implemented at the Doncaster demonstration site alongside a wider evaluation of the demonstration sites. Reference Parry, Barkham, Brazier, Dent-Brown, Hardy and Kendrick9 The aim was to assess whether IAPT provided additional benefit over and above existing services by contrasting them with comparator sites and the associated additional cost of the IAPT service.
Method
Study design
We identified two comparator sites by comparing the IAPT demonstration site with other sites, using data held on the National Primary Care Trust Database. Information on the size and type of population served based on deprivation, ethnicity, and age was used to select sites with similar characteristics to the IAPT site. We also used information on how well sites were doing based on average Quality and Outcomes Framework points, a voluntary annual reward and incentive programme for all general practitioners (GPs) in England that assesses areas of clinical care, organisation, patient experience and other services. Comparators were also matched on: (a) geographical location, (b) local implementation of ‘pathways to work’ (as one of IAPT's key aims was employment), (c) recent changes in organisational structure and (d) ethnic diversity. Sites within this pool were approached to gain additional information in relation to service configuration, capacity and case mix so that the most appropriate comparator sites could be selected. As a result, Wakefield and Barnsley were identified as the two comparator sites for Doncaster. The former are referred to as comparator sites and the latter is referred to as the IAPT site in the rest of the paper.
Sample
All the GPs at the IAPT and comparator sites were approached for recruitment into the study between 2007 and 2009. Patients aged between 16 and 64 who had a new or recurrent episode of depression or anxiety and who were identified by their GP as patients who were likely to benefit from psychological therapies were recruited into the study at both IAPT and comparator sites as these were the criteria used to identify patients for referral to IAPT. Patients were recruited at GP services prior to referral to IAPT, but not all of them used the IAPT service. Patients recruited to the study at the IAPT site therefore included some who were seen in the new service as well as patients who were seen in other mental healthcare services in Doncaster.
A total of 403/3391 patients (based on an estimate from GPs that sent back information on the number of packs given to patients) were recruited from 29 general practices within the IAPT (n = 289) and comparator sites (n = 114), with response rates of 14 and 8% respectively (Fig. 1). Two patients were excluded from the IAPT site as one died and the other completed all their questionnaires with reference to a different period, leaving 401 patients. Of these, 86% (IAPT site, 245; comparator sites, 100) and 83% (IAPT site, 234; comparator site, 98) were followed up at 4 and 8 months respectively. A total of 99% (IAPT site, 285; comparator site, 113) had usable outcomes data at baseline. We had cost and quality-adjusted life-year (QALY) data for 74.1% of individuals (IAPT site, 212; comparator site, 85) at 8 months. A total of 127 IAPT site study participants were identified as having been seen in the IAPT service. Of these, 50 patients had already been seen by the IAPT service at baseline prior to recruitment into the study. These patients were retained in the analysis.
Study flow chart for the Improving Access to Psychological Therapies (IAPT) and comparator sites.
GP, general practitioner; PROs, patient-reported outcome data.
Intervention
IAPT service
The intervention offered at the IAPT service took a stepped-care approach based on NICE guidelines. 7,8 Step 1 was watchful waiting; Step 2 was guided self-help including bibliotherapy with support, computerised CBT with support and CBT-based telephone support for problem-solving; and Step 3 was CBT with or without medication. Most of the patients seen (over 90%) were assigned to Step 2 at initial contact, where the main interventions offered were bibliotherapy and CBT-based problem-solving with support from a case manager for both. Support was offered either face to face or more often by telephone after the first session. This included providing support through the assigned intervention and assessing progress, medication support and signposting to other services. Computerised CBT was offered to a small number of patients at Step 2 and this was also supported by a case manager by telephone. The rest were offered Step 1, i.e. monitoring, or Step 3, i.e. face-to-face CBT. Reference Clark, Layard, Smithies, Richards, Suckling and Wright10 The IAPT service specialised in patients whose primary diagnosis by the GP was depression (95%) although many who were considered also had generalised anxiety disorder. Reference Clark, Layard, Smithies, Richards, Suckling and Wright10
Non-IAPT and comparator site mental healthcare service
General practitioners are the major primary care provider for depression and anxiety in the UK and act as gate keepers for counselling within practices and secondary care services such as clinical psychologists, psychotherapists or counsellors. Comparator sites had comprehensive primary and secondary healthcare provision for mental health, with a variety of interventions, including visiting a GP, primary care counselling and referral to mental health professionals in secondary care. These services matched the IAPT site services prior to the new service. Patients at the IAPT site who did not use the new service had access to these other services, referred to as non-IAPT.
Outcome measures
The IAPT service included a comprehensive evaluation framework with interviewer-administered patient-reported outcome (PRO) measures namely, the Patient Health Questionnaire-9 (PHQ-9) Reference Martin, Rief, Klaiberg and Braehler11 and Generalised Anxiety Disorder-7 (GAD-7) Reference Spitzer, Kroenke, Williams and Lowe12 collected at each contact with the patient. We used the same measures at baseline, 4 and 8 months in this study. Recruitment packs containing study information, consent forms and baseline measures were handed or posted to eligible patients by GPs whereas the follow-up measures were posted by research staff. For the evaluation we used the postal self-reported questionnaire to ensure comparability across study participants. We identified those who had made reliable and clinically significant (RCS) improvement on the PHQ-9 as this is the depression-specific measure, and depression was the primary diagnosis for the majority of those seen at the IAPT site. We used the Jacobson & Truax Reference Jacobson and Truax13 criteria alongside published clinical cut-offs. Reference Martin, Rief, Klaiberg and Braehler11 Patients were classified as having had an RCS improvement if their score changed by six points and moved from a clinical population at baseline (10 and above) to a non-clinical population at 8 months (9 or less). Reference Parry, Barkham, Brazier, Dent-Brown, Hardy and Kendrick9
The Short Form (SF-6D), Reference Brazier, Roberts and Deverill14 a preference-based generic measure of health designed for calculating QALYs, was included in the self-reported questionnaire and used for the cost-effectiveness analysis. It comprises six multi-level dimensions of health: physical functioning, role limitation, social functioning, bodily pain, mental health and vitality. The SF-6D utility values were derived using a valuation set from members of the UK general population. Reference Brazier, Roberts and Deverill14
Resource use
The analysis took a health and care perspective and so covered all available NHS and personal social services (PSS) resources, including the new IAPT service itself and the consequences for use of primary, community and secondary healthcare, and Social Services. We measured resource use by patients over an 8-month follow-up period. Resource use was collected using a self-reported postal questionnaire that consisted of two sections, one related to mental health service use, which asked about the number of times mental health professionals (psychiatrist, psychologist, community psychiatric nurse, psychotherapist/counsellor, other mental health professionals and voluntary sector services) were seen by respondents in the previous 4 months. The second section related to other primary and secondary healthcare and PSS use. This questionnaire was administered at baseline, 4 and 8 months. Improving Access to Psychological Therapies service use was reported as contact time by the IAPT service.
Costing resource use
The IAPT service was costed using a top-down approach using financial data and service provision for the duration of the study (2007–2009). Financial data were obtained for the costs of setting up and running the service for the 2 years of the study, including training, equipment, facilities and overheads, to provide estimates of the costs associated with IAPT. Set-up costs were a small proportion of total costs (less than 10%) and these were therefore apportioned to this 2-year period rather than the lifetime of the service. The service recorded contact (face to face or over the phone) time in minutes for each service user and this was used to calculate total contact time over the 2 years, which was combined with total cost data to generate an average cost per minute for the IAPT service. This value was used to allocate IAPT costs for those patients who were seen by the service.
Use of non-IAPT health and social care services was valued using published national unit cost data Reference Curtis15 and NHS reference costs 16 as well as from evaluations. Reference Munro, Nicholl, O'Cathain, Knowles and Morgan17 All health and social care services at the comparator sites were also valued using national unit costs.
A broader perspective of costs was taken by assessing productivity impact, which we valued using the lost number of days from work using a human capital approach. The cost of each day of lost employment was assumed to be equal to the age- and gender-specific national median daily wage rates. Full- or part-time status was also taken into account with differing wage rates applied for the two groups. 18
All costs are presented in British pounds for the years 2008/2009. Costs that were for previous years were up-rated to 2008/9 prices using the UK retail price index from the UK Office for National Statistics.
Data analysis
Parametric (independent samples t-test) and non-parametric (Fisher's exact test; χ2-test; Mann–Whitney U-test) tests were undertaken to compare sociodemographic, health, well-being and resource use at baseline between sites and those who were followed up with those who were not followed up at 4 and 8 months. We compared changes in costs and outcomes between the IAPT site and its comparator sites. Ordinary least squares multiple regressions were used to adjust for baseline characteristics: age, gender and baseline outcomes or costs in all tests of differences in outcomes and costs.
Economic evaluation
The economic evaluation included an assessment of the cost-effectiveness of the intervention in terms of its incremental cost per QALY, that is, the additional cost of providing one unit of additional benefit. We calculated the additional costs and QALYs as the difference in costs and QALYS between the IAPT and the comparator sites and assessed the ratio between these, i.e. incremental cost-effectiveness ratio (ICER = (CostIAPT–CostCOMPARATOR)/(QALYIAPT–QALYCOMPARATOR). The NICE threshold of £20 000 to £30 000 per QALY was used to assess the cost-effectiveness of the IAPT service, with ICERs falling below these values deemed as cost-effective. Following NICE guidelines, productivity costs were calculated separately and were not included in the incremental cost per QALY analysis.
In addition, we assessed the cost-effectiveness in terms of the proportion who achieved RCS improvement based on the PHQ-9. The ICER was based on additional costs and additional proportion achieving RCS improvement (i.e. the difference between IAPT and comparator sites).
Sensitivity analysis
A probability sensitivity analysis (PSA) was undertaken to examine the uncertainty surrounding the cost-effectiveness ratios. Estimates of QALY gains and cost per patient were bootstrapped with 10 000 replications and presented as cost-effectiveness acceptability curves (CEACs) which showed the probability of IAPT being cost-effective at different costs per QALY. This was done for the SF-6D and RCS analysis as well as for the EQ-5D Reference Brooks19 estimates discussed below.
The NICE reference case since 2008 has been based on the EQ-5D, a 5-dimension preference-based measure of health used to estimate QALYs. Reference Parry, Barkham, Brazier, Dent-Brown, Hardy and Kendrick9 In order to improve comparability with this metric, SF-6D was translated into EQ-5D using an empirical mapping function. A large patient data-set, the Health Outcomes Data Repository (n = 15 184) collected from in-patients and out-patients in a hospital in Wales, UK, was used to estimate this function. Reference Currie, McEwan, Peters, Patel and Dixon20 The data-set contains both EQ-5D and SF-6D scores and the preference-based indexes were used to model the relationship between EQ-5D and SF-6D. (The function used was: EQ-5D = 1.563572×SF-6D score – 0.3502361. Results were capped at 1.0 for the highest values.) This model was then applied to the IAPT data. Predicted EQ-5D scores were used to calculate QALY gains (or losses) and the incremental cost per QALY. The PSA was also carried out using the predicted EQ-5D scores. A further sensitivity analysis of the IAPT costs was carried out by using national unit costs based on the cost of seeing an NHS counsellor of £67 per session in place of IAPT unit costs to evaluate the effect of different costs on the cost-effectiveness of the service.
The IAPT service users at the IAPT site in our study were drawn from a larger population of IAPT patients. We assessed whether the former were significantly different from the latter in terms of outcomes to examine the generalisability of findings. Ethical approval for this study was received from the Leeds NHS Research Ethics Committee (Ref: 07/Q1205/54). Data analysis was carried out using Stata 9.2 for Windows.
Results
The average age of study participants was 41.1 (s.d. = 14.7) and majority were female (72.8%) and White (98%) with no statistically significant differences across the IAPT and comparator sites. There were 208/401 (52%) study participants in full- or part-time employment and 88/401 (22%) were unemployed. There were no differences in employment activity at baseline when comparing the IAPT site and its comparator sites.
Individuals who had dropped out of the study or had incomplete data at 8 months at the IAPT site had different characteristics from those who were followed up. At the IAPT site, those who dropped out were younger than those who were followed up and had complete cost and QALY data (mean (s.d.): 39.1 (14.5) v. 42.6 (15.0), t 281 = −1.7, P<0.1). There were no significant differences between the two groups in terms of baseline SF-6D scores but those who dropped out had significantly higher baseline PHQ-9 scores (mean (s.d.): 19.4 (6.4) v. 15.4 (7.0), t 136 = 4.5, P<0.01) and GAD-7 scores (mean (s.d.): 15.8 (5.0) v. 13.2 (5.8), t 141 = 3.6, P<0.01), indicating that they had poorer mental health than those who remained in the study. There were no significant differences in age, SF-6D scores or clinical characteristics for those who dropped out compared with those who were followed up at the comparator sites.
Patient-reported outcomes at baseline, 4 and 8 months and comparison of changes for Improving Access to Psychological Therapies (IAPT) and comparator sites
| IAPT site | Comparator site | Difference in changeFootnote a , Footnote b | |||
|---|---|---|---|---|---|
| n | Mean (s.d.) | n | Mean (s.d.) | Mean (95% CI)Footnote b | |
| PHQ-9 total score | |||||
| Baseline | 285 | 16.44 (7.10) | 113 | 15.12 (7.25) | |
| 4 months | 244 | 13.04 (7.68) | 96 | 12.11 (7.18) | –0.04 (–1.35 to 1.26) |
| 8 months | 231 | 11.81 (7.60) | 96 | 11.08 (7.54) | –0.06 (–1.42 to 1.31) |
| GAD-7 total score | |||||
| Baseline | 284 | 13.86 (5.71) | 113 | 13.30 (5.83) | |
| 4 months | 241 | 11.01 (6.48) | 97 | 10.53 (6.12) | 0.21 (–0.99 to 1.41) |
| 8 months | 231 | 9.92 (6.32) | 97 | 9.95 (6.29) | –0.19 (–1.47 to 1.10) |
| SF-6D | |||||
| Baseline | 271 | 0.61 (0.13) | 112 | 0.63 (0.12) | |
| 4 months | 232 | 0.65 (0.15) | 95 | 0.66 (0.14) | 0.010 (–0.015 to 0.034) |
| 8 months | 218 | 0.66 (0.15) | 91 | 0.69 (0.16) | –0.002 (–0.029 to 0.025) |
PHQ-9, Patient Health Questionnaire-9: 0 to 27 (low to high depression); GAD-7, Generalised Anxiety Disorder-7: 0 to 21 (low to high anxiety); SF-6D, Short Form-6 dimensions: 0.301 to 1 (low to high health utility).
a. Mean difference in change: (IAPT follow-up – baseline) – (Comparator follow-up – baseline).
b. Adjusted for gender and age.
Outcomes
Table 1 shows the PRO data for the IAPT and comparator site study participants at baseline, 4 months and 8 months. There were statistically significant reductions in the mental health scores for the PHQ-9 and GAD-7 at 4 and 8 months at both the IAPT site and its comparators, resulting in small differences between them that were not statistically significant. At baseline, 81% (232/285) of the patients at the IAPT site and 74% (84/113) at the comparator sites had PHQ-9 scores in the clinical range. At 8 months, 22% (46/209) had made reliable and clinical improvements at the IAPT site compared with 20% (17/84) at comparator sites.
At 4 and 8 months all the sites achieved improvements in the SF-6D with significant changes for all sites at both time points. The IAPT site had small improvements over the comparator sites that were not statistically significant at 4 months and this gain disappeared by 8 months (Table 1).
Employment
There were 111 individuals who were unemployed at baseline (IAPT site, 76; comparator site, 35). At 8-month follow-up, 10% (6/62) and 17% (4/24) of individuals who were followed up at the IAPT and comparator sites respectively were in full- or part-time employment. However, 14% (17/123) at the IAPT site and 4% (2/51) of individuals who were employed at baseline were unemployed at follow-up. Days off work fell over the 4 and 8 month periods in both IAPT and comparator sites but there were no significant differences between them (Table 2). The wide confidence intervals indicate a high level of variability.
Number of working days off over 8-month period
| IAPT site | Comparator site | Difference in changeFootnote a , Footnote b | |||
|---|---|---|---|---|---|
| Days of work | n | Mean (s.d.) | n | Mean (s.d.) | Mean (95% CI)Footnote b |
| Baseline | 151 | 16.0 (25.0) | 57 | 9.2 (21.8) | |
| 4 months | 123 | 11.1 (23.1) | 52 | 7.3 (16.0) | 2.3 (–4.5 to 9.1) |
| 8 months | 118 | 7.8 (22.3) | 51 | 4.3 (9.6) | 3.0 (–4.3 to 10.4) |
IAPT, Improving Access to Psychological Therapies.
a. Mean difference in change: (IAPT follow-up – baseline) – (comparator follow-up – baseline).
b. Adjusted for gender and age.
Resource use
There were significant increases in NHS psychological therapist use (including IAPT contacts) at the IAPT site compared with the comparator sites at 4 months (mean difference (s.d.): 0.78 (3.00) v. −0.09 (2.39), U = 10343, P<0.001). These differences were not sustained at 8 months. There were also significant reductions in GP surgery use at the IAPT and comparator sites, with larger reductions at the former (mean difference (s.d.): 1.36 (5.60) v. 0.40 (3.24), U = 10314, P<0.05), which were sustained at the 8-month follow-up (mean difference (s.d.): 1.68 (6.23) v. 0.95 (3.71), U = 9721, P<0.05). There was also some evidence of reductions in contacts with social workers, health visitors and other unspecified health services that were significantly different from the comparator site for the IAPT site.
Costs of services
Based on total costs and contact time in the IAPT service, average cost per minute over the 2 years was £4.33 and this value was used to allocate costs to IAPT study patients. The average contact time per patient that was seen was 129 min over an average of 3 sessions, giving an average cost per patient of £559. Costs associated with seeing an NHS psychological therapist (includes IAPT contacts) were significantly higher at the IAPT sites compared with the comparator sites as these included the intervention cost (Table 3). Costs for primary care and hospital services were slightly lower at the IAPT site compared with the comparator sites but not significantly so, whereas social work costs were significantly lower at the IAPT site compared with the comparator sites. Mean total NHS and PSS costs per patient at the IAPT site were higher than those at the comparator sites but the confidence intervals indicate that these were not significant differences (Table 3).
Costs of lost employment
The cost of lost employment was higher at the IAPT site than its comparator sites. However, the differences between the IAPT and the comparator sites were not statistically significant (Table 3).
Costs (£) per participant over 8-month follow-up period
| IAPT site (n = 211) Mean (s.d.), £ |
Comparator site (n =84) Mean (s.d.), £ |
Difference in change Mean (95% CI),Footnote a , Footnote b £ |
|
|---|---|---|---|
| NHS psychological therapyFootnote c | 322 (446) | 60 (194) | 258 (158 to 355) |
| Other mental healthcare | 25 (123) | 22 (60) | 2 (–20 to 24) |
| Primary care | 168 (176) | 157 (171) | –7(–46 to 35) |
| Secondary care | 664 (2045) | 637 (1496) | –21 (–490 to 450) |
| Social care | 12 (131) | 58 (265) | –49 (–93 to −5) |
| Total cost | 1190 (2193) | 934 (1666) | 163 (–331 to 660) |
| Productivity costs | 669 (2019) | 391 (990) | 266 (–175 to 710) |
IAPT, Improving Access to Psychological Therapies.
a. Mean difference in change: (IAPT follow-up – baseline) – (comparator follow-up – baseline).
b. Adjusted for gender and age.
c. National Health Service (NHS) psychotherapist includes IAPT contacts.
Cost-effectiveness analysis
The IAPT site had a small QALY gain over the comparator sites and combining this with the cost resulted in an ICER of £29 500 per QALY (Table 4). All estimates were associated with very large degrees of uncertainty and this was reflected in the wide confidence intervals. Figure 2 represents the CEAC of the ICERs from the bootstrapped replications for costs and QALYs for the IAPT site compared with its comparator sites. The probability that IAPT was cost-effective was below 40% at a cost of £30 000 per QALY. Sensitivity analysis using predicted EQ-5D scores resulted in a slightly higher QALY gain compared with using the SF-6D for IAPT. This increase resulted in a cost per QALY ratio of £16 857 with a probability of cost-effectiveness of 38% for a willingness to pay for a QALY of £20 000 and 53% at £30 000 (Fig. 2). The ICER based on RCS improvement for the PHQ-9 was £9440 per participant who achieved this criterion. The probability that IAPT was cost-effective at achieving RCS improvement is shown in Fig. 3.
When national unit costs were used in place of IAPT costs, the mean total NHS and PSS cost per patient was £1042 (95% CI 749–1334) in IAPT. This represented an incremental cost of £95 (95% CI −422 to 612), resulting in an ICER of £11 875 per QALY using SF-6D QALY gains and £3800 per participant achieving RCS improvement.
Generalisability of findings
Given the low response rate, there is a concern that the sample may be biased. To address this we compared the study sample with the full sample of those who used the IAPT service. We had access to PRO scores obtained by interviewer administration at the IAPT service for all service users (n = 4616), including participants in our study who completed the same instrument PROs by post for our study. The former are referred to as service scores and the latter as study scores. Baseline PHQ-9 service scores for all users were similar to the study scores for those seen at the IAPT site (mean (s.d.): 16.1 (6.2) v. 15.9 (7.1)), but the follow-up scores were substantially lower (mean (s.d.): 9.7 (7.5) v. 11.9 (7.6)). This difference may be because the IAPT patients recruited to the study had lower gains than other IAPT users. The service scores for our study patients were not significantly different from service scores of the other IAPT users at baseline (mean (s.d.): 16.1 (6.2) v. 16.4 (6.0)) and follow-up (mean (s.d.): 9.7 (7.5) v. 10.4 (7.3)), which suggests it is not the result of differences in outcome between the samples. Variation in methods of data collection at the service and in our study may have resulted in these differences. Timing of data collection varied between the service and our study as we followed patients for 8 months in the study but they may have continued to use IAPT services. The service scores reflect this longer period and our study scores may therefore be biased by the shorter time frame that is assessed. Differences may also arise from the service follow-up scores being collected by the service (when the patients may still be enjoying the immediate benefit of the contact) compared with the study follow-up, which is completed outside the service. Overall this should not make a difference, since the method of data collection is comparable between those who use the IAPT service and those who do not.
Cost-effectiveness: National Health Service (NHS) and personal social services (PSS) perspective
| IAPT site | Comparator site | Difference between IAPT and comparator site | ||
|---|---|---|---|---|
| Mean (95% CI) |
Mean (95% CI) |
Unadjusted, mean (95% CI) |
Adjusted, mean (95% CI)Footnote a , Footnote b |
|
| Total NHS and PSS costs (£) | 1 190 (892 to 1 488) | 934 (572 to 1 295) | 256 (–266 to 779) | 236 (–214 to 689) |
| SF-6D QALY gain | 0.026 (0.018 to 0.033) | 0.018 (0.007 to 0.029) | 0.007 (–0.006 to 0.021) | 0.008 (–0.005 to 0.021) |
| ICER, cost (£) per QALY | 36 571 | 29 500 | ||
| EQ-5D QALY gain | 0.038 (0.027 to 0.049) | 0.025 (0.009 to 0.040) | 0.013 (–0.007 to 0.033) | 0.014 (–0.005 to 0.032) |
| ICER, cost (£) per QALY | 19 692 | 16 857 | ||
| RCS PHQ-9 gain | 0.221 (0.164 to 0.278) | 0.205 (0.116 to 0.293) | 0.016 (–0.089 to 0.122) | 0.025 (–0.078 to 0.127) |
| ICER, cost (£) per RCS gain | 16 000 | 9 440 | ||
IAPT, Improving Access to Psychological Therapies; SF-6D, Short Form-six dimensions; QALY, quality-adjusted life-year; ICER, incremental cost-effectiveness ratio; RCS, reliable and clinically significant; PHQ-9, Patient Health Questionnaire.
a. Adjusted for gender and age.
b. Bootstrap confidence interval.
Short Form (SF-6D) and EQ-5D cost-effectiveness acceptability curves: Improving Access to Psychological Therapies (IAPT) v. comparator sites.
QALY, quality-adjusted life-year.
Discussion
This study found small differences between the improvements in patients at the IAPT site and those from the comparator sites. The differences tended to favour the IAPT site over the comparator sites. For example compared with its comparators the difference in SF-6D score was 0.01 at 4 months after controlling for background variables. However, none of these differences were significant at 4 months and by 8 months had disappeared. In all, 2.5% more patients had made RCS improvements at 8 months at the IAPT compared with the comparator sites. Access to IAPT services led to significant increases in the use of NHS psychological therapy, as expected, and reductions in GP use at the IAPT site, which were significantly different from changes in equivalent service use at the comparator sites.
Patient Health Questionnaire (PHQ-9) reliable and clinically significant (RCS) improvement cost-effectiveness acceptability curves: Improving Access to Psychological Therapies (IAPT) v. comparator sites.
RCS: proportion achieving reliable and clinically significant improvement.
The IAPT site was more expensive over the 8-month follow-up period than its comparator sites, although not significantly so. Cost-effectiveness analysis indicated that the IAPT service was just within the NICE threshold of £30 000 per QALY. The incremental cost of achieving RCS improvement was £9440 per participant. The results were sensitive to the method for valuing health states and costs. Using predicted EQ-5D values led to ICER values of £16 857 per QALY. When we assessed IAPT sites using national unit costs, this resulted in lower costs and cost-effectiveness ratios, which were under the £20 000 threshold. There is a reasonable argument for using EQ-5D values since this makes them more consistent with the NICE threshold. Whether national unit cost data should be applied to the IAPT service is debatable. Local costs may reflect some ‘learning effects’ as this was a demonstration site and the IAPT service has special features that may not be reflected in the national cost data. However, the average cost of IAPT at £4.33 per minute was higher than estimated costs of similar services such as face-to-face CBT at £1.07 per minute (2008/2009 prices). Reference Curtis15 This may be a reflection of the top-down costing approach, which can produce higher estimates than bottom-up approaches. Reference Drummond, O'Brien, Stoddart and Torrance21 Our results may therefore represent a conservative estimate of cost-effectiveness because of higher costs.
A small proportion of individuals who were unemployed at baseline were in full- or part-time employment at follow-up but there were also individuals who became unemployed. The costs associated with lost employment days did not differ significantly at follow-up when the IAPT site was compared with its comparator sites.
Previous studies
There is little evidence on the cost-effectiveness of IAPT in practice. Trial data reviewed for NICE guidelines indicated that the interventions that have been used in IAPT can be cost-effective. 7,8 However, how this translates in routine practice is not clear. Assessments of IAPT services have typically focused on the effectiveness of IAPT without considering costs. Reference Clark, Layard, Smithies, Richards, Suckling and Wright10,Reference Richards and Suckling22,Reference Glover, Webb and Evison23 Assessment of the initial roll-out of IAPT services (Wave 1) that followed the demonstration sites indicated that the interventions offered were clinically effective but there was notable variation across the 32 sites assessed. Reference Glover, Webb and Evison23 The assessment also reports low number of contacts per patient (median, 2) across the sites compared with Doncaster IAPT contact (median, 3). However, this assessment did not take into account improvements over and above what would have been usual care so direct comparisons cannot be made with the current study. Cost–benefit analysis of psychological therapies undertaken by Layard et al Reference Layard, Clark, Knapp and Mayraz24 assumed that QALYs gained would be 0.11 based on the EQ-5D, which is higher than QALY gains reported here of 0.014. This estimate highly inflated the QALY benefits to society of psychological therapies (estimated at £3300) whereas using our findings this would be equivalent to £420 (£30 000×0.014).
Limitations
The low response rate among the patients invited to participate means we may have failed to detect important differences in outcomes between the IAPT and its comparator sites because the study was inadequately powered. It also has implications for the representativeness of the samples. Our assessment of potential differences between the IAPT site study patients and the larger population of IAPT patients found that there were no significant differences in their service scores. This indicates that those who were recruited into the study were not different from the rest of the IAPT service users. However, service scores differed from study scores at follow-up. The timing of measures may account for some of these differences and there may also be administration-reporting bias with lower self-reported outcomes in the study underestimating the effects of IAPT. However, it may be that study scores that were administered by postal questionnaire were less affected by social desirability bias, as may have been the case for the IAPT-service measures, which were administered by IAPT staff. We could not verify whether non-IAPT service users were representative of the population from which they were drawn. In addition, study patients who were followed up and had QALY data at the IAPT site were younger and their mental health outcomes were better at baseline than those who dropped out, whereas this was not the case at the comparator sites, which means the changes at the IAPT site may have been biased by this attrition. However, it is difficult to establish the direction of bias as they may have benefited more or less from using IAPT services.
We did not include medication costs in the analysis and this may have had implications for cost differences as evidence suggests that medication use went up for IAPT service users. Reference Parry, Barkham, Brazier, Dent-Brown, Hardy and Kendrick9 Inclusion of medication costs may have resulted in an ICER, based on the SF-6D, above the cost-effectiveness threshold of £30 000 per QALY. However, follow-up over a longer period than the 8 months used in this study may have shown better clinical outcomes because of adherence to medication in this group.
Implications for policy
The IAPT initiative has been rolled out nationally, with the new government guidance indicating that it will continue to form an important part of mental healthcare provision. 25,26 The results of this study suggest that IAPT was within the upper NICE threshold of £30 000 and using the NICE reference case this fell below the £20 000 threshold but there was a large degree of uncertainty. As we assessed the demonstration site over the first 2 years of the service, where considerable learning effects with associated costs were likely, assessing other IAPT sites compared with usual care would provide further information on the cost-effectiveness of IAPT.
Acknowledgements
We would like to thank staff in Doncaster IAPT service for their support, the GPs and the practice managers in the practices which sent out questionnaires, and all study participants for taking part.
This project was commissioned by the NIHR Service Delivery and Organisation (NIHR SDO) programme under the management of the National Coordinating Centre for Service Delivery and Organisation (NCCSDO), which was based at the London School of Hygiene and Tropical Medicine (LSHTM). The project was managed by NCCSDO until 31 March 2009. Management of the NIHR SDO programme and its portfolio of projects transferred to the National Institute for Health Research Evaluations, Trials and Studies Coordinating Centre (NETSCC), based at the University of Southampton from 1 April 2009. From January 2012, the NIHR SDO programme merged with the NIHR Health Services Research (NIHR HSR) programme to establish the new NIHR Health Services and Delivery Research (NIHR HS&DR) programme.
The views and opinions expressed herein are those of the authors and do not necessarily reflect those of the HS&DR Programme, NIHR, NHS or the Department of Health.
eLetters
No eLetters have been published for this article.