People with psychosis die on average 15 years earlier than those without this condition.Reference Chang, Hayes, Perera, Broadbent, Fernandes and Lee1 The main cause is cardiovascular disease, which is associated with lifestyle factors and the side-effects of antipsychotic medication.Reference Hennekens, Hennekens, Hollar and Casey2,Reference Hirsch, Yang, Bresee, Jette, Patten and Pringsheim3 National guidelines on the management of people with psychosis recommend that people receive annual health checks in which risk factors for premature morality are screened and appropriate interventions are delivered.4 However, data from a national audit in England and Wales in 2011 revealed that less than a quarter of people were receiving these checks.Reference Crawford, Jayakumar, Lemmey, Zalewska, Patel and Cooper5 In response to this, NHS England sought to improve performance by introducing a financial incentive as part of the Commissioning for Quality and Innovation framework (CQUIN) programme.6 Although financial incentives are widely used in an effort to improve the quality of healthcare, no controlled studies have been conducted in secondary care settings. Although the balance of available evidence suggests that financial incentives improved targeted aspects of care, there is also some evidence that the quality of components of care that are not targeted can deteriorate.Reference Campbell, Reeves, Kontopantelis, Sibbald and Roland7,Reference Doran, Kontopantelis, Valderas, Campbell, Roland and Salisbury8 We therefore set out to examine the costs and benefits of this financial incentive by comparing the quality of care that people with psychosis received in England and in Wales before and after the introduction of the CQUIN.
Method
This was a longitudinal, controlled, observational study using data from the National Audit of Psychosis in England and Wales. The first two rounds of the audit were conducted in 2011 and 2013, prior to the introduction of the CQUIN. A further round of the audit was conducted 3 years after the introduction of the financial incentive in 2017. The three audits aimed to collect data from randomly selected samples of patients using secondary care mental services provided by English trusts and Welsh health boards over this 6-year period (trusts and health boards will be collectively referred to as ‘trusts’ throughout the remainder of this report).
The CQUIN on physical healthcare of people with psychosis set targets for the proportion of people who are screened for weight, blood pressure, cholesterol and glucose level, alcohol misuse and drug misuse, and for the proportion of people who screened positive who were offered appropriate interventions.9 It allowed local commissioners to withhold 0.2% of total funding per year to providers of mental health services depending on their performance.9,10 At first the CQUIN applied only to people who had an admission to hospital, it was then extended to all patients in 2015.
Participants
The study population was people aged 18 years or over with a clinical diagnosis of schizophrenia or schizoaffective disorder receiving care from secondary care mental health services.11 For each round of the audit, providers of mental health services in England and Wales were helped to generate a random sample of eligible patients. In the first two rounds of the audit the sample size was 100 per trust. In the third round of the audit the sample size for each trust was adjusted depending on its size between a minimum of 100 and a maximum of 300 patients. For each selected patient, staff working in the trust extracted data from electronic and other patient records and entered them on to a secure online data-collection platform. There were differences in the samples used for the audits in 2011, 2013 and 2017. In 2011 and 2013, the sample included people receiving care from early-intervention psychosis teams, but these patients were excluded from the audit in 2017. All those that took part in the 2011 and 2013 audit had a clinical diagnosis of schizophrenia or schizoaffective psychosis, but the 2017 audit collected data on a wider group of patients including those with drug-induced psychosis. To make a valid comparison between audits in different years, we excluded data from patients treated by early-intervention psychosis teams in the 2011 and 2013 audits and excluded patients who did not have a diagnosis of schizophrenia or schizoaffective disorder from the analysis of data from the 2017 audit.
Study outcomes and data analysis
The primary outcome measure was the proportion of patients in each year who were screened for seven risk factors for poor physical health and had documented evidence of interventions being offered for any risk factor that was identified during the screening – this is subsequently referred to as ‘complete screening and interventions’. The seven risk factors were those that the CQUIN focused on: tobacco use, excessive alcohol use, substance misuse, excessive body weight, hypertension, raised blood glucose and dyslipidaemia. Cut-off points for intervention were those recommended in the Lester Cardiometabolic tool, which was a specifically developed guidance and endorsed by the National Institute for Health and Care Excellence (NICE) to meet the needs of people with psychosis.Reference Shiers, Rafi and Cooper12 Possible interventions included advice about diet and exercise, treatment for cardiovascular disease, diabetes, dyslipidaemia or hypertension, psychological and pharmacological interventions to promote smoking cessation, and help to reduce or stop alcohol and substance misuse. The secondary outcomes for the study were the proportion of patients who were screened for each of the seven risk factors and the proportion who had documented evidence of being offered interventions for each item where the patient screened positive according to the Lester tool.
In addition to collecting the age, gender, ethnicity and ICD diagnosis13 of each participant, we obtained data on six non-incentivised indicators of the quality of care that each patient received. Each indicator was derived from national guidelines on the treatment of people with psychosis,4 and was selected prior to the analysis of data from the 2017 audit. Three indicators concerned the quality of prescribing and three covered psychosocial aspects of care: whether the patient was treated with two or more antipsychotic drugs (polypharmacy), whether the current dose of antipsychotic medication exceeded the upper limit of British National Formulary dose recommendations,14 whether if the patient was not in remission they were being prescribed clozapine, whether they had been offered cognitive–behavioural therapy (of any type) or family interventions and whether there was documented evidence of the patient having a current care plan. Information on care plans was only collected in the 2013 and 2017 audit.
Data were analysed using IBM SPSS version 21.0 for Windows. We calculated the frequency and proportion of patients meeting each measure in each round of the audit, in England, in Wales and in the total sample. Differences in proportions between each round of the audit were calculated together with 95% confidence limits.Reference Newcombe15
During the development of the first round of the audit, the National Research Ethics Service and the Ethics and Confidentiality Committee of the National Information Governance Board advised that formal ethical approval was not required because this was an audit and patient identifiable data were not being collected. The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.
Results
A total of 60 (93.8%) of 64 trusts provided data for the audit in 2011 and all trusts took part in the 2013 and 2017 rounds of the audit. The total number of patients for whom data were obtained was 5091 in 2011, 5608 in 2013 and 9949 in 2017. Having excluded people using early intervention is psychosis services from the 2011 and 2013 audits and excluded those with diagnoses other than schizophrenia or schizoaffective psychosis from the 2017 audit, the total number included in this analysis was 17 947. Demographic characteristics of these patients are presented in supplementary Table 1 available at https://doi.org/10.1192/bjp.2019.162. The proportion of the participants who were screened and offered interventions across England and Wales for the three rounds of the audit are presented in Table 1.
Levels of screening and intervention in total during the three rounds of the audit
a. All interventions were calculated as a proportion of patients offered an intervention where indicated.
*P < 0.05, **P < 0.01, ***P < 0.001.
There was a modest increase of 2.3% in the proportion of people with compete screening and interventions between 2011 and 2013 and a more marked increase of 18.4% between 2013 and 2017. Changes in these proportions within England and within Wales are presented in Table 2. Although an increase was seen in the proportion of people with documented evidence of complete screening and interventions in both England and Wales between 2013 and 2017, the increase in England was more marked than that in Wales (18.8% compared with 5.6%). Changes were especially marked in the proportion who were offered an intervention among those who screened positive in England (from 40.2% in 2013 to 67.6 in 2017, difference in proportions 27.5%, 95% CI 25.5–29.4), a difference that was not seen in Wales. The proportion who were offered all interventions identified during screening are presented in Fig. 1.
Levels of screening and intervention in England and Wales during the three rounds of the audit
a. All interventions were calculated as a proportion of patients offered an intervention where indicated.
*P < 0.05, **P < 0.01, ***P < 0.001.
The proportion of patients offered all interventions where a need was identified during screening in England and Wales across the three rounds of the audit.
Quality indicators on other aspects of care for people with psychosis in England and Wales across each of the three rounds of the audit are presented in Table 3. Within both England and Wales there was an increase in the proportion of patients who were treated using more than one antipsychotic drug between 2013 and 2017. In England, there were small but statistically significant reductions in the proportion of patients who were offered cognitive–behavioural therapy, family interventions and had a current care plan. These changes were not seen in Wales, where there was a trend towards more patients being referred for cognitive–behavioural therapy.
Quality of prescribing, psychological interventions and care planning in England and Wales during the three rounds of the audit
a. Proportion of patients on clozapine was calculated as a proportion of patients not in remission.
*P < 0.05, **P < 0.01, ***P < 0.001.
Discussion
Results of this research demonstrate that there was a marked improvement in the quality of physical healthcare of people with psychosis in England between 2013 and 2017. We cannot be certain that it was the introduction of the financial incentive that was responsible for this improvement, but a number of related findings suggest that it was. First, only modest improvements were seen in the quality of physical healthcare delivered to people with psychosis in England between 2011 and 2013, prior to the introduction of the CQUIN. Second, we found no evidence of significant improvements in the quality of other aspects of care received by people with psychosis during this period, and some evidence of falling standards of care. Third, the scale of the improvement between 2013 and 2017 was far greater in England than it was in Wales where no financial incentive was offered. Although a range of policies, training programmes and other initiatives aimed to improve the quality of physical healthcare received by people with psychosis during the period, these were implemented across both England and Wales. These included policy initiatives in England and Wales, and the audit programme itself, which provided benchmarked data to each trust and held local events for frontline clinicians aimed at highlighting good practice. The improvement in levels of interventions offered in England took place at a time when a primary care based financial incentive that aimed to promote monitoring the body mass index of people with psychosis was withdrawn in England but continued to operate in Wales.Reference Minchin, Roland, Richardson, Rowark and Guthrie16 Ensuring that people with psychosis are offered high-quality physical healthcare is a responsibility that is shared between primary and secondary care, so the improvements we found in England are even more striking given the reduced incentives provided to primary care services to improve physical healthcare for people with psychosis in England during this period.
Within the overall improvement in levels of screening and interventions, there was some reduction seen in the proportion of people screened for smoking and substance misuse. This finding is in keeping with previous research on the impact of financial incentives indicating that there is a ceiling effect, in which performance is unlikely to improve when baseline levels are already high.Reference Campbell, Reeves, Kontopantelis, Sibbald and Roland7,Reference Doran, Fullwood, Kontopantelis and Reeves17,Reference Calvert, Shankar, McManus, Lester and Freemantle18
Among the other quality indicators that we assessed, we found much less evidence of change. In England we found small reductions in the proportion of people offered cognitive–behavioural therapy and family interventions and the proportion who had a care plan between 2013 and 2017. These changes took place at a time when there was rapid growth of psychological therapy services in England through the ‘Increasing Access to Psychological Therapies’ programme.Reference Clark19 It is possible that this may have drawn psychological therapists away from working with people with psychosis in secondary care mental health settings. However, we did not see these changes in Wales where there was also an expansion of psychological therapy services,Reference Champion20 and we cannot rule out the possibility that the time and other resources that were spent meeting CQUIN targets between 2013 and 2017 contributed to a reduction in the quality of non-incentivised aspects of care offered to people with psychosis in England.
Study strengths and limitations
One of the key strengths of this study is the large sample size with data collected from nearly 18 000 patients. The other main strength of the study was the inclusion of control data from Wales, where no financial incentive was offered to providers of secondary care mental health services. Wales was an appropriate control as both the patient population and healthcare system are very similar to that of England. Both countries aim to follow NICE guidelines for the management of patients with psychosis and both took part in this series of audits. Consequently, differences between the two systems are minimal, with the largest difference within this patient population being the financial incentive that was implemented only in England.
Concerns have been raised about the reliability of data gathered for assessing whether financial incentives should be paid.Reference Petersen, Woodard, Urech, Daw and Sookanan21 One of the strengths of this study was that we used an independent source of data, from the national audit programme, to examine the performance of services before and after the introduction of the CQUIN. As part of the audit, quality control checks were carried out on four randomly selected sites and did not find evidence that gaming had taken place.22
The main limitation of the study is that we relied on information about physical health that was recorded in clinical records. Although we were able to collect data from all health boards in Wales, the amount of data we collected was in proportion to the respective populations of these two countries and was therefore much smaller than the amount we collected in England. All data were collected from patients using the UK National Health Service. Although we see no reason why the results of this study could not be applied to other secondary care financial incentive schemes in the UK, it may not be possible to generalise the results to other countries that organise and reward secondary care services in different ways.
Implications for services and future research
These data provide evidence for the impact of financial incentives in healthcare policy on the incentivised target during the period that it is implemented. Consequently, using a similar financial incentive to improve physical healthcare of other patients with severe mental illness may also be effective. A key consideration for future policy is the effect of removing the financial incentive in 2019. A study of financial incentives for screening used by Kaiser Permanente found that removal of financial incentives led to a worsening of screening.Reference Lester, Schmittdiel, Selby, Fireman, Campbell and Lee23 Screening for diabetic retinopathy and cervical cancer worsened around 3% and 1.6% per year, respectively, following removal of their attached incentive. Furthermore, screening for the two conditions dropped to below baseline, worse than before the incentives were introduced.Reference Lester, Schmittdiel, Selby, Fireman, Campbell and Lee23 More recently reductions in the quality of care delivered by primary care services in England have also been noted following the removal of a financial incentive.Reference Minchin, Roland, Richardson, Rowark and Guthrie16
The aim of the CQUIN programme is to try to support a change in clinical practice that can be sustained once the financial incentive is withdrawn. There is some evidence that the CQUIN along with other efforts to raise the importance of physical healthcare for people with psychosis has led to changes in attitudes and practice among providers of secondary care mental health services.Reference Quirk, Chee, Patterson, Snowdon, Lemmey and Tooke24,Reference Nazarko25 Whether such changes continue to have an impact on the quality of physical healthcare that people with psychosis receive once the incentive is removed remains to be seen. The evidence base for the impact that screening and interventions for cardiovascular disease have on long-term morbidity and mortality is mixed. Available evidence suggests that the use of financial incentives in primary care did not lead to improvements in rates of mortality from targeted conditions.Reference Ryan, Krinsky, Kontopantelis and Doran26 Better evidence about the impact of offering interventions for these risk factors among people with psychosis is needed if clinicians are to continue to use their limited resources to deliver these interventions in the future.
Although concerns have been raised about the possibility of negative effects of financial incentives, this is the first time that comparative data from secondary care services on aspects of care that were not incentivised have been analysed. Although the pattern of differences in non-incentives aspects of care between England and Wales before and after the introduction of the CQUIN was not consistent, our observation that some aspects of care appear to have deteriorated in England during this period highlights the importance of assessing both positive and negative effects of financial incentives in future studies.
Supplementary material
Supplementary material is available online at https://doi.org/10.1192/bjp.2019.162.
Acknowledgements
Applications for access to data from the National Clinical Audit of Psychosis should be made to HQIP. Details of the process for obtaining these data are available at: https://www.hqip.org.uk/national-programmes/accessing-ncapop-data/#.W-7gS0ca7oo.
Funding
The National Audit of Psychosis (NCAP) is managed by the Royal College of Psychiatrists’ College Centre for Quality Improvement. It is commissioned by the Healthcare Quality Improvement Partnership (HQIP) as part of the National Clinical Audit and Patient Outcomes Programme. The views expressed in this publication are those of the authors and not necessarily those of the NHS, the National Institute for Health Research, or the Department of Health. The Imperial Biomedical Research Centre Facility, which is funded by the National Institute of Health Research also provided support that contributed to the completion of this paper. The funder of the study played no part in the preparation of this paper. The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the HQIP programme or the Department of Health.
eLetters
No eLetters have been published for this article.