Within the National Health Service (NHS), support for people in mental health crisis is typically provided by multidisciplinary crisis resolution teams (CRTs), which aim to avoid in-patient admission by providing care at home via frequent visits. However, there is evidence that implementation of national guidelines for CRTs is highly variable, meaning that some people may not receive the intensity of support they need,Reference Lloyd-Evans, Osborn, Marston, Lamb, Ambler and Hunter1 with lack of therapeutic content and social contact frequently raised as issues.Reference Morant, Lloyd-Evans, Lamb, Fullarton, Brown and Paterson2 In addition, CRT care is often dependent on support from carers, which is problematic where there are no carers, and can lead to excessive burden even where there are carers.Reference Weich, Griffith, Commander, Bradby, Sashidharan and Pemberton3
Acute day units
Acute day units (ADUs), previously known as ‘day hospitals’,Reference Marshall, Crowther, Sledge, Rathbone and Soares-Weiser4 provide an additional clinical resource for those in mental health crisis. In England, there is no NHS-specified model, but ADUs typically offer on-site individual and group sessions during the day, with patients returning home overnight and at weekends. ADU care is provided by a multidisciplinary team, usually including nurses, therapists, psychiatrists and other mental health professionals.Reference Lamb, Davidson, Lloyd-Evans, Johnson, Heinkel and Steare5 As a result, patients are provided with structured days, more staff contact time and continuity than is available via CRTs, with opportunities for peer support from other clients, and a wider range of psychological, social and medical interventions. ADUs are often used concurrently with other services (e.g. CRTs, in-patient wards, crisis houses), with these services all referring to each other. Evidence of the effectiveness of ADUs is limited, with the most recent meta-analysis conducted in 2011.Reference Marshall, Crowther, Sledge, Rathbone and Soares-Weiser4 This review synthesised evidence from studies that compared ADUs with hospital admission and concluded that they provide a viable alternative to hospital admission for some, with similar effectiveness on readmission rates after discharge, employment, quality of life and treatment satisfaction, but quality of evidence overall was reported as low.Reference Marshall, Crowther, Sledge, Rathbone and Soares-Weiser4 Furthermore, to date there has been no comparison of outcomes and experiences of people using ADUs compared with those using CRTs, arguably a more directly comparable type of service.
The current study
We need evidence regarding any additional benefit ADUs may offer to patients, not only in terms of clinical outcomes, but also on patient-reported outcomes such as patient experience, well-being and quality of life. The aims of this study were to describe and compare the clinical and sociodemographic characteristics of people using ADUs and CRTs; and to compare outcomes in terms of readmission, well-being, depression and patient satisfaction for those who received ADU care with those who received only CRT care.
We hypothesised that people receiving ADU care would have fewer admissions, greater satisfaction and well-being, and less depression at 6 months, compared with those receiving CRT care alone.
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008. All procedures were approved by the London Bloomsbury Research Ethics Committee (reference number 16/LO/2160).
Design and setting
We established a quasi-experimental cohort study of people using ADU and CRT services and compared readmission to the acute care pathway during a 6-month period from baseline, as well as measures of depression, well-being and satisfaction with services. We recruited four NHS trusts with ADUs and CRTs in England. A brief characterisation of each NHS trust and ADU is available in Table 1. More detailed information about each of the NHS trusts is available in a set of case studies.Reference Morant, Davidson, Wackett, Lamb, Pinfold and Smith6 Participant recruitment took place between March 2017 and April 2019, with follow-up completed in September 2019.
NHS, National Health Service; ADU, acute day unit; CRT, crisis resolution team.
We invited people consecutively admitted to each service to participate in baseline interviews. Recruitment and data collection could occur at any point during the initial admission, and up to 14 days after discharge from the service. Telephone or online follow-up was carried out 8–12 weeks after baseline, with electronic health record (EHR) outcome data collected at 6 months after baseline.
Inclusion and exclusion criteria
Inclusion criteria were as follows: aged ≥18 years, used an ADU/CRT service for at least 1 week, read and understand English (or translator available), capacity to provide informed consent and did not pose too high a risk to others or themselves to participate (as judged by their current clinical team).
ADU participants could use CRTs concurrently or during the follow-up period; however, CRT participants were excluded if they used an ADU at any point during study period. This was to determine the benefits of ADUs as an addition to the acute care pathway, over and above CRTs.
Our main exposure was being under the care of an ADU (solely or in combination with CRT use) for at least 1 week before baseline, compared with CRT care only.
Our primary outcome was time to readmission for acute treatment, after discharge from CRT or ADU, during the 6-month study period (starting from the date of recruitment to the study). This was collected via service use data from EHRs. We defined readmission for acute treatment as any subsequent use of acute mental health services (CRT, crisis house, ADU or in-patient ward) after discharge from the service used at baseline during the subsequent 6-month study period.
Our secondary outcomes were self-reported satisfaction with mental health services (Client Satisfaction Questionnaire; CSQ),Reference Attkisson and Zwick7 well-being (Short Warwick–Edinburgh Mental Well-being Scale; SWEMWBS)Reference Haver, Akerjordet, Caputi, Furunes and Magee8 and depression (Center for Epidemiologic Studies Depression Scale; CES-D),Reference Andresen, Malmgren, Carter and Patrick9 collected via online questionnaire at baseline and 8–12 weeks later.
We collected demographic data and self-rated physical health via the baseline questionnaire. We collected data from EHRs on admissions and service use; clinical characteristics (ICD-10 diagnosis and any comorbid diagnoses, physical health diagnoses, substance misuse, smoking, medication, previous hospital admission); Health of the Nation Outcome Scales (HoNOS) scoresReference Wing, Beevor, Curtis, Park, Hadden and Burns10 and content of care (physical assessment carried out, carers involved in care, psychological input from service used at baseline). Where patients had multiple diagnoses recorded in their EHR, we recorded the diagnosis considered to be more severe. We recorded whether the person had a serious mental illness (SMI), typically defined as being diagnosed with schizophrenia, other non-organic psychoses or bipolar disorder.Reference Hardoon, Hayes, Blackburn, Petersen, Walters and Nazareth11
At baseline, ADU/CRT staff screened all people consecutively admitted to their service from the study start date. All those who met the inclusion criteria were approached by clinical or research staff and asked if they were willing to discuss participation further (except at NHS trusts where individuals had already given consent to be contacted directly about research projects: in this instance, researchers contacted people directly, once their eligibility and any risk-related safety requirements had been established from clinicians and patient records). Those who agreed to discuss the study were contacted by a researcher with an information sheet and an offer to answer any questions. Potential participants were given at least 24 hours to consider whether they would like to take part, and then if still interested, they provided written consent to a researcher, who also collected the baseline data. Consent and data collection could occur up to 14 days after discharge from initial service use.
Participants were offered £20 (vouchers) reimbursement for taking part (£10 for the baseline interview and £10 for the follow-up interview at 8–12 weeks after baseline). Participants were contacted by telephone by a researcher 8–12 weeks after baseline, to collect follow-up data. At 6 months after baseline, readmission data were collected from EHRs.
A priori, we estimated that a sample size of 400 patients per group (N = 800) would give 90% power to detect a difference of 12% in the proportions readmitted in each arm (with an assumption of 50% readmission in the CRT group), with alpha set to 0.05. It would also afford 90% power to detect an effect size difference of 0.3 on the CSQ. This calculation included inflation for clustering by NHS trust.
We calculated descriptive statistics comparing the baseline characteristics of people using ADUs versus those using CRTs for the sample as a whole, and within NHS trusts. We explored baseline differences in demographics, clinical data and content of care, using parametric and non-parametric tests as appropriate, as well as the proportion of people using ADUs versus CRTs who were admitted during the 6-month study period.
For our primary outcome, we compared time to readmission in ADU and CRT participants. We analysed this using Cox's regression to produce a hazard ratio. Cohort entry was the date of recruitment and cohort exit date was the date of readmission to acute care or the 6-month study end-point. We adjusted for variables chosen a priori, which previous research suggested may be relevant (trust, age, gender, SMI diagnosis, employment, baseline HoNOS score, baseline SWEMWBS score and whether the person had previously been an in-patient). Covariates were added using a stepwise procedure, starting with a bivariate model with NHS trust as a fixed effect, then adding the variables of age, gender and employment, followed by diagnosis, HoNOS score, SWEMWBS score and finally, whether the person had previously been an in-patient. Because of expected heterogeneity between different NHS trusts, we tested for an interaction between type of team (ADU/CRT) and NHS trust as a sensitivity analysis.
For the secondary outcomes, we analysed mean satisfaction, well-being and depression scores at weeks 8–12, using linear regression. Stata version 16 for Windows was used for all analyses.12
Figure 1 below shows the flow of participants into the study, and those followed up at 8–12 weeks. Only one participant declined consent for access to their EHR at the 6-month follow-up, giving a completion rate of >99% for the primary outcome of readmission.
Baseline sociodemographic and clinical characteristics
We recruited 743 participants, of whom 431 (57.9%) had received ADU care and 312 (42.1%) had only received CRT care. Because of the small number of Black and minority ethnic participants, we have used undesirably broad ethnic categories. The ADU and CRT groups were generally similar in terms of sociodemographic characteristics (full details available in Table 2). Those sociodemographic characteristics with statistically significant differences between the groups are outlined below.
ADU and CRT data are n (%) unless otherwise specified. P-values relate to t-tests for continuous variables and χ 2-tests for categorical variables.
ADU, acute day unit; CRT, crisis resolution team; SMI severe mental illness; CES-D, Center for Epidemiologic Studies Depression Scale; SWEMWBS, Short Warwick–Edinburgh Mental Well-being Scale; HoNOS, Health of the Nation Outcome Scales.
ADU participants were older than CRT participants, and less likely to be employed. Clinically, higher proportions of ADU participants were diagnosed with psychosis and personality disorders, and a higher proportion of CRT participants were diagnosed with anxiety disorders. A higher proportion of ADU participants had previously been admitted to a psychiatric in-patient ward.
In terms of content of care when using the ADU or CRT, a much larger proportion of ADU participants received a physical health assessment, had carers involved in their care and received psychological input, compared with CRT participants.
At baseline, CES-D scores were lower in ADU participants than in CRT participants, whereas SWEMWBS well-being scores were higher in ADU participants than CRT participants. The mean baseline total HoNOS score was higher for ADU participants than for CRT participants, indicating more severe difficulties for ADU participants.
There were slight differences in the length of index admission and time at which participants were recruited. ADU participants had a mean index admission lasting for 60 days (s.d. 38) before they were discharged, whereas CRT participants had a mean index admission of 43 days (s.d. 39) before discharge. The mean difference in length of index admission between ADU and CRT participants was −17 days, which was statistically significant (P < 0.0001, 95% CI −22.0 to −11.1). ADU participants were recruited at a mean of 24 days (s.d. 23) into their index admission (i.e. 24 days after being admitted to the service they were recruited from), and CRT participants were recruited a mean of 31 days (s.d. 24) into their index admission. The mean difference between ADU and CRT participants in time from index admission to recruited was 7 days, which was statistically significant (P = 0.0001, 95% CI 3.2–9.8).
Follow-up data were collected for >99% of participants for the primary outcome, and 76% of participants for the secondary outcomes. The only significant difference between those who did and did not complete the secondary outcome questionnaires was that completers were older (mean age 41.6, s.d. 13.7) than those lost to follow-up (median age 37.8, s.d. 13.0) (P = 0.001, 95% CI −6.2 to −1.5).
Primary outcome: readmission to acute care over 6-month follow-up period
For the primary outcome at the 6-month time point, 21.4% of ADU participants were readmitted, compared with 23.4% of CRT participants, with no statistically significant difference evident. The results from the Cox regression comparing time to admission in ADU versus CRT participants are shown in Table 2. Model 1 shows a bivariate analysis, including only team and NHS trust. Model 2 was adjusted for age, gender and employment, whereas model 3 included the remaining covariates (SMI diagnosis, whether participants had previously been an in-patient, baseline HoNOS score, baseline SWEMWBS score).
We included a cross-trust interaction between team (ADU or CRT) and NHS trust in model 4, which showed there were significant differences across NHS trusts in terms of the primary outcome of readmission (likelihood ratio test P < 0.001). Model 4 showed that within trust 1, the risk of readmission was statistically significantly lower for ADU participants than CRT participants. In trusts 2 and 4, there was no statistically significantly difference in risk of readmission. In trust 3, the risk of admission was statistically significantly higher for ADU participants than CRT participants. Full results are given in Table 3.
Data are hazard ratio (95% confidence interval), fully adjusted for all covariates.
ADU, acute day unit; CRT, crisis resolution team; NHS, National Health Service; SMI severe mental illness; HoNOS, Health of the Nation Outcome Scales; SWEMWBS, Short Warwick–Edinburgh Mental Well-being Scale.
Secondary outcomes: satisfaction, well-being and depression at 8–12 weeks
We found statistically significantly higher patient satisfaction (CSQ) scores and well-being scores (SWEMWBS), and lower levels of depression (CES-D) in ADU participants than CRT participants at the 8–12 week follow-up period. The full results the linear regressions examining these differences in fully adjusted models are shown in Table 4.
Data are coefficients from linear regression (95% confidence interval), fully adjusted for all covariates. Interpretation: CSQ and SWEMWBS are positively scored (higher score indicating better outcome), and the CES-D is negatively scored (lower score indicating better outcome).
CSQ, Client Satisfaction Questionnaire; SWEMWBS, Short Warwick–Edinburgh Mental Well-being Scale; CES-D, Center for Epidemiologic Studies Depression Scale; CRT, crisis resolution team; ADU, acute day unit; NHS, National Health Service; SMI severe mental illness; HoNOS, Health of the Nation Outcome Scales.
Summary of results
In a fully adjusted model, there was no significant difference between ADU and CRT participants in terms of readmission over 6 months. However, once an interaction effect was included in the analysis, significant differences between ADU and CRT participants became apparent, with ADU participants in different NHS Trusts being variously at increased, decreased or no different risk of readmission, compared with CRT participants. At 8–12 weeks, after accounting for baseline differences in participant characteristics, ADU participants had significantly higher satisfaction scores, with better well-being and lower depression scores than CRT participants. These results indicate that, despite serving a more unwell client group (as suggested by the lower proportion of ADU participants in employment, and higher proportions with SMI diagnoses and previous hospital admission), overall, ADUs produce comparable outcomes in terms of readmission, and better satisfaction, depression outcomes and well-being than CRTs.
Comparison with previous research
In terms of the primary outcome, readmission to acute services, we are not aware of previous research directly comparing ADU and CRT services. The most recent meta-analysis of research on ADUsReference Marshall, Crowther, Sledge, Rathbone and Soares-Weiser4 found no difference in subsequent readmission rates between people using ADU and in-patient services. However, this meta-analysis did find some evidence (albeit low quality) that those using ADUs had better outcomes than in-patients, regarding subsequent employment. This is particularly interesting given our finding that ADU participants in this cohort had lower rates of employment than CRT participants. One study investigated readmission of people using CRTs over 1 year,Reference Werbeloff, Chang, Broadbent, Hayes, Stewart and Osborn13 and found that psychotic disorders increased the risk of readmission. Their study found similar HoNOS scores in two CRT cohorts to our CRT participants (13.3 (s.d. 6.4) and 11.8 (s.d. 5.5)). The sample used in that study was similar in terms of gender (52% female), but had a smaller number of White participants (68%). Similarly, another study of people using CRTs found a readmission rate of 38%,Reference Johnson, Lamb, Marston, Osborn, Mason and Henderson14 which is comparable with our study. That study found similar CSQ scores in people using CRTs (mean score 26, s.d. 5) to our study, although the sample had a slightly different demographic profile (female 60% v. 51% in our study; White ethnicity 65% v. 83% in our study).Reference Johnson, Lamb, Marston, Osborn, Mason and Henderson14
We are not aware of previous research of people using ADUs and CRTs directly comparing the two groups on the secondary outcomes we used. Previous studies have compared outcomes for ADUs and in-patient wards, and satisfaction and readmissions among patients treated by CRTs. In a randomised controlled trial comparing day hospitals to in-patient care,Reference Priebe, Jones, Mccabe, Briscoe, Wright and Sleed15 satisfaction was measured and, similar to this study, found significantly higher satisfaction in the day hospital group than the in-patients group at discharge and 3 months later (although no difference at 12 months). The CSQ-8 has been used in several other studies of acute mental health services; for example, in a cohort study of people using standard acute in-patient wards compared with those using alternative residential crisis services (e.g. crisis houses), those using alternative services had significantly higher satisfaction scores (26.4 (s.d. 4.9) v. 23.1 (s.d. 6.6)).Reference Osborn, Lloyd-Evans, Johnson, Gilburt, Byford and Leese16 These scores are comparable with those found in our study (26.66 (s.d. 5.04) v. 24.37 (s.d. 6.57)), with greater satisfaction among those using ADUs and crisis houses than those using CRTs and in-patient wards.
We note that a previous study found similar interaction effects to our study, with different sites showing varying results.Reference Kallert, Priebe, McCabe, Kiejna, Rymaszewska and Nawka17 Although this study tested in-patient and day hospitals (rather than CRTs and ADUs), the finding suggests that, in terms of admissions at least, it is the content of care that matters, rather than the setting. The very wide confidence intervals evident in our study are probably because of a lack of statistical power, since the study was powered for an overall effect.
Strengths and limitations
There are four key strengths of this cohort study: the direct comparison of those using ADUs and CRTs, which has not been undertaken previously; the robust methodology (large sample size and adjustment for clinical and demographic differences at baseline); the range of geographical locations of participating services and high follow-up rates. As discussed above, there is a lack of evidence about ADUs in modern mental health settings, with the majority of work having been carried out some time ago, and on services that were substantially different from those available currently. This study provides a comprehensive overview of the sociodemographic and clinical characteristics of those using ADUs and CRTs, and offers insights into the comparative benefits of each type of service in terms of clinical and well-being outcomes. To our knowledge, this is the largest cohort study of people using ADUs to date, and methodologically rigorous, lending weight to these results. The participating services are located in areas serving urban, suburban and rural communities, with high and low deprivation levels. We were able to follow up on 99.99% of participants via their EHRs at the 6-month time point, which included our primary outcome of service use during the 6 months, and 76% of participants (ADU 75.4%, CRT 77.2%) at the 8–12 week time point, which included the secondary outcomes of satisfaction, well-being and depression.
There are four main limitations of the study: selection and attrition bias; differences in index admission and recruitment periods; power, and lack of randomisation of participants and EHR data quality. First, as is typical in cohort studies of this nature, we were reliant on people who were attending participating services being willing and able to take part. The nature and severity of illness of some individuals excluded them from the study – around a third of potential participants were excluded (from both service types) because of risk considerations – and as a result our findings may not be applicable to the most unwell people who use these services (which is a potential issue for any type of study). However, the sample recruited from both ADUs and CRTs included a variety of diagnoses, including people who had previously been admitted to in-patient care, and both groups had substantial use of mental health services at follow-up. Furthermore, many participants were single and unemployed, and diagnoses included psychosis and personality disorder. This indicates that our sample did include people with complex mental health needs. That said, the overall recruitment rate from those eligible for inclusion was 46% in ADUs and only 16% in CRTs. In part, this likely reflects the fact that face-to-face recruitment (possible in ADUs but not CRTs) tends to be easier than recruitment over the telephone (the only recruitment method used in CRTs). As a result, it may be that the CRT group in this sample is unrepresentative. However, the relatively high response rate at follow-up (76%), and lack of significant differences between those completing the follow-up questionnaire and those not completing, can give us confidence that attrition bias is not a concern.
Second, there were statistically significant differences between ADU and CRT participants in terms of the lengths of their index admission (with ADU participants on average being admitted for 17 days longer than CRT participants), and in terms of the time from index admission to recruitment (with CRT participants recruited an average of 7 days later in their index admission than ADU participants). The longer length of admission in ADUs could reflect the fact that this group appears to be more severely unwell, with higher rates of SMI and previous hospital admission, and lower rates of employment. Although there was a significant difference in time from index admission to recruitment, this was relatively small (7 days), and seems unlikely to have had a substantial effect on our primary or secondary outcomes.
Third, we recruited 744 people to the study, having originally aimed for a target of 800, which would have given 90% statistical power to detect a 12% difference in readmission between ADU and CRT participants. We found the readmission rate and the difference in admission rate were lower than accounted for in the sample size calculation. The difference between models 2 and 3 led to attenuation of the hazard ratio more in keeping with the presence of confounding than the loss of power when adjusting for the additional characteristics in model 3. The differences in satisfaction, well-being and depression, each favouring ADUs, were all statistically significant with convincing effect sizes.
The fact that participants were not randomised could be seen as a limitation; however, we were able to recruit a larger sample size than would have been likely in an RCT, and in the analysis we adjusted for a large number of baseline differences between the two groups, including previous history of admission.
Finally, we collected a large quantity of data from routinely entered EHRs, including our primary outcome, and as such were reliant on these records being accurate and up to date. We undertook a thorough cleaning process to correct obvious errors in this data, but this type of data is by nature not always of very high quality.
Implications for future research and practice
In future work it would be helpful to investigate why ADUs, despite being overwhelmingly popular with staff and patients, remain an underutilised model in the NHS acute care pathway. Work to produce a model of best practice, along with service implementation guidelines, would provide a valuable resource to commissioners and service managers looking to increase choice for people in mental health crisis in their areas. Research about the place of ADUs in the complex mental health landscape would be beneficial, including how NHS services work with voluntary sector provision in this area.
It would also be helpful to generate further evidence regarding the factors that make these models so hard to implement in a sustained way. Making comparisons internationally may help to determine whether they are more sustained in less financially constrained service systems. Given the lack of recent randomised controlled trial evidence about ADUs, and none at all comparing such units with other non-residential crisis services such as CRTs, a trial investigating different service models would be helpful to those planning, commissioning and running acute care pathways.
We have previously shown that the provision of ADUs in England is highly variable, with many parts of the country having no access to an ADU, and small numbers of people benefiting from them.Reference Lamb, Davidson, Lloyd-Evans, Johnson, Heinkel and Steare5 However, where such services are available, patients and staff of ADUs value the units highly in terms of providing daily structure and contact with others, and emotional, practical and peer support.Reference Morant, Davidson, Wackett, Lamb, Pinfold and Smith6 This cohort study provides evidence that people using ADUs had similar outcomes in terms of readmissions, and better outcomes for satisfaction, well-being and depression, compared with CRT participants, after adjusting for baseline differences and despite being a more severely unwell group. Those using ADUs were more likely to receive psychological input, a physical health check and have carers involved, which are all interventions recommended by professional bodies and patient and carer groups. The reasons for the better outcomes in ADUs are not clear. However, our research showed that people using ADUs were more likely to receive psychological input, physical healthcare and carer involvement. ADUs also offered more opportunities for peer support and longer duration of treatment. Future research could explore whether these ingredients could improve outcomes during or following episodes of CRT care.
The data that support the findings of this study are available from the corresponding author, D.L., upon reasonable request.
We are very grateful to all of the individuals who participated in our research: the staff, patients and carers of the ADUs and CRTs in our study sites. We are particularly grateful to our Patient and Public Involvement group members who are not authors, who provided valuable thoughts and comments on this work: Terry Harper, Hameed Khan and Jane Wackett. We are also grateful to the NHS Research and Development and National Institute for Health Research Clinical Research Networks (North Thames, West of England, Eastern and West Midlands), especially the staff who facilitated and undertook data collection (special thanks to Lisa Carmody, Rose Tierney, Emily Benson, Anthony Wood, Tim Banks, Natalie Stonebank, Clare Davey, Ali Hodges and Jo Morris).
D.L. ran the study, collected and analysed data, and wrote the first draft of this manuscript. T.S. collected data and provided comments on the manuscript. L.M. contributed to the data analysis and provided comments on the manuscript. A.C., S.J., J.B.K., B.L.-E., N.M., V.P. and S.W. contributed to the conception and design of the study, and provided comments on the manuscript. D.S. provided comments on the manuscript. D.O. conceived and designed the study, and provided comments on the manuscript.
This work was funded by the National Institute for Health Research Health Services and Delivery Research Programme (project number 15/24/17). The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the Health Services and Delivery Research Programme, National Institute for Health Research, NHS or the Department of Health.
Declaration of interest