Measures

Service content was measured using three instruments, the Camden Staff–Patient Activity Record (CaSPAR), the Camden Record of Inpatient Care Events (CaRICE) and the Camden Content of Care Questionnaire – Patient version (CCCQ–P), developed for this study. The instruments and their properties are fully described elsewhere. ^{Reference Lloyd-Evans, Slade, Osborn, Skinner and Johnson16} CaSPAR uses direct observation and staff report to measure the number and proportion of patients in contact with staff at predefined time points. CaRICE aggregates information provided daily by all staff to calculate the time in minutes of staff contact provided per patient. CCCQ–P is a patient-completed questionnaire about the types and frequency of care received during an admission. CaRICE and CCCQ–P categorise care into 21 types, from which subscale scores can be derived for provision of social, psychological and physical and pharmacological interventions. The categories of care used in CaRICE and CCCQ–P are listed in Table 1. Patient satisfaction was assessed using the Client Satisfaction Questionnaire (CSQ), an eight-item self-report measure. ^{Reference Attkisson and Zwick17}

Table 1

Content of care at three alternative and three standard services

	Score: mean (s.d.)
	Community alternatives	Standard services	Tidal Model ward	Tidal Model comparison service
Total care (21 categories) a
CaSPAR
Proportion of patients with staff, % (28 recordings per service)	12.3 (20.9)	11.8 (9.4)	10.8 (10.3)	8.8 (8.0)
CaRICE
Contact per patient per day, min (5 days' data per service)	149.6 (39.1)	135.9 (23.1)	109.6 (26.2)	82.6 (5.6)
CCCQ–P total care (range 0–147) (n = 217)	25.7 (16.2)	30.6 (15.5)	31.9 (15.3)	34.4 (18.9)
Social interventions (7 categories) b
CaRICE	60.7 (35.6)	43.8 (17.7)	43.3 (7.0)	32.6 (7.3)
CCCQ–P subscale (range 0–49) (n = 222)	7.4 (6.6)	7.9 (6.4)	8.7 (6.7)	7.6 (7.2)
Psychological interventions (5 categories) c
CaRICE	29.7 (24.6)	19.8 (5.7)	18.4 (10.5)	16.4 (2.2)
CCCQ–P subscale (range 0–35) (n = 221)	5.7 (6.3)	4.7 (5.8)	4.8 (5.5)	6.6 (6.9)
Physical and pharmacological interventions (6 categories) d
CaRICE	31.9 (30.0)	48.3 (16.6)	28.8 (15.4)	20.4 (6.4)
CCCQ–P subscale (range 0–42) (n = 220)	7.7 (5.8)	12.7 (5.8)	12.5 (3.9)	14.1 (5.7)

CaRICE, Camden Staff–Patient Activity Record; CCCQ–P, Camden Content of Care Questionnaire – Patient version

a. Categories as listed in subscales below plus assessment, care planning meetings, aftercare

b. Categories are housing, finances, legal issues, current activity, future activity, activities of daily living, family support

c. Categories are help with past events, current relationships, drugs and alcohol, symptom coping and illness education

d. Categories are medication review, concordance, practical help, physical healthcare, observations, restraint

Settings

Data were collected from four alternative and four local comparison standard in-patient services, a subset of those described by Johnson et al. ^{Reference Johnson, Lloyd-Evans, Morant, Gilburt, Shepherd and Slade14} The four alternatives were three community-based services – crisis team beds, a clinical crisis house and a non-clinical alternative (a crisis house for Black and minority ethnic service users) – and an in-patient ward using the Tidal Model, a nursing-led model designed to enhance collaboration and contact between in-patient staff and patients. ^{Reference Barker18}

Procedures

Twenty-three semi-structured interviews were conducted with stakeholders of the alternative services, including at each service the alternative service manager, representatives of the local standard in-patient service, the management of the local mental health trust and the commissioning agency, the manager from the local crisis and home treatment team, and a consultant psychiatrist from a community service referring patients to the alternative service. At the crisis team beds service, the alternative service manager was also the local crisis team manager, so one fewer interview was conducted than at other services. Interviews were recorded and transcribed verbatim.

Twenty-eight CaSPAR recordings were made by researchers at each service at preset time points (four per day) and including weekend and evening recordings. All staff were asked to complete CaRICE over a 5-day (Monday to Friday) recording period at each service, describing the interventions they had delivered during each working shift. Researchers were present during recording periods to distribute and collect forms. The CCCQ–P and CSQ were completed as a structured interview with patients at or close to their discharge from the service, describing the interventions they had received during the admission as a whole. Written informed consent was obtained from participants, following an initial approach from service staff. Researchers sought to recruit 40 participants at each service. All quantitative data were entered on an electronic database using SPSS for Windows, version 14.

Statistical analysis

Thematic content analysis was used to analyse data from stakeholder interviews, corresponding broadly to the procedures described by Coffey & Atkinson. ^{Reference Coffey and Atkinson19} A coding frame was developed collaboratively by two authors (B.L.E. and N.M.) and main themes explored using QSR Nvivo7 software (www.qsrinternational.com). The quantitative analyses of care provided at alternative and standard services compared the three community-based alternatives and their local comparison standard services. Data from the Tidal Model ward and its comparison service were excluded from main analyses. Its role was as a local standard acute ward. ^{Reference Johnson, Lloyd-Evans, Morant, Gilburt, Shepherd and Slade14} Stakeholders reported inadequate implementation of the model and that the resulting service did not offer an alternative to standard acute care. Descriptive data from the two excluded services are presented separately.

Linear regression was used to test whether community alternatives and standard services differed in intensity of staff–patient contact and types of care provided, measured by CaSPAR and CCCQ–P. The CaSPAR data were analysed adjusting for time and day of recordings and clustering by service. The CCCQ–P data were analysed adjusting for patient characteristics – Mental Health Act status (detained or voluntary), gender, age and ethnicity (Black, White, Asian or other) – and clustering by service. The limited power of data from CaRICE, which yields aggregate data for the types and intensity of care provided at a service as a whole, meant that clinically important differences between alternative and standard services were unlikely to achieve statistical significance. We therefore assessed not only whether differences between service types were statistically significant, but also the effect size (the difference in mean scores divided by the standard deviation of all data). Following standard advice on estimating effect size, our assumption was that a score of 0.2–0.5 indicates a small effect, 0.5–0.8 a medium effect and over 0.8 a large effect. ^{Reference Cohen20}

A model of patient satisfaction with residential acute care was explored using regression analyses. The relationship between patient satisfaction and service type was analysed, adjusting for patient characteristics and clustering by service. Additional adjustment for CCCQ–P variables was then made, to explore how care received relates to satisfaction and whether adjusting for types of care received affects the relationship between satisfaction and service type. The CCCQ–P data from all eight services were included in this model of satisfaction. The Tidal Model ward was included as a standard in-patient service in these explorations of the relationship between service type and patient satisfaction, so this part of the analysis involved comparing three community crisis houses with five acute in-patient wards.

Stakeholder interviews

A common perception was that alternative services provide more staff time and attention to patients than acute wards: 11 out of 23 interviewees made reference to this, including the service managers of all four alternatives. The Tidal Model was perceived by stakeholders as having been initially received positively by patients, increasing one-to-one staff contact with patients and facilitating collaborative agenda-setting in ward rounds. An erosion of its model fidelity and implementation was identified, however. Fatigue from the simultaneous implementation of too many different initiatives, lack of active management support, and resistance to change and to proactive patient engagement among nurses were identified as barriers to its sustained use, resulting in the model having minimal sustained impact on actual service delivery.

Stakeholders commenting on community alternatives did not identify explicit models of care being used. Types of intervention available were not felt to differ greatly from acute wards. Most commonly highlighted was the inability of community alternative services to provide 24 h medical support and lack of facilities to provide complex physical healthcare. In-patient service representatives suggested that greater service size and the skills of a professionally trained and eclectic staff group resulted in a greater range of care being available on acute wards. However, particularly for the clinical crisis house and crisis team beds, similarities in care to acute wards were emphasised as much as differences. Additional interventions were not consistently described as available in alternative services; rather, a distinctive style of care was proposed by stakeholders as characteristic of these services. Care was seen as typically more individual, collaborative and informal than on acute wards, resulting in patients experiencing greater choice and control during their stay. Excerpts from interview transcripts illustrating themes from analysis of stakeholder interviews are provided in the online supplement to this paper.

Quantitative comparison of care

Twenty-eight CaSPAR recordings were made at each service. The status (with staff or not) was identified for 99% of patients resident at recording times. Five days of CaRICE data were collected from each service. Completed forms were obtained from 871 (94.7%) of 919 eligible staff. Completed CCCQ–P and CSQ questionnaires were obtained from 314 (70.2%) of 447 patients who were approached to participate in the study. Missing data from CCCQ–P forms required the exclusion of up to 11 CCCQ–P responses from analyses.

Table 1 provides descriptive data from CaSPAR, CaRICE and CCCQ–P. Data from three community-based alternatives and from their comparison services are aggregated; results from the Tidal Model ward and its comparison service are presented separately. Scores for the Tidal Model ward were more similar to mean scores for standard in-patient wards than to mean scores for community alternatives on total care variables from all three measures and five out of six subscale variables. Results indicate that the intensity of contact and types of care provided at the Tidal Model ward at the time of recording were broadly similar to standard wards.

Statistical comparison of CaSPAR and CCCQ–P scores from community alternative services and standard wards is provided in Table 2, reporting results from regression analyses. No significant difference in intensity of staff–patient contact between alternative and standard services was identified by either measure. The CCCQ–P subscale data indicate significantly greater provision of psychological interventions and less provision of physical and pharmacological interventions at the alternative services, but no significant difference in social interventions.

Table 2

Regression analyses of CaSPAR and CCCQ–P data (adjustment for clustering by service undertaken in all analyses)

	R 2	Correlation coefficient (95% CI) a	P
CaSPAR
Total score
Model 1 b	< 0.01	–0.45 (–12.68 to 11.78)	0.93
Model 2 c	0.03	–0.45 (–12.98 to 12.03)	0.93
CCCQ–P
Total score (n = 217)
Model 1	0.02	4.85 (–2.70 to 12.41)	0.16
Model 2	0.06	3.31 (–2.27 to 8.90)	0.19
Social interventions subscale score (n = 222)
Model 1	< 0.01	0.49 (–1.00 to 1.97)	0.44
Model 2	0.03	0.02 (–2.09 to 2.14)	0.98
Psychological interventions subscale score (n = 221)
Model 1	0.01	–0.97 (–2.80 to 0.85)	0.23
Model 2	0.04	–1.33 (–2.48 to –0.18)	0.03 *
Physical and pharmacological interventions subscale score (n = 221)
Model 1	0.16	5.03 (–1.01 to 11.08)	0.09
Model 2	0.25	4.35 (0.75 to 7.96)	0.03 *

CaRICE, Camden Staff–Patient Activity Record; CCCQ–P, Camden Content of Care Questionnaire – Patient version

a. Negative correlation coefficient indicates higher score at alternative services

b. Model 1: relationship to service type (alternative v. standard service)

c. Model 2: relationship to service type, adjusting for patients' Mental Health Act status at admission, age, gender and ethnicity

^* P < 0.05

Analyses and estimates of effect size for community alternatives compared with standard services from CaRICE data are presented in Table 3. No significant difference in results was found between groups but the limited power of analyses using CaRICE data must be noted. Wide confidence intervals from t-tests reported in Table 3 indicate the possibility of clinically important differences in care between alternative and standard services. Examination of effect sizes suggests, consistent with CCCQ–P results, only a small difference between groups in intensity of care provided and a medium effect for alternative services of more psychological care and less physical and pharmacological care. A medium effect size was also found for more social care at alternative services: this indicates that the duration of social interventions may be greater than at standard wards; CCCQ–P data indicated that social interventions were not delivered more often or more varied in type.

Table 3

Patient contact: comparison of alternative and standard services

CaRICE domain	Contact per patient per day, min Mean difference a	95% CI	t	P	Estimate of effect size b
Total care score	13.7	–10.6 to 38.0	1.17	0.26	0.42 (small)
Social care	17.0	–4.4 to 38.3	1.66	0.11	0.59 (medium)
Psychological care	9.9	–3.9 to 23.7	1.52	0.15	0.54 (medium)
Physical and pharmacological care	–16.4	–34.8 to 2.0	–1.85	0.08	0.65 (medium)

CaRICE, Camden Record of Inpatient Events

a. Alternative service minus standard service

b. Mean difference/s.d.

Descriptive data from content of care measures from individual services are provided in online Table DS1. Despite some divergence of service scores across measures, marked features of services' care provision can be consistently identified. The non-clinical crisis house, which employed no medical or nursing staff, ranked lowest of all services on both CaRICE and CCCQ–P for physical and pharmacological interventions. The crisis team beds scored lowest of all services on CaRICE and CCCQ–P for social interventions but highly on both measures for psychological interventions. A lack of emphasis on social problems may be explained by the brief length of stay at the crisis team beds service, typically less than 1 week. The clinical crisis house, which provides a daily structured programme of activities within the residential unit, scored highest on CaSPAR total score and CaRICE and CCCQ–P for the current activity category.

Impact of care received on patient satisfaction

Results of a model of patient satisfaction using regression analysis including CSQ and CCCQ–P data are presented in Table 4. All CCCQ–P subscale scores were significantly positively associated with patient satisfaction, with more of each type of care linked to greater satisfaction. The R ² values indicate that variance in CSQ scores could be explained more by analysis including CCCQ–P total care score than any subscale score. The amount of care received by patients may therefore be more influential on their satisfaction with services than the broad types of care received. Patient satisfaction with community alternatives remained significantly greater than at standard wards after adjustment for all CCCQ–P variables, suggesting that differences in the amount of care or types of intervention provided might not be primary influences on patients' greater satisfaction with alternatives. The R ² values indicate that nearly three-quarters of the variance in patient satisfaction could not be explained by variables included in this model. The biggest change to the correlation coefficient for service type followed adjustment for CCCQ–P physical and pharmacological interventions: the increase in correlation coefficient following adjustment suggests that community alternatives are more acceptable than standard wards despite, not because of, providing less physical and pharmacological care.

Table 4

Relationship of service type, patient characteristics and care received to patient satisfaction: Client Satisfaction Questionnaire score is the dependent variable in all analyses ^a

Model: relationship to patient satisfaction	R 2	Correlation coefficient (95% CI)	P
Model 1
Service type, adjusting for patient characteristics (MHA status, gender, age, ethnicity) and clustering by service (n = 314)	0.13
Service type: b standard (reference category community alternative service)		–1.98 (–3.06 to –0.91)	<0.01
Model 2
Service type, adjusting for CCCQ–P social interventions score, patient characteristics and clustering by service (n = 306)	0.21
Service type b		–2.13 (–3.40 to –0.85)	0.01
CCCQ–P social interventions score		0.26 (0.20 to 0.32)	<0.01
Model 3
Service type, adjusting for CCCQ–P psychological interventions score, patient characteristics and clustering by service (n = 306)	0.23
Service type b		–2.01 (–3.14 to –0.88)	<0.01
CCCQ–P psychological interventions score		0.31 (0.21 to 0.40)	<0.01
Model 4
Service type, adjusting for CCCQ–P physical and pharmacological interventions score, patient characteristics and clustering by service (n = 307)	0.18
Service type b		–3.38 (–4.76 to –2.00)	<0.01
CCCQ–P physical and pharmacological interventions score		0.25 (0.13 to 0.37)	<0.01
Model 5
Service type, adjusting for CCCQ–P total score, patient characteristics and clustering by service (n = 303)	0.27
Service type b		–2.94 (–4.30 to –1.58)	<0.01
CCCQ–P total score		0.14 (0.10 to 0.18)	<0.01

CCCQ–P, Camden Content of Care Questionnaire – Patient version; CSQ, Client Satisfaction Questionnaire; MHA, Mental Health Act

a. Patients from eight services (three community alternatives and five hospital in-patient services): n = 314

b. Negative correlation coefficient for service type indicates higher score at alternative service

Limitations

This study has three main limitations. First, only a small number of patient characteristics could be adjusted for in comparisons of alternative and standard services. This limits understanding of how far differences in care reflect different needs and presentations of the patients admitted, or represent intrinsic differences in what services provide. Second, the study involves only one exemplar of each type of alternative. Evidence about care provision at different alternative service types is therefore preliminary. Third, limited depth of information is available about care at services. Although CaRICE and CCCQ–P measure 21 types of care, information about delivery of specific interventions, or the style or quality of care, is not provided and forms of interventions for which there is a clear evidence base will be grouped together with others for which the evidence is limited.

The measures of content of care developed for this study are methodologically innovative, overcoming a lack of existing measures appropriate for use in in-patient services. ^{Reference Lloyd-Evans, Slade and Johnson21} Further research has been proposed to build on the development and psychometric exploration of these measures that has already been undertaken, ^{Reference Lloyd-Evans, Slade, Osborn, Skinner and Johnson16} and to refine methods to investigate the care provided by in-patient mental health services. Given current knowledge about how to measure content of care in in-patient mental health services, the multiple method approach used in this study is appropriate. Triangulation of findings from more than one quantitative measure and qualitative data from patients and stakeholders can corroborate findings regarding similarities and differences in service provision at alternative and standard services. ^{Reference Gilburt, Slade, Rose, Lloyd-Evans, Johnson and Osborn22}

Clinical implications

There was little evidence that differences in types of care provided at community alternatives and standard services are of great clinical importance. The CCCQ–P data showed significantly greater provision of (broadly defined) psychological interventions by alternative services, but the CaRICE data indicated that this amounted to less than 10 min more per patient per day. Given the known poor physical health of mental health service users, ^{Reference Sokal, Messias, Dikerson, Keyenbuhl, Brown and Goldberg23} the provision of less physical healthcare in alternative services is of concern. Such services should ensure that their patients have proper access to community physical healthcare and screening that would be more routinely available in hospital settings. Since acute physical conditions may exacerbate or even cause mental health crises, alternative services also need mechanisms for ensuring patients receive a physical health check close to the point of admission. However, CCCQ–P item scores indicated that in all services except the non-clinical crisis house, patients typically received core psychiatric interventions such as medication prescription and provision. Lower levels of pharmacological interventions or safety measures such as observations might be appropriate for the client group at alternative services, who are less likely to be detained and more likely to be known to services, ^{Reference Johnson, Lloyd-Evans, Morant, Gilburt, Shepherd and Slade14} with a probable existing treatment plan. Stakeholders reported that shortfalls in pharmacological interventions at alternative services compared with acute wards might be mitigated by close collaboration with community services such as crisis teams. The conclusion that there is much similarity in care provision between alternative and standard wards is supported by stakeholder interviews and the lack of differences in service interventions identified by patients in qualitative interviews. ^{Reference Gilburt, Slade, Rose, Lloyd-Evans, Johnson and Osborn22} Although stakeholder interviews suggested potentially important differences from standard wards in the style of service provision at alternatives, service planners and referrers should not conclude that alternatives offer fundamentally different types of care from standard wards. They may conclude that alternatives can form part of mainstream acute care provision.

The model of satisfaction presented in this paper does not support wholesale change in the types of care provided by acute residential services, nor a highly critical appraisal of care provision on standard acute wards. All broad types of care were positively received by patients. Reduced provision of medical-type interventions at alternatives was not found to be related to their greater acceptability to patients. This study suggests that a focus on increasing the amount of contact and care provided to patients should be a priority for clinicians and managers, above changing the types of interventions available.

Neither the Tidal Model nor community crisis houses were found to address an expressed concern of patients, ^{Reference Baker6–8} and expectation of stakeholders, that the amount of staff contact available to patients needs to be increased. Interventions with a more specific focus and mechanism for increasing staff–patient contact may be required. Protected engagement time – when for set periods staff are relieved of administrative duties, the ward is closed to visitors and the office closed to staff – is one, as yet unevaluated, example of an attempt to achieve this. ²⁴ The daily activity programme at the clinical crisis house contributed to its high CaRICE and CCCQ–P item scores for current activity and top-ranking CaSPAR score for proportion of patients interacting with staff. Service managers should consider greater use of structured groups and recreational activities at services as a means to increase staff–patient contact.

Research implications

Nearly three-quarters of variance in patient satisfaction was unexplained by the model presented here. Stakeholders indicated that the nature of care may be more individual, consensual and informal at alternative services than on acute wards. This finding is corroborated by qualitative interviews with service users, who experienced less coercion and paternalistic care at alternative units, ^{Reference Gilburt, Slade, Rose, Lloyd-Evans, Johnson and Osborn22} and to an extent by Ward Atmosphere Scale data ^{Reference Osborn, Lloyd-Evans, Johnson, Gilburt, Byford and Leese25} which, before adjustment for patient variables, showed greater autonomy and support at alternative than standard services. The way in which things are done, rather than what is done, may be distinctive and important to patient experience at alternative services. Evaluation focusing on comparisons of service style or therapeutic alliance at alternative and standard wards may identify differences in care not measured by this study.

The lack of impact of the Tidal Model found in this study may be due to its inadequate implementation. Stakeholders from the Tidal Model ward recounted the difficulties in implementing innovation and changing culture on acute wards. A previous launch of the model within the trust had also been discontinued. Evaluations of the Tidal Model have typically involved newly implemented services and have been small-scale and of short duration. ^{Reference Stevenson, Barker and Fletcher26–Reference Lafferty and Davidson29} Research to define and measure fidelity to the Tidal Model, then evaluation of services where it is well established, would be useful.

Differences in care provision between the community alternatives in this study support the typology of alternative services developed from a national UK survey. ^{Reference Johnson, Gilburt, Lloyd-Evans, Osborn, Boardman and Leese13} Further investigation could establish whether there are consistent differences in service provision between types of alternative. This could inform the development and subsequent evaluation of models of acute residential care, assisting the goal of providing effective, acceptable services.