Participants

Potential participants for the patient experience survey were initially identified and approached by staff on each unit. Every person currently receiving in-patient treatment was eligible for inclusion. Staff informed researchers which service users were well enough and willing to discuss the study. We aimed to recruit 40 participants from each of the eight services, close to their point of discharge. Each participant gave informed consent to complete the questionnaires and all were reimbursed for their time.

Measures

Service user satisfaction was assessed with two established user-rated measures. The Client Satisfaction Questionnaire (CSQ) is an eight-item questionnaire measuring patient satisfaction with a mental health service. ^{Reference Attkisson and Zwick8} Scores range from 8 to 32, with higher scores indicating higher satisfaction. The Service Satisfaction Scale – Residential Form (SSS–Res) is a more detailed, 33-item scale measuring aspects of patient satisfaction with residential and in-patient mental health services. ^{Reference Greenfield and Attkisson9} The total SSS–Res score ranges from 33 to 165. The scale produces a total score and four subscales, with ratings for the staff programme (15–75), medication and aftercare (15–55), day and night availability (3–15) and facilities (4–20). Higher scores indicate higher satisfaction.

To assess service user experience we applied the user-rated Admission Experience Survey (AES), a 16-item measure that derives subscales related to perceived coercion at admission (0–5), having a ‘voice’ in decisions (0–3) and negative pressures (0–6). ^{Reference Gardner, Hoge, Bennett, Roth, Lidz and Monahan10} It is designed for assessing the process of admission to hospital or residential acute care and was developed for the MacArthur Coercion Study. Finally, the Ward Atmosphere Scale – Short Form (WAS) is a 40-item user-rated measure of service style, producing ten subscales (0–4), including a unit's practical orientation, levels of anger and aggression, levels of patient autonomy, staff control and support. ^{Reference Moos11}

We also recorded age at admission, gender, ethnicity and Mental Health Act status (involuntary or voluntary) during admission, and length of time between admission and completing the questionnaire.

Missing data

When questionnaires were incompletely filled, missing items were pro-rated with the mean of the other scores, up to a predetermined maximum number of replacements (CSQ up to two items, SSS–Res six items, AES subscales one item, WAS subscales one item). Measures with more than the maximum number of missing items were excluded.

Statistical analysis

Data were entered using SPSS version 14.0 for Windows, and then converted to Stata version 10 for further analysis. Satisfaction and experience outcomes were initially compared between each local pair of services and then aggregated according to alternative or traditional service. Data were inspected for normality of distribution and compared using t-tests and chi-squared tests if appropriate. Where distributions deviated from normal, non-parametric tests were employed. Potential confounding variables were also explored with univariate tests. Multiple linear regression was then used to explore associations between alternatives and satisfaction/experience outcomes. All regression models were adjusted for clustering by service, using the Stata cluster option. The proportion of variance in outcomes explained by each multivariate model (R ² statistic) was noted. When outcome variables were not normally distributed, a sensitivity analysis was performed, using ordinal logistic regression to explore whether significant associations were stable irrespective of method of regression analysis.

Strengths of the study

This is the first national in-depth study of patient experiences within residential alternatives to in-patient psychiatric care. Each service was chosen to represent one of the main types of alternative that had been identified in a previous national survey of all identifiable novel residential mental health services. ^{Reference Johnson, Gilburt, Lloyd-Evans, Boardman, Leese and Osborn5}

The greater levels of satisfaction within residential alternatives, especially in relation to the CSQ, were generally robust whichever form of analysis was used to explore them, including different statistical methods and adjusting for different possible explanatory variables such as geographical area and duration of admission. This difference in satisfaction is an important service user outcome, given that a comparison of clinical outcomes revealed no clear-cut clinical or economic advantage for either type of service. ^{Reference Slade, Byford, Barrett, Lloyd-Evans, Gilburt and Osborn13}

The variability in our quantitative experience findings is consistent with results from qualitative interviews undertaken with people admitted to the same services. Gilburt et al found that although many people preferred admission to residential alternatives, they also identified a number of similarities between the two types of service, including the type of care that they received during their stay. ^{Reference Gilburt, Slade, Rose, Lloyd-Evans, Johnson and Osborn14} However, there are some consistent themes between the two methodological approaches that add weight to our findings. In particular, the qualitative work also confirmed that patients favoured alternative services in terms of their atmosphere, with lower levels of disturbance and less experience of negative coercion. ^{Reference Gilburt, Slade, Rose, Lloyd-Evans, Johnson and Osborn14}

In a separate paper we compared the content of care provided in different service types, ^{Reference Lloyd-Evans, Johnson, Morant, Gilburt, Osborn and Jagielska12} and once again there were similarities between alternative residential services and traditional units, such as access to activities and the quantity of time spent with staff. This is consistent with our findings from the Ward Atmosphere Scale, where after adjustment, the only distinguishing features of alternatives were lower levels of anger and aggression.

Service user satisfaction seems to be one key difference between the services, yet it is not simply explained by the type of care received. ^{Reference Lloyd-Evans, Johnson, Morant, Gilburt, Osborn and Jagielska12} Our findings suggest this satisfaction may hinge on feeling less coerced at admission and having more influence over the type of care received. In conjunction with the findings from Gilburt et al, ^{Reference Gilburt, Slade, Rose, Lloyd-Evans, Johnson and Osborn14} this points to a future research agenda which is more focused on interpersonal and relationships variables rather than concentrating exclusively on interventions and outcomes.

Limitations

This study involved only one exemplar service from four of eight types of residential alternative, so the generalisability of our results to other services cannot be guaranteed. Our observational study was not based on random sampling or on random allocation to services, and therefore sampling bias and residual confounding are important considerations. It is possible that participants in the two types of services were not fully representative of all those in the services. Although we adjusted for demographic variables, including area and length of stay, we did not have further data to explore how differences in clinical presentation in the services (such as severity of illness) might account for the increased satisfaction in alternative services. However, detention under the Mental Health Act is one proxy for severity and although it attenuated some results, satisfaction remained greater in alternatives, with less negative experiences and perception of coercion. Johnson et al reported that although alternative services do cater for similar groups of patients to traditional in-patient units, they admit fewer patients with a diagnosis of psychosis and fewer patients who may pose a risk of violence to others. ^{Reference Johnson, Lloyd-Evans, Morant, Gilburt, Shepherd and Slade7} We were unable to control for these variables in this part of the study and it is possible that they explain some of the differences in admission experiences and satisfaction that we report. Similarly, higher levels of anger and aggression have previously been associated with lower levels of satisfaction, ^{Reference Svensson and Hansson15} as have perceived levels of coercion. ^{Reference Iversen, H⊘yer and Sexton16} Since anger and aggression were rated as more prevalent in our traditional in-patient units, this may explain the lower levels of satisfaction we observed in such settings.

We now require further understanding of the causes of satisfaction among in-patient service users. Our final model explained less than 20% of the variance in satisfaction, hence future research may require more in-depth explorations of factors present at the point of admission and then throughout each in-patient stay. Our participants completed their satisfaction ratings near to discharge and this might explain the relatively high levels of satisfaction across all units. Future research might assess satisfaction at multiple time-points during admission, and assess predictors of any changes in satisfaction within individuals, perhaps in relation to quality of care, clinical progression and recovery as well as environmental influences within the unit (such as the quality and nature of relationships with staff, other service users and agencies).

Satisfaction at discharge may be an important clinical outcome with potential impact far beyond the experience of an individual admission. It may shape individuals' perception of their care as a whole and their willingness to engage with a range of mental health services in the future. Since the residential alternative units in this study were not inferior in terms of costs and clinical outcomes in either the short term or the medium term, ^{Reference Slade, Byford, Barrett, Lloyd-Evans, Gilburt and Osborn13,Reference Byford, Sharac, Lloyd-Evans, Gilburt, Osborn and Leese17} greater satisfaction levels are a compelling reason to support further development of such services as one component of acute adult mental health services.