The past 25 years have witnessed an explosion in mental health services research. A 1980 review of research in community mental health services (Reference Braun, Kochansky and ShapiroBraun et al, 1981) cited a dozen studies and concluded that there was little evidence that the newer services sustained people longer outside hospitals. Stein & Test's landmark study of assertive community treatment (ACT) was particularly influential because not only did it demonstrate reduced hospitalisation along with improved clinical outcomes (Reference Stein and TestStein & Test, 1980), but the accompanying paper indicated that it could achieve this without increased costs (Reference Weisbrod, Test and SteinWeisbrod et al, 1980). Not surprisingly these two findings stimulated an enormous interest in developing and evaluating such programmes. When Mueser et al (Reference Mueser, Bond and Drake1998) reviewed the area they were able to cite 75 good-quality studies, and when Catty et al (Reference Catty, Burns and Knapp2002) did the same 4 years later they had over 90 studies to draw on. The vast majority of these studies focused on those with severe mental illness and invariably the samples consisted mainly of people with schizophrenia – over 80% in the UK700 trial and the Department of Veterans Affairs study (Reference Burns, Creed and FahyBurns et al, 1999; Reference Rosenheck, Neale and LeafRosenheck et al, 1995).
HOSPITALISATION AS THE COMMON OUTCOME MEASURE
Not surprisingly in evolving mental health services there is a considerable range of outcome measures used; these reflect both the evolution of measures and the importance placed on these various outcomes. For some researchers symptom control might have been the goal, for some social functioning and community stability, for others quality of life or risk reduction. Hospitalisation is, however, overwhelmingly the most consistent outcome reported. Hospitalisation has been assumed to be a proxy for relapse in schizophrenia in the absence of a consensus on a clinically meaningful alternative measure. The use of pre-agreed changes in symptom scores, for example a 20% decrease or reduction to an agreed level of the Positive and Negative Syndrome Scale (PANSS) score, as in trials of antipsychotics, has not found favour with clinicians for service evaluations. They are rejected because of their sensitivity to prior levels of disturbance and insensitivity to key clinical features, such as self-neglect or hostility, that have a disproportionate influence on clinical management. As a consequence, hospitalisation has come to dominate randomised controlled trials (RCTs) of community interventions because it benefits from the assumed consistency of admission threshold in any local health system despite the known variation of these between systems.
Reporting of hospitalisation
Hospitalisation is generally reported in one of three forms in community studies.
Number of admissions
This is the simplest approach and consists of recording any psychiatric admission during the study period. The frequency of admissions is usually recorded during the follow-up period and outcomes reported in terms of admitted v. not admitted. This reporting has the advantage that it is immediately obvious to the reader, who may know little of the local circumstances or details about admission. If there are many patients with repeated admissions during the follow-up period then the mean number of admissions in the study categories may also be presented.
Time to admission
Time to readmission has been more used in relapse prevention studies than in community care studies. The difference between the timings of relapse in the experimental and control services are presented either as mean durations or, more usually, with survival curves (e.g. Kaplan–Meier).
Duration of in-patient care
The most common presentation of hospitalisation as outcome is by days of in-patient care within the agreed follow-up period. In schizophrenia trials hospitalisation data are rarely normally distributed and usually have a pronounced skew. The majority of patients usually have no admissions and a small number of patients account for most of the in-patient days. Such data are best presented as medians rather than means, but planners prefer means so that they can calculate bed needs. It is increasingly common to assess bed-days with parametric statistics, presenting means, after subjecting the non-parametric results to bootstrapping techniques (Reference Efron and TibshiraniEfron & Tibshirani, 1993). An advantage of duration of care is that it permits the pooling of hospitalisation data between studies with differing follow-up periods, because the durations can be recalculated as, for example, days per month or days per year.
ADVANTAGES OF HOSPITALISATION AS AN OUTCOME MEASURE
Face validity
The most obvious advantage of hospitalisation as an outcome measure is its face validity. All clinicians have a sense of when people with schizophrenia may need admission and what admission means for the patient, the family and the service. This understanding may, of course, be more illusory than real; the threshold for admission and the experience of admission may be very different in inner-city London and in a small town in Switzerland. However, a finding that an intervention halves admission rates or duration is immediately understandable and translatable to the clinician's practice.
Utility
An understanding of changes in bed occupancy has direct utility for service planning. Indeed, it has been the translation of this outcome into projections of bed occupancy that has driven much of the research in this area and had an impact on service developments. There has been concern that the utility of research in this area has been exaggerated, either through naivety or in the service of economic imperatives. Well-recognised factors that inflate the effectiveness of newly established services (Reference CoidCoid, 1994), such as charismatic leaders, the recruitment of exceptional staff and the slow accrual of complex and resistant patients, have been ignored, leading to overoptimistic bed reductions.
Health economic analyses
Because hospitalisation is such a disproportionately expensive component of mental health services – still responsible for 80% of costs in many services (Reference Leff, Trieman and KnappLeff et al, 2000) – careful recording of it is essential to any form of cost analysis. Mental health economic analyses require careful reading and careful interpretation. More than in any other branch of medicine the extent of the costing exercise is open to real debate – how much should housing and unemployment be included, how is informal care costed, etc.? Small, apparently unconnected, changes in living conditions can completely reverse the economic benefits of interventions (Reference McCrone, Beecham and KnappMcCrone et al, 1994). Where studies include hospitalisation as an outcome such complications are unlikely, but conclusions about comparative costs within services require attention to local conditions. The difference between the costs of an in-patient day and an outpatient contact with a professional are not fixed. For example, the difference between the cost of an in-patient day and a case manager contact was much greater in Stein & Test's study (Reference Weisbrod, Test and SteinWeisbrod et al, 1980) than in the UK700 study (Reference Byford, Fiander and TorgersonByford et al, 2000). Consequently how many case manager contacts would be paid for by a saved day in hospital would be very different in the two studies.
Despite these caveats, hospitalisation data are an essential component of health economic analysis and can make a powerful case for expanding or contracting different components in an integrated service. Careful costing of hospitalisation was responsible for dispelling the early myth that deinstitutionalisation was inevitably cheaper than hospital care and helped to identify levels of disability at which hospital care was cheaper overall (Reference Knapp, Beecham and AndersonKnapp et al, 1990; Reference Hallam, Beecham and KnappHallam et al, 1994).
DISADVANTAGES OF HOSPITALISATION AS AN OUTCOME MEASURE
‘Negative’ therapeutic goal
The most common criticism of hospitalisation as an outcome is a sense of its inadequacy in conveying normal and desirable clinical aims. Surely, its critics demand, there is more to psychiatry than simply keeping beds empty? Patient and family groups are often dismissive of a reduction in hospitalisation as evidence that services are more interested in an outcome relevant to themselves (i.e. reduced costs or administrative convenience) than to patients and families – improved well-being, quality of life and symptom control. This is a powerful criticism and not easily dismissed. Repeated attempts to contextualise hospitalisation as an outcome (explaining that it is a benchmark for clinical success, a proxy, rather than a direct measure) are necessary but often unsuccessful.
There are also disadvantages from a service development and delivery perspective of an exclusive focus on reduced hospitalisation. Sustaining mental health services relies on recruiting and retaining committed, high-quality staff, and for this the day-to-day business of care must be centred on the individual well-being of the patient directly in front of the staff member. Maintaining focus and motivation for the staff member and engaging the patient require a clear therapeutic goal that can be shared and realised in that interaction (e.g. reducing distress, improving understanding of the illness or treatment, ensuring adherence to medication). Reducing bed occupancy is not one such shared goal. Reframing this as ‘promoting stability’ or ‘improving community tenure’ goes some way to presenting it as a desirable positive goal, but statistical probabilities are weak motivators in human behaviour. Clinical experience emphasises the need to identify the clinical practices and the interpersonal and patient-centred outcomes that lead to a goal of reduced hospitalisation (Reference Wright, Catty and WattWright et al, 2004) and enshrine these in operational policies (Reference Burns and FirnBurns & Firn, 2002).
Research distraction
Another criticism of hospitalisation as an outcome measure is that it can distract from efforts to explore the mechanisms of schizophrenia care. This criticism certainly does have salience in service development research (Reference Burns, Creed and FahyBurns et al, 1999), where preoccupation with organisation has led to a relative neglect of the operative components (Reference Wright, Catty and WattWright et al, 2004), but it is probably unwarranted in the area of schizophrenia outcomes. Current research in schizophrenia care demonstrates attention to a wide range of specified interventions, both pharmacological and psychosocial, and a wide range of outcome measures.
HOSPITALISATION AND RELAPSE
Independent assessment of relapse
Hospitalisation owes its current status as a research outcome principally to its assumed equivalence with relapse. Two recent developments question this legitimacy. First, the increased thresholds for admission in hard-pressed services or in tightly managed services may require a specific degree of severity of relapse for admission. Although agreed definitions of relapse in pharmaceutical trials have been long established based on agreed changes (either absolute or percentage point changes) in symptom ratings (such as PANSS score) they have rarely been used outside drug trials. Where there are regular ongoing assessments of clinical status as part of a study it could be possible to identify relapse independent of hospitalisation. Several ongoing naturalistic and observational studies, such as the Schizophrenia Care and Assessment Programme – UK (SCAP–UK; Reference Burns, Christova and CooperBurns et al, 2006), have attempted to construct relapse criteria from symptom changes, drug prescribing changes and changes in contact frequency. To date none of these attempts has been replicated in published studies.
The Lambeth Early Onset study of early intervention in psychosis has reported a reliable method for estimating relapse from regular systematised assessments of case notes (Reference Bebbington, Craig and GaretyBebbington et al, 2006). The assessed relapses were strongly correlated with independently assessed PANSS scores. Whether such an approach will erode the status of hospitalisation as an outcome measure is as yet unclear. A series of studies using such instruments might provide a guide to the relationship between relapse rates and hospitalisation rates in schizophrenia that can then be used to scale up the inevitably conservative hospitalisation rates.
Crisis resolution/home treatment studies
Unlike research into case management or assertive outreach, studies of crisis resolution/home treatment teams also use hospitalisation as a primary outcome measure but without the assumption that a change reflects a change in relapse rate. The clinical rationale of assertive outreach is that improved continuity of care leads to better clinical management and reduced relapse (Reference Stein and TestStein & Test, 1980) and that reduced hospitalisation is a reflection of this (Reference Marshall and LockwoodMarshall & Lockwood, 1998). In studies of crisis resolution/home treatment teams, however, the intervention comprises a different style of managing relapses, not preventing them (Reference Johnson, Nolan and PillingJohnson et al, 2005; Reference Glover, Arts and BabuGlover et al, 2006; Reference Killaspy, Bebbington and BlizardKillaspy et al, 2006). Thus a reduction in hospitalisation is a marker for more effective management of relapse (i.e. successful care in the home) not a marker for reduced relapse. The relationship between hospitalisation and relapse in these two different types of studies needs to be recognised for their interpretation.
MISINTERPRETATION OF HOSPITALISATION AS AN OUTCOME
Hospitalisation as an outcome measure in community studies draws its legitimacy in RCTs from the highly plausible assumption that the threshold applied in any local area will affect experimental and control groups equally. Thus any differences in hospitalisation rates can be attributed to differences between the two interventions. The dangers of extrapolating directly from model services, with their highly motivated staff, exclusion criteria and invisible incentives, have been well documented (Reference BachrachBachrach, 1989; Reference Tyrer, Coid and SimmondsTyrer et al, 1999) although the lesson is consistently ignored. Within an individual trial, however, difference in hospitalisation is generally a reliable guide to anticipated impacts. The wider generalisability of hospitalisation is a highly complex matter and failure to give it due consideration has led to significant mistakes in policy and planning.
Can hospitalisation rates be used in meta-analyses?
Meta-analyses of medical trials consolidate the outcomes from several small trials into a single result for that outcome, treating all the data as if from a single trial. The benefits of this approach, and the world-wide Cochrane Collaboration that supports it, is that conclusions can be established earlier (thereby introducing life-saving treatments and also avoiding unnecessary subsequent trials) and with greater confidence. The delay in introducing clot-busting drugs after myocardial infarction is often cited as the most convincing case for meta-analysis (Reference Antman, Lau and KupelnickAntman et al, 1992). The importance of meta-analyses has been emphasised for mental health research because of the preponderance of small, underpowered studies (Reference CoidCoid, 1994). Within the Cochrane Collaboration, difference in hospitalisation rates has been the most influential outcome in meta-analyses of community mental health services (Reference Marshall and LockwoodMarshall & Lockwood, 1998; Reference Marshall, Gray and LockwoodMarshall et al, 2001) although others are reported (e.g. loss to follow-up care, satisfaction with care, cost of care). Clinical and social functioning are often too inconsistently collected for influential findings to be presented.
The meta-analyses of hospitalisation for ACT teams (Reference Marshall and LockwoodMarshall & Lockwood, 1998) and case management (Reference Marshall, Gray and LockwoodMarshall et al, 2001) have been consistently cited to confirm that ACT reduces the need for hospitalisation compared with standard care. As a consequence, ACT has been mandated in many US and Australian states, Canadian provinces and increasingly across Europe. In the UK ACT teams are the basis for the reorganisation of mental health services required by the NHS Plan (Department of Health, 2000), with the establishment of over 170 teams. Close examination of the forest plots indicates that there is quite a lot of heterogeneity in the results. Some caution should therefore be exercised in applying meta-analytical techniques to hospitalisation outcomes and efforts should be made to understand the source of the heterogeneity.
Two potential sources of heterogeneity are immediately clear from a cursory examination of the forest plots. First, in the ACT meta-analysis the studies demonstrating major reductions are all from the USA, and the only non-American study included (Reference Muijen, Marks and ConnollyMuijen et al, 1992) demonstrates minimal reduction. In the case management analysis three of the studies are from the UK. This difference might indicate an impact of differing healthcare systems on the results of these two meta-analyses. There is also a suggestion that later studies indicate less benefit for ACT, although the difference is not as pronounced as that for the geographical differences. The importance of these observations becomes clear with the failure of any recent, high-quality European studies of ACT to replicate the reduction in hospitalisation. Indeed several recent European studies have been sufficiently powered that their failure to demonstrate reduction in hospitalisation can be interpreted as confirmation that there is no reduction. Hospitalisation is therefore not a reliable outcome in meta-analyses. Variation in hospitalisation as an outcome, on the other hand, has proved to be most useful by leading analyses that produce better understanding in service evaluations.
Control services are not placebos
Examination of the differences between US and European (predominantly UK) community care studies confirmed that the impression that US studies were more successful in reducing hospitalisation is indeed the case (Reference Burns, Catty and WattBurns et al, 2002). This holds despite evidence that the interventions were substantially similar (Reference Fiander, Burns and McHugoFiander et al, 2003). Home-based care in the US (the definition was widened to ensure consistency and to avoid post hoc rationalisation in labelling) did reduce in-patient care by a statistically significant mean of about 10 days a year compared with standard care, whereas in European studies it increased in-patient care by a non-significant average of 3 days a year. However, the conclusion that US experimental services kept patients out of hospital more is not supported. Mean days in hospital were essentially the same for experimental service patients in the US and Europe (19 and 21 days respectively); the differences stem from the differences in hospitalisation for the control services (means of 28 and 17 days respectively).
This exploration of variation in hospitalisation data confirms our earlier call for community psychiatry studies to pay much greater attention to service characterisation and, in particular, characterisation of the control services (Reference Burns and PriebeBurns & Priebe, 1996). Hospitalisation as an outcome measure certainly has some generalisability, but its limitations need to be considered when it is used as a basis for service planning.
Distinguishing effective ingredients
An important consequence of the heterogeneity of hospitalisation as an outcome is that it has stimulated a search for the sources of that heterogeneity and this has helped distinguish effective from more redundant components in complex interventions. In the systematic review of home-based care by Catty et al (Reference Catty, Burns and Knapp2002) we obtained data from the 60 of the 90 researchers to characterise their experimental services at the time of the investigations. The information was collected using 20 operationalised ‘components of care’, which were subjected both to cluster analysis to identify common characteristics of practice and to regression against reduction in hospitalisation to identify whether any were more strongly associated. Figure 1 shows the six regularly occurring components reported. The two found in a regression analysis to be significantly associated with reduction in hospitalisation are home visiting and joint health and social care. This is only a post hoc analysis and the sample was quite restricted. However, what it does do is indicate how hospitalisation as an outcome can be used to explore community mental health services in greater depth.
A subsequent study has demonstrated even more convincingly the utility of hospitalisation as an outcome measure to exploit differences in trials (Reference Burns, Marshall and CattyBurns et al, 2005). Meta-regression analysis allows skewed, non-parametric data to be used in a meta-analysis (Reference ThompsonThompson, 2001) and allows for multi-site studies to be analysed as, effectively, several independent studies. Substantially the same set of studies as those used in the home-based care review were used to obtain patient-level data. Hospitalisation data were assessed using an accepted model fidelity scale (Reference McGrew, Bond and DietzenMcGrew et al, 1994). This approach demonstrated that baseline bed use was the factor most strongly associated with reduction, but of the model fidelity factors it was predominantly the structural rather than staffing characteristics of the intensive approach which accounted for the outcome differences.
CONCLUSIONS
Hospitalisation as an outcome in schizophrenia research is likely to retain an important place mainly because of its obvious utility to planners and service providers. It also has a powerful advantage in its face validity to clinicians. Its limitations are obvious – it says little about individual patient outcomes and can convey a sense of being more interested in services (in particular their costs) and the professionals that staff them than in patient welfare. A closer examination demonstrates that reducing unnecessary hospitalisation has parallelled patient and carer wishes. Survey after survey has reported the desire to remain out of hospital as much as possible (Reference Drake and WallachDrake & Wallach, 1988). Given the choice, patients almost invariably opt for out-patient and community care or, failing that, day care.
Reducing unnecessary hospitalisation has also, arguably, increased the overall efficiency of mental healthcare. The disproportionate cost of in-patient care per patient contact (which is, after all, where the treatment occurs) reflects the capital costs, hotel costs and 24-hour staffing. In-patient care has declined for most physical disorders as the population increasingly has clean, well-heated accommodation affording adequate privacy. These extra costs of hospital care are justified when they add to safety or ensure adherence. However, for many patients it is not necessary and there is no clear evidence that treatments are any more effective for being delivered in hospitals than in clinics or patients' homes. Indeed, the difficulty of ‘transfer of learning’ from hospital to home is one of the underlying reasons for Stein & Test's emphasis on what they call ‘in vivo’ care in assertive outreach (Reference Stein and TestStein & Test, 1980).
Reducing hospitalisation is also in line with most current thinking in bioethics, where the emphasis has been on the provision of mental healthcare in the ‘least restrictive’ environment (Reference LinLin, 2003). Much of this ethical debate has centred around the care of legally detained patients. However, there is accumulating evidence of informal coercion in mental healthcare (Reference Monahan, Redlich and SwansonMonahan et al, 2005), suggesting that the distinction between voluntary and involuntary may be better conceptualised as a gradient rather than a dichotomy (Reference Bonnie and MonahanBonnie & Monahan, 2005). Patient and ethical views about legally enforced admission may, in some measure, also apply to most admissions.
The utility and apparent simplicity of hospitalisation as an outcome measure should not, however, blind us to its limitations. It is a good proxy for relapse in schizophrenia in well-functioning and coordinated services. However, it is a social sciences outcome that is not independent of context and it needs to be interpreted that way. Its reputation has been somewhat tarnished by overextrapolation; there is a need for greater caution in its interpretation to ensure its reputation is rehabilitated.
Consistent care components of home-based care. From Wright et al (Reference Wright, Catty and Watt2004). Reprinted with permission.
eLetters
No eLetters have been published for this article.