A fundamental process in community care is the construction of a care plan based upon assessment of an individual's ‘needs’ (Reference Brewin, Wing and MangenBrewin et al, 1987). Normally needs are assessed by an informal process, but for research purposes standardised methods have been developed that collect data systematically using formal interview schedules (Reference Brewin, Wing and MangenBrewin et al, 1987; Reference Marshall, Hogg and GathMarshall et al, 1995; Reference Phelan, Slade and ThornicroftPhelan et al, 1995) or structured self-reports (Reference Van Os, Altamura and Bobesvan Os et al, 2002). Surveys using these standardised methods have found surprisingly high levels of unmet need in representative populations of people with mental illness from across Europe, suggesting that many needs are overlooked by informal needs assessment (Reference Murray, Walker and MitchellMurray et al, 1996; Reference Slade, Phelan and ThornicroftSlade et al, 1998; Reference Salize, Kuestner and Torres-GonzalezSalize et al, 1999; Reference Middelboe, Mackeprang and HanssonMiddelboe et al, 2001). These findings have led to the proposal that routine care planning should be based on a standardised assessment of need (Reference Slade, Beck and BindmanSlade et al, 1999; Reference Lasalvia, Stefani and RuggeriLasalvia et al, 2000) - a proposition supported by the findings of improved patient outcome in two uncontrolled trials (Reference O'Leary and WebbO'Leary & Webb, 1996; Reference Lockwood and MarshallLockwood & Marshall, 1999). This trial is the first randomised evaluation of introducing standardised needs assessment into the care planning process as called for by a recent systematic review of the area (Reference Gilbody, House and SheldonGilbody et al, 2003).
METHOD
Setting and patient selection
Three National Health Service trusts in the north-west of England participated in the study (combined catchment population 640 000). Each trust was based in an urban area with deprivation levels above the national average. Recruitment was from October 1998 to October 1999, with follow-up at 12 months. Eligible subjects were patients who were being cared for in the community under the UK care programme approach and who met Goldman's criteria for severe mental disorder. Goldman's criteria specify that the patient should have: a diagnosis of schizophrenia, schizoaffective disorder, bipolar disorder, major depression or delusional disorder; a duration of 1 year since onset; and disability sufficiently severe to cause serious impairment of functioning in family responsibilities, occupation or accommodation (Reference Goldman, Gattozzi and TaubeGoldman et al, 1981). Patients were identified by screening the case-loads of all care coordinators from the participating trusts. A sample of six patients per care coordinator was then randomly selected (or, if fewer than six were available, a smaller even number was selected). Participating patients received a standardised assessment of need at baseline from a research psychiatric nurse, using the Cardinal Needs Schedule (Reference Marshall, Hogg and GathMarshall et al, 1995).
Study design
A potential confounding factor in this study was that the experience of receiving feedback from a needs assessment might, in itself, alter a care coordinator's practice, irrespective of the content of the feedback. Such an effect could plausibly result from improved assessment practices, better communication with patients or greater awareness of available interventions. ‘Contamination’ of this kind would not be undesirable because it would imply that feedback on a few patients could improve care for a much larger number of patients. It was therefore necessary for the study to incorporate both group and individual randomisation to clarify the contribution of the experience of feedback from the contents of feedback (see Fig. 1).
Patient flow diagram.
In the group randomisation, care co-ordinators were allocated to one of three arms: in arm 1 they received feedback from the standardised needs assessment on all of their participating patients; in arm 2 they received feedback on half of their participating patients, who were individually randomised (within care coordinators) to feedback or no feedback, and in arm 3 they received no feedback.
Randomisation
Randomisation of care coordinators and individual patients was by numbered sealed envelopes, using a computer-generated random allocation sequence. C.R. provided the allocation sequence and A.L. opened the envelopes and assigned care coordinators on the basis of a trial number supplied by the research nurses. Care coordinators were randomised by blocks (to ensure an equal balance across sites) and randomisation was stratified by professional status (nurse or non-nurse).
Assessment of need
The information required for the assessment of need was collected by a research psychiatric nurse who interviewed the patients, their main carer and the care coordinator. Patients and carers were interviewed at home where possible. Data from these interviews were entered on a computer program that identified problems across 16 areas of psychiatric and social functioning. The program then determined which of these problems required action (known as ‘cardinal problems’) using three criteria: the ‘cooperation criterion’, which takes account of the patient's view of the problem; the ‘carer stress criterion’, which takes account of the carer's view of the problem; and the ‘severity criterion’, which takes account of the nature and severity of the problem. When the program identified a cardinal problem, the research nurse and a consultant psychiatrist considered a comprehensive list of defined interventions for that problem and decided whether the patient had received, or was likely to benefit from, any of these interventions. A need was recorded when a patient had a cardinal problem for which there was at least one suitable intervention that had not been offered in the past year.
The intervention: needs feedback
Within 2 weeks of the baseline needs assessment the research nurse met with care coordinators in arms 1 and 2. The nurse gave a detailed explanation of the process of systematic needs assessment and then provided a report generated by the needs assessment program for each of the care coordinator's patients who had been randomised to receive feedback. For each patient this report explained: exactly why a problem had been identified in an area of functioning; why the problem was considered worth acting on; what type of intervention was required; and how the intervention might be obtained, by reference to a database of local services. This extensive database contained details of interventions provided by local voluntary and statutory organisations and was indexed by type of intervention. When a need for an intervention was identified (e.g. help with literacy), the database would provide information on how to obtain that intervention from a suitable local agency. The research nurse and care coordinator then discussed the report. Subsequently, the care coordinator contacted the patient to discuss the report and to formulate a care plan for obtaining any interventions that they and the patient agreed were required. The research nurse telephoned the care coordinator at 3 and 6 months to discuss progress in implementing the care plan and the care coordinator was invited to contact the nurse for clarification as required. All feedback sessions were recorded and a sample was rated for quality of feedback.
Control: informal needs assessment under the care programme approach
The care programme approach specifies that every patient has a care plan based on a regular assessment of need from a care coordinator, usually a community psychiatric nurse or social worker. At the time of the study, standard care programme approach practice in the participating trusts was for a care coordinator to carry out an unstructured needs assessment based on informal discussions with the patient, their carers and the clinical team. This assessment did not systematically cover particular areas of functioning or specify any particular questions that should be asked. It was left to the discretion of the care coordinator to decide what constituted a problem or a need. Care coordinators were not obliged to consider comprehensive lists of defined interventions when drawing up a care plan, nor did they have access to any up-to-date directory of agencies providing interventions.
Measurement of outcome
Follow-up at 12 months was conducted by a different research nurse who was masked to group allocation. The primary outcome measures were mental state (Brief Psychiatric Rating Scale, BPRS; Reference Overall and GorhamOverall & Gorham, 1962), social functioning (World Health Organization Psychiatric Disability Assessment Schedule, WHODAS; World Health Organization, 1988), quality of life (Manchester Short Assessment of Quality of Life, MANSA; Reference Priebe, Huxley and KnightPriebe et al, 1999) and patient satisfaction (Client Satisfaction Index, CSI; Reference Larsen, Attkison and HargreavesLarsen et al, 1979). Secondary outcomes were admission to hospital, needs at follow-up and needs met during the study (calculated as needs present at baseline that were no longer present at follow-up). The research nurses attended a Present State Examination training course and were trained to achieve reliable ratings on the BPRS. Interviews were recorded throughout the study and a random sample was scrutinised for reliability.
Statistical analysis
The main hypothesis was that patients would have a better outcome and fewer needs than controls if their care coordinators had received feedback from a standardised assessment. The secondary hypothesis was that exposing care coordinators to feedback on some of their patients would improve their care of other patients on whom they had not received feedback.
Sample size was determined by calculating the number of care coordinators necessary to have a 90% chance of detecting a clinically significant difference (defined as half of the standard deviation for each main outcome) at a 5% level of significance. The power calculation assumed that each care coordinator provided six patients and that the within-cluster correlation was 0.013.
Statistical analysis treated the care coordinator as a random effect, to adjust for the effect of clustering due to group randomisation and variability between care coordinators. Linear and non-linear random effect models were fitted using STATA (release 6.0 for Windows). Where appropriate, baseline values of the outcome and other known predictors of outcome were included as covariates. To investigate for contamination of the treatment effect among individually randomised patients, a preliminary analysis was carried out by adding an interaction term for method of randomisation (group or individual) and intervention group. For missing data, a logistic regression model was used to investigate possible predictors of non-response. Where missing data were predictable by covariates, these variables were added to the model to reduce any bias due to missing data.
Analysis of BPRS and WHODAS gave non-linear normal probability plots because these variables were skewed. Analyses using the clustered bootstrap (Reference Davison and HinkleyDavison & Hinkley, 1997) were used to check robustness. If a patient had no needs at baseline, then no needs could be met. Hence the analysis of met needs was based on clients with at least one need at baseline, using a logistic random effects model with needs nested within clients within the care coordinator (otherwise all analyses were by intention to treat). The number of medical and social needs at follow-up each had a maximum of eight, so the proportion of medical and social needs present at follow-up was modelled using a binomial model with a denominator of eight and the overall proportion of needs was modelled using a denominator of sixteen. A random effect term was added to account for the nesting of clients within care coordinators. This analysis used the GLAMM algorithm in STATA (Reference Rabe-Hesketh, Pickles and SkrondalRabe-Hesketh et al, 2001).
RESULTS
Seventy-two of 101 identified care coordinators agreed to participate (for characteristics, see Table 1). The participating care coordinators identified 952 potentially eligible patients. The trial aimed to recruit a random sample of six patients from the case-load of each care coordinator (i.e. a target of 432 patients), but in the end only 304 were recruited (for patient characteristics, see Table 2). Patient flow during the trial is shown in Fig. 1. At 12 months, 242 patients (79.6%) were re-interviewed.
Characteristics of care coordinators
| Profession | Arm 1 (n=18), feedback | Arm 2 (n=36), individual randomisation | Arm 3 (n=18), no feedback | Total (n=72) |
|---|---|---|---|---|
| Psychiatric nurse | 11 | 21 | 12 | 44 |
| Social worker | 6 | 13 | 4 | 23 |
| Occupational therapist | 1 | 0 | 2 | 3 |
| Other non-nurse | 0 | 2 | 0 | 2 |
Characteristics of study patients
| Randomisation method | ||||
|---|---|---|---|---|
| Cluster (arm 1) | Individual (arm 2) | Cluster (arm 3) | ||
| Feedback | Feedback | No feedback | No feedback | |
| n | 88 | 70 | 70 | 76 |
| Female (%) | 42 (47.7) | 35 (50.0) | 41 (58.6) | 32 (42.1) |
| Age (s.d.) | 42.2 (12.1) | 43.6 (11.6) | 44.1 (11.5) | 43.7 (11.6) |
| ICD-10 diagnosis1 | ||||
| F20-F29 Schizophrenia-like disorders | 44 | 43 | 38 | 44 |
| F30-F39 Affective disorders | 39 | 22 | 30 | 31 |
| F40-F48 Neurotic disorders | 4 | 3 | 0 | 1 |
| F60-F69 Personality disorders | 1 | 1 | 1 | 0 |
| F99 Unspecified mental disorder | 0 | 1 | 1 | 0 |
A preliminary analysis was carried out to determine whether there was evidence of a time × randomisation method interaction for each outcome variable. For variables other than satisfaction there was no evidence of an interaction, so a main effects analysis is presented for these variables (Tables 3, 4, 5) without disaggregating the data by method of randomisation. For satisfaction there was evidence of a differential treatment effect as the interaction term approached conventional levels of significance (interaction mean effect=0.21, P=0.07, 95% CI -0.02 to -0.44). Hence the interaction term was retained in the model for satisfaction, resulting in separate estimates of the intervention effect for this variable by method of randomisation (Table 5).
Number of needs at baseline
| Baseline needs | Medical | Social | Combined | |||
|---|---|---|---|---|---|---|
| Control | Intervention | Control | Intervention | Control | Intervention | |
| 0 | 101 (69%) | 92 (58%) | 80 (55%) | 96 (61%) | 61 (42%) | 61 (39%) |
| 1 | 30 (21%) | 42 (27%) | 47 (32%) | 40 (25%) | 45 (31%) | 43 (27%) |
| 2 | 12 (8%) | 19 (12%) | 14 (10%) | 19 (12%) | 21 (14%) | 34 (22%) |
| 3 | 1 (1%) | 5 (3%) | 4 (3%) | 2 (1%) | 12 (8%) | 12 (8%) |
| 4 | 2 (1%) | 1 (1%) | 1 (1%) | 4 (3%) | 5 (3%) | |
| 5 | 1 (1%) | 2 (1%) | ||||
| 6 | 2 (1%) | 1 (1%) | ||||
| No. of clients | 146 | 158 | 146 | 158 | 146 | 158 |
| No. of needs | 65 | 95 | 91 | 88 | 156 | 183 |
| Mean no. of needs per client | 0.45 | 0.60 | 0.62 | 0.57 | 1.07 | 1.16 |
Met needs at 12-month follow-up
| Met needs | Medical | Social | Combined | |||
|---|---|---|---|---|---|---|
| Control | Intervention | Control | Intervention | Control | Intervention | |
| 0 | 3 (8%) | 7 (13%) | 6 (12%) | 5 (10%) | 5 (7%) | 6 (8%) |
| 1 | 26 (68%) | 29 (54%) | 33 (63%) | 33 (63%) | 35 (52%) | 36 (45%) |
| 2 | 8 (21%) | 14 (26%) | 9 (17%) | 12 (23%) | 16 (24%) | 25 (31%) |
| 3 | 1 (3%) | 4 (7%) | 3 (6%) | 1 (2%) | 6 (9%) | 8 (10%) |
| 4 | 1 (2%) | 1 (2%) | 2 (3%) | 3 (4%) | ||
| 5 | 2 (3%) | 1 (1%) | ||||
| 6 | 1 (1%) | 1 (1%) | ||||
| No. of clients | 38 | 54 | 52 | 52 | 67 | 80 |
| No. of needs at baseline | 51 | 77 | 75 | 73 | 126 | 150 |
| No. of met needs | 45 (88%) | 69 (90%) | 64 (85%) | 64 (87%) | 109 (87%) | 133 (89%) |
| Adjusted odds ratio1 (95% CI) | 1.10 (0.31-3.98) | 1.25 (0.45-3.51) | 1.28 (0.57-2.88) | |||
| P | 0.88 | 0.67 | 0.52 | |||
| Between care coordinator variance | 0.82 | 0.15 | 0.48 | |||
Adjusted mean difference between intervention and control
| Control | Intervention | Adjusted mean difference (95% CI) | P | Intracluster correlation (care coordinator) | |||
|---|---|---|---|---|---|---|---|
| No. of patients | Mean (s.d.) | No. of patients | Mean (s.d.) | ||||
| BPRS | 115 | 5.19 (4.69) | 127 | 5.56 (4.27) | 0.34 (−0.67 to 1.35)1 | 0.51 | 0.05 |
| WHODAS | 115 | 0.69 (0.56) | 126 | 0.68 (0.62) | 0.016 (−0.13 to 0.16)2 | 0.83 | 0.09 |
| MANSA | 100 | 4.58 (0.88) | 114 | 4.66 (0.81) | 0.032 (−0.147 to 0.210)3 | 0.73 | 0 |
| CSI | |||||||
| Individual randomisation | 44 | 2.24 (0.53) | 47 | 2.20 (0.55) | − 0.05 (−0.22 to 0.12)4 | 0.55 | 0 |
| Group randomisation | 52 | 2.15 (0.42) | 61 | 2.35 (0.44) | 0.16 (0.01 to 0.31)5 | 0.03 | 0 |
Table 3 gives the number of needs recorded at baseline. At follow-up 88% of baseline needs were met, either because there was no longer a cardinal problem in the area or because the patient had been offered all suitable interventions (Table 4). The adjusted odds ratio (adjusted for care coordinator type) for a need being met in the intervention group relative to the control group was 1.28 (95% CI 0.57-2.88, P=0.52). A sensitivity analysis that included only met needs where there is no longer a cardinal problem did not substantially alter the original finding. In terms of the number of needs remaining at follow-up (which includes baseline needs still not met and any new needs that had arisen), the estimated odds ratio (OR) was not significant (OR=0.98, 95% CI 0.64-1.51, P=0.94) after adjustment for baseline needs and care coordinator type. Thus, there was no evidence that the intervention reduced the number of needs at follow-up. Two sensitivity analyses were performed to examine the effects of missing needs data. The first analysis assumed that all baseline needs on which there were missing data at follow-up remained unmet, whereas the second analysis assumed that all were met. Neither analysis made a substantial difference to the original findings. A further sensitivity analysis compared met needs and needs at follow-up separately for each method of randomisation, but again this did not substantially alter the original finding.
The intervention had no effect on mental state, social functioning or quality of life (Table 5) or likelihood of admission (adjusted OR=1.34, 95% CI 0.58-3.09; 9.7% admitted control, 12.1% feedback). However, there was an effect on satisfaction that depended on the method of randomisation (Table 5). When patients were randomised individually (i.e. within arm 2) there was no difference in satisfaction between the control and intervention patients (adjusted mean difference=-0.05, 95% CI -0.22 to 0.12, P=0.55), but when the randomisation was by care coordinator (i.e. arm 1 v. arm 3) there was a significant difference between control and intervention (adjusted mean difference=0.16, 95% CI 0.01-0.31, P=0.03). This difference according to method of randomisation suggests contamination in the individually randomised arm of the trial.
DISCUSSION
Interpretation of findings
The main hypothesis (that patients would have a better outcome if their care coordinators received feedback from the needs assessment) was not supported for any outcome except patient satisfaction. For this outcome we found that in the cluster-randomised arms the patients of care coordinators who received feedback (arm 1) were more satisfied than those of care coordinators who did not (arm 3). This finding was not replicated in the individually randomised arm (arm 2, where all care coordinators had the experience of feedback, but on half their patients). This suggests that our secondary hypothesis (that exposing care coordinators to feedback on some of their patients would improve their care of other patients on whom they had not received feedback) was supported for the outcome of satisfaction. Taken together, these findings indicate that increased patient satisfaction results from care coordinators experiencing the process of feedback rather than the content of that feedback. We speculate that this is because the experience of feedback fostered a change in the care coordinator's interaction with the patient by, for example, clarifying the care coordinator's explanation of the care planning process.
Why did the intervention not improve clinical outcome?
There are two obvious reasons for the failure of standardised needs assessment to enhance the informal care planning process: the overall level of need at baseline was at the lower end of what would have been expected from previous surveys (see Table 3); and the control groups, which received an informal needs assessment under the care programme approach, showed unexpectedly high reductions in their levels of need (87% of needs identified at baseline in the control groups were met at follow-up; see Table 4). Both of these findings suggest that the informal needs assessment that takes place under the care programme approach was already effective at meeting needs, so there was little scope for additional improvement. To have any chance of being clinically effective the Cardinal Needs Schedule would need to be applied in populations of patients with a higher level of baseline need, such as those who are homeless or difficult to engage. The Schedule might also be more effective if the list of suitable interventions were more tightly restricted to those for which there is a strong evidence base, such as those currently recommended in the National Institute for Clinical Excellence (NICE) treatment guidelines for schizophrenia (NICE, 2002). It is possible that other standardised needs assessments will prove more effective than the Cardinal Needs Assessment at enhancing routine care planning, but this remains to be demonstrated in randomised controlled trials.
Should standardised needs assessment be used in routine clinical practice?
With the introduction of the minimum dataset, all mental health services in the UK will be required to undertake routine outcome assessment (Reference GloverGlover, 2000) and it has been proposed that a standardised needs assessment should be used for this purpose (Reference SladeSlade, 2002). Our trial suggests that although routine use of standardised needs assessments might make patients more satisfied, this advantage has to be weighed against the lack of clinical benefit and the loss of the therapist time spent completing the assessment. There is some evidence to suggest that simple self-report needs assessments would produce benefits in patient satisfaction that are comparable with those produced by more complex schedules (Reference Van Os, Altamura and Bobesvan Os et al, 2004).
Clinical Implications and Limitations
CLINICAL IMPLICATIONS
-
▪ Care planning based on an informal assessment of patients' needs appears to be effective at identifying and meeting needs.
-
▪ The introduction of a standardised method of needs assessment does not convey any additional benefits.
-
▪ Patient satisfaction could be improved if care coordinators had some experience of feedback from a standardised assessment of need.
LIMITATIONS
-
▪ Other standardised needs assessments might prove more effective than the Cardinal Needs Assessment, but this remains to be demonstrated.
-
▪ The overall level of need in the sample was lower than that seen in representative samples of severely mentally ill people.
-
▪ The proposed explanation for improved satisfaction in the cluster-randomised group remains tentative.
Acknowledgements
We thank Francis Creed for involvement in the initial conception of the study and Brian Farraghar for help with the power calculation. We thank all the service users and care coordinators who participated in the study. This research was supported by the Wellcome Trust (grant no. 051641/Z/97/2).
eLetters
No eLetters have been published for this article.