Recruitment

Over the period of study there were 96 admissions to the in-patient unit, of which 41 were emergencies and 55 were elective. The patients admitted had a mean age of 15.05 (range 13-18). There were 69 girls and 27 boys. The eating disorders service saw 54 new referrals for out-patient treatment for anorexia nervosa. All these patients were female, with a similar mean age (15.6 years). Clinician-rated HoNOSCAs were completed for all in-patients and out-patients at all rating points, comprising 361 scoring episodes.

The self-rated version of HoNOSCA was completed by all adolescents at admission and discharge from the in-patient unit, although two failed to record mid-treatment review ratings. All out-patients completed self-ratings at each time point. There were therefore 359 self-ratings, representing a completion rate of 99.4%.

The completion of the self-rated version took 10-15 minutes for the first rating and about 5 minutes for subsequent ratings.

Reliability of HoNOSCA—SR

Split-half reliability showed a correlation between the odd— and even-numbered scores of 0.73 based on a series of 65. Test—retest reliability of HoNOSCA—SR was explored in a subset of 24 patients to whom the instrument was administered on two occasions, a week apart. As these were in-patients with moderately severe disorder, it was assumed that there was unlikely to have been much change in severity of difficulties between the two rating points. The intraclass correlation coefficients varied from 0.32 for item 1 (antisocial and aggressive behaviour) to 0.88 for item 2 (attention and concentration). The total score (Pearson's) correlation between time 1 and time 2 was 0.806, P < 0.001 (Table 1).

Table 1

Test—retest reliability for the self-rated version of HoNOSCA (n=24)

Scale item number	Kappa correlation coefficient
1	0.318
2	0.882
3	0.565
4	0.357
5	0.636
6	0.731
7	0.412
8	0.739
9	0.357
10	0.565
11	0.560
12	0.578
13	0.399

Validity of HoNOSCA—SR

The validity of HoNOSCA—SR was explored by correlating it with the SDQ and (for the eating disorder series) the modified Morgan—Russell severity schedule, which provides a global severity measure covering physical, psychological and social adjustment (Reference Morgan and HaywardMorgan & Hayward, 1988). This measure has been widely used as a severity measure in anorexia nervosa (Reference Gowers, Weetman and ShoreGowers et al, 2000). The HoNOSCA—SR gave a correlation of 0.66 (P < 0.001) with the adolescent-rated SDQ (n=39 cases); this compares with a correlation of 0.397 between the clinician-rated version of HoNOSCA and the SDQ (Reference Yates, Garralda and HigginsonYates et al, 1999). For the patients with eating disorders, HoNOSCA—SR correlated moderately well with the total (self-report) EDI—2 score (r=0.629, n=53, P < 0.001), but only weakly (and inversely, given the direction of scoring) with the clinician-rated Morgan—Russell total score (r=-0.32, n=54,P=0.018).

Patient view of clinical severity and outcome

At admission to the in-patient unit, or at presentation to the out-patient service, the adolescent patients recorded comparable levels of difficulty to the clinician ratings. For the in-patients (n=96) the mean total admission score was 17.8, compared with the mean clinician rating of 19.6. For the out-patients the mean scores were 14.9 and 18.8, respectively. There were, however, notable differences between scale items (Fig. 1), with clinicians tending to give higher ratings to difficulties with peer relationships and school attendance, while the young people themselves drew attention to problems with attention and concentration (question 2) and scholastic and language skills (question 5).

Fig. 1

Comparison of clinician-rated and self-reported scores on the Health of the Nation Outcome Scales for Children and Adolescents (HoNOSCA) at admission and discharge for 96 adolescent in-patients. This scale scores for (1) aggression, (2) concentration, (3) self-injury, (4) substance misuse, (5) language, (6) physical illness, (7) hallucinations, (8) somatic disturbance, (9) emotion, (10) peer relationships, (11) independence, (12) family relationships and (13) school attendance; see Appendix for full details.

During the course of treatment both the clinician-rated and self-rated versions of HoNOSCA demonstrated sensitivity to change. The in-patients reported a significant change over time, but slightly less than was reported by the staff (mean admission and discharge scores for 57 patients admitted for longer than 6 weeks were 18.44 and 12.82 self-rated, compared with clinician ratings of 18.35 and 9.75). The out-patients with eating disorders showed changes comparable with the clinician ratings. The changes in ratings over time for a consecutive smaller series from both services who were treated for a minimum of 4 months is shown in Table 2.

Table 2

Mean clinician-rated and self-report HoNOSCA scores over time for in-patient and out-patient cases for all patients in treatment for 18 weeks

	In-patient (n=18)		Out-patient (n=17)
	Self-report	Clinician	Self-report	Clinician
First assessment	16.22	20.17	15.00	18.00
Week 6	12.11	14.89	13.65	13.71
Week 12	10.72	11.94	10.53	11.06
Discharge/week 18	9.56	10.11	8.88	9.76

Considering each scale item in turn, both raters reported improvement across the board, but young people were more positive about improvements in family relationships (item 12) and clinicians about behavioural items (items 1-3) (see Fig. 1).

Agreement between clinician and patient ratings

In individual cases agreement between clinician and patient ratings was poor, with a correlation (Pearson's r) between HoNOSCA and HoNOSCA—SR of 0.27 at presentation, rising to 0.58 at discharge. There were notable differences by diagnosis, for the in-patients. Those with psychoses tended to see themselves as having fewer problems than the clinician's assessment, while those with emotional (chiefly affective) disorders rated themselves as having more. A subgroup of patients with incipient borderline personality disorder with poor peer relationships and self-harm were the highest scorers on HoNOSCA—SR (mean admission score 24.6). The out-patient eating-disorder group showed much higher levels of agreement, with a correlation of 0.58 at presentation and quite good agreement (r=0.72) at discharge.

Strengths and limitations of HoNOSCA

Clinicians, patients and commissioners all require comprehensive outcome data in order to evaluate services and individual treatment response. The HoNOSCA assessment has become well established as a clinician-rated general outcome measure (Reference Garralda, Yates and HigginsonGarralda et al, 2000), balancing simplicity with a reasonable level of reliability and validity (Reference Brann, Coleman and LukBrann et al, 2001). It has been shown to correlate highly with the Children's Global Assessment Scale (Reference Shaffer, Gould and BrasicShaffer et al, 1983) and to complement the Paddington Complexity Scale (PCS) in terms of its coverage of mental health problems in children and adolescents (Reference Yates, Garralda and HigginsonYates et al, 1999).

A survey of 120 child and adolescent mental health services who had received training in the use of HoNOSCA suggested that half were using it 6 months later (further details available from the author upon request). Not all have been persuaded that the enthusiasm for HoNOSCA is fully justified, however (Reference JaffaJaffa, 2000), and other measures have since been developed. Alongside the PCS, the Strengths and Difficulties Questionnaire (Reference GoodmanGoodman, 1997) is a notable addition to the field. Incorporating both parent and child versions, the SDQ raises the question of who is the most relevant rater of a child's difficulties: clinician or child? One of the major reservations about HoNOSCA is that there is no patient perspective, and that clinicians might inevitably see progress when their interventions are being assessed.

Adolescent views of severity

Despite theoretical difficulties of insight and sometimes poor engagement between adolescents and mental health services, this study shows that adolescents are aware of their difficulties in a range of domains and will freely admit to them. They often do not see difficulties in the same areas as their treating clinicians, however. It is noteworthy that adolescents themselves rate the subjective items (such as difficulty with concentration and attention) as more problematic than staff do. This discrepancy highlights the importance of therapeutic engagement in identifying difficulties and hence common treatment aims.

Adolescent views on outcome

Despite the above reservations, in the two services studied adolescents reported significant improvements with treatment, which spanned the duration of intervention. This gave some support to the average length of treatment of 3-4 months. In the out-patient service, this view matched the progress seen by staff.

Agreement between clinician and patient ratings

The correlation between HoNOSCA and HoNOSCA—SR is weak, particularly for in-patients. These patients have been rated at the point of admission, and their attitude to admission may be reflected in the scores. Approximately one-third was suffering from an acute psychotic disorder, with predictable levels of suspicion and lack of insight. The greater level of agreement between clinicians and patients in the out-patient service requires explanation. This could be related to the diagnostic homogeneity of the out-patient group, but people with eating disorders are not noted for acknowledging difficulties. A more likely explanation is that in an out-patient service clinicians are more reliant on patient and informant report in completing ratings, whereas in-patient staff can observe rather than take the patient's word for performance in such areas as peer relationships. It may therefore be that despite the lower levels of agreement, the clinician ratings for the in-patients are more objective.

Therapeutic use of HoNOSCA—SR

The two services have used the clinician— and self-rated scores to monitor progress through treatment with the patients themselves. In the out-patient eating disorder service particularly, where patients are encouraged to change behaviour, the scores provide regular feedback in addition to physical measures and the scores on specific eating disorder self-report questionnaires. Thus, patients with anorexia nervosa might be encouraged to gain weight and judge for themselves whether this results in an improvement or deterioration in general functioning by their own account. Progress in the first 6 weeks can then be used to boost motivation for the next phase of treatment. The possibilities for the therapeutic use of HoNOSCA to provide feedback to patients look promising.

CLINICAL IMPLICATIONS

1. • The self-rated version of the Health of the Nation Outcome Scales for Children and Adolescents (HoNOSCA) offers a useful perspective on adolescent mental health outcomes to support the clinically rated version.

2. • Adolescents with mental health problems report similar overall levels of difficulty to those noted by clinicians, but are often concerned about different areas of functioning.

3. • A combination of self-report and clinician-rated outcome measures can be used therapeutically to monitor progress and enhance motivation.

LIMITATIONS

1. • The self-rated HoNOSCA should be completed with the help of a member of staff who is not directly involved in treatment, to maximise patient understanding and quality of scoring.

2. • Correlations between clinician-rated and self-reported HoNOSCA total scores are poor.

3. • The self-reported HoNOSCA, like the clinician-rated version, may not address the mental health problems of all adolescents presenting to child and adolescent mental health services.

Self-rated version of the Health of the Nation Outcome Scales for Children and Adolescents

The patient is asked to assess the following items, selecting answers from the categories ‘not at all’, ‘insignificantly’, ‘mild but definitely’, ‘moderately’ or ‘severely’.