Experience and application of HoNOS65

Aims and method The implementation of the Health of the Nation Outcomes Scales for Old Age Psychiatry services (HoNOS65+) in a National Health Service trust is described. Some preliminary data are reported for illustrative purposes. Results Lessons learnt from this process are identified, and further work needed both on the glossary for the scales and on systems for trapping data is suggested. Clinical implications These remain uncertain. It may be that HoNOS65+ will remain a purely administrative tool.

The HoNOS (Health of the Nation Outcome Scales) were devised, with funding from the Department of Health, by the Royal College of Psychiatrists Research Unit (CRU) in order to measure global change in the health and social functioning of mentally ill people (Wing et al, 1996). The primary aim of the scales was that they be used by multi-disciplinary clinicians in the mental health field to help achieve one of the three targets for improving mental health identified in the Government's Health of the Nation strategy -namely "to improve signifi cantly the health and social functioning of mentally ill people". The CRU made it clear that they considered this to include: improvement in mental, physical and/or social functioning, over and above what would be expected without intervention; and maintenance of an 'optimal functional state' by preventing, slowing and/or mitigating deterioration.
In the development of HoNOS, the CRU completed four stages of work: (a) Start-up phase, which included a litera ture search, and the first draft of the score, which included 20 items, one global, each rated 0-3. (b) Two pilot projects were set up, whereby data and user comments were analysed by the CRU. Changes to the scales included a reduction to 12 items, a more detailed glossary and 0-4 rating points.
Psychiatric Bulletin (1999), 23, 203-206 (c) Field trials which included 22 sites, during which data from 2706 patients were analysed. (d) Finally, trials of reliability which were undertaken independently in two academic units in Nottingham and Manchester.
The final HoNOS was based on this experience. Speciality versions of HoNOS were developed for child and adolescent mental health, learning disability and old age psychiatry (HoNOS65+). This article describes the implementation of HoNOS65+ in a community-orientated, openaccess old age psychiatry service based on the "Guy's model" (Shulman & Arie, 1991) of service delivery. No attempt has been made to evaluate the implementation; theoretical and political issues about outcomes measurement and HoNOS are not addressed in this report, and the data presented should be regarded as illustrative only.

Background
The Mental Health in the Elderly Directorate of Lewisham & Guy's Mental Health Trust carried out a strategic review in 1995-1996. This made several recommendations, one of which was to move from a process orientation to an outcomes one. As a start towards this, it was decided to implement a change measure in everyday clinical practice, and although HoNOS only measured change and not the effectiveness of specific interventions, it looked straightforward and easy to use. It was hoped that it would encourage clinicians in the Directorate to collect more focused service user data, and had the potential for use as a tool for exploring case mix/case load management, as well as, eventually, evaluating the impact of specific interventions.

Implementation
In December 1996, five clinicians from different disciplines completed the CRU Training the Trainers for HoNOS' course. (It is worth noting that although the course was useful in providing the opportunity for clinicians to understand the background to, and use of, the HoNOS scales, it did not provide a framework for training others, or consider the necessary processes and struc tures for implementation in a clinical service.) The Mental Health in the Elderly Service management agreed that the service would take part in the CRU pilot of HoNOS65+; data used within the directorate would also be fed back to the CRU.
The five clinicians established a 'HoNOS Development Group', with a lead clinician and administrator for each ward/team. The group developed a standard training pack, which included facilitation guidelines, information on HoNOS and an agreed structure and process for the implementation of HoNOS65+. Five half-day training sessions for all qualified clinical staff and administrators were set up. It was thought essential to involve the administrative staff as they have a crucial role in ensuring the commu nication of all HoNOS data to the central point where it was collated and analysed. The HoNOS Development Group included a researcher in volved in the development of better methods for measuring activity by mental health services for contracting purposes, who proved crucial in data trapping and analysis. Support from the aca demic department allowed HoNOS65+ data to be linked to data from the trust community team patient information system, SISYPHUS. It was agreed that HoNOS would be piloted within the Trust from the end of February 1997. The HoNOS Development Group met monthly to review progress, discuss any difficulties and make changes to procedures and systems.
Over the following months, minor changes to the wording of the glossary were made (see below) and 'rules' for the use of HoNOS65+ gradually tightened up. At first, all patients referred since the start of the project had to have one rating before discharge was permitted. By the end of April 1997 all patients, irrespec tive of referral date, had to have at least one rating before discharge was allowed. By the end of October 1997 all patients had to have at least two ratings (at referral and at discharge) before discharge was allowed. The sanction was that SISYPHUS would not remove a patient from the active list of a keyworker had the rules not been allowed. A similar protocol was implemented for in-patients, but without any sanctions.
A major training exercise was carried out at the start of 1997, and this was repeated in July, to obtain feedback on the glossary and its use. At this point 50 staff rated a single video interview of a patient together with additional information. As a result of this exercise and the feedback, further changes were made to the wording of the glossary.
The main difficulties encountered were: (a) One clinical area did not have a lead clinician based with them. This led to a low rate of HoNOS65+ use until one was appointed. (b) The HoNOS65+ glossary produced by the CRU proved too unclear to use in training clinicians, particularly on Scales 10 ('Pro blems with activities of daily living'), 11 ('Problems with living conditions') and 12 ('Problems with activities'). These had to be rewritten.  (c) Feedback of HoNOS data to clinicians was initially slow. A summary of progress was given to all teams at three months. However, the time lapse in collecting and analysing sufficient data was meaningful, particularly for patients with more than one rating, this stoked the fires of cynicism about the process among some clinicians, these smoulder still. (d) The total HoNOS scores, summing all the scale scores, did not seem a very useful indication of change in individual service users. The individual scale scores seem more useful. (e) Clinicians found the paperwork laborious; administrators sometimes had difficulty encouraging clinicians to complete forms. (f) At the July review issues like the multi dimensional nature of several scales, unclear index time periods, the impossi bility of rating delirium in the setting of dementia, the non-linearity of the Activ ities of Daily Living Scale (complex and basic skills are rated in parallel), as well as various other inconsistencies were re vealed. These were reported to the CRU at a seminar in November 1997. (g) There are significant conceptual and prac tical problems in the analysis and summa tion of multiple ratings by different raters at different stages in patients' careers. In an attempt to simplify these, analysis was confined to pairs of ratings in the same patient -further development in this area is necessary.

Findings
There was a linear accumulation of HoNOS ratings so that by June 1998 there were 2612 ratings available for analysis in 1072 patients. Six hundred and fifty-two patients had at least two ratings by this time, and for analysis these were arranged in pairs of different sorts (e.g. sequen tial, or bounding certain episodes like admission to an acute assessment ward). As an example of the sort of analysis that might be possible. Table 1 lists the mean change in HoNOS scale scores for one particular pair of ratings (referral to/dis charge from the service) broken down by a selection of diagnostic groups for all patients discharged during the year starting 1 April 1997 (dual ratings were not compulsory until October 1997, and other changes to procedures and systems were incremental so rating pairs for patients were only available for approximately 25% of patients discharged by the service during this year. The monthly rate of rating pairs per discharge rose during the year from 1% to just over 50%; with a monthly increment of approxi mately 5% in the last five months.) Table 1 also illustrates the mean change in individual scales broken down by the same diagnostic groups. With the possible exception of change in total score for patients with affective disorders, total score change appeared to represent an accumulation of change in many of the scales. In patients with affective disorders, however, there was a particu larly substantial contribution to the change in total score by the depression scale score change.
The results of the rating of a single videoed patient assessment by 50 raters revealed con siderable difficulty in achieving acceptable interrater reliability with this staff group in scales concerned with activities of daily living, problems with living conditions and work and leisure activities. There were problems inherent in the glossary. Difficulties with scales concerned with physical illness, hallucinations and delusions, depression and other problems appeared to be due to lack of clarity about the reference time period to be covered by HoNOSratings -an issue which has since been resolved by the CRU.
These very preliminary data suggest that some of the practical problems of implementing HoNOS65+ can be solved, but that much work remains to be done. We are in a position to make a number of recommendations for the implemen tation of HoNOS, and to identify a number of issues which await resolution.

Recommendations for implementing HoNOS
(a) Train one clinician to work as a lead clinician for each community multidisciplinary team or in-patient ward. Ensure that the clinicians are in a position of authority to facilitate the use of HoNOS and ensure that any changes agreed are implemented.

Issues that await resolution
The glossary must be revised so that it is easier to use. We are working with the CRU and others to develop a glossary in tabular form which differs as little as possible from the original, which we are piloting, together with a short version. It seems that the sort of clarity de manded of a research instrument is equally, if not more definitely demanded by a clinical instrument.
The time period covered must be standardised. This may be different for different services; for instance, rehabilitation psychiatry outcomes may need measures that cover longer periods of assessment than acute services like old age psychiatry.
These issues must be clarified nationally, so that improvements and developments of the HoNOS are common across all users. The CRU is now involved in further work to this end.
However useful to management, it is not yet clear that HoNOS data are useful to individual clinicians. This subsumes two issues: whether the data are of any use, and how data can be summarised so that it remains rich in infor mation without becoming daunting. We are exploring these issues. It may be that the real culture change occurs when individual HoNOS ratings are used at discharge from the service as a check on progress -clinicians may be encouraged to defend discharge decisions in the face of unchanged or greater HoNOS scale scores from those at referral. However, it is possible that this will never happen, and the scale scores will remain an administrative tool only.
Whether HoNOS scores can be useful in the evaluation of case-mix, or (by record linkage to intervention data) clinical effectiveness, remains to be seen. It seems likely that these issues must await resolution of the practical issues raised by this work.