The College Performance indicators in child and adolescent psychiatry

A small working group was set up in 1985to look at the general problem of audit and evaluation in child psychiatry. From an early stage it was clear that the group's work would be largely concerned with the KÃ¶rner Report1 and the performance indicators developed by the DHSS2.


The problem
On analysing the KÃ ¶rnerReport, we concluded that the recommended minimum data set was not well suited to describe clinical activity in those medical specialties that had moved away from traditional hospital practice. This included not only child and adolescent psychiatry but also adult and, indeed, old age psychiatry, as well as community paediatrics. For this reason, this article may be of interest to colleagues working in these areas.

Research into the KÃ ¶rner measures in child and adolescent psychiatry
We pinpointed some of the problems that would arise if the KÃ ¶rnerframework were to be used as a basis for Performance Indicators, as has now happened. We identified three major problems: ( 1) KÃ ¶rnerfocuses on GPs as a referral source, yet child and adolescent psychiatrists take re ferrals from several community agencies.
(2) KÃ ¶rnertakes the referral letter as the measure of a case, while child and adolescent psy chiatrists often see several members of a fam ily; indeed, it is not unusual for a whole family to be referred.

A pilot project
Three questionnaires were completed by volunteers: 50 consultants, 20 senior registrars, ÃOE2 registrars and 9 clinical assistants. They consisted of: ( 1) a form for monthly statistical returns (2) an attempt to delineate the context of child psychiatric practice (3) a weekly diary of direct work with children and families, indirect work (consultation, etc), teaching, management and travel.

Monthly statistics
Here we report results for the 48 consultants who completed this part of the questionnaires.

Comment
This represents quite a wide variety of referral totals.

Cases referred and individuals actually seen
Some clinicians saw, on average, only one and onethird people for every referral letter, whereas some saw more than four for every case referred.

Performance indicators Comment
This discrepancy is not reflected in any KÃ ¶rneror Performance Indicator proposals.

Comment
GP referrals do constitute the largest referral source but neglect of other sources would give a significantly distorted picture.

Waiting Lists
Eleven consultants were operating without a waiting list, 14had one of less than 10,10 had a waiting list of 10 up to 20, while 13 had a list of 20 and over.

Consultation
A very broad definition was used, i.e., cases where the child psychiatrist gave advice on assessment, treat ment or management but where there was no formal referral. The activity could be on individual style consultation, Ã la Caplan, but could also include ad vice given in the context of a case conference, or in a quasi educational setting as long as it was advice that had the aim of providing help to the family about which the consultation occurred, rather than for staff teaching purposes. It included consultation with pro fessionals in the clinic team about patients who are not the psychiatrists' responsibility and telephone consultations.
The factor that distinguished consultation from referral was that the clinician did not take over clini cal responsibility for the case.
Consultation is not acknowledged by KÃ ¶rnerso we were interested to see how much it was being used in practice. Comment Again, the KÃ ¶rnerconcept of the consultant out patient clinic requires modification in child and adolescent psychiatry.

The weekly diary
Fifty consultants kept a diary for one week. The pur pose of this was to attempt to define how clinicians were apportioning their time in broad categories.

Conclusion
The aim of this data presentation is to demonstrate the marginal relevance of the KÃ ¶rner principles to the measurement of professional activity in child psy chiatry. Thus, child psychiatrists take referrals from a variety of agencies, they see a number of people in the referred patient's family and social network and they often offer consultation as well as seeing individ ual patients. The diary data also highlighted the fact that, along with colleagues in other specialties, child psychiatrists frequently work long hours and spend significant time in study, teaching, research and man agement. These are all activities which are not measured by performance indicators, yet are crucial aspects of the consultant role.

Current performance indicators
The working party reviewed the Performance Indi cators published by the DHSS in 1987 and represen tatives have attended workshops conducted by the DHSS. The following points were put forward: "This report ushers in a new era for the analysis of Health Service activity. However, there are some grave problems for those branches of medicine where the pattern of work varies from the more traditional medical model. Child and Adolescent Psychiatry falls firmly into this group.
The data is to be collected according to the KÃ ¶rner mini mum data set. It is likely to give a very distorted picture of the work pattern of the consultant child and adolescent psychiatrist." "There are five more obvious reasons why this is so. These are: Referral sources: child and adolescent psychiatrists ac cept referrals from a wide range of professionals in the community, including social workers, educational psy chologists and school doctors. Thus, the primary health care team may. in many cases, not be the prime source of referral, although they should be informed of what is happening. It is usual, indeed universal, for more than one family member to be seen and there are variations in practice as to the amount of work that is done with other family members and the amount that is done with the patient who is identified by the referral letter. In many cases, the family is referred rather than an individual patient.
It is increasingly common for the child psychiatrist to work with other professionals who deal with the child directly. This can be in Health Service settings, such as paediatric wards or special care baby units: in Social Services settings, such as children's homes or day nurseries, etc; or in Educational settings, such as the School Psychological Service or in schools for mal adjusted children. This is a well established and recog nised practice that would be completely overlooked by the proposed indices for the minimum data set.
A considerable amount of work in child psychiatry has gone into developing the intensive delivery of treat ment so that therapies can be accomplished in a rela tively small number of out-patient attendances rather than over periods of many months. This is often achieved by the intensive input of several professionals on each visit. It would penalise the development of this desirable programme if the number of out-patient at tendances (rather than the nature of treatments given) were used as an output indicator.
A number of reorganisations in the Health and Social Services have, in recent years, eroded oppor tunities for close multi-disciplinary collaboration. This collaboration lies at the core of child psychiatric prac tice. The collection of totally separate data to judge performance of different disciplines (psychiatry, clini cal and educational psychology, social work, nursing, etc) will add to this fragmentation." "Other problems concern the amount of work done on the telephone and the need to attend case conferences if proper community work is to be done." There was concern, therefore, that the use of KÃ ¶rner data for performance indicators would give a dis torted picture of the pattern of activity which would favour an inappropriate "medical" way of working and, at worst, which may encourage a rigid "doctor" oriented approach.
With regard to in-patient services, the following principle is set out.

Performance indicators
"As far as possible, mentally ill children and adolescents should be treated in the community, and should be kept out of large institutions. This implies that these patients should be treated as out-patients. We have recommended indicators to help to identify where this policy has been achieved." The working group viewed this principle with the utmost concern since, in application, it will cut across the practice not only of child psychiatry but of medi cal practice in general. It is true that child psychiatry is practised substantially on an out-patient basis but there is an important minority of children with depression, anorexia nervosa, psychoses and other severe disorders that require day and in-patient care. We agreed that it is generally undesirable for adoles cents to be in large psychiatric hospitals; they should be in small special in-patient units when this is indi cated. To this end, all districts should have in-patient and day patient facilities for children and adolescents and Performance Indicators should be developed to ensure that this is the case.
Of the many performance indicators offered for child and adolescent psychiatry2, we thought that the following could help in reflecting some aspects of clinical activity.
"Finished consultant episodes related to resident population." "Number of consultants related to the total number of day patient attendances." "Hospital nursing staff per occupied bed day for child psychiatry." "Number of referral attendances related to number of attendances." "Sessions cancelled as a proportion of sessions arranged." "Total out-patient attendances related to resident population."

Conclusion: a constructive reply
The need for better quality information for manage ment of the health service is not questioned. How ever, the current performance indicators for child 97 and adolescent psychiatry seem to be doomed to fail ure, since they are based on the KÃ ¶rnerdata set which we have shown to provide a distorted picture of clinical activity in the specialty.
We recommend, as a start, the collection of data under the following headings:3 First attenders with referral source and counted both as one referral for each referral letter and as number of individuals seen.
Consultant initiated (or re-) attendances counted as one per referral letter and as number of individuals seen.
Non-attenders at first appointment and follow-up. Waiting list counted as number of referral letters. Consultation by number of hours spent and number of cases consulted about.
Number of hours in study, teaching and research. This is only a small step towards understanding the clinical activity of the child and adolescent psy chiatrist. Much more research is needed into the content of the clinical activity and its effectiveness. Experience has shown that this can only be done effectively by peer review, audit and clinical trials by the specialty concerned. More resources are needed for research and coordination which will move us towards accurate ways of measuring effective clinical activity.