Inconsistencies in risk assessment

An audit of case notes and a survey of in-patients was carried out to evaluate risk assessment on an in-patient ward. We found considerable inconsistencies between the risk assessment records in medical and nursing notes. A systematic survey found higher levels of risk than either set of notes, but combining the notes improved the quality of risk assessment compared to the survey. We suggest three key areas for action to improve risk assessment.

The public eye is often turned towards 'failures' in the management of psychiatric patients. Accounts of assults involving people in contact with mental health services can lead to dramatic headlines and vitriolic editorials. Despite grow ing public opinion that mental health profes sionals should be responsible for the behaviour of patients, we know that in practice we have limited influence preventing risks becoming realities. However, mental health professionals are in a good position to assess and record risks with a view to preparing care plans to manage risks. Standards of risk assessment have been criticised in several high profile enquiries (Ritchie et oÃ-, 1994;Blom-Cooper et al, 1995), and mental health services are looking at ways of improving the monitoring and management of risks. The Royal College of Psychiatrists has published a guide to the management of violence (Royal College of Psychiatrists, 1996) and the College Research Unit is currently working on clinical practice guidelines for the management of violent patients (Royal College of Psychiatrists, 1998). Self-harm and suicide receive less media attention, but result in considerable morbidity and mortality. The Government has set targets to reduce the suicide rate to less than 9.4 per 100 000 and to reduce the 1990 rate in severely mentally ill people by 33% in the year 2000 (Department of Health, 1993).
In the 12-month period before this two-week study, there were 202 incidents of self-harm or violence on the ward concerned. These incidents were reported through the standard incident recording procedures of the hospital. Seventyfour episodes of self-harm and 26 violent incidents caused physical harm to patients or staff. This project began as a two-week special ised study module, which is a new development within the psychiatry clerkship of the medical school at the University of Leeds. The aims of the project were: (a) to investigate whether risk assessments were being recorded in medical and nur sing notes for in-patients on an acute psychiatric ward. (b) to compare the medical and nursing records of risk assessment, for consistency. (c) to compare these risk assessments with a risk assessment interview designed for the project.

The study
Twelve patients were interviewed during their admission to an adult acute admissions general psychiatry ward. The ward has 28 beds and covers two sectors with a combined adult population of 82 000. There are two small community teams linked to the ward, but there is no regular access to special care beds. Each patient gave consent. Each patient was inter viewed with the risk assessment tool and this was then compared with the medical and nursing notes.
In the hospital there are separate nursing and medical records which are combined after each admission. During the hospital admission, how ever, the nursing records are kept separately from the medical notes. In the nursing notes, the admission entry, care plan and ward round entries were reviewed. In the medical notes the admission entry and ward round entries were examined, with particular attention to the history and mental state taken on admission. The notes were scanned for entries recording the assessment of current self-harm and violence, and past history of self-harm and violence. When present, the entries were further classified as indicating a positive risk or no risk. An inter pretation was made about the level of the positive risks recorded in the nursing and medical records based on severity and duration of risk, opportunities and any qualitative information recorded. The risk was rated as low, medium or high.
The risk assessment tool was designed and administered by P.H. The pilot tool was a semistructured interview with sections for current AUDIT self-harm and violence. Each section contained 10-15 items and tick boxes to record positive or negative response. Each positive response scored one point and the total for each section gave an overall numerical score of risk. The items were devised from a study of the literature and consultation with consultant colleagues in gen eral and forensic psychiatry. The interview was piloted with patients and then revised, leaving 10 items for each risk area. The interview took five minutes to complete.
The patients were categorised as low, medium or high risk according to the interview scores. Scores of eight or more were rated as high risk, from five to seven, medium risk and less than five, low risk. It was assumed that each item contributed equally to the overall score. The judgements about risk in case notes were made by the same person who administered the inter views, introducing the possibility of bias.

Findings
Cose notes review The results are given in Table 1. The table  records whether an assessment of current risk of self-harm and violence was recorded and also notes whether there was a record of a history of self-harm and a history of violence in the current admission notes. The table also gives how often the risk was positive. In general, there were less records of risk assessment in the nursing notes (23) than in the medical notes (33). Entries in nursing notes on current risk of self-harm and past history of self-harm were made 58% of the time. Current risk of violence was recorded for 50% of patients. Notes on a past history of violence appeared in 25% of notes. Entries in medical notes also varied. Every file contained an entry on current self-harm and suicidal intent (whether positive or absent). Seventy-five per cent contained an entry on current risk of violence, 58% contained entries on history of self-harm and 42% contained notes on previous violence.
There was poor correlation between the nur sing records and the medical records for all categories of risk assessment. There was a record of a current self-harm assessment in both sets of notes forjust over half the in-patients involved in the study. In the other categories of risk assessment, the inconsistencies were in the majority. Even when both sets of notes contained a record they did not all agree on positive risks. However, the number of positive risk assess ments was increased if the separate note systems were combined.

Comparison ojease notes to risk assessment scores
The level of risk from the interview was compared with the combined level of recorded risk pro duced by integrating the nursing and medical records together. The results of the comparison showed that for current self-harm risk, concor dance was fairly good. Using the low, medium and high categories, two patients scored more on the interview than in case notes and two scored less, with the remaining eight scoring the same level of risk. For risk of violence, concordance was almost as good. Seven patients were rated at the same risk level by the interview and case notes. Four patients were rated at higher risk and one at lower risk by the interview.

Comment
This was a small-scale, focused project carried out by a medical student with guidance from consultants. The case note review was an audit. This was combined with a research interview which was limited in scope by the short time available for the study. We used a small sample of patients because we suspected that incon sistencies in risk assessment were widespread and would be readily detectable.
We found that risk assessment on the ward was carried out in an unsystematic way and results were not recorded consistently in nursing and medical records. The lack of recording does not mean that risk assessments were not done. It was noticeable that there were fewer negative risk assessments than positive. It is possible staff felt that negative results were not as Table 1. Comparison of the nursing and medical notes and risk assessment interview for 12 patients Risk of self-harm Risk of violence History of self-harm History of violenceNursing7(4) 6(6) 7(6) 3(1)Medical12(5) 9(5) 7(7) 5(5)Nursing and medical7(4) 5(4) 5(3) 3(1)Nursing or medical (combined)12(8) 9(6)Interview12 (8) 12 (7) The first number in each column is the number of case notes with entries about risk assessment followed by the number of positive entries in parentheses.
important to record, thus reducing the apparent thoroughness of their assessments. A systematic enquiry by interview found slightly higher ratings of risks than current practice on the ward. The only consistent risk assessment was the record ing of risk of self-harm and suicide in the medical notes. This is the only aspect of risk assessment subject to an existing audit standard. For over four years there has been a medical case note standard which requires the following in the mental state examination: "suicidal ideation must be noted as either present or not". Largeand small-scale audits over the past four years have shown that this standard is adhered to in over 95% of case notes. There are no standards in medical or nursing notes for recording risks of any other sort. Furthermore, there are no written protocols for sharing risk assessment infor mation, other than in the supervision register policy. The audit showed that a posi hoc integration of the two sets of notes gave higher numbers or positive risk assessments which tallied well with the interview tool. Thus the information was mostly collected but had be come lost and confused by the two recording systems which did not interact satisfactorily.
During this study we observed several other features of risk assessment recording. The medical and nursing notes were difficult to follow. Entries were sometimes illegible and the notes were poorly ordered. Risk assessment entries were not collated into one place which made the assessment of past history difficult, requiring lengthy painstaking analysis of the notes. Since past behaviour is arguably one of the best predictors of future risk, the existing format of our medical and nursing notes is unsatisfactory for risk assessment recording.
It appeared that as a patient became well known to the service, case note entries became less formal. This meant that the notes became more difficult to follow. Even though the patients' regular clinical team might have been aware of the situation there was often little in the records to describe current risks to anyone else. This meant it would be very difficult to communicate risk to other staff not intimately involved in a patient's care, such as a duty doctor assessing a request for self-discharge. We also found that a few patients had been involved in an incident of self-harm or violence during their hospital stay. There had not always been a recorded re assessment of risk following the incidents. Risk assessments were thus not fully integrated into planning management.
Our results suggest that if risk assessments are better organised and communicated, then routine clinical practice could be almost as good as a systematic enquiry. Furthermore where a standard existed to encourage regular enquiry the risk assessment was of equal value to the systematic enquiry. It should be possible to attain very high standards of risk assessment through appropriate audit, training and record keeping. Multiple sets of case notes appear to confound risk assessment. Our study did not extend to evaluating the predictive accuracy of risk assessments. It is important to do this once risk assessments are routinely available to audit, otherwise we may fail to improve the quality of assessments. Over emphasising risks and defen sive practice could lead to inappropriately re strictive regimes, and neglect of risks could miss the chance to prevent harm.
We have proposed the following recommenda tions locally. These may merit consideration in other trusts that recognise similar issues in their risk assessment procedures.
(a) Audit standards are implemented for recording risk assessments and manage ment plans. (b) Risk assessments are made by a multidisciplinary team when more than one person is involved with a patient's care. Teams should train together specifically to develop their risk assessment and risk management strategies. (c) Risk assessments are recorded in such a way as to be immediately available, sys tematic, up-to-date and clearly linked to risk management. Risk assessments should be collated in a single place in paper records and efforts should be made to audit predictive accuracy.
Following the study, we presented our findings locally and hope to introduce more standards through the clinical audit group. We have implemented team training in risk assessment and management, and we are developing paperbased and computer-based records. Parallel developments in the trust include a new multidisciplinary case note system for wards (abolish ing the separate nursing and medical notes) and the addition of risk assessment to all Care Programme Approach documentation. The pro ject will be repeated to evaluate the effects of these interventions.