Seena Fazel, Wellcome Trust Senior Research Fellow, Department of Psychiatry, University of Oxford
Achim Wolf, DPhil Student & Research Assistant, Department of Psychiatry, University of Oxford
11 May 2017

The study by Quinlivan and colleagues (1) could be interpreted to suggest that clinician and patient ratings are better than actuarial tools in predicting self-harm after an emergency hospital presentation with self-harm. However, we would argue that this is an incorrect interpretation.



First, the clinical evaluation appears to have occurred after these tools were completed by the same clinician, and, although they were masked to the overall score, the clinical impression will therefore have been strongly informed by the items in these suicide risk assessment tools. In fact, the study does not appear to be a comparison between actuarial tools and a distinct, unstructured clinical judgement, but a comparison between actuarial tools and what is called a structured clinical judgement approach (where structured questions about relevant risk factors are asked, and then a clinical judgement is made about an individual’s overall risk level). Clarification of the exact procedure used is important for interpreting the findings.



Second, the authors correctly point out that, on the basis of ROC curves, risk assessment tools performed no better than clinician ratings. The other way of looking at this, however, is that clinician ratings performed no better than risk scales. In particular, the Manchester Self-Harm Rule (2), a 4-item tool, performed just as well. Importantly, the authors found no evidence of between-hospital heterogeneity for this tool’s performance. Clinician ratings on the other hand showed substantial heterogeneity between hospitals, with specificity ranging from 58% to 82%. The lack of variability in the actuarial tools could be argued to be an advantage when performance between clinician rating and assessment tool is no different. Furthermore, tools like this will be considerably quicker, leaving more clinician time for risk management (as opposed to assessment).



Third, the clinicians were based in teaching hospitals (Brighton, Bristol, Derby, Manchester, and Oxford) with longstanding research interests in self-harm. Whether the reported predictive accuracy of clinician ratings is generalizable to non-specialist centres is an empirical question.



Fourth, the patient rating may also have been influenced by the questions asked by the tools (which tend to be categorical and therefore easy to work out what constitutes a risk factor). In a sense, then, the patient rating is a form of structured judgement.



Comparing risk tools with clinicians may not be informative, or even feasible, as clinical interviews already include many of the items used in risk tools. Instead, future research should compare actuarial scores with or without additional clinician input. In other words, if clinicians disagree with the risk level provided by actuarial tools, does this reclassification lead to an improvement in predictive performance? As the AUCs for the tools in this study ranged from 0.55 to 0.72, there may be considerable room for improvement by incorporating novel and modifiable risk factors as has been shown in violence risk assessment in patients with severe mental illness (3). Ultimately, randomized studies will be required to establish the effects of different approaches to risk assessment on patient and service outcomes.



1 Quinlivan L, Cooper J, Meehan D, Longson D, Potokar J, Hulme T, Marsden J, Brand F, Lange K, Riseborough E, Page L. Predictive accuracy of risk scales following self-harm: multicentre, prospective cohort study. British Journal of Psychiatry. 2017. DOI: 10.1192/bjp.bp.116.189993

2 Cooper J, Kapur N, Dunning J, Guthrie E, Appleby L, Mackway-Jones K. A clinical tool for assessing risk after self-harm. Annals of Emergency Medicine. 2006; 48(4), 459-466.

3Fazel S, Wolf A, Larsson H, Lichtenstein P, Mallett S, Fanshawe TR. Identification of low risk of violent crime in severe mental illness with a clinical prediction tool (Oxford Mental Illness and Violence tool [OxMIV]): a derivation and validation study. Lancet Psychiatry. 2017. DOI: 10.1016/S2215-0366(17)30109-8 ... More

Conflict of interest: S.F. has published on risk assessment, including a platform for risk assessment tools (OxRisk). A.W. is currently researching violence risk assessment.

Emily Kruger, Assistant Psychologist, Child and Adolescent Mental Health Service (CAMHS)
26 March 2017

Thank you for publishing such an interesting, in depth study. As one who is very interested in ways to recognise and prevent self harm and crisis situations, I am pleased to see an article dedicating research to such an important area. Thus, in order to prevent crisis situations, it is paramount that reliable and valid measures are used to assess in clinical settings.

I only have one criticism of the paper. In terms of your introduction, I would have liked to see more information R.E self harm and risk factors of self-harming behaviours, in order to set the scene. In addition, it may have been useful to state what self harm consists of and the controversies surrounding this in order to inform a ready of whom may be unaware of the details.

Again, I thank you for a compelling read.

Many thanks,

Emily Kruger ... More

Conflict of interest: None Declared