There are increasing expectations of public protection from violent and other high-risk behaviours. Forensic and general psychiatrists are requested to carry out risk assessments, and healthcare and criminal justice professionals can be criticised if they make wrong decisions on release and detention of offenders. Risk assessment instruments can classify individuals according to probability of violence and are considered superior to clinical judgement. The most commonly used are actuarial instruments and structured professional judgement. Actuarial assessments are criticised because the group average method on which they are constructed means they cannot estimate risks with a high level of certainty when applied to an individual. Reference Hart, Michie and Cooke1 An increasing number of structured professional judgement instruments have been developed, with the aim of improving risk management. However, most items in these instruments are not independently predictive. Reference Coid, Yang, Ullrich, Zhang, Sizmur and Farrington2 Furthermore, meta-analysis demonstrated that if the intention is to predict violence, these instruments cannot predict better than ‘moderate’ levels of accuracy and are essentially interchangeable. Reference Singh, Grann and Fazel3
Psychopathy is a rare personality disorder, affecting less than 0.5% of the population, but has a disproportionate impact on violence at population level compared with other mental disorders, Reference Coid and Yang4 with exceptionally high levels of criminal recidivism. Unfortunately, evidence suggests that predictive efficacy of risk assessment instruments may be reduced among patients with diagnoses of personality disorder and drug dependence compared with those with major mental disorder. Reference Gray, Taylor and Snowden5 We tested predictive accuracy of three instruments specifically developed to assess risk of future violent behaviour, according to different categories of mental disorder in a large, high-risk sample of released male prisoners in England and Wales. We aimed to investigate how confident clinicians can be in making their decisions when guided by structured risk assessment methods.
Method
Study setting and participants have previously been described. Reference Coid, Yang, Ullrich, Zhang, Sizmur and Roberts6 Briefly, male prisoners in England and Wales (n = 1396), aged ⩾18 years, were interviewed 6-12 months before release by trained interviewers if serving a determinate sentence of ⩾2 years for a sexual or violent offence.
The Structured Clinical Interview for DSM-IV Axis II personality disorders diagnosed personality disorder, a module of the Schedule for Affective Disorders and Schizophrenia identified lifetime schizophrenia, drug use disorders and depressive disorder. Alcohol use disorder was measured using the Alcohol Use Disorders Identification Test. Psychopathy scores using the Psychopathy Checklist Revised (PCL-R) were divided into high, medium and low score ranges, where ⩾30 indicates a psychopathic personality.
Outcome was a binary indicator for having at least one conviction for a violent offence within 3 years recorded in the UK Police National Computer.
We tested three instruments commonly used in research and clinical practice: the Violence Risk Assessment Guide (VRAG), Reference Quinsey, Harris, Rice and Cormier7 a 12-item actuarial instrument; the 20-item Historical, Clinical, Risk Management-20 (HCR-20), Reference Webster, Douglas, Eaves and Hart8 a structured professional judgement; and the Offenders Group Reconviction Scale-II (OGRS-II), Reference Taylor9 a statistical risk score. Although designed to assess risk of any offending, it has good predictive accuracy for future violent offending among men. Reference Coid, Yang, Ullrich, Zhang, Sizmur and Roberts6
Reliability of the HCR-20 and PCL-R were good with intraclass correlation coefficients of 0.98 and 0.85 respectively. Reference Coid, Yang, Ullrich, Zhang, Sizmur and Roberts6
Results
Online Table DS1 shows predictive accuracy of the three instruments using area-under-the-curve (AUC) measures in a final sample of 1224 male prisoners. Instrument total scores were used for calculation of AUC statistics. Since there is no pre-specified cut off for the VRAG and the OGRS-II, the median of each instrument was applied conventionally as threshold for predicting individuals as violent/non-violent for the purpose of calculating percentage correctly classified, positive (PPV) and negative predictive values (NPV).
As shown in Table DS1, the performance of each instrument was moderate to good when no mental disorder was present, no more than moderate for participants diagnosed with schizophrenia and depression, with some further reduction among those with substance use disorders. For participants with antisocial and other personality disorder, their predictive efficacy ranged from poor to no more than chance. No instrument was statistically better than chance for individuals with psychopathy.
We performed a formal statistical test (chi-squared test using command roccomp in Stata version 12 for Windows) to investigate whether the AUCs of the instruments were statistically different in each diagnostic group. The OGRS-II and VRAG demonstrated significantly higher levels of predictive accuracy than the HCR-20 for all diagnostic categories except schizophrenia (online Table DS2).
Online Table DS3 provides information on PPV and NPV of the three instruments. All PPVs were below 50.0 indicative that problems in accuracy of prediction occurred primarily due to false positives.
Discussion
Our findings have major implications for risk assessment in criminal populations. Routine use of these risk assessment instruments will have major limitations in settings with high prevalence of severe personality disorder, such as secure psychiatric hospitals and prisons. These instruments are more accurate for use with patients in community mental health services, general psychiatric in-patient units, primary care and in the general population where prevalences are much lower. However, their level of accuracy with an individual with psychopathy is no better than flipping a coin. Both actuarial risk instruments demonstrated significant but clinically modest superiority over the HCR-20 which covers more potential risk and dynamic factors. However, most instruments demonstrate shrinkage of accuracy when applied to populations with different characteristics from those on which they were standardised. Furthermore, the HCR-20 and VRAG were developed to predict future violence and not specifically violent reconviction, whereas the OGRS-II was designed for prediction of general criminal recidivism. However, standardised instruments lack outcome specificity, with those designed to predict violence predicting general recidivism and vice versa. Structured risk assessment instruments may be limited to measuring a general construct of criminal risk rather than specific tendencies to violence as originally intended. Reference Glover, Nicholson, Hemmati, Bernfield and Quinsey10
The prisoner cohort included a large sample, prospectively interviewed prior to release. However, it did not include life-sentence prisoners as it would not have been possible to assess them prospectively in anticipation of their release date. The generalisability of the findings is therefore restricted to similar populations. Furthermore, violent outcome was defined on the basis of criminal records, did not include self-report, and may have therefore underestimated the occurrence of violence during follow-up. Application of the HCR-20 differs in clinical practice where an overall estimate of risk is given avoiding the assignment of scores. Future research should establish whether this approach is superior in predicting violence in people with psychopathy.
Both general and forensic psychiatrists who are requested to carry out risk assessments for violence should be mindful of the limitations of some risk assessment instruments when assessing those with psychopathy/psychopathic traits. Our findings correspond to previous research showing lower accuracy among psychiatric patients with pesonality disorder and drug dependence, with AUCs of comparable magnitude. Reference Gray, Taylor and Snowden5 Taken together, our findings emphasise that the first step in any assessment should be a comprehensive diagnostic assessment before assessing risk of future violence.
Our findings do not explain whether failure of three risk assessment instruments to predict violence in individuals with psychopathy was due to inherent limitations in their construction and standardisation or the inherent and unpredictable behaviour of these individuals, characteristic of their life course. One possible explanation is that items included in the three instruments demonstrate little variation in psychopathic offenders and, therefore, are unsuitable to differentiate those at high and low risk for violent reoffending. Alternatively, the ability of individuals with psychopathy to lie convincingly, con and manipulate even experienced clinicians may mean that accuracy can only be achieved with new actuarial tools excluding any items requiring subjective clinical judgement.
Funding
The Ministry of Justice (formerly the Home Office) funded the Prisoner Cohort Study awarded to J.W.C. J.W.C., C.K. and S.U. are funded by the National Institute of Health Research under its Programme Grants for Applied Research funding stream (PP-PG-0407-10500). The views expressed in this article are those of the authors and not necessarily those of the Ministry of Justice, the NHS, the National Institute of Health Research or the Department of Health.
eLetters
No eLetters have been published for this article.