Skip to main content
×
×
Home

Information:

  • Access
  • Cited by 24
  • Cited by
    This article has been cited by the following publications. This list is generated based on data provided by CrossRef.

    Kashiwagi, Hiroko Kikuchi, Akiko Koyama, Mayuko Saito, Daisuke and Hirabayashi, Naotsugu 2018. Strength-based assessment for future violence risk: a retrospective validation study of the Structured Assessment of PROtective Factors for violence risk (SAPROF) Japanese version in forensic psychiatric inpatients. Annals of General Psychiatry, Vol. 17, Issue. 1,

    SanSegundo, Miriam Sánchez Ferrer-Cascales, Rosario Bellido, Jesús H. Bravo, Mar P. Oltra-Cucarella, Javier and Kennedy, Harry G. 2018. Prediction of Violence, Suicide Behaviors and Suicide Ideation in a Sample of Institutionalized Offenders With Schizophrenia and Other Psychosis. Frontiers in Psychology, Vol. 9, Issue. ,

    Scurich, Nicholas 2018. The case against categorical risk estimates. Behavioral Sciences & the Law, Vol. 36, Issue. 5, p. 554.

    Völlm, Birgit A. Clarke, Martin Herrando, Vicenç Tort Seppänen, Allan O. Gosek, Paweł Heitzman, Janusz and Bulten, Erik 2018. European Psychiatric Association (EPA) guidance on forensic psychiatry: Evidence based assessment and treatment of mentally disordered offenders. European Psychiatry, Vol. 51, Issue. , p. 58.

    Labrecque, Ryan M. Campbell, Christopher M. Elliott, Jaycee King, Megan Christmann, Molly Page, Kari McVay, John and Roller, Katie 2018. An Examination of the Inter-Rater Reliability and Rater Accuracy of the Level of Service/Case Management Inventory. Corrections, Vol. 3, Issue. 2, p. 105.

    Fazel, Seena and Wolf, Achim 2018. Selecting a risk assessment tool to use in practice:a 10-point guide. Evidence Based Mental Health, Vol. 21, Issue. 2, p. 41.

    Shepherd, Stephane M. and Sullivan, Danny 2017. Covert and Implicit Influences on the Interpretation of Violence Risk Instruments. Psychiatry, Psychology and Law, Vol. 24, Issue. 2, p. 292.

    Brook, Michael 2017. Structured Approaches to Violence Risk Assessment: A Critical Review. Psychiatric Annals, Vol. 47, Issue. 9, p. 454.

    de Vries Robbé, Michiel and Willis, Gwenda M. 2017. Assessment of protective factors in clinical practice. Aggression and Violent Behavior, Vol. 32, Issue. , p. 55.

    Fazel, Seena Wolf, Achim Larsson, Henrik Lichtenstein, Paul Mallett, Susan and Fanshawe, Thomas R 2017. Identification of low risk of violent crime in severe mental illness with a clinical prediction tool (Oxford Mental Illness and Violence tool [OxMIV]): a derivation and validation study. The Lancet Psychiatry, Vol. 4, Issue. 6, p. 461.

    Marett, Christopher P. and Mossman, Douglas 2017. From Ballpark to Courtroom: How Baseball Explains Risk Assessment. Psychiatric Annals, Vol. 47, Issue. 9, p. 443.

    Fazel, Seena Wolf, Achim Fimińska, Zuzanna Larsson, Henrik and Sasayama, Daimei 2016. Mortality, Rehospitalisation and Violent Crime in Forensic Psychiatric Patients Discharged from Hospital: Rates and Risk Factors. PLOS ONE, Vol. 11, Issue. 5, p. e0155906.

    Muñoz Vicente, José Manuel and López-Ossorio, Juan José 2016. Valoración psicológica del riesgo de violencia: alcance y limitaciones para su uso en el contexto forense. Anuario de Psicología Jurídica, Vol. 26, Issue. 1, p. 130.

    Fazel, Seena Chang, Zheng Fanshawe, Thomas Långström, Niklas Lichtenstein, Paul Larsson, Henrik and Mallett, Susan 2016. Prediction of violent reoffending on release from prison: derivation and external validation of a scalable tool. The Lancet Psychiatry, Vol. 3, Issue. 6, p. 535.

    Monteith, Scott Glenn, Tasha Geddes, John Whybrow, Peter C. and Bauer, Michael 2016. Big data for bipolar disorder. International Journal of Bipolar Disorders, Vol. 4, Issue. 1,

    Judges, Rachel Egan, Vincent and Broad, Grant 2016. A Critique of the Historical Clinical Risk–20, Version 3, Risk Assessment Instrument. Journal of Forensic Psychology Practice, Vol. 16, Issue. 4, p. 304.

    Sedgwick, Ottilie Young, Susan Das, Mrigendra and Kumari, Veena 2016. Objective predictors of outcome in forensic mental health services—a systematic review. CNS Spectrums, Vol. 21, Issue. 06, p. 430.

    Gulati, Gautam Cornish, Robert Al-Taiar, Hasanen Miller, Christopher Khosla, Vivek Hinds, Christopher Price, Jonathan Geddes, John and Fazel, Seena 2016. Web-Based Violence Risk Monitoring Tool in Psychoses: Pilot Study in Community Forensic Patients. Journal of Forensic Psychology Practice, Vol. 16, Issue. 1, p. 49.

    Tiffin, Paul A. Kitchen, Charlotte E.W. and Weir, Steven 2015. Innovations in Practice: Piloting a new child and adolescent risk assessment suite in the UK. Child and Adolescent Mental Health, Vol. 20, Issue. 4, p. 225.

    Sher, Leo and Rice, Timothy 2015. Prevention of homicidal behaviour in men with psychiatric disorders. The World Journal of Biological Psychiatry, Vol. 16, Issue. 4, p. 212.

    ×

Actions:

      • Send article to Kindle

        To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

        Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

        Find out more about the Kindle Personal Document Service.

        Rates of violence in patients classified as high risk by structured risk assessment instruments
        Available formats
        ×
        Send article to Dropbox

        To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

        Rates of violence in patients classified as high risk by structured risk assessment instruments
        Available formats
        ×
        Send article to Google Drive

        To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

        Rates of violence in patients classified as high risk by structured risk assessment instruments
        Available formats
        ×
Export citation

Abstract

Background

Rates of violence in persons identified as high risk by structured risk assessment instruments (SRAIs) are uncertain and frequently unreported by validation studies.

Aims

To analyse the variation in rates of violence in individuals identified as high risk by SRAIs.

Method

A systematic search of databases (1995–2011) was conducted for studies on nine widely used assessment tools. Where violence rates in high-risk groups were not published, these were requested from study authors. Rate information was extracted, and binomial logistic regression was used to study heterogeneity.

Results

Information was collected on 13 045 participants in 57 samples from 47 independent studies. Annualised rates of violence in individuals classified as high risk varied both across and within instruments. Rates were elevated when population rates of violence were higher, when a structured professional judgement instrument was used and when there was a lower proportion of men in a study.

Conclusions

After controlling for time at risk, the rate of violence in individuals classified as high risk by SRAIs shows substantial variation. In the absence of information on local base rates, assigning predetermined probabilities to future violence risk on the basis of a structured risk assessment is not supported by the current evidence base. This underscores the need for caution when such risk estimates are used to influence decisions related to individual liberty and public safety.

Footnotes

Declaration of interest

None.

Violence risk assessment is an increasing part of psychiatric practice. Psychiatrists, psychologists and other health professionals seeking to manage the risk of their patients acting violently have a range of structured risk assessment instruments (SRAIs) to assist them. These instruments score a patient on variables associated with violence. Such scores are then either combined mathematically (the ‘actuarial’ approach) or assist clinicians in making risk classifications (the ‘structured professional judgement’ approach). The most widely used instruments have satisfactory psychometric qualities in a range of settings and populations, 1 and are reported to provide more accurate predictions of violence than unstructured clinical assessments. 2

Despite the widespread use of SRAIs by mental health practitioners in general and forensic settings, 3-6 the role of these instruments remains the subject of extensive debate. 3,7 Some of the controversy relates to the applicability of group-derived risk estimates to an individual case. 8 Studies of the predictive validity of SRAIs have shown that they can be used to rank individuals in terms of their likelihood of violence. At follow-up, for most groups studied, a randomly selected person who had engaged in violence will have scored more highly than a randomly selected person who had not in approximately 70% of cases. 9,10 This is a measure of SRAIs’ ability to establish what has been referred to as the ‘relative’ risk for participants in a sample. 11

In clinical practice, however, the absence of a comparison group puts a premium on establishing an individual’s ‘absolute’ violence risk. 11 The actuarial approach to this task uses the violence rates of groups with a given score in past follow-up studies as an estimate of the likelihood that a future person with that score will act violently. In such schemes, individuals are assigned a numerical probability based on their score. The alternative, structured professional judgement (SPJ) approach allocates patients to one of several classes (e.g. high, moderate or low risk). Both approaches depend for their clinical usefulness on the rates of violence for people with similar scores or classifications being stable. Research suggests that this is not the case when SRAIs are used to predict sexual offending, 11-13 and it has been argued on theoretical grounds that it may not be the case for violent offending either. 14

If violence rates in groups classified as high risk are unstable across study populations, there are a number of possible explanations. Several authors have suggested that for statistical reasons the rate of violence among high-risk cases should rise as the rate of violence rises in the population as a whole. 13,14 This explanation seems to preclude any numerical probability from being reliably assigned to an individual’s violence risk without reference to local base rates. It is also possible that the clinical discretion granted to SPJ users, but not to clinicians employing actuarial approaches, leads them to classify different groups of individuals as high risk. Further variation may result from features of the design of the study, such as follow-up in a hospital rather than in the community, or reliance on criminal conviction as opposed to self-reported violence as an outcome. 15 The prevalence of known risk factors for violence, such as age and gender, will also vary from one study to another. Finally, differences between countries and legal systems may lead to preventive measures, such as effective treatment and supervision, being more widely used.

Despite the increasing use of SRAIs, the rates of violence in groups classified as high risk by these instruments have not been systematically described, nor have the sources of any variability been studied. We examined data from predictive validity studies of the most widely used instruments to investigate the extent and sources of variation in rates of violence by individuals judged to be high risk. Specifically, we examined the degree to which variation in rates of violence in high-risk groups was explained by:

  1. (a) the rate of violence in individuals not classified as high risk;

  2. (b) the type of instrument administered;

  3. (c) methodological differences;

  4. (d) sample characteristics;

  5. (e) geographic location.

We focused on rates of violent behaviour in high-risk patients, as these are the individuals for whom clinicians are most likely to consider additional treatment, in-patient stay or supervision for the purposes of risk reduction.

Method

Risk assessment instruments

The nine risk assessment instruments most commonly used in clinical practice according to a recent international survey were included, 6 five of which were actuarial and four of which employed the SPJ approach. Actuarial instruments comprised the Level of Service Inventory - Revised (LSI-R), 16 the Psychopathy Checklist - Revised (PCL-R), 17 the Sex Offender Risk Appraisal Guide (SORAG), 18 the Static-99, 19 and the Violence Risk Appraisal Guide (VRAG). 18 The SPJ instruments comprised the Historical, Clinical, Risk Management - 20 (HCR-20), 20 the Sexual Violence Risk - 20 (SVR-20), 21 the Spousal Assault Risk Assessment (SARA) and the Structured Assessment of Violence Risk in Youth (SAVRY). 22,23

Systematic search

A systematic search was conducted to identify studies that measured the predictive validity of the nine instruments (Fig. 1). PsycINFO, EMBASE, Medline and the US National Criminal Justice Reference Service Abstracts were searched between 1 January 1995 and 1 January 2011 using the acronyms and full names of the instruments as keywords. This search was supplemented with studies identified through references, annotated bibliographies and correspondence with risk assessment experts. Investigations from any country in any language were considered for inclusion. Unpublished studies (government reports, conference presentations, Master’s theses and doctoral dissertations) were also considered. Actuarial instruments’ development studies were excluded, as were studies that used only select scales of an instrument and retrospective studies where risk assessment instrument coders were not masked to outcome.

To be included in the study, the rate of violence for participants classed as high risk (standardised according to the most recent version of the instruments’ manuals) and information on time at risk must have been available either in the published article or from the authors. In studies where several instruments were administered, rate data were included for each instrument and counted separately. When studies used samples composed of the same participants, the study with the largest sample size was included.

The initial search for predictive validity studies identified 468 investigations relating to the nine instruments. The rate of violence in individuals judged to be at high risk according to instruments’ manuals and time at risk information was available in the manuscripts of 21 eligible studies. Information from the remaining studies was requested directly from study authors and obtained for 26 studies. As none of these studies reported the rate of violence in individuals administered the LSI-R and judged to be at high risk, that instrument was excluded from further analysis. Details of the included studies can be found in online Table DS1.

Data extraction

One author (J.P.S.) extracted rate information, study features and sample characteristics from the validity studies using a standardised coding sheet. As a measure of quality control, six (12.8%) of the included articles were randomly selected and coded by another author (A.B.). A high level of interrater agreement was established (κ = 0.95). 24

Statistical analysis

Descriptive statistics were calculated for the mean rate of violence in individuals classed as high risk both for each instrument and overall for all instruments combined. Rates of violence in the high-risk group were defined as the ratios of the number of offending individuals in the high-risk group to the total number of individuals classified as high risk. Annualised rates were defined as violence rate divided by length of follow-up and capped at 100%. Variability in rates was measured using the I 2 index, calculated based on χ2 differences between individual sample rates and the overall rate weighted by inverse variances. The I 2 index describes the percentage of variation across samples due to between-study variability rather than sampling error alone.

Univariate binomial logistic regression analyses were then conducted to examine sources of variation in the rate of violence in the groups judged to be at high risk. The first source of variation considered was that samples with overall higher rates of violence might produce higher rates in individuals classed as high risk, specifically. By definition, the overall rate of violence included the individuals judged to be at high risk. Therefore, to avoid double counting, we employed as a proxy the rate of violence in participants not classified as high risk.

Univariate analyses were also conducted to assess the effects of the following additional variables on rates of violence for individuals classed as high risk: the type of risk assessment instrument (actuarial v. SPJ), outcome location (community v. other), choice of outcome measure (criminal conviction v. other), gender (percentage of sample that was male), mean sample age (in years) and geographic location (North America v. other). The fitting algorithms weighted the data contribution of the different studies by taking into account the variances in each study.

Predictors found to be significant at the P<0.05 level in the univariate analyses were entered into a multivariable logistic regression model to estimate adjusted effects. Backward elimination was used to drop non-significant effects from the model at the α = 0.05 significance level. Odds ratios and their corresponding 95% confidence intervals were used to interpret the remaining significant effects. All regression analyses were two-tailed and controlled for sample size, time at risk and study design (prospective v. non-prospective).

The present study aimed to explain variability in rates of violence in high-risk groups using information from individual studies. It did not seek to estimate the overall magnitude of an effect comparing experimental and control groups. For this reason, binomial logistic regression taking into account the variances of the rates of violence in high-risk groups in individual studies was preferred to meta-analytic approaches such as those based on log odds ratios. To assess the sensitivity of our results to the choice of data-analytic method, however, a post hoc general linear model meta-analysis was conducted of the log odds of violence in groups classified as high risk as a function of the covariates discussed above weighting the studies by their inverse variances. Log odds of violence were modelled, rather than the log odds ratios employed by other meta-analytic reviews, owing to the absence of a control group. All statistical analyses were performed using SAS version 9.3 for Windows.

Results

Information was collected on 13 045 participants in 57 samples from 47 independent studies (Fig. 1). 25-71 The instrument with the most predictive validity studies in which violence rates in high-risk groups were available was the VRAG (10 studies, 21.3%). The average sample was composed of 244 men (s.d. = 412) and had a mean age of 32.4 years (s.d. = 9.3). Approximately half of studies (21 studies, 44.7%) relied on criminal conviction as their outcome, with most studies using outcomes resulting from a violent incident in the community; such incidents were reported in 36 studies (76.6%). Studies were conducted in 13 countries, namely Argentina, Austria, Canada, Denmark, Finland, Germany, The Netherlands, New Zealand, Serbia, Spain, Sweden, the UK and the USA (Table DS1).

Outcome rates

Across instruments, the mean rate of violence for individuals classified as high risk was 54.8% (s.d. = 27.9, median = 57.6, inter-quartile range (IQR) 33.3-76.1, range 0.0-100.0) over an average time at risk of 55.4 months (s.d. = 41.8, range 0.9-194.4). The mean annualised rate of violence in high-risk groups was 23.1% (s.d. = 28.7, median = 12.9, IQR 6.5-19.0, range 0.0-100.0; I 2 = 92%). The mean, standard deviation and range of rates per year of violence in individuals classed at high risk using each of the eight instruments are shown in Table 1. The distribution of annualised rates with respective 95% confidence intervals for actuarial and SPJ instruments are displayed in Figs 2 and 3. The I 2 indices of rates for actuarial and SPJ instruments are 89% and 76% respectively, indicating that the majority of variability in violence rates is not due to chance.

Binomial logistic regression

Univariate analyses demonstrated an increased rate of violence in non-high-risk groups, the use of an SPJ instrument, non-community follow-up, an outcome other than conviction, fewer men in a sample and younger participants were associated with increased rates of violence in high-risk groups (Table 2). Backward elimination multivariable logistic regression analysis revealed which factors remained significant after adjustment for other variables (Table 2). The odds of violence in the high-risk group were found to increase by 7% for every unit increase (absolute increase of 1%) in the violence rate of the non-high-risk groups. Individuals classed as high risk by actuarial instruments had 25% lower odds of committing a violent act than individuals classed as high risk by SPJ instruments. Finally, for every 1% decrease in the percentage of men in a study, there was a 2% increase in the odds of violence in individuals classified as high risk.

Table 1 Annualised rate of violence in individuals classified as high risk by eight widely used structured risk assessment instruments

Annualised rate, %
Instrument High-risk group k a Min. Max. Mean s.d.
Actuarial
    PCL-R b Scores ⩾30 8 0.0 100.0 18.0 33.5
    SORAG Scores ⩾+20 6 0.0 19.0 10.4 6.6
    Static-99 Scores ⩾6 9 3.4 13.6 7.1 3.5
    VRAG Scores ⩾+14 10 6.5 75.0 22.4 20.9
SPJ
    HCR-20 Professional judgement 9 5.0 100.0 40.6 43.8
    SARA Professional judgement 3 16.5 22.9 19.4 3.2
    SAVRY Professional judgement 9 4.3 100.0 38.6 27.6
    SVR-20 Professional judgement 3 3.8 5.6 8.5 6.7

HCR-20, Historical, Clinical, Risk Management-20; Max., maximum; Min., minimum; PCL-R, Psychopathy Checklist - Revised; SARA, Spousal Assault Risk Assessment; SAVRY, Structured Assessment of Violence Risk in Youth; SORAG, Sex Offender Risk Appraisal Guide; SPJ, structured professional judgement; SVR-20, Sexual Violence Risk-20; VRAG, Violence Risk Appraisal Guide.

a. Number of samples.

b. Instrument not originally developed for the purpose of forensic risk assessment.

Fig. 1 Systematic search for predictive validity studies of commonly used structured violence risk assessment instruments.

Sensitivity analysis

General linear model meta-analysis of log odds for violence weighted by inverse variances produced similar results to logistic regression. In particular, backward elimination dropped the same predictors in the same order. The remaining significant predictors were the same, with similar effects.

Discussion

For structured instruments to be of greatest use to clinicians, the violence rates for high-risk groups in different clinical settings and different patient populations should be similar. The principal finding of this study is that, after adjusting for sample size, time at risk and study design, overall rates of violence in groups deemed high risk varied substantially both within and between instruments. Although the median annual rate of violence in high-risk groups is 12.9%, half of samples reported rates that were either below 6.5% or above 19.0%. The importance of considering local base rates of violence in the risk assessment process has been discussed theoretically; 72 however, this variation in high-risk groups has (to our knowledge) not been demonstrated previously and provides empirical support for caution in the use of risk estimates derived from SRAIs to influence clinical decisions related to individual liberty and public safety (detention in general and forensic psychiatric hospitals, discharge from psychiatric hospital, release from prison and length of community supervision). 8,73 In particular it calls into question recommendations for the use of SRAIs that do not emphasise the role of local base rates. 74

Fig. 2 Actuarial risk assessment instruments: annualised rates of violence in high-risk groups.

Fig. 3 Structured professional judgement risk assessment instruments: annualised rates of violence in high-risk groups.

Table 2 Predictors of rates of violence in individuals classified as high risk: binomial logistic regression

Rate of violence in high-risk group (k = 57) a
Univariate Multivariable
OR (95% CI) P OR (95% CI) P
General risk level of sample: rate in non-high-risk groups 1.07 (1.06-1.08) <0.0001 1.07 (1.06-1.07) <0.0001
Type of risk assessment instrument: actuarial v. SPJ 0.53 (0.45-0.63) <0.0001 0.75 (0.62-0.92) 0.004
Location of outcome: community v. other 0.36 (0.28-0.46) <0.0001 NS b
Choice of outcome measure: conviction v. other 0.66 (0.55-0.78) <0.0001 NS c
Gender: percentage men 0.97 (0.96-0.98) 0.002 0.98 (0.97-0.99) <0.0001
Age: mean age of participants 0.96 (0.95-0.98) <0.0001 NS d
Geographic location: North America v. other 1.02 (0.87-1.20) 0.80 NA

NA, not applicable; NS, not significant and dropped from model; OR, odds ratio; SPJ, structured professional judgement.

a. All analyses adjusted for sample size, time at risk and study design (prospective v. non-prospective); k = number of samples.

b. Variable dropped second from backward stepwise model.

c. Variable dropped third from backward stepwise model.

d. Variable dropped first from backward stepwise model.

The variation in rates of violence in patients classified as high risk is not random, with elevated rates in studies where the rate of violence for non-high-risk patients (and hence in all patients) is increased, where an SPJ instrument is used, and in which there are fewer men. It has been argued elsewhere that applying probabilities from groups to individuals for the purposes of violence risk assessment is not reliable. 8 The data we report point to a different problem: that rates of violence in high-risk groups depend on local factors, and no general assumptions can be made about the probability of violent behaviour. That the rate of violence in patients classified as high risk varies with the base rate of the overall sample and that such local base rate information should be incorporated into individual risk assessments concords with Bayes’ theorem. 75 However, there currently exists no resource that allows practitioners to systematically identify local base rates for different forms of violence in different psychiatric populations.

Previous studies have suggested that providing individuals undertaking risk assessment with such prevalence information might improve predictive accuracy. 76,77 Empirical evidence also suggests, however, that even when base rates are taken into account there remains substantial variation in the predictive accuracy achieved by different instruments in different settings. 10 Future research will be of greatest assistance to clinicians where it takes into account all of these variables in examining the performance of SRAI risk classifications.

Using the operational definitions of this study, individuals classified as high risk by actuarial assessments have reduced violence rates compared with those classified as high risk by SPJ instruments. As actuarial and SPJ approaches have similar predictive validity, 10,78 this is unlikely to be a consequence of SPJ instruments being more successful at identifying those who will be violent. It is more likely a consequence of the procedure that we, like previous researchers, 32 followed in defining an actuarial category corresponding to SPJ high risk. Nominal labels such as ‘high’, ‘moderate’ and ‘low’ are known to be interpreted inconsistently by clinicians and others. 79 Future comparisons of actuarial and SPJ approaches should consider using different ways of generating categories with similar levels of risk, perhaps by reserving high-risk classifications for actuarial scores higher than those we used.

Being male is a known demographic risk factor for violence. When a known risk factor is the source of systematic variation in violence rates for groups classified as high risk, this suggests that that risk factor is being allocated an incorrect weight. The error is capable of being made in either direction. In this case, structured assessments attributed too much weight to the increased risk associated with a person being male. The discovered effect is substantial: if the percentage of a sample is increased by, say, 10%, the odds of violence decrease by 22%. The explanation for this may lie in the threshold for admitting women to secure settings being higher than that for men, resulting in a population of women at higher risk of violence. This finding is consistent with previous research suggesting that being male is less of a risk factor in mental health populations than in the general population, 80-82 and that violence in female mental health populations is underrecognised. 83

Limitations of the study

There are several potential limitations to this review. First, we did not seek to examine the full range of SRAIs available, of which there are over 150. 84 We did, however, choose those most commonly used in clinical practice according to a number of recent surveys. Second, we were unable to obtain rate information from all eligible studies as this information was rarely reported in manuscripts. However, the data presented here show that variation in rates of violence for individuals classified as high risk is a general phenomenon and is not limited to a particular instrument. Consequently, it seems unlikely that the use of different instruments or the inclusion of all eligible studies would have resulted in a more homogeneous set of samples. Third, the reliability of information used to administer the included risk assessment tools and the reliability of information used to determine outcome occurrence was not routinely reported to allow us to adjust for these potential moderators. It may be that there is less variation in violence rates for high-risk patients when assessments are made and outcomes detected using more reliable sources of information. Fourth, we did not have the necessary information to be able to investigate how findings might have changed when different thresholds were used to class individuals as high risk. It may be that using higher thresholds (resulting in increased specificity) could have resulted in an improved ability to identify a small group of individuals at very high risk, although this would have also resulted in an increased rate of false negative predictions. Fifth, we needed to rely on the percentage of men in samples in our analyses rather than investigating the rate of violence in male and female high-risk groups separately. Therefore, caution may be warranted in interpreting the findings relating to gender as there may have been an aggregation bias in our continuous covariate. Finally, the number of samples for individual instruments was too small to investigate sources of rate heterogeneity in each instrument separately.

Future directions

In addition to addressing these shortcomings, future studies could examine whether variation in rates of violence for patients classified as high risk can be reduced by providing clinicians with information on outcome rates in individuals with similar clinical, geographical or criminal history backgrounds. A second area for future research concerns the operational definition of the term ‘high risk’. Some of the variation between rates of violence in groups with the label may be a consequence of the inconsistent use of this term, suggesting that the use of more detailed operational definitions may reduce the degree of variation across studies and instruments.

As unstructured clinical assessments of violence risk remain common in practice, 6 a third area in need of further research is the investigation of variation in rates of violence in patients judged to be at high risk without the use of an SRAI. Data from social psychology suggest that clinicians using unstructured methods are likely also to take insufficient account of base rates in assessing risk. 85 Finally, the possibility that more effective supervision affects rates of violence in patients classed as high risk warrants further investigation.

Implications of the study

After controlling for time at risk, the rate of violence in patients judged to be at high risk by SRAIs is not constant, varying considerably and systematically within and between instruments. Therefore, it does not seem possible to use SRAIs to assign reliably a predetermined numerical probability to the potential for an individual to act violently. This raises the question of whether, if practitioners cannot make a reasonable estimate of the base rate for the population in question and hence cannot estimate the likelihood of a future violent act for a member of a category, they should be using high-risk categorisations at all. These findings support recommendations for caution, given the present state of knowledge, in the use of such probabilistic risk estimates to influence decisions related to individual liberty and public safety. The results of individual risk assessments should be reported with explicit acknowledgement of the possible sources of error associated with their use.

Funding

S.F. is funded by the Wellcome Trust.

Acknowledgements

We are grateful to the following study authors for providing tabular data for the analyses: April Beckmann, Sarah Beggs, Susanne Bengtson Pedersen, Klaus-Peter Dahle, Rebecca Dempster, Kevin Douglas, Reinhard Eher, Jorge Folino, Monica Gammelgård, Grant Harris, Leslie Helmus, Andreas Hill, Hilda Ho, Drew Kingston, P. Randy Kropp, Henny Lodewijks, Jan Looman, Karin Arbach Lucioni, Jeremy Mills, Catrin Morrissey, Martin Rettenberger, Marnie Rice, Michael Seto, Robert Snowden, Jodi Viljoen, and Zoe Walkington.

References

1 Otto, R, Douglas, K. Handbook of Violence Risk Assessment. Routledge, 2010.
2 Conroy, MA, Murrie, DC. Forensic Assessment of Violence Risk. Wiley, 2007.
3 Buchanan, A, Binder, R, Norko, M, Swartz, M. Psychiatric violence risk assessment. Am J Psychiatry 2012; 169: 340.
4 Higgins, N, Watts, D, Bindman, J, Slade, M, Thornicroft, G. Assessing violence risk in general adult psychiatry. Psychiatr Bull 2005; 29: 131–3.
5 Tolman, A, Mullendore, K. Risk evaluations for the courts: is service quality a function of specialization? Prof Psychol Res Prac 2003; 34: 225–32.
6 Viljoen, JL, McLachlan, K, Vincent, GM. Assessing violence risk and psychopathy in juvenile and adult offenders: a survey of clinical practices. Assessment 2010; 17: 377–95.
7 Tyrer, P. An agitation of contrary opinions. Br J Psychiatry 2007; 190: s12.
8 Hart, SD, Michie, C, Cooke, DJ. Precision of actuarial risk assessment instruments. Evaluating the ‘margins of error’ of group v. individual predictions of violence. Br J Psychiatry 2007; 190: s605.
9 Cook, N. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation 2007; 115: 928–35.
10 Singh, JP, Grann, M, Fazel, S. A comparative study of violence risk assessment tools: a systematic review and metaregression analysis of 68 studies involving 25,980 participants. Clin Psychol Rev 2011; 31: 499513.
11 Helmus, L, Hanson, RK, Thornton, D, Babchishin, KM, Harris, AJR. Absolute recidivism rates predicted by Static-99R and Static-2002R sex offender risk assessment tools vary across samples: a meta-analysis. Crim Justice Behav 2012; 39: 1148–71.
12 Looman, J. Comparison of two risk assessment instruments for sexual offenders. Sex Abuse 2006; 18: 193206.
13 Helmus, L, Hanson, K, Thornton, D. Reporting Static-99 in light of new research on recidivism norms. Forum 2009; 21: 3845.
14 Mossman, D. Another look at interpreting risk categories. Sex Abuse 2006; 18: 4163.
15 Singh, JP, Fazel, S. Forensic risk assessment: a metareview. Crim Justice Behav 2010; 37: 965–88.
16 Andrews, DA, Bonta, J. LSI-R: The Level of Service Inventory – Revised. Multi-Health Systems, 1995.
17 Hare, RD. The Hare Psychopathy Checklist – Revised. Multi-Health Systems, 2003.
18 Quinsey, VL, Harris, GT, Rice, ME, Cormier, CA. Violent Offenders: Appraising and Managing Risk. American Psychological Association, 2006.
19 Harris, AJR, Phenix, A, Hanson, RK, Thornton, D. Static-99 Coding Rules: Revised 2003. Solicitor General Canada, 2003.
20 Webster, CD, Douglas, KS, Eaves, D, Hart, SD. HCR-20: Assessing Risk for Violence. Simon Fraser University, 1997.
21 Boer, DP, Hart, SD, Kropp, PR, Webster, CD. Manual for the Sexual Violence Risk-20. Professional Guidelines for Assessing Risk of Sexual Violence. Simon Fraser University, 1997.
22 Kropp, PR, Hart, SD, Webster, CD, Eaves, D. Spousal Assault Risk Assessment Guide (SARA). Multi-Health Systems, 1999.
23 Borum, R, Bartel, P, Forth, A. Manual for the Structured Assessment of Violence Risk in Youth (SAVRY): Version 1.1. University of South Florida, 2003.
24 Landis, JR, Koch, GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159–74.
25 Arbach, K, Pueyo, AA. Violence risk assessment in mental disorders with the HCR-20. Papeles del Psicologo 2007; 28: 174–86.
26 Beggs, SM, Grace, RC. Psychopathy, intelligence, and recidivism in child molesters: evidence of an interaction effect. Crim Justice Behav 2008; 35: 683–95.
27 Bengtson, S. Is newer better? A cross-validation of the Static-2002 and the Risk Matrix 2000 in a Danish sample of sexual offenders. Psychol Crim Law 2008; 14: 85106.
28 Caperton, JD. Predicting Recidivism among Sex Offenders: Utility of the STATIC-99, Minnesota Sex Offender Screening Tool-Revised, and Psychopathy Checklist-Revised. Sam Houston State University, 2005.
29 Dahle, KP. Strengths and limitations of actuarial prediction of criminal reoffense in a German prison sample: a comparative study of LSI-R, HCR-20 and PCL-R. Int J Law Psychiatr 2006; 29: 431–42.
30 De Vogel, V, de Ruiter, C. The HCR-20 in personality disordered female offenders: a comparison with a matched sample of males. Clin Psychol Psychother 2005; 12: 226–40.
31 De Vogel, V, de Ruiter, C, van, Beek, Mead, G. Predictive validity of the SVR-20 and Static-99 in a Dutch sample of treated sex offenders. Law Hum Behav 2004; 28: 235–51.
32 Dempster, RJ. Prediction of Sexually Violent Recidivism: A Comparison of Risk Assessment Instruments. Simon Fraser University, 1998.
33 Dempster, RJ. Understanding Errors in Risk Assessment: The Application of Differential Prediction Methodology. Simon Fraser University, 2001.
34 Douglas, KS, Ogloff, JRP, Hart, SD. Evaluation of a model of violence risk assessment among forensic psychiatric patients. Psychiatr Serv 2003; 54: 1372–9.
35 Douglas, KS, Yeomans, M, Boer, DP. Comparative validity analysis of multiple measures of violence risk in a sample of criminal offenders. Crim Justice Behav 2005; 32: 479510.
36 Eher, R, Rettenberger, M, Matthes, A. Aktuarische prognose bei Sexualstraftaetern [Actuarial predictions of sexual offenders]. Mschr Krim 2009; 92: 1827.
37 Eher, R, Rettenberger, M, Schilling, F, Pfäfflin, F. Failure of Static-99 and SORAG to predict reoffense categories in relevant sexual offender subtypes: a prospective study. Sex Offend Treat 2008; 3: 114.
38 Folino, J, Almiron, M, Ricci, MA. Factores de riesgo de recidiva violenta en mujeres filicidas [Violent recidivism risk factor in filicidal women]. Vertex 2007; 18: 258–67.
39 Friendship, C, Mann, RE, Beech, AR. Evaluation of a national prison-based treatment program for sexual offenders in England and Wales. J Interpers Viol 2003; 18: 744–59.
40 Gammelgard, M, Koivisto, AM, Eronen, M, Kaltiala-Heino, R. The predictive validity of the Structured Assessment of Violence Risk in Youth (SAVRY) among institutionalised adolescents. J Forensic Psychiatr Psychol 2008; 19: 352–70.
41 Gibas, AL, Kropp, PR, Hart, SD, Stewart, L. Validity of the SARA in a Canadian Sample of Incarcerated Adult Males. International Association of Forensic Mental Health Services, 2008.
42 Grann, M, Belfrage, H, Tengstrom, A. Actuarial assessment of risk for violence: predictive validity of the VRAG and the historical part of the HCR-20. Crim Justice Behav 2000; 27: 97114.
43 Grann, M, Langstrom, N, Tengstrom, A, Kullgren, G. Psychopathy (PCL-R) predicts violent recidivism among criminal offenders with personality disorders in Sweden. Law Hum Behav 1999; 23: 205–17.
44 Gretton, H, Abramowitz, C. SAVRY: Contribution of Items and Scales to Clinical Risk Judgments and Criminal Outcomes. American Psychology Law Society, 2002.
45 Harris, GT, Rice, ME, Quinsey, VL, Lalumiere, ML, Boer, D, Lang, C. A multisite comparison of actuarial risk instruments for sex offenders. Psychol Assess 2003; 15: 413–25.
46 Helmus, LMD, Hanson, RK. Predictive validity of the Static-99 and Static-2002 for sex offenders on community supervision. Sex Offend Treat 2007; 2: 114.
47 Hill, A, Habermann, N, Klusmann, D, Berner, W, Briken, P. Criminal recidivism in sexual homicide perpetrators. Int J Offender Ther Comp Criminol 2008; 52: 520.
48 Ho, H, Thomson, L, Darjee, R. Violence risk assessment: the use of the PCL-SV, HCR-20, and VRAG to predict violence in mentally disordered offenders discharged from a medium secure unit in Scotland. J Forensic Psychiatr Psychol 2009; 4: 523–41.
49 Jovanovic, AA, Tosevski, DL, Ivkonvic, M, Damjanovic, A, Gasic, MJ. Predicting violence in veterans with posttraumatic stress disorder. Vojnosanit Pregl 2009; 66: 1321.
50 Kingston, DA, Yates, PM, Firestone, P, Babchishin, K, Bradford, JM. Long-term predictive validity of the Risk Matrix 2000: a comparison with the Static-99 and the Sex Offender Risk Appraisal Guide. Sex Abuse 2008; 20: 466–84.
51 Kloezeman, KC. Violent Behavior on Inpatient Psychiatric Units: The HCR-20 Violence Risk Assessment Scheme. University of Hawaii, 2004.
52 Kropp, PR, Hart, SD. The Spousal Assault Risk Assessment (SARA) guide: reliability and validity in adult male offenders. Law Hum Behav 2000; 24: 101–18.
53 Lodewijks, HPB, Doreleijers, TAH, de Ruiter, C. SAVRY risk assessment in violent Dutch adolescents: relation to sentencing and recidivism. Crim Justice Behav 2008; 35: 696709.
54 Lodewijks, HPB, de Ruiter, C, Doreleijers, TAH. Gender differences in violent outcome and risk assessment in adolescent offenders after residential treatment. Int J Forensic Ment Health 2008; 7: 105–41.
55 Lodewijks, HPB, Doreleijers, TAH, de Ruiter, C, Borum, R. Predictive validity of the Structured Assessment of Violence Risk in Youth (SAVRY) during residential treatment. Int J Law Psychiatr 2008; 31: 263–71.
56 Looman, J, Abracen, J. Comparison of measures of risk for recidivism in sexual offenders. J Interpers Viol 2010; 25: 791807.
57 McEachran, A. The Predictive Validity of the PCL:YV and the SAVRY in a Population of Adolescent Offenders. Simon Fraser University, 2001.
58 Meyers, JR, Schmidt, F. Predictive validity of the Structured Assessment of Violence Risk in Youth (SAVRY) with juvenile offenders. Crim Justice Behav 2008; 35: 344–55.
59 Mills, JF, Kroner, DG. The effect of discordance among violence and general recidivism risk estimates on predictive accuracy. Crim Behav Ment Health 2006; 16: 155–66.
60 Morrissey, C, Hogue, T, Mooney, P, Allen, C, Johnston, S, Hollin, C, et al. Predictive validity of the PCL-R in offenders with intellectual disability in a high secure hospital setting: institutional aggression. J Forensic Psychiatr Psychol 2007; 18: 115.
61 Pedersen, L, Rasmussen, K, Elsass, P. Risk assessment: the value of structured professional judgments. Int J Forensic Ment Health 2010; 9: 7481.
62 Polvi, NH. The Prediction of Violence in Pre-trial Forensic Patients: The Relative Efficacy of Statistical versus Clinical Predictions of Dangerousness. Simon Fraser University, 1999.
63 Reeves, KA, Kropp, PR, Cairns, K. An Independent Validation Study of the SARA. International Association of Forensic Mental Health Services, 2004.
64 Rettenberger, M, Eher, R. Predicting reoffense in sexual offender subtypes: a prospective validation study of the German version of the Sexual Offender Risk Appraisal Guide (SORAG). Sex Offend Treat 2007; 2: 112.
65 Rice, ME, Harris, GT. Men who molest their sexually immature daughters: is a special examination required? J Abnorm Psychol 2002; 111: 329–39.
66 Seto, MC, Barbaree, HE. Psychopathy, treatment behavior, and sex offender recidivism. J Interpers Viol 1999; 14: 1235–48.
67 Snowden, RJ, Gray, NS, Taylor, J, MacCulloch, MJ. Actuarial prediction of violent recidivism in mentally disordered offenders. Psychol Med 2007; 37: 1539–49.
68 Snowden, RJ, Gray, NS, Taylor, J, Fitzgerald, S. Assessing risk of future violence among forensic psychiatric inpatients with the Classification of Violence Risk (COVR). Psychiatr Serv 2009; 60: 1522–6.
69 Viljoen, JL, Scalora, M, Cuadra, L, Bader, S, Chavez, V, Ullman, D, et al. Assessing risk for violence in adolescents who have sexually offended: a comparison of the J-SOAP-II, J-SORRAT-II, and SAVRY. Crim Justice Behav 2008; 35: 523.
70 Walkington, Z, O'Keeffe, C, Thomas, S. Predicting Violence Recidivism in Violent Juveniles: A UK Trial. HM Prison Service, 2006.
71 Wormith, JS, Olver, ME, Stevenson, HE, Girard, L. The long-term prediction of offender recidivism using diagnostic, personality, and risk/need approaches to offender assessment. Psychol Serv 2007; 4: 287305.
72 Szmukler, G. Violence risk prediction in practice. Br J Psychiatry 2001; 178: 84–5.
73 Fazel, S, Singh, JP, Doll, H, Grann, M. The prediction of violence and antisocial behaviour: a systematic review and meta-analysis of the utility of risk assessment instruments in 73 samples involving 24,827 individuals. BMJ 2012; 345: e4692.
74 Eastman, N, Adshead, G, Fox, S, Latham, R, Whyte, S. Forensic Psychiatry. Oxford University Press, 2012.
75 Bayes, T, Price, R. An essay towards solving a problem in the doctrine of chances. Phil Trans Royal Society London 1763, 53: 370418.
76 Arkes, HR. Impediments to accurate clinical judgment and possible ways to minimize their impact. J Consult Clin Psychol 1981; 49: 323–30.
77 Arkes, HR. Costs and benefits of judgment errors: implications for debiasing. Psychol Bull 1991; 110: 486–98.
78 Guy, LS. Performance Indicators of the Structured Professional Judgment Approach for Assessing Risk for Violence to Others: A Meta-analytic Survey. Simon Fraser University, 2008.
79 Hilton, N, Carter, A, Harris, G, Sharpe, A. Does using nonnumerical terms to describe risk aid violence risk communication? J Interpers Viol 2008; 23: 171–88.
80 Lidz, C, Mulvey, E, Gardner, W. The accuracy of predictions of violence to others. JAMA 1993; 269: 1007–11.
81 McNiel, DE, Binder, RL. Correlates of accuracy in the assessment of psychiatric inpatients' risk of violence. Am J Psychiatry 1995; 152: 901–6.
82 Rabinowitz, J, Garelik-Wyler, R. Accuracy and confidence in clinical assessment of psychiatric inpatients risk of violence. Int J Law Psychiatry 1999; 22: 99106.
83 Dean, K, Walsh, E, Moran, P, Tyrer, P, Creed, F, Byford, S, et al. Violence in women with psychosis in the community: prospective study. Br J Psychiatry 2006; 188: 264–70.
84 Singh, JP, Serper, M, Reinharth, J, Fazel, S. Structured assessment of violence risk in schizophrenia and other psychiatric disorders: a systematic review of the validity, reliability, and item content of 10 available instruments. Schizophr Bull 2011; 37: 899912.
85 Tversky, A, Kahneman, D. Judgment under uncertainty: heuristics and biases. Science 1974; 185: 1124–31.