Matthew Large, Psychiatrist
28 March 2014

The recent meta-analysis by Singh et al. examined the proportion of violent people among those classified as high-risk, known as the positive predictive value (PPV). They found that PPV is highly variable between studies and is most strongly associated with the base rate of violence in the whole risk-assessed group (1). They conclude that risk assessment is not a reliable indicator of absolute risk. We agree. The increased focus on the PPV of high-risk categories is a welcome development because it leads to a consideration of the number of people who might need to be assessed as high-risk for every true positive (actually violent) person. The number needed to assess is often a more relevant measure than those derived from the receiver operator curve and it clearly illustrates the lack of meaning in recent debates about the extent to which group data applies to individuals (2) and the margins of error in particular risk predictions (3). However, we believe the debate about risk assessment nowneeds to move beyond abstract notions relating solely to probability. A probability after all is simply a number between 0 and 1, a number that isuninformative unless it is a probability of something specific.

Although not cited in the recent paper, we systematically examined PPV of risk categorization after making generous assumptions about the statistical power of risk assessment (4). Unlike the present paper, our paper focused on the main factor that actually determines base rates and thus PPV - the definition of violence according to violence severity. For example, using a risk assessment instrument with a sensitivity and specificity of 80% for the detection of different outcomes, the PPV for criminally violent behavior over a year by people with schizophrenia mightbe about 4% under optimal conditions, while the same figure for homicide would be 0.04%.

In the primary risk research, including that used by Singh et al. in the recent meta-analysis, a wide spectrum of violent events is amalgamatedinto an omnibus violent category. These events range from common assault all the way to homicide. Each of these diverse events has different base rates and consequences, with more severe violence having lower base rate but leading to greater losses.

Risk assessment in mental health should to start to consider the dimension of resulting loss. In areas outside mental health, risk is not aprobability but is a quantum of loss - that is why we pay our insurance premium in money, yet have little idea of the likelihood of the loss of our possessions. In our view any study that does not consider the magnitude of resulting loss should not really be thought of as a 'risk assessment' and more properly should be referred to as 'probability assessment'. While quantification of loss poses significant challenges, considering a definition of risk that includes the loss component re-emphasizes two complex, important and unanswered questions

i)What actual psychiatric interventions in terms of cost/side effects/benefits are indicated for those who are regarded as high-risk, and yet should be withheld from patients classified as low risk? If the harm we consider is not severe, no costly, restrictive or intrusive treatment can be justified. If the harm considered is severe, it will alsobe rare. Therefore what costly and intrusive intervention can be justified for the tiny proportion of false positives, or if the intervention is not costly or intrusive, why withhold it from low-risk patients, who will commit many of the future acts of violence? (5)ii)Is there evidence that an overall reduction in violence can be achieved by applying this cost/effective and acceptable intervention to a group who are more likely to offend while denying it to those who as a group less likely to offend? Will the additional resources spent on preventing violence by high-risk patients be justified in terms of harm reduction? (6)

At the end of the recent paper the authors recommend that risk assessments be provided with a qualification explaining their limitations.Here we agree as well. Perhaps it should be "this risk assessment providesan estimate of an uncertain probability of an unspecified event with no consideration of the consequences"

References

1.Singh JP, Fazel S, Gueorguieva R, Buchanan A. Rates of violence inpatients classified as high risk by structured risk assessment instruments. The British journal of psychiatry : the journal of mental science. 2014; 204: 180-7.2.Scurich N, Monahan J, John RS. Innumeracy and unpacking: bridging the nomothetic/idiographic divide in violence risk assessment. Law and human behavior. 2012; 36(6): 548-54.3.Hart SD, Michie C, Cooke DJ. Precision of actuarial risk assessment instruments: evaluating the 'margins of error' of group v. individual predictions of violence. The British journal of psychiatry Supplement. 2007; 49: s60-5.4.Large MM, Ryan CJ, Singh SP, Paton MB, Nielssen OB. The predictive value of risk categorization in schizophrenia. Harvard review of psychiatry. 2011; 19(1): 25-33.5.Large M, Ryan CJ. Screening for suicide: a comment on Steeg et al. Psychological medicine. 2012; 42(9): 2011-2; author reply 2-3.6.Wand T, Large M. Little evidence for the usefulness of violence risk assessment. The British journal of psychiatry : the journal of mental science. 2013; 202: 468.

... More

Conflict of interest: None declared

Manjeet S. Bhatia
07 March 2014

The article by Jay P. Singh and co-authors is an excellent one in highlighting the cautious interpretation of studies on rates of violence among psychiatric patients classified as High Risk.Not only there is huge variation, their interpretation without having the base rates is not appropriate. There are many other factors which influence the rates of violence e.g. setting (indoor or outdoor),pattern of patients in the setting (differ in psychiatric hospital and general hospital setting),typeof disorder ( schizophrenia /mood disorder/substance abuse),disorder variables (age- group, type of disorder,acute/chronic,co-morbid disorder present or absent, type and nature of treatment given),compliance variables (patient as well as family factors),type of assessment scale used (structured/standardized), prevalence of violent behaviour in community)and social variables linked to violent behaviour . More studies (country and disorder based after controlling different variables)are required to reach a definite conclusion.

... More

Conflict of interest: None declared