Case ascertainment and operational definitions

The sample comprised all 425 patients discharged from the three special (high secure) hospitals in England and Wales between 1 January 1982 and 31 December 1983. Records of each patient's criminal convictions were obtained from the Home Office. These records covered the individual's convictions during the calendar year of discharge, during which each had been out of special hospital care for an average of 0.5 years, and for each successive year to 10.5 years. The records also covered each individual's criminal career prior to discharge. All other data were collected from the medical records departments of the three hospitals. Details of the members of the sample and of their subsequent criminal conviction have been published (Reference BuchananBuchanan, 1998).

The Mental Health Act 1983 requires, for detention exceeding 28 days, the allocation of a patient's mental disorder to one or more of four classes: mental illness, psychopathic disorder, mental impairment and severe mental impairment. The legal and administrative nature of these classes distinguishes them from clinical diagnoses. Many patients have more than one diagnosis, not all of which are reflected in the legal class they are assigned. Significantly also, a patient's legal classification does not usually change as his or her condition fluctuates. Legal classes nevertheless contain diagnostically distinct groups of patients. The most recent survey of all three special hospitals found an 89% ‘overlap’ (and a kappa coefficient of 0.8) between legal class and clinical diagnosis (Reference Taylor, Leese and WilliamsTaylor et al, 1998). Thus, 85% of special hospital patients classified as mentally ill meet ICD–10 (World Health Organization, 1992) criteria for psychosis and only 3% of patients who meet those criteria do not carry a mental illness classification (Reference Taylor, Leese and WilliamsTaylor et al, 1998). Ninety-three per cent of men in Broadmoor special hospital and 97% of women in all three special hospitals who are classified as having a psychopathic disorder have one or more Axis II diagnoses under DSM–III (American Psychiatric Association, 1980). The mean numbers of DSM–III Axis II categories for those classified as having psychopathic disorder are 2.7 for men and 3.7 for women (Reference CoidCoid, 1992) and the most prevalent categories are antisocial, borderline and narcissistic (Reference Reiss, Grubin and MeuxReiss et al, 1999). All special hospital patients with ICD–10 severe learning difficulties have been found to be detained under severe mental impairment and 88% of those with mild to moderate learning disabilities are detained under mental impairment (Reference Taylor, Leese and WilliamsTaylor et al, 1998).

The first hypothesis was examined using any criminal conviction recorded within 10.5 years of discharge as the dependent variable. To investigate the second and third hypotheses the dependent variable was changed, first to reflect convictions received at different points over the 10.5 years and second to include only conviction on a serious charge. ‘Serious’ was operationally defined to comprise murder, attempted murder, threat or conspiracy to murder, manslaughter, wounding, grievous bodily harm, actual bodily harm (multiple), child stealing, buggery, attempted buggery, rape, indecent assault, incest, gross indecency with children, robbery, kidnapping, aggravated burglary and arson. Buggery, attempted buggery, incest and gross indecency with children do not require non-consensual contact. They were included because at the time the members of the sample were convicted they were usually charged in England and Wales in cases of child abuse where the necessity of the victim giving evidence was sought to be avoided.

Characteristics of the sample

Of the 425 patients in the sample, 349 (82%) were men, and the mean age on discharge was 38 years (range 18–75, s.d.=10.9). Further data were missing for one patient. The Mental Health Act 1983 allows a patient to be placed in more than one class. Only 24 (6%) of the sample had received a dual classification, however, suggesting that legal class had been allocated to reflect the primary diagnosis rather than the full range of the patient's psychopathology. One hundred and ninety (45%) had been classified as mentally ill, 141 (33%) as having psychopathic disorder, 72 (17%) as mentally impaired and 45 (11%) as severely mentally impaired.

Patients had been convicted of a mean of 4.6 and median of 2 offences (range 0–38, interquartile range 1–6) at the time of their discharge. On discharge, 281 (66%) went to be patients in other hospitals, 122 (29%) went home or to supported accommodation and 22 (5%) went back to prison. For patients categorised as mentally ill, the proportion becoming patients elsewhere was 68% and the proportion going home or to supported accommodation was 24%. For those categorised as having a psychopathic disorder these proportions were 60% and 36% respectively and for patients categorised as mentally impaired they were 76% and 23%. Eighty-seven (20%) had been convicted of an offence within 2.5 years of discharge, 118 (28%) within 6.5 years and 134 (32%) within 10.5 years. For patients with mental illness these proportions were 17%, 23% and 29% respectively; for patients with personality disorder they were 29%, 40% and 44% and for those with mental impairment they were 15%, 20% and 25%. When other variables were controlled for, the correlates of conviction for this sample were age, number of convictions at discharge and legal class of psychopathic disorder. Prior to controlling for other variables, correlates included additionally gender and destination on discharge (Reference BuchananBuchanan, 1998).

Statistical procedure

The variables and the order in which they were entered were chosen to place recognised risk factors for conviction (Reference Bonta, Law and HansonBonta et al, 1998; Reference Monahan, Steadman and SilverMonahan et al, 2001) in a sequence that reflected the resources required to obtain the information that they comprise. Logistic regression equations were fitted to the data with each of the outcomes as dependent variables. The equations used as independent variables: first, age and gender; then, in addition, the number of convictions the patient had received prior to leaving special hospital care; and, third, in addition, each patient's legal classification. The change in likelihood at each of the three stages was used to assess the statistical significance of the new information added (twice the change in log likelihood having an approximate chi-squared distribution with degrees of freedom equal to the change in the number of parameters).

The predicted probabilities of each patient being convicted were then used to generate the coordinates of a receiver operating characteristic curve. Receiver operating characteristic analyses describe the accuracy of a procedure for classification. The curves are generated by plotting sensitivity against 1 —specificity at different thresholds. When used in conjunction with logistic regression, as here, the area under the curve (AUC) is the probability that a randomly selected case will score higher than a randomly selected non-case on the linear predictor (that is, the risk factors combined using the regression weights).

Receiver operating characteristic analyses were developed in the 1950s to describe the performance of radar (Reference Altman and BlandAltman & Bland, 1994). They have been used to describe the independent contributions of Gleason score, prostate-specific antigen and clinical judgement in staging prostate cancer (Reference Partin, Kattan and SubongPartin et al, 1997; Reference Swets, Dawes and MonahanSwets et al, 2000), in the evaluation of screening programmes for hypercholesterolaemia (Reference Umans-Eckenhausen, Defesche and SijbrandsUmans-Eckenhausen et al, 2001) and in the identification of predictors of treatment response in multiple sclerosis (Reference Wandinger, Lunemann and WengertWandinger et al, 2003). They have also been used to describe the success of attempts to predict general (Reference Rice and HarrisRice & Harris, 1995) and sexual (Reference Hanson and ThorntonHanson & Thornton, 2000) recidivism. They have not been employed previously to describe the contributions of different types of information to the prediction of conviction in a psychiatric population.

The description of the accuracy of a procedure for classification that a receiver operating characteristic analysis provides through the AUC does not vary with the base rate and takes into account that the sensitivity and specificity of any procedure based on a quantitative measure (such as probability of conviction) will change with the threshold that the procedure uses in order to classify. We computed the areas under the receiver operating characteristic curve at each stage and bootstrapped each change in these with 1000 replications to obtain the 95% bias-corrected confidence interval (Reference Efron and TibshiraniEfron & Tibshirani, 1993). Where this excluded zero the change is noted as being significant at an approximate critical level of 0.05. Sensitivity and specificity were calculated using two probability cutoffs for predicting a case as positive: the first was 0.5 and the second was the value that maximised the sum of the sensitivity and specificity. This latter approach causes the choice of cut-off to change depending on the outcome, and is ‘optimal’ (in terms of the costs of misclassification) if the cost of a false negative result is the same as that of a false positive one (Reference AltmanAltman, 1995).

Using the sample prevalence, the number needed to detain (NND; Reference Buchanan and LeeseBuchanan & Leese, 2001) was then calculated for the optimal cut-off. The NND is the number of people who would have to be detained in order to prevent one person from being convicted. It is the inverse of positive predictive value and, unlike the AUC, is a base-rate sensitive measure that can only be calculated when the prevalence of the behaviour to be prevented is known. Also unlike the AUC, however, it describes the practical consequences of using a prediction technique to detain those thought likely to offend. Because the NND is sensitive to the rate of the behaviour to be prevented it varies with the length of time – in this case 2.5 years, 6.5 years or 10.5 years – over which a prediction is made.

Destination on discharge is associated with conviction in this sample (Reference BuchananBuchanan, 1998) and patients categorised as having a psychopathic disorder were more likely to be discharged to unsupported settings. The final models for 10.5 years were refitted including destination as an independent variable in case this could have been a confounding factor. Two sensitivity analyses were also conducted. In the first, the AUC and other measures of predictive ability were recalculated using jack-knifing. Jack-knifing is a method of cross-validation in which cases are left out one by one, the model re-estimated with each omission and the new model used to predict the omitted case. It was used to estimate the loss of predictive accuracy that would result from using the same prediction technique with a different sample. In the second sensitivity analysis the effect of dual classification was examined by treating legal class as a single, mutually exclusive, categorical variable. Finally, we examined whether the relative contributions of the different types of information derived from the sequence in which information was added to the model.

Limitations of the method

The method that we have used suggests at least four qualifications to the results that we report. First, the point of entry into the study of the members of our sample – discharge from special hospital – does not represent a rigid boundary between being at risk and not being at risk of conviction. More than half of these patients were discharged to other hospitals. Although some were convicted of acts committed in those other hospitals (Reference Buchanan, Taylor and GunnBuchanan et al, 2004), the risk of conviction differs, for instance in terms of the opportunity to offend, the vulnerability of potential victims and the likelihood of an offence being prosecuted, while the person remains an in-patient. Imprisonment or readmission during the follow-up period will have reduced the time at risk for some members of the sample but not for others. We have not corrected for either of these effects. Second, although diagnosis – in particular, a diagnosis of antisocial personality disorder – is the only clinically derived variable that correlates consistently with future offending (Reference Bonta, Law and HansonBonta et al, 1998), a patient's legal classification conveys only a fraction of the clinical information that can be collected in the course of an admission to psychiatric hospital. Similar analyses using more detailed clinical information may demonstrate a greater role for such information in the assessment of risk. Third, we used a long follow-up period. Although our subsidiary analyses using shorter follow-up periods failed to demonstrate a greater role for clinically derived information, none covered a period of less than 2.5 years. It is possible that the relative contributions of different types of information are different when predictions are sought over days or weeks. Finally, criminal conviction is only one of the proxy measures available for offending behaviour. Although there is little evidence that the correlates of offending vary according to which proxy is used (see Reference Farrington, Maguire, Morgan and ReinerFarrington, 1997), it is possible that a more sensitive, less specific measure such as arrest would produce different results.

This study and previous research

We have demonstrated the applicability of a statistical approach used elsewhere in medicine to the quantification of the contributions made by different kinds of information in the prediction of criminal conviction. The ability of the approach to quantify the relative contributions of those variables does not appear to be dependent on the sequence in which information is added to the regression equations. The results of jack-knifing suggest that the approach will be robust when used on other similar samples. The results nevertheless require replication using different samples and different variables, including variables that relate to a patient's clinical condition.

Our finding that demographic information and data from an individual's criminal record make significant contributions to the prediction of conviction on any charge, whereas legal class does not add to the accuracy that can be achieved using these two sources of information alone, is consistent with reviews suggesting that the long-term predictors of recidivism are the same for offenders with or without a psychiatric disorder (Reference Bonta, Law and HansonBonta et al, 1998). That this remains the case regardless of the period over which predictions are made (when that period is measured in years) is consistent with earlier work showing that the overall accuracy of predictions is independent of time at risk (Reference MossmanMossman, 1994; Reference Buchanan and LeeseBuchanan & Leese, 2001). The finding that legal class is more important in respect of serious convictions suggests, contrary to what had been thought to be the case, that predictors operate differently according to offence type.

For serious convictions the NND is higher than that for all convictions because the prevalence of serious convictions is lower and NND is a base-rate sensitive measure. The probability cut-off was chosen in a neutral way (to maximise the sum of sensitivity and specificity) and other valid choices could lead to lower values of NND, albeit at the expense of a higher false negative rate. It is possible that more sophisticated statistical methods, for example classification and regression tree analysis, would permit more accurate predictions to be made. Such methods demand substantially larger data-sets than logistic regression, however, in order to obtain robust predictors (Reference Thomas, Leese and WalshThomas et al, 2005).

Our AUC values should be compared with those reported elsewhere only with caution. The logistic model used for prediction has not been validated by applying it to a different sample from that used to estimate the parameters. The results of jack-knifing suggest that, at least for similar samples, the consequent loss of predictive accuracy would not be great, probably because in this case the sample size is reasonably large in comparison with the number of predictors. Such validation nevertheless invariably produces some ‘shrinkage’ in the AUC.

Mossman (Reference Mossman1994) described a mean AUC for all prediction techniques of 0.78 and areas of 0.71 and 0.67 for validated actuarial and clinical approaches respectively. We report AUC values for any offence and serious offences of 0.73 and 0.75 respectively. The regression equations in this study used fewer actuarial and clinical data than previous studies. With the above reservation, therefore, the results suggest that when long-term predictions of conviction on any charge are sought, the incremental value of information beyond that contained in basic demographic data and a person's criminal record may be limited. The results suggest also that the clinical information generating a legal classification of psychopathic disorder is more important, and criminal record less important, when only serious offences are sought to be predicted. We are not aware of a previous quantitative description of such an effect but the finding is consistent with suggestions that criminal record is a more potent predictor of general recidivism than of more serious crimes (Reference Bonta, Law and HansonBonta et al, 1998). It may be that a diagnosis of personality disorder, and hence a classification of psychopathic disorder, is operating as a proxy for histories of violence and other criminal behaviour of which the clinical team is aware but of which the person has not been convicted, and that this effect is more pronounced when the offence is serious. A second possibility is that the legal class ‘psychopathic disorder’ is operating as a proxy for ‘psychopathy’ (Reference HareHare, 1970) and that psychopathy is independently associated with conviction after discharge (Reference Harris, Rice and CormierHarris et al, 1991). It is unclear from previous research, however, that such an effect would be limited to serious offending (Reference Cooke, Michie and RyanCooke et al, 2001; Reference Stadtland, Kleindienst and KrönerStadtland et al, 2005).

A third possibility is that adding the legal class ‘psychopathic disorder’ permits the prediction model to benefit from the reduced heterogeneity of offending by people convicted of sexual offences (Reference Hanson and BussitèreHanson & Bussière, 1998). Our method classified the majority of sexual offences as serious (Reference BuchananBuchanan, 1998) and any ‘breeding true’ of sexual offending therefore increased the rate of serious offending disproportionately. Further inspection of our data suggested that this was the most likely explanation. Patients classed as having a psychopathic disorder were more likely than other patients to have been convicted of a serious sexual offence prior to discharge (psychopathic disorder 30%, mental illness 7.9%, mental impairment 21%, severe mental impairment 2%).

Assessment and resources

The quality of any risk assessment is limited by the availability of accurate information (Reference HollowayHolloway, 1997). The amount and quality of the information that can be obtained is limited, in turn, by resources. Some information required by actuarial techniques, such as age and gender, is readily available. Other data, such as those contained in a criminal record, are easy to obtain in some cases but not in others. Information such as diagnosis requires more detailed – and time-consuming – assessment. These results suggest that screening patients for a history of violent behaviour may be as effective as more complicated and resource-intensive assessments of the risk of future violence, at least where all types of conviction are sought to be prevented. This seems especially likely if, as others have suggested, patients themselves will report most of the violence that can be ascertained by other means (Reference Steadman, Mulvey and MonahanSteadman et al, 1998).

Implications for future research

Special hospital patients in England and Wales are atypical of psychiatric patients. Further research using this approach should examine whether the relative contributions of different types of information are the same for different samples. It should also use the full range of actuarial and clinical variables that have been shown to be correlated with conviction. Of the clinical data not examined here, a history of substance misuse seems the most obvious candidate to assist in the assessment of risk (Reference Harris and RiceHarris & Rice, 1997; Reference Steadman, Mulvey and MonahanSteadman et al, 1998). Not all aspects of a person's criminal record will be of equal importance, and age at first conviction (for instance) may be a more powerful predictor than total number of convictions received.

CLINICAL IMPLICATIONS

1. ▪ The level of accuracy in predicting conviction on any charge that can be achieved using socio-demographic and criminal record data alone appears to approach that reported using more sophisticated and resource-intensive methods.

2. ▪ In predicting conviction on any charge, such clinical information as is reflected in the Mental Health Act 1983 class of disorder adds little to the accuracy that can be achieved using socio-demographic and criminal record data alone.

3. ▪ Clinically derived data appear to add significantly to the accuracy of predictions when the convictions sought to be predicted are serious.

LIMITATIONS

1. ▪ Patients discharged from special (high secure) hospital are atypical.

2. ▪ The study used a limited amount of the information that is often available.

3. ▪ The study did not examine conviction over periods shorter than 2.5 years.