Development and evaluation of a screening tool for identifying prisoners with severe mental illness

AIMS AND METHOD 

To develop and evaluate a screening tool based on the observational skills of prison officers to identify adult male prisoners with severe mental illness. The tool was developed from open-ended interviews with officers and diagnostic interviews with prisoners. A case-comparison study was used to evaluate the tool. Fifty prisoners identified using the tool and 50 randomly selected prisoners underwent diagnostic interviews to determine the proportion in each group with severe mental illness.

RESULTS 

Five behavioural indicators of severe mental illness were identified and incorporated into the tool. In the evaluation, 19 out of 50 (38%) of the cases identified were found to have severe mental illness compared with none in the comparison group.

CLINICAL IMPLICATIONS 

The simple tool shows promise for the identification of prisoners with severe mental illness by prison officers. It does require further evaluation in other prison settings.


AIMS AND METHOD
To develop and evaluate a screening tool based on the observational skills of prison officers to identify adult male prisoners with severe mental illness. The tool was developed from open-ended interviews with officers and diagnostic interviews with prisoners. A case-comparison study was used to evaluate the tool. Fifty prisoners identified using the tool and 50 randomly selected prisoners underwent diagnostic interviews to determine the proportion in each group with severe mental illness.

RESULTS
Five behavioural indicators of severe mental illness were identified and incorporated into the tool. In the evaluation, 19 out of 50 (38%) of the cases identified were found to have severe mental illness compared with none in the comparison group.

CLINICAL IMPLICATIONS
The simple tool shows promise for the identification of prisoners with severe mental illness by prison officers. It does require further evaluation in other prison settings.
Many prisoners have severe mental health problems (Gunn et al, 1991;Maden et al, 1995;Birmingham et al, 1996;Office for National Statistics, 1998). Traditional methods of health screening at reception into prison are quite ineffective (Mitchison et al, 1994;Birmingham et al, 1997;Parsons et al, 2001). Once prisoners with mental illness find their way on to ordinary prison wings there is a significant risk that their mental health problems will remain unrecognised and their needs will remain unmet (Birmingham et al, 1998). A new prison reception health screen is being introduced (Grubin et al, 2002). This looks more promising than the traditional screen, but no matter how effective it is some prisoners with mental illness will pass through undetected. Others will develop mental illness at a later stage in prison.
The Department of Health sees the developing prison mental health in-reach services, which comprise multidisciplinary teams like community mental health teams, as the vehicle for improving mental healthcare for prisoners (Department of Health & Prison Service, 2001). According to the National Health Service Plan, 5000 prisoners at any one time should be in receipt of comprehensive mental health services in prison, all those with severe mental illness will be in receipt of treatment and no prisoner with serious mental illness will leave prison without a care plan and a care coordinator (Department of Health, 2000). In order to achieve this, prison mental health in-reach teams have first to identify those who have serious mental health problems. Reception health screening will play a key part, but other screening methods, for example using the observational skills of prison officers to identify prisoners with severe mental illness, could make an important contribution (Birmingham, 1999).

Participants
The study was conducted at Her Majesty's Prison (HMP) Winchester, a local prison housing adult male prisoners.
At the time of the study the prison held approximately 550 men (30% sentenced and 70% on remand). Of these prisoners, 500 were housed on four ordinary prison wings of roughly equal capacity.
The study involved prison staff and prisoners on the ordinary prison wings where we intended the screening tool to be used. Staff and prisoners in the prison healthcare centre and the segregation unit were therefore excluded.
Developing the tool This stage of the study was conducted between October and December 2002. We conducted open-ended interviews with 18 prison officers working on each of the ordinary prison wings at HMP Winchester to identify patterns of abnormal behaviour exhibited by prisoners with possible mental illness. We did this by asking officers to identify prisoners they considered to be odd, strange or with behavioural disturbances and to describe the behaviours they observed. We then conducted semistructured diagnostic interviews with ten of these prisoners to identify current, severe mental illness (functional psychotic disorders and major mood disorders). The semistructured diagnostic interview used consisted of a schedule for collecting demographic details based on that used in the Durham remand study (Birmingham et al, 1996) and a number of instruments used extensively in prison settings (Gunn et al, 1991;Maden et al, 1995;Office for National Statistics, 1998). These comprised the Schedule for Schizophrenia and Affective Disorders -Lifetime Version (SADS-L; Endicott & Spitzer, 1978) to generate current DSM-IV diagnoses of mental disorder (American Psychiatric Association, 1994); the Alcohol Use Disorders Identification Test (AUDIT; Saunders et al, 1993) to detect hazardous and harmful alcohol consumption in the previous year, and an adaptation of the Severity of Dependence Questionnaire (SODQ; Phillips et al, 1987) used in the Durham remand study to detect harmful use of illicit drugs over the same period (the interview schedule is available from the authors on request). DSM-IV Birmingham & Mullee Screening tool for prisoners with severe mental illness original papers diagnoses of personality disorder were based on supplementary questions included when there was an index of suspicion.
We selected out the behaviours identified by prison officers that were most consistently associated with a finding of severe mental illness (based on the semistructured diagnostic interview) and grouped these into five themes. We used these themes to develop the criteria in the tool. These criteria were written in the terminology used by the prison officers (Box 1). We included a sixth criterion to allow prison officers to specify any other symptoms that they thought were suggestive of mental illness.
Evaluating the tool A clinical research associate (and honorary specialist registrar) in forensic psychiatry (S.W., see Acknowledgements) made repeated visits to the four wings at HMP Winchester to evaluate the tool starting in January 2003. On each occasion she approached each of the officers on duty on the wing on an individual basis and asked them to identify every prisoner housed there who in their opinion met one or more of the criteria listed in the tool. Every prisoner identified as such was approached for inclusion in the study. Those who gave consent were administered the semi-structured diagnostic interview, described above, by S.W. She continued to visit the prison once or twice a week until 50 participants (cases) were recruited. She visited one wing on each occasion, visiting each wing in turn. None of the prisoners who participated in the evaluation of the tool had been interviewed to assist with its development.
At the same time S.W. recruited a comparison group of prisoners who did not satisfy any of the six criteria in the screening tool. She recruited one comparison prisoner for every case recruited. This was achieved by random selection (using a computer-generated list) from prison cell numbers to identify another prisoner housed on the same landing as each case. She administered the semi-structured diagnostic interview to those who gave consent.
Sample size and data analysis Due to lack of existing research in this field, the sample size of 50 per group was determined by the feasibility of data collection within the time-frame of the study.
The semi-structured diagnostic interview schedule was designed for processing by scanner (using Teleform, Verity, CA, USA). Data management and statistical analysis was performed using Statistical Package for Social Sciences (SPSS) for Windows (SPSS, Chicago). Chi-squared tests were used to test for association between mental health disorder and prisoner group (case or comparison).

Ethical issues
The study was approved by the Prison Health Research Ethics Committee and Southampton and South West Hampshire Local Research Ethics Committee.
The prison officers and prisoners who were approached to take part were given a verbal and written explanation of the study. A risk-based protocol was used to define the circumstances under which researchers would be required to disclose confidential information to the prison medical officer.

Development
Six of the ten prisoners interviewed during the development of the tool were found to have current, severe mental illness. The prison officers who participated in the study identified a number of behavioural abnormalities in these men that were attributed to severe mental illness. It was not difficult for us to divide these abnormalities into five separate themes from which the criteria in the tool were derived (Box 1). A sixth criterion (Any other symptoms that are likely to suggest the inmate has a mental illness?) was also included. Evaluation S.W. made 29 visits to the prison wings between January and October 2003 to recruit the 50 cases. A considerable number of prison officers participated in the study. The screening tool proved to be very easy to use with the officers who identified between one and three cases on each wing during each visit.
All 50 prisoners identified by officers as meeting one or more of the screening tool criteria (cases) gave consent and completed the interview. Of the 54 prisoners approached for the comparison group, 50 (93%) consented and were fully interviewed; 4 declined. Their demeanour, reasons for refusal and information contained within their prison inmate medical records did not suggest any of these men had a mental disorder.
The two groups of prisoners were comparable in terms of age, prisoner status and ethnicity (Table 1). Both groups were representative of the wider population at HMP Winchester in terms of prisoner status (30% sentenced and 70% on remand). Prisoners from the case group were more likely than those from the comparison group to be have been living in temporary accommodation or have been homeless prior to imprisonment. They were also more likely to be single than those in the comparison group. Of the 50 cases who were identified, 47 (94%) met two or more of the screening tool criteria. The remaining three cases were all said to exhibit persistently erratic or bizarre behaviour (criterion 2).
The prevalence of mental disorder was considerably higher among cases compared with the comparison group (Table 2). Of the 50 prisoners in the case group, 19 (38%) had severe mental illness (functional psychotic disorders or major mood disorders). None of the comparison group had severe mental illness.

Principal findings
We have developed a short and simple screening tool that has proved to be quick and easy to use with prison officers at HMP Winchester. During 29 prison visits conducted over a 9-month period, prison officers identified 50 men by using the tool. Of these men, 19 (38%) had severe mental illness whereas none of the 50 men in a randomly selected comparison group had severe mental illness. The case and comparison groups were similar in other respects, except for relationship and housing status prior to imprisonment. We attribute this to the fact that prisoners with mental illness are more likely to have been in unstable accommodation prior to imprisonment and to be single than their counterparts without mental illness (Birmingham et al, 1996).

Strengths and weaknesses of the study
This screening tool was developed to use the observational skills of prison officers rather than relying on officers having to interpret signs and symptoms of possible mental disorder. It was developed for use on ordinary prison wings where most prisoners are located. Prison officers and prisoners were actively involved in developing the screening tool and it was written in language that prison officers could easily understand. We are not aware of any other published studies of this nature.
Our study sample was reasonably large, and the refusal rate was very low. The main outcome measure (severe mental illness) was derived from interview data using diagnostic instruments that have been used extensively in prison research.
We acknowledge that our study has limitations. The ideal method for evaluating the tool would have been to carry out a large-scale cross-sectional survey of prisoners to establish the point prevalence of mental disorder in this population and allocating every participant to either the case group or the comparison group depending on prison officers' observations. This would have allowed us to calculate the sensitivity, specificity and predictive value Birmingham & Mullee Screening tool for prisoners with severe mental illness original papers  of the tool for identifying prisoners with severe mental illness. We concluded, however, that the constraints of the prison environment and the turnover of the prison population means that it would be virtually impossible to conduct a large enough cross-sectional study for this purpose. We accept that the design of our study is imperfect, but we believe that it was the only feasible way of carrying out an initial evaluation of the tool. The external validity of the tool is limited by the fact that it was developed using information gathered from a relatively small number of prison officers at one local prison and then evaluated at the same prison. This limits the generalisability of the findings to other male prisons and there is no evidence to support the use of the tool in young offender or female prisoner populations.
We did not use a diagnostic instrument for personality disorder. This was intentional because the study focuses on severe mental illness. The extra time taken and the nature of the questions asked would almost certainty have resulted in a higher refusal rate.
We did not systematically collect information from the prison officers concerning any prior knowledge they had regarding the mental health of the cases they identified. Therefore, we cannot exclude the possibility that factors such as this influenced their judgement when identifying cases. This did not appear to be so. The officers seemed to concentrate on the criteria in the tool and very few volunteered information that suggested that they knew anything more about the prisoners they identified.
Another limitation stems from the fact that we did not systematically record who among the case and control groups had been identified as having mental health problems at reception and who was already known by or had been referred to the mental health inreach team at the prison. It was our impression that a reasonable proportion of cases did fit this description, but a substantial minority of cases with severe mental illness were undetected.
Putting the findings into context National point prevalence studies of mental disorder in adult male prisoners which used the Schedule for Affective Disorders and Schizophrenia (Endicott & Spitzer, 1978) found that approximately 2.5% of sentenced prisoners and 6% of prisoners on remand had either a psychotic disorder or a major mood disorder (Gunn et al, 1991;Maden et al, 1995). If these findings are applied to HMP Winchester then it would be reasonable to expect that there would have been around 27 men with severe mental illness in the prison at any time during our study. During 2003, about half of the 30 prison healthcare centre beds were occupied by prisoners with severe mental illness. Therefore, we estimate that at any single point during the study there would have been approximately 12 prisoners with severe mental illness on the four prison wings. This amounts to about 1 in 40 prisoners.
Prison mental health in-reach teams need to find effective methods for identifying prisoners with severe mental illness who have been missed by prison reception health screening as well as those who have developed severe mental illness while in prison. If prison mental health in-reach staff were faced with having to screen every prisoner located on a prison wing for mental illness, then in a local prison, such as Winchester, around 40 prisoners would have to be screened to stand a chance of identifying one with severe mental illness. This approach is not practical or possible in any prison.
The design of our study does not allow us to calculate the specificity, sensitivity and predictive value of the screening tool. However, our findings suggest that this simple tool looks promising as a screening tool for use by prison mental health in-reach staff. Like the new prison reception health screen (Grubin et al, 2002), it does not rely on staff having to make judgements they are not necessarily trained to make, it uses simple yes/no criteria linked to a protocol.

Future research
We would stress that the tool needs to be evaluated further in other male prisons to determine whether it is effective when used by prison officers to identify prisoners with severe mental illness in these settings. It also needs to be tested in clinical practice when it is applied by prison mental health in-reach staff rather than a researcher. Similar tools should be developed for use in young offender and women's prisons.

Declaration of interest
The study was supported by a grant from the North and Mid Hants Health Authority.