Criminally invalid: the treatment outcome profile form for substance misuse

AIMS AND METHOD 

To audit completion of the Treatment Outcome Profile (TOP) form in individuals attending substance misuse services in England. Forms are completed at the start of treatment and every 3 months thereafter. All forms at 3-drug treatment services were inspected over 6 months.

RESULTS 

Forms were inspected for 200 service users; 86% were fully completed. Two-thirds (67%) of service users had no declared funding for illicit drug use in the previous month (mean spending £988; s.e. = 149) despite denying any paid employment and criminal activity.

CLINICAL IMPLICATIONS 

The section on crime in the TOP form is unreliable and completely invalid.

To audit completion of theTreatment Outcome Profile (TOP) form in individuals attending substance misuse services in England. Forms are completed at the start of treatment and every 3 months thereafter. All forms at 3-drug treatment services were inspected over 6 months.

RESULTS
Forms were inspected for 200 service users; 86% were fully completed. Two-thirds (67%) of service users had no declared funding for illicit drug use in the previous month (mean spending »988; s.e. = 149) despite denying any paid employment and criminal activity.

CLINICAL IMPLICATIONS
The section on crime in theTOP form is unreliable and completely invalid.
In October 2007, the Treatment Outcomes Profile (TOP): A Structured Interview for the Evaluation of Substance Misuse Treatment form 1 became a mandatory instrument that is to be completed for all individuals in treatment for illicit drug use with the National Health Service (NHS) substance misuse services. The results are reported back to local NHS commissioning organisations (drug and alcohol action teams). The TOP form is a single-page questionnaire with four domains: substance misuse, injecting behaviour, criminal activity and health/social functioning. It is to be completed by the patient's key worker at first contact and then every 3 months. The National Treatment Agency for Substance Misuse explicitly states in its patient information leaflet (Treatment Outcome Profile (TOP): Making Your Drug Treatment Better) that the TOP form 'will enable treatment services to be assessed on the basis of the outcomes they achieve for their service users.' The TOP form was developed from the Maudsley Addiction Profile 2 and tested on several hundred service users before its introduction in October 2007.

Method
The TOP forms are completed at the start of treatment and every 3 months thereafter in all individuals attending NHS substance misuse services in England. The form is based on a semi-structured interview and quantifies drug use in terms of estimated quantity or spending over the preceding 28 days. It also makes an assessment of physical and mental health, criminal activity and a variety of health and social indicators over the preceding 28 days.
All available forms were inspected over the period October 2007 to March 2008 at three drug and alcohol services in South Essex. This included a mixture of TOP forms from initial attendance and 3-monthly reviews, and was part of an audit to ensure that the forms were correctly completed. Data from the forms were summarised, including the reported spending on class A drugs (heroin and cocaine). Spending on cannabis, amphetamine and alcohol was not analysed, as less than 10% of total spending in the sample involved these drugs, whereas opiate and cocaine use was the primary problem for 86% of the sample. Where necessary, quantity used was converted to street drug prices using information collated from a variety of sources. 3 Individual TOP forms were completed by the 24 key workers based at the three teams.

Results
We inspected the TOP forms for 200 individuals; 86% were fully completed. However, analysis of the audit results showed that 67% of the sample had no declared funding for class A drug use (other than benefits), with a mean spending of »988 (s.e. = »149) in the previous month despite denying any paid employment and denying any criminal activity (69% male, n = 123; mean age 34 years, s.e. = 0.5). For the sample as a whole (n = 200), mean spending was also »988 over the previous month (s.e. = 137; 73% male; mean age 32 years, s.e. = 0.5).

Discussion
Over the past decade, the UK government has been committed to reducing crime especially by targeting illicit drug use. 4 Drug treatment agencies have been the flagship of this approach, with massive increases in government spending. There are currently around 180 000 illicit drug users (mainly opiate users) in treatment with the 200 UK community drug and alcohol services (http://drugs.homeoffice.gov.uk/treatment/ strategy/?version=2). Government spending on these services in the 2006/07 financial year approached »600 million (www.nta.nhs.uk/about/funding/drug___ treatment___services.aspx). Results from the TOP forms, especially indices of criminal activity, are likely to be used by the Department of Health to justify this level of spending and inform funding decisions in the future.
The audit confirmed that the vast majority of TOP forms were completed to an acceptable level. Funding for British drug treatment agencies is now dependent on submitting the results from these forms and they are routinely entered into the database or 'return' that are sent each month to fund-holders. It is therefore reassuring, although not entirely unexpected, that these forms are being completed.
How to explain the considerable disparity between self-declared drug use and available income?
The results of the study indicate a massive disparity between income (legal and criminal) and spending on illicit drugs, with half of our sample declaring no funding despite a mean spending of over »900 every 28 days. Most unemployed people, including class A drug users, claim benefits of »200-300 per month. 5 It seems extremely unlikely that drug use at this level is funded by gifts from relatives or friends. The street drug prices were compiled from a variety of sources, including the National Criminal Intelligence Agency and 80 front-line drug agencies, police forces and drug action teams in 20 towns and cities throughout the UK. 3 The prices of typical quantities of illicit drugs are displayed for the major conurbations throughout Britain. As this information was obtained from reports of staff, outreach workers and service users attending local treatment agencies, it is likely to be reliable.
The other explanation for the disparity between reported drug use and spending is the falsification of information provided at interview by the service users. Admittedly, it is possible that users could exaggerate their consumption, but they would have little reason to do this to the extent revealed by the study. Moreover, other studies have suggested that typical class A drug users in Britain spend »40 per day (around »1000 per month) on heroin and often more on cocaine. 6,7 Many surveys have concluded that self-reporting of drug use is reliable and valid when events are recent and individuals do not face negative consequences. 8^12 For example, the UK National Treatment Outcome Research Study reported that the concordance rate between self-reported use and results from a urine specimen for heroin and cocaine was 92%. Only 2-3% of clients who reported not using heroin or cocaine tested positive. 13 Hser et al assessed the reliability of retrospective self-report by 323 drug addicts at a 10-year interval with respect to opiate use and criminality. The diagnostic accuracy and predictive power of the self-report was found to be very high (over 80%). 14 In regard to self-report of cocaine use, Simpson et al reported that 8.8% of people denied cocaine use but subsequently tested positive. This was based on a random sample of 352 interviewees who presented with cocaine misuse as part of the US Drug Abuse Treatment Outcome Study (DATOS). 15 However, these results are based on prolonged interviews in research samples rather than brief initial assessments in routine clinical teams where self-report may be less reliable.
The most likely explanation for the disparity between drug use and funding is that individuals are not reporting criminal activity. There is a vast body of research on self-report and concealment of delinquent and deviant activity including crime and sexual behaviour. 16^20 It is also self-evident that many service users who are on court diversion programmes (such as the drug rehabilitation requirements) are unlikely to admit to illegal activity as this will be reported automatically to the court during their monthly reviews.

TOP and MAP
The Maudsley Addiction Profile (MAP) is a well-validated, brief, semi-structured interview developed to assess the substance use, risk-behaviour, health and social functioning of illicit drug users. 6 A health professional is required to complete the various sections, which takes approximately 12 minutes. There is no requirement for formal training to use the MAP. There is considerable overlap between the MAP and the TOP form and they were produced by many of the same authors. Both instruments are completed by clinicians following an interview with the clients, both have excellent psychometric properties (with correlation coefficients exceeding 0.75 between different indices) and have been validated in samples of several hundred clients at more than 50 community drug and alcohol services. 1,6 They are relatively short (although TOP is significantly shorter) and are eminently practicable for use in front-line clinical services, although the researchers were frequently made aware of complaints by staff regarding the ever increasing and often irrelevant paperwork required by substance misuse services and mental health services in general. It would be highly desirable if the TOP form could be used to replace, rather than supplement, some of the other documentation often known as the 'basic data-set'.
Are self-reports valid? Problems with the validity of self-reported criminal activity in routine clinical settings have previously been Luty et al Criminally invalid original papers observed in relation to the Maudsley Addiction Profile. 6 Of the 206 service users in this study, 96 were not working and denied committing any acquisitive crimes in the previous month, although they spent an average of »595 (s.e. = 96) on heroin and cocaine over this period. It is difficult to see how they were able to fund this illicit drug use without recourse to criminal activity, particularly as their total income from benefits amounted to only »230 per month each, and this includes payments to cover food and accommodation. Even with enhancements for disability or child support, this amount remains less than »500 per month. 5 It is possible that these individuals were receiving an undeclared income from informal work as well as fraudulently claiming unemployment benefits. However, it seems unlikely that they were willing to reveal the amount they were spending on illicit drugs and yet withhold information on undeclared employment.

Conclusion
The effectiveness of NHS community drug and alcohol services will be assessed using data derived from the TOP forms. Moreover, future government policy and funding is likely to be determined by reports based on the TOP form (especially criminal activity), hence it is important that this information is valid. Unfortunately, our study indicates that data on criminal activity from the TOP form are invalid assuming the results of this study are typical throughout the UK. The criminal activity section of the TOP form is unreliable and completely invalid.