Scientific consensus indicates that adults with serious mental illnesses, including schizophrenia, can validly and reliably self-report (Swanson et al. Reference Swanson, Swartz, Van Dorn, Elbogen, Wagner, Rosenheck, Stroup, McEvoy and Lieberman2006; Lincoln et al. Reference Lincoln, Ziegler, Lüllmann, Müller and Rief2010; Baumstarck et al. Reference Baumstarck, Boyer, Boucekine, Aghababian, Parola, Lançon and Auquier2013) intra-psychic, cognitive, and behavioural functioning, including drug use (Wolford et al. Reference Wolford, Rosenberg, Drake, Mueser, Oxman, Hoffman, Vidaver, Luckoor and Carrieri1999). Bahorik and colleagues (Reference Bahorik, Newhill, Queen and Eack2013) depart from this viewpoint, concluding that adults with schizophrenia significantly under-reported drug use in the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) study. However, there are reasons to question their findings.
Representativeness of analytical sample
Bahorik and colleagues' analytical sample consisted of 1042 patients with schizophrenia; the full CATIE trial included 1460 patients (Reimherr et al. Reference Reimherr, Swartz, Olsen, Stroup and Lieberman2010). Based on their stated sample selection criteria, they should have analysed a larger sample: at baseline there were 1445 urine drug screens (UDS) and 1130 radioimmunoassays (RIAs) of hair. The only way to arrive close to N = 1042 would be by first selecting those with a valid hair RIA. Moreover, comparisons between patients selected in and out of their analytical sample were not reported; characteristics may have differed systematically between those who did and did not have RIA results.
Selective inclusion and exclusion of UDS data
Bahorik and colleagues used RIAs of hair as a gold standard and first-order inclusion criterion, a measure that was missing for over one-fifth of all CATIE participants, but then discarded one-quarter of valid UDS data for the remaining participants. Specifically, they excluded participants who did not have a valid RIA of hair (even if they did have a valid UDS), but included UDS for participants who had a valid RIA of hair. If it is appropriate to include UDS for some participants, then it is appropriate to include UDS for all participants. If Bahorik and colleagues were interested in matching the 90-day RIA of hair detection period with the 90-day retrospective self-report, they should have excluded all UDS results, which generally detect use over a shorter period of time (DuPont & Baumgartner, Reference DuPont and Baumgartner1995; Verstraete, Reference Verstraete2004). Still, it has been argued that RIA of hair should not be used as a gold standard (Ledgerwood et al. Reference Ledgerwood, Goldberger, Risk, Lewis and Kato Price2008).
Conflicting results
Bahorik and colleagues' finding that, ‘rates of under-reported drug use are considerable’ is not replicable when using all available CATIE data, as shown in our prior paper (Van Dorn et al. Reference Van Dorn, Desmarais, Young, Sellers and Swartz2012), as well as in analyses undertaken for this letter. First, of 1448 patients for whom self-report and biological test data were available, 155 (10.7%) patients denied drug use, but had a positive biological test, while 114 (7.9%) patients self-reported drug use, but had negative biological tests, suggesting that patients were almost as likely to over-report as they were to under-report. Second, Bahorik and colleagues show a negative predictive value and a positive predictive value of 0.72 and 0.76, respectively, for any drug use; we found values of 0.89 and 0.61. They report sensitivity and specificity of 0.42 and 0.91; we found values of 0.68 and 0.86. Their kappa of 0.37 improved to 0.52 in our reanalysis. Third, we computed areas under the curve (AUC) of receiver-operating characteristic of self-report compared to biological tests and found acceptable, not poor, accuracy (AUC = 0.73, s.e. = 0.02, 95% CI = 0.70–77). While it is beyond the scope of this letter, there is also reason to question the validity of Bahorik and colleagues' multivariable results in light of these divergent findings. Fourth, Bahorik and colleagues do not report how they handled positive biological tests attributable to prescribed medication. Prior CATIE publications (Swartz et al. Reference Swartz, Wagner, Swanson, Stroup, McEvoy, Canive, Miller, Reimherr, McGee, Khan, Van Dorn, Rosenheck and Lieberman2006; Van Dorn et al. Reference Van Dorn, Desmarais, Young, Sellers and Swartz2012; Desmarais et al. Reference Desmarais, Van Dorn, Sellers, Young and Swartz2013) describe a protocol for handling such situations: participants who test positive for a prescribed medication in biological tests are not considered to be using. Given their claim of increased under-reporting compared to those reported herein and in prior CATIE publications, the possibility that Bahorik and colleagues (mis)attributed under-reporting to appropriate use of prescribed medications should not be overlooked.
Representation of extant research
Bahorik and colleagues' representation of the extant research also deserves comment. They assert that adults with schizophrenia are at increased risk of under-reporting drug use during an ‘acute crisis, but not when their symptoms were stabilized’, citing Stone and colleagues (1993); yet, there is nothing to support this statement in the Stone et al. paper. The authors also cite Møller & Linaker's (Reference Møller and Linaker2010) study of 48 patients, stating that approximately 14% under-reported ‘their use of drugs’. However, Møller & Linaker found that one in seven patients under-reported their drug use problems compared to clinician ratings on the Drug Use Scale (DUS), which is not the same as under-reporting their drug use. Moreover, research shows that DUS ratings frequently over-identify disordered use (Desmarais et al. Reference Desmarais, Van Dorn, Sellers, Young and Swartz2013). Bahorik and colleagues also misreport the findings of Galletly and colleagues (Reference Galletly, Field and Prior1993): one-quarter of patients – not all patients, as Bahorik and colleagues state – who had a positive UDS failed to disclose their drug use. Finally, Bahorik and colleagues' argument that self-report measures ‘could considerably underestimate actual rates of use’ in outpatient samples is not supported by contemporary research. Research on inpatient samples from the late 1980s and early 1990s showed under-reporting of drug use; however, more recent research concludes that self-report is an accurate approach to assessing drug use (Wolford et al. Reference Wolford, Rosenberg, Drake, Mueser, Oxman, Hoffman, Vidaver, Luckoor and Carrieri1999; Van Dorn et al. Reference Van Dorn, Desmarais, Young, Sellers and Swartz2012; Desmarais et al. Reference Desmarais, Van Dorn, Sellers, Young and Swartz2013).
Summary
Bahorik and colleagues' efforts to examine the potential for under-reporting drug use in the CATIE data are laudable. In contrast with their conclusions, findings presented herein and in our previous publications are consistent with previous research showing that individuals with schizophrenia can adequately and accurately self-report their substance use. In fact, biological tests add little incrementally to the accuracy of multi-modal assessment protocols (Wolford et al. Reference Wolford, Rosenberg, Drake, Mueser, Oxman, Hoffman, Vidaver, Luckoor and Carrieri1999; Van Dorn et al. Reference Van Dorn, Desmarais, Young, Sellers and Swartz2012; Desmarais et al. Reference Desmarais, Van Dorn, Sellers, Young and Swartz2013). As Bahorik and colleagues note, ‘much remains to be learned’ about the reporting of drug use in this population; however, their framing of the research, sample selection, findings, and conclusions misconstrue the relationship between self-report and biological tests in the CATIE data.
Acknowledgements
Funding for Dr Van Dorn's time was provided by NIDA Award number 1R03DA030850 to Dr Van Dorn. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIDA or the NIH.
Declaration of Interest
None.
