Alcohol problems are a significant public health problem. Reference Room, Babor and Rehm1,Reference Saitz2 Alcohol consumption has been estimated to cause 3.8% of all deaths and 4.6% of disability. Reference Rehm, Mathers, Popova, Thavorncharoensap, Teerawattananon and Patra3 Alcohol problems in general include alcohol use disorder, alcohol dependence and acute intoxication. Alcohol use disorders include a spectrum of excessive drinking often also described as alcohol abuse (DSM-IV), 4 hazardous drinking (WHO) 5 or harmful drinking (ICD-10) 6 (see Appendix 1). In the general population hazardous drinking is seen in 30–40%, Reference Compton, Conway, Stinson, Colliver and Grant7 with lifetime alcohol misuse or dependence found in about 10% compared with 16–36% of out-patients. Reference O'Connor and Schottenfeld8,9 In primary care approximately 7–30% of attendees have at-risk drinking or an alcohol use problem. Reference Saitz2,Reference Reid, Fiellin and O'Connor10–Reference Fiellin, Reid and O'Connor12 In hospital settings the point prevalence of alcohol use disorder varies between 7 and 25% Reference Gerke, Hapke, Rumpf and John13–Reference Santora and Hutton18 and approximately 4% have alcohol dependence. Reference Coder, Freyer-Adam and Bischof19 In psychiatric out-patients with serious mental illness, a recent meta-analysis showed rates of 10% (current use) and 20% (lifetime use) for alcohol use disorder. Reference Koskinen, Löhönen, Koponen, Isohanni and Miettunen20 The highest rates are seen in psychiatric in-patients where prevalence rates may be as high as 50%. Reference Barry, Fleming, Greenley, Widlak, Kropp and McKee21–Reference McCloud, Barnaby, Omu, Drummond and Aboud23 In spite of these high prevalence rates it seems that only a minority of alcohol problems are detected and treated. Studies conducted in the USA, Australia, UK and Finland indicate that clinicians frequently do not screen for alcohol use disorder and fail to address the problem in at least a third to a half of cases even when the diagnosis is known. Reference Kaner, Heather, Brodie, Lock and McAvoy24–Reference Rydon, Redman, Sanson-Fisher and Reid28 In most cases, diagnosis is made by clinical judgement without the use of scales, blood tests or reference to diagnostic criteria. Reference Berner, Zeidler, Kriston, Mundle, Lorenz and Härter29,Reference Aalto and Seppa30 Patient surveys suggest that only 30–40% are asked about their alcohol habits Reference Deitz, Rohde, Bertolucci and Dufour31–Reference Edlund, Jürgen and Wells33 and a small percentage of those with alcohol problems report receiving advice to cut down. Reference Hasin, Grant, Dufour and Endicott34 Several effective treatment packages including brief alcohol interventions have been developed and are potentially effective. Reference Wallace, Cutler and Haines35–Reference D'Onofrio and Degutis37 However, such interventions can only be effective when alcohol problems are recognised. Numerous studies of screening tools and biomarkers have been conducted but it is important to clarify how much improvement in the identification of alcohol problems occurs with their use above and beyond that achieved from routine clinical judgement alone. Reference Neumann, Gentilello, Neuner, Weiss-Gerlach, Schürmann and Schröder38 Early research suggests that about a third of individuals with alcohol problems are detected by their general practitioner. Reference Rydon, Redman, Sanson-Fisher and Reid28 The comparable detection rate from general hospital and psychiatric settings is unknown, although some previous work has suggested that hospital specialists detect most people with drinking problems at admission. Reference Nielsen and Gluud39
The aim of this study was to clarify accuracy of clinical judgement as well as clinical recording of clinicians working in (a) primary care, (b) general hospital and (c) psychiatric settings in identifying broadly defined alcohol problems. We hypothesised that mental health specialists would have the greatest success and primary care doctors the least success when working without assistance of scales or tools.
Inclusion and exclusion criteria
The principal inclusion criteria were studies that examined the clinical judgement and clinical recording of clinicians in the identification of those with alcohol problems. We defined alcohol problems broadly to give a representative picture of routine practice. We grouped these problems into alcohol use disorder with and without dependence. We allowed studies to use a self-report or interview-based criterion definition of alcohol use disorder. Although we did not specifically exclude ICD-10-based studies, all interview-based studies used DSM criteria (Appendix 1). In an attempt to minimise heterogeneity we looked at studies examining acute intoxication and alcohol dependence separately from alcohol use disorder without intoxication or dependence. We excluded studies that did not present data on alcohol use, were based on vignettes, had insufficient primary data or failed to employ an adequate criterion standard (e.g. studies based on self-reported alcohol use). Reference Ramsay, Vredenburgh and Gallagher40–Reference Cheeta, Drummond, Oyefeso, Phillips, Deluca and Perryman46
Search and information sources
A systematic search, critical appraisal and meta-analysis were conducted. The following abstract databases were searched from inception to September 2011. In MEDLINE/Embase (266 hits) and PsycINFO (20 hits), relevant keywords, MESH terms, title terms and limits were applied (available from the author on request). In four full-text collections (Science Direct, Ingenta Select, Ovid Full text, Blackwell-Wiley Interscience) the search terms were used as a full text search and citation search (261 hits). The abstract databases SCOPUS (179 hits) and Web of Knowledge (113 hits) were searched, using relevant search terms as a text word search, and using key papers in a reverse citation search. Non-English language articles were searched and one relevant study was found. We also contacted several experts in the field for unpublished and very recently published work.
Data extraction and appraisal
We adhered to standards in the PRISMA guidelines for meta-analyses. Reference Moher, Liberati, Tetzlaff and Altman47 Data extraction was conducted independently by two authors using a data extraction form in Microsoft excel. The form was developed from previous systematic reviews of diagnostic accuracy according to principles of PRISMA and the Cochrane Collaboration (available from the author on request). Reference Elamin, Flynn, Bassler, Briel, Alonso-Coello and Karanicolas48 Variables extracted were country of study, setting (e.g. primary care, general hospital), patient characteristics (e.g. age, gender), reference standard (including cut-off if relevant), method used to determine clinician judgement, sample size, positive cases and negative cases (as identified by reference standard), sensitivity, specificity, true positives (i.e. clinician judgement and reference standard both suggest alcoholism), false positives (i.e. clinician judgement suggests alcoholism but reference standard does not), false negatives (clinician judgement indicates no alcoholism but reference standard suggests alcoholism) and true negatives (both clinician judgement and reference standard both judge no alcoholism). To establish validity of the data extraction for the primary outcomes, true positive, false positives, false negatives and true negatives extracted from papers were recalculated from prevalence, sensitivity and specificity data in order to identify any inconsistencies or errors in extraction. Any inconsistencies were resolved by double-checking data from the paper and discussion with one of the authors (A.J.M.). Appraisal of each article was conducted by all authors independently using QUADAS. Reference Whiting, Rutjes, Reitsma, Bossuyt and Kleijnen49 This is a standardised quality appraisal form and is the recommended tool for a number of organisations such as the Cochrane Collaboration and the National Institute for Health and Clinical Excellence.
Given high heterogeneity, we used random effects bivariate meta-analysis to synthesise the data and provide pooled estimates of sensitivity and specificity using the metandi commands in Stata 10 on Windows. This method fits a two-level model, with independent binomial distributions for the true positives and true negatives conditional on the sensitivity and specificity in each study, and a bivariate normal model for the logit transforms of sensitivity and specificity between studies. Reference Reitsma, Glas, Rutjes, Scholten, Bossuyt and Zwinderman50 A summary receiver operator characteristic (ROC) curve, where each data point represents a separate study, was then constructed using the bivariate model to produce a 95% confidence ellipse within ROC space. Heterogeneity was assessed using the I Reference Saitz2 statistic. Reference Higgins, Thompson, Deeks and Altman51 Partial verification bias, differential verification bias and incorporation bias was assessed for each study (online Table DS1). Finally, publication bias was assessed formally using Begg-Mazumdar’s test. Reference Begg and Mazumdar52
Study description and methods
We identified 48 studies of clinical accuracy including 39 on alcohol use disorder, 4 concerning intoxication and 5 examining alcohol dependence (Fig. 1). The sample size of individual studies ranged from 35 to 3014 individuals (mean 490.6, s.d. = 644.7). Twenty-one studies identified alcohol use disorder on the basis of a structured interview, four intoxication studies used blood alcohol concentration and the remainder used self-report measures of alcohol use (online Table DS2). High heterogeneity was found for most analyses (I ranged from 92.0 to 94.5% depending on analysis). Publication bias was assessed using funnel plots but no evidence of bias was detected (Fig. 2).
We examined the prevalence of each type of alcohol problem according to setting. In primary care the pooled prevalence of alcohol use disorder was 16.7% (95% CI 10.0–24.6). It was 12.1% (95% CI 7.2–18.0) when identified by interview and 22.7% (95% CI 10.5–37.9) when identified by self-report. In hospital settings the prevalence of alcohol use disorder was 33.5% (95% CI 16.2–53.5%) (Martin et al Reference Martin, Heymann, Neumann, Schmidt, Soost and Mazurek53 was excluded from the prevalence calculation as the study pre-selected a high-risk sample). However, the prevalence was 43.4% (95% CI 5.1–87.5) when identified by interview and 28.1% (95% CI 20.5–36.5) when identified by self-report. In mental health settings the prevalence of alcohol use disorder was 21.7% (95% CI 10.4–35.7), with insufficient studies to stratify by self-report/interview. Finally, the prevalence of alcohol dependence was 12.1% (n = 4, 95% CI 9.3–15.1) in hospital settings, whereas the prevalence of intoxication reached 52% (n = 2, 95% CI 7.7–94.0) identified by analysing blood alcohol levels in emergency departments.
Identification by primary care physicians
Alcohol use disorder
Across 12 studies, involving 10 997 people with problem drinking, primary care physicians recorded alcohol problems in medical records in 27.3% (95% CI 16.9–39.1), but actually recognised alcohol use disorder in 41.7% (95% CI 23.0–61.7) using clinical judgement (Table 1). In studies relying on interview-based gold standard detection the sensitivity was 44.0% (95% CI 21.4–68.0). There were only two studies with specificity data, both based on clinical judgement, and primary care physicians correctly reassured 93.1% (95% CI 86.7–97.6) of people without a drinking problem (detection specificity). At a prevalence of 20%, the positive predictive value would be 60.2% and the negative predictive value 86.5%. Thus, a primary care practitioner would typically identify 8 cases, missing 12. They would correctly identify 75 non-drinkers, falsely diagnosing 5. Thus the fraction correctly identified would be 83%.
|Alcohol use disorder||Alcohol dependence||Alcohol intoxication|
|Professional group||Sensitivity % (95% CI)||Specificity % (95% CI)||Sensitivity % (95% CI)||Specificity % (95% CI)||Sensitivity % (95% CI)||Specificity % (95% CI)|
|Primary care physicians||41.7 (23.0–61.7)||93.1 (86.7–97.6)||No studies||No studies||No studies||No studies|
|Hospital specialists||52.4 (35.9–68.7)||88.2 (80.9–93.9)||No studies||No studies||89.8 (70.4–99.4)||61.4 (11.4–98.7)|
|Mental health professionals||54.7 (16.8–89.6)||83.6 (56.3–98.8)||No studies||No studies||No studies||No studies|
|Clinical recording (chart notation)|
|Primary care physicians||27.3 (16.9–39.1)||No studies||No studies||No studies||No studies||No studies|
|Hospital specialists||37.2 (28.4–46.4)||95.2 (94.2–96.1)||41.7 (16.5–69.5)||No studies||75.6 (68.1–82.3).||No studies|
|Mental health professionals||28.2 (15.0–44.9) a||No studies||No studies||No studies||No studies||No studies|
a Based on one study only.
Alcohol dependence and alcohol intoxication
No studies were found.
Identification by hospital staff in medical settings
Alcohol use disorder
Across 23 studies involving 10 837 people with alcohol use disorder, health professionals correctly recorded alcohol problems in 37.2% (95% CI 28.4–46.4) of case notes. Based on clinical judgement, at interview they correctly identified 52.4% (95% CI 35.9–68.7). There was no difference in detection sensitivity when stratified by gold standard (robust clinical interview) 41.2% (95% CI 28.8–54.2) v. self-report 42.7% (95% CI 31.1–54.6). In nine studies reporting specificity, hospital staff identified 93.1% (95% CI 89.1–96.3) of those without problem drinking with no difference by case ascertainment or outcome method.
At a prevalence of 20%, the positive predictive value would be 52.5% and the negative predictive value 88.2%. Thus, hospital doctors would typically identify 10 cases, missing 10. Hospital doctors would typically correctly identify 71 non-drinkers, falsely diagnosing 9. Thus, the fraction correctly identified would be 81%.
Alcohol dependence and alcohol intoxication
There were four studies assessing the identification of alcohol dependence in hospital settings. Hospital staff accurately recorded 41.7% (95% CI 16.5–69.5) of such cases in medical notes. However, no data were reported on specificity in these settings.
Four studies examined the ability of doctors and nurses working in trauma centres to identify acute alcohol intoxication (defined by a high blood alcohol concentration). Health professionals were able to identify intoxication in 89.8% (95% CI 70.4–99.4) of cases based on clinical judgement and recorded this in the notes in 75.6% (95% CI 68.1–82.3). However, their specificity (based on clinical judgement) was low at 61.4% (n = 2, 95% CI 11.4–98.7).
Identification by mental health professionals
Alcohol use disorder
We located four studies involving a small sample of 784 patients, but as there was only one study using a chart review method (showing a sensitivity of 28.2%), we were only able to pool studies
of clinical judgement, reducing the sample size to 384. Mental health professionals identified 54.7% (95% CI 16.8–89.6) of those with alcohol use disorder. Based on clinical judgement, their detection specificity was 83.6% (95% CI 56.3–98.8). At a prevalence of 20%, the positive predictive value would be 45.4% and negative predictive value 88.1%. Thus, a mental health professional would typically identify 11 cases, missing 9. They would correctly identify 67 non-drinkers, falsely diagnosing 13. Thus, the fraction correctly identified would be 78%.
Alcohol dependence and alcohol intoxication
Only one small study was found. Rienzi Reference Rienzi54 reported that mental health practitioners had a sensitivity of 82.9% (95% CI 67.3–91.9) when looking for self-reported alcohol dependency (defined using the Michigan Alcoholism Screening Test (MAST)).
Figure 3 shows the results for the hierarchical summary ROC curve analysis for clinical identification of alcohol use disorder.
We found 39 studies examining the clinical identification of alcohol use disorder, 5 studies involving alcohol dependence and 4 involving acute alcohol intoxication, with a total sample of 23 472 participants. Although the overall sample size was large there was sparse data on dependence and intoxication, especially in mental health settings (Table 1). Our findings indicate that all healthcare professionals have considerable difficulty with the identification of problem drinking in clinical practice, identifying about half of those with alcohol use disorder based on clinical judgement and correctly recording alcohol use disorder in the notes in only one in three cases. It should be noted that this data were based on single assessments to inform clinical opinion. Only one study examined the effect of cumulative assessment, finding that detection sensitivity improved from 16 to 34% after three consultations. Reference Kip, Neumann, Jugel, Kleinwaechter, Weiss-Gerlach and Guill56 Previous studies have found that clinicians have most difficulty identifying individuals with milder alcohol problems and better success with dependence but we found no significant difference. Reference Cheeta, Drummond, Oyefeso, Phillips, Deluca and Perryman46 The most successful group were emergency department specialists who were able to identify acute alcohol intoxication in nine out of ten people. This is in itself important, as at least 33% of people seen in the emergency department for trauma have evidence of legal intoxication. Reference Gentilello, Villaveces, Ries, Nason, Daranciang and Donovan55,Reference Reyna, Hollis and Hulbus57 However, even here emergency department specialists made a significant number of false positive errors.
Using clinical judgement primary care physicians identified about four in ten of attendees with alcohol use disorder, although their medical records were accurate in less than three out of ten. This is in line with previous work suggesting that most of those presenting in primary care are not detected routinely. Reference Rydon, Redman, Sanson-Fisher and Reid28 Recognition by hospital specialists and mental health professionals has been much less discussed. Reference Freimuth44 Hospital staff also had difficulty with recognition with about half of people with alcohol use disorder identified. This is important because alcohol use disorder can exacerbate severity of illness and prognosis in people with several mental disorders such as schizophrenia and depression. Reference Potvin, Sepehry and Stip58–Reference Dixon60 Our finding of lower accuracy in indentifying problems among mental health professionals may be surprising but it has been previously found that alcohol problems are often not discussed even in mental health settings. Reference Lubman, Hides, Jorm and Morgan61,Reference Weisner and Matzger62 Further, most mental health practitioners are not expert in alcohol problems, rarely use standardised instruments for alcohol problems and have variable access to dedicated specialist alcohol services. 63–Reference Berner, Langlotz, Kriston and Härter65
Barriers to the recognition of alcohol problems
Many factors have been cited as barriers to appropriate and prompt recognition. These include clinician confidence as to what constitutes alcohol misuse, Reference Rush, Ellis, Crowe and Powell66 inadequate training, Reference Cornuz, Ghali, Di Carlantonio, Pecoud and Paccaud67 lack of contractual incentives, Reference Wilson, Lock, Heather, Cassidy, Christie and Kaner68 lack of time, Reference Aira, Kauhanen, Laricaara and Rautio69 fear of labelling due to the stigma associated with substance misuse Reference Bander, Goldman, Schwartz, Rabinowitz and English70 and a belief that patients will not honestly disclose their drinking practices. Reference Beich, Gannik and Malterud71–Reference Ferguson, Ries and Russo73 In most cases patients accept being questioned about their drinking habits. Reference Miller, Thomas and Mallin74,Reference Makela, Havio and Seppa75 Our data on the similar prevalence of alcohol problems by self-report compared with interview and lack of substantial diagnostic differences by criterion reference do not support the hypothesis that people will not disclose their drinking history if asked in a sensitive manner. Therefore we suggest that the most significant modifiable predictor remains the willingness of the clinician to ask about alcohol habits appropriately. Reference Rush, Powell, Crowe and Ellis76 In self-report surveys, health professionals report that they often enquire about drinking behaviours. Reference Herbert and Bass77–79 Yet they also express the belief that clinical questions will not be answered honestly by patients Reference Bander, Goldman, Schwartz, Rabinowitz and English70 and are concerned that asking about drinking might harm the patient–provider relationship. Reference Thom and Tellez72,Reference Arborelius and Thakker80
Observational studies of clinician enquiry show that, in general, screening for alcohol problems is not routine in primary care Reference Berner, Zeidler, Kriston, Mundle, Lorenz and Härter29,Reference Aalto and Seppa30,Reference Bradley, Curry, Koepsell and Larson81,Reference Wenrich, Paauw, Carline, Curtis and Ramsey82 or in specialist settings. Reference Huang, Yu, Chen, Chen, Shen and Chen83 Several studies found that clinicians discuss alcohol use in about 10–15% of consultations but few discussions are specifically prompted by concerns over drinking habits. Reference Vinson, Elder, Werner, Vorel and Nutting84–Reference Arndt, Schultz, Turvey and Petersen86,Reference Huang, Yu, Chen, Chen, Shen and Chen83 On videotaped or observed interviews, alcohol-related discussions were often superficial and yielded little information regarding patients’ drinking practices. Reference Wenrich, Paauw, Carline, Curtis and Ramsey82,Reference Larsson, Saljo and Aronsson87,Reference Lawner, Doot, Gausas, Doot and See88 Interviews where at-risk drinking discussions took place typically lasted only 1–2 min. Reference Bradley, Epler, Bush, Sporleder, Dunn and Cochran89,Reference Vinson, Galliher, Reidinger and Kappus90 Of those clinicians that look for alcohol problems, nearly all prefer asking quantity–frequency questions, about a third say they use the CAGE questions, and 15% cite use of biochemical markers. Reference Huang, Yu, Chen, Chen, Shen and Chen83,Reference Reid, Tinetti, Brown and Concato91,Reference Spandorfer, Israel and Turner92 D’Amico et al examined the practices of physicians towards over 7000 individuals visiting them. Reference D'Amico, Paddock, Burnam and Kung93 The practitioners asked 29% about their drinking (and 44% of problem drinkers about their drinking) over 1 year. Of those asked about alcohol problems, only 21% received advice (49% in the case of problem drinkers). Reference D'Amico, Paddock, Burnam and Kung93 Clinicians are least likely to raise the subject of problem drinking with White people, women and widows, Reference Arndt, Schultz, Turvey and Petersen86 prioritising discussion with healthier, younger males who misuse tobacco and alcohol. Reference Bertakis and Azari94 Less studied is the issue of whether clinicians may also find distinguishing problematic alcohol use from non-problematic use difficult.
In response to these concerns the Institute of Medicine, the National Institute on Alcohol Abuse and Alcoholism (NIAAA), the American Medical Association and the American Society of Addiction Medicine have all recommended that clinicians routinely ask patients about alcohol use. 79,95–97 However, the Scottish Intercollegiate Guidelines Network advocates clinical assessment with judicious use of questionnaires only where there is suspicion of alcohol problems. 98 The NIAAA and the US Preventive Services Task Force (USPSTF) recommend population screening to identify problem drinking; that is, clinicians should ask all attendees whether they drink, and assess the specific quantity, frequency and pattern of consumption, but they did not recommend a specific tool. 99 In 2004 the USPSTF recommended that screening should be accompanied by behavioural counselling interventions to reduce alcohol misuse by adults in primary care settings. 100 The NIAAA also recommended targeted screening (case finding) in that all patients who drink alcohol should be screened with the CAGE questions. 101 To date, variations of the AUDIT (Alcohol Use Disorder Identification Test), CAGE and MAST have been the most common questionnaires for alcohol problems but these tools are difficult to use in a primary care practice. Reference Thom and Tellez72,79,Reference Wenrich, Paauw, Carline, Curtis and Ramsey82,Reference Beich, Thorsen and Rollnick102,Reference Phelps and Johnson103 No single laboratory test or combination of tests has been shown to be appropriate for screening. Reference Hoeksema and De Bock104,Reference Reynaud, Schwan, Loiseaux-Meunier, Albuisson and Deteix105
The main limitation in this data synthesis is lack of data from some settings and a lack of consistency in terminology for alcohol use disorder. Reference Kunda106,Reference Morse and Flavin107 It is disappointing that few studies were conducted in Europe, and none in the UK. Some problems in terminology are to be anticipated given we have examined studies spanning more than 25 years of clinical practice. A second limitation is the reliance on self-report criterion methods such as the CAGE, AUDIT and MAST in some studies. However, we adjusted for this by examining both interview-based and self-report standards separately. In primary care and medical settings there was no difference in sensitivity or specificity but in mental health settings, in studies relying on an interview-based gold standard, detection sensitivity was significantly lower 36.0% (95% CI 16.5–58.2) compared with self-report 79.8% (95% CI 70.0–88.1). In addition, we found no statistically significant difference in the prevalence of alcohol use disorder whether defined by interview or self-report.
Given the clear findings that most, if not all health professionals struggle to accurately identify those with alcohol problems (including dependency) is there any evidence that interventions improve recognition rates? There is some evidence that education programmes can improve diagnostic habits. Reference Seale, Shellenberger, Boltri, Okosun and Barton108 A meta-analysis of 15 methods in 12 trials aimed at increasing primary care practitioner rates of screening for, and giving advice about, hazardous and harmful alcohol consumption concluded it is possible to increase the engagement of physicians although effects of identification of alcohol problems were unclear. Reference Anderson, Laurant, Kaner, Wensing and Grol109 There is an urgent need to trial combined screening, education and brief alcohol interventions in adequate samples in both mental health and general hospital settings. We suggest that such trials specifically measure detection sensitivity and detection specificity as well as patient outcomes. We also suggest that such trials compare the performance of screening against unassisted clinical accuracy in order to clarify which methods most help clinicians identify people with problem drinking.
|Our catagorisation||Source definition|
|Alcohol use disorder||Hazardous use (WHO) 5|
|Alcohol abuse (DSM-IV-TR) 4|
|Alcoholism or alcoholic (primary authors’ own definition, from original publications, see online Table DS2)|
|Risky drinking or at-risk drinking (primary authors’ own definition, from original publications, see online Table DS2)|
|Alcohol use disorder with dependence||Alcohol dependence (DSM-IV-TR) 4|
Thanks to the staff of the postgraduate library, Leicester General Hospital. Thanks also to Alex Sutton, Professor of Medical Statistics, University of Leicester for statistical advice.