I. Introduction
Alcohol abuse is one of the leading causes of death, especially among males and younger consumers. According to the World Health Organization, more than 200 health conditions are connected to harmful alcohol use, including liver and cardiovascular diseases, road injuries and violence, cancers, suicides, tuberculosis, and sexually transmitted diseases (WHO, 2019). The Centers for Disease Control and Prevention (CDC) reports that “more than 140,000 people die from excessive alcohol use in the U.S. each year” (CDC, 2022). Worldwide, a similar count by the WHO amounts to 3 million deaths, more than tuberculosis and HIV/AIDS (WHO, 2019). The picture becomes even more worrisome if we expand the analysis to the negative consequences on chronic diseases, fetal alcohol spectrum disorders, mental health, and other issues of interest for policy makers. According to the CDC, if we consider the sole healthcare, workplace productivity, collisions, and criminal justice, the costs of excessive alcohol use represent a 249 billion $/year loss for the U.S. economy, while the OECD estimates that in the OECD countries the GDP is “1.6% lower due to diseases caused by alcohol consumption above the 1–1.5 drinks per day cap” (WHO, 2019).Footnote 1
While excessive alcohol consumption has indisputable negative consequences, the effect of moderate alcohol consumption is less clear. For a long time, a consistent body of literature has shown the positive or null effect of moderate consumption (Barboriak et al, Reference Barboriak, Anderson and Hoffmann1979; Baum-Baicker, Reference Baum-Baicker1985; Castelli, Reference Castelli1979; Stason et al, Reference Stason, Neff, Miettinen and Jick1976). In most observational studies, the association between alcohol use and health was U- or J-shaped, and moderate alcohol use (one to two drinks per day) was found to have a mild negative or possibly null association with cardiovascular diseases (Reynolds et al, Reference Reynolds, Lewis, Nolen, Kinney, Sathya and He2003; Ronksley et al, Reference Ronksley, Brien, Turner, Mukamal and Ghali2011) and diabetes (Carlsson et al, Reference Carlsson, Hammar and Grill2005). This belief was so rooted in the academic community that, in the Dietary Guidelines for Americans 2010 provided by the U.S. Government, the health advice was that moderate alcohol use is protective against the aforementioned diseases and reduces all-cause mortality (DGAC, 2010). However, in the last decade, this consensus has gradually weakened. A series of scientific studies, such as Griswold et al (Reference Griswold, Fullman, Hawley, Arian, Zimsen, Tymeson, Venkateswaran, Tapp, Forouzanfar, Salama and Abate2018), started to claim that there is no safe level of alcohol intake (Mehta and Sheron, Reference Mehta and Sheron2019; Topiwala et al, Reference Topiwala, Ebmeier, Maullin-Sapey and Nichols2022; Wood et al, Reference Wood, Kaptoge, Butterworth, Willeit, Warnakula, Bolton, Paige, Paul, Sweeting, Burgess and Bell2018). Subsequently, prestigious newspapers—e.g. the New York Times in 2018 and 2023—quickly republished this information, adopting this new point of view (NYT, 2023, 2018). The same dynamic has also pervaded the institutions responsible for public health and policy. For instance, the Dietary Guidelines for Americans 2015 do not claim beneficial effects of moderate consumption anymore and do not recommend that people start drinking for any reason (DGAC, 2015, 2020). Furthermore, in September 2022, the WHO Regional Office for Europe approved the “European framework for action on alcohol 2022–2025” (WHO Europe, 2022). According to this document, the WHO suggests reducing per capita alcohol consumption by 2025 (from a 2010 baseline) by 10%. Concretely, they suggest that European governments undertake actions such as increasing taxes, implementing minimum pricing policies, increasing minimum age restrictions, introducing total bans in and around sporting and cultural events, limiting content and frequency of commercial communications, and so forth.Footnote 2 In June 2022, the Irish Government informed the European Commission (EC) of its intention to introduce health warnings about the risks of cancer and liver diseases linked to alcohol intake (European Parliament, 2022). Despite protests from Italy, Spain, and six other EU member states, the request did not receive any objection from the EC during a six-month moratorium period (Giuffrida, Reference Giuffrida2023).
The WHO strategy appears to be well defined; according to it, there is no safe level of alcohol intake, and there is no distinction among typologies of alcoholic beverages. For instance, spirits and wines should be subjected to the same restrictions, irrespective of the average consumption and the modalities (e.g., mostly during lunch, or binge drinking). This approach seems to be at odds with the previous WHO’s strategy (WHO, 2010), which mainly focused on reducing the harmful level of consumption. The proposal is also distant from the Cancer Plan approved by the European Parliament in February 2022 (EC, 2022), which focused on tackling excessive—rather than average—consumption. This shows that even among institutions there is no consensus on the effects of moderate alcohol intake. Assuming that European countries implemented the policies proposed by the WHO, a 10% reduction in alcohol consumption might cause company bankruptcies, job losses, and damage to tourism. The alcoholic beverage industry includes producers, distributors, sellers, and hospitality providers, such as hotels, and has a relevant role in the economy. In Southern Europe, some regions have created specific food and wine tours. The value of the global alcoholic industry was estimated to be more than 500 billion $ in 2020, without including hospitality (WIRE, Reference WIRE2021). The Institute for Alcohol Studies (IAS, 2020) estimates that in the UK, including hospitality services, the alcoholic beverage industry was worth 46 billion £ in 2014, accounting for 2.5% of GDP and 770,000 jobs. If European countries implemented the policies proposed by the WHO, a 10% reduction in alcohol consumption may inflict significant damage to the alcohol-related industry, including the tourism sector (Berkhout et al, Reference Berkhout, Bertling, Bleeker, De Wit, Kruis, Stokkel and Theuws2013; Oxford Economics, 2016). While the net impact on the economy could still be positive, due to substitution effects, improved health outcomes, and increased productivity and tax revenues (Connolly et al, Reference Connolly, Bhattacharya, Lisenkova and McGregor2019; Sachdev et al, Reference Sachdev, Roy and Allan2023; Wada et al, Reference Wada, Chaloupka, Powell and Jernigan2017), these effects are generally difficult to quantify in advance and might not be present at all. For example, no significant improvement in terms of public health can be achieved by a policy that reduces the average (moderate) alcohol consumption but is unable to modify risky behaviors such as binge drinking and alcohol abuse in general. In conclusion, policy makers should be cautious before implementing strategies that can negatively shrink an important sector of the economy and have highly uncertain positive outcomes.
In front of clear scientific evidence on serious health issues arising from moderate alcohol consumption, the economic consequences could eventually take second place. However, it appears that there is no scientific consensus on this topic. The crucial point raised by the WHO Europe is that “to identify a ‘safe’ level of alcohol consumption, valid scientific evidence would need to demonstrate that at and below a certain level, there is no risk of illness or injury associated with alcohol consumption” (WHO, 2023). However, it is not clear with whom the burden of proof lies and the reverse can be claimed, that is, in order to identify an “unsafe” level of alcohol consumption, valid scientific evidence would need to demonstrate that at and below a certain level, there is a risk of illness or injury associated with alcohol consumption. The main issue is that, when studying the effects of alcohol (as well as those of food and drugs), it is not possible to conduct long-term randomized controlled trials (RCTs) because of both ethical and practical reasons (Au Yeung et al, Reference Au Yeung, Jiang, Cheng, Cowling, Liu, Zhang, Lam, Leung and Schooling2013; Poli et al, Reference Poli, Marangoni, Avogaro, Barba, Bellentani, Bucci, Cambieri, Catapano, Costanzo, Cricelli and De Gaetano2013). Inevitably, the literature on the long-term effects of alcohol on health relies on observational studies where articles show at least one of the following statistical and methodological flaws: publication bias, omitted variable bias, reverse causality, inclusion of former drinkers in the teetotalers’ group (“sick-quitter hypothesis”), poor recall of past alcohol consumption, underestimation of the real alcohol intake, non-distinction for ethnicity, and inappropriate use of linear models in place of non-linear models.
Nowadays, one of the most used methods to remedy some of these problems is the Mendelian randomization (MR) approach. This methodology is similar to an instrumental variables approach and enables assessing the causal role of moderate alcohol consumption in a specific population, wherein a genetic variant influences alcohol metabolism and subsequently impacts alcohol use (van de Luitgaarden et al, Reference van de Luitgaarden, van Oort, Bouman, Schoonmade, Schrieks, Grobbee, van der Schouw, Larsson, Burgess, van Bal-legooijen and Onland-Moret2022). While MR appears very promising, it requires more assumptions than RCT and is likely to suffer from significant biases (Adam, Reference Adam2019; Nitsch et al, Reference Nitsch, Molokhia, Smeeth, DeStavola, Whittaker and Leon2006; VanderWeele et al, Reference VanderWeele, Tchetgen, Cornelis and Kraft2014). In fact, three assumptions must be fulfilled for the validity of MR: (1) the genetic variant must be associated with the exposure of interest; (2) the genetic variant should not associate with confounders; and (3) the genetic variant only affects the outcome through the exposure to alcohol. There are many situations in which these assumptions might be violated. One such situation is genetic pleiotropy, i.e., a scenario in which a genetic variant affects both the drinking habits and the smoking behavior. As pointed out by Nitsch et al (Reference Nitsch, Molokhia, Smeeth, DeStavola, Whittaker and Leon2006), MR cannot replace RCT and presents various potential sources of bias, such as inadequate phenotype definition, the presence of gene–environment or gene–gene interactions, the possibility of reverse causation, and linkage disequilibrium.
In this study, we explore all the aforementioned sources of bias and examine the policy implications of our findings. First, we address the issue of publication bias by extracting confidence intervals (CIs) from all alcohol-related papers published in the last decades and available on PubMed. Second, we conduct a rigorous methodological analysis of a sample of papers from WHO’s 2018 report, the CDC website, and studies that use the MR approach. Finally, we discuss our findings.
II. Publication bias
It is a well-known fact that it is easier to publish works in the presence of statistically significant results (Emerson et al, Reference Emerson, Warme, Wolf, Heckman, Brand and Leopold2010). This encourages researchers to only submit manuscripts that present significant findings and, in some cases, even to manipulate their results to obtain a p-value less than .05. Ultimately, this causes a drastic increase in false-positive results (Dumas-Mallet et al, Reference Dumas-Mallet, Button, Boraud, Gonon and Munafò2017). Allen and Mehler (Reference Allen and Mehler2019) find that the most effective way to prevent this problem is to adopt study preregistration and registered reports (RRs), where the hypotheses and analysis pipelines are declared publicly before collecting the data. The authors survey 113 published biomedical and psychological science RRs compiled by the Center for Open Science and find that 60.5% do not get statistically significant results, compared to a share between 5% and 20% for the traditional literature. There are several strategies that researchers can rely on to force the results. One way is to include or exclude some observations, which is particularly common in medical science due to the exclusion criteria of patients from trials based on age, co-morbidity, and co-prescribing. In their systematic review, He et al (Reference He, Morales and Guthrie2020) find that the median exclusion rate from trials was 77.1% of patients. Another way to influence the results is to choose a statistical model which achieves the target. In their experiment, Silberzahn et al (Reference Silberzahn, Uhlmann, Martin, Anselmi, Aust, Awtrey, Bahn´ık, Bai, Bannard, Bonnier and Carlsson2018) ask 29 teams involving 61 analysts to address the same research question using the same identical dataset. The question is whether soccer referees exhibit a higher propensity to give red cards to players with darker skin tones compared to those with lighter skin tones. Both the statistical methodologies and the results vary widely across the teams of researchers, with the estimated effect sizes ranging from 0.89 to 2.93 in odds ratio units; 20 teams find a statistically significant positive effect while 9 do not.
The problem of publication bias is so common that in many disciplines academics have created new journals devoted specifically to publish articles with non-significant results in order to provide an unbiased vision of the reality. In Psychology, there is the Journal of Articles in Support of the Null Hypothesis; in Medicine, the Journal of Negative Results in Biomedicine (where negative should be interpreted as non-significant); in Ecology and Evolutionary Biology, there is the Journal of Negative Results (same as before); and in Economics, there is the Series of Unsurprising Results in Economics.
To investigate publication bias in the alcohol-related literature, we adopt the same strategy used by Barnett and Wren (Reference Barnett and Wren2019) and van Zwet and Cator (Reference van Zwet and Cator2021). By using a modified version of the algorithm created by Georgescu and Wren (Reference Georgescu and Wren2018), we extract CIs from the abstracts of all the published papers uploaded on PubMed from 1980 to 2022 containing the word “alcohol” in the title or the expression “moderate alcohol” in the abstract. PubMed Central carefully controls whether journals meet certain scientific quality standards. We exclude those journals which are considered potentially predatory in the updated Beall’s list provided by Open Access Journals (Singh, Reference Singh2023). After selecting only the papers containing at least one CI in the abstract, we obtain a data set consisting of 6,763 papers and a total of 19,981 CIs. Subsequently, we convert the CIs to z-values.
Clearly, the proposed approach does not allow to summarize the literature, neither it can be used to assess the quality of single articles. However, it provides correct information regarding which z-values are presented more frequently in the abstract. This allows to investigate publication bias in a large number of articles for which carrying out a systematic review would be unfeasible.
If no publication bias was present, and the null hypothesis was always true, the z-values should follow a standard normal distribution. When the null is false, we should observe a compound distribution, i.e., a mixture of normal distributions, each centered at the corresponding value of the parameter under the alternative. A clear sign of publication bias is found when the empirical distribution of the z-values presents some unexpected change of behavior around commonly used critical values, e.g., z = −1.96 and z = 1.96 (van Zwet and Cator, Reference van Zwet and Cator2021).
As shown in Figure 1, our results display a suspiciously low number of z-values between −1.96 and +1.96, which suggests the presence of publication bias. To further illustrate this phenomenon, in the online supplementary materials, we present the distribution of the absolute z-values, which shows a pronounced jump right in proximity of the threshold of statistical significance (see Figure A1). In addition, Figure A2, also available in the online supplementary materials, replicates the histogram of Figure 1 by including only those studies reporting the expression “moderate alcohol” in the title or in the abstract; the histogram provides similar results with a drop in the number of studies with z-value right below +1.96 and right above −1.96.
Figure 1. Distribution of z-values in alcohol-related papers from PubMed. Note: In the figure, the two dashed lines represent the ±1.96 points. For this analysis, we selected from PubMed the papers containing the word “alcohol” in the title or the expression “moderate alcohol” in the abstract, which simultaneously displayed at least one CI in the abstract. The period of analysis is from 1980 to 2022 and the final dataset consists of 6,763 papers and a total of 19,981 CIs.
To investigate time trends, we compute for each year the ratio between the number of CIs just above the significance level threshold (1.96 < |z-value| < 2.58, which corresponds to a p-value between 0.01 and 0.05) and the number of CIs just below (1.64 < |z-value| < 1.96, which corresponds to a p-value between 0.05 and 0.10). Figure A3, available in the online supplementary materials, shows how the ratio increases over time, suggesting that publication bias is becoming even more common. As a last step of this analysis, we investigate the relationship between the SCImago Journal Rank (SJR) and the presence of insignificant results (p > .05) for a subsample of 3,217 papers for which the SJR was readily available. The mean and median of the SJR are slightly—yet significantly—higher in the group of papers displaying at least one insignificant result in the abstract (mean: 2.10 vs. 1.97; median: 1.76 vs. 1.54, respectively; see Figure A4 in the online supplementary materials). This suggests that higher impact journals are more likely to publish results that do not achieve statistical significance, thus reducing the severity of the publication bias.
III. Assessment of existing research
In accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, in this section, we aim to conduct a comprehensive analysis of 49 published articles on the effects of alcohol consumption on health. The goal is to identify potential limitations or biases in individual studies that may impact the validity of their results.
A systematic search was conducted across three distinct sources: the most recent Global Status Report on Alcohol Consumption and Health (WHO, 2019), the CDC website’s Alcohol & Public Health section, and a meta-literature review by van de Luitgaarden et al. focusing on articles utilizing the Mendelian approach to evaluate the effects of alcohol (van de Luitgaarden et al, Reference van de Luitgaarden, van Oort, Bouman, Schoonmade, Schrieks, Grobbee, van der Schouw, Larsson, Burgess, van Bal-legooijen and Onland-Moret2022). The rationale behind selecting these three sources aligns with two specific criteria. First, there is a desire to assess papers referenced by two of the most significant health institutions globally. As their political relevance could potentially influence governmental decisions, we consider it essential to scrutinize the papers cited in their reports and websites. Second, we aimed to explore potential biases in papers employing the Mendelian approach. This methodological strategy is currently one of the most promising for assessing the effects of alcohol on health. However, its implementation implies addressing various other methodological challenges.
We considered original empirical studies that attempt to establish the effects of alcohol consumption on specific diseases, overall mortality, and risky behaviors such as unprotected sex. From the Global Status Report on Alcohol Consumption and Health, we excluded 161 meta-analyses, literature reviews, reports, or newspaper articles, and 214 articles that are not relevant to our analysis, such as papers evaluating the impact of policies contrasting alcohol abuse or studies on the determinants of alcohol consumption. From the report, we identified 22 papers to be included in our analysis. On the day of access, the CDC web page Alcohol & Public Health presented 109 publications, but just 5 of them met our inclusion criteria, while 40 are reports or literature reviews and 64 are papers on different subjects. Regarding the third source, after excluding a meta-analysis, 23 of the 24 papers were used in our research. We identified only 1 case of overlap between 2 of the 3 sources; therefore, the final number of observations was 49. A flow diagram illustrating the study selection process is presented in Figure 2, delineating the stages of identification, screening, eligibility, and inclusion of articles, while a complete list of the selected studies is presented in the online supplementary material (see Table A1). At least two researchers collaboratively screened each record, jointly assessing inclusion or exclusion. In cases of persistent disagreement, the third researcher was consulted.
Figure 2. Flowchart. Note: Flowchart of the selection of original empirical studies that attempt to establish the effects of alcohol consumption on specific diseases, overall mortality and risky behaviors such as unprotected sex.
We assessed the presence of potential sources of bias, including omitted variable bias, estimation method, assessment of alcohol consumption, and the validity of studies based on MR. Our criteria evaluated the definition of the exposure and the comparability of different groups and are inspired by commonly used, well-established tools, such as the Newcastle-Ottawa scale (Wells et al, Reference Wells, Shea, O'Connell, Peterson, Welch, Losos and Tugwell2000). Compared with existing methods, we attempt to provide greater detail regarding possible sources of bias, and consider aspects, such as the use of linear instead of non-linear models, that are relevant to our problem but are scarcely investigated in the literature.
In total, we defined 16 binary indicators (+3 that only apply to papers that use MR), such that a value of 1 indicates a “good” trait (i.e., the corresponding source of bias is either absent, or handled correctly), while a value of 0 indicates a “bad” trait (i.e., the paper is likely to suffer from some bias). Occasionally, we assigned a score of 0.5, whenever the authors handled the issue partially. A missing value was assigned when the bias was not relevant, such as an assessment of diet in a work studying the effect of alcohol intake on car accidents. A complete list of our binary indicators is available in Table 1. Scores were independently assigned by two researchers for each paper. In cases of tied scores, the value was accepted. However, in cases of divergent assessments, the third researcher was consulted to review the paper and provide a definitive evaluation. To determine to which extent the authors have taken into account the potential sources of bias, we also calculated a score for each paper, defined as the mean of all the binary indicators.
Table 1. List and description of the 19 binary indicators and the computed score
In Figure A5, we present the distribution of the score for the selected 49 papers, from highest to lowest. The mean value of the score is around 0.5, indicating that, on average, papers control for half of the potential sources of bias. Figure 3 displays the percentage of papers that appropriately analyze each specific source of bias. All of the MR papers validate the second assumptions, and nearly all the papers in our sample include controls for sex and age. However, only one paper examines the differential impacts of specific alcoholic beverages, such as wine and spirits. Additionally, other relevant aspects such as life-time alcohol consumption patterns or use of drugs are considered by a minority of the papers. We tested the existence of a relationship between the year of publication of the selected papers and their score, but no significant trend was observed. Finally, we examined the association between the SCImago Journal Ranking, measured both in the year of publication and in 2021, and our score. Our findings suggest that journals with higher SJR values in the year of publication tend to have slightly higher scores. However, the difference disappears when we consider the normalized SJR values for 2021 (see Figure A6 in the online supplementary materials).
Figure 3. Sources of bias in alcohol related papers. Note: Percentage of papers that appropriately analyze each specific source of bias when necessary. For instance, if a paper aimed at estimating the impact of alcohol consumption on the probability of car accidents, we do not consider variables such as cigarette smoking or physical activity.
We want to highlight that the scoring system used in this review is not without limitations. It is based on the subjective judgment of the researchers, and it may not capture all potential sources of bias. Despite these limitations, the results of our analysis show that papers included in our study suffer from various sources of bias that may undermine the validity of their estimations of the effects of alcohol. Some might argue that health policies are based on more comprehensive studies, such as meta-analyses (e.g. Shield et al, Reference Shield, Soerjomataram and Rehm2016), which combine the results from multiple individual studies. Indeed, a substantial number of papers cited by the WHO report and the CDC website are meta-analyses. These studies are particularly appreciated by health institutions, as they allow the use of a large number of observations and improve statistical inference (Griswold et al, Reference Griswold, Fullman, Hawley, Arian, Zimsen, Tymeson, Venkateswaran, Tapp, Forouzanfar, Salama and Abate2018). Nonetheless, as underlined by some scholars, increasing the sample size does not eliminate the bias (Oster, Reference Oster2023). The mean of multiple biased estimators is still biased, unless the individual biases have zero mean and “cancel out.” The problem is even more severe in the presence of the aforementioned publication bias, which reduces the chances that studies finding non-significant relationships get published.
IV. Conclusions
The evidence provided in the previous sections shows that the literature on (moderate) alcohol consumption and health suffers from various statistical and methodological weaknesses. We have explored two critical aspects of alcohol-related studies: the pervasive influence of publication bias, and the several methodological and statistical biases affecting this stream of literature. With respect to the former, our analysis has unveiled that the statistical bias could be very strong, and worsening over time. Regarding the latter, the prevalence of methodological biases underscores the complexity of reaching valid conclusions, particularly when dealing with low intake levels. In fact, because of moral and practical reasons, it is not possible to carry on RCTs, where the treatment sample is forced to drink a certain amount of alcohol every day for many years, while the control sample is forbidden to. Therefore, we have to rely on studies based on observational data, which may suffer from reverse causality, omitted variable bias, specific potential sources of bias related to alcohol consumption, estimation method issues, and lack of validation of the three Mendelian assumptions. Since it is objectively impossible to control for all the relevant variables, all studies are expected to be more or less biased.
Despite all the above limitations, the negative effect of alcohol abuse is so strong that it cannot be denied. What is unclear is whether moderate alcohol consumption is harmful and, in case, what is the safe limit we should not exceed. Excessive consumption of alcohol, marijuana, cigarettes, red meat, cheese, butter, eggs, coffee, etc. is surely unhealthy. However, it is difficult—if not impossible—to say whether one glass of wine per day, one cigarette per week, or one beef steak per month has negative and detectable effects on health, net of other omitted confounding elements.
In the past, those authors claiming that a moderate amount of alcohol is healthy should have used some caution. Similarly, the same caution should now be used when affirming that a small amount is unsafe. In conclusion, given the methodological limitations in detecting the effects of modest alcohol quantities, from a scientific point of view, it is incorrect to claim that “there is no safe level.” We should rather say that “we are unable to determine if there is a safe amount” and, likely, we will never be.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/jwe.2025.10077.
Acknowledgements
We thank the participants of the EuAWE Conference (Crete, May 2023) and of the Wine Workshop in Dijon (France, 2023). We are especially thankful to Daria Piacentino, Mirco Tonin, and Filippo Vernia for their useful suggestions. Finally, we would like to thank the Editors and an anonymous referee.
Competing interests
The authors declare none.
Open access
