Skip to main content
×
Home
    • Aa
    • Aa

Bias Amplification and Bias Unmasking

  • Joel A. Middleton (a1), Marc A. Scott (a2), Ronli Diakow (a3) and Jennifer L. Hill (a4)
Abstract

In the analysis of causal effects in non-experimental studies, conditioning on observable covariates is one way to try to reduce unobserved confounder bias. However, a developing literature has shown that conditioning on certain covariates may increase bias, and the mechanisms underlying this phenomenon have not been fully explored. We add to the literature on bias-increasing covariates by first introducing a way to decompose omitted variable bias into three constituent parts: bias due to an unobserved confounder, bias due to excluding observed covariates, and bias due to amplification. This leads to two important findings. Although instruments have been the primary focus of the bias amplification literature to date, we identify the fact that the popular approach of adding group fixed effects can lead to bias amplification as well. This is an important finding because many practitioners think that fixed effects are a convenient way to account for any and all group-level confounding and are at worst harmless. The second finding introduces the concept of bias unmasking and shows how it can be even more insidious than bias amplification in some cases. After introducing these new results analytically, we use constructed observational placebo studies to illustrate bias amplification and bias unmasking with real data. Finally, we propose a way to add bias decomposition information to graphical displays for sensitivity analysis to help practitioners think through the potential for bias amplification and bias unmasking in actual applications.

  • View HTML
    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Bias Amplification and Bias Unmasking
      Available formats
      ×
      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your Dropbox account. Find out more about sending content to Dropbox.

      Bias Amplification and Bias Unmasking
      Available formats
      ×
      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your Google Drive account. Find out more about sending content to Google Drive.

      Bias Amplification and Bias Unmasking
      Available formats
      ×
Copyright
Corresponding author
e-mail: joel.middleton@berkeley.edu (corresponding author)
Footnotes
Hide All

Edited by Prof. R. Michael Alvarez

Authors’ note: For replication files, see Middleton (2016). Supplementary Materials for this article are available on the Political Analysis Web site.

Footnotes
Linked references
Hide All

This list contains references from the content that can be linked to their source. For a full set of references and notes please see the PDF or HTML where available.

J. D. Angrist , G. Imbens , and D. Rubin 1996. Identification of causal effects using instrumental variables. Journal of the American Statistical Association 91(434):444–55.

P. Austin , P. Grootendorst , and G. Anderson 2007. A comparison of the ability of different propensity score models to balance measured variables between treated and untreated subjects: a Monte Carlo study. Statistics in Medicine 26:734–53.

R. Breen , K. Karlson , and A. Holm 2013. Total, direct, and indirect effects in logit and probit models. Sociological Methods and Research 42(2):164191.

M. Brookhart , T. Sturmer , R. Glynn , J. Rassen , and S. Schneeweiss 2010. Confounding control in healthcare database research. Medical Care 48:S11420.

J. Bhattacharya , and W. Vogt 2007. Do instrumental variables belong in propensity scores? NBER Working Paper 343, National Bureau of Economic Research, MA.

K. A. Clarke 2005. The phantom menace. Conflict Management and Peace Science 22:341352.

K. A. Clarke 2009. Return of the phantom menace. Conflict Management and Peace Science 26:4666.

S. R. Cole , R. W. Platt , E. F. Schisterman , H. Chu , D. Westreich , D. Richardson , and C. Poole 2010. Illustrating bias due to conditioning on a collider. International Journal of Epidemiology 39(2):417420.

T. Dunning , and J. Nilekani 2013. Ethnic quotas and political mobilization: caste, parties, and distribution in Indian village councils. American Political Science Review 107:3556.

D. A. Freedman 2008. Randomization does not justify logistic regression. Statistical Science 23(2):237–49.

T. Frisell , S. Oberg , R. Kuja-Halkola , and A. Sjolander 2012. Sibling comparison designs: bias from non-shared confounders and measurement error. Epidemiology 23(5):713–20.

S. Greenland 2002. Quantifying biases in causal models: classical confounding vs. collider-stratification bias. Epidemiology 14:300306.

J. Heckman , and R. Robb 1985. Alternative methods for estimating the impact of interventions. In Longitudinal analysis of labor market data, eds. J. J. Heckman and B. Singer Cambridge University Press, Cambridge, UK.

J. Heckman , and R. Robb 1986. Alternative methods for solving the problem of selection bias in evaluating the impact of treatments on outcomes. In Drawing inferences from self-selected samples, ed. H. Wainer New Jersey: Lawrence Erlbaum Associates.

M. Lechner 2001. Identification and estimation of causal effects of multiple treatments under the conditional independence assumption. In Econometric evaluations of active labor market policies in Europe, eds. M. Lechner and F. Pfeiffer , Heidelberg: Physica.

W. Liu , M. A. Brookhart , S. Schneeweiss , X. Mi , and S. Setoguchi 2012. Implications of M-bias in epidemiologic studies: a simulation study. American Journal of Epidemiology 176:938–48.

J.A. Myers , J. A. Rassen , J. J. Gagne , K. F. Huybrechts , S. Schneeweiss , K. J. Rothman , M. M. Joffe , and R. J. Glynn 2011. Effects of adjusting for instrumental variables on bias and precision of effect estimates. American Journal of Epidemiology 174(11):1213–22.

J. Pearl 2011. Invited commentary: Understanding bias amplification. American Journal of Epidemiology 174(11):1223–27.

P. R. Rosenbaum 2002. Observational studies. Springer, New York, NY.

D. B. Rubin 1974. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology 66:688.

D. B. Rubin 1978. Bayesian inference for causal effects: the role of randomization. The Annals of Statistics 6(1):3458.

D. R. Shaw , D. P. Green , J. G. Gimpel , and A. S. Gerber 2012. Do robotic calls from credible sources influence voter turnout or vote choice? Evidence from a randomized field experiment. Journal of Political Marketing 11(4):231–45.

E. F. Schisterman , S. R. Cole , and R. W. Platt 2009. Overadjustment bias and unnecessary adjustment in epidemiologic studies. Epidemiology 20:488–95.

M. E. Sobel 2006. What do randomized studies of housing mobility demonstrate? Causal inference in the face of interference. Journal of the American Statistical Association 101:1398–407.

R. Wyss , M. Lunt , M. A. Brookhart , R. J. Glynn , and T. Strürmer 2014. Reducing bias amplification in the presence of unmeasured confounding through out-of-sample estimation strategies for the disease risk score. Journal of Causal Inference 2(2):131–46.

T. J. VanderWeele , and O. A. Arahc 2011. Unmeasured confounding for general outcomes, treatments, and confounders: bias formulas for sensitivity analysis. Epidemiology 22(1):4252.

Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Political Analysis
  • ISSN: 1047-1987
  • EISSN: 1476-4989
  • URL: /core/journals/political-analysis
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×
MathJax
Type Description Title
PDF
Supplementary Materials

Middleton et al. supplementary material
Supplementary Material

 PDF (114 KB)
114 KB
UNKNOWN
Supplementary Materials

Middleton et al. supplementary material
Appendix

 Unknown (12 KB)
12 KB

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 77 *
Loading metrics...

Abstract views

Total abstract views: 193 *
Loading metrics...

* Views captured on Cambridge Core between 4th January 2017 - 20th September 2017. This data will be updated every 24 hours.