Published online by Cambridge University Press: 04 November 2016
The effect of conditioning on an additional covariate on confounding bias depends, in part, on covariates that are unobserved. We characterize the conditions under which the interaction between a covariate that is available for conditioning and one that is not can affect bias. When the confounding effects of two covariates, one of which is observed, are countervailing (in opposite directions), conditioning on the observed covariate can increase bias. We demonstrate this possibility analytically, and then show that these conditions are not rare in actual data. We also consider whether balance tests or sensitivity analysis can be used to justify the inclusion of an additional covariate. Our results indicate that neither provide protection against overadjustment.
Kevin A. Clarke is an Associate Professor in the Department of Political Science, University of Rochester, Rochester, NY 14627-0146 (email@example.com). Brenton Kenkel is an Assistant Professor in the Department of Political Science, Vanderbilt University, Nashville, TN 37203 (firstname.lastname@example.org). Miguel R. Rueda is an Assistant Professor in the Department of Political Science, Emory University, Atlanta, GA 30233 (email@example.com). A previous version of this paper was given at the 27th Annual Summer Meeting of the Society for Political Methodology. The authors thank the participants for their comments. The authors thank Jake Bowers, John Jackson, Michael Peress, the editor, and two anonymous reviewers for helpful comments and discussion. Brad Smith provided excellent research assistance. Errors remain the authors own. To view supplementary material for this article, please visit https://doi.org/10.1017/psrm.2016.46