Hostname: page-component-76fb5796d-dfsvx Total loading time: 0 Render date: 2024-04-25T14:26:04.788Z Has data issue: false hasContentIssue false
  • English
  • Français

Reducing Model Misspecification and Bias in the Estimation of Interactions

Published online by Cambridge University Press:  23 July 2021

Matthew Blackwell Open the ORCID record for Matthew Blackwell [Opens in a new window]  and
Michael P. Olson
Matthew Blackwell*
Affiliation:
Department of Government, Harvard University, Cambridge, MA, USA. Email: mblackwell@gov.harvard.edu, URL: http://www.mattblackwell.org
Michael P. Olson
Affiliation:
Department of Political Science, Washington University in St. Louis, St. Louis, MO, USA. Email: michael.p.olson@wustl.edu, URL: http://www.michaelpatrickolson.com
*
Corresponding author Matthew Blackwell
Get access Rights & Permissions [Opens in a new window]

Abstract

Analyzing variation in treatment effects across subsets of the population is an important way for social scientists to evaluate theoretical arguments. A common strategy in assessing such treatment effect heterogeneity is to include a multiplicative interaction term between the treatment and a hypothesized effect modifier in a regression model. Unfortunately, this approach can result in biased inferences due to unmodeled interactions between the effect modifier and other covariates, and including these interactions can lead to unstable estimates due to overfitting. In this paper, we explore the usefulness of machine learning algorithms for stabilizing these estimates and show how many off-the-shelf adaptive methods lead to two forms of bias: direct and indirect regularization bias. To overcome these issues, we use a post-double selection approach that utilizes several lasso estimators to select the interactions to include in the final model. We extend this approach to estimate uncertainty for both interaction and marginal effects. Simulation evidence shows that this approach has better performance than competing methods, even when the number of covariates is large. We show in two empirical examples that the choice of method leads to dramatically different conclusions about effect heterogeneity.

Keywords

interactionsregressionmachine learninglasso
Type
Article
Information
Political Analysis , Volume 30 , Issue 4 , October 2022 , pp. 495 - 514
Copyright
© The Author(s) 2021. Published by Cambridge University Press on behalf of the Society for Political Methodology

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Edited by Jeff Gill

References

Ansolabehere, S., Hirano, S., and Snyder, J. M. Jr. 2007. “What Did the Direct Primary Do to Party Loyalty in Congress?” In Process, Party and Policy Making: Further New Perspectives on the History of Congress, edited by Brady, D. and McCubbins, M. D.. Palo Alto, CA: Stanford University Press.Google Scholar
Bansak, K. 2021. “A Generalized Framework for the Estimation of Causal Moderation Effects with Randomized Treatments and Non-Randomized Moderators.” Journal of the Royal Statistical Society: Series A 184(1):65–86.CrossRefGoogle Scholar
Beiser-McGrath, J., and Beiser-McGrath, L. F.. 2020. “Problems with Products? Control Strategies for Models with Interaction and Quadratic Effects.” Political Science Research and Methods 8(4):707–730.CrossRefGoogle Scholar
Belloni, A., Chernozhukov, V., and Hansen, C.. 2014a. “Inference on Treatment Effects After Selection Among High-Dimensional Controls.” The Review of Economic Studies 81(2):608–650.CrossRefGoogle Scholar
Belloni, A., Chernozhukov, V., Hansen, C., and Kozbur, D.. 2016. “Inference in High-Dimensional Panel Models with an Application to Gun Control.” Journal of Business & Economic Statistics 34(4):590–605.CrossRefGoogle Scholar
Belloni, A., Chernozhukov, V., and Kato, K.. 2014b. “Uniform Post-Selection Inference for Least Absolute Deviation Regression and Other Z-Estimation Problems.” Biometrika 102(1):77–94.CrossRefGoogle Scholar
Berry, W. D., DeMeritt, J. H. R., and Esarey, J.. 2010. “Testing for Interaction in Binary Logit and Probit Models: Is a Product Term Essential?” American Journal of Political Science 54(1):248–266.CrossRefGoogle Scholar
Blackwell, M., and Olson, M.. 2021. “Replication Data for: Reducing Model Misspecification and Bias in the Estimation of Interactions.” https://doi.org/10.7910/DVN/HZYFRI, Harvard Dataverse, V1, UNF:6:frTpXxD+sbB6H4uyIxDIgw== [fileUNF].CrossRefGoogle Scholar
Brambor, T., Clark, W. R., and Golder, M.. 2006. “Understanding Interaction Models: Improving Empirical Analyses.” Political Analysis 14(1):63–82.CrossRefGoogle Scholar
Braumoeller, B. F. 2004. “Hypothesis Testing and Multiplicative Interaction Terms.” International Organization 58(4):807–820.CrossRefGoogle Scholar
Chatterjee, A., and Lahiri, S. N.. 2011. “Bootstrapping Lasso Estimators.” Journal of the American Statistical Association 106(494):608–625.CrossRefGoogle Scholar
Chipman, H. A., George, E. I., and McCulloch, R. E.. 2010. “BART: Bayesian Additive Regression Trees.” Annals of Applied Statistics 4(1):266–298.CrossRefGoogle Scholar
Esarey, J., and Sumner, J. L.. 2018. “Marginal Effects in Interaction Models: Determining and Controlling the False Positive Rate.” Comparative Political Studies 51(9):1144–1176.CrossRefGoogle Scholar
Escribà-Folch, A., Meseguer, C., and Wright, J.. 2018. “Remittances and Protest in Dictatorships.” American Journal of Political Science 62(4):889–904.CrossRefGoogle Scholar
Franzese, R. J., and Kam, C.. 2009. Modeling and Interpreting Interactive Hypotheses in Regression Analysis. Ann Arbor: University of Michigan Press.Google Scholar
Hainmueller, J., and Hazlett, C.. 2014. “Kernel Regularized Least Squares: Reducing Misspecification Bias with a Flexible and Interpretable Machine Learning Approach.” Political Analysis 22(2):143–168.CrossRefGoogle Scholar
Hainmueller, J., Mummolo, J., and Xu, Y.. 2019. “How Much Should We Trust Estimates from Multiplicative Interaction Models? Simple Tools to Improve Empirical Practice.” Political Analysis 27(2):163–192.CrossRefGoogle Scholar
Hirano, S., and Snyder, J. M. Jr. 2007. “The Decline of Third-Party Voting in the United States.” Journal of Politics 69(1):1–16.CrossRefGoogle Scholar
Hirano, S., and Snyder, J. M. Jr. 2019. Primary Elections in the United States. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
Imai, K., and Ratkovic, M.. 2013. “Estimating Treatment Effect Heterogeneity in Randomized Program Evaluation.” The Annals of Applied Statistics 7(1):443–470.CrossRefGoogle Scholar
Imbens, G. W. 2004. “Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review.” Review of Economics and Statistics 86(1):4–29.CrossRefGoogle Scholar
Kam, C. D., and Trussler, M. J.. 2017. “At the Nexus of Observational and Experimental Research: Theory, Specification, and Analysis of Experiments with Heterogeneous Treatment Effects.” Political Behavior 39(4):789–815.CrossRefGoogle Scholar
Keele, L., Stevenson, R. T., and Elwert, F.. 2020. “The Causal Interpretation of Estimated Associations in Regression Models.” Political Science Research and Methods 8(1):1–13.CrossRefGoogle Scholar
Künzel, S. R., Sekhon, J. S., Bickel, P. J., and Yu, B.. 2019. “Metalearners for Estimating Heterogeneous Treatment Effects Using Machine Learning.” Proceedings of the National Academy of Sciences 116(10):4156–4165.CrossRefGoogle ScholarPubMed
MacKinnon, J., and White, H.. 1985. “Some Heteroskedasticity-Consistent Covariance Matrix Estimators with Improved Finite Sample Properties.” Journal of Econometrics 29(3):305–325.CrossRefGoogle Scholar
Ratkovic, M., and Tingley, D.. 2017. “Sparse Estimation and Uncertainty with Application to Subgroup Analysis.” Political Analysis 25(1):1–40.CrossRefGoogle Scholar
Tibshirani, R. 1996. “Regression Shrinkage and Selection via the Lasso.” Journal of the Royal Statistical Society. Series B (Methodological) 58(1):267–288.CrossRefGoogle Scholar
VanderWeele, T. 2015. Explanation in Causal Inference: Methods for Mediation and Interaction. Oxford, UK: Oxford University Press.Google Scholar
Vansteelandt, S., VanderWeele, T. J., Tchetgen, E. J., and Robins, J. M.. 2008. “Multiply Robust Inference for Statistical Interactions.” Journal of the American Statistical Association 103(484):1693–1704.CrossRefGoogle ScholarPubMed
Ware, A. 2002. The American Direct Primary. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
Zou, H. 2006. “The Adaptive Lasso and Its Oracle Properties.” Journal of the American Statistical Association 101(476):1418–1429.CrossRefGoogle Scholar
Supplementary material: PDF

Blackwell and Olson supplementary material

Blackwell and Olson supplementary material

Download Blackwell and Olson supplementary material(PDF)
PDF 771.5 KB