Skip to main content
×
Home
    • Aa
    • Aa

How Much Is Minnesota Like Wisconsin? Assumptions and Counterfactuals in Causal Inference with Observational Data

  • Luke Keele (a1) and William Minozzi (a2)
Abstract

Political scientists are often interested in estimating causal effects. Identification of causal estimates with observational data invariably requires strong untestable assumptions. Here, we outline a number of the assumptions used in the extant empirical literature. We argue that these assumptions require careful evaluation within the context of specific applications. To that end, we present an empirical case study on the effect of Election Day Registration (EDR) on turnout. We show how different identification assumptions lead to different answers, and that many of the standard assumptions used are implausible. Specifically, we show that EDR likely had negligible effects in the states of Minnesota and Wisconsin. We conclude with an argument for stronger research designs.

Copyright
Corresponding author
e-mail: ljk20@psu.edu (corresponding author)
Footnotes
Hide All

Authors' note: We thank Michael Hanmer and Devin Caughey for generously sharing code and data. For comments and suggestions, we thank Mike Alvarez, Curt Signorino, Shigeo Hirano, Robert Erikson, Mike Ting, Walter Mebane, Michael Hanmer, Betsy Sinclair, Jonathan Nagler, Don Green, Rocío Titiunik, and seminar participants at Columbia University, the University of Michigan, and the University of Rochester. A previous version of this article was presented at the 2010 Annual Meeting of the Society of Political Methodology, Iowa City, IA, and APSA 2010. Replication files and information can be found in Keele (2012).

Footnotes
Linked references
Hide All

This list contains references from the content that can be linked to their source. For a full set of references and notes please see the PDF or HTML where available.

Joshua D. Angrist , and Jörn-Steffen Pischke . 2010. The credibility revolution in empirical economics: How better research design is taking the con out of econometrics. Journal of Economic Perspectives 24(2): 330.

Joshua D. Angrist , Guido W. Imbens , and Donald B. Rubin 1996. Identification of causal effects using instrumental variables. Journal of the American Statistical Association 91(434): 444–55.

Stephen Ansolabehere , and David M. Konisky 2006. The introduction of voter registration and its effect on turnout. Political Analysis 14(1): 83100.

Jason Barabas . 2004. How deliberation affects policy opinions. American Political Science Review 98(4): 687702.

Craig Leonard Brians , and Bernard Grofman . 1999. When registration barriers fall, who votes? An empirical test of a rational choice model. Public Choice 99: 161–76.

Craig Leonard Brians , and Bernard Grofman . 1999. When registration barriers fall, who votes? An empirical test of a rational choice model. Public Choice 99: 161–76.

Craig Leonard Brians , and Bernard Grofman . 2001. Election day registration's effect on U.S. voter turnout. Social Science Quarterly 82: 170–83.

Devin Caughey , and Jasjeet S. Sekhon 2011. Elections and the regression discontinuity design: Lessons from close U.S. house races, 1942–2008. Political Analysis 19(4): 385408.

T. D. Cook , and W. R. Shadish 1994. Social experiments: Some developments over the past fifteen years. Annual Review of Psychology 45: 545–80.

T. D. Cook , W. R. Shadish , and Vivian C. Wong 2008. Three conditions under which experiments and observational studies produce comparable causal estimates: New findings from within-study comparisons. Journal of Policy Analysis and Management 27(4): 724–50.

Stephen G. Donald , and Kevin Lang . 2007. Inference with differences-in-differences and other panel data. Review of Economics and Statistics 89(2): 221–33.

Jinyong Hahn , Petra Todd , and Wilbert van der Klaauw . 2001. Identification and estimation of treatments effects with a regression-discontinuity design. Econometrica 69(1): 201–9.

Michael J. Hanmer 2007. An alternative approach to estimating who is most likely to respond to changes in registration laws. Political Behavior 29(1): 130.

Michael J. Hanmer 2009. Discount voting. New York: Cambridge University Press.

Benjamin Highton , and Raymond E. Wolfinger 1998. Estimating the effects of the national voter registration act of 1993. Political Behavior 20(1): 79104.

Paul W. Holland 1986. Statistics and causal inference. Journal of the American Statistical Association 81: 945–60.

Paul W. Holland 1986. Statistics and causal inference. Journal of the American Statistical Association 81: 945–60.

Guido W. Imbens 2003. Sensitivity to exogeneity assumptions in program evaluation. American Economic Review Papers and Proceedings 93(2): 126–32.

Guido W. Imbens 2010. Better LATE than nothing: Some comments on Deaton (2009) and Heckman and Urzua (2009). Journal of Economic Literature 48(2): 399423.

Guido W. Imbens , and Joshua D. Angrist 1994. Identification and estimation of local average treatment effects. Econometrica 62(2): 467–76.

Guido W. Imbens , and Karthik Kalyanaraman . 2012. Optimal bandwidth choice for the regression discontinuity estimator. Review of Economic Studies 79(3): 933–59.

Guido W. Imbens , and Thomas Lemieux 2008. Regression discontinuity designs: A guide to practice. Journal of Econometrics 142(2): 615–35.

Stephen Knack . 2001. Election-day registration: The second wave. American Politics Research 29: 6578.

David S. Lee 2008. Randomized experiments from non-random selection in U.S. House elections. Journal of Econometrics 142(2): 675–97.

David S. Lee , and Thomas Lemieux . 2010. Regression discontinuity designs in economics. Journal of Economic Literature 48(2): 281355.

Charles F. Manski 1997. Monotone treatment response. Econometrica 65(5): 1311–34.

Justin McCrary . 2008. Manipulation of the running variable in the regression discontinuity design: A density test. Journal of Econometrics 142(2): 698714.

S. L. Rhine 1995. Registration reform and turnout change in American states. American Politics Quarterly 23: 409–27.

Paul R. Rosenbaum 2002a. Attributing effects to treatment in matched observational studies. Journal of the American Statistical Association 97(457): 110.

Paul R. Rosenbaum 2002b. Observational studies. 2nd ed. New York: Springer.

Paul R. Rosenbaum 2005a. Heterogeneity and causality: Unit heterogeneity and design sensitivity in observational studies. American Statistician 59(2): 147–52.

Paul R. Rosenbaum 2005a. Heterogeneity and causality: Unit heterogeneity and design sensitivity in observational studies. American Statistician 59(2): 147–52.

Paul R. Rosenbaum 2005b. Observational study. In Encyclopedia of statistics in behavioral science, eds. S. Everitt Brian and C. Howell David , Vol. 3, 1451–62. Hoboken, NJ: John Wiley and Sons.

Paul R. Rosenbaum 2010. Design of observational studies. New York: Springer.

Paul R. Rosenbaum , and Jeffrey H. Silber 2009. Amplification of sensitivity analysis in matched observational studies. Journal of the American Statistical Association 104(488): 1398–405.

Paul R. Rosenbaum , and Jeffrey H. Silber 2009. Amplification of sensitivity analysis in matched observational studies. Journal of the American Statistical Association 104(488): 1398–405.

Donald B. Rubin 1978. Bayesian inference for causal effects: The role of randomization. Annals of Statistics 6: 3458.

Donald B. Rubin 1978. Bayesian inference for causal effects: The role of randomization. Annals of Statistics 6: 3458.

Jasjeet S. Sekhon 2011. Multivariate and propensity score matching software with automated balance optimization: The matching package for R. Journal of Statistical Software 42(7): 152.

Jasjeet S. Sekhon , and Rocío Titiunik . 2012. When natural experiments are neither natural nor experiments. American Political Science Review 106(1): 3557.

J. Allison Sovey , and Donald P. Green 2011. Instrumental variables estimation in political science: A readers' guide. American Journal of Political Science 55(1): 188200.

Richard J. Timpone 1998. Structure, behavior, and voter turnout in the United States. American Political Science Review 92(1): 145–58.

Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Political Analysis
  • ISSN: 1047-1987
  • EISSN: 1476-4989
  • URL: /core/journals/political-analysis
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×
MathJax

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 20 *
Loading metrics...

Abstract views

Total abstract views: 57 *
Loading metrics...

* Views captured on Cambridge Core between 4th January 2017 - 17th August 2017. This data will be updated every 24 hours.