Skip to main content
    • Aa
    • Aa

Do Get-Out-the-Vote Calls Reduce Turnout? The Importance of Statistical Methods for Field Experiments

  • KOSUKE IMAI (a1)

In their landmark study of a field experiment, Gerber and Green (2000) found that get-out-the-vote calls reduce turnout by five percentage points. In this article, I introduce statistical methods that can uncover discrepancies between experimental design and actual implementation. The application of this methodology shows that Gerber and Green's negative finding is caused by inadvertent deviations from their stated experimental protocol. The initial discovery led to revisions of the original data by the authors and retraction of the numerical results in their article. Analysis of their revised data, however, reveals new systematic patterns of implementation errors. Indeed, treatment assignments of the revised data appear to be even less randomized than before their corrections. To adjust for these problems, I employ a more appropriate statistical method and demonstrate that telephone canvassing increases turnout by five percentage points. This article demonstrates how statistical methods can find and correct complications of field experiments.

Corresponding author
Kosuke Imai is Assistant Professor, Department of Politics, Princeton University, Princeton NJ, 08544 (kimai@Princeton.Edu;
Hide All
Achen Christopher H. 1986. The Statistical Analysis of Quasi-experiments. Berkeley: University of California Press.
Adams Williams C., and Dennis J. Smith. 1980. “Effects of Telephone Canvassing on Turnout and Preferences: A Field Experiment.” Public Opinion Quarterly 44: 38995.
Angrist Joshua D., and Guido W. Imbens. 1995. “Two-stage least squares estimation of average causal effects in models with variable treatment intensity.” Journal of the American Statistical Association 90: 43142.
Angrist Joshua D., Guido W. Imbens, and Donald B. Rubin. 1996. “Identification of Causal Effects Using Instrumental Variables (with Discussion).” Journal of the American Statistical Association 91: 44455.
Balke Alexander, and Judea Pearl. 1997. “Bounds on treatment effects from studies with imperfect compliance.” Journal of the American Statistical Association 92: 117176.
Bound John, David A. Jaeger, and Regina M. Baker. 1995. “Problems with instrumental variables estimation when the correlation between the instruments and the endogenous explanatory variable is weak.” Journal of the American Statistical Association 90: 44350.
Campbell Donald T., and Julian C. Stanley. 1963. Experimental and Quasi-experimental Designs for Research. Chicago: RAND McNally.
Dao James. 2000. “The 2000 Campaign: The Voters; Ringing Phones, Chiming Doorbells, Stuffed E-Mailboxes: The Great Voter Roundup.” New York Times.
Dehejia Rajeev H., and Sadek Wahba. 1999. “Causal Effects in Nonexperimental Studies: Reevaluating the Evaluation of Training Programs.” Journal of the American Statistical Association 94: 105362.
Eldersveld Samuel J. 1956. “Experimental Propaganda Techniques and Voting Behavior.” American Political Science Review 50: 15465.
Fisher Ronald A. 1938. “Presidential Address: The First Session of the Indian Statistical Conference, Calcutta, 1938.” Sankhya 4: 1417.
Gerber Alan S., and Donald P. Green. 2000. “The Effects of Canvassing, Telephone Calls, and Direct Mail on Voter Turnout: A Field Experiment.” American Political Science Review 94: 65363.
Gerber Alan S., and Donald P. Green. 2001. “Do Phone Calls Increase Voter Turnout? A Field Experiment.” Public Opinion Quarterly 65: 7585.
Gerber Alan S., Donald P. Green, and Edward H. Kaplan. 2002. “The Illusion of Learning from Observational Research.” Presented at the Economic Science Association.
Gosnell Harold F. 1927. Getting-Out-the-Vote: An Experiment in the Stimulation of Voting. Chicago: University of Chicago Press.
Green Donald P., and Alan S. Gerber. 2001. “Getting Out the Youth Vote: Results from Randomized Field Experiments.” Technical Report. Yale University.
Green Donald P., and Alan S. Gerber. 2002. “Reclaiming the Experimental Tradition in Political Science.” In Political Science: State of the Discipline. Vol. III, ed. I. Katznelson and H. V. Milner. New York: W. W. Norton, 80532.
Heckman James, Hidehiko Ichimura, Jeffrey Smith, and Petra Todd. 1998. “Characterizing Selection Bias Using Experimental Data.” Econometrica 66: 101798.
Ho Daniel E., Kosuke Imai, Gary King, and Elizabeth A. Stuart. 2004. “Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference.” Technical Report, http://GKing.Harvard.Edu/.
Holland Paul W. 1986. “Statistics and Causal Inference (with Discussion).” Journal of the American Statistical Association 81: 94560.
Horiuchi Yusaku, Kosuke Imai, and Naoko Taniguchi. 2005. “Designing and Analyzing Randomized Experiments.” Technical Report,
Howell William, and Paul E. Peterson. 2002. The Education Gap: Vouchers and Urban Schools. Washington, DC: Brookings.
Imai Kosuke. 2003. “Essays on Political Methodology.” Ph.D. dissertation, Department of Government, Harvard University.
Imai Kosuke, and David A. van Dyk. 2004. “Causal Inference with General Treatment Regimes: Generalizing the Propensity Score.” Journal of the American Statistical Association, Theory and Methods 99: 85466.
Kinder Donald R., and Thomas R. Palfrey, eds. 1993. Experimental Foundations of Political Science. Ann Arbor: University of Michigan Press.
King Gary, and Langche Zeng. 2003. “When Can History Be Our Guide? The Pitfalls of Counterfactual Inference.” Technical Report, http://GKing.Harvard.Edu/.
McCullagh Peter, and John A. Nelder. 1989. Generalized Linear Models. 2nd Ed. London: Chapman & Hall.
Mill John Stuart. 1930/1843. A System of Logic, Ratiocinative and Inductive: Being a Connected View of the Principles of Evidence and the Methods of Scientific Investigation. London: Longman.
Miller Roy E., David E. Bositis, and Denise L. Baer. 1981. “Stimulating Voter Turnout in a Primary: Field Experiment with a Precinct Committeeman.” International Political Science Review 2: 44560.
Rosenbaum Paul R., and Donald B. Rubin. 1983. “The Central Role of the Propensity Score in Observational Studies for Causal Effects.” Biometrika 70: 4155.
Rosenbaum Paul R., and Donald B. Rubin. 1985a. “The Bias Due to Incomplete Matching.” Biometrics 41: 10316.
Rosenbaum Paul R., and Donald B. Rubin. 1985b. “Constructing a Control Group Using Multivariate Matched Sampling Methods That Incorporate the Propensity Score.” American Statistician 39: 3338.
Wantchekon Leonard. 2003. “Clientelism and Voting Behavior: Evidence from a Field Experiment in Benin.” World Politics 55: 399422.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

American Political Science Review
  • ISSN: 0003-0554
  • EISSN: 1537-5943
  • URL: /core/journals/american-political-science-review
Please enter your name
Please enter a valid email address
Who would you like to send this to? *


Altmetric attention score

Full text views

Total number of HTML views: 8
Total number of PDF views: 83 *
Loading metrics...

Abstract views

Total abstract views: 255 *
Loading metrics...

* Views captured on Cambridge Core between September 2016 - 21st October 2017. This data will be updated every 24 hours.