Skip to main content
×
Home
    • Aa
    • Aa

Estimation of Heterogeneous Treatment Effects from Randomized Experiments, with Application to the Optimal Planning of the Get-Out-the-Vote Campaign

  • Kosuke Imai (a1) and Aaron Strauss (a2)
Abstract

Although a growing number of political scientists are conducting randomized experiments, many of them only report the average treatment effects and do not systematically explore the variation in treatment effects across subpopulations. This is unfortunate from a scientific point of view because heterogeneous treatment effects can provide additional substantive insights. This current state of affairs is also problematic from a policy makers' perspective since such studies do not identify subgroups for which treatments are effective. In this paper, we propose a formal two-step framework that first identifies heterogeneous treatment effects from a randomized experiment and then uses this information to derive an optimal policy about which treatment should be given to whom. Our proposed method avoids the risk of false discoveries that are likely in post hoc subgroup analysis routinely conducted in the discipline. We discuss our methodology in the context of get-out-the-vote randomized field experiments and show how the proposed two-step framework can be applied in real-world settings.

    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Estimation of Heterogeneous Treatment Effects from Randomized Experiments, with Application to the Optimal Planning of the Get-Out-the-Vote Campaign
      Available formats
      ×
      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your Dropbox account. Find out more about sending content to Dropbox.

      Estimation of Heterogeneous Treatment Effects from Randomized Experiments, with Application to the Optimal Planning of the Get-Out-the-Vote Campaign
      Available formats
      ×
      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your Google Drive account. Find out more about sending content to Google Drive.

      Estimation of Heterogeneous Treatment Effects from Randomized Experiments, with Application to the Optimal Planning of the Get-Out-the-Vote Campaign
      Available formats
      ×
Copyright
Corresponding author
e-mail: kimai@princeton.edu (corresponding author)
Footnotes
Hide All

Authors' note: The first version of this paper was circulated in December 2008 under the title of “Planning the Optimal Get-out-the-vote Campaign.” We thank useful comments from seminar participants at Columbia University and the University of Wisconsin, Madison, as well as three anonymous reviewers and the editor. Supplementary materials for this article are available on the Political Analysis Web site.

Footnotes
Linked references
Hide All

This list contains references from the content that can be linked to their source. For a full set of references and notes please see the PDF or HTML where available.

Kevin Arceneaux , and Robin Kolodny . 2009. Educating the least informed: Group endorsements in a grassroots campaign. American Journal of Political Science 53: 755–70.

James O. Berger 1985. Statistical decision theory and Bayesian analysis. 2nd ed. New York: Springer.

Hugh A. Chipman , Edward I. George , and Robert E. McCulloch 2010. BART: Bayesian additive regression trees. Annals of Applied Statistics 4: 266–98.

Stephen R. Cole , and Elizabeth A. Stuart 2010. Generalizing evidence from randomized clinical trials to target populations: The ACTG 320 trial. American Journal of Epidemiology 172: 107–15.

Allison Dale , and Aaron Strauss . 2009. Don't forget to vote: Text message reminders as a mobilization tool. American Journal of Political Science 53: 787804.

Rajeev Dehejia . 2005. Program evaluation as a decision problem. Journal of Econometrics 125: 141–73.

James N. Druckman , Donald P. Green , James H. Kuklinski , and Arthur Lupia . 2006. The growth and development of experimental research in political science. American Political Science Review 100: 627–35.

Samuel J. Eldersveld 1956. Experimental propaganda techniques and voting behavior. American Political Science Review 50: 154–65.

Alan S. Gerber , and Donald P. Green 2000. The effects of canvassing, telephone calls, and direct mail on voter turnout: A field experiment. American Political Science Review 94: 653–63.

Alan S. Gerber , Donald P. Green , and Christopher W. Larimer 2008. Social pressure and voter turnout: Evidence from a large-scale field experiment. American Political Science Review 102: 3348.

Alan S. Gerber , Donald P. Green , and Ron Shachar . 2003. Voting may be habit-forming: Evidence from a randomized field experiment. American Journal of Political Science 47: 540–50.

George W. Hartmann 1936. A field experiment on the comparative effectiveness of “emotional” and “rational” political leaflets in determining election results. Journal of Abnormal Psychology 31: 99114.

Yusaku Horiuchi , Kosuke Imai , and Naoko Taniguchi . 2007. Designing and analyzing randomized experiments: Application to a Japanese election survey experiment. American Journal of Political Science 51: 669–87.

Rebecca B. Morton , and Kenneth C. Williams 2010. Experimental political science and the study of causality: From nature to the lab. New York: Cambridge University Press.

David W. Nickerson 2008. Is voting contagious?: Evidence from two field experiments. American Political Science Review 102: 4957.

Stuart J. Pocock , Susan E. Assmann , Laura E. Enos , and Linda E. Kasten 2002. Subgroup analysis, covariate adjustment and baseline comparisons in clinical trial reporting: current practice and problems. Statistics in Medicine 21: 2917–30.

Brian D. Ripley 1996. Pattern recognition and neural networks. Cambridge: Cambridge University Press.

Donald B. Rubin 1990. Comments on “On the Application of Probability Theory to Agricultural Experiments. Essay on Principles. Section 9” by J. Splawa-Neyman translated from the Polish and edited by D.M. Dabrowska and T.P. Speed Statistical Science 5: 472–80.

L. J. Savage 1951. The theory of statistical decision. Journal of the American Statistical Association 46: 5567.

Jörg Stoye . 2009. Minimax regret treatment choice with finite samples. Journal of Econometrics 151: 7081.

Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Political Analysis
  • ISSN: 1047-1987
  • EISSN: 1476-4989
  • URL: /core/journals/political-analysis
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×
MathJax
Type Description Title
PDF
Supplementary Materials

Imai and Strauss supplementary material
Appendix

 PDF (67 KB)
67 KB

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 13 *
Loading metrics...

Abstract views

Total abstract views: 50 *
Loading metrics...

* Views captured on Cambridge Core between 4th January 2017 - 29th June 2017. This data will be updated every 24 hours.