Skip to main content
×
×
Home

The Minimal Persuasive Effects of Campaign Contact in General Elections: Evidence from 49 Field Experiments

  • JOSHUA L. KALLA (a1) and DAVID E. BROOCKMAN (a2)
Abstract

Significant theories of democratic accountability hinge on how political campaigns affect Americans’ candidate choices. We argue that the best estimate of the effects of campaign contact and advertising on Americans’ candidates choices in general elections is zero. First, a systematic meta-analysis of 40 field experiments estimates an average effect of zero in general elections. Second, we present nine original field experiments that increase the statistical evidence in the literature about the persuasive effects of personal contact tenfold. These experiments’ average effect is also zero. In both existing and our original experiments, persuasive effects only appear to emerge in two rare circumstances. First, when candidates take unusually unpopular positions and campaigns invest unusually heavily in identifying persuadable voters. Second, when campaigns contact voters long before election day and measure effects immediately—although this early persuasion decays. These findings contribute to ongoing debates about how political elites influence citizens’ judgments.

Copyright
Corresponding author
Joshua L. Kalla is a Graduate Student, Department of Political Science, University of California, Berkeley. kalla@berkeley.edu, http://polisci.berkeley.edu/people/person/joshua-kalla.
David E. Broockman is an Assistant Professor, Stanford Graduate School of Business. dbroockman@stanford.edu, https://people.stanford.edu/dbroock/.
Footnotes
Hide All

We acknowledge seminar participants at the Midwest Political Science Association conference, the Northeast Political Methodology Meeting at NYU, Berkeley, Stanford, and the University of British Columbia, as well as Adam Berinsky, Donald Green, Avi Feller, Shanto Iyengar, Jon Krosnick, Gabriel Lenz, Joel Middleton, Daron Shaw, Jas Sekhon, Eric Schickler, Laura Stoker, and Lynn Vavreck for helpful feedback. All remaining errors are our own. The original studies reported herein were conducted by Working America. The authors served as unpaid consultants to Working America in their personal capacity.

Footnotes
References
Hide All
Abramowitz Alan I. 2010. The Disappearing Center: Engaged Citizens, Polarization, and American Democracy. New Haven, CT: Yale University Press.
Adams William C., and Smith Dennis J.. 1980. “Effects of Telephone Canvassing on Turnout and Preferences: A Field Experiment.” Public Opinion Quarterly 44 (3): 389–95.
Ansolabehere Stephen. 2006. “The Paradox of Minimal Effects.” In Capturing Campaign Effects, ed. Brady Henry E. and Johnston Richard. Ann Arbor: University of Michigan Press, 2944.
Anzia Sarah F. 2011. “Election Timing and the Electoral Influence of Interest Groups.” The Journal of Politics 73 (2): 412–27.
Arceneaux Kevin. 2005. “Using Cluster Randomized Field Experiments to Study Voting Behavior.” Annals of the American Academy of Political and Social Science 601 (1): 169–79.
Arceneaux Kevin. 2007. “I’m Asking for Your Support: The Effects of Personally Delivered Campaign Messages on Voting Decisions and Opinion Formation.” Quarterly Journal of Political Science 2 (1): 4365.
Arceneaux Kevin, and Nickerson David W.. 2010. “Comparing Negative and Positive Campaign Messages: Evidence From Two Field Experiments.” American Politics Research 38 (1): 5483.
Arceneaux Kevin, and Kolodny Robin. 2009. “Educating the Least Informed: Group Endorsements in a Grassroots Campaign.” American Journal of Political Science 53 (4): 755–70.
Bailey Michael A., Hopkins Daniel J., and Rogers Todd. 2016. “Unresponsive, Unpersuaded: The Unintended Consequences of Voter Persuasion Efforts.” Political Behavior 38 (3): 713–46.
Barabas Jason, and Jerit Jennifer. 2010. “Are Survey Experiments Externally Valid?American Political Science Review 104 (2): 226–42.
Barton Jared, Castillo Marco, and Petrie Ragan. 2014. “What Persuades Voters? A Field Experiment on Political Campaigning.” The Economic Journal 124 (574): F293F326.
Bawn Kathleen, Cohen Martin, Karol David, Masket Seth, Noel Hans, and Zaller John. 2012. “A Theory of Political Parties: Groups, Policy Demands and Nominations in American Politics.” Perspectives on Politics 10 (3): 571–97.
Berelson Bernard R., Lazarsfeld Paul F., and McPhee William N.. 1954. Voting: A Study of Opinion Formation in a Presidential Campaign. Chicago, IL: University of Chicago Press.
Berinsky Adam J. 2009. In Time of War: Understanding American Public Opinion From World War II to Iraq. Chicago, IL: University of Chicago Press.
Berinsky Adam J. 2017. “Measuring Public Opinion with Surveys.” Annual Review of Political Science 20: 309–29.
Brady Henry E., Johnston Richard, and Sides John. 2006. “The Study of Political Campaigns.” In Capturing Campaign Effects, ed. Brady Henry E. and Johnston Richard. Ann Arbor, MI: University of Michigan Press, 126.
Broockman David, and Green Donald. 2014. “Do Online Advertisements Increase Political Candidates’ Name Recognition or Favorability? Evidence from Randomized Field Experiments.” Political Behavior 36 (2): 263–89.
Broockman David E., and Kalla Joshua L.. 2016. “Durably Reducing Transphobia: A Field Experiment on Door-to-door Canvassing.” Science 352 (6282): 220–4.
Broockman David, Kalla Joshua L., and Sekhon Jasjeet S.. 2017. “The Design of Field Experiments With Survey Outcomes: A Framework for Selecting More Efficient, Robust, and Ethical Designs.” Political Analysis.
Brox Brian J., and Shaw Daron R.. 2009. “Political Parties, American Campaigns, and Effects on Outcomes.” In Handbook of Party Politics, ed Katz Richard S. & Crotty William. London: Sage, 146–59.
Campbell Angus, Converse Philip E., Miller Warren E., and Stokes Donald E.. 1960. The American Voter. Chicago: University of Chicago Press.
Cantoni Enrico and Pons Vincent. 2017. “Do Interactions with Candidates Increase Voter Support and Participation? Experimental Evidence from Italy.” MPSA Conference Paper.
Cardy Emily Arthur. 2005. “An Experimental Field Study of the GOTV and Persuasion Effects of Partisan Direct Mail and Phone Calls.” The Annals of the American Academy of Political and Social Science 601 (1): 2840.
Carpenter Daniel. 2016. “Recruitment by Petition: American Antislavery, French Protestantism, English Suppression.” Perspectives on Politics 14 (3): 700–23.
Chong Dennis, and Druckman James N. 2007. “A Theory of Framing and Opinion Formation in Competitive Elite Environments.” Journal of Communication 57 (1): 99118.
Cubbison William. 2015. “The Marginal Effects of Direct Mail on Vote Choice.” Paper presented at the Annual Meeting of the Midwest Political Science Association. URL: http://media.wix.com/ugd/3a8c0a_47330c730f56431f8f982a3d842f434a.pdf.
Cunow Saul, and Schwenzfeier Meg. 2015. Working America 2014 Persuasion Experiments Results Memo. Technical Report Analyst Institute.
Deaton Angus. 2010. “Instruments, Randomization, and Learning about Development.” Journal of Economic Literature 48 (2): 424–55.
DellaVigna Stefano, and Gentzkow Matthew. 2010. “Persuasion: Empirical Evidence.” Annual Review of Economics 2: 643–69.
Doherty David, and Adler E. Scott. 2014. “The Persuasive Effects of Partisan Campaign Mailers.” Political Research Quarterly 67 (3): 562573.
Druckman James N. 2001. “The Implications of Framing Effects for Citizen Competence.” Political Behavior 23 (3): 225–56.
Druckman James N. 2004a. “Political Preference Formation: Competition, Deliberation, and the (Ir) Relevance of Framing Effects.” American Political Science Review 98 (4): 671–86.
Druckman James N. 2004b. “Priming the Vote: Campaign Effects in a U.S. Senate Election.” Political Psychology 25 (04): 577–94.
Druckman James N., Peterson Erik, and Slothuus Rune. 2013. “How Elite Partisan Polarization Affects Public Opinion Formation.” American Political Science Review 107 (01): 5779.
Druckman James N., and Leeper Thomas J.. 2012a. “Is Public Opinion Stable? Resolving the Micro/Macro Disconnect in Studies of Public Opinion.” Daedalus 141 (4): 5068.
Druckman James N., and Leeper Thomas J.. 2012b. “Learning More from Political Communication Experiments: Pretreatment and Its Effects.” American Journal of Political Science 56 (4): 875–96.
Endres Kyle. 2016. “The Accuracy of Microtargeted Policy Positions.” PS: Political Science & Politics 49 (4): 771–4.
Enos Ryan D. and Fowler Anthony. 2016. “Aggregate Effects of Large-Scale Campaigns on Voter Turnout.” Political Science Research and Methods: 119. https://doi.org/10.1017/psrm.2016.21
Enos Ryan D., and Hersh Eitan D.. 2015. “Party Activists as Campaign Advertisers: The Ground Campaign as a Principal-Agent Problem.” American Political Science Review 109 (2): 252–78.
Follman Dean A., and Proschan Michael A.. 1999. “Valid Inference in Random Effects Meta-analysis.” Biometrics 55 (3): 732–7.
Foos Florian. 2017. “First Impressions - Lasting Impressions? The Short- and Long-term Effects of Candidate Contact on Voting Intentions During a High-salience Campaign.” MPSA Conference Paper.
Fowler Erika Franklin, Ridout Travis N., and Franz Michael M.. 2016. “Political Advertising in 2016: The Presidential Election as Outlier?The Forum: A Journal of Applied Research in Contemporary Politics 4 (14): 445–69.
Franco Annie, Malhotra Neil, and Simonovits Gabor. 2014. “Publication Bias in the Social Sciences: Unlocking the File Drawer.” Science 345 (6203): 1502–05.
Freeder Sean, Lenz Gabriel S., and Turney Shad. 2017. “The Importance of Knowing ‘What Goes With What’: Reinterpreting the Evidence on Policy Attitude Stability.” Working paper, available at https://www.ocf.berkeley.edu/~glenz/wgww/WGWW.pdf.
Gelman Andrew, and King Gary. 1993. “Why Are American Presidential Election Campaign Polls so Variable When Votes Are so Predictable?British Journal of Political Science 23 (4): 409–51.
Gelman Andrew, Goel Sharad, Rivers Douglas, Rothschild David et al. 2016. “The Mythical Swing Voter.” Quarterly Journal of Political Science 11 (1): 103–30.
Gerber Alan S. 2004. “Does Campaign Spending Work? Field Experiments Provide Evidence and Suggest New Theory.” American Behavioral Scientist 47 (5): 541–74.
Gerber Alan S., Kessler Daniel P., and Meredith Marc. 2011. “The Persuasive Effects of Direct Mail: A Regression Discontinuity Based Approach.” The Journal of Politics 73 (1): 140–55.
Gerber Alan S., Karlan Dean, and Bergan Daniel. 2009. “Does the Media Matter? A Field Experiment Measuring the Effect of Newspapers on Voting Behavior and Political Opinions.” American Economic Journal: Applied Economics 1 (2): 3552.
Gerber Alan S., and Green Donald P.. 2000. “The Effects of Canvassing, Telephone Calls, and Direct Mail on Voter Turnout: A Field Experiment.” American Political Science Review 94 (3): 653–63.
Gerber Alan S., Green Donald P., and Larimer Christopher W.. 2008. “Social Pressure and Voter Turnout: Evidence from a Large-Scale Field Experiment.” American Political Science Review 102 (1): 3348.
Gerber Alan S., Gimpel James, Green Donald, and Shaw Daron. 2011. “How Large and Long-lasting Are the Persuasive Effects of Televised Campaign Ads? Results from a Randomized Experiment.” American Political Science Review 105 (1): 135–50.
Green Donald, Palmquist Bradley, and Schickler Eric. 2002. Partisan Hearts & Minds: Political Parties and the Social Identities of Voters. New Haven, CT: Yale University Press.
Green Donald P., and Gerber Alan S.. 2015. Get Out the Vote: How to Increase Voter Turnout. 3rd ed. Brookings.
Green Donald P., Krasno Jonathan S., Coppock Alexander, Farrer Benjamin D., Lenoir Brandon, and Zingher Joshua N.. 2016. “The Effects of Lawn Signs on Vote Outcomes: Results from Four Randomized Field Experiments.” Electoral Studies 41: 143–50.
Green Donald P., McGrath Mary C., and Aronow Peter M.. 2013. “Field Experiments and the Study of Voter Turnout.” Journal of Elections, Public Opinion and Parties 23 (1): 2748.
Hersh Eitan D. 2015. Hacking the Electorate: How Campaigns Perceive Voters. New York: Cambridge University Press.
Hersh Eitan D., and Schaffner Brian F.. 2013. “Targeted Campaign Appeals and the Value of Ambiguity.” The Journal of Politics 75 (2): 520–34.
Hill Seth J., Lo James, Vavreck Lynn, and Zaller John. 2013. “How Quickly We Forget: The Duration of Persuasion Effects from Mass Communication.” Political Communication 30 (4): 521–47.
Hillygus D. Sunshine, and Shields Todd G.. 2008. The Persuadable Voter: Wedge Issues in Presidential Campaigns. Princeton, NJ: Princeton University Press.
Holbrook Thomas M. 1996. Do Campaigns Matter? Thousand Oaks, CA: Sage.
Issenberg Sasha. 2012. The Victory Lab: The Secret Science of Winning Campaigns. New York: Crown.
Iyengar Shanto, and Simon Adam F.. 2000. “New Perspectives and Evidence on Political Communication and Campaign Effects.” Annual Review of Psychology 51 (1): 149–69.
Iyengar Shanto, Sood Gaurav, and Lelkes Yphtach. 2012. “Affect, Not Ideology: A Social Identity Perspective on Polarization.” Public Opinion Quarterly 76 (3): 405–31.
Jacobson Gary C. 2015. “How do Campaigns Matter?Annual Review of Political Science 18: 3147.
Johnston Richard, Hagen Michael G., and Jamieson Kathleen Hall. 2004. The 2000 Presidential Election and the Foundation of Party Politics. New York, NY: Cambridge University Press.
Kalla Joshua L., and Sekhon Jasjeet S.. 2017. “Do TV Ads Persuade? A Randomized Experiment in the 2016 Presidential Election.” Working Paper.
Keane Laura, and Nickerson David W.. 2013. “A Field Experiment on Nonpartisan Mobilization and Persuasion Down-Ballot.” Working Paper, URL: https://www.researchgate.net/publication/254065918_A_Field_Experiment_on_Nonpartisan_Mobilization_and_Persuasion_Down-Ballot.
Klapper Joseph T. 1960. The Effects of Mass Communication. New York, NY: Free Press.
Klar Samara, and Krupnikov Yanna. 2016. Independent Politics. Cambridge University Press.
Lau Richard R., Sigelman Lee, and Rovner Ivy Brown. 2007. “The Effects of Negative Political Campaigns: A Meta-Analytic Reassessment.” Journal of Politics 69 (4): 1176–209.
Lauderdale Benjamin E. 2016. “Partisan Disagreements Arising from Rationalization of Common Information.” Political Science Research and Methods 4 (3): 477–92.
Lazarsfeld Paul F., Berelson Bernard R., and Gaudet Hazel. 1948. The People’s Choice: How the Voter Makes Up His Mind in a Presidential Campaign. New York, NY: Columbia University Press.
Leeper Thomas J., and Slothuus Rune. 2015. “Can Citizens Be Framed? How Information, Not Emphasis, Changes Opinions.” Unpublished paper, Aarhus University.
Lenz Gabriel S. 2012. Follow the Leader? How Voters Respond to Politicians’ Performance and Policies. Chicago, IL: University of Chicago Press.
Miller Roy E., and Robyn Dorothy L.. 1975. “A Field Experimental Study of Direct Mail in a Congressional Primary Campaign: What Effects Last Until Election Day.” Experimental Study of Politics 4 (3): 136.
Nickerson David W. 2005. “Partisan Mobilization Using Volunteer Phone Banks and Door Hangers.” Annals of the American Academy of Political and Social Science 601 (1): 1027.
Nickerson David W. 2007 a. “Don’t Talk to Strangers: Experimental Evidence of the Need for Targeting.” Presented at the 2007 Annual Meeting of the Midwest Political Science Assocation. Available at https://www.scribd.com/document/98714549/Nickerson-independents.
Nickerson David W. 2007 b. “Quality Is Job One: Professional and Volunteer Voter Mobilization Calls.” American Journal of Political Science 51 (2): 269–82.
Nickerson David W., and Rogers Todd. 2010. “Do You Have a Voting Plan? Implementation Intentions, Voter Turnout, and Organic Plan Making.” Psychological Science 21 (2): 194–9.
Niven David. 2013. “Hiding Racism from Ourselves: A Field Experiment on Race Effects in a Statewide Election.” Paper presented at the Annual Meeting of the Midwest Political Science Association, Chicago, April 11–14, 2013.
Page Benjamin I., and Shapiro Robert Y.. 1992. The Rational Public: Fifty Years of Trends in Americans’ Policy Preferences. Chicago, IL: University of Chicago Press.
Paluck Elizabeth Levy, and Green Donald P.. 2009. “Deference, Dissent, and Dispute Resolution: An Experimental Intervention using Mass Media to Change Norms and Behavior in Rwanda.” American Political Science Review 103 (4): 622–44.
Panagopoulos Costas. 2016. “All about that base: Changing campaign strategies in US presidential elections.” Party Politics 22 (2): 179–90.
Petty Richard E. and Cacioppo John T.. 1986. “The Elaboration Likelihood Model of Persuasion.” In Advances in Experimental Social Psychology Volume 19, ed Leonard Berkowitz. New York, NY: Academic Press, 123205.
Potter Philip B. K. and Gray Julia. 2008. “Does Costly Signaling Matter? Preliminary Evidence from a Field Experiment.” Working paper, available at http://www.belfercenter.org/sites/default/files/files/publication/Potterr%202008%20FINAL%20DOC.pdf
Rogers Todd, and Nickerson David W.. 2013. “Can Inaccurate Beliefs about Incumbents Be Changed? And Can Reframing Change Votes?” Working Paper RWP13-018, Harvard Kennedy School. URL: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2271654.
Rogers Todd, and Middleton Joel. 2015. “Are Ballot Initiative Outcomes Influenced by the Campaigns of Independent Groups? A Precinct-Randomized Field Experiment Showing That They Are.” Political Behavior 37 (3): 567–93.
Sadin Meredith L. 2014. “Campaigning with Class: Ambivalent Stereotypes and Candidate Wealth in U.S. Elections.” PhD thesis, Princeton University.
Schickler Eric. 2016. Racial Realignment: The Transformation of American Liberalism, 1932–1965. Princeton, NJ: Princeton University Press.
Sears David O. and Funk Carolyn L.. 1999. “Evidence of the Long-term Persistence of Adults’ Political Predispositions.” The Journal of Politics 61 (1): 128.
Shaw Daron, Blunt Christopher, and Seaborn Brent. 2017. “Testing Overall and Synergistic Campaign Effects in a Partisan Statewide Election.” Working paper.
Shaw Daron R., Green Donald P., Gimpel James G., and Gerber Alan S.. 2012. “Do Robotic Calls From Credible Sources Influence Voter Turnout or Vote Choice? Evidence From a Randomized Field Experiment.” Journal of Political Marketing 11 (4): 231–45.
Shaw Daron R., and Gimpel James G.. 2012. “What if We Randomize the Governor’s Schedule? Evidence on Campaign Appearance Effects From a Texas Field Experiment.” Political Communication 29 (2): 137–59.
Sides John, and Vavreck Lynn. 2013. The Gamble. Princeton, NJ: Princeton University Press.
Simon Paul. 1994. P.S.: The Autobiography of Paul Simon. Boulder, CO: Taylor Trade Publishing.
Smidt Corwin D. 2017. “Polarization and the Decline of the American Floating Voter.” American Journal of Political Science 61 (2): 365–81.
Sniderman Paul M. and Theriault Sean M.. 2004. “The Structure of Political Argument and the Logic of Issue Framing.” In Studies in Public Opinion: Attitudes, Nonattitudes, Measurement Error, and Change, ed. Saris Willem E. and Sniderman Paul M., Princeton, NJ: Princeton University Press: 133–65.
Strauss Aaron B. 2009. “Political Ground Truth: How Personal Issue Experience Counters Partisan Biases” PhD thesis, Princeton University.
Ternovski John, Green Jennifer, and Kalla Joshua. 2012. Working America’s 2011 Persuasion Canvass Experiment on SB5 in Ohio. Technical report Analyst Institute.
Vavreck Lynn. 2009. The Message Matters: The Economy and Presidential Campaigns. Princeton, NJ: Princeton University Press.
Wantchekon Leonard. 2003. “Clientelism and Voting Behavior: Evidence from a Field Experiment in Benin.” World politics 55 (3): 399422.
Wlezien Christopher, and Erikson Robert S.. 2002. “The Timeline of Presidential Election Campaigns.” Journal of Politics 64 (4): 969–93.
Zaller John R. 1992. The Nature and Origins of Mass Opinion. New York, NY: Cambridge University Press.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

American Political Science Review
  • ISSN: 0003-0554
  • EISSN: 1537-5943
  • URL: /core/journals/american-political-science-review
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×
Type Description Title
PDF
Supplementary materials

Kalla and Broockman supplementary material 1
Appendix

 PDF (2.2 MB)
2.2 MB
UNKNOWN
Supplementary materials

Kalla and Broockman Dataset
Dataset

 Unknown

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 80
Total number of PDF views: 955 *
Loading metrics...

Abstract views

Total abstract views: 4993 *
Loading metrics...

* Views captured on Cambridge Core between 28th September 2017 - 25th February 2018. This data will be updated every 24 hours.