Skip to main content
×
×
Home

The Minimal Persuasive Effects of Campaign Contact in General Elections: Evidence from 49 Field Experiments

  • JOSHUA L. KALLA (a1) and DAVID E. BROOCKMAN (a2)
Abstract

Significant theories of democratic accountability hinge on how political campaigns affect Americans’ candidate choices. We argue that the best estimate of the effects of campaign contact and advertising on Americans’ candidates choices in general elections is zero. First, a systematic meta-analysis of 40 field experiments estimates an average effect of zero in general elections. Second, we present nine original field experiments that increase the statistical evidence in the literature about the persuasive effects of personal contact tenfold. These experiments’ average effect is also zero. In both existing and our original experiments, persuasive effects only appear to emerge in two rare circumstances. First, when candidates take unusually unpopular positions and campaigns invest unusually heavily in identifying persuadable voters. Second, when campaigns contact voters long before election day and measure effects immediately—although this early persuasion decays. These findings contribute to ongoing debates about how political elites influence citizens’ judgments.

Copyright
Corresponding author
Joshua L. Kalla is a Graduate Student, Department of Political Science, University of California, Berkeley. kalla@berkeley.edu, http://polisci.berkeley.edu/people/person/joshua-kalla.
David E. Broockman is an Assistant Professor, Stanford Graduate School of Business. dbroockman@stanford.edu, https://people.stanford.edu/dbroock/.
Footnotes
Hide All

We acknowledge seminar participants at the Midwest Political Science Association conference, the Northeast Political Methodology Meeting at NYU, Berkeley, Stanford, and the University of British Columbia, as well as Adam Berinsky, Donald Green, Avi Feller, Shanto Iyengar, Jon Krosnick, Gabriel Lenz, Joel Middleton, Daron Shaw, Jas Sekhon, Eric Schickler, Laura Stoker, and Lynn Vavreck for helpful feedback. All remaining errors are our own. The original studies reported herein were conducted by Working America. The authors served as unpaid consultants to Working America in their personal capacity.

Footnotes
References
Hide All
Abramowitz, Alan I. 2010. The Disappearing Center: Engaged Citizens, Polarization, and American Democracy. New Haven, CT: Yale University Press.
Adams, William C., and Smith, Dennis J.. 1980. “Effects of Telephone Canvassing on Turnout and Preferences: A Field Experiment.” Public Opinion Quarterly 44 (3): 389–95.
Ansolabehere, Stephen. 2006. “The Paradox of Minimal Effects.” In Capturing Campaign Effects, ed. Brady, Henry E. and Johnston, Richard. Ann Arbor: University of Michigan Press, 2944.
Anzia, Sarah F. 2011. “Election Timing and the Electoral Influence of Interest Groups.” The Journal of Politics 73 (2): 412–27.
Arceneaux, Kevin. 2005. “Using Cluster Randomized Field Experiments to Study Voting Behavior.” Annals of the American Academy of Political and Social Science 601 (1): 169–79.
Arceneaux, Kevin. 2007. “I’m Asking for Your Support: The Effects of Personally Delivered Campaign Messages on Voting Decisions and Opinion Formation.” Quarterly Journal of Political Science 2 (1): 4365.
Arceneaux, Kevin, and Nickerson, David W.. 2010. “Comparing Negative and Positive Campaign Messages: Evidence From Two Field Experiments.” American Politics Research 38 (1): 5483.
Arceneaux, Kevin, and Kolodny, Robin. 2009. “Educating the Least Informed: Group Endorsements in a Grassroots Campaign.” American Journal of Political Science 53 (4): 755–70.
Bailey, Michael A., Hopkins, Daniel J., and Rogers, Todd. 2016. “Unresponsive, Unpersuaded: The Unintended Consequences of Voter Persuasion Efforts.” Political Behavior 38 (3): 713–46.
Barabas, Jason, and Jerit, Jennifer. 2010. “Are Survey Experiments Externally Valid?American Political Science Review 104 (2): 226–42.
Barton, Jared, Castillo, Marco, and Petrie, Ragan. 2014. “What Persuades Voters? A Field Experiment on Political Campaigning.” The Economic Journal 124 (574): F293F326.
Bawn, Kathleen, Cohen, Martin, Karol, David, Masket, Seth, Noel, Hans, and Zaller, John. 2012. “A Theory of Political Parties: Groups, Policy Demands and Nominations in American Politics.” Perspectives on Politics 10 (3): 571–97.
Berelson, Bernard R., Lazarsfeld, Paul F., and McPhee, William N.. 1954. Voting: A Study of Opinion Formation in a Presidential Campaign. Chicago, IL: University of Chicago Press.
Berinsky, Adam J. 2009. In Time of War: Understanding American Public Opinion From World War II to Iraq. Chicago, IL: University of Chicago Press.
Berinsky, Adam J. 2017. “Measuring Public Opinion with Surveys.” Annual Review of Political Science 20: 309–29.
Brady, Henry E., Johnston, Richard, and Sides, John. 2006. “The Study of Political Campaigns.” In Capturing Campaign Effects, ed. Brady, Henry E. and Johnston, Richard. Ann Arbor, MI: University of Michigan Press, 126.
Broockman, David, and Green, Donald. 2014. “Do Online Advertisements Increase Political Candidates’ Name Recognition or Favorability? Evidence from Randomized Field Experiments.” Political Behavior 36 (2): 263–89.
Broockman, David E., and Kalla, Joshua L.. 2016. “Durably Reducing Transphobia: A Field Experiment on Door-to-door Canvassing.” Science 352 (6282): 220–4.
Broockman, David, Kalla, Joshua L., and Sekhon, Jasjeet S.. 2017. “The Design of Field Experiments With Survey Outcomes: A Framework for Selecting More Efficient, Robust, and Ethical Designs.” Political Analysis.
Brox, Brian J., and Shaw, Daron R.. 2009. “Political Parties, American Campaigns, and Effects on Outcomes.” In Handbook of Party Politics, ed Katz, Richard S. & Crotty, William. London: Sage, 146–59.
Campbell, Angus, Converse, Philip E., Miller, Warren E., and Stokes, Donald E.. 1960. The American Voter. Chicago: University of Chicago Press.
Cantoni, Enrico and Pons, Vincent. 2017. “Do Interactions with Candidates Increase Voter Support and Participation? Experimental Evidence from Italy.” MPSA Conference Paper.
Cardy, Emily Arthur. 2005. “An Experimental Field Study of the GOTV and Persuasion Effects of Partisan Direct Mail and Phone Calls.” The Annals of the American Academy of Political and Social Science 601 (1): 2840.
Carpenter, Daniel. 2016. “Recruitment by Petition: American Antislavery, French Protestantism, English Suppression.” Perspectives on Politics 14 (3): 700–23.
Chong, Dennis, and Druckman, James N. 2007. “A Theory of Framing and Opinion Formation in Competitive Elite Environments.” Journal of Communication 57 (1): 99118.
Cubbison, William. 2015. “The Marginal Effects of Direct Mail on Vote Choice.” Paper presented at the Annual Meeting of the Midwest Political Science Association. URL: http://media.wix.com/ugd/3a8c0a_47330c730f56431f8f982a3d842f434a.pdf.
Cunow, Saul, and Schwenzfeier, Meg. 2015. Working America 2014 Persuasion Experiments Results Memo. Technical Report Analyst Institute.
Deaton, Angus. 2010. “Instruments, Randomization, and Learning about Development.” Journal of Economic Literature 48 (2): 424–55.
DellaVigna, Stefano, and Gentzkow, Matthew. 2010. “Persuasion: Empirical Evidence.” Annual Review of Economics 2: 643–69.
Doherty, David, and Adler, E. Scott. 2014. “The Persuasive Effects of Partisan Campaign Mailers.” Political Research Quarterly 67 (3): 562573.
Druckman, James N. 2001. “The Implications of Framing Effects for Citizen Competence.” Political Behavior 23 (3): 225–56.
Druckman, James N. 2004a. “Political Preference Formation: Competition, Deliberation, and the (Ir) Relevance of Framing Effects.” American Political Science Review 98 (4): 671–86.
Druckman, James N. 2004b. “Priming the Vote: Campaign Effects in a U.S. Senate Election.” Political Psychology 25 (04): 577–94.
Druckman, James N., Peterson, Erik, and Slothuus, Rune. 2013. “How Elite Partisan Polarization Affects Public Opinion Formation.” American Political Science Review 107 (01): 5779.
Druckman, James N., and Leeper, Thomas J.. 2012a. “Is Public Opinion Stable? Resolving the Micro/Macro Disconnect in Studies of Public Opinion.” Daedalus 141 (4): 5068.
Druckman, James N., and Leeper, Thomas J.. 2012b. “Learning More from Political Communication Experiments: Pretreatment and Its Effects.” American Journal of Political Science 56 (4): 875–96.
Endres, Kyle. 2016. “The Accuracy of Microtargeted Policy Positions.” PS: Political Science & Politics 49 (4): 771–4.
Enos, Ryan D. and Fowler, Anthony. 2016. “Aggregate Effects of Large-Scale Campaigns on Voter Turnout.” Political Science Research and Methods: 119. https://doi.org/10.1017/psrm.2016.21
Enos, Ryan D., and Hersh, Eitan D.. 2015. “Party Activists as Campaign Advertisers: The Ground Campaign as a Principal-Agent Problem.” American Political Science Review 109 (2): 252–78.
Follman, Dean A., and Proschan, Michael A.. 1999. “Valid Inference in Random Effects Meta-analysis.” Biometrics 55 (3): 732–7.
Foos, Florian. 2017. “First Impressions - Lasting Impressions? The Short- and Long-term Effects of Candidate Contact on Voting Intentions During a High-salience Campaign.” MPSA Conference Paper.
Fowler, Erika Franklin, Ridout, Travis N., and Franz, Michael M.. 2016. “Political Advertising in 2016: The Presidential Election as Outlier?The Forum: A Journal of Applied Research in Contemporary Politics 4 (14): 445–69.
Franco, Annie, Malhotra, Neil, and Simonovits, Gabor. 2014. “Publication Bias in the Social Sciences: Unlocking the File Drawer.” Science 345 (6203): 1502–05.
Freeder, Sean, Lenz, Gabriel S., and Turney, Shad. 2017. “The Importance of Knowing ‘What Goes With What’: Reinterpreting the Evidence on Policy Attitude Stability.” Working paper, available at https://www.ocf.berkeley.edu/~glenz/wgww/WGWW.pdf.
Gelman, Andrew, and King, Gary. 1993. “Why Are American Presidential Election Campaign Polls so Variable When Votes Are so Predictable?British Journal of Political Science 23 (4): 409–51.
Gelman, Andrew, Goel, Sharad, Rivers, Douglas, Rothschild, David et al. 2016. “The Mythical Swing Voter.” Quarterly Journal of Political Science 11 (1): 103–30.
Gerber, Alan S. 2004. “Does Campaign Spending Work? Field Experiments Provide Evidence and Suggest New Theory.” American Behavioral Scientist 47 (5): 541–74.
Gerber, Alan S., Kessler, Daniel P., and Meredith, Marc. 2011. “The Persuasive Effects of Direct Mail: A Regression Discontinuity Based Approach.” The Journal of Politics 73 (1): 140–55.
Gerber, Alan S., Karlan, Dean, and Bergan, Daniel. 2009. “Does the Media Matter? A Field Experiment Measuring the Effect of Newspapers on Voting Behavior and Political Opinions.” American Economic Journal: Applied Economics 1 (2): 3552.
Gerber, Alan S., and Green, Donald P.. 2000. “The Effects of Canvassing, Telephone Calls, and Direct Mail on Voter Turnout: A Field Experiment.” American Political Science Review 94 (3): 653–63.
Gerber, Alan S., Green, Donald P., and Larimer, Christopher W.. 2008. “Social Pressure and Voter Turnout: Evidence from a Large-Scale Field Experiment.” American Political Science Review 102 (1): 3348.
Gerber, Alan S., Gimpel, James, Green, Donald, and Shaw, Daron. 2011. “How Large and Long-lasting Are the Persuasive Effects of Televised Campaign Ads? Results from a Randomized Experiment.” American Political Science Review 105 (1): 135–50.
Green, Donald, Palmquist, Bradley, and Schickler, Eric. 2002. Partisan Hearts & Minds: Political Parties and the Social Identities of Voters. New Haven, CT: Yale University Press.
Green, Donald P., and Gerber, Alan S.. 2015. Get Out the Vote: How to Increase Voter Turnout. 3rd ed. Brookings.
Green, Donald P., Krasno, Jonathan S., Coppock, Alexander, Farrer, Benjamin D., Lenoir, Brandon, and Zingher, Joshua N.. 2016. “The Effects of Lawn Signs on Vote Outcomes: Results from Four Randomized Field Experiments.” Electoral Studies 41: 143–50.
Green, Donald P., McGrath, Mary C., and Aronow, Peter M.. 2013. “Field Experiments and the Study of Voter Turnout.” Journal of Elections, Public Opinion and Parties 23 (1): 2748.
Hersh, Eitan D. 2015. Hacking the Electorate: How Campaigns Perceive Voters. New York: Cambridge University Press.
Hersh, Eitan D., and Schaffner, Brian F.. 2013. “Targeted Campaign Appeals and the Value of Ambiguity.” The Journal of Politics 75 (2): 520–34.
Hill, Seth J., Lo, James, Vavreck, Lynn, and Zaller, John. 2013. “How Quickly We Forget: The Duration of Persuasion Effects from Mass Communication.” Political Communication 30 (4): 521–47.
Hillygus, D. Sunshine, and Shields, Todd G.. 2008. The Persuadable Voter: Wedge Issues in Presidential Campaigns. Princeton, NJ: Princeton University Press.
Holbrook, Thomas M. 1996. Do Campaigns Matter? Thousand Oaks, CA: Sage.
Issenberg, Sasha. 2012. The Victory Lab: The Secret Science of Winning Campaigns. New York: Crown.
Iyengar, Shanto, and Simon, Adam F.. 2000. “New Perspectives and Evidence on Political Communication and Campaign Effects.” Annual Review of Psychology 51 (1): 149–69.
Iyengar, Shanto, Sood, Gaurav, and Lelkes, Yphtach. 2012. “Affect, Not Ideology: A Social Identity Perspective on Polarization.” Public Opinion Quarterly 76 (3): 405–31.
Jacobson, Gary C. 2015. “How do Campaigns Matter?Annual Review of Political Science 18: 3147.
Johnston, Richard, Hagen, Michael G., and Jamieson, Kathleen Hall. 2004. The 2000 Presidential Election and the Foundation of Party Politics. New York, NY: Cambridge University Press.
Kalla, Joshua L., and Sekhon, Jasjeet S.. 2017. “Do TV Ads Persuade? A Randomized Experiment in the 2016 Presidential Election.” Working Paper.
Keane, Laura, and Nickerson, David W.. 2013. “A Field Experiment on Nonpartisan Mobilization and Persuasion Down-Ballot.” Working Paper, URL: https://www.researchgate.net/publication/254065918_A_Field_Experiment_on_Nonpartisan_Mobilization_and_Persuasion_Down-Ballot.
Klapper, Joseph T. 1960. The Effects of Mass Communication. New York, NY: Free Press.
Klar, Samara, and Krupnikov, Yanna. 2016. Independent Politics. Cambridge University Press.
Lau, Richard R., Sigelman, Lee, and Rovner, Ivy Brown. 2007. “The Effects of Negative Political Campaigns: A Meta-Analytic Reassessment.” Journal of Politics 69 (4): 1176–209.
Lauderdale, Benjamin E. 2016. “Partisan Disagreements Arising from Rationalization of Common Information.” Political Science Research and Methods 4 (3): 477–92.
Lazarsfeld, Paul F., Berelson, Bernard R., and Gaudet, Hazel. 1948. The People’s Choice: How the Voter Makes Up His Mind in a Presidential Campaign. New York, NY: Columbia University Press.
Leeper, Thomas J., and Slothuus, Rune. 2015. “Can Citizens Be Framed? How Information, Not Emphasis, Changes Opinions.” Unpublished paper, Aarhus University.
Lenz, Gabriel S. 2012. Follow the Leader? How Voters Respond to Politicians’ Performance and Policies. Chicago, IL: University of Chicago Press.
Miller, Roy E., and Robyn, Dorothy L.. 1975. “A Field Experimental Study of Direct Mail in a Congressional Primary Campaign: What Effects Last Until Election Day.” Experimental Study of Politics 4 (3): 136.
Nickerson, David W. 2005. “Partisan Mobilization Using Volunteer Phone Banks and Door Hangers.” Annals of the American Academy of Political and Social Science 601 (1): 1027.
Nickerson, David W. 2007 a. “Don’t Talk to Strangers: Experimental Evidence of the Need for Targeting.” Presented at the 2007 Annual Meeting of the Midwest Political Science Assocation. Available at https://www.scribd.com/document/98714549/Nickerson-independents.
Nickerson, David W. 2007 b. “Quality Is Job One: Professional and Volunteer Voter Mobilization Calls.” American Journal of Political Science 51 (2): 269–82.
Nickerson, David W., and Rogers, Todd. 2010. “Do You Have a Voting Plan? Implementation Intentions, Voter Turnout, and Organic Plan Making.” Psychological Science 21 (2): 194–9.
Niven, David. 2013. “Hiding Racism from Ourselves: A Field Experiment on Race Effects in a Statewide Election.” Paper presented at the Annual Meeting of the Midwest Political Science Association, Chicago, April 11–14, 2013.
Page, Benjamin I., and Shapiro, Robert Y.. 1992. The Rational Public: Fifty Years of Trends in Americans’ Policy Preferences. Chicago, IL: University of Chicago Press.
Paluck, Elizabeth Levy, and Green, Donald P.. 2009. “Deference, Dissent, and Dispute Resolution: An Experimental Intervention using Mass Media to Change Norms and Behavior in Rwanda.” American Political Science Review 103 (4): 622–44.
Panagopoulos, Costas. 2016. “All about that base: Changing campaign strategies in US presidential elections.” Party Politics 22 (2): 179–90.
Petty, Richard E. and Cacioppo, John T.. 1986. “The Elaboration Likelihood Model of Persuasion.” In Advances in Experimental Social Psychology Volume 19, ed Leonard Berkowitz. New York, NY: Academic Press, 123205.
Potter, Philip B. K. and Gray, Julia. 2008. “Does Costly Signaling Matter? Preliminary Evidence from a Field Experiment.” Working paper, available at http://www.belfercenter.org/sites/default/files/files/publication/Potterr%202008%20FINAL%20DOC.pdf
Rogers, Todd, and Nickerson, David W.. 2013. “Can Inaccurate Beliefs about Incumbents Be Changed? And Can Reframing Change Votes?” Working Paper RWP13-018, Harvard Kennedy School. URL: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2271654.
Rogers, Todd, and Middleton, Joel. 2015. “Are Ballot Initiative Outcomes Influenced by the Campaigns of Independent Groups? A Precinct-Randomized Field Experiment Showing That They Are.” Political Behavior 37 (3): 567–93.
Sadin, Meredith L. 2014. “Campaigning with Class: Ambivalent Stereotypes and Candidate Wealth in U.S. Elections.” PhD thesis, Princeton University.
Schickler, Eric. 2016. Racial Realignment: The Transformation of American Liberalism, 1932–1965. Princeton, NJ: Princeton University Press.
Sears, David O. and Funk, Carolyn L.. 1999. “Evidence of the Long-term Persistence of Adults’ Political Predispositions.” The Journal of Politics 61 (1): 128.
Shaw, Daron, Blunt, Christopher, and Seaborn, Brent. 2017. “Testing Overall and Synergistic Campaign Effects in a Partisan Statewide Election.” Working paper.
Shaw, Daron R., Green, Donald P., Gimpel, James G., and Gerber, Alan S.. 2012. “Do Robotic Calls From Credible Sources Influence Voter Turnout or Vote Choice? Evidence From a Randomized Field Experiment.” Journal of Political Marketing 11 (4): 231–45.
Shaw, Daron R., and Gimpel, James G.. 2012. “What if We Randomize the Governor’s Schedule? Evidence on Campaign Appearance Effects From a Texas Field Experiment.” Political Communication 29 (2): 137–59.
Sides, John, and Vavreck, Lynn. 2013. The Gamble. Princeton, NJ: Princeton University Press.
Simon, Paul. 1994. P.S.: The Autobiography of Paul Simon. Boulder, CO: Taylor Trade Publishing.
Smidt, Corwin D. 2017. “Polarization and the Decline of the American Floating Voter.” American Journal of Political Science 61 (2): 365–81.
Sniderman, Paul M. and Theriault, Sean M.. 2004. “The Structure of Political Argument and the Logic of Issue Framing.” In Studies in Public Opinion: Attitudes, Nonattitudes, Measurement Error, and Change, ed. Saris, Willem E. and Sniderman, Paul M., Princeton, NJ: Princeton University Press: 133–65.
Strauss, Aaron B. 2009. “Political Ground Truth: How Personal Issue Experience Counters Partisan Biases” PhD thesis, Princeton University.
Ternovski, John, Green, Jennifer, and Kalla, Joshua. 2012. Working America’s 2011 Persuasion Canvass Experiment on SB5 in Ohio. Technical report Analyst Institute.
Vavreck, Lynn. 2009. The Message Matters: The Economy and Presidential Campaigns. Princeton, NJ: Princeton University Press.
Wantchekon, Leonard. 2003. “Clientelism and Voting Behavior: Evidence from a Field Experiment in Benin.” World politics 55 (3): 399422.
Wlezien, Christopher, and Erikson, Robert S.. 2002. “The Timeline of Presidential Election Campaigns.” Journal of Politics 64 (4): 969–93.
Zaller, John R. 1992. The Nature and Origins of Mass Opinion. New York, NY: Cambridge University Press.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

American Political Science Review
  • ISSN: 0003-0554
  • EISSN: 1537-5943
  • URL: /core/journals/american-political-science-review
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×
Type Description Title
PDF
Supplementary materials

Kalla and Broockman supplementary material 1
Appendix

 PDF (2.2 MB)
2.2 MB
UNKNOWN
Supplementary materials

Kalla and Broockman Dataset
Dataset

 Unknown

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed