Skip to main content
×
×
Home

Does Counter-Attitudinal Information Cause Backlash? Results from Three Large Survey Experiments

  • Andrew Guess (a1) and Alexander Coppock (a2)
Abstract

Several theoretical perspectives suggest that when individuals are exposed to counter-attitudinal evidence or arguments, their pre-existing opinions and beliefs are reinforced, resulting in a phenomenon sometimes known as ‘backlash’. This article formalizes the concept of backlash and specifies how it can be measured. It then presents the results from three survey experiments – two on Mechanical Turk and one on a nationally representative sample – that find no evidence of backlash, even under theoretically favorable conditions. While a casual reading of the literature on information processing suggests that backlash is rampant, these results indicate that it is much rarer than commonly supposed.

  • View HTML
    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Does Counter-Attitudinal Information Cause Backlash? Results from Three Large Survey Experiments
      Available formats
      ×
      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      Does Counter-Attitudinal Information Cause Backlash? Results from Three Large Survey Experiments
      Available formats
      ×
      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      Does Counter-Attitudinal Information Cause Backlash? Results from Three Large Survey Experiments
      Available formats
      ×
Copyright
Corresponding author
*Corresponding author. Email: aguess@princeton.edu
References
Hide All
Ansolabehere, S, Rodden, J Snyder, JM (2008) The strength of issues: using multiple measures to gauge preference stability, ideological constraint, and issue voting. American Political Science Review 102 (2):215232.
Arceneaux, K Johnson, M (2013) Changing Minds Or Changing Channels?: Partisan News in an Age of Choice . Chicago, IL: University of Chicago Press.
Bakshy, E, Messing, S Adamic, LA (2015) Exposure to ideologically diverse news and opinion on Facebook. Science 348 (6239):11301132.
Barberá, P (2014) How social media reduces mass political polarization. Evidence from Germany, Spain, and the U.S. Working Paper.
Bartels, L (2002) Beyond the running tally: Partisan bias in political perceptions. Political Behavior 24 (2):117150.
Benoît, J-P Dubra, J (2016) A theory of rational attitude polarization. Available at SSRN 2529494.
Berinsky, A, Huber, G Lenz, G (2012) Evaluating online labor markets for experimental research: Amazon.com’s Mechanical Turk. Political Analysis 20 (3):351368.
Berinsky, AJ (2015) Rumors and health care reform: experiments in political misinformation. British Journal of Political Science 47 (2):122.
Berry, JM Sobieraj, S (2013) The Outrage Industry: Political Opinion Media and the New Incivility. Oxford: Oxford University Press.
Bishin, BG et al. (2016) Opinion backlash and public attitudes: Are political advances in gay rights counterproductive? American Journal of Political Science 60 (3):625648.
Buhrmester, MD, Kwang, T Gosling, SD (2011) Amazon’s Mechanical Turk: a new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science 6 (1):35.
Bullock, JG (2009) Partisan bias and the Bayesian ideal in the study of public opinion. The Journal of Politics 71 (3):11091124.
Bullock, JG et al. (2015) Partisan bias in factual beliefs about politics. Quarterly Journal of Political Science 10 (4):519578.
Chandler, J, Mueller, P Paolacci, G (2014) Nonnaïveté among Amazon Mechanical Turk workers: consequences and solutions for behavioral researchers. Behavior Research Methods 46 (1):112130.
Chipman, HA, George, EI McCulloch, RE (2007) Bayesian ensemble learning. Advances in Neural Information Processing Systems 19, 265.
Chipman, HA, George, EI McCulloch, RE (2010) BART: Bayesian additive regression trees. Annals of Applied Statistics 4 (1):266298.
Clifford, S Jerit, J (2014) Is there a cost to convenience? An experimental comparison of data quality in laboratory and online studies. Journal of Experimental Political Science 1 (2):120131.
Coppock, A (2016) Randomizr: Easy to use tools for common forms of random assignment and sampling. R package version 0.5.0.
Coppock, A (2017) Generalizing from survey experiments conducted on Mechanical Turk: a replication approach. Political Science Research and Methods. doi: 10.1017/psrm.2018.10.
Coppock, A Green, DP (2015) Assessing the correspondence between experimental results obtained in the lab and field: a review of recent social science research. Political Science Research and Methods 3 (1):113131.
Druckman, JN (2012) The politics of motivation. Critical Review 24 (2):199216.
Druckman, JN, Levendusky, MS McLain, A (2018) No need to watch: how the effects of partisan media can spread via inter-personal discussions. American Journal of Political Science 62 (1):99112.
Gelman, A Loken, E (2016) The statistical crisis in science. The Best Writing on Mathematics 2015, 305.
Gerber, A Green, D (1999) Misperceptions about perceptual bias. Annual Review of Political Science 2 (1):189210.
Green, DP Kern, HL (2012) Modeling heterogeneous treatment effects in survey experiments with Bayesian additive regression trees. Public Opinion Quarterly 76 (3):491511.
Guess, Andrew Coppock, Alexander (2018) “Replication Data for: Does Counter-Attitudinal Information Cause Backlash? Results from Three Large Survey Experiments”, https://doi.org/10.7910/DVN/J7WNTM, Harvard Dataverse, V1.
Guess, A et al. (2018) Avoiding the echo chamber about echo chambers: why selective exposure to like-minded political news is less prevalent than you think. The Knight Foundation White Paper.
Guess, AM (N.d.) Media choice and moderation: evidence from online tracking data. Unpublished manuscript.
Hauser, DJ Schwarz, N (2016) Attentive turkers: mturk participants perform better on online attention checks than do subject pool participants. Behavior Research Methods 48 (1):400407.
Hill, JL (2011) Bayesian nonparametric modeling for causal inference. Journal of Computational and Graphical Statistics 20 (1):217240.
Holland, PW (1986) Statistics and causal inference. Journal of the American Statistical Association 81 (396):945960.
Humphreys, M, Sanchez de la Sierra, R van der Windt, P (2013) Fishing, commitment, and communication: a proposal for comprehensive nonbinding research registration. Political Analysis 21 (1):120.
Kahan, D (2013) Fooled Twice, Shame on Who? Problems with Mechanical Turk Study Samples, Part 2. Available at http://www.culturalcognition.net/blog/2013/7/10/fooled-twice-shame-on-who-problems-with-mechanical-turk-stud.html, accessed 1 January 2016.
Kahan, DM (2012) Ideology, motivated reasoning, and cognitive reflection: an experimental study. Judgment and Decision Making 8, 407424.
Kerr, NL (1998) Harking: hypothesizing after the results are known. Personality and Social Psychology Review 2 (3):196217.
Kuhn, D Lao, J (1996) Effects of evidence on attitudes: is polarization the norm. Psychological Science 7 (2):115120.
Kuklinski, JH et al. (2001) The political environment and citizen competence. American Journal of Political Science 45 (2):410424.
Kunda, Z (1990) The case for motivated reasoning. Psychological Bulletin 108, 480498.
Lau, R Redlawsk, D (2006) How Voters Decide: Information Processing During Election Campaigns. Cambridge: Cambridge University Press.
Lazarsfeld, PF, Berelson, B Gaudet, H (1944) The People’s Choice: How the Voter Makes Up His Mind in a Presidential Campaign. New York: Columbia University Press.
Leeper, TJ (2017) MTurkR: Access to Amazon Mechanical Turk Requester API via R. R package version 0.8.0.
Leeper, TJ Slothuus, R (2014) Political parties, motivated reasoning, and public opinion formation. Advances in Political Psychology 35, 129156.
Leeper, TJ Thorson, E (2015) Minimal sponsorship-induced bias in web survey data. Paper presented at the 2015 annual meeting of the Midwest Political Science Association, Chicago, IL.
Levendusky, MS (2013) Why do partisan media polarize viewers? American Journal of Political Science 57 (3):611623.
Lodge, M Taber, CS (2013) The Rationalizing Voter. Cambridge: Cambridge University Press.
Lord, CS, Ross, L Lepper, M (1979) Biased assimilation and attitude polarization: the effects of prior theories on subsequently considered evidence. Journal of Personality and Social Psychology 37, 20982109.
Miller, AG et al. (1993) The attitude polarization phenomenon: role of response measure, attitude extremity, and behavioral consequences of reported attitude change. Journal of Personality and Social Psychology 64 (4):561574.
Moore, RT (2015) blockTools: blocking, assignment, and diagnosing interference in randomized experiments. R package version 0.6-2.
Mullinix, KJ et al. (2015) The generalizability of survey experiments. Journal of Experimental Political Science 2, 109138.
Mummolo, J Peterson, E (2017) Demand effects in survey experiments: an empirical assessment. doi: 10.2139/ssrn.2956147.
Nyhan, B Reifler, J (2010) When corrections fail: the persistence of political misperceptions. Political Behavior 32 (2):303330.
Nyhan, B Reifler, J (2015) Does correcting myths about the flu vaccine work? An experimental evaluation of the effects of corrective information. Vaccine 33 (3):459464.
Nyhan, B et al. (2014) Effective messages in vaccine promotion: A randomized trial. Pediatrics 133 (4):835842.
Prior, M (2007) ) Post-Broadcast Democracy: How Media Choice Increases Inequality in Political Involvement and Polarizes Elections . Cambridge: Cambridge University Press.
Prior, M et al. (2015) You cannot be serious: the impact of accuracy incentives on partisan bias in reports of economic perceptions. Quarterly Journal of Political Science 10 (4):489518.
Redlawsk, DP (2002) Hot cognition or cool consideration? Testing the effects of motivated reasoning on political decision making. The Journal of Politics 64 (4):10211044.
Redlawsk, DP, Civettini, AJW Emmerson, KM (2010) The affective tipping point: do motivated reasoners ever “Get It”? Political Psychology 31 (4):563593.
Sears, DO (1986) College sophomores in the laboratory: influences of a narrow data base on social psychology’s view of human nature. Journal of Personality and Social Psychology 51 (3):515530.
Strickland, AA, Taber, CS Lodge, M (2011) Motivated reasoning and public opinion. Journal of Health Politics, Policy and Law 36 (6):935944.
Taber, CS, Cann, D Kucsova, S (2009) The motivated processing of political arguments. Political Behavior 31 (2):137155.
Taber, CS Lodge, M (2006) Motivated skepticism in the evaluation of political beliefs. American Journal of Political Science 50 (3):755769.
Tversky, A Kahneman, D (1974) Judgment under uncertainty: heuristics and biases. Science 185 (4157):11241131.
White, A et al. (2016) Investigator characteristics and respondent behavior in online surveys. Journal of Experimental Political Science 5 (10):5667.
Wood, T Porter, E (2018) The elusive backfire effect: mass attitudes’ steadfast factual adherence. Political Behavior. Forthcoming.
Zaller, JR (1992) The Nature and Origins of Mass Opinion. New York: Cambridge University Press.
Zhou, J (2016) Boomerangs versus javelins: how polarization constrains communication on climate change. Environmental Politics 25 (5):788811.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

British Journal of Political Science
  • ISSN: 0007-1234
  • EISSN: 1469-2112
  • URL: /core/journals/british-journal-of-political-science
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Keywords

Type Description Title
UNKNOWN
Supplementary materials

Guess and Coppock Dataset
Dataset

 Unknown
PDF
Supplementary materials

Guess and Coppock supplementary material
Online Appendix

 PDF (2.8 MB)
2.8 MB

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed