Hostname: page-component-586b7cd67f-rcrh6 Total loading time: 0 Render date: 2024-11-26T20:07:52.943Z Has data issue: false hasContentIssue false
  • English
  • Français

Does Counter-Attitudinal Information Cause Backlash? Results from Three Large Survey Experiments

Published online by Cambridge University Press:  05 November 2018

Andrew Guess  and
Alexander Coppock
Andrew Guess*
Affiliation:
Department of Politics, Princeton University
Alexander Coppock
Affiliation:
Department of Political Science, Yale University
*
*Corresponding author. Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Several theoretical perspectives suggest that when individuals are exposed to counter-attitudinal evidence or arguments, their pre-existing opinions and beliefs are reinforced, resulting in a phenomenon sometimes known as ‘backlash’. This article formalizes the concept of backlash and specifies how it can be measured. It then presents the results from three survey experiments – two on Mechanical Turk and one on a nationally representative sample – that find no evidence of backlash, even under theoretically favorable conditions. While a casual reading of the literature on information processing suggests that backlash is rampant, these results indicate that it is much rarer than commonly supposed.

Keywords

public opinionattitude polarizationmotivated reasoningbackfire effect
Type
Articles
Information
British Journal of Political Science , Volume 50 , Issue 4 , October 2020 , pp. 1497 - 1515
Copyright
© Cambridge University Press 2018

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Ansolabehere, S, Rodden, J and Snyder, JM (2008) The strength of issues: using multiple measures to gauge preference stability, ideological constraint, and issue voting. American Political Science Review 102 (2):215–232.10.1017/S0003055408080210CrossRefGoogle Scholar
Arceneaux, K and Johnson, M (2013) Changing Minds Or Changing Channels?: Partisan News in an Age of Choice. Chicago, IL: University of Chicago Press.10.7208/chicago/9780226047447.001.0001CrossRefGoogle Scholar
Bakshy, E, Messing, S and Adamic, LA (2015) Exposure to ideologically diverse news and opinion on Facebook. Science 348 (6239):1130–1132.10.1126/science.aaa1160CrossRefGoogle ScholarPubMed
Barberá, P (2014) How social media reduces mass political polarization. Evidence from Germany, Spain, and the U.S. Working Paper.Google Scholar
Bartels, L (2002) Beyond the running tally: Partisan bias in political perceptions. Political Behavior 24 (2):117–150.10.1023/A:1021226224601CrossRefGoogle Scholar
Benoît, J-P and Dubra, J (2016) A theory of rational attitude polarization. Available at SSRN 2529494.10.2139/ssrn.2754316CrossRefGoogle Scholar
Berinsky, A, Huber, G and Lenz, G (2012) Evaluating online labor markets for experimental research: Amazon.com’s Mechanical Turk. Political Analysis 20 (3):351–368.10.1093/pan/mpr057CrossRefGoogle Scholar
Berinsky, AJ (2015) Rumors and health care reform: experiments in political misinformation. British Journal of Political Science 47 (2):1–22.Google Scholar
Berry, JM and Sobieraj, S (2013) The Outrage Industry: Political Opinion Media and the New Incivility. Oxford: Oxford University Press.Google Scholar
Bishin, BG et al. (2016) Opinion backlash and public attitudes: Are political advances in gay rights counterproductive? American Journal of Political Science 60 (3):625–648.10.1111/ajps.12181CrossRefGoogle Scholar
Buhrmester, MD, Kwang, T and Gosling, SD (2011) Amazon’s Mechanical Turk: a new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science 6 (1):3–5.10.1177/1745691610393980CrossRefGoogle ScholarPubMed
Bullock, JG (2009) Partisan bias and the Bayesian ideal in the study of public opinion. The Journal of Politics 71 (3):1109–1124.10.1017/S0022381609090914CrossRefGoogle Scholar
Bullock, JG et al. (2015) Partisan bias in factual beliefs about politics. Quarterly Journal of Political Science 10 (4):519–578.10.1561/100.00014074CrossRefGoogle Scholar
Chandler, J, Mueller, P and Paolacci, G (2014) Nonnaïveté among Amazon Mechanical Turk workers: consequences and solutions for behavioral researchers. Behavior Research Methods 46 (1):112–130.10.3758/s13428-013-0365-7CrossRefGoogle ScholarPubMed
Chipman, HA, George, EI and McCulloch, RE (2007) Bayesian ensemble learning. Advances in Neural Information Processing Systems 19, 265.Google Scholar
Chipman, HA, George, EI and McCulloch, RE (2010) BART: Bayesian additive regression trees. Annals of Applied Statistics 4 (1):266–298.10.1214/09-AOAS285CrossRefGoogle Scholar
Clifford, S and Jerit, J (2014) Is there a cost to convenience? An experimental comparison of data quality in laboratory and online studies. Journal of Experimental Political Science 1 (2):120–131.10.1017/xps.2014.5CrossRefGoogle Scholar
Coppock, A (2016) Randomizr: Easy to use tools for common forms of random assignment and sampling. R package version 0.5.0.Google Scholar
Coppock, A (2017) Generalizing from survey experiments conducted on Mechanical Turk: a replication approach. Political Science Research and Methods. doi: 10.1017/psrm.2018.10.CrossRefGoogle Scholar
Coppock, A and Green, DP (2015) Assessing the correspondence between experimental results obtained in the lab and field: a review of recent social science research. Political Science Research and Methods 3 (1):113–131.10.1017/psrm.2014.10CrossRefGoogle Scholar
Druckman, JN (2012) The politics of motivation. Critical Review 24 (2):199–216.10.1080/08913811.2012.711022CrossRefGoogle Scholar
Druckman, JN, Levendusky, MS and McLain, A (2018) No need to watch: how the effects of partisan media can spread via inter-personal discussions. American Journal of Political Science 62 (1):99–112.10.1111/ajps.12325CrossRefGoogle Scholar
Gelman, A and Loken, E (2016) The statistical crisis in science. The Best Writing on Mathematics 2015, 305.Google Scholar
Gerber, A and Green, D (1999) Misperceptions about perceptual bias. Annual Review of Political Science 2 (1):189–210.10.1146/annurev.polisci.2.1.189CrossRefGoogle Scholar
Green, DP and Kern, HL (2012) Modeling heterogeneous treatment effects in survey experiments with Bayesian additive regression trees. Public Opinion Quarterly 76 (3):491–511.10.1093/poq/nfs036CrossRefGoogle Scholar
Guess, Andrew and Coppock, Alexander (2018) “Replication Data for: Does Counter-Attitudinal Information Cause Backlash? Results from Three Large Survey Experiments”, https://doi.org/10.7910/DVN/J7WNTM, Harvard Dataverse, V1.CrossRefGoogle Scholar
Guess, A et al. (2018) Avoiding the echo chamber about echo chambers: why selective exposure to like-minded political news is less prevalent than you think. The Knight Foundation White Paper.Google Scholar
Guess, AM (N.d.) Media choice and moderation: evidence from online tracking data. Unpublished manuscript.Google Scholar
Hauser, DJ and Schwarz, N (2016) Attentive turkers: mturk participants perform better on online attention checks than do subject pool participants. Behavior Research Methods 48 (1):400–407.10.3758/s13428-015-0578-zCrossRefGoogle ScholarPubMed
Hill, JL (2011) Bayesian nonparametric modeling for causal inference. Journal of Computational and Graphical Statistics 20 (1):217–240.10.1198/jcgs.2010.08162CrossRefGoogle Scholar
Holland, PW (1986) Statistics and causal inference. Journal of the American Statistical Association 81 (396):945–960.10.1080/01621459.1986.10478354CrossRefGoogle Scholar
Humphreys, M, Sanchez de la Sierra, R and van der Windt, P (2013) Fishing, commitment, and communication: a proposal for comprehensive nonbinding research registration. Political Analysis 21 (1):1–20.10.1093/pan/mps021CrossRefGoogle Scholar
Kahan, D (2013) Fooled Twice, Shame on Who? Problems with Mechanical Turk Study Samples, Part 2. Available at http://www.culturalcognition.net/blog/2013/7/10/fooled-twice-shame-on-who-problems-with-mechanical-turk-stud.html, accessed 1 January 2016.Google Scholar
Kahan, DM (2012) Ideology, motivated reasoning, and cognitive reflection: an experimental study. Judgment and Decision Making 8, 407–424.Google Scholar
Kerr, NL (1998) Harking: hypothesizing after the results are known. Personality and Social Psychology Review 2 (3):196–217.10.1207/s15327957pspr0203_4CrossRefGoogle ScholarPubMed
Kuhn, D and Lao, J (1996) Effects of evidence on attitudes: is polarization the norm. Psychological Science 7 (2):115–120.10.1111/j.1467-9280.1996.tb00340.xCrossRefGoogle Scholar
Kuklinski, JH et al. (2001) The political environment and citizen competence. American Journal of Political Science 45 (2):410–424.10.2307/2669349CrossRefGoogle Scholar
Kunda, Z (1990) The case for motivated reasoning. Psychological Bulletin 108, 480–498.10.1037/0033-2909.108.3.480CrossRefGoogle ScholarPubMed
Lau, R and Redlawsk, D (2006) How Voters Decide: Information Processing During Election Campaigns. Cambridge: Cambridge University Press.10.1017/CBO9780511791048CrossRefGoogle Scholar
Lazarsfeld, PF, Berelson, B and Gaudet, H (1944) The People’s Choice: How the Voter Makes Up His Mind in a Presidential Campaign. New York: Columbia University Press.Google Scholar
Leeper, TJ (2017) MTurkR: Access to Amazon Mechanical Turk Requester API via R. R package version 0.8.0.10.32614/RJ-2016-020CrossRefGoogle Scholar
Leeper, TJ and Slothuus, R (2014) Political parties, motivated reasoning, and public opinion formation. Advances in Political Psychology 35, 129–156.10.1111/pops.12164CrossRefGoogle Scholar
Leeper, TJ and Thorson, E (2015) Minimal sponsorship-induced bias in web survey data. Paper presented at the 2015 annual meeting of the Midwest Political Science Association, Chicago, IL.Google Scholar
Levendusky, MS (2013) Why do partisan media polarize viewers? American Journal of Political Science 57 (3):611–623.10.1111/ajps.12008CrossRefGoogle Scholar
Lodge, M and Taber, CS (2013) The Rationalizing Voter. Cambridge: Cambridge University Press.10.1017/CBO9781139032490CrossRefGoogle Scholar
Lord, CS, Ross, L and Lepper, M (1979) Biased assimilation and attitude polarization: the effects of prior theories on subsequently considered evidence. Journal of Personality and Social Psychology 37, 2098–2109.10.1037/0022-3514.37.11.2098CrossRefGoogle Scholar
Miller, AG et al. (1993) The attitude polarization phenomenon: role of response measure, attitude extremity, and behavioral consequences of reported attitude change. Journal of Personality and Social Psychology 64 (4):561–574.10.1037/0022-3514.64.4.561CrossRefGoogle Scholar
Moore, RT (2015) blockTools: blocking, assignment, and diagnosing interference in randomized experiments. R package version 0.6-2.Google Scholar
Mullinix, KJ et al. (2015) The generalizability of survey experiments. Journal of Experimental Political Science 2, 109–138.10.1017/XPS.2015.19CrossRefGoogle Scholar
Mummolo, J and Peterson, E (2017) Demand effects in survey experiments: an empirical assessment. doi: 10.2139/ssrn.2956147.CrossRefGoogle Scholar
Nyhan, B and Reifler, J (2010) When corrections fail: the persistence of political misperceptions. Political Behavior 32 (2):303–330.10.1007/s11109-010-9112-2CrossRefGoogle Scholar
Nyhan, B and Reifler, J (2015) Does correcting myths about the flu vaccine work? An experimental evaluation of the effects of corrective information. Vaccine 33 (3):459–464.10.1016/j.vaccine.2014.11.017CrossRefGoogle Scholar
Nyhan, B et al. (2014) Effective messages in vaccine promotion: A randomized trial. Pediatrics 133 (4):835–842.10.1542/peds.2013-2365CrossRefGoogle ScholarPubMed
Prior, M (2007) ) Post-Broadcast Democracy: How Media Choice Increases Inequality in Political Involvement and Polarizes Elections. Cambridge: Cambridge University Press.10.1017/CBO9781139878425CrossRefGoogle Scholar
Prior, M et al. (2015) You cannot be serious: the impact of accuracy incentives on partisan bias in reports of economic perceptions. Quarterly Journal of Political Science 10 (4):489–518.10.1561/100.00014127CrossRefGoogle Scholar
Redlawsk, DP (2002) Hot cognition or cool consideration? Testing the effects of motivated reasoning on political decision making. The Journal of Politics 64 (4):1021–1044.10.1111/1468-2508.00161CrossRefGoogle Scholar
Redlawsk, DP, Civettini, AJW and Emmerson, KM (2010) The affective tipping point: do motivated reasoners ever “Get It”? Political Psychology 31 (4):563–593.10.1111/j.1467-9221.2010.00772.xCrossRefGoogle Scholar
Sears, DO (1986) College sophomores in the laboratory: influences of a narrow data base on social psychology’s view of human nature. Journal of Personality and Social Psychology 51 (3):515–530.10.1037/0022-3514.51.3.515CrossRefGoogle Scholar
Strickland, AA, Taber, CS and Lodge, M (2011) Motivated reasoning and public opinion. Journal of Health Politics, Policy and Law 36 (6):935–944.10.1215/03616878-1460524CrossRefGoogle ScholarPubMed
Taber, CS, Cann, D and Kucsova, S (2009) The motivated processing of political arguments. Political Behavior 31 (2):137–155.10.1007/s11109-008-9075-8CrossRefGoogle Scholar
Taber, CS and Lodge, M (2006) Motivated skepticism in the evaluation of political beliefs. American Journal of Political Science 50 (3):755–769.10.1111/j.1540-5907.2006.00214.xCrossRefGoogle Scholar
Tversky, A and Kahneman, D (1974) Judgment under uncertainty: heuristics and biases. Science 185 (4157):1124–1131.10.1126/science.185.4157.1124CrossRefGoogle ScholarPubMed
White, A et al. (2016) Investigator characteristics and respondent behavior in online surveys. Journal of Experimental Political Science 5 (10):56–67.10.1017/XPS.2017.25CrossRefGoogle Scholar
Wood, T and Porter, E (2018) The elusive backfire effect: mass attitudes’ steadfast factual adherence. Political Behavior. Forthcoming.10.1007/s11109-018-9443-yCrossRefGoogle Scholar
Zaller, JR (1992) The Nature and Origins of Mass Opinion. New York: Cambridge University Press.10.1017/CBO9780511818691CrossRefGoogle Scholar
Zhou, J (2016) Boomerangs versus javelins: how polarization constrains communication on climate change. Environmental Politics 25 (5):788–811.10.1080/09644016.2016.1166602CrossRefGoogle Scholar
Supplementary material: Link

Guess and Coppock Dataset

Dataset

https://doi.org/10.7910/DVN/J7WNTM
Link
Supplementary material: PDF

Guess and Coppock supplementary material

Online Appendix

Download Guess and Coppock supplementary material(PDF)
PDF 2.8 MB