Hostname: page-component-669899f699-swprf Total loading time: 0 Render date: 2025-04-30T07:37:31.109Z Has data issue: false hasContentIssue false
  • English
  • Français

Breaking Bad? How Survey Experiments Prime Americans for War Crimes

Published online by Cambridge University Press:  03 November 2020

Charli Carpenter Open the ORCID record for Charli Carpenter [Opens in a new window] ,
Alexander H. Montgomery Open the ORCID record for Alexander H. Montgomery [Opens in a new window]  and
Alexandria Nylen Open the ORCID record for Alexandria Nylen [Opens in a new window]
Get access Rights & Permissions [Opens in a new window]

Abstract

What affects Americans’ sensitivity to international laws and norms on the use of force? A wealth of recent IR literature tackles this question through experimental surveys using fictional scenarios and treatments to explore precisely when Americans would approve of government policies that would violate the laws of war. We test whether such survey experiments may themselves be affecting public sensitivity to these norms—or even Americans’ understanding of the content of the norms themselves. We show that being invited to express a preference regarding war crimes in survey settings has a negative impact on Americans’ understanding of US legal and ethical obligations in war and that reporting previous findings can inflate support for war crimes. We conclude with suggestions for future experimental survey design in international relations and international law.

Type
Reflection
Information
Perspectives on Politics , Volume 19 , Issue 3 , September 2021 , pp. 912 - 924
Copyright
© The Author(s), 2020. Published by Cambridge University Press on behalf of the American Political Science Association

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

Footnotes

*

Data replication sets are available in Harvard Dataverse at: https://doi.org/10.7910/DVN/B6QSZ1

We are grateful to the participants in University of Massachusetts-Amherst’s Approaches and Methods workshop in February 2020 and at the ISA International Security Studies Section Conference panel on Nuclear Norms in October 2019, to a number of anonymous reviewers, and to generous banks of written comments by Jeff Kaplow and Kevin Young. All flaws and errors are our own.

References

Ackerly, Brooke A., Stern, Maria, and True, Jacqui. 2006. Feminist Methodologies for International Relations. Cambridge: Cambridge University Press.10.1017/CBO9780511617690CrossRefGoogle Scholar
Ackerly, Brooke A., and True, Jacqui. 2008. “Reflexivity in Practice: Power and Ethics in Feminist Research on International Relations.” International Studies Review 10 (4): 693–707.10.1111/j.1468-2486.2008.00826.xCrossRefGoogle Scholar
Barabas, Jason, and Jerit, Jennifer. 2010. “Are Survey Experiments Externally Valid?” American Political Science Review 104 (2): 226–42.10.1017/S0003055410000092CrossRefGoogle Scholar
Búzás, Zoltán I. 2017. “Evading International Law: How Agents Comply with the Letter of the Law but Violate Its Purpose.” European Journal of International Relations 23 (4): 857–83.10.1177/1354066116679242CrossRefGoogle Scholar
Carpenter, Charli, and Montgomery, Alexander H.. 2019. “Americans Want Their Leaders to Obey the Laws of War.” Foreign Policy, June 27. https://foreignpolicy.com/2019/06/27/americans-want-their-leaders-to-obey-the-laws-of-war./Google Scholar
Carpenter, Charli, and Montgomery, Alexander H.. 2020. “The Stopping Power of Norms: Saturation Bombing, Civilian Immunity, and U.S. Attitudes toward the Norms and Laws of War.” International Security (forthcoming).10.1162/isec_a_00392CrossRefGoogle Scholar
Chilton, Adam S. 2015. “The Laws of War and Public Opinion: An Experimental Study.” Journal of Institutional and Theoretical Economics JITE 171 (1): 181–201.10.1628/093245615X14188909230370CrossRefGoogle Scholar
Chilton, Adam S., and Posner, Eric A.. 2015. “An Empirical Study of Political Bias in Legal Scholarship.” Journal of Legal Studies 44 (2): 277–314.10.1086/684302CrossRefGoogle Scholar
Chilton, Adam S., and Tingley, Dustin. 2013. “Why the Study of International Law Needs Experiments.” Columbia Journal of Transnational Law 52: 173.Google Scholar
Chilton, Adam S., and Versteeg, Mila. 2016. “International Law, Constitutional Law, and Public Support for Torture.” Research & Politics 3 (1).10.1177/2053168016636413CrossRefGoogle Scholar
Conrad, Courtenay R., Croco, Sarah E., Gomez, Brad T., and Moore, Will H.. 2015. “Threat Perception and Public Support of Government Torture.” Working Paper. University of California, Merced.Google Scholar
Cox, Robert W. 1981. “Social Forces, States and World Orders: Beyond International Relations Theory.” Millennium: Journal of International Studies 10 (2): 126–55.10.1177/03058298810100020501CrossRefGoogle Scholar
Daniel, J. Furman III, and Musgrave, Paul. 2017. “Synthetic Experiences: How Popular Culture Matters for Images of International Relations.” International Studies Quarterly 61 (3): 503–16.10.1093/isq/sqx053CrossRefGoogle Scholar
Deitelhoff, Nicole, and Zimmermann, Lisbeth. 2019. “Norms under Challenge: Unpacking the Dynamics of Norm Robustness.” Journal of Global Security Studies 4 (1): 2–17.10.1093/jogss/ogy041CrossRefGoogle Scholar
Druckman, James N. 2001. “The Implications of Framing Effects for Citizen Competence.” Political Behavior 23 (3): 225–56.10.1023/A:1015006907312CrossRefGoogle Scholar
Eagleton-Pierce, Matthew. 2011. “Advancing a Reflexive International Relations.” Millennium: Journal of International Studies 39 (3): 805–23.10.1177/0305829811402709CrossRefGoogle Scholar
Eichenberg, Richard C. 2005. “Victory Has Many Friends: US Public Opinion and the Use of Military Force, 1981–2005.” International Security 30 (1): 140–177.10.1162/0162288054894616CrossRefGoogle Scholar
Fujii, Lee Ann. 2012. “Research Ethics 101: Dilemmas and Responsibilities.” PS: Political Science & Politics 45 (4): 717–23.Google Scholar
Gaines, Brian J., Kuklinski, James H., and Quirk, Paul J.. 2007. “The Logic of the Survey Experiment Reexamined.” Political Analysis 15 (1): 1–20.CrossRefGoogle Scholar
Galtung, Johan, and Ruge, Mari Holmboe. 1965. “The Structure of Foreign News: The Presentation of the Congo, Cuba and Cyprus Crises in Four Norwegian Newspapers.” Journal of Peace Research 2 (1): 64–90.10.1177/002234336500200104CrossRefGoogle Scholar
Gartner, Scott Sigmund. 2008. “The Multiple Effects of Casualties on Public Support for War: An Experimental Approach.” American Political Science Review 102 (1): 95–106.10.1017/S0003055408080027CrossRefGoogle Scholar
Hamati-Ataya, Inanna. 2010. “Knowing and Judging in International Relations Theory: Realism and the Reflexive Challenge.” Review of International Studies 36 (4): 1079–1101.CrossRefGoogle Scholar
Hamati-Ataya, Inanna. 2013. “Reflectivity, Reflexivity, Reflexivism: IR’s ‘Reflexive Turn’ — and Beyond.” European Journal of International Relations 19 (4): 669–94.10.1177/1354066112437770CrossRefGoogle Scholar
Haraway, Donna. 1988. “Situated Knowledges: The Science Question in Feminism and the Privilege of Partial Perspective.” Feminist Studies 14 (3): 575–99.10.2307/3178066CrossRefGoogle Scholar
Haworth, Alida R., Sagan, Scott D., and Valentino, Benjamin A.. 2019. “What Do Americans Really Think about Conflict with Nuclear North Korea? The Answer Is Both Reassuring and Disturbing.” Bulletin of the Atomic Scientists 75 (4): 179–186.10.1080/00963402.2019.1629576CrossRefGoogle Scholar
Hochschild, Jennifer L. 1981. What’s Fair? American Beliefs about Distributive Justice. Cambridge, MA: Harvard University Press.Google Scholar
Holsti, Ole Rudolf. 2009. Public Opinion and American Foreign Policy.Rev.ed. Ann Arbor: University of Michigan Press.Google Scholar
Horowitz, Michael C. 2016. “Public Opinion and the Politics of the Killer Robots Debate.” Research & Politics 3 (1): 1–8.10.1177/2053168015627183CrossRefGoogle Scholar
Human Rights Watch. 2012. Losing Humanity: The Case against Killer Robots. New York: Human Rights Watch.Google Scholar
Hurd, Ian. 2017. How to Do Things with International Law. Princeton: Princeton University Press.10.2307/j.ctvc77cj7CrossRefGoogle Scholar
Jackson, Patrick Thaddeus. 2011. The Conduct of Inquiry in International Relations. London: Routledge.Google Scholar
Jackson, Patrick Thaddeus. 2015. “Must International Studies Be a Science?” Millennium: Journal of International Studies 43 (3): 942–65.10.1177/0305829815579307CrossRefGoogle Scholar
Jurkovich, Michelle. 2019. “What Isn’t a Norm? Redefining the Conceptual Boundaries of ‘Norms’ in the Human Rights Literature.” International Studies Review, August 20. https://academic.oup.com/isr/advance-article/doi/10.1093/isr/viz040/5550876.Google Scholar
Kahl, Colin H. 2007. “In the Crossfire or the Crosshairs? Norms, Civilian Casualties, and US Conduct in Iraq.” International Security 32 (1): 7–46.CrossRefGoogle Scholar
Keohane, Robert O. 2009. “Political Science as a Vocation.” PS: Political Science and Politics 42 (2): 359–63.Google Scholar
Kertzer, Joshua D., and Zeitzoff, Thomas. 2017. “A Bottom-Up Theory of Public Opinion about Foreign Policy.” American Journal of Political Science 61 (3): 543–58.10.1111/ajps.12314CrossRefGoogle Scholar
Kreps, Sarah. 2014. “Flying under the Radar: A Study of Public Attitudes towards Unmanned Aerial Vehicles.” Research & Politics 1 (1): 1–7.10.1177/2053168014536533CrossRefGoogle Scholar
Kreps, Sarah, and Wallace, Geoffrey P. R.. 2016. “International Law, Military Effectiveness, and Public Support for Drone Strikes.” Journal of Peace Research 53 (6): 830–844.10.1177/0022343316657405CrossRefGoogle Scholar
Kunda, Ziva. 1990. “The Case for Motivated Reasoning.” Psychological Bulletin 108 (3): 480–98.10.1037/0033-2909.108.3.480CrossRefGoogle ScholarPubMed
Lipari, Lisbeth. 2000. “Toward a Discourse Approach to Polling.” Discourse Studies 2 (2): 187–215.10.1177/1461445600002002003CrossRefGoogle Scholar
Luskin, Robert C., Fishkin, James S., and Jowell, Roger. 2002. “Considered Opinions: Deliberative Polling in Britain.” British Journal of Political Science 32 (3): 455–87.10.1017/S0007123402000194CrossRefGoogle Scholar
Lynch, Jake. 2015. “Peace Journalism: Theoretical and Methodological Developments.” Global Media and Communication 11 (3): 193–99.10.1177/1742766515606297CrossRefGoogle Scholar
Lynch, Jake, and McGoldrick, Annabel. 2005. Peace Journalism. Stroud, UK: Hawthorn Press.Google Scholar
Meron, Theodor. 2000. “The Martens Clause, Principles of Humanity, and Dictates of Public Conscience.” American Journal of International Law 94 (1): 78–89.10.2307/2555232CrossRefGoogle Scholar
Mutz, Diana C. 2002. “The Consequences of Cross-Cutting Networks for Political Participation.” American Journal of Political Science 46 (4): 838–55.10.2307/3088437CrossRefGoogle Scholar
Nylen, Alexandria, and Carpenter, Charli. 2019. “Questions of Life and Death.” European Journal of International Security 4 (2): 142–62.10.1017/eis.2019.3CrossRefGoogle Scholar
Pachirat, Timothy. 2017. Among Wolves: Ethnography and the Immersive Study of Power. London: Routledge.Google Scholar
Panke, Diana, and Petersohn, Ulrich. 2012. “Why International Norms Disappear Sometimes.” European Journal of International Relations 18 (4): 719–42.10.1177/1354066111407690CrossRefGoogle Scholar
Parkinson, Sarah E. 2018. “The Dignity of Complexity: Being Human in Political Science.” Qualitative and Multi-Method Research 16 (1): 39–41.Google Scholar
Pauly, Reid B. C. 2018. “Would U.S. Leaders Push the Button? Wargames and the Sources of Nuclear Restraint.” International Security 43 (2): 151–92.10.1162/isec_a_00333CrossRefGoogle Scholar
Perla, Peter P, and McGrady, Ed. 2011. “Why Wargaming Works.” Naval War College Review 64 (3).Google Scholar
Press, Daryl G., Sagan, Scott D., and Valentino, Benjamin A.. 2013. “Atomic Aversion: Experimental Evidence on Taboos, Traditions, and the Non-Use of Nuclear Weapons.” American Political Science Review 107 (1): 188–206.10.1017/S0003055412000597CrossRefGoogle Scholar
Rathbun, Brian C. 2012. “Politics and Paradigm Preferences: The Implicit Ideology of International Relations Scholars.” International Studies Quarterly 56 (3): 607–22.10.1111/j.1468-2478.2012.00749.xCrossRefGoogle Scholar
Rathbun, Brian C., and Stein, Rachel. 2019. “Greater Goods: Morality and Attitudes toward the Use of Nuclear Weapons.” Journal of Conflict Resolution, December 27. https://doi.org/10.1177/0022002719879994.CrossRefGoogle Scholar
Ron, James, Lavine, Howard, and Golden, Shannon. 2019. “No, Americans Don’t Support Airstrikes That Kill Civilians, Even When They Target Terrorists.” Washington Post, June 13. https://www.washingtonpost.com/politics/2019/05/06/no-americans-dont-support-airstrikes-that-kill-civilians-even-when-they-target-terrorists/.Google Scholar
Sagan, Scott D., and Valentino, Benjamin A.. 2017. “Revisiting Hiroshima in Iran: What Americans Really Think about Using Nuclear Weapons and Killing Noncombatants.” International Security 42 (1): 41–79.10.1162/ISEC_a_00284CrossRefGoogle Scholar
Sandholtz, Wayne. 2019. “Norm Contestation, Robustness, and Replacement.” Journal of Global Security Studies 4 (1): 139–46.10.1093/jogss/ogy042CrossRefGoogle Scholar
Schaffer, Frederic Charles. 2014. “Thin Descriptions: The Limits of Survey Research on the Meaning of Democracy.” Polity 46 (3): 303–30.10.1057/pol.2014.14CrossRefGoogle Scholar
Shannon, Vaughn P. 2000. “Norms Are What States Make of Them: The Political Psychology of Norm Violation.” International Studies Quarterly 44 (2): 293–316.10.1111/0020-8833.00159CrossRefGoogle Scholar
Simmons, Beth A. 2009. Mobilizing for Human Rights: International Law in Domestic Politics. Cambridge: Cambridge University Press.10.1017/CBO9780511811340CrossRefGoogle Scholar
Smyth, Marie, and Robinson, Gillian. 2001. Researching Violently Divided Societies: Ethical and Methodological Issues. New York: United Nations University Press.Google Scholar
Tomz, Michael R., and Weeks, Jessica L. P.. 2013. “Public Opinion and the Democratic Peace.” American Political Science Review 107 (4): 849–65.10.1017/S0003055413000488CrossRefGoogle Scholar
Valentino, Benjamin A., and Weinberg, Ethan M.. 2017. “More than Words? ‘Genocide,’ Holocaust Analogies, and Public Opinion in the United States.” Journal of Human Rights 16 (3): 276–92.10.1080/14754835.2016.1239067CrossRefGoogle Scholar
Vitalis, Robert. 2015. White World Order, Black Power Politics: The Birth of American International Relations. Ithaca: Cornell University Press.Google Scholar
Wallace, Geoffrey P. R. 2013. “International Law and Public Attitudes toward Torture: An Experimental Study.” International Organization 67 (1): 105–140.10.1017/S0020818312000343CrossRefGoogle Scholar
Wallace, Geoffrey P. R. 2014. “Martial Law? Military Experience, International Law, and Support for Torture.” International Studies Quarterly 58 (3): 501–14.10.1111/isqu.12092CrossRefGoogle Scholar
Wallace, Geoffrey P. R. 2019. “Condemning or Condoning the Perpetrators? International Humanitarian Law and Attitudes toward Wartime Violence.” Law & Social Inquiry 44 (1): 192–226.10.1017/lsi.2018.22CrossRefGoogle Scholar
Walsh, James Igoe. 2015. “Precision Weapons, Civilian Casualties, and Support for the Use of Force.” Political Psychology 36 (5): 507–523.10.1111/pops.12175CrossRefGoogle Scholar
Wendt, Alexander. 2015. Quantum Mind and Social Science: Unifying Physical and Social Ontology. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
Wiener, Antje. 2018. Contestation and Constitution of Norms in Global International Relations. Cambridge: Cambridge University Press.10.1017/9781316718599CrossRefGoogle Scholar
Yanow, Dvora, and Schwartz-Shea, Peregrine. 2016. “Encountering Your IRB 2.0: What Political Scientists Need to Know.” PS: Political Science & Politics 49 (2): 277–86.Google Scholar
Young, Kevin L. 2020. “Progress, Complexity and Science: Moving from Alienated to Engaged Pluralism.” Working Paper, University of Massachusetts-Amherst.CrossRefGoogle Scholar
Zaller, John, and Feldman, Stanley. 1992. “A Simple Theory of the Survey Response: Answering Questions versus Revealing Preferences.” American Journal of Political Science 36 (3): 579–616.10.2307/2111583CrossRefGoogle Scholar
Supplementary material: Link

Carpenter et al. Dataset

Dataset

https://doi.org/10.7910/DVN/B6QSZ1
Link