Hostname: page-component-586b7cd67f-r5fsc Total loading time: 0 Render date: 2024-11-28T15:11:13.226Z Has data issue: false hasContentIssue false
  • English
  • Français

Getting the Message? Choice, Self-Selection, and the Efficacy of Social Movement Arguments

Published online by Cambridge University Press:  11 September 2020

Paul F. Testa Open the ORCID record for Paul F. Testa [Opens in a new window] ,
Tarah Williams ,
Kylee Britzman  and
Matthew V. Hibbing
Paul F. Testa*
Affiliation:
Department of Political Science, Brown University, 111 Thayer St, Providence, RI02912, USA, Twitter: @ProfPaulTesta
Tarah Williams
Affiliation:
Department of Political Science, Allegheny College, Allegheny, PA, USA
Kylee Britzman
Affiliation:
Social Sciences Program, Lewis-Clark State College, Lewiston, ID, USA, Twitter: @kyleebritzman
Matthew V. Hibbing
Affiliation:
Department of Political Science, UC Merced, Merced, CA, USA, Twitter: @matthibbing
*
*Corresponding author. Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

The dynamics of choice and self-selection are central features of politics but absent from most experimental designs. We show how designs that incorporate choice, by allowing some subjects the option to receive or avoid treatment, can be extended by randomizing conditional on subjects’ treatment choices to answer further questions of interest while preserving statistical power. We apply this design to study how the gender of messengers for the #MeToo social movement conditions who receives the movement’s message and how they respond. Our results, from both convenience and nationally representative samples, suggest that #MeToo movement’s message reaches a wide audience with the intended effect. The potential for backlash in response to the message appears limited but more likely when this message is delivered by a woman.

Keywords

self-selectionpreference-incorporating choice and assignment designeffect heterogeneitysource cuessocial movementspolitical communication
Type
Research Article
Information
Journal of Experimental Political Science , Volume 8 , Issue 3 , Winter 2021 , pp. 296 - 309
Copyright
© The Author(s) 2020. Published by Cambridge University Press on behalf of The Experimental Research Section of the American Political Science Association

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Support for this research was provided by Brown University and the Carrie Chapman Catt Prize for Research on Women and Politics from Iowa State University. The authors report no conflict of interests associated with this project. No outside funding was used to conduct the study, and the authors hold no additional positions outside their academic posts. The authors thank the editor, Kaye Usry, Andy Bloeser, and three anonymous reviewers for helpful feedback. The data, code, and any additional materials required to replicate all analyses in this article are available at the Journal of Experimental Political Science Dataverse within the Harvard Dataverse Network, at: https://doi.org/10.7910/DVN/CWEHBA under Testa et al. (2020)

References

Arceneaux, Kevin, Johnson, Martin and Murphy, Chad 2012. Polarized Political Communication, Oppositional Media Hostility, and Selective Exposure. The Journal of Politics 74(01): 174–86. http://www.journals.uchicago.edu/doi/pdfplus/10.1017/S002238161100123X CrossRefGoogle Scholar
Arceneaux, Kevin and Kolodny, Robin 2009. Educating the Least Informed: Group Endorsements in a Grassroots Campaign. American Journal of Political Science 53(4): 755–70. http://doi.wiley.com/10.1111/j.1540–5907.2009.00399.x CrossRefGoogle Scholar
Atkeson, Lonna Rae and Rapoport, Ronald B 2003. The More Things Change the More They Stay the Same: Examining Gender Differences in Political Attitude Expression, 1952–2000. Public Opinion Quarterly 67(4):495–521.CrossRefGoogle Scholar
Barnes, Tiffany D. and Cassese, Erin C. 2017. American Party Women: A Look at the Gender Gap within Parties. Political Research Quarterly 70(1): 127–41. http://journals.sagepub.com/doi/10.1177/1065912916675738 CrossRefGoogle Scholar
Benford, Robert D. and Snow, David A. 2000. Framing Processes and Social Movements: An Overview and Assessment. Annual Review of Sociology 26(1): 611–39.CrossRefGoogle Scholar
Berinsky, Adam J. 2017. Rumors and Health Care Reform: Experiments in Political Misinformation. British Journal of Political Science 47(2): 241–62. http://people-press.org/files/legacy-pdf/537.pdf CrossRefGoogle Scholar
Berinsky, Adam J, Huber, Gregory A and Lenz, Gabriel S 2012. Evaluating Online Labor Markets for Experimental Research: Amazon.com”s Mechanical Turk . Political Analysis 20(3): 351–68.CrossRefGoogle Scholar
Bittner, Amanda and Goodyear-Grant, Elizabeth 2017. Sex isn’t Gender: Reforming Concepts and Measurements in the Study of Public Opinion. Political Behavior 39(4): 1019–41.CrossRefGoogle Scholar
Cameron, A Colin and Trivedi, Pravin K 2005. Microeconometrics: Methods and applications. New York: Cambridge University Press.CrossRefGoogle Scholar
de Benedictis-Kessner, Justin, Baum, Matthew A., Berinsky, Adam J. and Yamamoto, Teppei 2019. Persuading the Enemy: Estimating the Persuasive Effects of Partisan Media with the Preference-Incorporating Choice and Assignment Design. American Political Science Review 113(4): 902–16. https://www.cambridge.org/core/product/identifier/S0003055419000418/type/journal_article CrossRefGoogle Scholar
Deckman, Melissa 2018. “#MeToo and the Midterms - Gender Watch 2018.” http://www.genderwatch2018.org/metoo-and-the-midterms/ Google Scholar
Gaines, Brian J and Kuklinski, James H 2011. Experimental Estimation of Heterogeneous Treatment Effects Related to Self-Selection. American Journal of Political Science 55(3): 724–36. http://www.jstor.org/stable/23024947 CrossRefGoogle Scholar
Goren, P, Federico, C M and Kittilson, M C 2009. Source Cues, Partisan Identities, and Political Value Expression. American Journal of Political Science 53(4): 805–20.CrossRefGoogle Scholar
Hansen, Susan B. 1997. Talking About Politics: Gender and Contextual Effects on Political Proselytizing. The Journal of Politics 59(01): 73–103.CrossRefGoogle Scholar
Huckfeldt, Robert R and Sprague, John 1995. Citizens, Politics and Social Communication: Information and Influence in an Election Campaign. New York: Cambridge University Press.CrossRefGoogle Scholar
Huff, Connor and Tingley, Dustin 2015. “Who are these people?” Evaluating the demographic characteristics and political preferences of MTurk survey respondents.” Research & Politics 2(3): 2053168015604648.CrossRefGoogle Scholar
Imbens, Guido W. and Rubin, Donald B. 2015. Causal Inference in Statistics, Social, and Biomedical Sciences. New York: Cambridge University Press.CrossRefGoogle Scholar
Iyengar, S, Hahn, K. S., Krosnick, J. A. and Walker, J. 2008. Selective Exposure to Campaign Communication: The Role of Anticipated Agreement and Issue Public Membership. Journal of Politics 70(1): 186–200.CrossRefGoogle Scholar
Johnston, Hank and Noakes, John A. 2005. Frames of Protest: Social Movements and the Framing Perspective. Rowman & Littlefield Publishers.Google Scholar
Kam, C. D. 2005. Who Toes the Party Line? Cues, Values, and Individual Differences. Political Behavior 27(2): 163–82.CrossRefGoogle Scholar
Karpowitz, Christopher F. and Mendelberg, Tali 2014. The Silent Sex: Gender, Deliberation, and Institutions. Princeton, NJ: Princeton University Press.Google Scholar
Knox, Dean, Yamamoto, Teppei, Baum, Matthew A. and Berinsky, Adam J. 2019. Design, Identification, and Sensitivity Analysis for Patient Preference Trials. Journal of the American Statistical Association. https://www.tandfonline.com/action/journalInformation?journalCode=uasa20 CrossRefGoogle Scholar
Leeper, Thomas J. 2017. How Does Treatment Self-Selection Affect Inferences About Political Communication? Journal of Experimental Political Science 4(1): 21–33. https://www.cambridge.org/core/product/identifier/S205226301700001X/type/journal_article CrossRefGoogle Scholar
Long, Qi, Little, Roderick J. and Lin, Xihong 2008. “Causal Inference in Hybrid Intervention Trials Involving Treatment Choice. Journal of the American Statistical Association 103(482):474–84. http://www.tandfonline.com/action/journalInformation?journalCode=uasa20 CrossRefGoogle Scholar
Ma, Debbie S., Joshua, Correll and Wittenbrink, Bernd 2015. The Chicago Face Database: A Free Stimulus Set of Faces and Norming Data. Behavior Research Methods 47(4): 1122–35. http://link.springer.com/10.3758/s13428–014–0532–5 CrossRefGoogle ScholarPubMed
McDermott, Monika L. 1998. Race and Gender Cues in Low-Information Elections. Political Research Quarterly 51(4): 895–918.CrossRefGoogle Scholar
Mendez, Jeanette Morehouse and Osborn, Tracy 2010. Gender and the Perception of Knowledge in Political Discussion. Political Research Quarterly 63(2): 269–79.CrossRefGoogle Scholar
Miratrix, Luke W., Sekhon, Jasjeet S., Theodoridis, Alexander G. and Campos, Luis F. 2018. Worth Weighting? How to Think About and Use Weights in Survey Experiments. Political Analysis 26(3): 275–91. https://www.cambridge.org/core/product/identifier/S1047198718000013/type/journal_article CrossRefGoogle Scholar
Nicholson, Stephen P. 2012. Polarizing Cues. American Journal of Political Science 56(1): 52–66. http://doi.wiley.com/10.1111/j.1540–5907.2011.00541.x CrossRefGoogle ScholarPubMed
Rücker, Gerta 1989. A Two-Stage Trial Design for Testing Treatment, Self-Selection and Treatment Preference Effects. Statistics in Medicine 8(4):477–85. http://doi.wiley.com/10.1002/sim.4780080411 Google Scholar
Stroud, N. J. 2008. “Media Use and Political Predispositions: Revisiting the Concept of Selective Exposure.” Political Behavior 30(3): 341–66.CrossRefGoogle Scholar
Stroud, N. J. 2010. “Polarization and Partisan Selective Exposure.” Journal of Communication 60(3): 556–76.Google Scholar
Testa, Paul, Williams, Tarah, Britzman, Kylee and Hibbing, Matthew 2020. Replication Data for: “Getting the Message? Choice, Self-Selection, and the Efficacy of Social Movement Arguments, Harvard Dataverse, V1. https://doi.org/10.7910/DVN/CWEHBA CrossRefGoogle Scholar
Torgerson, David J., Klaber-Moffett, Jennifer and Russell, Ian T. 1996. Patient Preferences in Randomised Trials: Threat or Opportunity? Journal of Health Services Research & Policy 1(4):194–7. http://journals.sagepub.com/doi/10.1177/135581969600100403 CrossRefGoogle ScholarPubMed
Winter, Nicholas J. G. 2010. Masculine Republicans and Feminine Democrats: Gender and Americans’ Explicit and Implicit Images of the Political Parties. Political Behavior 32(4): 587–618. https://link.springer.com/content/pdf/10.1007%2Fs11109–010–9131-z.pdf CrossRefGoogle Scholar
Supplementary material: Link

Testa et al. Dataset

Dataset

https://doi.org/10.7910/DVN/CWEHBA
Link
Supplementary material: PDF

Testa et al. supplementary material

Testa et al. supplementary material

Download Testa et al. supplementary material(PDF)
PDF 808.3 KB