Hostname: page-component-7b9c58cd5d-g9frx Total loading time: 0 Render date: 2025-03-15T18:13:04.478Z Has data issue: false hasContentIssue false
  • English
  • Français

News and Geolocated Social Media Accurately Measure Protest Size Variation

Published online by Cambridge University Press:  30 June 2020

ANTON SOBOLEV ,
M. KEITH CHEN Open the ORCID record for M. KEITH CHEN [Opens in a new window] ,
JUNGSEOCK JOO Open the ORCID record for JUNGSEOCK JOO [Opens in a new window]  and
ZACHARY C. STEINERT-THRELKELD Open the ORCID record for ZACHARY C. STEINERT-THRELKELD [Opens in a new window]
ANTON SOBOLEV*
Affiliation:
Yale University
M. KEITH CHEN*
Affiliation:
University of California, Los Angeles
JUNGSEOCK JOO*
Affiliation:
University of California, Los Angeles
ZACHARY C. STEINERT-THRELKELD*
Affiliation:
University of California, Los Angeles
*
Anton Sobolev, Postdoctoral Associate, Leitner Program on Political Economy, Yale University, [email protected].
M. Keith Chen, Professor, Anderson School of Management, University of California, Los Angeles, [email protected].
Jungseock Joo, Assistant Professor, Department of Communication, University of California, Los Angeles, [email protected].
Zachary C. Steinert-Threlkeld, Assistant Professor, Luskin School of Public Affairs, University of California, Los Angeles, [email protected].
Get access Rights & Permissions [Opens in a new window]

Abstract

Larger protests are more likely to lead to policy changes than small ones are, but whether or not attendance estimates provided in news or generated from social media are biased is an open question. This letter closes the question: news and geolocated social media data generate accurate estimates of protest size variation. This claim is substantiated using cellphone location data from more than 10 million individuals during the 2017 United States Women’s March protests. These cellphone estimates correlate strongly with those provided in news media as well as three size estimates generated using geolocated tweets, one text-based and two based on images. Inferences about protest attendance from these estimates match others’ findings about the Women’s March.

Type
Letter
Information
American Political Science Review , Volume 114 , Issue 4 , November 2020 , pp. 1343 - 1351
Copyright
© The Author(s), 2020. Published by Cambridge University Press on behalf of the American Political Science Association.

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

We thank Pablo Barberá, Brett Carter, Benjamin A.T. Graham, Will Hobbs, Nazita Lajevardi, Natalia Lamberova, John McCarthy, Clark McPhail, Jeremy Pressman, Michael Shalev, and Thomas Zeitzoff for their valuable feedback. The editor and anonymous reviewers also provided constructive criticism. Ryan Fox Squire and Aaron Hoffman assisted with the SafeGraph data. Alexander Chan and Christina Indudhara assisted with detecting duplicate faces. This work is supported by NSF RIDIR #1831848, the UCLA Big Data Transdisciplinary Seed Grant, and a UCLA California Center for Population Research Seed Grant. All remaining errors remain inadvertently. Replication files are available at the American Political Science Review Dataverse: https://doi.org/10.7910/DVN/TRLSJA.

References

Amenta, Edwin, Caren, Neal, Chiarello, Elizabeth, and Su, Yang. 2010. “The Political Consequences of Social Movements.” Annual Review of Sociology 36: 287–307.CrossRefGoogle Scholar
Andrews, Kenneth T., Caren, Neal, and Browne, Alyssa. 2018. “Protesting Trump.” Mobilization: An International Quarterly 23 (4): 393–400.CrossRefGoogle Scholar
Barbera, Pablo. 2013. “streamR.” URL: https://cran.r-project.org/web/packages/streamR/.Google Scholar
Barranco, Jose, and Wisler, Dominique. 1999. “Validity Event and Analysis Systematicity of Newspaper Data in Event Analysis.” European Sociological Review 15 (3): 301–322.CrossRefGoogle Scholar
Barry, Ann Marie Seward. 1997. Visual Intelligence: Perception, Image, and Manipulation in Visual Communication. SUNY Press.Google Scholar
Baum, Matthew A., and Zhukov, Yuri M.. 2015. “Filtering Revolution: Reporting Bias in International Newspaper Coverage of the Libyan Civil War.” Journal of Peace Research 52 (3): 384–400.Google Scholar
Beyerlein, Kraig, Barwis, Peter, Crubaugh, Bryant, and Carnesecca, Cole. 2018. “A New Picture of Protest: The National Study of Protest Events.” Sociological Methods & Research 47 (3): 384–429.Google Scholar
Beyerlein, Kraig, Ryan, Peter, Abu-Hazeemand, Aliyah, Pauley, Amity. 2018. “The 2017 Women’s March: A National Study of Solidarity Events.” Mobilization: An International Quarterly 23 (4): 425–449.CrossRefGoogle Scholar
Biggs, Michael. 2016. “Size Matters: Quantifying Protest by Counting Participants.” Sociological Methods & Research 47 (3): 351–383.CrossRefGoogle Scholar
Botta, Federico, Moat, Helen Susannah, and Preis, Tobias. 2015. “Quantifying Crowd Size with Mobile Phone and Twitter data.” Royal Society Open Science 2 (5):150162.CrossRefGoogle ScholarPubMed
Chen, M. Keith, and Rohla, Ryne. 2018. “The Effect of Partisanship and Political Advertising on Close Family Ties.” Science 360 (6392):1020–1024.CrossRefGoogle ScholarPubMed
Chenoweth, Erica, and Pressman, Jeremey. 2017. “Crowd Counting Consortium.” URL: https://sites.google.com/view/crowdcountingconsortium/home.Google Scholar
Choi-Fitzpatrick, Austin, Juskauskas, Tautvydas, and Sabur, Boby. 2018. “All the Protestors Fit to Count: Using Geospatial Affordances to Estimate Protest Event Size.” Interface: A Journal for and about Social Movements 10 (1): 297–321.Google Scholar
Clark, David H., and Regan, Patrick M.. 2018. “Mass Mobilization Protest Data.” The Mass Mobilization Project. URL: https://www.binghamton.edu/massmobilization/about.html.Google Scholar
Cox, Joseph. 2019. “I Gave a Bounty Hunter $300. Then He Located Our Phone.” Vice Media Group. URL: https://motherboard.vice.com/en_us/article/nepxbz/i-gave-a-bounty-hunter-300-dollars-located-phone-microbilt-zumigo-tmobile.Google Scholar
Davidson, Thomas, and Berezin, Mabel. 2018. “Britain First and the UK Independence Party: Social Media and Movement-Party Dynamics.” Mobilization: An International Quarterly 23 (4): 485–510.CrossRefGoogle Scholar
de Montjoye, Yves-Alexandre, Gambs, Sébastien, Blondel, Vincent, Canright, Geoffrey, De Cordes, Nicolas, Deletaille, Sébastien, Engø-Monsen, Kenth, et al. 2018. “On the Privacy-Conscientious Use of Mobile Phone Data.” Scientific Data 5: 180286. URL: http://dx.doi.org/10.1038/sdata.2018.286.CrossRefGoogle ScholarPubMed
Earl, Jennifer, Martin, Andrew, McCarthy, John D., and Soule, Sarah A.. 2004. “The Use of Newspaper Data in the Study of Collective Action.” Annual Review of Sociology 30: 65–80.CrossRefGoogle Scholar
Fassiotto, Magali, and Soule, Sarah A.. 2017. “Loud and Clear: the Effect of Protest Signals on Congressional Attention.” Mobilization: An International Quarterly 22 (1): 17–38.Google Scholar
Fisher, Dana R. , Jasny, Lorien, and Dow, Dawn M.. 2018. “Why Are We Here? Patterns of Intersectional Motivations across the Resistance.” Mobilization: An International Quarterly 23 (4): 451–468.CrossRefGoogle Scholar
Gerner, Deborah J., and Schrodt, Philip A.. 1998. “The Effects of Media Coverage on Crisis Assessment and Early Warning in the Middle East.” In Early Warning and Early Response, eds. Schmeidl, Susanne and Adelman, Howard. New York: Columbia University Press.Google Scholar
Graber, Doris A. 1996. “Say It with Pictures.” The ANNALS of the American Academy of Political and Social Science 546 (1): 85–96.CrossRefGoogle Scholar
Hawelka, Bartosz, Sitko, Izabela, Beinat, Euro, Sobolevsky, Stanislav, Kazakopoulos, Pavlos, and Ratti, Carlo. 2014. “Geo-located Twitter as Proxy for Global Mobility Patterns.” Cartography and Geographic Information Science 41 (3): 260–271.Google ScholarPubMed
Herkenrath, Mark, and Knoll, Alex. 2011. “Protest Events in International Press Coverage: An Empirical Critique of Cross-national Conflict Databases.” International Journal of Comparative Sociology 52 (3): 163–180.CrossRefGoogle Scholar
Hlavac, Marek (2018). “stargazer: Well-formatted Regression and Summary Statistics Tables. R package version 5.2.2. Available at https://CRAN.R-project.org/package=stargazer.Google Scholar
Joo, Jungseock, and Steinert-Threlkeld, Zachary C.. 2019. “Image as Data: Automated Visual Content Analysis for Political Science.” Working paper. URL: https://arxiv.org/abs/1810.01544.Google Scholar
Lohmann, Susanne. 1994. “The Dynamics of Informational Cascades: The Monday Demonstrations in Leipzig, East Germany, 1989-91.” World Politics 47 (1): 42–101.CrossRefGoogle Scholar
Mann, Leon. 1974. “Counting the Crowd: Effects of Editorial Policy on Estimates.” Journalism Quarterly 51 (2): 278–285.CrossRefGoogle Scholar
McAdam, Doug, and Su, Yang. 2002. “The War at Home: Antiwar Protests and Congressional Voting, 1965 to 1973.” American Sociological Review 67 (5): 696–721.CrossRefGoogle Scholar
McCarthy, John D., McPhail, Clark, and Smith, Jackie. 1996. “Images of Protest: Dimensions of Selection Bias in Media Coverage of Washington Demonstrations.” American Sociological Review 61 (3): 478–499.CrossRefGoogle Scholar
McCarthy, John D., McPhail, Clark, Smith, Jackie, and Crishock, Louis J.. 1999. “Electronic and Print Media Representations of Washington, D.C. Demonstrations, 1982 and 1991: A Demography of Description Bias.” In Acts of Dissent: New Developments in the Study of Protest, eds. Rucht, Dieter, Koopmans, Ruud, and Neidhardt, Friedhelm. Lanham, MD: Rowman & Littlefield Publishers, 113–130.Google Scholar
McKane, Rachel G., and McCammon, Holly J.. 2018. “Why We March: The Role of Grievances, Threats, and Movement Organizational Resources in the 2017 Women’s Marches.” Mobilization: An International Quarterly 23 (4): 401–424.CrossRefGoogle Scholar
McPhail, Clark, and McCarthy, John. 2004. “Who Counts and How: Estimating the Size of Protests.” Contexts 3 (3): 12–18.CrossRefGoogle Scholar
Myers, Daniel J., and Caniglia, Beth Schaefer. 2004. “All the Rioting That’s Fit to Print: Selection Effects in National Newspaper Coverage of Civil Disorders, 1968-1969.” American Sociological Review 69 (4): 519–543.CrossRefGoogle Scholar
Nam, Taehyun. 2006. “What You Use Matters: Coding Protest Data.” PS: Political Science & Politics 39 (2): 281–287.Google Scholar
Opp, Karl-Dieter, and Gern, Christiane. 1993. “Dissident Groups, Personal Networks, and Spontaneous Cooperation: The East German Revolution of 1989.” American Sociological Review 58 (5): 659–680.CrossRefGoogle Scholar
Pfeffer, Jürgen, and Mayer, Katja. 2018. “Tampering with Twitter’s Sample API.” EPJ Data Science 7 (50): 1–21.CrossRefGoogle Scholar
Raleigh, Clionadh, Linke, Andrew, Hegre, Havard, and Karlsen, Joakim. 2010. “Introducing ACLED: An Armed Conflict Location and Event Dataset: Special Data Feature.” Journal of Peace Research 47 (5): 651–660.CrossRefGoogle Scholar
Salehyan, Idean, Hendrix, Cullen, Hammer, Jesse, Case, Christina, Linebarger, Christopher, Stull, Emily, and Williams, Jennifer. 2012. “Social Conflict in Africa: A New Database.” International Interactions 38 (4): 503–511.CrossRefGoogle Scholar
Schweingruber, David, and McPhail, Clark. 1999. “A Method for Systematically Observing and Recording Collective Action.” Sociological Methods & Research 27 (4): 451–498.CrossRefGoogle Scholar
Shalev, Michael, and Rotman, Assaf. 2020. “Using Location Data from Mobile Phones to Study Participation in Mass Protests.” Sociological Methods and Research. Published online 3 April 2020. https://doi.org/10.1177/0049124120914926.Google Scholar
Sobolev, Anton, Chen, M. Keith, Joo, Jungseock, and Steinert-Threlkeld, Zachary C.. 2020. “Replication Data for: News and Geolocated Social Media Accurately Measure Protest Size Variation.” Harvard Dataverse, V1, https://doi.org/10.7910/DVN/TRLSJA.CrossRefGoogle Scholar
Steinert-Threlkeld, Zachary C. 2018. Twitter as Data. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
Steinert-Threlkeld, Zachary C., Chan, Alexandar, and Joo, Jungseock. 2020. “How Violence Affects Protests.” Working paper. URL: https://preprints.apsanet.org/engage/apsa/article-details/5e1f9019cc45bb0019b9d894.Google Scholar
Stephan, Maria J., and Chenoweth, Erica. 2008. “Why Civil Resistance Works.” International Security 33 (1): 7–44.Google Scholar
Tarrow, Sidney, and Meyer, David S., eds. 2018. The Resistance: The Dawn of the Anti-Trump Opposition Movement. Oxford: Oxford University Press.Google Scholar
Tilly, Charles, and Tarrow, Sidney. 2015. Contentious Politics. Second edition.Oxford: Oxford University Press.Google Scholar
Traag, Vincent A., Quax, Rick, and Sloot, Peter M. A.. 2017. “Modelling the Distance Impedance of Protest Attendance.” Physica A: Statistical Mechanics and Its Applications 468: 171–182.CrossRefGoogle Scholar
Uba, Katrin. 2005. “Political Protest and Policy Change: The Direct Impacts of Indian Anti-privatization Mobilizations, 1990-2003.” Mobilization: An International Journal 10 (3): 1990–2003.Google Scholar
Valentino-DeVries, Jennifer, Singer, Natasha, Keller, Michael H., and Krolik, Aaron. 2018. “Your Apps Know Where You Were Last Night, and They’re Not Keeping It Secret.”. URL: https://www.nytimes.com/interactive/2018/12/10/business/location-data-privacy-apps.html?smid=tw-share.Google Scholar
Walgrave, Stefaan, and Verhulst, Joris. 2011. “Selection and Response Bias in Protest Surveys.” Mobilization: An International Journal 16 (2): 203–222.CrossRefGoogle Scholar
Walgrave, Stefaan, and Vliegenthart, Rens. 2012. “The Complex Agenda-setting Power of Protest: Demonstrations, Media, Parliament, Government, and Legislations in Belgium, 1993-2000.” Mobilization: An International Journal 17 (2): 129–156.CrossRefGoogle Scholar
Walgrave, Stefaan, and Wouters, Ruud. 2014. “The Missing Link in the Diffusion of Protest: Asking Others.” American Journal of Sociology 119 (6): 1670–1709.CrossRefGoogle ScholarPubMed
Walgrave, Stefaan, Wouters, Ruud, and Ketelaars, Pauline. 2016. “Response Problems in the Protest Survey Design: Evidence from Fifty-one Protest Events in Seven Countries.” Mobilization: An International Journal 21 (1): 83–104.CrossRefGoogle Scholar
Weidmann, Nils B., and Rød, Espen Geelmuyden. 2018. “Coding Protest Events in Autocracies.” In The Internet and Political Protest in Autocracies. Oxford: Oxford University Press.Google Scholar
Woolley, John T. 2000. “Using Media-Based Data in Studies of Politics.” American Journal of Political Science 44 (1): 156–173.CrossRefGoogle Scholar
Wouters, Ruud, and Camp, Kirsten Van. 2017. “Less than Expected? How Media Cover Demonstration Turnout.” The International Journal of Press/Politics 22 (4): 450–470.Google ScholarPubMed
Wouters, Ruud, and Walgrave, Stefaan. 2017. “Demonstrating Power: How Protest Persuades Political Representatives.” American Sociological Review 82 (2): 361–383.CrossRefGoogle Scholar
Yip, Paul S. F., Watson, Ray, Chan, K.S., Lau, Eric H. Y., Chen, Feng, Xu, Ying, Xi, Liqun, Cheung, Derek Y. T., Ip, Brian Y. T., and Liu, Danping. 2010. “Estimation of the Number of People in a Demonstration.” Australian & New Zealand Journal of Statistics 52 (1): 17–26.CrossRefGoogle Scholar
Supplementary material: PDF

Sobolev et al. supplementary material

Sobolev et al. supplementary material

Download Sobolev et al. supplementary material(PDF)
PDF 9.2 MB
Supplementary material: Link

Sobolev et al. Dataset

Dataset

https://doi.org/10.7910/DVN/TRLSJA
Link
Submit a response

Comments

No Comments have been published for this article.