Hostname: page-component-586b7cd67f-t7fkt Total loading time: 0 Render date: 2024-11-22T03:17:58.977Z Has data issue: false hasContentIssue false
  • English
  • Français

Quota sampling using Facebook advertisements

Published online by Cambridge University Press:  05 December 2018

Baobao Zhang Open the ORCID record for Baobao Zhang [Opens in a new window] ,
Matto Mildenberger ,
Peter D. Howe ,
Jennifer Marlon ,
Seth A. Rosenthal  and
Anthony Leiserowitz
Baobao Zhang*
Affiliation:
Department of Political Science, Yale University, New Haven, CT, USA
Matto Mildenberger
Affiliation:
Department of Political Science, University of California, Santa Barbara, Santa Barbara, CA, USA
Peter D. Howe
Affiliation:
Department of Environment and Society, Utah State University, Logan, UT, USA
Jennifer Marlon
Affiliation:
School of Forestry and Environmental Studies, Yale University, New Haven, CT, USA
Seth A. Rosenthal
Affiliation:
School of Forestry and Environmental Studies, Yale University, New Haven, CT, USA
Anthony Leiserowitz
Affiliation:
School of Forestry and Environmental Studies, Yale University, New Haven, CT, USA
*
*Corresponding author. Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Researchers in different social science disciplines have successfully used Facebook to recruit subjects for their studies. However, such convenience samples are not generally representative of the population. We developed and validated a new quota sampling method to recruit respondents using Facebook advertisements. Additionally, we published an R package to semi-automate this quota sampling process using the Facebook Marketing API. To test the method, we used Facebook advertisements to quota sample 2432 US respondents for a survey on climate change public opinion. We conducted a contemporaneous nationally representative survey asking identical questions using a high-quality online survey panel whose respondents were recruited using probability sampling. Many results from the Facebook-sampled survey are similar to those from the online panel survey; furthermore, results from the Facebook-sampled survey approximate results from the American Community Survey (ACS) for a set of validation questions. These findings suggest that using Facebook to recruit respondents is a viable option for survey researchers wishing to approximate population-level public opinion.

Type
Research Notes
Information
Political Science Research and Methods , Volume 8 , Issue 3 , July 2020 , pp. 558 - 564
Copyright
Copyright © The European Political Science Association 2018

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Berinsky, AJ, Huber, GAandLenz, GS (2012) Evaluating Online Labor Markets for Experimental Research: Amazon. com’s Mechanical Turk. Political Analysis 20(3), 351–368.CrossRefGoogle Scholar
Bond, RandMessing, S (2015) Quantifying Social Media’s Political Space: Estimating Ideology from Publicly Revealed Preferences on Facebook. American Political Science Review 109(01), 62–78.CrossRefGoogle Scholar
Broockman, DEandGreen, DP (2014) Do Online Advertisements Increase Political Candidates’ Name Recognition or Favorability? Evidence from Randomized Field Experiments. Political Behavior 36(2), 263–289.CrossRefGoogle Scholar
Hirano, S, Lenz, GS, Pinkovskiy, MandSnyder, JM (2015) Voter Learning in State Primary Elections. American Journal of Political Science 59(1), 91–108.CrossRefGoogle Scholar
Huff, CandTingley, D (2015) “Who Are These People?” Evaluating the Demographic Characteristics and Political Preferences of Mturk Survey Respondents. Research & Politics 2(3), 2053168015604648.CrossRefGoogle Scholar
Jäger, K (2017) The potential of online sampling for studying political activists around the world and across time. Political Analysis 25(3), 329–343.CrossRefGoogle Scholar
Kapp, JM, Peters, CandOliver, DP (2013) Research Recruitment Using Facebook Advertising: Big Potential, Big Challenges. Journal of Cancer Education 28(1), 134–137.CrossRefGoogle ScholarPubMed
Kosinski, M, Matz, SC, Gosling, SD, Popov, VandStillwell, D (2015) Facebook as a Research Tool for the Social Sciences: Opportunities, Challenges, Ethical Considerations, and Practical Guidelines. American Psychologist 70(6), 543.CrossRefGoogle ScholarPubMed
Kosinski, M, Stillwell, DandGraepel, T (2013) Private Traits and Attributes Are Predictable from Digital Records of Human Behavior. Proceedings of the National Academy of Sciences 110(15), 5802–5805.CrossRefGoogle ScholarPubMed
Ramo, DEandProchaska, JJ (2012) Broad Reach and Targeted Recruitment Using Facebook for an Online Survey of Young Adult Substance Use. Journal of Medical Internet Research 14(1), e28.CrossRefGoogle ScholarPubMed
Ramo, DE, Rodriguez, TM, Chavez, K, Sommer, MJandProchaska, JJ (2014) Facebook Recruitment of Young Adult Smokers for a Cessation Trial: Methods, Metrics, and Lessons Learned. Internet Interventions 1(2), 58–64.CrossRefGoogle ScholarPubMed
Ryan, TJ (2012) What Makes Us Click? Demonstrating Incentives for Angry Discourse with Digital-Age Field Experiments. The Journal of Politics 74(4), 1138–1152.CrossRefGoogle Scholar
Samuels, DJandZucco, C (2013) Using Facebook as a Subject Recruitment Tool for Survey-Experimental Research. Working Paper, available at SSRN 2101458.Google Scholar
Samuels, DJandZucco, C (2014) The Power of Partisanship in Brazil: Evidence from Survey Experiments. American Journal of Political Science 58(1), 212–225.CrossRefGoogle Scholar
Sances, MW (2017) Ideology and Vote Choice in US Mayoral Elections: Evidence from Facebook Surveys. Political Behavior 40(3), 737–762.Google Scholar
Sances, MW (2018) Missing the Target? Using Surveys to Validate Social Media Ad Targeting. Working Paper, Unpublished paper.CrossRefGoogle Scholar
Santoso, LP, Stein, RandStevenson, R (2016) Survey Experiments with Google Consumer Surveys: Promise and Pitfalls for Academic Research in Social Science. Political Analysis 24(3), 356–373.CrossRefGoogle Scholar
Smith, AandAnderson, M (2018) Social Media Use in 2018. Survey Report, Washington, DC.Google Scholar
Stewart, N, Ungemach, C, Harris, AJ, Bartels, DM, Newell, BR, Paolacci, GandChandler, J (2015) The Average Laboratory Samples a Population of 7,300 Amazon Mechanical Turk Workers. Judgment and Decision Making 10(5), 479.Google Scholar
Youyou, W, Kosinski, MandStillwell, D (2015) Computer-Based Personality Judgments Are More Accurate Than Those Made by Humans. Proceedings of the National Academy of Sciences 112(4), 1036–1040.CrossRefGoogle ScholarPubMed
Supplementary material: PDF

Zhang et al. supplementary material

Zhang et al. supplementary material

Download Zhang et al. supplementary material(PDF)
PDF 459.8 KB
Supplementary material: Link

Zhang et al. Dataset

Dataset

https://doi.org/10.7910/DVN/XTXS4P
Link