Skip to main content
×
×
Home

Quota sampling using Facebook advertisements

  • Baobao Zhang (a1), Matto Mildenberger (a2), Peter D. Howe (a3), Jennifer Marlon (a4), Seth A. Rosenthal (a4) and Anthony Leiserowitz (a4)...
Abstract

Researchers in different social science disciplines have successfully used Facebook to recruit subjects for their studies. However, such convenience samples are not generally representative of the population. We developed and validated a new quota sampling method to recruit respondents using Facebook advertisements. Additionally, we published an R package to semi-automate this quota sampling process using the Facebook Marketing API. To test the method, we used Facebook advertisements to quota sample 2432 US respondents for a survey on climate change public opinion. We conducted a contemporaneous nationally representative survey asking identical questions using a high-quality online survey panel whose respondents were recruited using probability sampling. Many results from the Facebook-sampled survey are similar to those from the online panel survey; furthermore, results from the Facebook-sampled survey approximate results from the American Community Survey (ACS) for a set of validation questions. These findings suggest that using Facebook to recruit respondents is a viable option for survey researchers wishing to approximate population-level public opinion.

Copyright
Corresponding author
*Corresponding author. Email: baobao.zhang@yale.edu
References
Hide All
Berinsky, AJ, Huber, GA and Lenz, GS (2012) Evaluating Online Labor Markets for Experimental Research: Amazon. com’s Mechanical Turk. Political Analysis 20(3), 351368.
Bond, R and Messing, S (2015) Quantifying Social Media’s Political Space: Estimating Ideology from Publicly Revealed Preferences on Facebook. American Political Science Review 109(01), 6278.
Broockman, DE and Green, DP (2014) Do Online Advertisements Increase Political Candidates’ Name Recognition or Favorability? Evidence from Randomized Field Experiments. Political Behavior 36(2), 263289.
Hirano, S, Lenz, GS, Pinkovskiy, M and Snyder, JM (2015) Voter Learning in State Primary Elections. American Journal of Political Science 59(1), 91108.
Huff, C and Tingley, D (2015) “Who Are These People?” Evaluating the Demographic Characteristics and Political Preferences of Mturk Survey Respondents. Research & Politics 2(3), 2053168015604648.
Jäger, K (2017) The potential of online sampling for studying political activists around the world and across time. Political Analysis 25(3), 329343.
Kapp, JM, Peters, C and Oliver, DP (2013) Research Recruitment Using Facebook Advertising: Big Potential, Big Challenges. Journal of Cancer Education 28(1), 134137.
Kosinski, M, Matz, SC, Gosling, SD, Popov, V and Stillwell, D (2015) Facebook as a Research Tool for the Social Sciences: Opportunities, Challenges, Ethical Considerations, and Practical Guidelines. American Psychologist 70(6), 543.
Kosinski, M, Stillwell, D and Graepel, T (2013) Private Traits and Attributes Are Predictable from Digital Records of Human Behavior. Proceedings of the National Academy of Sciences 110(15), 58025805.
Ramo, DE and Prochaska, JJ (2012) Broad Reach and Targeted Recruitment Using Facebook for an Online Survey of Young Adult Substance Use. Journal of Medical Internet Research 14(1), e28.
Ramo, DE, Rodriguez, TM, Chavez, K, Sommer, MJ and Prochaska, JJ (2014) Facebook Recruitment of Young Adult Smokers for a Cessation Trial: Methods, Metrics, and Lessons Learned. Internet Interventions 1(2), 5864.
Ryan, TJ (2012) What Makes Us Click? Demonstrating Incentives for Angry Discourse with Digital-Age Field Experiments. The Journal of Politics 74(4), 11381152.
Samuels, DJ and Zucco, C (2013) Using Facebook as a Subject Recruitment Tool for Survey-Experimental Research. Working Paper, available at SSRN 2101458.
Samuels, DJ and Zucco, C (2014) The Power of Partisanship in Brazil: Evidence from Survey Experiments. American Journal of Political Science 58(1), 212225.
Sances, MW (2017) Ideology and Vote Choice in US Mayoral Elections: Evidence from Facebook Surveys. Political Behavior 40(3), 737762.
Sances, MW (2018) Missing the Target? Using Surveys to Validate Social Media Ad Targeting. Working Paper, Unpublished paper.
Santoso, LP, Stein, R and Stevenson, R (2016) Survey Experiments with Google Consumer Surveys: Promise and Pitfalls for Academic Research in Social Science. Political Analysis 24(3), 356373.
Smith, A and Anderson, M (2018) Social Media Use in 2018. Survey Report, Washington, DC.
Stewart, N, Ungemach, C, Harris, AJ, Bartels, DM, Newell, BR, Paolacci, G and Chandler, J (2015) The Average Laboratory Samples a Population of 7,300 Amazon Mechanical Turk Workers. Judgment and Decision Making 10(5), 479.
Youyou, W, Kosinski, M and Stillwell, D (2015) Computer-Based Personality Judgments Are More Accurate Than Those Made by Humans. Proceedings of the National Academy of Sciences 112(4), 10361040.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Political Science Research and Methods
  • ISSN: 2049-8470
  • EISSN: 2049-8489
  • URL: /core/journals/political-science-research-and-methods
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×
Type Description Title
UNKNOWN
Supplementary materials

Zhang et al. Dataset
Dataset

 Unknown

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed