Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Home
Hostname: page-component-55597f9d44-rn2sj Total loading time: 0.274 Render date: 2022-08-17T11:09:36.187Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "useRatesEcommerce": false, "useNewApi": true } hasContentIssue true
  • English
Political Science Research and Methods Political Science Research and Methods

Article contents

Quota sampling using Facebook advertisements

Published online by Cambridge University Press:  05 December 2018

Baobao Zhang Open the ORCID record for Baobao Zhang[Opens in a new window] ,
Matto Mildenberger ,
Peter D. Howe ,
Jennifer Marlon ,
Seth A. Rosenthal  and
Anthony Leiserowitz
Baobao Zhang*
Affiliation:
Department of Political Science, Yale University, New Haven, CT, USA
Matto Mildenberger
Affiliation:
Department of Political Science, University of California, Santa Barbara, Santa Barbara, CA, USA
Peter D. Howe
Affiliation:
Department of Environment and Society, Utah State University, Logan, UT, USA
Jennifer Marlon
Affiliation:
School of Forestry and Environmental Studies, Yale University, New Haven, CT, USA
Seth A. Rosenthal
Affiliation:
School of Forestry and Environmental Studies, Yale University, New Haven, CT, USA
Anthony Leiserowitz
Affiliation:
School of Forestry and Environmental Studies, Yale University, New Haven, CT, USA
*
*Corresponding author. Email: baobao.zhang@yale.edu
Get access Rights & Permissions[Opens in a new window]

Abstract

Researchers in different social science disciplines have successfully used Facebook to recruit subjects for their studies. However, such convenience samples are not generally representative of the population. We developed and validated a new quota sampling method to recruit respondents using Facebook advertisements. Additionally, we published an R package to semi-automate this quota sampling process using the Facebook Marketing API. To test the method, we used Facebook advertisements to quota sample 2432 US respondents for a survey on climate change public opinion. We conducted a contemporaneous nationally representative survey asking identical questions using a high-quality online survey panel whose respondents were recruited using probability sampling. Many results from the Facebook-sampled survey are similar to those from the online panel survey; furthermore, results from the Facebook-sampled survey approximate results from the American Community Survey (ACS) for a set of validation questions. These findings suggest that using Facebook to recruit respondents is a viable option for survey researchers wishing to approximate population-level public opinion.

Type
Research Notes
Information
Political Science Research and Methods , Volume 8 , Issue 3 , July 2020 , pp. 558 - 564
Copyright
Copyright © The European Political Science Association 2018

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Berinsky, AJ, Huber, GAandLenz, GS (2012) Evaluating Online Labor Markets for Experimental Research: Amazon. com’s Mechanical Turk. Political Analysis 20(3), 351–368.CrossRefGoogle Scholar
Bond, RandMessing, S (2015) Quantifying Social Media’s Political Space: Estimating Ideology from Publicly Revealed Preferences on Facebook. American Political Science Review 109(01), 62–78.CrossRefGoogle Scholar
Broockman, DEandGreen, DP (2014) Do Online Advertisements Increase Political Candidates’ Name Recognition or Favorability? Evidence from Randomized Field Experiments. Political Behavior 36(2), 263–289.CrossRefGoogle Scholar
Hirano, S, Lenz, GS, Pinkovskiy, MandSnyder, JM (2015) Voter Learning in State Primary Elections. American Journal of Political Science 59(1), 91–108.CrossRefGoogle Scholar
Huff, CandTingley, D (2015) “Who Are These People?” Evaluating the Demographic Characteristics and Political Preferences of Mturk Survey Respondents. Research & Politics 2(3), 2053168015604648.CrossRefGoogle Scholar
Jäger, K (2017) The potential of online sampling for studying political activists around the world and across time. Political Analysis 25(3), 329–343.CrossRefGoogle Scholar
Kapp, JM, Peters, CandOliver, DP (2013) Research Recruitment Using Facebook Advertising: Big Potential, Big Challenges. Journal of Cancer Education 28(1), 134–137.CrossRefGoogle ScholarPubMed
Kosinski, M, Matz, SC, Gosling, SD, Popov, VandStillwell, D (2015) Facebook as a Research Tool for the Social Sciences: Opportunities, Challenges, Ethical Considerations, and Practical Guidelines. American Psychologist 70(6), 543.CrossRefGoogle ScholarPubMed
Kosinski, M, Stillwell, DandGraepel, T (2013) Private Traits and Attributes Are Predictable from Digital Records of Human Behavior. Proceedings of the National Academy of Sciences 110(15), 5802–5805.CrossRefGoogle ScholarPubMed
Ramo, DEandProchaska, JJ (2012) Broad Reach and Targeted Recruitment Using Facebook for an Online Survey of Young Adult Substance Use. Journal of Medical Internet Research 14(1), e28.CrossRefGoogle ScholarPubMed
Ramo, DE, Rodriguez, TM, Chavez, K, Sommer, MJandProchaska, JJ (2014) Facebook Recruitment of Young Adult Smokers for a Cessation Trial: Methods, Metrics, and Lessons Learned. Internet Interventions 1(2), 58–64.CrossRefGoogle ScholarPubMed
Ryan, TJ (2012) What Makes Us Click? Demonstrating Incentives for Angry Discourse with Digital-Age Field Experiments. The Journal of Politics 74(4), 1138–1152.CrossRefGoogle Scholar
Samuels, DJandZucco, C (2013) Using Facebook as a Subject Recruitment Tool for Survey-Experimental Research. Working Paper, available at SSRN 2101458.Google Scholar
Samuels, DJandZucco, C (2014) The Power of Partisanship in Brazil: Evidence from Survey Experiments. American Journal of Political Science 58(1), 212–225.CrossRefGoogle Scholar
Sances, MW (2017) Ideology and Vote Choice in US Mayoral Elections: Evidence from Facebook Surveys. Political Behavior 40(3), 737–762.Google Scholar
Sances, MW (2018) Missing the Target? Using Surveys to Validate Social Media Ad Targeting. Working Paper, Unpublished paper.CrossRefGoogle Scholar
Santoso, LP, Stein, RandStevenson, R (2016) Survey Experiments with Google Consumer Surveys: Promise and Pitfalls for Academic Research in Social Science. Political Analysis 24(3), 356–373.CrossRefGoogle Scholar
Smith, AandAnderson, M (2018) Social Media Use in 2018. Survey Report, Washington, DC.Google Scholar
Stewart, N, Ungemach, C, Harris, AJ, Bartels, DM, Newell, BR, Paolacci, GandChandler, J (2015) The Average Laboratory Samples a Population of 7,300 Amazon Mechanical Turk Workers. Judgment and Decision Making 10(5), 479.Google Scholar
Youyou, W, Kosinski, MandStillwell, D (2015) Computer-Based Personality Judgments Are More Accurate Than Those Made by Humans. Proceedings of the National Academy of Sciences 112(4), 1036–1040.CrossRefGoogle ScholarPubMed
Supplementary material: PDF

Zhang et al. supplementary material

Zhang et al. supplementary material

Download Zhang et al. supplementary material(PDF)
PDF 460 KB
Supplementary material: Link

Zhang et al. Dataset

Dataset

https://doi.org/10.7910/DVN/XTXS4P
Link
26
Cited by

Save article to Kindle

To save this article to your Kindle, first ensure coreplatform@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Quota sampling using Facebook advertisements
Available formats
×

Save article to Dropbox

To save this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Dropbox account. Find out more about saving content to Dropbox.

Quota sampling using Facebook advertisements
Available formats
×

Save article to Google Drive

To save this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Google Drive account. Find out more about saving content to Google Drive.

Quota sampling using Facebook advertisements
Available formats
×
×

Reply to: Submit a response

Please enter your response.

Your details

Please enter a valid email address.

Conflicting interests

Do you have any conflicting interests? *