Skip to main content Accessibility help
×
×
Home

Online Polls and Registration-Based Sampling: A New Method for Pre-Election Polling

  • Michael J. Barber (a1), Christopher B. Mann (a2), J. Quin Monson (a3) and Kelly D. Patterson (a4)

Abstract

This article outlines a new method for surveys to study elections and voter attitudes. Pre-election surveys often suffer from an inability to identify and survey the likely electorate for the upcoming election. We propose a new and inexpensive method to conduct representative surveys of the electorate. We demonstrate the performance of our method in producing a representative sample of the future electorate that can be used to study campaign dynamics and many other issues. We compare pre-election outcome forecasts to election outcomes in seven primary and general election surveys conducted prior to the 2008 and 2010 primary and general elections in three states. The results indicate that the methodology produces representative samples, including in low-turnout elections such as primaries where traditional methods have difficulty consistently sampling the electorate. This new methodology combines Probability Proportional to Size (PPS) sampling, mailed invitation letters, and online administration of the questionnaire. The PPS sample is drawn based on a model employing variables from the publicly available voter file to produce a probability of voting score for each individual voter. The proposed method provides researchers a valuable tool to study the attitudes of the voting public.

Copyright

Corresponding author

e-mail: cmann13@lsu.edu (corresponding author)

Footnotes

Hide All

Authors' note: We thank Anand Sohkey and his colleagues at University of Colorado—Boulder for their assistance in conducting the 2010 primary surveys in Colorado and Matthew Frei of the Center for the Study of Elections and Democracy (CSED) at Brigham Young University (BYU) for his assistance in administering the surveys. We thank Lonna Atkeson, Larry Bartels, John Love, Mark A. Schulman, the anonymous reviewers, and participants in a faculty research seminar at BYU for helpful comments. Previous versions of this paper were presented at the annual meetings of the American Association of Public Opinion Research in 2010 and 2011. This research would not have been possible without the generous support of CSED at BYU, the University of Colorado—Boulder, and the University of Miami. All errors are the responsibility of the authors. Supplementary materials for this article are available on the Political Analysis Web site. Replication data are available on the Political Analysis Dataverse at http://dx.doi.org/10.7910/DVN/22908.

Footnotes

References

Hide All
Alvarez, R. Michael, Sherman, Robert P., and VanBeselaere, Carla. 2003. Subject acquisition for web-based surveys. Political Analysis 1: 2343.
Ansolabehere, Stephen, and Hersh, Eitan. 2012. Misreporting, sample selection, and participation. Political Analysis 20: 437–59.
Atkeson, Lonna R., and Tafoya, Lorraine M. 2008. Surveying political activists: The effectiveness of a mixed mode survey design. Journal of Elections, Public Opinion & Parties 18: 367.
Atkeson, Lonna, Adams, Alex, Bryant, Lisa, Zilberman, Luciana, and Saunders, Kyle. 2011. Considering mixed mode surveys for questions in political behavior: Using the internet and mail to get quality data at reasonable costs. Political Behavior 33: 161–78.
Baker, Reg, Michael Brick, J., Bates, Nancy A., Battaglia, Mike, Couper, Mick P., Dever, Jill A., Gile, Krista J., and Tourangeau, Roger. 2013. Report of the AAPOR Task Force on Non-Probability Sampling. May 2013 (updated June 22, 2013). http://www.aapor.org/Reports1.htm (accessed September 18, 2013).
Baker, Reg, Blumberg, Stephen J., Michael Brick, J., Couper, Mick P., Courtright, Melanie, Michael Dennis, J., Dillman, Don, Frankel, Martin R., Garland, Philip, Groves, Robert M., Kennedy, Courtney, Krosnick, Jon, Lavrakas, Paul J., Lee, Sunghee, Link, Michael, Piekarski, Linda, Rao, Kumar, Thomas, Randall K., and Zahs, Dan. 2010. AAPOR Report on Online Panels. Public Opinion Quarterly 74: 711–81.
Barber, Michael J. 2013. Ideological donors, contribution limits, and the polarization of state legislatures. Presented at the Annual Meeting of the American Political Science Association.
Bartels, Larry. 1991. Constituency opinion and congressional policy making: The Reagan defense buildup. American Political Science Review 85: 457–74.
Berrens, Robert P., Bohara, Alok K., Jenkins-Smith, Hank, Silva, Carol, and Weimer, David L. 2003. The advent of internet surveys for political research: A comparison of telephone and internet samples. Political Analysis 11: 122.
Burden, Barry. 1997. Deterministic and probabilistic voting models. American Journal of Political Science 41: 1150–69.
Butler, Daniel, and Nickerson, David. 2011. Can learning constituency opinion affect how legislators vote? Results from a field experiment. Quarterly Journal of Political Science 6: 5583.
Chang, LinChiat, and Krosnick, Jon. 2009. National surveys via RDD telephone interviewing versus the internet: Comparing sample representativeness and response quality. Public Opinion Quarterly 73: 641–78.
Chang, LinChiat, and Krosnick, Jon. 2010. Comparing oral interviewing with self-administered computerized questionnaires: An experiment. Public Opinion Quarterly 74: 154–67.
Couper, Mick P. 2000. Web surveys: A review of issues and approaches. Public Opinion Quarterly 64: 464–94.
Crespi, Irving. 1988. Pre-election polling: Sources of accuracy and error. New York. Russell Sage Foundation.
Dillman, Don A., and Christian, Leah M. 2005. Survey mode as a source of instability in responses across surveys. Field Methods 17: 3052.
Dillman, Don A., Glenn, Phelps, Tortora, Robert, Swift, Karen, Kohrell, Julie, Berck, Jodi, and Messer, Benjamin L. 2009. Response rate and measurement differences in mixed-mode surveys using mail, telephone, interactive voice response (IVR) and the internet. Social Science Research 38: 118.
Fishbein, Martin, and Ajzen, Icek. 1975. Belief, attitude, intention, and behavior. Reading, MA: Addison-Wesley Publishing.
Freedman, Paul, and Goldstein, Ken. 1997. Building a probable electorate from pre-election polls: A two-stage approach. Public Opinion Quarterly 60: 574–87.
Frei, Matthew D., Quin Monson, J., Murray, Leah, and Patterson, Kelly D. 2012. Tea for only two: The ousting of Utah Senator Robert Bennett. In Tea Party Effects on 2010 U.S. Senate Elections: Stuck in the Middle to Lose, eds. Miller, William J. and Walling, Jeremy D., 105–20. Lanham, MD: Lexington Books.
Green, Donald P., and Gerber, Alan S. 2006. Can registration-based sampling improve the accuracy of midterm election forecasts? Public Opinion Quarterly 70: 197223.
Gilens, Martin. 2012. Affluence and influence: Economic inequality and political power in America. Princeton, NJ: Princeton University Press.
Groves, Robert M. 2006. Non-response rates and non-response bias in household surveys. Public Opinion Quarterly 70: 646–75.
Groves, Robert M., and Peytcheva, Emilia. 2008. The impact of nonresponse rates on nonresponse bias. Public Opinion Quarterly 72: 167–89.
Hoek, Janet, and Daves, Robert P. 1997. Turnout prediction: A comparison of methodologies. Presented at the Annual Meeting of the American Association for Public Opinion Research, Norfolk, VA.
Jann, B. 2006. gsample: Stata module to draw a random sample. http://ideas.repec.org/c/boc/bocode/s456716.html (accessed February 8, 2012).
Juster, F. Thomas. 1960. Prediction and consumer buying intentions. American Economic Review 50: 604–22.
Karpowitz, Christopher F., King-Meadows, Tyson, Quin Monson, J., and Pope, Jeremy C. 2013. Carrying many credentials: Partisan, racial, and religious dynamics in the Utah Fourth District. Presented at the Annual Meeting of the Midwest Political Science Association.
Karpowitz, Christopher F., Quin Monson, J., Patterson, Kelly D., and Pope, Jeremy C. 2011. Tea time in America? The impact of the Tea Party movement on the 2010 midterm elections. PS: Political Science & Politics 44: 303–9.
Keeter, Scott, Miller, Carolyn, Kohut, Andrew, Groves, Robert M., and Presser, Stanley. 2000. Consequences of reducing nonresponse in a national telephone survey. Public Opinion Quarterly 64: 125–48.
Keeter, Scott, Kennedy, Courtney, Dimock, Michael, Best, Jonathan, and Craighill, Peyton. 2006. Gauging the impact of growing non-response on estimates from a national RDD telephone survey. Public Opinion Quarterly 70: 759–79.
Larcinese, Valentino. 2007. Does political knowledge increase turnout? Evidence from the 1997 British general election. Public Choice 131: 387411.
Lavrakas, Paul J. 1993. Telephone survey methods: Sampling, selection, and supervision. 2nd ed. Newbury Park, CA: Sage Publications.
Lohr, Sharon L. 2010. Sampling: Design and Analysis. 2nd ed. Boston: Brooks/Cole, Cengage Learning.
Malchow, Hal. 2008. Political Targeting. 2nd ed. Washington, DC: Predicted Lists LLC Publishing.
Malhotra, Neil, and Krosnick, Jon A. 2007. The effect of survey mode and sampling on inferences about political attitudes and behavior: Comparing the 2000 and 2004 ANES to internet surveys with nonprobability samples. Political Analysis 15: 286323.
Mann, Christopher B. 2005. Do advance letters improve preelection forecast accuracy? Public Opinion Quarterly 69: 561–71.
Martin, Elizabeth, Traugott, Michael, and Kennedy, Courtney. 2005. A review and proposal for a new measure of poll accuracy. Public Opinion Quarterly 69: 342–69.
Perry, Paul. 1960. Election survey procedures of the Gallup poll. Public Opinion Quarterly 24: 531–42.
Perry, Paul. 1979. Certain problems in election survey methodology. Public Opinion Quarterly 43: 312–25.
Petrocik, John R. 1991. An algorithm for estimating turnout as a guide to predicting elections. Public Opinion Quarterly 55: 643–47.
Pew Research Center for the People and the Press. 2012. Assessing the representativeness of public opinion surveys. http://www.people-press.org/files/legacy-pdf/Assessing%20the%20Representativeness%20of%20Public%20Opinion%20Surveys.pdf (accessed August 5, 2012).
Peytchev, Andy, Baxter, Rodney K., and Carley-Baxter, Lisa R. 2009. Not all survey effort is equal: Reduction of non-response bias and non-response error. Public Opinion Quarterly 73: 785806.
Rogers, Todd, and Aida, Masa. 2011. Vote self-prediction hardly predicts who will vote, and is (misleadingly) unbiased. American Politics Research. Published online before print September 5, 2013, doi: 10.1177/1532673X13496453.
Sanders, David, Clarke, Harold D., Stewart, Marianne C., and Whiteley, Paul. 2007. Does mode matter for modeling political choice? Evidence from the 2005 British Election Study. Political Analysis 15: 257–85.
Scheiber, Noam. 2012. The internal polls that made Mitt Romney think he'd win. New Republic November 30. 2012. http://www.newrepublic.com/blog/plank/110597/exclusive-the-polls-made-mitt-romney-think-hed-win (accessed December 1, 2012).
Silver, Brian D., Anderson, Barbara A., and Abramson, Paul R. 1986. Who overreports voting? American Political Science Review 80: 613–24.
Sokhey, Anand E., and Djupe, Paul A. 2014. Name generation in interpersonal political network data: Results from a series of experiments. Social Networks 36: 147–61.
Stephenson, Laura B., and Crête, Jean. 2011. Studying political behavior: A comparison of internet and telephone surveys. International Journal of Public Opinion Research 23: 2455.
Traugott, Michael. 2012. Methodological trends and controversies in the media's use of opinion polls. In Opinion polls and the media: Reflecting and shaping public opinion, eds. Holtz-Bacha, Christina and Strömbäck, Jesper, 6990. New York: Palgrave-Macmillan.
Traugott, Michael W., and Tucker, Clyde. 1984. Strategies for predicting whether a citizen will vote and estimation of electoral outcomes. Public Opinion Quarterly 48: 330–43.
Traugott, Michael. 2005. The accuracy of the national pre-election polls in the 2004 presidential election. Public Opinion Quarterly 69: 642–54.
Traugott, Michael W., and Wlezien, Christopher. 2009. The dynamics of poll performance during the 2008 presidential nomination contest. Public Opinion Quarterly 73: 866–94.
Visser, Penny S., Krosnick, Jon A., Marquette, Jesse, and Curtin, Michael. 1996. Mail surveys for election forecasting? An evaluation of the Columbus Dispatch poll. Public Opinion Quarterly 60: 181227.
Voss, D. Stephen, Gelman, Andrew, and King, Gary. 1995. Pre-election survey methodology: Details from eight polling organizations, 1988 and 1992. Public Opinion Quarterly 59: 98132.
Weisberg, Herbert F. 2005. The total survey error approach: A guide to the new science of survey research. Chicago: University of Chicago Press.
Yeager, David S., Krosnick, Jon, Chang, LinChiat, Javitz, Harold S., Levendusky, Matthew S., Simpser, Alberto, and Wang, Rui. 2011. Comparing the accuracy of RDD telephone surveys and internet surveys conducted with probability and non-probability samples. Public Opinion Quarterly 75: 709–47.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Political Analysis
  • ISSN: 1047-1987
  • EISSN: 1476-4989
  • URL: /core/journals/political-analysis
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×
MathJax
Type Description Title
PDF
Supplementary materials

Barber et al. supplementary material
Supplementary Material

 PDF (2.3 MB)
2.3 MB
WORD
Supplementary materials

Barber et al. supplementary material
Supplementary Material

 Word (3.2 MB)
3.2 MB

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed