Hostname: page-component-7dd5485656-tbj44 Total loading time: 0 Render date: 2025-10-28T06:28:06.229Z Has data issue: false hasContentIssue false
  • English
  • Français

Validation: What Big Data Reveal About Survey Misreporting and the Real Electorate

Published online by Cambridge University Press:  04 January 2017

Stephen Ansolabehere  and
Eitan Hersh
Stephen Ansolabehere
Affiliation:
Harvard University
Eitan Hersh*
Affiliation:
Institution for Social and Policy Studies, Yale University, New Haven, CT 06520-8209
*
e-mail: eitan.hersh@yale.edu (corresponding author)
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

Social scientists rely on surveys to explain political behavior. From consistent overreporting of voter turnout, it is evident that responses on survey items may be unreliable and lead scholars to incorrectly estimate the correlates of participation. Leveraging developments in technology and improvements in public records, we conduct the first-ever fifty-state vote validation. We parse overreporting due to response bias from overreporting due to inaccurate respondents. We find that nonvoters who are politically engaged and equipped with politically relevant resources consistently misreport that they voted. This finding cannot be explained by faulty registration records, which we measure with new indicators of election administration quality. Respondents are found to misreport only on survey items associated with socially desirable outcomes, which we find by validating items beyond voting, like race and party. We show that studies of representation and participation based on survey reports dramatically misestimate the differences between voters and nonvoters.

Information

Type
Research Article
Information
Political Analysis , Volume 20 , Issue 4 , Autumn 2012 , pp. 437 - 459
Copyright
Copyright © The Author 2012. Published by Oxford University Press on behalf of the Society for Political Methodology 

Footnotes

Authors' note: We thank the editors and anonymous reviewers for their helpful feedback. For materials to replicate the statistical analysis in this article, see Ansolabehere and Hersh (2012).

References

Ansolabehere, Stephen, and Hersh, Eitan. 2010. The quality of voter registration records: A state-by-state analysis. Caltech/MIT Voting Technology Project Report.Google Scholar
Ansolabehere, Stephen, and Hersh, Eitan. 2011a. Who Really Votes? In Facing the challenge of democracy? Explorations in the analysis of public opinion and political participation, eds. Highton, Benjamin and Sniderman, Paul. Princeton, NJ: Princeton University Press.Google Scholar
Ansolabehere, Stephen, and Hersh, Eitan. 2011b. Gender by race: The massive participation gender gap in American politics. Paper presented at the annual meeting of the American Political Science Association, Seattle, WA.Google Scholar
Ansolabehere, Stephen, and Hersh, Eitan. 2012. Replication data for validation: What big data reveal about survey misreporting and the real electorate [dataset]. IQSS Dataverse Network [Distributor] V1 [Version]. http://hdl.handle.net/1902.1/18300 (accessed July 30, 2012).Google Scholar
Ansolabehere, Stephen, Hersh, Eitan, Gerber, Alan, and Doherty, David. 2010. Voter registration list quality pilot studies: Report on detailed results and report on methodology. Washington, DC: Pew Center on the States.Google Scholar
Belli, Robert F., Traugott, Michael W., Young, Margaret, and McGonagle, Katherine A. 1999. Reducing vote overreporting in surveys: Social desirablity, memory failure, and source monitoring. Public Opinion Quarterly 63(1): 90–108.CrossRefGoogle Scholar
Belli, Robert F., Traugott, Michael W., and Beckmann, Matthew N. 2001. What leads to voting overreports? Contrasts of overreporters to validated voters and admitted nonvoters in the American National Election Studies. Journal of Official Statistics 17(4): 479–98.Google Scholar
Belli, Robert F., Moore, Sean E., and VanHoewyk, John. 2006. An experimental comparison of question forms used to reduce voter overreporting. Electoral Studies 25(4): 751–9.Google Scholar
Berent, Matthew K., Krosnick, Jon A., and Lupia, Arthur. 2011. The quality of government records and “overestimation” of registration and turnout in surveys: Lessons from the 2008 ANES Panel Study's Registration and Turnout Validation Exercises. American National Election Studies Working Paper No. nes012554.Google Scholar
Bernstein, Robert, Chadha, Anita, and Montjoy, Robert. 2001. Overreporting voting: Why it happens and why it matters. Public Opinion Quarterly 65(1): 22–44.Google Scholar
Burden, Barry C. 2000. Voter turnout and the National Election Studies. Political Analysis 8(4): 389–98.Google Scholar
Burden, Barry C. 2003. Internal and external effects on the accuracy of NES turnout: Reply. Political Analysis 11(2): 193–5.CrossRefGoogle Scholar
Burns, Nancy, Schlozman, Kay Lehman, and Verba, Sidney. 2001. The private roots of public action. Cambridge, MA: Harvard University Press.CrossRefGoogle Scholar
Campbell, James E. 2010. Explaining politics, not polls: Reexamining macropartisanship with recalibrated NES Data. Public Opinion Quarterly 74(4): 616–42.Google Scholar
Cassel, Carol A. 2003. Overreporting and electoral participation research. American Politics Research 31(1): 81–92.Google Scholar
Cassel, Carol A. 2004. Voting records and validated voting studies. Public Opinion Quarterly 68(1): 102–8.Google Scholar
Citrin, Jack, Schickler, Eric, and Sides, John. 2003. What if everyone voted? Simulating the impact of increased turnout in senate elections. American Journal of Political Science 47(1): 75–90.Google Scholar
Denscombe, Martyn. 2006. Web-based questionnaires and the mode effect: An evaluation based on completion rates and data contents of near-identical questionnaires delivered in different modes. Social Science Computer Review 24(2): 246–54.CrossRefGoogle Scholar
Deufel, Benjamin J., and Kedar, Orit. 2010. Race and turnout in U.S. elections: Exposing hidden effects. Public Opinion Quarterly 74(2): 286–318.Google Scholar
Duff, Brian, Hanmer, Michael J., Park, Won-Ho, and White, Ismail K. 2007. Good excuses: Understanding who votes with an improved turnout question. Public Opinion Quarterly 71(1): 67–90.Google Scholar
Fullerton, Andrew S., Dixon, Jeffrey C., and Borch, Casey. 2007. Bringing registration into models of vote overreporting. Public Opinion Quarterly 71(4): 649–60.Google Scholar
Harbaugh, William T. 1996. If people vote because they like to, then why do so many of them lie? Public Choice 89(1): 63–76.Google Scholar
Highton, Benjamin, and Wolfinger, Raymond E. 2001. The political implications of higher turnout. British Journal of Political Science 31(01): 179–223.Google Scholar
Karp, Jeffrey A., and Brockington, David. 2005. Social desirability and response validity: A comparative analysis of overreporting voter turnout in five countries. Journal of Politics 67(3): 825–40.Google Scholar
Katosh, John P., and Traugott, Michael W. 1981. The consequences of validated and self-reported voting measures. Public Opinion Quarterly 45(4): 519–35.Google Scholar
Katz, Jonathan N., and Katz, Gabriel. 2010. Correcting for survey misreports using auxiliary information with an application to estimating turnout. American Journal of Political Science 54(3): 815–35.Google Scholar
Malhotra, Neil, and Krosnick, Jon A. 2007. The effect of survey mode and sampling on inferences about political attitudes and behavior: Comparing the 2000 and 2004 ANES to Internet surveys with nonprobability samples. Political Analysis 15(3): 286–323.CrossRefGoogle Scholar
Martinez, Michael D. 2003. Comment on “voter turnout and the National Election Studies.” Political Analysis 11(2): 187–92.CrossRefGoogle Scholar
McDonald, Michael P. 2003. On the overreport bias of the National Election Study turnout rate. Political Analysis 11(2): 180–6.Google Scholar
McDonald, Michael P. 2011. “Turnout, 1980–2010.” United States Elections Project. http://elections.gmu.edu/voter_turnout.htm (accessed May 16, 2011).Google Scholar
Presser, Stanley, Traugott, Michael W., and Traugott, Santa. 1990. Vote “over” reporting in surveys: The records or the respondents. ANES Technical Report Series: Doc. nes010157. Ann Arbor, MI: American National Election Studies.Google Scholar
Rosenstone, Steven J., and Leege, David C. 1994. An update on the National Election Studies. PS: Political Science and Politics 27(4): 693–8.Google Scholar
Rosenstone, Steven J., and Hansen, John Mark. 1993. Mobilization, participation, and democracy in America. New York: Macmillan.Google Scholar
Sigelman, Lee. 1982. The nonvoting voter in voting research. American Journal of Political Science 26(1): 47–56.Google Scholar
Silver, Brian D., Anderson, Barbara A., and Abramson, Paul R. 1986. Who overreports voting? American Political Science Review 80(2): 613–24.Google Scholar
University, Stanford and the University of Michigan [producers and distributors]. 2010. American National Election Studies Time Series Cumulative Data File [dataset] version. http://electionstudies.org/studypages/cdf/cdf.htm (accessed June 24, 2010).Google Scholar
Traugott, Michael W., Traugott, Santa M., and Presser, Stanley. 1992. Revalidation of self-reported vote. ANES Technical Report Series: Doc. nes010160. Ann Arbor, MI: American National Election Studies.Google Scholar
Verba, Sidney, Schlozman, Kay L., and Brady, Henry E. 1995. Voice and equality: Civic voluntarism in American Politics. Cambridge, MA: Harvard University Press.Google Scholar
Wolfinger, Raymond E., and Rosenstone, Steven J. 1980. Who votes? New Haven, CT: Yale University Press.Google Scholar
Supplementary material: PDF

Ansolabehere and Hersh supplementary material

Supplementary Material

Download Ansolabehere and Hersh supplementary material(PDF)
PDF 123.6 KB