Skip to main content Accessibility help
×
Home
Hostname: page-component-65d66dc8c9-zvvs7 Total loading time: 0.375 Render date: 2021-09-28T21:18:48.718Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "metricsAbstractViews": false, "figures": true, "newCiteModal": false, "newCitedByModal": true, "newEcommerce": true, "newUsageEvents": true }

Validation: What Big Data Reveal About Survey Misreporting and the Real Electorate

Published online by Cambridge University Press:  04 January 2017

Stephen Ansolabehere
Affiliation:
Harvard University
Eitan Hersh*
Affiliation:
Institution for Social and Policy Studies, Yale University, New Haven, CT 06520-8209
*
e-mail: eitan.hersh@yale.edu (corresponding author)
Rights & Permissions[Opens in a new window]

Abstract

HTML view is not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

Social scientists rely on surveys to explain political behavior. From consistent overreporting of voter turnout, it is evident that responses on survey items may be unreliable and lead scholars to incorrectly estimate the correlates of participation. Leveraging developments in technology and improvements in public records, we conduct the first-ever fifty-state vote validation. We parse overreporting due to response bias from overreporting due to inaccurate respondents. We find that nonvoters who are politically engaged and equipped with politically relevant resources consistently misreport that they voted. This finding cannot be explained by faulty registration records, which we measure with new indicators of election administration quality. Respondents are found to misreport only on survey items associated with socially desirable outcomes, which we find by validating items beyond voting, like race and party. We show that studies of representation and participation based on survey reports dramatically misestimate the differences between voters and nonvoters.

Type
Research Article
Copyright
Copyright © The Author 2012. Published by Oxford University Press on behalf of the Society for Political Methodology 

Footnotes

Authors' note: We thank the editors and anonymous reviewers for their helpful feedback. For materials to replicate the statistical analysis in this article, see Ansolabehere and Hersh (2012).

References

Ansolabehere, Stephen, and Hersh, Eitan. 2010. The quality of voter registration records: A state-by-state analysis. Caltech/MIT Voting Technology Project Report.Google Scholar
Ansolabehere, Stephen, and Hersh, Eitan. 2011a. Who Really Votes? In Facing the challenge of democracy? Explorations in the analysis of public opinion and political participation, eds. Highton, Benjamin and Sniderman, Paul. Princeton, NJ: Princeton University Press.Google Scholar
Ansolabehere, Stephen, and Hersh, Eitan. 2011b. Gender by race: The massive participation gender gap in American politics. Paper presented at the annual meeting of the American Political Science Association, Seattle, WA.Google Scholar
Ansolabehere, Stephen, and Hersh, Eitan. 2012. Replication data for validation: What big data reveal about survey misreporting and the real electorate [dataset]. IQSS Dataverse Network [Distributor] V1 [Version]. http://hdl.handle.net/1902.1/18300 (accessed July 30, 2012).Google Scholar
Ansolabehere, Stephen, Hersh, Eitan, Gerber, Alan, and Doherty, David. 2010. Voter registration list quality pilot studies: Report on detailed results and report on methodology. Washington, DC: Pew Center on the States.Google Scholar
Belli, Robert F., Traugott, Michael W., Young, Margaret, and McGonagle, Katherine A. 1999. Reducing vote overreporting in surveys: Social desirablity, memory failure, and source monitoring. Public Opinion Quarterly 63(1): 90108.CrossRefGoogle Scholar
Belli, Robert F., Traugott, Michael W., and Beckmann, Matthew N. 2001. What leads to voting overreports? Contrasts of overreporters to validated voters and admitted nonvoters in the American National Election Studies. Journal of Official Statistics 17(4): 479–98.Google Scholar
Belli, Robert F., Moore, Sean E., and VanHoewyk, John. 2006. An experimental comparison of question forms used to reduce voter overreporting. Electoral Studies 25(4): 751–9.CrossRefGoogle Scholar
Berent, Matthew K., Krosnick, Jon A., and Lupia, Arthur. 2011. The quality of government records and “overestimation” of registration and turnout in surveys: Lessons from the 2008 ANES Panel Study's Registration and Turnout Validation Exercises. American National Election Studies Working Paper No. nes012554.Google Scholar
Bernstein, Robert, Chadha, Anita, and Montjoy, Robert. 2001. Overreporting voting: Why it happens and why it matters. Public Opinion Quarterly 65(1): 2244.CrossRefGoogle ScholarPubMed
Burden, Barry C. 2000. Voter turnout and the National Election Studies. Political Analysis 8(4): 389–98.CrossRefGoogle Scholar
Burden, Barry C. 2003. Internal and external effects on the accuracy of NES turnout: Reply. Political Analysis 11(2): 193–5.CrossRefGoogle Scholar
Burns, Nancy, Schlozman, Kay Lehman, and Verba, Sidney. 2001. The private roots of public action. Cambridge, MA: Harvard University Press.Google Scholar
Campbell, James E. 2010. Explaining politics, not polls: Reexamining macropartisanship with recalibrated NES Data. Public Opinion Quarterly 74(4): 616–42.CrossRefGoogle Scholar
Cassel, Carol A. 2003. Overreporting and electoral participation research. American Politics Research 31(1): 8192.CrossRefGoogle Scholar
Cassel, Carol A. 2004. Voting records and validated voting studies. Public Opinion Quarterly 68(1): 102–8.CrossRefGoogle Scholar
Citrin, Jack, Schickler, Eric, and Sides, John. 2003. What if everyone voted? Simulating the impact of increased turnout in senate elections. American Journal of Political Science 47(1): 7590.CrossRefGoogle Scholar
Denscombe, Martyn. 2006. Web-based questionnaires and the mode effect: An evaluation based on completion rates and data contents of near-identical questionnaires delivered in different modes. Social Science Computer Review 24(2): 246–54.CrossRefGoogle Scholar
Deufel, Benjamin J., and Kedar, Orit. 2010. Race and turnout in U.S. elections: Exposing hidden effects. Public Opinion Quarterly 74(2): 286318.CrossRefGoogle Scholar
Duff, Brian, Hanmer, Michael J., Park, Won-Ho, and White, Ismail K. 2007. Good excuses: Understanding who votes with an improved turnout question. Public Opinion Quarterly 71(1): 6790.CrossRefGoogle Scholar
Fullerton, Andrew S., Dixon, Jeffrey C., and Borch, Casey. 2007. Bringing registration into models of vote overreporting. Public Opinion Quarterly 71(4): 649–60.CrossRefGoogle Scholar
Harbaugh, William T. 1996. If people vote because they like to, then why do so many of them lie? Public Choice 89(1): 6376.CrossRefGoogle Scholar
Highton, Benjamin, and Wolfinger, Raymond E. 2001. The political implications of higher turnout. British Journal of Political Science 31(01): 179223.CrossRefGoogle Scholar
Karp, Jeffrey A., and Brockington, David. 2005. Social desirability and response validity: A comparative analysis of overreporting voter turnout in five countries. Journal of Politics 67(3): 825–40.CrossRefGoogle Scholar
Katosh, John P., and Traugott, Michael W. 1981. The consequences of validated and self-reported voting measures. Public Opinion Quarterly 45(4): 519–35.CrossRefGoogle Scholar
Katz, Jonathan N., and Katz, Gabriel. 2010. Correcting for survey misreports using auxiliary information with an application to estimating turnout. American Journal of Political Science 54(3): 815–35.CrossRefGoogle Scholar
Malhotra, Neil, and Krosnick, Jon A. 2007. The effect of survey mode and sampling on inferences about political attitudes and behavior: Comparing the 2000 and 2004 ANES to Internet surveys with nonprobability samples. Political Analysis 15(3): 286323.CrossRefGoogle Scholar
Martinez, Michael D. 2003. Comment on “voter turnout and the National Election Studies.” Political Analysis 11(2): 187–92.CrossRefGoogle Scholar
McDonald, Michael P. 2003. On the overreport bias of the National Election Study turnout rate. Political Analysis 11(2): 180–6.CrossRefGoogle Scholar
McDonald, Michael P. 2011. “Turnout, 1980–2010.” United States Elections Project. http://elections.gmu.edu/voter_turnout.htm (accessed May 16, 2011).Google Scholar
Presser, Stanley, Traugott, Michael W., and Traugott, Santa. 1990. Vote “over” reporting in surveys: The records or the respondents. ANES Technical Report Series: Doc. nes010157. Ann Arbor, MI: American National Election Studies.Google Scholar
Rosenstone, Steven J., and Leege, David C. 1994. An update on the National Election Studies. PS: Political Science and Politics 27(4): 693–8.Google Scholar
Rosenstone, Steven J., and Hansen, John Mark. 1993. Mobilization, participation, and democracy in America. New York: Macmillan.Google Scholar
Sigelman, Lee. 1982. The nonvoting voter in voting research. American Journal of Political Science 26(1): 4756.CrossRefGoogle Scholar
Silver, Brian D., Anderson, Barbara A., and Abramson, Paul R. 1986. Who overreports voting? American Political Science Review 80(2): 613–24.CrossRefGoogle Scholar
University, Stanford and the University of Michigan [producers and distributors]. 2010. American National Election Studies Time Series Cumulative Data File [dataset] version. http://electionstudies.org/studypages/cdf/cdf.htm (accessed June 24, 2010).Google Scholar
Traugott, Michael W., Traugott, Santa M., and Presser, Stanley. 1992. Revalidation of self-reported vote. ANES Technical Report Series: Doc. nes010160. Ann Arbor, MI: American National Election Studies.Google Scholar
Verba, Sidney, Schlozman, Kay L., and Brady, Henry E. 1995. Voice and equality: Civic voluntarism in American Politics. Cambridge, MA: Harvard University Press.Google Scholar
Wolfinger, Raymond E., and Rosenstone, Steven J. 1980. Who votes? New Haven, CT: Yale University Press.Google Scholar
Supplementary material: PDF

Ansolabehere and Hersh supplementary material

Supplementary Material

Download Ansolabehere and Hersh supplementary material(PDF)
PDF 124 KB
You have Access
185
Cited by

Send article to Kindle

To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Validation: What Big Data Reveal About Survey Misreporting and the Real Electorate
Available formats
×

Send article to Dropbox

To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

Validation: What Big Data Reveal About Survey Misreporting and the Real Electorate
Available formats
×

Send article to Google Drive

To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

Validation: What Big Data Reveal About Survey Misreporting and the Real Electorate
Available formats
×
×

Reply to: Submit a response

Please enter your response.

Your details

Please enter a valid email address.

Conflicting interests

Do you have any conflicting interests? *