Hostname: page-component-89b8bd64d-72crv Total loading time: 0 Render date: 2026-05-09T15:31:48.855Z Has data issue: false hasContentIssue false
  • English
  • Français

Does Survey Mode Still Matter? Findings from a 2010 Multi-Mode Comparison

Published online by Cambridge University Press:  04 January 2017

Stephen Ansolabehere  and
Brian F. Schaffner
Stephen Ansolabehere
Affiliation:
Harvard University, 1737 Cambridge Street, CGIS Knafel Building 410, Cambridge, MA 02138. e-mail: sda@gov.harvard.edu
Brian F. Schaffner*
Affiliation:
University of Massachusetts, Thompson Hall 316, 200 Hicks Way, Amherst, MA 01003
*
e-mail: schaffne@polsci.umass.edu (corresponding author)
Get access Rights & Permissions [Opens in a new window]

Abstract

In this article, we present data from a three-mode survey comparison study carried out in 2010. National surveys were fielded at the same time over the Internet (using an opt-in Internet panel), by telephone with live interviews (using a national Random Digit Dialing (RDD) sample of landlines and cell phones), and by mail (using a national sample of residential addresses). Each survey utilized a nearly identical questionnaire soliciting information across a range of political and social indicators, many of which can be validated with government data. Comparing the findings from the modes using a Total Survey Error approach, we demonstrate that a carefully executed opt-in Internet panel produces estimates that are as accurate as a telephone survey and that the two modes differ little in their estimates of other political indicators and their correlates.

Information

Type
Symposium on Advances in Survey Methodology
Information
Political Analysis , Volume 22 , Issue 3 , Summer 2014 , pp. 285 - 303
Copyright
Copyright © The Author 2014. Published by Oxford University Press on behalf of the Society for Political Methodology 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

AAPOR Opt-In Online Panel Task Force. 2010. AAPOR Report on Online Panels. http://www.aapor.org/AAPOR_Releases_Report_on_Online_Survey_Panels/2263.htm.Google Scholar
Ansolabehere, Stephen. 2009. Guide to the 2008 Cooperative Congressional Election Survey. http://projects.iq.harvard.edu/cces/data?dvn_subpage=/faces/study/StudyPage.xhtml?globalId=hdl:1902.1/14003.Google Scholar
Ansolabehere, Stephen, and Schaffner, Brian F. 2013. Replication Data for: Does Survey Mode Still Matter? Findings from a 2010 Multi-Mode Comparison. http://dx.doi.org/10.7910/DVN/23025UNF:5:lIok9HT/VfayP4ULEfvB4A==IQSS Dataverse Network [Distributor] V1 [Version].Google Scholar
Atkeson, Lonna Rae, Bryant, Lisa A., Adams, Alex N., Zilberman, Luciana, and Saunders, Kyle L. 2011. Considering mixed mode surveys for questions in political behavior: Using the Internet and mail to get quality data at reasonable costs. Political Behavior 33: 161-78.Google Scholar
Atkeson, Lonna Rae, and Tafoya, Lori. 2008. Surveying political activists: An examination of the effectiveness of a mixed mode (Internet and mail) survey design. Journal of Elections, Public Opinion, and Parties 18 (4): 367-86.CrossRefGoogle Scholar
Biemer, Paul P. 2010. Total survey error: Design, implementation, and evaluation. Public Opinion Quarterly 74 (5): 817-48.CrossRefGoogle Scholar
Blumberg, Stephen J., and Luke, Julian V. 2009. Wireless Substitution: Early Release of Estimates from the National Health Interview Survey, January-June 2011. Centers for Disease Control. http://www.cdc.gov/nchs/data/nhis/earlyrelease/wireless201112.pdf.Google Scholar
Chang, Linchiat, and Krosnick, Jon A. 2009. National surveys via RDD telephone interviewing versus the Internet: Comparing sample representativeness and response quality. Public Opinion Quarterly 73 (4): 641-78.CrossRefGoogle Scholar
Chang, Linchiat. 2010. Comparing oral interviewing with self-administered computerized questionnaires: An experiment. Public Opinion Quarterly 74 (1): 154-67.Google Scholar
Chatt, Cindy, Michael Dennis, J., Li, Rick, and Pulliam, Paul. 2003. Data collection mode effects controlling for sample origins in a panel survey: Telephone versus Internet. Paper presented at the Annual Meeting of the Midwest Chapter of the American Association for Public Opinion Research, Chicago.Google Scholar
Curtin, R., Presser, S., and Singer, E. 2005. Changes in telephone survey nonresponse over the past quarter century. Public Opinion Quarterly 69: 87–98.Google Scholar
Carpini, Delli, Michael, X., and Keeter, Scott. 1997. What Americans know about politics and why it matters. New Haven, CT: Yale University Press.Google Scholar
Duncan, W. Jack. 1979. Mail questionnaires in survey research: A review of response inducement techniques. Journal of Management 5: 39–55.CrossRefGoogle Scholar
Groves, Robert M., Fowler, Floyd J. Jr., Couper, Mick P., Lepkowski, James M., and Singer, Eleanor. 2009. Survey Methodology. 2nd ed. Hoboken, NJ: J. Wiley.Google Scholar
Groves, Robert M., and Lyberg, Lars. 2010. Total survey error: Past, present, and future. Public Opinion Quarterly 74 (5): 849-79.CrossRefGoogle Scholar
Hill, Seth J., Lo, James, Vavreck, Lynn, and Zaller, John. 2007. The opt-in Internet panel: Survey mode, sample methodology, and the implications for political research. Unpublished manuscript.Google Scholar
Kenski, Kate, and Jomini Stroud, Natalie. 2006. Connections between Internet use and political efficacy, knowledge, and participation. Journal of Broadcasting & Electronic Media 50 (2): 173-92.Google Scholar
Link, Michael W., and Mokdad, Ali H. 2005. Alternative models for health surveillance surveys: An experiment with web, mail, and telephone. Epidemiology 16 (5): 701-4.Google Scholar
Malhotra, Neil, and Krosnick, Jon A. 2007. The effect of survey mode and sampling on inferences about political attitudes and behavior: Comparing the 2000 and 2004 ANES to Internet surveys with nonprobability samples. Political Analysis 15: 286–323.CrossRefGoogle Scholar
Pasek, Josh, and Krosnick, Jon A. 2010. Measuring intent to participate and participation in the 2010 census and their correlates and trends: Comparisons of RDD telephone and non-probability sample Internet survey data. Statistical Research Division of the US Census Bureau (#2010–15).Google Scholar
Rivers, Douglas. 2006. Sample matching: Representative sampling from Internet panels. http://www.websm.org/db/12/16408/Web%20Survey%20Bibliography/Sample_matching_Representative_sampling_from_Internet_panels/.Google Scholar
Rivers, Douglas. 2007. Sampling for web surveys. Paper prepared for the 2007 Join Statistical Meetings, Salt Lake City, UT.Google Scholar
Rivers, Douglas. 2009. Second thoughts about Internet surveys. Pollster.com (September 6). http://www.pollster.com/blogs/doug_rivers.php.Google Scholar
Sanders, David, Clarke, Harold D., Stewart, Marianne C., and Whiteley, Paul. 2007. Does mode matter for modeling political choice? Evidence from the 2005 British Election Study. Political Analysis 15: 257-85.CrossRefGoogle Scholar
Smith, Tom W. 2011. Refining the total survey error perspective. International Journal of Public Opinion Research 23 (4): 464-84.Google Scholar
Yeager, David S., Krosnick, Jon A., Chang, LinChiat, Javitz, Harold S., Levendusky, Matthew S., Simpser, Alberto, and Wang, Rui. 2009. Comparing the accuracy of RDD telephone surveys and Internet surveys conducted with probability and non-probability samples. Unpublished manuscript.Google Scholar