Skip to main content Accessibility help

The Effect of Survey Mode and Sampling on Inferences about Political Attitudes and Behavior: Comparing the 2000 and 2004 ANES to Internet Surveys with Nonprobability Samples

  • Neil Malhotra (a1) and Jon A. Krosnick (a2)

Since the inception of the American National Election Study (ANES) in the 1940s, data have been collected via face-to-face interviewing in the homes of members of area probability samples of American adults, the same gold-standard approach used by the U.S. Census Bureau, other federal agencies, and some nongovernment researchers for many of the most high-profile surveys conducted today. This paper explores whether comparable findings about voters and elections would be obtained by a different, considerably less expensive method: Internet data collection from nonprobability samples of volunteer respondents. Comparisons of the 2000 and 2004 ANES data (collected via face-to-face interviewing with national probability samples) with simultaneous Internet surveys of volunteer samples yielded many differences in the distributions of variables and in the associations between variables (even controlling for differences between the samples in reported interest in politics). Accuracy was higher for the face-to-face/probability sample data than for the Internet/volunteer sample data in 88% of the possible comparisons. This suggests that researchers interested in assuring the accuracy of their findings in describing populations should rely on face-to-face surveys of probability samples rather than Internet samples of volunteer respondents.

Hide All
Chang, LinChiat, and Krosnick, Jon A. 2002. Comparing self-administered computer surveys and auditory interviews: An experiment. Paper presented at the American Association for Public Opinion Research Annual Meeting, St. Petersburg, FL.
Converse, Philip E. 1964. The nature of belief systems in mass publics. In Ideology and discontent, ed. Apter, D. E., 206–61. New York: Free Press.
Erikson, Robert S., and Tedin, Kent L. 2007. American public opinion. 7th ed., updated. New York: Longman.
Holbrook, Allyson L., Green, Melanie C., and Krosnick, Jon A. 2003. Telephone vs. face-to-face interviewing of national probability samples with long questionnaires: Comparisons of respondent satisficing and social desirability response bias. Public Opinion Quarterly 67: 79125.
Rossi, Peter H., Wright, James D., and Anderson, Andy B. 1983. Handbook of survey research. Orlando, FL: Academic Press.
Saris, Willem. 1998. Ten years of interviewing without interviewers: The Telepanel. In Computer assisted survey information collection, ed. Couper, Mick P., Baker, Reginald P., Bethlehem, Jelke, Clark, Cynthia Z. F., Martin, Jean, Nicholls, William L. II, and O'Reilly, James M., 409–29. New York: Wiley.
Sears, David O. 1986. College sophomores in the laboratory: Influences of a narrow database on social psychology's view of human nature. Journal of Personality and Social Psychology 51 (September): 515–30.
Verba, Sidney, Schlozman, Kay Lehman, and Brady, Henry E. 1995. Voice and equality: Civic voluntarism in American politics. Cambridge, MA: Harvard University Press.
Wolfinger, Raymond E., and Rosenstone, Steven J. 1980. Who votes? New Haven, CT: Yale University Press.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Political Analysis
  • ISSN: 1047-1987
  • EISSN: 1476-4989
  • URL: /core/journals/political-analysis
Please enter your name
Please enter a valid email address
Who would you like to send this to? *


Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed