Skip to main content Accesibility Help
×
×
Home
  • Get access
    Check if you have access via personal or institutional login
  • Cited by 138
  • Cited by
    This chapter has been cited by the following publications. This list is generated based on data provided by CrossRef.

    Giner-Sorolla, Roger 2019. From crisis of evidence to a “crisis” of relevance? Incentive-based answers for social psychology’s perennial relevance worries. European Review of Social Psychology, Vol. 30, Issue. 1, p. 1.

    Kertzer, Joshua D. and Tingley, Dustin 2018. Political Psychology in International Relations: Beyond the Paradigms. Annual Review of Political Science, Vol. 21, Issue. 1, p. 319.

    Fahey, Kevin Weissert, Carol S. and Uttermark, Matthew J. 2018. Extra, Extra, (Don’t) Roll-off about It! Newspaper Endorsements for Ballot Measures. State Politics & Policy Quarterly, Vol. 18, Issue. 1, p. 93.

    Glas, Jeffrey M. and Taylor, J. Benjamin 2018. The Silver Screen and Authoritarianism: How Popular Films Activate Latent Personality Dispositions and Affect American Political Attitudes. American Politics Research, Vol. 46, Issue. 2, p. 246.

    Pandey, Ashish Chandwani, Rajesh and Navare, Ajinkya 2018. How can mindfulness enhance moral reasoning? An examination using business school students. Business Ethics: A European Review, Vol. 27, Issue. 1, p. 56.

    Loepp, Eric 2018. Who Else Is Running? Reference Dependence in Candidate Evaluations. American Politics Research, p. 1532673X1880388.

    Lee, Taedong An, Jungbae Sohn, Hyodong and Yoo, In Tae 2018. An Experiment of Community-Based Learning Effects on Civic Participation. Journal of Political Science Education, p. 1.

    Sisti, Davide Preti, Antonello Giambartolomei, Nazzario Amatori, Stefano and Rocchi, Marco BL 2018. Analyzing Human Random Generation: an Approach Based on the Zener Card Test. Journal of Cognitive Enhancement,

    Kanol, Direnç 2018. Knowledge of Lobbying Regulations and Attitudes Toward Politics: Findings from a Survey Experiment in Cyprus. Public Integrity, Vol. 20, Issue. 2, p. 163.

    Bouwman, Robin van Thiel, Sandra van Deemen, Ad and Rouwette, Etiënne 2018. Accountability and Coalitions: Evidence from a Negotiation Experiment. Public Administration Review, Vol. 78, Issue. 1, p. 37.

    Rothschild, Jacob E. Howat, Adam J. Shafranek, Richard M. and Busby, Ethan C. 2018. Pigeonholing Partisans: Stereotypes of Party Supporters and Partisan Polarization. Political Behavior,

    von Sikorski, Christian Knoll, Johannes and Matthes, Jörg 2018. A New Look at Celebrity Endorsements in Politics: Investigating the Impact of Scandalous Celebrity Endorsers and Politicians’ Best Responses. Media Psychology, Vol. 21, Issue. 3, p. 403.

    Jungkunz, Sebastian Helbling, Marc and Schwemmer, Carsten 2018. Xenophobia before and after the Paris 2015 attacks: Evidence from a natural experiment. Ethnicities, p. 146879681875726.

    Runst, Petrik 2018. Does Immigration Affect Demand for Redistribution? - An Experimental Design. German Economic Review, Vol. 19, Issue. 4, p. 383.

    Donovan, Kathleen M. Tillyer, Rob and Klahm, Charles F. 2018. Understanding perceptions of citizen demeanour: using an experimental design to understand the impact of encounter and observer characteristics. Policing and Society, Vol. 28, Issue. 4, p. 480.

    Daniels, Shanna R. and Jordan, Samantha L. 2018. The Effect of Paternalism on Incivility: Exploring Incivility Climate as an Important Boundary Condition. Journal of Leadership & Organizational Studies, p. 154805181879581.

    Preti, Antonio Massidda, Davide Cella, Matteo Raballo, Andrea Scanu, Rosanna Tronci, Debora Gabbrielli, Mersia Muratore, Tamara Carta, Mauro Giovanni and Petretto, Donatella Rita 2018. Factor mixture analysis of paranoia in young people. Social Psychiatry and Psychiatric Epidemiology,

    Stucki, Iris Pleger, Lyn E. and Sager, Fritz 2018. The Making of the Informed Voter: A Split-Ballot Survey on the Use of Scientific Evidence in Direct-Democratic Campaigns. Swiss Political Science Review, Vol. 24, Issue. 2, p. 115.

    Roth, M. Garrett 2018. How I learned to stop worrying and love polarization. Journal of Elections, Public Opinion and Parties, p. 1.

    Carlin, Ryan E. and Love, Gregory J. 2018. Political Competition, Partisanship and Interpersonal Trust in Electoral Democracies. British Journal of Political Science, Vol. 48, Issue. 01, p. 115.

    ×
  • Print publication year: 2011
  • Online publication date: June 2012

4 - Students as Experimental Participants

Summary

An experiment entails randomly assigning participants to various conditions or manipulations. Given common consent requirements, this means experimenters need to recruit participants who, in essence, agree to be manipulated. The ensuing practical and ethical challenges of subject recruitment have led many researchers to rely on convenience samples of college students. For political scientists who put particular emphasis on generalizability, the use of student participants often constitutes a critical, and according to some reviewers, fatal problem for experimental studies.

In this chapter, we investigate the extent to which using students as experimental participants creates problems for causal inference. First, we discuss the impact of student subjects on a study's internal and external validity. In contrast to common claims, we argue that student subjects do not intrinsically pose a problem for a study's external validity. Second, we use simulations to identify situations when student subjects are likely to constrain experimental inferences. We show that such situations are relatively limited; any convenience sample poses a problem only when the size of an experimental treatment effect depends on a characteristic on which the convenience sample has virtually no variance. Third, we briefly survey empirical evidence that provides guidance on when researchers should be particularly attuned to taking steps to ensure appropriate generalizability from student subjects. We conclude with a discussion of the practical implications of our findings.

Recommend this book

Email your librarian or administrator to recommend adding this book to your organisation's collection.

Cambridge Handbook of Experimental Political Science
  • Online ISBN: 9780511921452
  • Book DOI: https://doi.org/10.1017/CBO9780511921452
Please enter your name
Please enter a valid email address
Who would you like to send this to *
×
References
Anderson, Craig A., and Bushman, Brad J.. 1997. “External Validity of ‘Trivial’ Experiments: The Case of Laboratory Aggression.” Review of General Psychology 1: 19–41.
Ansolabehere, Stephen, and Iyengar, Shanto. 1995. Going Negative: How Political Advertisements Shrink and Polarize the Electorate. New York: Free Press.
Arceneaux, Kevin, and Johnson, Martin. 2008. “Choice, Attention, and Reception in Political Communication Research.” Paper presented at the annual meeting of the International Society for Political Psychology, Paris.
Aronson, Elliot, Brewer, Marilynn B., and Carlsmith, J. Merill. 1985. “Experimentation in Social Psychology.” In Handbook of Social Psychology. 3rd ed., eds. Lindzey, Gardner and Aronson, Elliot. New York: Random House, 441–86.
Aronson, Elliot, and Carlsmith, J. Merill. 1968. “Experimentation in Social Psychology.” In Handbook of Social Psychology. 2nd ed., eds. Lindzey, Gardner and Aronson, Elliot. Reading, MA: Addison-Wesley, 1–79.
Aronson, Elliot, Wilson, Timothy D., and Brewer, Marilynn B.. 1998. “Experimentation in Social Psychology.” In The Handbook of Social Psychology. 4th ed., eds. Gilbert, Daniel T., Fiske, Susan T., and Lindzey, Gardner. Boston: McGraw-Hill, 99–142.
Barabas, Jason, and Jerit, Jennifer. 2010. “Are Survey Experiments Externally Valid?” American Political Science Review 104: 226–42.
Baron, Reuben M., and Kenny, David A.. 1986. “The Moderator–Mediator Variable Distinction in Social Psychological Research: Conceptual, Strategic, and Statistical Considerations.” Journal of Personality and Social Psychology 51: 1173–82.
Benz, Matthias, and Meier, Stephan. 2008. “Do People Behave in Experiments as in the Field?: Evidence from Donations.” Experimental Economics 11: 268–81.
Berkowitz, Leonard, and Donnerstein, Edward. 1982. “External Validity Is More Than Skin Deep: Some Answers to Criticisms of Laboratory Experiments.” American Psychologist 37: 245–57.
Campbell, Angus, Converse, Philip, Miller, Warren, and Stokes, Donald. 1960. The American Voter. Chicago: The University of Chicago Press. [Reprinted in 1980.]
Campbell, Donald T. 1969. “Prospective: Artifact and Control.” In Artifact in Behavioral Research, eds. Rosenthal, Robert and Rosnow, Robert. New York: Academic Press, 351–82.
Chong, Dennis, and Druckman, James N.. 2007a. “Framing Public Opinion in Competitive Democracies.” American Political Science Review 101: 637–55.
Chong, Dennis, and Druckman, James N.. 2007b. “Framing Theory.” Annual Review of Political Science 10: 103–26.
Converse, Jean M., and Schuman, Howard. 1974. Conversations at Random: Survey Research as Interviewers See It. New York: Wiley.
Depositario, Dinah Pura T., Nayga, Rodolfo M.., Wu, Ximing, and Laude, Tiffany P.. 2009. “Should Students Be Used as Subjects in Experimental Auctions?” Economic Letters 102: 122–24.
Dickhaut, John W., Leslie Livingstone, J., and Watson, David J. H.. 1972. “On the Use of Surrogates in Behavioral Experimentation.” The Accounting Review 47 (Suppl): 455–71.
Druckman, James N. 2001. “On the Limits of Framing Effects.” Journal of Politics 63: 1041–66.
Druckman, James N. 2009. “Competing Frames in a Campaign.” Unpublished paper, Northwestern University.
Druckman, James N., Green, Donald P., Kuklinski, James H., and Lupia, Arthur. 2006. “The Growth and Development of Experimental Research Political Science.” American Political Science Review 100: 627–36.
Franklin, Charles H. 1991. “Efficient Estimation in Experiments.” The Political Methodologist 4: 13–15.
Friedman, Daniel, and Sunder, Shyam. 1994. Experimental Economics: A Primer for Economists. New York: Cambridge University Press.
Friedman, Milton. 1953. Essays in Positive Economics. Chicago: The University of Chicago Press.
Gaines, Brian J., Kuklinski, James H., and Quirk, Paul J.. 2007. “The Logic of the Survey Experiment Reexamined.” Political Analysis 15: 1–20.
Gerber, Alan S., Gimpel, James G., Green, Donald P., and Shaw, Daron R.. 2007. “The Influence of Television and Radio Advertising on Candidate Evaluations: Results from a Large Scale Randomized Experiment.” Unpublished paper, Yale University.
Gerber, Alan S., and Green, Donald P.. 2008. “Field Experiments and Natural Experiments.” In The Oxford Handbook of Political Methodology, eds. Box-Steffensmeier, Janet M., Brady, Henry E., and Collier, David. Oxford: Oxford University Press, 357–81.
Gordon, Michael E., Allen Slade, L., and Schmitt, Neal. 1986. “The ‘Science of the Sophomore’ Revisited: From Conjecture to Empiricism.” Academy of Management Review 11: 191–207.
Groves, Robert M., and Peytcheva, Emilia. 2008. “The Impact of Nonresponse Rates on Nonresponse Bias: A Meta-Analysis.” Public Opinion Quarterly 72: 167–89.
Guala, Francesco. 2005. The Methodology of Experimental Economics. New York: Cambridge University Press.
Henrich, Joseph, Heine, Steven J., and Norenzayan, Ara. 2009. “The Weirdest People in the World: How Representative Are Experimental Findings from American University Students? What Do We Really Know about Human Psychology?” Unpublished paper, University of British Columbia.
James, William L., and Sonner, Brenda S.. 2001. “Just Say No to Traditional Student Samples.” Journal of Advertising Research 41: 63–71.
Kam, Cindy D. 2005. “Who Toes the Party Line?: Cues, Values, and Individual Differences.” Political Behavior 27: 163–82.
Kam, Cindy D. 2007. “When Duty Calls, Do Citizens Answer?” Journal of Politics 69: 17–29.
Kam, Cindy D., and Franzese, Jr Robert J.. 2007. Modeling and Interpreting Interactive Hypotheses in Regression Analysis. Ann Arbor: University of Michigan Press.
Kam, Cindy D., Wilking, Jennifer R., and Zechmeister, Elizabeth J.. 2007. “Beyond the ‘Narrow Data Base’: Another Convenience Sample for Experimental Research.” Political Behavior 29: 415–40.
Kinder, Donald R. 1998. “Communication and Opinion.” Annual Review of Political Science 1: 167–97.
Kruglanski, Arie W. 1975. “The Human Subject in the Psychology Experiment: Fact and Artifact.” In Advances in Experimental Social Psychology, ed. Berkowitz, Leonard. New York: Academic Press, 101–47.
Lau, Richard R., and Redlawsk, David P.. 2006. How Voters Decide: Information Processing in Election Campaigns. Cambridge: Cambridge University Press.
Liyanarachchi, Gregory A. 2007. “Feasibility of Using Student Subjects in Accounting Experiments: A Review.” Pacific Accounting Review 19: 47–67.
MacDonald, Paul. 2003. “Useful Fiction or Miracle Maker: The Competing Epistemological Foundations of Rational Choice Theory.” American Political Science Review 97: 551–65.
McDermott, Rose. 2002. “Experimental Methodology in Political Science.” Political Analysis 10: 325–42.
Milgram, Stanley. 1963. “Behavioral Study of Obedience.” Journal of Abnormal and Social Psychology 67: 371–78.
Mintz, Alex, Redd, Steven B., and Vedlitz, Arnold. 2006. “Can We Generalize from Student Experiments to the Real World in Political Science, Military Affairs, and International Relations?” Journal of Conflict Resolution 50: 757–76.
Mook, Douglas G. 1983. “In Defense of External Invalidity.” American Psychologist 38: 379–87.
Morton, Rebecca B., and Williams, Kenneth C.. 2008. “Experimentation in Political Science.” In The Oxford Handbook of Political Methodology, eds. Box-Steffensmeier, Janet M., Brady, Henry E., and Collier, David. Oxford: Oxford University Press, 339–56.
Peterson, Robert A. 2001. “On the Use of College Students in Social Science Research: Insights from a Second-Order Meta-Analysis.” Journal of Consumer Research 28: 450–61.
Plott, Charles R. 1991. “Will Economics Become an Experimental Science?” Southern Economic Journal 57: 901–19.
Roth, Alvin E. 1995. “Introduction to Experimental Economics.” In The Handbook of Experimental Economics, eds. Kagel, John H. and Roth, Alvin E.. Princeton, NJ: Princeton University Press, 3–109.
Sears, David O. 1986. “College Sophomores in the Laboratory: Influence of a Narrow Data Base on Social Psychology's View of Human Nature.” Journal of Personality and Social Psychology 51: 515–30.
Shadish, William R., Cook, Thomas D., and Campbell, Donald T.. 2002. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston: Houghton Mifflin.
Simon, Herbert A. 1963. “Problems of Methodology Discussion.” American Economic Review Proceedings 53: 229–31.
Simon, Herbert A. 1979. “Rational Decision Making in Business Organizations.” American Economic Review 69: 493–513.
Slothuus, Rune. 2009. “The Political Logic of Party Cues in Opinion Formation.” Paper presented at the annual meeting of the Midwest Political Science Association, Chicago.
Sniderman, Paul M., Brody, Richard A., and Tetlock, Philip E.. 1991. Reasoning and Choice: Explorations in Political Psychology. Cambridge: Cambridge University Press.
Sniderman, Paul M., and Theriault, Sean M.. 2004. “The Structure of Political Argument and the Logic of Issue Framing.” In Studies in Public Opinion, eds. Saris, Willem E. and Sniderman, Paul M.. Princeton, NJ: Princeton University Press, 133–65.
Stevens, Charles D., and Ash, Ronald A.. 2001. “The Conscientiousness of Students in Subject Pools: Implications of ‘Laboratory’ Research.” Journal of Research in Personality 35: 91–97.
Transue, John E., Lee, Daniel J., and Aldrich, John H.. 2009. “Treatment Spillover Effects across Survey Experiments.” Political Analysis 17: 143–61.
Zaller, John. 1992. The Nature and Origins of Mass Opinion. New York: Cambridge University Press.
Zimbardo, Phillip. 1973. “A Pirandellian Prison.” New York Times Magazine, April 8.