Skip to main content Accessibility help
×
Home
Hostname: page-component-684899dbb8-gblv7 Total loading time: 0.607 Render date: 2022-05-20T11:05:20.951Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "useRatesEcommerce": false, "useNewApi": true }

Assessing the Correspondence between Experimental Results Obtained in the Lab and Field: A Review of Recent Social Science Research*

Published online by Cambridge University Press:  04 August 2014

Abstract

A small but growing social science literature examines the correspondence between experimental results obtained in lab and field settings. This article reviews this literature and reanalyzes a set of recent experiments carried out in parallel in both the lab and field. Using a standardized format that calls attention to both the experimental estimates and the statistical uncertainty surrounding them, the study analyzes the overall pattern of lab-field correspondence, which is found to be quite strong (Spearman's ρ = 0.73). Recognizing that this correlation may be distorted by the ad hoc manner in which lab-field comparisons are constructed (as well as the selective manner in which results are reported and published), the article concludes by suggesting directions for future research, stressing in particular the need for more systematic investigation of treatment effect heterogeneity.

Type
Original Articles
Copyright
Copyright © The European Political Science Association 2014 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

*Alexander Coppock is a PhD student, Department of Political Science, Columbia University, 420 West 118th Street, Suite 710, New York, NY 10025 (ac3242@columbia.edu). Donald P. Green is Professor of Political Science, Columbia University, 420 West 118th Street, Suite 710, New York, NY 10025 (dpg2110@columbia.edu). We are grateful to Oliver Armantier, Peter Aronow, Lindsay Dolan, Uri Gneezy, John List, Jennifer Jerit, Andrej Tusicisny and two anonymous reviewers for helpful comments, Glenn Harrison and Nicholas Valentino for graciously providing replication data, and to Johannes Abeler and Ido Erev for providing supplementary information. To view supplementary material for this article, please visit http://dx.doi.org/10.1017/psrm.2014.10

References

Abeler, J., Marklein, F.. 2013. ‘Fungibility, Labels, and Consumption’. Unpublished manuscript.Google Scholar
Arceneaux, K.Nickerson, D.W.. 2009. ‘Who Is Mobilized to Vote? A ReAnalysis of 11 Field Experiments’. American Journal of Political Science 53(1):116.CrossRefGoogle Scholar
Armantier, O.Boly, A.. 2013. ‘Comparing Corruption in the Laboratory and in the Field in Burkina Faso and in Canada’. The Economic Journal 123(573):11681187.CrossRefGoogle Scholar
Ashraf, N., Berry, J.Shapiro, J.M.. 2010. ‘Can Higher Prices Stimulate Product Use? Evidence from a Field Experiment in Zambia’. American Economic Review 100(December):23832413.CrossRefGoogle Scholar
Baird, S., Ferreira, F., Özler, B., Woolcock, M.. 2013. ‘Relative Effectiveness of Conditional and Unconditional Cash Transfers for Schooling Outcomes in Developing Countries: A Systematic Review’. Technical report, Campbell Systematic Reviews.Google Scholar
Banerjee, A., Kumar, S., Pande, R., Su, F.. 2010. ‘Do Informed Voters Make Better Choices? Experimental Evidence from Urban India’. Unpublished manuscript.Google Scholar
Benz, M.Meier, S.. 2008. Do People Behave in Experiments as in the Field? Evidence from Donations. Experimental Economics 11(3):268281.CrossRefGoogle Scholar
Bond, R.Smith, P.B.. 1996. ‘Culture and Conformity: A Meta-analysis of Studies Using Asch's (1952b, 1956) Line Judgment Task’. Psychological Bulletin 119(1):111137.CrossRefGoogle Scholar
Bornstein, G., Erev, I.Rosen, O.. 1990. ‘Intergroup Competition as a Structural Solution to Social Dilemmas’. Social Behaviour 5(4):247260.Google Scholar
Camerer, C.F. forthcoming. ‘The Promise and Success of Lab-Field Generalizability in Experimental Economics: A Critical Reply to Levitt and List’. In G. Frechette and A. Schotter (eds), Methods of Modern Experimental Economics. Oxford: Oxford University Press.Google Scholar
Castro, L., Scartascini, C.. 2013. ‘Tax Compliance and Enforcement in the Pampas: Evidence from a Field Experiment’. Washington, DC: Inter-American Development Bank, 2013.Google Scholar
Charness, G., Schram, A.. 2013. ‘Social and Moral Norms in Allocation Choices in the Laboratory’. Unpublished manuscript.Google Scholar
Chong, A., De La O, A.L., Karlan, D.S., Wantchekon, L.. 2010. ‘Information Dissemination and Local Governments Electoral Returns, Evidence from a Field Experiment in Mexico’. Unpublished manuscript.Google Scholar
Cohen, J.Dupas, P.. 2010. ‘Free Distribution or Cost-sharing? Evidence from a Randomized Malaria Prevention Experiment’. Quarterly Journal of Economics 125(1):145.CrossRefGoogle Scholar
Cook, T.D.Campbell, D.T.. 1979. Quasi-Experimentation: Design & Analysis Issues for Field Settings. Boston, MA: Houghton Mifflin.Google Scholar
Engel, C. 2011. ‘Dictator Games: a Meta Study’. Experimental Economics 14(4):583610.CrossRefGoogle Scholar
Erev, I., Bornstein, G.Galili, R.. 1993. ‘Constructive Intergroup Competition as a Solution to the Free Rider Problem: A Field Experiment’. Journal of Experimental Social Psychology 29:463478.CrossRefGoogle Scholar
Falk, A.Heckman, J.J.. 2009. ‘Lab Experiments are a Major Source of Knowledge in the Social Sciences’. Science 326(5952):535538.CrossRefGoogle Scholar
Fellner, G., Sausgruber, R.Traxler, C.. 2013. ‘Testing Enforcement Strategies in the Field: Threat, Moral Appeal and Social Information’. Journal of the European Economic Association 11(3):634660.CrossRefGoogle Scholar
Gerber, A.S. 2004. ‘Does Campaign Spending Work?: Field Experiments Provide Evidence and Suggest New Theory’. American Behavioral Scientist 47(5):541574.CrossRefGoogle Scholar
Gerber, A.S.Green, D.P.. 2012. Field Experiments: Design, Analysis, and Interpretation. New York: W.W. Norton.Google Scholar
Gerber, A.S., Green, D.P.Larimer, C.W.. 2008. ‘Social Pressure and Voter Turnout: Evidence from a Large-Scale Field Experiment’. American Political Science Review 102(1):3348.CrossRefGoogle Scholar
Glennerster, R.Takavarasha, K.. 2013. Running Randomized Evaluations: A Practical Guide. Princeton, NJ: Princeton University Press.Google Scholar
Gneezy, U., Haruvy, E.Yafe, H.. 2004. ‘The Inefficiency of Splitting the Bill’. The Economic Journal 114:265280.CrossRefGoogle Scholar
Gneezy, U.List, J.A.. 2006. ‘Putting Behavioral Economics to Work: Testing for Gift Exchange in Labor Markets Using Field Experiments’. Econometrica 74(5):13651384.CrossRefGoogle Scholar
Gneezy, U., Niederle, M.Rustichini, A.. 2003. ‘Performance in Competitive Environments: Gender Differences’. The Quarterly Journal of Economics 118(3):10491074.CrossRefGoogle Scholar
Gneezy, U.Rustichini, A.. 2000. ‘Pay Enough or Don't Pay at All’. The Quarterly Journal of Economics 115(3):791810.CrossRefGoogle Scholar
Gneezy, U.Rustichini, A.. 2004. ‘Gender and Competition at a Young Age’. American Economic Review 94(2):377381.CrossRefGoogle Scholar
Green, D.P., Tusicisny, A. 2013. ‘Statistical Analysis of Results from Laboratory Studies in Experimental Economics: A Critique of Current Practice’. Unpublished manuscript.Google Scholar
Großer, J.Schram, A. 2010. ‘Public Opinion Polls, Voter Turnout, and Welfare: An Experimental Study’. American Journal of Political Science 54(3):700717.CrossRefGoogle Scholar
Habyarimana, J., Humphreys, M., Posner, D.N.Weinstein, J.M.. 2009. Coethnicity: Diversity and the Dilemmas of Collective Action. New York: Russell Sage Foundation.Google Scholar
Harder, V.S., Stuart, E.A.Anthony, J.C.. 2010. ‘Propensity Score Techniques and the Assessment of Measured Covariate Balance to Test Causal Associations in Psychological Research’. Psychological Methods 15(3):234249.CrossRefGoogle Scholar
Harrison, G.W.List, J.A.. 2004. ‘Field Experiments’. Journal of Economic Literature 42(4):10091055.CrossRefGoogle Scholar
Harrison, G.W.List, J.A.. 2008. ‘Naturally Occurring Markets and Exogenous Laboratory Experiments: A Case Study of the Winners Curse’. The Economic Journal 118(528):822843.CrossRefGoogle Scholar
Henrich, J., Boyd, R., Bowles, S., Camerer, C.F., Fehr, E., Gintis, H.McElreath, R.. 2001. ‘In Search of Homo Economicus: Behavioral Experiments in 15 Small-Scale Societies’. American Economic Review 91(2):7378.CrossRefGoogle Scholar
Henrich, J., Heine, S.J.Norenzayan, A.. 2010. ‘The Weirdest People in the World?’ Behavioral and Brain Sciences 33:6183.CrossRefGoogle Scholar
Hotz, V.J., Imbens, G.W.Mortimer, J.H.. 2005. ‘Predicting the Efficacy of Future Training Programs using Past Experiences at other Locations’. Journal of Econometrics 125(1–2):241270.CrossRefGoogle Scholar
Humphreys, M., Sanchez de la Sierra, R.van der Windt, P.. 2013. ‘Fishing, Commitment, and Communication: A Proposal for Comprehensive Nonbinding Research Registration’. Political Analysis 21(1):120.CrossRefGoogle Scholar
Humphreys, M., Weinstein, J. M.. 2010. ‘Policing Politicians: Citizen Empowerment and Political Accountability in Uganda’. Unpublished manuscript.Google Scholar
Jerit, J., Barabas, J.Clifford, S.. 2013. ‘Comparing Contemporaneous Laboratory and Field Experiments on Media Effects’. Public Opinion Quarterly 77(1):256282.CrossRefGoogle Scholar
Johnson, N.D.Mislin, A.A.. 2011. ‘Trust Games: A Meta-analysis’. Journal of Economic Psychology 32(5):865889.CrossRefGoogle Scholar
Kessler, J.Vesterlund, L.. forthcoming. ‘The External Validity of Laboratory Experiments: Qualitative rather than Quantitative Effects’. In G. Frechette and A. Schotter (eds), Methods of Modern Experimental Economics. Oxford: Oxford University Press.Google Scholar
King, E.B.Ahmad, A.S.. 2010. An Experimental Field Study Of Interpersonal Discrimination Toward Muslim Job Applicants. Personnel Psychology 63(4):881906.CrossRefGoogle Scholar
Lalancette, M.-F.Standing, L.. 1990. Asch Fails Again. Social Behavior and Personality 18(1):712.CrossRefGoogle Scholar
Levitt, S.D.List, J.A. 2007. ‘Viewpoint: On the Generalizability of Lab Behaviour to the Field’. Canadian Journal of Economics 40(2):347370.CrossRefGoogle Scholar
List, J.A. 2006. ‘The Behavioralist Meets the Market: Measuring Social Preferences and Reputation Effects in Actual Transactions’. Journal of Political Economy 114(1):137.CrossRefGoogle Scholar
Milgram, S. 1963. ‘Behavioral Study of Obedience’. The Journal of Abnormal and Social Psychology 67(4):371378.CrossRefGoogle Scholar
Morton, R.B.Williams, K.C.. 2010. Experimental Political Science and the Study of Causality: From Nature to the Lab. New York: Cambridge University Press.CrossRefGoogle Scholar
Oosterbeek, H., Sloof, R.van de Kuilen, G.. 2004. ‘Cultural Differences in Ultimatum Game Experiments: Evidence from a Meta-Analysis’. Experimental Economics 7(2):171188.CrossRefGoogle Scholar
Paluck, Elizabeth Levy. 2009. ‘Reducing Intergroup Prejudice and Conflict Using the Media: A Field Experiment in Rwanda’. Journal of Personality and Social Psychology 96(3):574587.CrossRefGoogle Scholar
Rondeau, D.List, J.A.. 2008. ‘Matching and Challenge Gifts to Charity: Evidence from Laboratory and Natural Field Experiments’. Experimental Economics 11(3):253267.CrossRefGoogle Scholar
Rosenthal, R. 1979. ‘The File Drawer Problem and Tolerance for Null Results’. Psychological Bulletin 86(3):638641.CrossRefGoogle Scholar
Schultz, W.P., Khazian, A.M.Zaleski, A.C.. 2008. ‘Using Normative Social Influence to Promote Conservation Among Hotel Guests’. Social Influence 3(1):423.CrossRefGoogle Scholar
Sears, D.O. 1986. ‘College Sophomores in the Laboratory: Influences of a Narrow Data Base on Social Psychology's View of Human Nature’. Journal of Personality and Social Psychology 51(3):515530.CrossRefGoogle Scholar
Shadish, W.R., Cook, T.D.Campbell, D.T.. 2002. Experimental and Quasi-experimental Designs for Generalized Causal Inference. Boston, MA: Houghton, Mifflin and Company.Google Scholar
Shang, J., Reed, A.Croson, R.. 2008. ‘Identity Congruency Effects on Donations’. Journal of Marketing Research 45:351361.CrossRefGoogle Scholar
Stewart, R., Rooyen, C.V., Korth, M., Chereni, A., Rebelo Da Silva, N.de Wet, T.. 2012. Do Micro-credit, Micro-savings and Micro-leasing Serve as Effective Financial Inclusion Interventions Enabling Poor People, and Especially Women, to Engage in Meaningful Economic Opportunities in Low- and Middle-income Countries. London: EPPI-Centre, University of London.Google Scholar
Student. 1908. ‘Probable Error of a Correlation Coefficient’. Biometrika 6(2):302310.CrossRefGoogle Scholar
Valentino, N.A., Traugott, M.W.Hutchings, V.L.. 2002. ‘Group Cues and Ideological Constraint: A Replication of Political Advertising Effects Studies in the Lab and in the Field’. Political Communication 19:2948.CrossRefGoogle Scholar
Supplementary material: Link

Coppock and Green Datasets

Link
Supplementary material: Link

Coppock & Green Dataset

Link
Supplementary material: Link

Coppock and Green code and data to replicate results

Link
Supplementary material: File

Coppock and Green Supplementary Material

Appendix

Download Coppock and Green Supplementary Material(File)
File 122 KB
18
Cited by

Save article to Kindle

To save this article to your Kindle, first ensure coreplatform@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Assessing the Correspondence between Experimental Results Obtained in the Lab and Field: A Review of Recent Social Science Research*
Available formats
×

Save article to Dropbox

To save this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Dropbox account. Find out more about saving content to Dropbox.

Assessing the Correspondence between Experimental Results Obtained in the Lab and Field: A Review of Recent Social Science Research*
Available formats
×

Save article to Google Drive

To save this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Google Drive account. Find out more about saving content to Google Drive.

Assessing the Correspondence between Experimental Results Obtained in the Lab and Field: A Review of Recent Social Science Research*
Available formats
×
×

Reply to: Submit a response

Please enter your response.

Your details

Please enter a valid email address.

Conflicting interests

Do you have any conflicting interests? *