Hostname: page-component-848d4c4894-x5gtn Total loading time: 0 Render date: 2024-05-17T04:27:08.665Z Has data issue: false hasContentIssue false

Citizen science can help to alleviate the generalizability crisis

Published online by Cambridge University Press:  10 February 2022

Courtney B. Hilton
Department of Psychology, Harvard University, Cambridge, MA02138,
Samuel A. Mehr
Department of Psychology, Harvard University, Cambridge, MA02138, Data Science Initiative, Harvard University, Cambridge, MA02138,; School of Psychology, Victoria University of Wellington, Kelburn Parade, Wellington6012, New Zealand


Improving generalization in psychology will require more expansive data collection to fuel more expansive statistical models, beyond the scale of traditional lab research. We argue that citizen science is uniquely positioned to scale up data collection and, that in spite of certain limitations, can help to alleviate the generalizability crisis.

Open Peer Commentary
Copyright © The Author(s), 2022. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)


Awad, E., Dsouza, S., Kim, R., Schulz, J., Henrich, J., Sharff, A., … Rahwan, I. (2018). The moral machine experiment. Nature, 563(7729), 5964.CrossRefGoogle ScholarPubMed
Baribault, B., Donkin, C., Little, D. R., Trueblood, J. S., Oravecz, Z., van Ravenzwaaij, D., … Vandekerckhove, J. (2018). Metastudies for robust tests of theory. Proceedings of the National Academy of Sciences, 115(11), 26072612.CrossRefGoogle ScholarPubMed
Cooper, S., Khatib, F., Treuille, A., Barbero, J., Lee, J., Beenen, M., … Players, F. (2010). Predicting protein structures with a multiplayer online game. Nature, 466(7307), 756760.CrossRefGoogle ScholarPubMed
de Leeuw, J. R. (2015). jsPsych: A JavaScript library for creating behavioral experiments in a Web browser. Behavior Research Methods, 47(1), 112.CrossRefGoogle Scholar
Hartshorne, J. K., de Leeuw, J., Goodman, N., Jennings, M., & O'Donnell, T. J. (2019). A thousand studies for the price of one: Accelerating psychological science with Pushkin. Behavior Research Methods, 51(4), 122.CrossRefGoogle Scholar
Hilton, C., Crowley de-Thierry, L., Yan, R., Martin, A., & Mehr, S. (2021). Children infer the behavioral contexts of unfamiliar songs. PsyArXiv. doi: 10.31234/ Scholar
Hilton, C. B., Moser, C. J., Bertolo, M., Lee-Rubin, H., Amir, D., Bainbridge, C. M., … Mehr, S. A. (2021). Acoustic regularities in infant-directed vocalizations across cultures. bioRxiv. doi: 10.1101/2020.04.09.032995Google Scholar
Lourenco, S. F., & Tasimi, A. (2020). No participant left behind: Conducting science during COVID-19. Trends in Cognitive Sciences, 24(8), 583584.CrossRefGoogle ScholarPubMed
ManyBabies Consortium. (2020). Quantifying sources of variability in infancy research using the infant-directed-speech preference. Advances in Methods and Practices in Psychological Science, 3, 2452.CrossRefGoogle Scholar
Mehr, S. A., Singh, M., Knox, D., Ketter, D., Pickens-Jones, D., Atwood, S., … Glowacki, L. (2019). Universality and diversity in human song. Science, 366(6468), eaax0868.CrossRefGoogle ScholarPubMed
Mehr, S. A., Singh, M., York, H., Glowacki, L., & Krasnow, M. M. (2018). Form and function in human song. Current Biology, 28(3), 356368.e5.CrossRefGoogle ScholarPubMed
Peirce, J., Gray, J. R., Simpson, S., MacAskill, M., Höchenberger, R., Sogo, H., … Lindeløv, J. K. (2019). PsychoPy2: Experiments in behavior made easy. Behavior Research Methods, 51(1), 195203.CrossRefGoogle ScholarPubMed
Scott, K., & Schulz, L. (2017). Lookit (Part 1): A new online platform for developmental research. Open Mind, 1(1), 414.CrossRefGoogle Scholar
Sheskin, M., Scott, K., Mills, C. M., Bergelson, E., Bonawitz, E., Spelke, E. S., … Schulz, L. (2020). Online developmental science to foster innovation, access, and impact. Trends in Cognitive Sciences, 24(9), 675678.CrossRefGoogle ScholarPubMed
Smaldino, P. E., & McElreath, R. (2016). The natural selection of bad science. Royal Society Open Science, 3(9), 160384.CrossRefGoogle ScholarPubMed
Stuart, E. A. (2010). Matching methods for causal inference: A review and a look forward. Statistical Science, 25(1), 121.CrossRefGoogle Scholar
Thompson, W. F., Schellenberg, E. G., & Husain, G. (2001). Arousal, mood, and the Mozart effect. Psychological Science, 12(3), 248251.CrossRefGoogle ScholarPubMed
Way, S. F., Garcia-Gathright, J., & Cramerr, H. (2020). Local trends in global music streaming. Proceedings of the Fourteenth International AAAI Conference on Web and Social Media, 10.Google Scholar
Yarkoni, T., & Westfall, J. (2017). Choosing prediction over explanation in psychology: Lessons from machine learning. Perspectives on Psychological Science, 12(6), 11001122.CrossRefGoogle ScholarPubMed