Hostname: page-component-7d684dbfc8-rcw2t Total loading time: 0 Render date: 2023-09-30T14:03:18.636Z Has data issue: false Feature Flags: { "corePageComponentGetUserInfoFromSharedSession": true, "coreDisableEcommerce": false, "coreDisableSocialShare": false, "coreDisableEcommerceForArticlePurchase": false, "coreDisableEcommerceForBookPurchase": false, "coreDisableEcommerceForElementPurchase": false, "coreUseNewShare": true, "useRatesEcommerce": true } hasContentIssue false

Generalizability, transferability, and the practice-to-practice gap

Published online by Cambridge University Press:  10 February 2022

Joshua R. de Leeuw
Department of Cognitive Science, Vassar College, Poughkeepsie, NY12604,;
Benjamin A. Motz
Department of Psychological and Brain Sciences, Indiana University, Bloomington, IN47405,;;;;;
Emily R. Fyfe
Department of Psychological and Brain Sciences, Indiana University, Bloomington, IN47405,;;;;;
Paulo F. Carvalho
Human-Computer Interaction Institute, Carnegie Mellon University, Pittsburgh, PA15213, USA.;
Robert L. Goldstone
Department of Psychological and Brain Sciences, Indiana University, Bloomington, IN47405,;;;;;


Emphasizing the predictive success and practical utility of psychological science is an admirable goal but it will require a substantive shift in how we design research. Applied research often assumes that findings are transferable to all practices, insensitive to variation between implementations. We describe efforts to quantify and close this practice-to-practice gap in education research.

Open Peer Commentary
Copyright © The Author(s), 2022. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)



Co-first authors.


Agarwal, P. K., Roediger, H. L., McDaniel, M. A., & McDermott, K. B. (2020). How to use retrieval practice to improve learning. St. Louis, MO: Washington University in St. Louis. Retrieved from Scholar
Baribault, B., Donkin, C., Little, D. R., Trueblood, J. S., Oravecz, Z., van Ravenzwaaij, D., … Vandekerckhove, J. (2018). Metastudies for robust tests of theory. Proceedings of the National Academy of Sciences, 115(11), 26072612.CrossRefGoogle ScholarPubMed
Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14(1), 458. ScholarPubMed
Fyfe, E., de Leeuw, J. R., Carvalho, P. F., Goldstone, R., Sherman, J., Admiraal, D., … Motz, B. (2021). ManyClasses 1: Assessing the generalizable effect of immediate versus delayed feedback across many college classes. Advances in Methods and Practices in Psychological Science, 4(3), 124. Scholar
Gurung, R. A., & Burns, K. (2019). Putting evidence-based claims to the test: A multi-site classroom study of retrieval practice and spaced practice. Applied Cognitive Psychology, 33(5), 732743. Scholar
Hawe, P., Shiell, A., & Riley, T. (2009). Theorising interventions as events in systems. American Journal of Community Psychology, 43(3–4), 267276. ScholarPubMed
Koedinger, K. R., Booth, J. L., & Klahr, D. (2013). Instructional complexity and the science to constrain it. Science, 342(6161), 935937. Scholar
Lincoln, Y. S., & Guba, E. G. (1986). But is it rigorous? Trustworthiness and Authenticity in Naturalistic evaluation. New Directions for Program Evaluation, 1986(30), 7384.CrossRefGoogle Scholar
Makel, M. C., & Plucker, J. A. (2014). Facts are more important than novelty: Replication in the education sciences. Educational Researcher, 43(6), 304316. Scholar
Moreira, B. F. T., Pinto, T. S. S., Starling, D. S. V., & Jaeger, A. (2019). Retrieval practice in classroom settings: A review of applied research. Frontiers in Education, 4(5), 116. Scholar
Motz, B. A., Carvalho, P. F., de Leeuw, J. R., & Goldstone, R. L. (2018). Embedding experiments: Staking causal inference in authentic educational contexts. Journal of Learning Analytics, 5(2), 4759. Scholar
Schneider, M. (2018). A more systematic approach to replicating research. IES Director's Blog, Institute of Education Sciences. Retrieved from Scholar
Yang, C., Luo, L., Vadillo, M. A., Yu, R., & Shanks, D. R. (2021). Testing (quizzing) boosts classroom learning: A systematic and meta-analytic review. Psychological Bulletin, 147(4), 399435. ScholarPubMed