Skip to main content Accessibility help
×
Home

Scaling Up Intervention Studies to Investigate Real-Life Foreign Language Learning in School

  • Detmar Meurers (a1), Kordula De Kuthy (a1), Florian Nuxoll (a1), Björn Rudzewitz (a1) and Ramon Ziai (a1)...

Abstract

Intervention studies typically target a focused aspect of language learning that is studied over a relatively short time frame for a relatively small number of participants in a controlled setting. While for many research questions, this is effective, it can also limit the ecological validity and relevance of the results for real-life language learning. In educational science, large-scale randomized controlled field trials (RCTs) are seen as the gold standard method for addressing this challenge—yet they require intervention to scale to hundreds of learners in their varied, authentic contexts.

We discuss the use of technology in support of large-scale interventions that are fully integrated in regular classes in secondary school. As an experimentation platform, we developed a web-based workbook to replace a printed workbook widely used in German schools. The web-based FeedBook provides immediate scaffolded feedback to students on form and meaning for various exercise types, covering the full range of constructions in the seventh-grade English curriculum.

Following the conceptual discussion, we report on the first results of an ongoing, yearlong RCT. The results confirm the effectiveness of the scaffolded feedback, and the approach makes students and learning process variables accessible for the analysis of learning in a real-world context.

  • View HTML
    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Scaling Up Intervention Studies to Investigate Real-Life Foreign Language Learning in School
      Available formats
      ×

      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      Scaling Up Intervention Studies to Investigate Real-Life Foreign Language Learning in School
      Available formats
      ×

      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      Scaling Up Intervention Studies to Investigate Real-Life Foreign Language Learning in School
      Available formats
      ×

Copyright

This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.

Corresponding author

*Corresponding author. E-mail: dm@sfs.uni-tuebingen.de

References

Hide All
Aljaafreh, A. L., & Lantolf, J. P. (1994). Negative feedback as regulation and second language learning in the zone of proximal development. The Modern Language Journal, 78(4), 465483.
Amaral, L., & Meurers, D. (2011). On using intelligent computer-assisted language learning in real-life foreign language teaching and learning. ReCALL, 23(1), 424.
Andringa, S., & Godfroid, A. (2019). SLA for all? Reproducing SLA research in non-academic samples. OSF Project, created January 25, 2019. Retrieved from https://osf.io/mp47b/
Bates, D., Maechler, M., & Bolker, B. (2012). lme4: Linear mixed-effects models using S4 classes. R Package, R Foundation for Statistical Computing. Vienna, Austria.
Bitchener, J., & Storch, N. (2016). Written corrective feedback for L2 development (Vol. 96). Bristol, UK: Multilingual Matters.
Chinkina, M., & Meurers, D. (2016). Linguistically-aware information retrieval: Providing input enrichment for second language learners. In Proceedings of the 11th workshop on innovative use of NLP for building educational applications (BEA) (pp. 188198). San Diego, CA: ACL. Retrieved from http://aclweb.org/anthology/W16-0521.pdf
Chinkina, M., & Meurers, D. (2017). Question generation for language learning: From ensuring texts are read to supporting learning. In Proceedings of the 12th workshop on innovative use of NLP for building educational applications (BEA) (pp. 334344). Copenhagen, Denmark. Retrieved from http://aclweb.org/anthology/W17-5038.pdf
Corbett, A. T., Koedinger, K. R., & Anderson, J. R. (1997). Intelligent tutoring systems. In Helander, M. G., Landauer, T. K., & Prabhu, P. V. (Eds.), Handbook of human computer interaction (pp. 849874). Amsterdam, The Netherlands: North-Holland.
Dede, C. (2005). Scaling up: Evolving innovations beyond ideal settings to challenging contexts of practice. In Sawyer, R. (Ed.), Cambridge handbook of the learning sciences (pp. 551566). Cambridge, UK: Cambridge University Press.
Dörnyei, Z., & Katona, L. (1992). Validation of the C-test amongst Hungarian EFL learners. Language Testing, 9(2), 187206.
Ellis, R. (2003). Task-based language learning and teaching. Oxford, UK: Oxford University Press.
Ellis, R. (2016). Focus on form: A critical review. Language Teaching Research, 20(3), 405428. doi: 10.1177/1362168816628627
Ferris, D. R. (2004). The “grammar correction” debate in L2 writing: Where are we, and where do we go from here? (and what do we do in the meantime …?). Journal of Second Language Writing, 13(1), 4962. doi: 10.1016/j.jslw.2004.04.005
Ferris, D. R. (2010). Second language writing research and written corrective feedback in SLA: Intersections and practical applications. Studies in Second Language Acquisition, 32(2), 181201.
Finn, B., & Metcalfe, J. (2010). Scaffolding feedback to maximize long-term error correction. Memory & Cognition, 38(7), 951961. doi: 10.3758/MC.38.7.951
Gaspard, H., Häfner, I., Parrisius, C., Trautwein, U., & Nagengast, B. (2017). Assessing task values in five subjects during secondary school: Measurement structure and mean level differences across grade level, gender, and academic subject. Contemporary Educational Psychology, 48, 6784. doi: 10.1016/j.cedpsych.2016.09.003
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81112. doi: 10.3102/003465430298487
Hedges, L. V., & Schauer, J. (2018). Randomised trials in education in the USA. Educational Research, 60(3), 265275.
Heift, T. (2010). Developing an intelligent language tutor. CALICO Journal, 27(3), 443459.
Heift, T., & Hegelheimer, V. (2017). Computer-assisted corrective feedback and language learning. In Nassaji, H. & Kartchava, E. (Eds.), Corrective feedback in second language teaching and learning (pp. 5165). New York, NY: Routledge.
Heift, T., & Schulze, M. (2007). Errors and intelligence in computer-assisted language learning: Parsers and pedagogues. New York, NY: Routledge.
Hicks, K. L., Foster, J. L., & Engle, R. W. (2016). Measuring working memory capacity on the web with the Online Working Memory Lab (the OWL). Journal of Applied Research in Memory and Cognition, 5, 478489. doi: 10.1016/j.jarmac.2016.07.010
Kulik, J. A., & Fletcher, J. (2016). Effectiveness of intelligent tutoring systems: A meta-analytic review. Review of Educational Research, 86(1), 4278.
Kultusministerium. (2016). Englisch als erste Fremdsprache [English as a first foreign language]. Bildungsplan des Gymnasiums 2016 [State curriculum for academic track schools 2016]. Retrieved from http://www.bildungsplaene-bw.de/,Lde/LS/BP2016BW/ALLG/GYM/E1 (Ministerium für Kultus, Jugend und Sport, Baden Württemberg)
Lang, C., Siemens, G., Wise, A., & Gašević, D. (Eds.). (2017). The handbook of learning analytics. Society for Learning Analytics Research. Retrieved from https://solaresearch.org/hla-17
Li, S. (2017). Cognitive differences and ISLA. In Loewen, S. & Sato, M. (Eds.), The Routledge handbook of instructed second language acquisition (pp. 396417). New York, NY: Routledge.
Loewen, S. (2018). Instructed second language acquisition. In Phakiti, A., De Costa, P., Plonsky, L., & Starfield, S. (Eds.), The Palgrave handbook of applied linguistics research methodology (pp. 663680). London, UK: Springer.
Loewen, S., & Sato, M. (Eds.). (2017). The Routledge handbook of instructed second language acquisition. New York, NY: Routledge.
Long, M. H., & Robinson, P. (1998). Focus on form: Theory, research, and practice. In Doughty, C. & Williams, J. (Eds.), Focus on form in classroom second language acquisition (pp. 1541). Cambridge, UK: Cambridge University Press.
Lynch, C., Ashley, K., Aleven, V., & Pinkwart, N. (2006). Defining ill-defined domains; a literature survey. In Proceedings of the workshop on intelligent tutoring systems for ill-defined domains, held during the 8th international conference on intelligent tutoring systems (pp. 110). Jhongli, Taiwan. Retrieved from http://people.cs.pitt.edu/~collinl/Papers/Ill-DefinedProceedings.pdf#page=7
Lyster, R., & Saito, K. (2010). Oral feedback in classroom SLA: A meta-analysis. Studies in Second Language Acquisition, 32(2), 265302.
Mackey, A. (2012). Input, interaction and corrective feedback in L2 learning. Oxford, UK: Oxford University Press.
Mackey, A. (2017). Classroom-based research. In Loewen, S. & Sato, M. (Eds.), The Routledge handbook of instructed second language acquisition (pp. 541561). New York, NY: Routledge.
MacWhinney, B. (2017). A shared platform for studying second language acquisition. Language Learning, 67(S1), 254275.
McDonald, S.-K., Keesler, V. A., Kauffman, N. J., & Schneider, B. (2006). Scaling-up exemplary interventions. Educational Researcher, 35(3), 1524.
Meurers, D., & Dickinson, M. (2017). Evidence and interpretation in language learning research: Opportunities for collaboration with computational linguistics. Language Learning, 67(S1), 6695.
Meurers, D., Ziai, R., Ott, N., & Bailey, S. (2011). Integrating parallel analysis modules to evaluate the meaning of answers to reading comprehension questions. IJCEELL. Special Issue on Automatic Free-text Evaluation, 21(4), 355369. Retrieved from http://purl.org/dm/papers/Meurers.Ziai.ea-11.pdf
Nagata, N. (2009). Robo-Sensei's NLP-based error detection and feedback generation. CALICO Journal, 26(3), 562579. Retrieved from http://purl.org/calico/nagata09.pdf
Nagata, N. (2010). Some design issues for an online Japanese textbook. CALICO Journal, 27(3), 460476.
Nassaji, H., & Kartchava, E. (2017). Corrective feedback in second language teaching and learning: Research, theory, applications, implications. New York, NY: Taylor & Francis.
Nye, B., Konstantopoulos, S., & Hedges, L. V. (2004). How large are teacher effects? Educational Evaluation and Policy Analysis, 26(3), 237257.
Nye, B. D., Graesser, A. C., & Hu, X. (2014). Autotutor and family: A review of 17 years of natural language tutoring. International Journal of Artificial Intelligence in Education, 24(4), 427469.
Pallotti, G. (2007). An operational definition of the emergence criterion. Applied Linguistics, 28(3), 361382. doi: 10.1093/applin/amm018
Pienemann, M. (2012). Processability theory and teachability. The Encyclopedia of Applied Linguistics. doi: 10.1002/9781405198431.wbeal0958
Plonsky, L., & Ziegler, N. (2016). The CALL-SLA interface: Insights from a second-order synthesis. Language Learning & Technology, 20(2), 1737.
Quixal, M., & Meurers, D. (2016). How can writing tasks be characterized in a way serving pedagogical goals and automatic analysis needs? CALICO Journal, 33, 1948. Retrieved from http://purl.org/dm/papers/Quixal.Meurers-16.html
Rudzewitz, B., Ziai, R., De Kuthy, K., & Meurers, D. (2017). Developing a web-based workbook for English supporting the interaction of students and teachers. In Proceedings of the Joint 6th Workshop on NLP for Computer Assisted Language Learning and 2nd Workshop on NLP for Research on Language Acquisition. Retrieved from http://aclweb.org/anthology/W17-0305.pdf
Rudzewitz, B., Ziai, R., De Kuthy, K., Möller, V., Nuxoll, F., & Meurers, D. (2018). Generating feedback for English foreign language exercises. In Proceedings of the 13th workshop on innovative use of NLP for building educational applications (BEA) (pp. 127136). ACL. Retrieved from http://aclweb.org/anthology/W18-0513.pdf
Ruiz Hernández, S. E. (2018). Individual differences and instructed second language acquisition: Insights from intelligent computer assisted language learning (Doctoral dissertation). Eberhard Karls Universität Tübingen. Retrieved from http://hdl.handle.net/10900/86575
Russell, J., & Spada, N. (2006). The effectiveness of corrective feedback for the acquisition of L2 grammar. In Norris, J. M. & Ortega, L. (Eds.), Synthesizing research on language learning and teaching (Vol. 13, pp. 133164). Amsterdam, The Netherlands: John Benjamins.
Sato, M., & Loewen, S. (2019). Methodological strengths, challenges, and joys of classroom-based quasi-experimental research. In Keyser, R. M. D. & Botana, G. P. (Eds.), Doing SLA research with implications for the classroom: Reconciling methodological demands and pedagogical applicability (pp. 3154). Amsterdam, The Netherlands: John Benjamins.
Sheen, Y. (2011). Corrective feedback, individual differences and second language. New York, NY: Springer.
Truscott, J. (1996). The case against grammar correction in L2 writing classes. Language Learning, 46(2), 327369.
Vaughn, S., Martinez, L. R., Wanzek, J., Roberts, G., Swanson, E., & Fall, A.-M. (2017). Improving content knowledge and comprehension for English language learners: Findings from a randomized control trial. Journal of Educational Psychology, 109(1), 22.
Ziai, R., & Meurers, D. (2018). Automatic focus annotation: Bringing formal pragmatics alive in analyzing the Information Structure of authentic data. In Proceedings of the 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (pp. 117128). New Orleans, LA: ACL. Retrieved from http://aclweb.org/anthology/N18-1011.pdf
Ziai, R., Rudzewitz, B., De Kuthy, K., Nuxoll, F., & Meurers, D. (2018). Feedback strategies for form and meaning in a real-life language tutoring system. In Proceedings of the 7th workshop on Natural Language Processing for Computer-Assisted Language Learning (NLP4CALL) (pp. 9198). ACL. Retrieved from http://aclweb.org/anthology/W18-7110
Ziegler, N., & Mackey, A. (2017). Interactional feedback in synchronous computer-mediated communication. In Corrective feedback in second language teaching and learning: Research, theory, applications, implications (Vol. 66, pp. 8094). New York, NY: Taylor & Francis.
Ziegler, N., Meurers, D., Rebuschat, P., Ruiz, S., Moreno Vega, J. L., Chinkina, M., … Grey, S. (2017). Interdisciplinary research at the intersection of CALL, NLP, and SLA: Methodological implications from an input enhancement project. Language Learning, 67, 209231. doi: 10.1111/lang.12227

Keywords

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed