Skip to main content Accessibility help
×
Home

Effects of modality preference and working memory capacity on captioned videos in enhancing L2 listening outcomes

  • Emily Fen Kam (a1), Yeu-Ting Liu (a2) and Wen-Ta Tseng (a3)

Abstract

Captioned video is widely used to enhance second language (L2) learners’ exposure to oral input beyond the classroom setting, and captioning has been found to provide an instantaneous, useful visual aid for parsing and understanding L2 oral discourse. Nevertheless, a meta-analysis has shown that captioning exerts a selective effect on L2 learners with different profiles. This study investigated whether L2 learners’ modality preferences (visual vs. auditory) and working memory capacity (high vs. low) would modulate the effect of full captions on L2 listening outcome. Results from 60 participants revealed that both cognitive variables affected their L2 listening to different extents. Notably, working memory capacity modulates the impact of L2 learners’ preferred modality on their listening outcome. Modality preference did not exert any significant impact on the listening outcome of L2 learners with lower working memory capacity. For L2 learners with high working memory capacity, their modality preference played a pivotal role in modulating their listening outcome; in this case, auditory learners had the best listening performance viewing the video without captions, whereas visual learners did best when watching the captioned video. These findings speak to the need for taking individual differences into consideration when employing captioned videos.

Copyright

Corresponding author

*All correspondence regarding this publication should be addressed to Yeu-Ting Liu (Email: yeutingliu@ntnu.edu.tw).

References

Hide All
Alptekin, C. & Erçetin, G. (2009) Assessing the relationship of working memory to L2 reading: Does the nature of comprehension process and reading span task make a difference? System, 37(4): 627639. https://doi.org/10.1016/j.system.2009.09.007
Chang, C.-C., Tseng, K.-H. & Tseng, J.-S. (2011) Is single or dual channel with different English proficiencies better for English listening comprehension, cognitive load and attitude in ubiquitous learning environment? Computers & Education, 57(4): 23132321. https://doi.org/10.1016/j.compedu.2011.06.006
Chen, Y.-R., Liu, Y.-T. & Todd, A. G. (2018) Transient but effective? Captioning and adolescent EFL learners’ spoken vocabulary acquisition. English Teaching & Learning, 42(1): 2556. https://doi.org/10.1007/s42321-018-0002-8
Friedman, N. P. & Miyake, A. (2004) The reading span test and its predictive power for reading comprehension ability. Journal of Memory and Language, 51(1): 136158. https://doi.org/10.1016/j.jml.2004.03.008
Garza, T. J. (1991) Evaluating the use of captioned video materials in advanced foreign language learning. Foreign Language Annals, 24(3): 239258. https://doi.org/10.1111/j.1944-9720.1991.tb00469.x
Gass, S., Winke, P., Isbell, D. R. & Ahn, J. (2019) How captions help people learn languages: A working-memory, eye-tracking study. Language Learning & Technology, 23(2): 84104. https://doi.org/10125/44684
Hasler, B. S., Kersten, B. & Sweller, J. (2007) Learner control, cognitive load and instructional animation. Applied Cognitive Psychology, 21(6): 713729. https://doi.org/10.1002/acp.1345
Huang, H.-C. & Eskey, D. E. (1999) The effects of closed-captioned television on the listening comprehension of intermediate English as a second language (ESL) students. Journal of Educational Technology Systems, 28(1): 7596. https://doi.org/10.2190/RG06-LYWB-216Y-R27G
Kozan, K., Erçetin, G. & Richardson, J. C. (2015) Input modality and working memory: Effects on second language text comprehension in a multimedia learning environment. System, 55: 6373. https://doi.org/10.1016/j.system.2015.09.001
Lee, M. & Révész, A. (2018) Promoting grammatical development through textually enhanced captions: An eye-tracking study. The Modern Language Journal, 102(3): 557577. https://doi.org/10.1111/modl.12503
Lee, P.-J., Liu, Y.-T. & Tseng, W.-T. (2019) One size fits all? In search of the desirable caption display for second language learners with different caption reliance in listening comprehension. Language Teaching Research. Advance online publication. https://doi.org/10.1177/1362168819856451
Leveridge, A. N. & Yang, J. C. (2013) Testing learner reliance on caption supports in second language listening comprehension multimedia environments. ReCALL, 25(2): 199214. https://doi.org/10.1017/S0958344013000074
Leveridge, A. N. & Yang, J. C. (2014) Learner perceptions of reliance on captions in EFL multimedia listening comprehension. Computer Assisted Language Learning, 27(6): 545559. https://doi.org/10.1080/09588221.2013.776968
Liu, Y.-T. & Todd, A. G. (2014) Dual-modality input in repeated reading for foreign language learners with different learning styles. Foreign Language Annals, 47(4): 684706. https://doi.org/10.1111/flan.12113
Linck, J. A., Osthus, P., Koeth, J. T. & Bunting, M. F. (2014) Working memory and second language comprehension and production: A meta-analysis. Psychonomic Bulletin & Review, 21(4): 861883. https://doi.org/10.3758/s13423-013-0565-2
Markham, P. & Peter, L. (2003) The influence of English language and Spanish language captions on foreign language listening/reading comprehension. Journal of Educational Technology Systems, 31(3): 331341. https://doi.org/10.2190/BHUH-420B-FE23-ALA0
Markham, P. L., Peter, L. A. & McCarthy, T. J. (2001) The effects of native language vs. target language captions on foreign language students’ DVD video comprehension. Foreign Language Annals, 34(5): 439445. https://doi.org/10.1111/j.1944-9720.2001.tb02083.x
Mayer, R. E. (2001) Multimedia learning. New York: Cambridge University Press. https://doi.org/10.1017/CBO9781139164603
Mayer, R. E. (2005) Principles for managing essential processing in multimedia learning: Segmenting, pretraining, and modality principles. In Mayer, R. E. (ed.), Cambridge handbook of multimedia learning. New York: Cambridge University Press, 169182. https://doi.org/10.1017/CBO9780511816819.012
Mayer, R. E. & Moreno, R. (2003) Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist, 38(1): 4352. https://doi.org/10.1207/S15326985EP3801_6
Mirzaei, M. S., Meshgi, K., Akita, Y. & Kawahara, T. (2017) Partial and synchronized captioning: A new tool to assist learners in developing second language listening skill. ReCALL, 29(2): 178199. https://doi.org/10.1017/S0958344017000039
Montero Perez, M., Noortgate, W. V. D. & Desmet, P. (2013) Captioned video for L2 listening and vocabulary learning: A meta-analysis. System, 41(3): 720739. https://doi.org/10.1016/j.system.2013.07.013
Moreno, R. & Mayer, R. E. (2007) Interactive multimodal learning environments. Educational Psychology Review, 19(3): 309326. https://doi.org/10.1007/s10648-007-9047-2
Peters, E. (2019) The effect of imagery and on-screen text on foreign language vocabulary learning from audio-visual input. TESOL Quarterly, 53: 10081032. https://doi.org/10.1002/tesq.531
Pujadas, G. & Muñoz, C. (2019) Extensive viewing of captioned and subtitled TV series: A study of L2 vocabulary learning by adolescents. The Language Learning Journal, 47(4): 479496. https://doi.org/10.1080/09571736.2019.1616806
Reid, J. M. (ed.) (1995) Learning styles in the ESL/EFL classroom. Boston: Heinle & Heinle.
Révész, A. (2012) Working memory and the observed effectiveness of recasts on different L2 outcome measures. Language Learning, 62(1): 93132. https://doi.org/10.1111/j.1467-9922.2011.00690.x
Sagarra, N. (2017) Longitudinal effects of working memory on L2 grammar and reading abilities. Second Language Research, 33(3): 341363. https://doi.org/10.1177/0267658317690577
Seufert, T., Schütze, M. & Brünken, R. (2009) Memory characteristics and modality in multimedia learning: An aptitude–treatment–interaction study. Learning and Instruction, 19(1): 2842. https://doi.org/10.1016/j.learninstruc.2008.01.002
Suárez, M. D. M. & Gesa, F. (2019) Learning vocabulary with the support of sustained exposure to captioned video: Do proficiency and aptitude make a difference? The Language Learning Journal, 47(4): 497517. https://doi.org/10.1080/09571736.2019.1617768
Sueyoshi, A. & Hardison, D. M. (2005) The role of gestures and facial cues in second language listening comprehension. Language Learning, 55(4): 661699. https://doi.org/10.1111/j.0023-8333.2005.00320.x
Taylor, G. (2005) Perceived processing strategies of students watching captioned video. Foreign Language Annals, 38(3): 422427. https://doi.org/10.1111/j.1944-9720.2005.tb02228.x
Turner, M. L. & Engle, R. W. (1989) Is working memory capacity task dependent? Journal of Memory and Language, 28(2): 127154. https://doi.org/10.1016/0749-596X(89)90040-5
Winke, P., Gass, S. & Sydorenko, T. (2010) The effects of captioning videos used for foreign language listening activities. Language Learning & Technology, 14(1): 6586.
Winke, P., Gass, S. & Sydorenko, T. (2013) Factors influencing the use of captions by foreign language learners: An eye‐tracking study. The Modern Language Journal, 97(1): 254275. https://doi.org/10.1111/j.1540-4781.2013.01432.x

Keywords

Effects of modality preference and working memory capacity on captioned videos in enhancing L2 listening outcomes

  • Emily Fen Kam (a1), Yeu-Ting Liu (a2) and Wen-Ta Tseng (a3)

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed.