Skip to main content
    • Aa
    • Aa

Partial and synchronized captioning: A new tool to assist learners in developing second language listening skill

  • Maryam Sadat Mirzaei (a1), Kourosh Meshgi (a2), Yuya Akita (a3) and Tatsuya Kawahara (a4)

This paper introduces a novel captioning method, partial and synchronized captioning (PSC), as a tool for developing second language (L2) listening skills. Unlike conventional full captioning, which provides the full text and allows comprehension of the material merely by reading, PSC promotes listening to the speech by presenting a selected subset of words, where each word is synched to its corresponding speech signal. In this method, word-level synchronization is realized by an automatic speech recognition (ASR) system, dedicated to the desired corpora. This feature allows the learners to become familiar with the correspondences between words and their utterances. Partialization is done by automatically selecting words or phrases likely to hinder listening comprehension. In this work we presume that the incidence of infrequent or specific words and fast delivery of speech are major barriers to listening comprehension. The word selection criteria are thus based on three factors: speech rate, word frequency and specificity. The thresholds for these features are adjusted to the proficiency level of the learners. The selected words are presented to aid listening comprehension while the remaining words are masked in order to keep learners listening to the audio. PSC was evaluated against no-captioning and full-captioning conditions using TED videos. The results indicate that PSC leads to the same level of comprehension as the full-captioning method while presenting less than 30% of the transcript. Furthermore, compared with the other methods, PSC can serve as an effective medium for decreasing dependence on captions and preparing learners to listen without any assistance.

Linked references
Hide All

This list contains references from the content that can be linked to their source. For a full set of references and notes please see the PDF or HTML where available.

A. Baddeley (1992) Working memory. Science, 255(5044): 556559.

S. A. Bird and J. N. Williams (2002) The effect of bimodal input on implicit and explicit memory: An investigation into the benefits of within-language subtitling. Applied Psycholinguistics, 23(4): 509533.

G. Buck (2001) Assessing listening. Cambridge: Cambridge University Press.

A. C. S. Chang (2009) Gains to L2 listeners from reading while listening vs. listening only in comprehending short stories. System, 37(4): 652663.

A. Coxhead (2000) A new academic word list. TESOL Quarterly, 34(2): 213238.

M. Danan (1992) Reversed subtitling and dual coding theory: New directions for foreign language instruction. Language Learning, 42(4): 497527.

M. Danan (2004) Captioning and subtitling: Undervalued language learning strategies. META, 49(1): 6677.

N. C. Ellis (2003) Constructions, chunking, and connectionism: The emergence of second language structure. In Doughty, C. J. and Long, M. H. (eds.), The handbook of second language acquisition. Oxford: Blackwell, 63103.

D. Gardner and M. Davies (2013) A new academic vocabulary list. Applied Linguistics, 35(3): 305327.

T. J. Garza (1991) Evaluating the use of captioned video materials in advanced foreign language learning. Foreign Language Annals, 24(3): 239258.

A. Gilmore (2007) Authentic materials and authenticity in foreign language learning. Language Teaching, 40(2): 97118.

C. Goh (2000) A cognitive perspective on language learners’ listening comprehension problems. System, 28(1): 5575.

R. Griffiths (1992) Speech rate and listening comprehension: Further evidence of the relationship. TESOL Quarterly, 26(2): 385390.

A. W. Inhoff and K. Rayner (1986) Parafoveal word processing during eye fixations in reading: Effects of word frequency. Perception & Psychophysics, 40(6): 431439.

J. King (2002) Using DVD feature films in the EFL classroom. Computer Assisted Language Learning, 15(5): 509523.

O. Korat (2010) Reading electronic books as a support for vocabulary, story comprehension and word reading in kindergarten and first grade. Computers & Education, 55(1): 2431.

A. N. Leveridge and J. C. Yang (2013) Testing learner reliance on caption supports in second language listening comprehension multimedia environments. ReCALL, 25(2): 199214.

R. J. Lund (1991) A comparison of second language listening and reading comprehension. The Modern Language Journal, 75(2): 196204.

P. Markham and L. Peter (2003) The influence of English language and Spanish language captions on foreign language listening/reading comprehension. Journal of Educational Technology Systems, 31(3): 331341.

R. E. Mayer and R. Moreno (2003) Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist, 38(1): 4352.

R. E. Mayer , H. Lee and A. Peebles (2014) Multimedia learning in a second language: A cognitive load perspective. Applied Cognitive Psychology, 28(5): 653660.

J. Medwell (1998) The talking books project: Some further insights into the use of talking books to develop reading. Reading, 32(1): 38.

M. Montero Perez , W. Van den Noortgate and P. Desmet (2013) Captioned video for L2 listening and vocabulary learning: A meta-analysis. System, 41(3): 720739.

M. Montero Perez , E. Peters and P. Desmet (2014a) Is less more? Effectiveness and perceived usefulness of keyword and full captioned video for L2 listening comprehension. ReCALL, 26(1): 2143.

I. S. P. Nation (2006) How large a vocabulary is needed for reading and listening? Canadian Modern Language Review, 63(1): 5982.

A. Paivio (1990) Mental representations: A dual coding approach. Oxford: Oxford University Press.

J. T. Pujolà (2002) CALLing for help: Researching language learning strategies using help facilities in a web-based multimedia program. ReCALL, 14(2): 235262.

A. Révész and T. Brunfaut (2013) Text characteristics of task input and difficulty in second language listening comprehension. Studies in Second Language Acquisition, 35(1): 3165.

J. Sweller (1994) Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction, 4(4): 295312.

S. Tauroza and D. Allison (1990) Speech rates in British English. Applied Linguistics, 11(1): 90105.

G. Taylor (2005) Perceived processing strategies of students watching captioned video. Foreign Language Annals, 38(3): 422427.

L. Vandergrift (2007) Recent developments in second and foreign language listening comprehension research. Language Teaching, 40(3): 191210.

R. Vanderplank (1988) The value of teletext sub-titles in language learning. ELT Journal, 42(4): 272281.

R. Vanderplank (2010) Déjà vu? A decade of research on language laboratories, television and video in language learning. Language Teaching, 43(1): 137.

P. Winke , S. Gass and T. Sydorenko (2013) Factors influencing the use of captions by foreign language learners: An eye-tracking study. The Modern Language Journal, 97(1): 254275.

Y. Zhao (1997) The effects of listeners’ control of speech rate on second language comprehension. Applied Linguistics, 18(1): 4968.

Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

  • ISSN: 0958-3440
  • EISSN: 1474-0109
  • URL: /core/journals/recall
Please enter your name
Please enter a valid email address
Who would you like to send this to? *


Type Description Title
Supplementary Materials

Mirzaei supplementary material
Mirzaei supplementary material 1

 Word (265 KB)
265 KB


Full text views

Total number of HTML views: 8
Total number of PDF views: 147 *
Loading metrics...

Abstract views

Total abstract views: 458 *
Loading metrics...

* Views captured on Cambridge Core between 2nd March 2017 - 19th September 2017. This data will be updated every 24 hours.