Skip to main content
×
×
Home

Processing of native and foreign language subtitles in films: An eye tracking study

  • MARIE-JOSÉE BISSON (a1), WALTER J. B. VAN HEUVEN (a1), KATHY CONKLIN (a1) and RICHARD J. TUNNEY (a1)
Abstract

Foreign language (FL) films with subtitles are becoming increasingly popular, and many European countries use subtitling as a cheaper alternative to dubbing. However, the extent to which people process subtitles under different subtitling conditions remains unclear. In this study, participants watched part of a film under standard (FL soundtrack and native language subtitles), reversed (native language soundtrack and FL subtitles), or intralingual (FL soundtrack and FL subtitles) subtitling conditions while their eye movements were recorded. The results revealed that participants read the subtitles irrespective of the subtitling condition. However, participants exhibited more regular reading of the subtitles when the film soundtrack was in an unknown FL. To investigate the incidental acquisition of FL vocabulary, participants also completed an unexpected auditory vocabulary test. Because the results showed no vocabulary acquisition, the need for more sensitive measures of vocabulary acquisition are discussed. Finally, the reading of the subtitles is discussed in relation to the saliency of subtitles and automatic reading behavior.

  • View HTML
    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Processing of native and foreign language subtitles in films: An eye tracking study
      Available formats
      ×
      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      Processing of native and foreign language subtitles in films: An eye tracking study
      Available formats
      ×
      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      Processing of native and foreign language subtitles in films: An eye tracking study
      Available formats
      ×
Copyright
The online version of this article is published within an Open Access environment subject to the conditions of the Creative Commons Attribution-NonCommercial-ShareAlike licence <http://creativecommons.org/licenses/by-nc-sa/3.0/>. The written permission of Cambridge University Press must be obtained for commercial re-use.
Corresponding author
ADDRESS FOR CORRESPONDENCE Correspondence concerning this article should be addressed to Marie-Josée Bisson, School of Psychology, University of Nottingham, Nottingham NG7 2RD, UK. E-mail: lpxmb@nottingham.ac.uk
References
Hide All
Bird, S. A., & Williams, J. N. (2002). The effect of bimodal input on implicit and explicit memory: An investigation into the benefits of within-language subtitling. Applied Psycholinguistics, 23, 509533.
Borras, I., & Lafayette, R. C. (1994). Effects of multimedia courseware subtitling on the speaking performance of college-students of French. Modern Language Journal, 78, 6175.
Burnham, D., Leigh, G., Noble, W., Jones, C., Tyler, M., Grebennikov, L., et al. (2008). Parameters in television captioning for deaf and hard-of-hearing adults: Effects of caption rate versus text reduction on comprehension. Journal of Deaf Studies and Deaf Education, 13, 391404.
Cerf, M., Frady, E. P., & Koch, C. (2009). Faces and text attract gaze independent of the task: Experimental data and computer model. Journal of Vision, 9, 115.
D'Ydewalle, G., & De Bruycker, W. (2007). Eye movements of children and adults while reading television subtitles. European Psychologist, 12, 196205.
D'Ydewalle, G., Muylle, P., & van Rensbergen, J. (1985). Attention shifts in partially redundant information situations. In Groner, R., McConkie, G. W., & Menz, C. (Eds.), Eye movements and human information processing (pp. 375384). North-Holland: Elsevier.
D'Ydewalle, G., Praet, C., Verfaillie, K., & van Rensbergen, J. (1991). Watching subtitled television: Automatic reading behavior. Communication Research, 18, 650665.
D'Ydewalle, G., & van de Poel, M. (1999). Incidental foreign-language acquisition by children watching subtitled television programs. Journal of Psycholinguistic Research, 28, 227244.
D'Ydewalle, G., & van Rensbergen, J. (1989). Developmental studies of text-picture interactions in the perception of animated cartoons with text. In Mandl, H. & Levin, J. R. (Eds.), Knowledge acquisition from text and pictures (pp. 233248). North-Holland: Elsevier.
D'Ydewalle, G., van Rensbergen, J., & Pollet, J. (1987). Reading a message when the same message is available auditorily in another language: The case of subtitling. In O'Regan, J. K. & Levy-Schoen, A. (Eds.), Eye movements from physiology to cognition (pp. 313321). Amsterdam: Elsevier.
De Linde, Z., & Kay, N. (1999). Processing subtitles and film images: Hearing vs deaf viewers. Translator, 5, 4560.
Hillenburg, S. (Director). (2004). Spongebob Squarepants The Movie [Animated motion picture]. Nickleodeon.
Hu, C.-F. (2008). Use orthography in L2 auditory word learning: Who benefits? Reading and Writing, 21, 823841.
Inhoff, A. W., & Rayner, K. (1986). Parafoveal word processing during eye fixations in reading: Effects of word frequency. Perception & Psychophysics, 40, 431439.
Itti, L., & Koch, C. (2000). A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Research, 40, 14891506.
Itti, L., & Koch, C. (2001). Computational modelling of visual attention. Nature Reviews Neuroscience, 2, 194203.
Jensema, C. J., El Sharkawy, S., Danturthi, R. S., Burch, R., & Hsu, D. (2000). Eye movement patterns of captioned television viewers. American Annals of the Deaf, 145, 275285.
Koolstra, C. M., & Beentjes, J. W. J. (1999). Children's vocabulary acquisition in a foreign language through watching subtitled television programs at home. Educational Technology Research and Development, 47, 5160.
Koolstra, C. M., Peeters, A. L., & Spinhof, H. (2002). The pros and cons of dubbing and subtitling. European Journal of Communication, 17, 325354.
Laberge, D., & Samuels, S. J. (1974). Toward a theory of automatic information processing in reading. Cognitive Psychology, 6, 293323.
Lambert, W. E., & Holobow, N. E. (1984). Combinations of printed script and spoken dialogue that show promise for students of a foreign-language. Canadian Journal of Behavioural Science–Revue Canadienne Des Sciences Du Comportement, 16, 111.
MacLeod, C. (1991). Half a century of research on the Stroop effect: An integrative review. Psychological Bulletin, 109, 163203.
Mital, P. K., Smith, T. J., Hill, R., & Henderson, J. M. (2010). Clustering of gaze during dynamic scene viewing is predicted by motion. Cognitive Computation, 3, 115.
Mitterer, H., & McQueen, J. M. (2009). Foreign subtitles help but native-language subtitles harm foreign speech perception. Plos One, 4, A146A150.
Nation, I. S. P. (2001). Learning vocabulary in another language. Cambridge: Cambridge University Press.
Paribakht, S. T., & Wesche, M. (1997). Vocabulary enhancement activities and reading for meaning in second language vocabulary acquisition. In Coady, J. & Huckin, T. N. (Eds.), Second language vocabulary acquisition: A rationale for pedagogy (pp. 174200). Cambridge: Cambridge University Press.
Perego, E., Del Missier, F., Porta, M., & Mosconi, M. (2010). The cognitive effectiveness of subtitle processing. Media Psychology, 13, 243272.
Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124, 372422.
Rayner, K. (2009). Eye movements and attention in reading, scene perception, and visual search. Quarterly Journal of Experimental Psychology, 62, 14571506.
Rayner, K., & Raney, G. E. (1996). Eye movement control in reading and visual search: Effects of word frequency. Psychonomic Bulletin & Review, 3, 245248.
Ricketts, J., Bishop, D. V., & Nation, K. (2009). Orthographic facilitation in oral vocabulary acquisition. Quarterly Journal of Experimental Psychology, 62, 19481966.
Rosenthal, J., & Ehri, L. C. (2008). The mnemonic value of orthography for vocabulary learning. Journal of Educational Psychology, 100, 175191.
Samuels, S. J. (1994). Toward a theory of automatic information processing in reading, revisited. In Ruddell, R. B., Ruddell, M. R., & Singer, H. (Eds.), Theoretical models and processes of reading (4th ed., pp. 816837). Newark, DE: International Reading Association.
Schepens, J., Dijkstra, T., & Grootjen, F. (2012). Distribution of cognates in Europe as based on the Levenshtein distance. Bilingualism: Language and Cognition, 15, 157166.
Schmitt, N. (2010). Researching vocabulary: A vocabulary research manual. Houndmills: Palgrave Macmillan.
Stroop, J. R. (1935). Studies of interference in serial verbal reactions. Journal of Experimental Psychology, 18, 643662.
Sydorenko, T. (2010). Modality of input and vocabulary acquisition. Language Learning & Technology, 14, 5073.
van Heuven, W. J. B. (2010). Eyetracker Output Utility (Version 1.20 beta 14) [Computer program]. Retrieved from http://www.psychology.nottingham.ac.uk/staff/wvh/eou/index.html
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Applied Psycholinguistics
  • ISSN: 0142-7164
  • EISSN: 1469-1817
  • URL: /core/journals/applied-psycholinguistics
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed