Skip to main content

Visual speech fills in both discrimination and identification of non-intact auditory speech in children*


To communicate, children must discriminate and identify speech sounds. Because visual speech plays an important role in this process, we explored how visual speech influences phoneme discrimination and identification by children. Critical items had intact visual speech (e.g. bæz) coupled to non-intact (excised onsets) auditory speech (signified by /–bz). Children discriminated syllable pairs that differed in intactness (i.e. bæz:/–b/æz) and identified non-intact nonwords (/–b/æz). We predicted that visual speech would cause children to perceive the non-intact onsets as intact, resulting in more same responses for discrimination and more intact (i.e. bæz) responses for identification in the audiovisual than auditory mode. Visual speech for the easy-to-speechread /b/ but not for the difficult-to-speechread /g/ boosted discrimination and identification (about 35–45%) in children from four to fourteen years. The influence of visual speech on discrimination was uniquely associated with the influence of visual speech on identification and receptive vocabulary skills.

Corresponding author
Address for correspondence: Susan Jerger, School of Behavioral and Brain Sciences, GR4.1, University of Texas at Dallas, 800 W. Campbell Rd, Richardson, TX 75080, USA. tel: 512-216-2961; e-mail:
Hide All

This research was supported by the National Institute on Deafness and Other Communication Disorders, grant DC-00421 to the University of Texas at Dallas. Dr Abdi would like to acknowledge the support of an EURIAS fellowship at the Paris Institute for Advanced Studies (France), with the support of the European Union's 7th Framework Program for research, and from funding from the French State managed by the “Agence Nationale de la Recherche (program: Investissements d'avenir, ANR-11-LABX-0027-01 Labex RFIEA+).” Sincere appreciation to speech science colleagues for their guidance and advice to adopt a perceptual criterion for editing the non-intact stimuli. We appreciate Dr Nancy Tye-Murray's comments on an earlier version of this paper. We thank the children and parents who participated and the research staff who assisted, namely Aisha Aguilera, Carissa Dees, Nina Dinh, Nadia Dunkerton, Alycia Elkins, Brittany Hernandez, Cassandra Karl, Demi Krieger, Michelle McNeal, Jeffrey Okonye, and Kimberly Periman of the University of Texas at Dallas (data collection, analysis, presentation), and Derek Hammons and Scott Hawkins of the University of Texas at Dallas and Dr Brent Spehar and Dr Nancy Tye-Murray of Washington University School of Medicine (stimuli recording and editing, computer programming).

Hide All
Abdi, H., Edelman, B., Valentin, D. & Dowling, W. (2009). Experimental design and analysis for psychology. New York: Oxford University Press.
Aslin, R. & Smith, L. (1988). Perceptual development. Annual Review of Psychology 39, 435–73.
Boothroyd, A. (1988). Linguistic factors in speechreading. Volta Review 90, 7787.
Boothroyd, A., Eisenberg, L. S. & Martinez, A. S. (2010). An on-line imitative test of speech-pattern contrast perception (OlimSpac): developmental effects in normally hearing children. Journal of Speech, Language, and Hearing Research 53, 531–42.
Bouton, S., Cole, P. & Serniclaes, W. (2012). The influence of lexical knowledge on phoneme discrimination in deaf children with cochlear implants. Speech Communication 54, 189–98.
Briscoe, J., Bishop, D. & Norbury, C. (2001). Phonological processing, language, and literacy: a comparison of children with mild-to-moderate sensorineural hearing loss and those with specific language impairment. Journal of Child Psychology and Psychiatry 42, 329–40.
Burnham, D. & Dodd, B. (2004). Auditory–visual speech integration by prelinguistic infants: perception of an emergent consonant in the McGurk effect. Developmental Psychobiology 44, 209–20.
Calvert, G., Spence, C. & Stein, B. (2004). The handbook of multisensory processes. Cambridge, MA: MIT Press.
Campbell, R. (1988). Tracing lip movements: making speech visible. Visible Language 22, 3257.
Carney, A. (1996). Audition and the development of oral communication competency. In Bess, F., Gravel, J. & Tharpe, A. (eds), Amplication for children with auditory deficits, 2954. Nashville, TN: Bill Wilkerson Center Press.
Clopper, C., Pisoni, D. & Tierney, A. (2006). Effects of open-set and closed-set task demands on spoken word recognition. Journal of the American Academy of Audiology 17, 331–49.
Conway, C. & Pisoni, D. (2008). Neurocognitive basis of implicit learning of sequential structure and its relation to language processing. Annals of the New York Academy of Sciences 1145, 113–31.
Desjardins, R., Rogers, J. & Werker, J. (1997). An exploration of why preschoolers perform differently than do adults in audiovisual speech perception tasks. Journal of Experimental Child Psychology 66, 85110.
Dodd, B. & Campbell, R. (Eds.) (1987). Hearing by eye: the psychology of lip-reading. London: Lawrence Erlbaum.
Dunn, L. & Dunn, D. (2007). The Peabody Picture Vocabulary Test-IV, 4th ed. Minneapolis, MN: NCS Pearson.
Dupont, S., Aubin, J. & Menard, L. (2005). A study of the McGurk effect in 4- and 5-year-old French Canadian children. ZAS Papers in Linguistics 40, 117.
Edwards, J. & Lahey, M. (1998). Nonword repetitions of children with specific language impairment: exploration of some explanations for their inaccuracies. Applied Psycholinguistics 19, 279309.
Edwards, J., Munson, B. & Beckman, M. (2011). Lexicon–phonology relationships and dynamics of early language development—a commentary on Stoel-Gammon's ‘Relationships between lexical and phonological development in young children'. Journal of Child Language 38, 3540.
Erdener, D. & Burnham, D. (2013). The relationship between auditory-visual speech perception and language-specific speech perception at the onset of reading instruction in English-speaking children. Journal of Experimental Child Psychology 114, 120–38.
Files, B., Tjan, B., Jiang, J. & Bernstein, L. (2015). Visual speech discrimination and identification of natural and synthetic consonant stimuli. Frontiers in Psychology 6, 878. Online: <doi: 10.3389/fpsyg.2015.00878>.
Fort, M., Spinelli, E., Savariaux, C. & Kandel, S. (2010). The word superiority effect in audiovisual speech perception. Speech Communication 52, 525–32.
Fort, M., Spinelli, E., Savariaux, C. & Kandel, S. (2012). Audiovisual vowel monitoring and the word superiority effect in children. International Journal of Behavioral Development 36, 457–67.
Gathercole, S. (2006). Nonword repetition and word learning: the nature of the relationship. Applied Psycholinguistics 27, 513–43.
Gogate, L., Walker-Andrews, A. & Bahrick, L.. (2001). The intersensory origins of word comprehension: an ecological-dynamic systems view. Developmental Science 4, 137.
Hnath-Chisolm, T., Laipply, E. & Boothroyd, A. (1998). Age-related changes on a children's test of sensory-level speech perception capacity. Journal of Speech, Language, and Hearing Research 41, 94106.
Holt, R., Kirk, K. & Hay-McCutcheon, M. (2011). Assessing multimodal spoken word-in-sentence recognition in children with normal hearing and children with cochlear implants. Journal of Speech, Language, and Hearing Research 54, 632–57.
Hornickel, J., Skoe, E., Nicol, T., Zecker, S. & Kraus, N. (2009). Subcortical differentiation of stop consonants relates to reading and speech-in-noise perception. Proceedings of the National Academy of Sciences of the United States of America 106, 13022–27.
Jerger, S., Damian, M. F., Spence, M. J., Tye-Murray, N. & Abdi, H. (2009). Developmental shifts in children's sensitivity to visual speech: a new multimodal picture-word task. Journal of Experimental Child Psychology 102, 4059.
Jerger, S., Damian, M., Tye-Murray, N. & Abdi, H. (2014). Children use visual speech to compensate for non-intact auditory speech. Journal of Experimental Child Psychology 126, 295312.
Jerger, S., Damian, M., Tye-Murray, N. & Abdi, H. (2017). Children perceive speech onsets by ear and eye. Journal of Child Language 44, 185215.
Jerger, S., Martin, R. & Jerger, J. (1987). Specific auditory perceptual dysfunction in a learning disabled child. Ear and Hearing 8, 7886.
Kushnerenko, E., Tomalski, P., Ballieux, H., Potton, A., Birtles, D., Frostick, C. & Moore, D. (2013). Brain responses and looking behavior during audiovisual speech integration in infants predict auditory speech comprehension in the second year of life. Frontiers in Psychology 4, 432. Online: <doi: 10.3389/fpsyg.2013.00432. eCollection 2013>.
Lalonde, K. & Holt, R. (2014). Cognitive and linguistic sources of variance in 2-year-olds’ speech-sound discrimination: a preliminary investigation. Journal of Speech, Language, and Hearing Research 57, 308–26.
Lalonde, K. & Holt, R. (2015). Preschoolers benefit from visually salient speech cues. Journal of Speech, Language, and Hearing Research 58, 135–50.
Lalonde, K. & Holt, R. (2016). Audiovisual speech perception development at varying levels of perceptual processing. Journal of the Acoustical Society of America 139, 1713–23.
Lewkowicz, D. & Hansen-Tift, A. (2012). Infants deploy selective attention to the mouth of a talking face when learning speech. Proceedings of the National Academy of Sciences of the United States of America 109, 1431–6.
Martin, R., Breedin, S. & Damian, M. (1999). The relation of phoneme discrimination, lexical access, and short-term memory: a case study and interactive activation account. Brain and Language 70, 437–82.
Massaro, D., Thompson, L., Barron, B. & Laren, E. (1986). Developmental changes in visual and auditory contributions to speech perception. Journal of Experimental Child Psychology 41, 93113.
McConachie, H. & Moore, V. (1994). Early expressive language of severely visually impaired children. Developmental Medicine & Child Neurology 36, 230–40.
McGurk, H. & MacDonald, M. (1976). Hearing lips and seeing voices. Nature 264, 746–8.
Menn, L. & Stoel-Gammon, C. (2009). Phonological development. In Gleason, J. & Ratner, N. (eds), The development of language, 58103. Boston, MA: Pearson.
Mills, A. (1987). The development of phonology in the blind child. In Dodd, B. & Campbell, R. (eds), Hearing by eye: the psychology of lipreading, 145–61. London: Erlbaum.
Rato, A. (2014). Effects of perceptual training on the identification of English vowels by native speakers of European Portuguese. Concordia Working Papers in Applied Linguistics 5, 529–46.
Rosenblum, L., Schmuckler, M. & Johnson, J. (1997). The McGurk effect in infants. Perception & Psychophysics 59, 347–57.
Ross, L., Molholm, S., Blanco, D., Gomez-Ramirez, M., Saint-Amour, D. & Foxe, J. (2011). The development of multisensory speech perception continues into the late childhood years. European Journal of Neuroscience 33, 2329–37.
Rubin, P., Turvey, M. & van Gelder, P. (1976). Initial phonemes are detected faster in spoken words than in spoken nonwords. Perception & Psychophysics 19, 394–8.
Snowling, M. & Hulme, C. (1994). The development of phonological skills. Philosophical Transactions of the Royal Society of London B 346, 21–7.
Sommers, M., Kirk, K. & Pisoni, D. (1997). Some considerations in evaluating spoken word recognition by normal-hearing, noise-masked normal-hearing, and cochlear implant listeners I: the effects of response format. Ear & Hearing 18, 8999.
Teinonen, T., Aslin, R., Alku, P. & Csibra, G. (2008). Visual speech contributes to phonetic learning in 6-month-old infants. Cognition 108, 850–5.
Tremblay, C., Champoux, R., Voss, P., Bacon, B., Lepore, F. & Theoret, H. (2007). Speech and non-speech audio-visual illusions: a developmental study. PLoS One 2(8), online: <e742.DOI:10.1371/journal.pone.0000742>.
Tsao, F., Liu, H. & Kuhl, P. (2004). Speech perception in infancy predicts language development in the second year of life: a longitudinal study. Child Development 75, 1067–84.
Tye-Murray, N. (2014). Foundations of aural rehabilitation: children, adults, and their family members, 4th ed. Boston, MA: Cengage Learning.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Journal of Child Language
  • ISSN: 0305-0009
  • EISSN: 1469-7602
  • URL: /core/journals/journal-of-child-language
Please enter your name
Please enter a valid email address
Who would you like to send this to? *


Altmetric attention score

Full text views

Total number of HTML views: 7
Total number of PDF views: 53 *
Loading metrics...

Abstract views

Total abstract views: 277 *
Loading metrics...

* Views captured on Cambridge Core between 20th July 2017 - 21st March 2018. This data will be updated every 24 hours.