Hostname: page-component-8448b6f56d-42gr6 Total loading time: 0 Render date: 2024-04-23T16:30:30.764Z Has data issue: false hasContentIssue false
  • English
  • Français

How much do visual cues help listeners in perceiving accented speech?

Published online by Cambridge University Press:  26 October 2018

YI ZHENG Open the ORCID record for YI ZHENG [Opens in a new window]  and
ARTHUR G. SAMUEL
YI ZHENG*
Affiliation:
Stony Brook University
ARTHUR G. SAMUEL
Affiliation:
Stony Brook University; Basque Center on Cognition, Brain, and Language; and Ikerbasque, Basque Foundation for Science
*
ADDRESS FOR CORRESPONDENCE Yi Zheng, Department of Psychology, Stony Brook University, Stony Brook, NY 11794-2500. E-mail: yizheng.psychology@gmail.com
Get access Rights & Permissions [Opens in a new window]

Abstract

It has been documented that lipreading facilitates the understanding of difficult speech, such as noisy speech and time-compressed speech. However, relatively little work has addressed the role of visual information in perceiving accented speech, another type of difficult speech. In this study, we specifically focus on accented word recognition. One hundred forty-two native English speakers made lexical decision judgments on English words or nonwords produced by speakers with Mandarin Chinese accents. The stimuli were presented as either as videos that were of a relatively far speaker or as videos in which we zoomed in on the speaker’s head. Consistent with studies of degraded speech, listeners were more accurate at recognizing accented words when they saw lip movements from the closer apparent distance. The effect of apparent distance tended to be larger under nonoptimal conditions: when stimuli were nonwords than words, and when stimuli were produced by a speaker who had a relatively strong accent. However, we did not find any influence of listeners’ prior experience with Chinese accented speech, suggesting that cross-talker generalization is limited. The current study provides practical suggestions for effective communication between native and nonnative speakers: visual information is useful, and it is more useful in some circumstances than others.

Keywords

accentlexical decisionspeechvisual distanceword recognition
Type
Original Article
Information
Applied Psycholinguistics , Volume 40 , Issue 1 , January 2019 , pp. 93 - 109
Check for updates
Copyright
© Cambridge University Press 2018 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Adank, P., & Devlin, J. T. (2010). On-line plasticity in spoken sentence comprehension: Adapting to time-compressed speech. Neuroimage, 49, 1124–1132.Google Scholar
Adank, P., & Janse, E. (2010). Comprehension of a novel accent by young and older listeners. Psychology and Aging, 25, 736–740.Google Scholar
Baese-Berk, M. M., Bradlow, A. R., & Wright, B. A. (2013). Accent-independent adaptation to foreign accented speech. Journal of the Acoustical Society of America, 133, EL174–EL180.Google Scholar
Banai, K., & Lavner, Y. (2012). Perceptual learning of time-compressed speech: More than rapid adaptation. PLOS ONE, 7, e47099.Google Scholar
Banks, B., Gowen, E., Munro, K. J., & Adank, P. (2015a). Audiovisual cues benefit recognition of accented speech in noise but not perceptual adaptation. Frontiers in Human Neuroscience, 9, 1–13.Google Scholar
Banks, B., Gowen, E., Munro, K. J., & Adank, P. (2015b). Cognitive predictors of perceptual adaptation to accented speech. Journal of the Acoustical Society of America, 137, 2015–2024.Google Scholar
Barros, P. C. M. D. (2010). “It’s easier to understand”: the effect of a speaker’s accent, visual cues, and background knowledge on listening comprehension (Unpublished doctoral dissertation, Kansas State University).Google Scholar
Bernstein, L. E., Auer, E. T. Jr., Jiang, J., & Eberhardt, S. P. (2013). Auditory perceptual learning for speech perception can be enhanced by audiovisual training. Frontiers in Neuroscience, 7, 34.Google Scholar
Borjas, G. J. (2000). Foreign-born teaching assistants and the academic performance of undergraduates. American Economic Review, 90, 355–359.Google Scholar
Bradlow, A. R., & Bent, T. (2003). Listener adaptation to foreign-accented English. In M. J. Solé, D. Recasens, & J. Romero (Eds.), Proceedings of the 15th International Congress of Phonetic Sciences (pp. 2881–2884). Barcelona: Universitat Autònoma de Barcelona.Google Scholar
Bradlow, A. R., & Bent, T. (2008). Perceptual adaptation to non-native speech. Cognition, 106, 707–729.Google Scholar
Chun, C. W. (2008). Comments on “Evaluation of the usefulness of the Versant for English test: A response”: The author responds. Language Assessment Quarterly, 5, 168–172.Google Scholar
Davis, T. M. (1988). Open doors: Report on International Educational Exchange. Washington, DC: Institute of International Education.Google Scholar
Downey, R., Farhady, H., Present-Thomas, R., Suzuki, M., & Van Moere, A. (2008). Evaluation of the usefulness of the Versant for English Test: A response. Language Assessment Quarterly, 5, 160–167.Google Scholar
Erber, N. P. (1969). Interaction of audition and vision in the recognition of oral speech stimuli. Journal of Speech, Language, and Hearing Research, 12, 423–425.Google Scholar
Erber, N. P. (1971). Auditory and audiovisual reception of words in low-frequency noise by children with normal hearing and by children with impaired hearing. Journal of Speech, Language, and Hearing Research, 14, 496–512.Google Scholar
Finder, A. (2005, June 24). Unclear on American campus: What the foreign teachers said. New York Times, pp. A1, A18.Google Scholar
Fitch, F., & Morgan, S. E. (2003). “Not a lick of English”: Constructing the ITA identity through student narratives. Communication Education, 52, 297–310.Google Scholar
Gass, S., & Varonis, E. M. (1984). The effect of familiarity on the comprehensibility of nonnative speech. Language learning, 34, 65–87.Google Scholar
Grossman, L. A. (2011). The effects of mere exposure on responses to foreign-accented speech (Unpublished master’s thesis, San Jose State University).Google Scholar
Hazan, V., Kim, J., & Chen, Y. (2010). Audiovisual perception in adverse conditions: Language, speaker and listener effects. Speech Communication, 52, 996–1009.Google Scholar
Hazan, V., Sennema, A., Faulkner, A., Ortega-Llebaria, M., Iba, M., & Chung, H. (2006). The use of visual cues in the perception of non-native consonant contrasts a. Journal of the Acoustical Society of America, 119, 1740–1751.Google Scholar
Hosoda, M., Stone-Romero, E. F., & Walter, J. N. (2007). Listeners’ cognitive and affective reactions to English speakers with standard American English and Asian accents. Perceptual and Motor Skills, 104, 307–326.Google Scholar
Janse, E., & Adank, P. (2012). Predicting foreign-accent adaptation in older adults. Quarterly Journal of Experimental Psychology, 65, 1563–1585.Google Scholar
Jongman, A., Wade, T., & Sereno, J. (2003). On improving the perception of foreign-accented speech. In Proceedings of the 15th International Congress of Phonetic Sciences (pp. 1561–1564).Google Scholar
Kawase, S., Hannah, B., & Wang, Y. (2014). The influence of visual speech information on the intelligibility of English consonants produced by non-native speakers. Journal of the Acoustical Society of America, 136, 1352–1362.Google Scholar
Kawase, T., Sakamoto, S., Hori, Y., Maki, A., Suzuki, Y., & Kobayashi, T. (2009). Bimodal audio–visual training enhances auditory adaptation process. Neuroreport, 20, 1231–1234.Google Scholar
Larraza, S., Samuel, A. G., & Oñederra, M. L. (2016). Listening to accented speech in a second language: First language and age of acquisition effects. Journal of Experimental Psychology: Learning, Memory, and Cognition, 42, 1774–1797.Google Scholar
MacLeod, A., & Summerfield, Q. (1987). Quantifying the contribution of vision to speech perception in noise. British Journal of Audiology, 21, 131–141.Google Scholar
Miller, G., & Isard, S. (1963). Some perceptual consequences of linguistic rules. Journal of Verbal Learning and Verbal Behavior, 2, 217–228.Google Scholar
Navarra, J., & Soto-Faraco, S. (2007). Hearing lips in a second language: Visual articulatory information enables the perception of second language sounds. Psychological Research, 71, 4–12.Google Scholar
Pilling, M., & Thomas, S. (2011). Audiovisual cues and perceptual learning of spectrally distorted speech. Language and Speech, 54, 487–497.Google Scholar
Ross, L. A., Saint-Amour, D., Leavitt, V. M., Javitt, D. C., & Foxe, J. J. (2007). Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments. Cerebral Cortex, 17, 1147–1153.Google Scholar
Rubin, D. L. (1992). Nonlanguage factors affecting undergraduates’ judgments of nonnative English-speaking teaching assistants. Research in Higher Education, 33, 511–531.Google Scholar
Sumby, W. H., & Pollack, I. (1954). Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America, 26, 212–215.Google Scholar
Sumner, M., & Samuel, A. G. (2009). The effect of experience on the perception and representation of dialect variants. Journal of Memory and Language, 60, 487–501.Google Scholar
Wang, Y., Behne, D. M., & Jiang, H. (2008). Linguistic experience and audio-visual perception of non-native fricatives. Journal of the Acoustical Society of America, 124, 1716–1726.Google Scholar
Wang, Y., Behne, D. M., & Jiang, H. (2009). Influence of native language phonetic system on audio-visual speech perception. Journal of Phonetics, 37, 344–356.Google Scholar
Wayne, R. V., & Johnsrude, I. S. (2012). The role of visual speech information in supporting perceptual learning of degraded speech. Journal of Experimental Psychology: Applied, 18, 419.Google Scholar
Witteman, M. J., Weber, A., & McQueen, J. M. (2013). Foreign accent strength and listener familiarity with an accent codetermine speed of perceptual adaptation. Attention, Perception, & Psychophysics, 75, 537–556.Google Scholar
Yi, H. G., Phelps, J. E., Smiljanic, R., & Chandrasekaran, B. (2013). Reduced efficiency of audiovisual integration for nonnative speech. Journal of the Acoustical Society of America, 134, EL387–EL393.Google Scholar
Zheng, Y., & Samuel, A. G. (2017). Does seeing an Asian face make speech sound more accented? Attention, Perception, & Psychophysics, 79, 1841–1859 Google Scholar
Zhou, J. (2014). Managing anxiety: A case study of an international teaching assistant’s interaction with American students. Journal of International Students, 4, 177–190.Google Scholar
Supplementary material: File

Zheng and Samuel supplementary material

Appendices A-C

Download Zheng and Samuel supplementary material(File)
File 1.1 MB