Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-5b777bbd6c-kmmxp Total loading time: 0 Render date: 2025-06-23T12:31:25.354Z Has data issue: false hasContentIssue false

8 - Lending a helping hand to hearing: another motor theory of speech perception

Published online by Cambridge University Press:  01 September 2009

By
Jeremy I. Skipper ,
Howard C. Nusbaum  and
Steven L. Small
Edited by
Michael A. Arbib
Jeremy I. Skipper
Affiliation:
Department of Neurology, The University of Chicago, Chicago, IL 60637, USA
Howard C. Nusbaum
Affiliation:
Department of Psychology and the Brain Research Imaging Center, The University of Chicago, Chicago IL 60637, USA
Steven L. Small
Affiliation:
Departments of Psychology and Neurology and the Brain Research Imaging Center, The University of Chicago, Chicago, IL 60637, USA
Michael A. Arbib
Affiliation:
University of Southern California
Get access

Summary

… any comprehensive account of how speech is perceived should encompass audiovisual speech perception. The ability to see as well as hear has to be integral to the design, not merely a retro-fitted after-thought.

Summerfield (1987)

The “lack of invariance problem” and multisensory speech perception

In speech there is a many-to-many mapping between acoustic patterns and phonetic categories. That is, similar acoustic properties can be assigned to different phonetic categories or quite distinct acoustic properties can be assigned to the same linguistic category. Attempting to solve this “lack of invariance problem” has framed much of the theoretical debate in speech research over the years. Indeed, most theories may be characterized as to how they deal with this “problem.” Nonetheless, there is little evidence for even a single invariant acoustic property that uniquely identifies phonetic features and that is used by listeners (though see Blumstein and Stevens, 1981; Stevens and Blumstein, 1981).

Phonetic constancy can be achieved in spite of this lack of invariance by viewing speech perception as an active process (Nusbaum and Magnuson, 1997). Active processing models like the one to be described here derive from Helmholtz who described visual perception as a process of “unconscious inference” (see Hatfield, 2002). That is, visual perception is the result of forming and testing hypotheses about the inherently ambiguous information available to the retina.

Type
Chapter
Information
Action to Language via the Mirror Neuron System , pp. 250 - 286
Publisher: Cambridge University Press
Print publication year: 2006

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Book purchase

Temporarily unavailable

References

Allison, T., Puce, A., and McCarthy, G., 2000. Social perception from visual cues: role of the STS region. Trends Cogn. Sci. 4: 267–278.CrossRefGoogle ScholarPubMed
Arbib, M., and Rizzolatti, G., 1997. Neural expectations: a possible evolutionary path from manual skills to language. Commun. Cogn. 29: 393–424.Google Scholar
Arbib, M. A., Érdi, P., and Szentágothai, J., 1998. Neural Organization: Structure, Function and Dynamics. Cambridge, MA: MIT PressGoogle Scholar
Awh, E., Smith, E. E., and Jonides, J., 1995. Human rehearsal processes and the frontal lobes: PET evidence. Ann. N. Y. Acad. Sci., 769: 97–117.CrossRefGoogle ScholarPubMed
Beattie, G., and Coughlan, J., 1999. An experimental investigation of the role of iconic gestures in lexical access using the tip-of-the-tongue phenomenon. Br. J. Psychol. 90: 35–56.CrossRefGoogle ScholarPubMed
Belin, P., Zatorre, R. J., Lafaille, P., Ahad, P., and Pike, B., 2000. Voice-selective areas in human auditory cortex. Nature 403: 309–312.CrossRefGoogle ScholarPubMed
Belin, P., Zatorre, R. J., and Ahad, P., 2002. Human temporal-lobe response to vocal sounds. Brain Res. Cogn. Brain Res. 13: 17–26.CrossRefGoogle ScholarPubMed
Bernstein, L. E., Eberhardt, S. P., and Demorest, M. E., 1998. Single-channel vibrotactile supplements to visual perception of intonation and stress. J. Acoust. Soc. America 85: 397–405.CrossRefGoogle Scholar
Bertelson, P., Vroomen, J., and Gelder, B., 2003. Visual recalibration of auditory speech identification: a McGurk aftereffect. Psychol. Sci. 14: 592–597.CrossRefGoogle ScholarPubMed
Binder, J. R., Frost, J. A., Hammeke, T. A., et al., 2000. Human temporal lobe activation by speech and nonspeech sounds. Cereb. Cortex 10: 512–528.CrossRefGoogle ScholarPubMed
Binkofski, F., Amunts, K., Stephan, K. M., et al., 2000. Broca's region subserves imagery of motion: a combined cytoarchitectonic and fMRI study. Hum. Brain Map. 11: 273–285.3.0.CO;2-0>CrossRefGoogle ScholarPubMed
Blumstein, S. E., and Stevens, K. N., 1981. Phonetic features and acoustic invariance in speech. Cognition 10: 25–32.CrossRefGoogle Scholar
Bookheimer, S., 2002. Functional MRI of language: new approaches to understanding the cortical organization of semantic processing. Annu. Rev. Neurosci. 25: 151–188.CrossRefGoogle ScholarPubMed
Bookheimer, S. Y., Zeffiro, T. A., Blaxton, T., Gaillard, W., and Theodore, W., 1995. Regional cerebral blood flow during object naming and word reading. Hum. Brain Map. 3: 93–106.CrossRefGoogle Scholar
Braun, A. R., Guillemin, A., Hosey, L., and Varga, M., 2001. The neural organization of discourse: an H215O-PET study of narrative production in English and American sign language. Brain 124: 2028–2044.CrossRefGoogle Scholar
Buccino, G., Binkofski, F., Fink, G. R., et al., 2001. Action observation activates premotor and parietal areas in a somatotopic manner: an fMRI study. Eur. J. Neurosci. 13: 400–404.Google Scholar
Buccino, G., Lui, F., Canessa, N., et al., 2004. Neural circuits involved in the recognition of actions performed by nonconspecifics: an fMRI study. J. Cogn. Neurosci. 16: 114–126.CrossRefGoogle Scholar
Buchsbaum, B., Hickok, G., and Humphries, C., 2001. Role of left posterior superior temporal gyrus in phonological processing for speech perception and production. Cogn. Sci. 25: 663–678.CrossRefGoogle Scholar
Burton, M. W., Small, S. L., and Blumstein, S. E., 2000. The role of segmentation in phonological processing: an fMRI investigation. J. Cogn. Neurosci. 12: 679–690.CrossRefGoogle Scholar
Callan, D. E., Callan, A. M., Kroos, C., and Vatikiotis-Bateson, E., 2001. Multimodal contribution to speech perception revealed by independent component analysis: a single-sweep EEG case study. Brain Res. Cogn. Brain Res. 10: 349–353.CrossRefGoogle ScholarPubMed
Callan, D. E., Tajima, K., Callan, A. M., et al., 2003. Learning-induced neural plasticity associated with improved identification performance after training of a difficult second-language phonetic contrast. Neuroimage 19: 113–124.CrossRefGoogle ScholarPubMed
Callan, D. E., Jones, J. A., Callan, A. M., and Akahane-Yamada, R., 2004. Phonetic perceptual identification by native- and second-language speakers differentially activates brain regions involved with acoustic phonetic processing and those involved with articulatory-auditory/orosensory internal models. Neuroimage 22: 1182–1194.CrossRefGoogle ScholarPubMed
Calvert, G. A., Campbell, R., and Brammer, M. J., 2000. Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr. Biol. 10: 649–657.CrossRefGoogle ScholarPubMed
Campbell, R., MacSweeney, M., Surguladze, S., et al., 2001. Cortical substrates for the perception of face actions: an fMRI study of the specificity of activation for seen speech and for meaningless lower-face acts (gurning). Brain Res. Cogn. Brain Res. 12: 233–243.CrossRefGoogle Scholar
Chao, L. L., and Martin, A., 2000. Representation of manipulable man-made objects in the dorsal stream. Neuroimage 12: 478–484.CrossRefGoogle ScholarPubMed
Church, R. B., and Goldin-Meadow, S., 1986. The mismatch between gesture and speech as an index of transitional knowledge. Cognition 23: 43–71.CrossRefGoogle ScholarPubMed
Cooper, W. E., 1979. Speech Perception and Production: Studies in Selective Adaptation. Norwood, NJ: Ablex.Google Scholar
Corina, D. P., McBurney, S. L., Dodrill, C., et al., 1999. Functional roles of Broca's area and SMG: evidence from cortical stimulation mapping in a deaf signer. Neuroimage 10: 570–581.CrossRefGoogle Scholar
Damasio, H., Grabowski, T. J., Tranel, D., Hichwa, R. D., and Damasio, A. R., 1996. A neural basis for lexical retrieval. Nature 380: 499–505.CrossRefGoogle ScholarPubMed
Dehaene, S., Dupoux, E., Mehler, J., et al., 1997. Anatomical variability in the cortical representation of first and second language. Neuroreport 8: 3809–3815.CrossRefGoogle ScholarPubMed
Devlin, J. T., Matthews, P. M., and Rushworth, M. F., 2003. Semantic processing in the left inferior prefrontal cortex: a combined functional magnetic resonance imaging and transcranial magnetic stimulation study. J. Cogn. Neurosci. 15: 71–84.CrossRefGoogle ScholarPubMed
Diehl, R. L., Lotto, A. J., and Holt, L. L., 2004. Speech perception. Annu. Rev. Psychol. 55: 149–179.CrossRefGoogle ScholarPubMed
Dodd, B., 1977. The role of vision in the perception of speech. Perception 6: 31–40.CrossRefGoogle Scholar
Eliades, S. J., and Wang, X., 2003. Sensory-motor interaction in the primate auditory cortex during self-initiated vocalizations. J. Neurophysiol. 89: 2194–2207.CrossRefGoogle ScholarPubMed
Embick, D., Marantz, A., Miyashita, Y., O'Neil, W., and Sakai, K. L., 2000. A syntactic specialization for Broca's area. Proc. Natl. Acad. Sci. USA 97: 6150–6154.CrossRefGoogle ScholarPubMed
Fadiga, L., Craighero, L., Buccino, G., and Rizzolatti, G., 2002. Speech listening specifically modulates the excitability of tongue muscles: a tms study. Eur. J. Neurosci. 15: 399–402.CrossRefGoogle ScholarPubMed
Federmeier, K. D., and Kutas, M., 2001. Meaning and modality: influences of context, semantic memory organization, and perceptual predictability on picture processing. J. Exp. Psychol. Learn. Mem. Cogn. 27: 202–224.CrossRefGoogle ScholarPubMed
Ferrari, P. F., Gallese, V., Rizzolatti, G., and Fogassi, L., 2003. Mirror neurons responding to the observation of ingestive and communicative mouth actions in the monkey ventral premotor cortex. Eur. J. Neurosci. 17: 1703–1714.CrossRefGoogle ScholarPubMed
Fogassi, L., Gallese, V., Fadiga, L., and Rizzolatti, G., 1998. Neurons responding to the sight of goal directed hand/arm actions in the parietal area PF (7b) of the macaque monkey. Soc. Neurosci. Abstr. 24, 257.Google Scholar
Gallese, V., Fadiga, L., Fogassi, L., and Rizzolatti, G., 1996. Action recognition in the premotor cortex. Brain 119: 593–609.CrossRefGoogle ScholarPubMed
Gentilucci, M., Santunione, P., Roy, A. C., and Stefanini, S., 2004a. Execution and observation of bringing a fruit to the mouth affect syllable pronunciation. Eur. J. Neurosci. 19: 190–202.CrossRefGoogle ScholarPubMed
Gentilucci, M., Stefanini, S., Roy, A. C., and Santunione, P., 2004b. Action observation and speech production: study on children and adults. Neuropsychologia 42: 1554–1567.CrossRefGoogle ScholarPubMed
Gerlach, C., Law, I., and Paulson, O. B., 2002. When action turns into words: activation of motor-based knowledge during categorization of manipulable objects. J. Cogn. Neurosci. 14: 1230–1239.CrossRefGoogle ScholarPubMed
Geschwind, N., 1965. The organization of language and the brain. Science 170: 940–944.CrossRefGoogle Scholar
Gold, B. T., and Buckner, R. L., 2002. Common prefrontal regions coactivate with dissociable posterior regions during controlled semantic and phonological tasks. Neuron 35: 803–812.CrossRefGoogle ScholarPubMed
Goodale, M. A., and Milner, A. D., 1992. Separate visual pathways for perception and action. Trends Neurosci. 15: 20–25.CrossRefGoogle ScholarPubMed
Grafton, S. T., Fadiga, L., Arbib, M. A., and Rizzolatti, G., 1997. Premotor cortex activation during observation and naming of familiar tools. Neuroimage 6: 231–236.CrossRefGoogle ScholarPubMed
Grant, K. W., and Greenberg, S., 2001. Speech intelligibility derived from asynchronous processing of auditory-visual information. Proceedings of the Workshop on Audio-Visual Speech Processing, Scheelsminde, Denmark, pp. 132–137.
Guenther, F. H., and Ghosh, S. S., 2003. A model of cortical and cerebellar function in speechz. Proceedings of the 15th International Congress of Phonetic Sciences, pp. 169–173.Google Scholar
Guenther, F. H., and Perkell, J. S., 2004. A neural model of speech production and its application to studies of the role of auditory feedback in speech. In Maassen, B., Kent, R., Peters, H., Lieshout, P., and Hulstijn, W. (eds.) Speech Motor Control In Normal and Disordered Speech.Oxford, UK: Oxford University Press, pp. 29–49.Google Scholar
Hasegawa, M., Carpenter, P. A., and Just, M. A., 2002. An fMRI study of bilingual sentence comprehension and workload. Neuroimage 15: 647–660.CrossRefGoogle ScholarPubMed
Hashimoto, Y., and Sakai, K. L., 2003. Brain activations during conscious self-monitoring of speech production with delayed auditory feedback: an fMRI study. Hum. Brain Map. 20: 22–28.CrossRefGoogle Scholar
Hatfield, G., 2002. Perception as unconscious inference. In Heyer, D. and Mausfield, R. (eds.) Perception and the Physical World: Psychological and Philosophical Issues in Perception. New York: John Wiley, pp. 115–143.CrossRef
Haxby, J. V., Gobbini, M. I., Furey, M. L., et al., 2001. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 293: 2425–2430.CrossRefGoogle ScholarPubMed
Heim, S., Opitz, B., Muller, K., and Friederici, A. D., 2003. Phonological processing during language production: fMRI evidence for a shared production-comprehension network. Brain Res. Cogn. Brain Res. 16: 285–296.CrossRefGoogle ScholarPubMed
Hickok, G., and Poeppel, D., 2004. Dorsal and ventral streams: a framework for understanding aspects of the functional anatomy of language. Cognition 92: 67–99.CrossRefGoogle ScholarPubMed
Hickok, G., Erhard, P., Kassubek, J., et al., 2000. A functional magnetic resonance imaging study of the role of left posterior superior temporal gyrus in speech production: implications for the explanation of conduction aphasia. Neurosci. Lett. 287: 156–160.CrossRefGoogle ScholarPubMed
Houde, J. F., and Jordan, M. I., 1998. Sensorimotor adaptation in speech production. Science 279: 1213–1216.CrossRefGoogle ScholarPubMed
Houde, J. F., Nagarajan, S. S., Sekihara, K., and Merzenich, M. M., 2002. Modulation of the auditory cortex during speech: an meg study. J. Cogn. Neurosci. 14: 1125–1138.CrossRefGoogle Scholar
Humphries, C., Willard, K., Buchsbaum, B., and Hickok, G., 2001. Role of anterior temporal cortex in auditory sentence comprehension: an fMRI study. Neuroreport 12: 1749–1752.CrossRefGoogle Scholar
Iacoboni, M., 2005. Understanding others: imitation, language, empathy. In Hurley, S. and Chater, N. (eds.) Perspectives on Imitation: From Cognitive Neuroscience to Social Science, vol. 1, Mechanisms of Imitation and Imitation in Animals. Cambridge, MA: MIT Press, pp. 77–99.Google Scholar
Iacoboni, M., Woods, R. P., Brass, M., et al., 1999. Cortical mechanisms of human imitation. Science 286: 2526–2528.CrossRefGoogle ScholarPubMed
Iacoboni, M., Koski, L. M., Brass, M., et al., 2001. Reafferent copies of imitated actions in the right superior temporal cortex. Proc. Natl Acad. Sci. USA 98: 13995–13999.CrossRefGoogle ScholarPubMed
Jancke, L., Wustenberg, T., Scheich, H., and Heinze, H. J., 2002. Phonetic perception and the temporal cortex. Neuroimage 15: 733–746.CrossRefGoogle ScholarPubMed
Jeannerod, M., 1997. The Cognitive Neuroscience of Action. Oxford, UK: Blackwell.Google Scholar
Jones, J. A., and Callan, D. E., 2003. Brain activity during audiovisual speech perception: an fMRI study of the McGurk effect. Neuroreport 14: 1129–1133.CrossRefGoogle ScholarPubMed
Jonides, J., Schumacher, E. H., Smith, E. E., et al., 1998. The role of parietal cortex in verbal working memory. J. Neurosci. 18: 5026–5034.CrossRefGoogle ScholarPubMed
Jordan, M., and Rumelhart, D., 1992. Forward models: supervised learning with a distal teacher. Cogn. Sci. 16: 307–354.CrossRefGoogle Scholar
Josse, G., Suriyakham, L. W., Skipper, J. I., et al., 2005. Language-associated gesture modulates areas of the brain involved in action observation and language comprehension. Poster presented at The Organization for Human Brain Mapping, Toronto, Canada.Google Scholar
Jusczyk, P. W., 1981. Infant speech perception: a critical appraisal. In Eimas, P. D. and Miller, J. L. (eds.) Perspectives on the Study of Speech. Hillsdale, NJ: Lawrence Erlbaum, pp. 113–164.Google Scholar
Kaas, J. H., and Hackett, T. A., 2000. Subdivisions of auditory cortex and processing streams in primates. Proc. Natl Acad. Sci. USA 97: 11793–11799.CrossRefGoogle ScholarPubMed
Kahl, R. (ed.), 1971. Selected Writings of Hermann von Helmholtz. Middletown, CT: Wesleyan University Press.Google Scholar
Kendon, A., 1987. On gesture: its complementary relationship with speech. In Siegman, A. and Feldstein, S. (eds.) Nonverbal Communication.Hillsdale, NJ: Lawrence Erlbaum, pp. 65–97.Google Scholar
Kendon, A. 1994. Do gestures communicate? A review. Res. Lang. Soc. Interact. 27: 175–200.CrossRefGoogle Scholar
Kluender, K. R., Diehl, R. L., and Killeen, P. R., 1987. Japanese quail can learn phonetic categories. Science 237: 1195–1197.CrossRefGoogle ScholarPubMed
Kohler, E., Keysers, C., Umiltá, M. A., et al., 2002. Hearing sounds, understanding actions: action representation in mirror neurons. Science 297: 846–848.CrossRefGoogle ScholarPubMed
Koski, L., Wohlschlager, A., Bekkering, H., et al., 2002. Modulation of motor and premotor activity during imitation of target-directed actions. Cereb. Cortex 12: 847–855.CrossRefGoogle ScholarPubMed
Kuhl, P. K., and Meltzoff, A. N., 1982. The bimodal perception of speech in infancy. Science 218: 1138–1141.CrossRefGoogle ScholarPubMed
Liberman, A. M., and Mattingly, I. G., 1985. The motor theory of speech perception revised. Cognition 21: 1–36.CrossRefGoogle ScholarPubMed
MacSweeney, M., Woll, B., Campbell, R., et al., 2002. Neural correlates of British sign language comprehension: spatial processing demands of topographic language. J. Cogn. Neurosci. 14: 1064–1075.CrossRefGoogle ScholarPubMed
Marslen-Wilson, W., and Welsh, A., 1978. Processing interactions and lexical access during word recognition in continuous speech. Cogn. Psychol. 10: 29–63.CrossRefGoogle Scholar
Martin, A., Haxby, J. V., Lalonde, F. M., Wiggs, C. L., and Ungerleider, L. G., 1995. Discrete cortical regions associated with knowledge of color and knowledge of action. Science 270: 102–105.CrossRefGoogle Scholar
Martin, A., Wiggs, C. L., Ungerleider, L. G., and Haxby, J. V., 1996. Neural correlates of category-specific knowledge. Nature 379: 649–652.CrossRefGoogle ScholarPubMed
Massaro, D. W., 1998. Perceiving Talking Faces: From Speech Perception to a Behavioral Principle. Cambridge, MA: MIT Press.Google Scholar
McGurk, H., and MacDonald, J., 1976. Hearing lips and seeing voices. Nature 264: 746–748.CrossRefGoogle ScholarPubMed
McNeill, D., 1992. Hand and Mind: What Gestures Reveal about Thought. Chicago, IL: University of Chicago Press.Google Scholar
Miall, R. C., 2003. Connecting mirror neurons and forward models. Neuroreport 14: 2135–2137.CrossRefGoogle ScholarPubMed
Miller, J. D., 1977. Perception of speech sounds by animals: evidence for speech processing by mammalian auditory mechanisms. In Bullock, T. H. (ed.) Recognition of Complex Acoustic Signals. Berlin: Dahlem Konferenzen, pp. 49–58.Google Scholar
Moore, C. J., and Price, C. J., 1999. A functional neuroimaging study of the variables that generate category-specific object processing differences. Brain 122: 943–962.CrossRefGoogle ScholarPubMed
Mottonen, R., Krause, C. M., Tiippana, K., and Sams, M., 2002. Processing of changes in visual speech in the human auditory cortex. Brain Res. Cogn. Brain Res. 13: 417–425.CrossRefGoogle ScholarPubMed
Mummery, C. J., Patterson, K., Hodges, J. R., and Price, C. J., 1998. Functional neuroanatomy of the semantic system: divisible by what?J. Cogn. Neurosci. 10: 766–777.CrossRefGoogle Scholar
Munhall, K. G., Gribble, P., Sacco, L., and Ward, M., 1996. Temporal constraints on the McGurk effect. Percept. Psychophys. 58: 351–362.CrossRefGoogle ScholarPubMed
Munhall, K. G., Jones, J. A., Callan, D. E., Kuratate, T., and Vatikiotis-Bateson, E., 2004. Visual prosody and speech intelligibility: head movement improves auditory speech perception. Psychol. Sci. 15: 133–137.CrossRefGoogle ScholarPubMed
Nicholson, K. G., Baum, S., Cuddy, L. L., and Munhall, K. G., 2002. A case of impaired auditory and visual speech prosody perception after right hemisphere damage. Neurocase 8: 314–322.CrossRefGoogle ScholarPubMed
Numminen, J., Salmelin, R., and Hari, R., 1999. Subject's own speech reduces reactivity of the human auditory cortex. Neurosci. Lett. 265: 119–122.CrossRefGoogle ScholarPubMed
Nusbaum, H. C., and Magnuson, J., 1997. Talker normalization: phonetic constancy as a cognitive process. In Johnson, K. and Mullennix, J. W. (eds.) Talker Variability in Speech Processing.San Diego, CA: Academic Press, pp. 109–132.Google Scholar
Nusbaum, H. C., and Morin, T. M., 1992. Paying attention to differences among talkers. In Tohkura, Y., Sagisaka, Y., and Vatikiotis-Bateson, E. (eds.) Speech Perception, Production, and Linguistic Structure.Tokyo: Ohmasha Publishing, pp. 113–134.Google Scholar
Nusbaum, H. C., and Schwab, E. C., 1986. The role of attention and active processing in speech perception. In Schwab, E. C. and Nusbaum, H. C. (eds.) Pattern Recognition by Humans and Machines, vol. 1, Speech Perception.New York: Academic Press, pp. 113–157.Google Scholar
Olson, I. R., Gatenby, J. C., and Gore, J. C., 2002. A comparison of bound and unbound audio-visual information processing in the human cerebral cortex. Brain Res. Cogn. Brain Res. 14: 129–138.CrossRefGoogle ScholarPubMed
Paulesu, E., Frith, C. D., and Frackowiak, R. J., 1993a. The neural correlates of the verbal component of working memory. Nature 362: 342–345.CrossRefGoogle ScholarPubMed
Paulesu, P., Frith, C. D., Bench, C. J., et al., 1993b. Functional anatomy of working memory: The articulatory loop. J. Cereb. Blood Flow Metab. 13: 551.Google Scholar
Paus, T., Perry, D. W., Zatorre, R. J., Worsley, K. J., and Evans, A. C., 1996. Modulation of cerebral blood flow in the human auditory cortex during speech: role of motor-to-sensory discharges. Eur. J. Neurosci. 8: 2236–2246.CrossRefGoogle ScholarPubMed
Perani, D., Schnur, T., Tettamanti, M., et al., 1999. Word and picture matching: a PET study of semantic category effects. Neuropsychologia 37: 293–306.CrossRefGoogle ScholarPubMed
Perkell, J. S., Matthies, M. L., Svirsky, M. A., and Jordan, M. I., 1995. Goal-based speech motor control: a theoretical framework and some preliminary data. J. Phonet. 23: 23–35.CrossRefGoogle Scholar
Petrides, M., and Pandya, D. N., 1999. Dorsolateral prefrontal cortex: comparative cytoarchitectonic analysis in the human and the macaque brain and corticocortical connection patterns. Eur. J. Neurosci. 11: 1011–1036.CrossRefGoogle ScholarPubMed
Petrides, M., and Pandya, D. N. 2002. Comparative cytoarchitectonic analysis of the human and the macaque ventrolateral prefrontal cortex and corticocortical connection patterns in the monkey. Eur. J. Neurosci. 16: 291–310.CrossRefGoogle ScholarPubMed
Plaut, D. C., and Kello, C. T., 1999. The emergence of phonology from the interplay of speech comprehension and production: a distributed connectionist approach. In MacWhinney, B. (ed.) The Emergence of Language.Mahwah, NJ: Lawrence Erlbaum, pp. 381–415.Google Scholar
Poldrack, R. A., Wagner, A. D., Prull, M. W., et al., 1999. Functional specialization for semantic and phonological processing in the left inferior prefrontal cortex. Neuroimage 10: 15–35.CrossRefGoogle ScholarPubMed
Posner, M. I., and DiGirolamo, G. J., 2000. Attention in cognitive neuroscience: an overview. In Gazzaniga, M. S. (ed.) The New Cognitive Neuroscience, 2nd edn. Cambridge, MA: MIT Press, pp. 621–632.Google Scholar
Pulvermüller, F., Harle, M., and Hummel, F., 2000. Neurophysiological distinction of verb categories. Neuroreport 11: 2789–2793.CrossRefGoogle ScholarPubMed
Pulvermüller, F., Harle, M., and Hummel, F. 2001. Walking or talking? Behavioral and neurophysiological correlates of action verb processing. Brain Lang. 78: 43–168.CrossRefGoogle ScholarPubMed
Rauschecker, J. P., and Tian, B., 2000. Mechanisms and streams for processing of “what” and “where” in auditory cortex. Proc. Natl Acad. Sci. USA 97: 11800–11806.CrossRefGoogle Scholar
Records, N. L., 1994. A measure of the contribution of a gesture to the perception of speech in listeners with aphasia. J. Speech Hear. Res. 37: 1086–1099.CrossRefGoogle ScholarPubMed
Reisberg, D., McLean, J., and Goldfield, A., 1987. Easy to hear but hard to understand: a lipreading advantage with intact auditory stimuli. In Dodd, B. and Campbell, R. (eds.) Hearing by Eye: The Psychology of Lipreading.Hillsdale, NJ: Lawrence Erlbaum, pp. 97–114.Google Scholar
Rémez, R. E., Rubin, P. E., Pisoni, D. B., and Carrell, T. I., 1981. Speech perception without traditional speech cues. Science 212: 947–950.CrossRefGoogle ScholarPubMed
Renwick, M., Shattuck-Hufnagel, S., and Yasinnik, Y., 2001. The timing of speech-accompanying gestures with respect to prosody. J. Acoust. Soc. America 115: 2397–2397.CrossRefGoogle Scholar
Riecker, A., Ackermann, H., Wildgruber, D., et al., 2000. Articulatory/phonetic sequencing at the level of the anterior perisylvian cortex: a functional magnetic resonance imaging (fMRI) study. Brain Lang. 75: 259–276.CrossRefGoogle ScholarPubMed
Risberg, A., and Lubker, J., 1978. Prosody and speechreading. Speech Transmission Lab. Q. Progr. Rep. Status Report 4: 1–16.Google Scholar
Rizzolatti, G., and Craighero, L., 2004. The mirror-neuron system. Annu. Rev. Neurosci. 27: 169–192.CrossRefGoogle ScholarPubMed
Rizzolatti, G., Fogassi, L., and Gallese, V., 2002. Motor and cognitive functions of the ventral premotor cortex. Curr. Opin. Neurobiol. 12: 149–154.CrossRefGoogle ScholarPubMed
Roberts, M., and Summerfield, Q., 1981. Audiovisual presentation demonstrates that selective adaptation in speech perception is purely auditory. Percept. Psychophys. 30: 309–314.CrossRefGoogle ScholarPubMed
Rogers, W. T., 1978. The contribution of kinesic illustrators towards the comprehension of verbal behavior within utterances. Hum. Commun. Res. 5: 54–62.CrossRefGoogle Scholar
Romanski, L. M., Averbeck, B. B., and Diltz, M., 2005. Neural representation of vocalizations in the primate ventrolateral prefrontal cortex. J. Neurophysiol. 93: 734–747.CrossRefGoogle ScholarPubMed
Saldaña, H. M., and Rosenblum, L. D., 1994. Selective adaptation in speech perception using a compelling audiovisual adaptor. J. Acoust. Soc. America 95: 3658–3661.CrossRefGoogle ScholarPubMed
Sams, M., Aulanko, R., Hamalainen, M., et al., 1991. Seeing speech: visual information from lip movements modifies activity in the human auditory cortex. Neurosci. Lett. 127: 141–145.CrossRefGoogle ScholarPubMed
Schumacher, E. H., Lauber, E., Awh, E., et al., 1996. PET evidence for an amodal verbal working memory system. Neuroimage 3: 79–88.CrossRefGoogle ScholarPubMed
Scott, S. K., and Wise, R. J. S., 2003. PET and fMRI studies of the neural basis of speech perception. Speech Commun. 41: 23–34.CrossRefGoogle Scholar
Seltzer, B., and Pandya, D. N., 1978. Afferent cortical connections and architectonics of the superior temporal sulcus and surrounding cortex in the rhesus monkey. Brain Res. 149: 1–24.CrossRefGoogle ScholarPubMed
Shergill, S. S., Brammer, M. J., Fukuda, R., et al., 2002. Modulation of activity in temporal cortex during generation of inner speech. Hum. Brain Map. 16: 219–227.CrossRefGoogle ScholarPubMed
Singer, M. A., and Goldin-Meadow, S., 2005. Children learn when their teacher's gestures and speech differ. Psychol. Sci. 16: 85–89.CrossRefGoogle ScholarPubMed
Skipper, J. I., Wassenhove, V., Nusbaum, H. C., and Small, S. L., 2004. Hearing lips and seeing voices in the brain: motor mechanisms of speech perception. Poster presented at 11th Annual Meeting of the Cognitive Neuroscience Society, San Francisco, CA.
Skipper, J. I., Nusbaum, H. C., and Small, S. L., 2005a. Listening to talking faces: motor cortical activation during speech perception. Neuroimage 25: 76–89.CrossRefGoogle ScholarPubMed
Skipper, J. I., Nusbaum, H. C., Wassenhove, V., et al., 2005b. The role of ventral premotor and primary motor cortex in audiovisual speech perception. Poster presented at The Organization for Human Brain Mapping, Toronto, Canada.
Skipper, J. I., Wymbs, N. F., Lobo, N., Cherney, L. R., and Small, S. L., 2005c. Common motor circuitry for audiovisual speech imitation and audiovisual speech perception. Poster presented at The Organization for Human Brain Mapping, Toronto, Canada.Google Scholar
Stevens, K. N., and Blumstein, S. E., 1981. The search for invariant acoustic correlates of phonetic features. In Eimas, P. D. and Miller, J. L. (eds.) Perspectives on the Study of Speech.Hillsdale, NJ: Lawrence Erlbaum, pp. 1–39.Google Scholar
Stevens, K. N., and Halle, M., 1967. Remarks on analysis by synthesis and distinctive features. In Walthen-Dunn, W. (ed.) Models for the Perception of Speech and Visual Form. Cambridge, MA: MIT Press, pp. 88– 102.Google Scholar
Stromswold, K., Caplan, D., Alpert, N., and Rauch, S., 1996. Localization of syntactic comprehension by positron emission tomography. Brain Lang. 52: 452–473.CrossRefGoogle ScholarPubMed
Sumby, W. H., and Pollack, I., 1954. Visual contribution of speech intelligibility in noise. J. Acoust. Soc. America 26: 212–215.CrossRefGoogle Scholar
Summerfield, A. Q., 1987. Some preliminaries to a comprehensive account of audio-visual speech perception. In Dodd, B. and Campbell, R. (eds.) Hearing by Eye: The Psychology of Lip Reading.Hillsdale, NJ: Lawrence Erlbaum, pp. 3–51.Google Scholar
Sundara, M., Namasivayam, A. K., and Chen, R., 2001. Observation-execution matching system for speech: a magnetic stimulation study. Neuroreport 12: 1341–1344.CrossRefGoogle ScholarPubMed
Surguladze, S. A., Calvert, G. A., Brammer, M. J., et al., 2001. Audio-visual speech perception in schizophrenia: an fMRI study. Psychiatry Res. 106: 1–14.CrossRefGoogle Scholar
Tranel, D., 2001. Combs, ducks, and the brain. Lancet 357: 1818–1819.CrossRefGoogle Scholar
Tzourio, N., Nkanga-Ngila, B., and Mazoyer, B., 1998. Left planum temporale surface correlates with functional dominance during story listening. Neuroreport 9: 829–833.CrossRefGoogle ScholarPubMed
Ungerleider, L. G., and Mishkin, M., 1982. Two cortical visual systems. In Ingle, D. J., Goodale, M. A., and Mansfield, R. J. W. (eds.) Analysis of Visual Behavior.Cambridge, MA: MIT Press, pp. 549–586.Google Scholar
Vroomen, J., Linden, S., Keetels, M., Gelder, B., and Bertelson, P., 2004. Selective adaptation and recalibration of auditory speech by lipread information: dissipation. Speech Commun. 44: 55–61.CrossRefGoogle Scholar
Wagner, A. D., Koutstaal, W., Maril, A., Schacter, D. L., and Buckner, R. L., 2000. Task-specific repetition priming in left inferior prefrontal cortex. Cereb. Cortex 10: 1176–1184.CrossRefGoogle ScholarPubMed
Watkins, K. E., Strafella, A. P., and Paus, T., 2003. Seeing and hearing speech excites the motor system involved in speech production. Neuropsychologia 41: 989–994.CrossRefGoogle ScholarPubMed
Wildgruber, D., Ackermann, H., and Grodd, W., 2001. Differential contributions of motor cortex, basal ganglia, and cerebellum to speech motor control: effects of syllable repetition rate evaluated by fMRI. Neuroimage 13: 101–109.CrossRefGoogle ScholarPubMed
Wilson, S. M., Saygin, A. P., Sereno, M. I., and Iacoboni, M., 2004. Listening to speech activates motor areas involved in speech production. Nature Neurosci. 7: 701–702.CrossRefGoogle ScholarPubMed
Wise, R. J., Greene, J., Buchel, C., and Scott, S. K., 1999. Brain regions involved in articulation. Lancet 353: 1057–1061.CrossRefGoogle ScholarPubMed
Wise, R. J., Scott, S. K., Blank, S. C., et al., 2001. Separate neural subsystems within ‘Wernicke's area’. Brain 124: 83–95.CrossRefGoogle ScholarPubMed
Wolpert, D. M., and Kawato, M., 1998. Multiple paired forward and inverse models for motor control. Neur. Networks 11: 1317–1329.CrossRefGoogle ScholarPubMed
Wright, T. M., Pelphrey, K. A., Allison, T., McKeown, M. J., and McCarthy, G., 2003. Polysensory interactions along lateral temporal regions evoked by audiovisual speech. Cereb. Cortex 13: 1034–1043.CrossRefGoogle ScholarPubMed
Wymbs, N. F., Nusbaum, H. C., and Small, S. L., 2004. The informed perceiver: neural correlates of linguistic expectation and speech perception. Poster presented at 11th Annual Meeting of the Cognitive Neuroscience Society, San Francisco, CA.Google Scholar
Zatorre, R. J., and Belin, P., 2001. Spectral and temporal processing in human auditory cortex. Cereb. Cortex 11: 946–953.CrossRefGoogle ScholarPubMed
Zatorre, R. J., Meyer, E., Gjedde, A., and Evans, A. C., 1996. PET studies of phonetic processing of speech: review, replication, and reanalysis. Cereb. Cortex 6: 21–30.CrossRefGoogle ScholarPubMed

Save book to Kindle

To save this book to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×