Skip to main content Accessibility help
×
Home

The Bilingual Language Interaction Network for Comprehension of Speech*

  • ANTHONY SHOOK (a1) and VIORICA MARIAN (a1)

Abstract

During speech comprehension, bilinguals co-activate both of their languages, resulting in cross-linguistic interaction at various levels of processing. This interaction has important consequences for both the structure of the language system and the mechanisms by which the system processes spoken language. Using computational modeling, we can examine how cross-linguistic interaction affects language processing in a controlled, simulated environment. Here we present a connectionist model of bilingual language processing, the Bilingual Language Interaction Network for Comprehension of Speech (BLINCS), wherein interconnected levels of processing are created using dynamic, self-organizing maps. BLINCS can account for a variety of psycholinguistic phenomena, including cross-linguistic interaction at and across multiple levels of processing, cognate facilitation effects, and audio-visual integration during speech comprehension. The model also provides a way to separate two languages without requiring a global language-identification system. We conclude that BLINCS serves as a promising new model of bilingual spoken language comprehension.

Copyright

Corresponding author

Address for correspondence: Anthony Shook, Department of Communication Sciences and Disorders, Northwestern University, 2240 Campus Drive, Evanston, IL 60208, USAa-shook@northwestern.edu

Footnotes

Hide All
*

The authors would like to thank the members of the Northwestern Bilingualism and Psycholinguistics Laboratory, as well as Dr. Ping Li and two anonymous reviewers for their helpful comments. This research was funded in part by grant R01HD059858 to the second author, and the John D. and Lucille H. Clarke Scholarship to the first author.

Footnotes

References

Hide All
Allopenna, P., Magnuson, J., & Tanenhaus, M. (1998). Tracking the time course of spoken word recognition using eye movements: Evidence for continuous mapping models. Journal of Memory and Language, 38, 419439.
Ameel, E., Storms, G., Malt, B., & Sloman, S. (2005). How bilinguals solve the naming problem. Journal of Memory and Language, 52, 309329.
Bartolotti, J., & Marian, V. (2012). Language learning and control in monolinguals and bilinguals. Cognitive Science, 36 (6), 11291147.
Bates, E., Devescovi, A., & Wulfeck, B. (2001). Psycholinguistics: A cross-language perspective. Annual Review of Psychology, 52 (May 2000), 369396.
Bitan, T., Burman, D. D., Chou, T.-L., Lu, D., Cone, N. E., Cao, F., Bigio, J. D., & Booth, J. R. (2007). The interaction between orthographic and phonological information in children: An fMRI study. Human Brain Mapping, 28 (9), 880891.
Blumenfeld, H., & Marian, V. (2007). Constraints on parallel activation in bilingual spoken language processing: Examining proficiency and lexical status using eye-tracking. Language and Cognitive Processes, 22 (5), 633660.
Brysbaert, M., & New, B. (2009). Moving beyond Kucera and Francis: A critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English. Behavioral Research Methods, 41 (4), 977990.
Burgess, C., & Lund, K. (1997). Modelling parsing constraints with high-dimensional context space. Language & Cognitive Processes, 12 (2), 177210.
Chater, N., & Christiansen, M. H. (2008). Computational models in psycholinguistics. In Sun, R. (ed.), Cambridge handbook of computational cognitive modeling, pp. 477504. New York: Cambridge University Press.
Cleland, A. A., Gaskell, M. G., Quinlan, P. T., & Tamminen, J. (2006). Frequency effects in spoken and visual word recognition: Evidence from dual-task methodologies. Journal of Experimental Psychology: Human Perception and Performance, 32 (1), 104119.
Costa, A., Caramazza, A., & Sebastián-Gallés, N. (2000). The cognate facilitation effect: Implications for models of lexical access. Journal of Experimental Psychology: Learning, Memory, and Cognition, 26 (5), 12831296.
Costa, A., & Santesteban, M. (2004). Lexical access in bilingual speech production: Evidence from language switching in highly proficient bilinguals and L2 learners. Journal of Memory and Language, 50 (4), 491511.
Costa, A., Santesteban, M., & Ivanova, I. (2006). How do highly proficient bilinguals control their lexicalization process? Inhibitory and language-specific selection mechanisms are both functional. Journal of Experimental Psychology: Learning, Memory, and Cognition, 32, 10571074.
Cuetos, F., Glez-Nosti, M., Barbón, A., & Brysbaert, M. (2011). SUBTLEX-ESP: Spanish word frequencies based on film subtitles. Psicológica, 32, 133143.
De Groot, A. M. B., & Kroll, J. (eds.) (1997). Tutorials in bilingualism: Psycholinguistic perspectives. Mahwah, NJ: Lawrence Erlbaum.
Dijkstra, T., Grainger, J., & van Heuven, W. J. B. (1999). Recognition of cognates and interlingual homographs: The neglected role of phonology. Journal of Memory and Language, 41, 496518.
Dijkstra, T., & van Heuven, W. J. B. (1998). The BIA model and bilingual word recognition. In Grainger, J. & Jacobs, A. M. (eds.), Localist connectionist approaches to human cognition, pp. 189225. Mahwah, NJ: Lawrence Erlbaum
Dijkstra, T., & van Heuven, W. J. B. (2002). The architecture of the bilingual word recognition system: From identification to decision. Bilingualism: Language and Cognition, 5 (3), 175197.
Dong, Y., Gui, Sh., & MacWhinney, B. (2005). Shared and separate meanings in the bilingual mental lexicon. Bilingualism: Language and Cognition, 8 (3), 221238.
Finkbeiner, M., Almeida, J., Janssen, N., & Caramazza, A. (2006). Lexical selection in bilingual speech production does not involve language suppression. Journal of Experimental Psychology: Learning, Memory, and Cognition, 32 (5), 10751089.
Finkbeiner, M., Forster, K., Nicol, J., & Nakamura, K. (2004). The role of polysemy in masked semantic and translation priming. Journal of Memory and Language, 51 (1), 122.
FitzPatrick, I., & Indefrey, P. (2010). Lexical competition in nonnative speech comprehension. Journal of Cognitive Neuroscience, 22 (6), 11651178.
Forster, K. I. (1976). Accessing the mental lexicon. In Wales, R. J. & Walker, E. (eds.), New approaches to language mechanisms, pp. 257287. Amsterdam: North-Holland.
Gentilucci, M., & Cattaneo, L. (2005). Automatic audiovisual integration in speech perception. Experimental Brain Research, 167 (1), 6675.
Gollan, T. H., & Ferreira, V. S. (2009). Should I stay or should I switch? A cost-benefit analysis of voluntary language switching in young and aging bilinguals. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35 (3), 640665.
Graham, R., & Belnap, K. (1986). The acquisition of lexical boundaries in English by native speakers of Spanish. International Review of Applied Linguistics, 24, 275286.
Grainger, J., & Dijkstra, T. (1992). On the representation and use of language information in bilinguals. In Harris, R. J. (ed.), Cognitive processing in bilinguals (vol. 83), pp. 207220. Amsterdam: Elsevier.
Grasemann, U., Kiran, S., Sandberg, C., & Miikkulainen, R. (2011). In Laaksonen, J. & Honkela, T. (eds.), Proceedings of WSOM11, 8th Workshop on Self-organizing Maps, pp. 207217. Espoo: Springer Verlag.
Green, D. W. (1998). Mental control of the bilingual lexico-semantic system. Bilingualism: Language and Cognition, 1, 6782.
Grosjean, F. (1988). Exploring the recognition of guest words in bilingual speech. Language and Cognitive Processes, 3, 233274.
Grosjean, F. (1997). Processing mixed languages: Issues, findings and models. In De Groot, & Kroll, (eds.), pp. 225–254.
Hartsuiker, R. J., Pickering, M. J., & Veltkamp, E. (2004). Is syntax separate or shared between languages? Cross-linguistic syntactic priming in Spanish–English bilinguals. Psychological Science, 15 (6), 409414.
Huettig, F., & Altmann, G. T. M. (2005). Word meaning and the control of eye fixation: Semantic competitor effects and the visual world paradigm. Cognition, 96, B2332.
Huettig, F., & McQueen, J. M. (2007). The tug of war between phonological, semantic and shape information in language-mediated visual search. Journal of Memory and Language, 57, 460482.
International Phonetic Association. (1999). Handbook of the International Phonetic Association: A guide to the use of the International Phonetic Alphabet. Cambridge: Cambridge University Press.
Ju, M., & Luce, P. A. (2004). Falling on sensitive ears. Psychological Science, 15, 314318.
Jurgens, D., & Stevens, K. (2010). The S-Space package: An open source package for word space models. Proceedings of the ACL 2010 System Demonstrations, pp. 30–35. Association for Computational Linguistics.
Kanekama, Y., & Downs, D. (2009). Effects of speechreading and signal-to-noise ratio on understanding American English by American and Indian adults. Journal of the Acoustical Society of America, 126 (4), 2314.
Kaushanskaya, M., & Marian, V. (2007). Non-target language recognition and interference: Evidence from eye-tracking and picture naming. Language Learning, 57 (1), 119163.
Kaushanskaya, M, Yoo, J., & Marian, V. (2011). The effect of second-language experience on native-language processing. Vigo International Journal of Applied Linguistics, 8, 5477.
Knoeferle, P., Crocker, M. W., Scheepers, C., & Pickering, M. J. (2005). The influence of the immediate visual context on incremental thematic role assignment: Evidence from eye-movements in depicted events. Cognition, 95 (1), 95127.
Kohonen, T. (1995). Self-organizing maps. Berlin; Heidelberg; New York: Springer.
Kramer, A. F., & Donchin, E. (1987). Brain potentials as indices of orthographic and phonological interaction during word matching. Journal of Experimental Psychology: Learning, Memory, and Cognition, 13 (1), 7686.
Kroll, J. F., & De Groot, A. M. B. (1997). Lexical and conceptual memory in the bilingual: Mapping form to meaning in two languages. In De Groot, & Kroll, (eds.), pp. 169–199.
Kroll, J. F. & De Groot, A. M. B. (eds.) (2005). Handbook of bilingualism: Psycholinguistic approaches. New York: Oxford University Press.
Kroll, J. F., Dijkstra, T., Janssen, N., & Schriefers, H. (2000). Selecting the language in which to speak: Experiments on lexical access in bilingual production. Proceedings of the 41st Annual Meeting of the Psychonomic Society, New Orleans, LA.
Kroll, J. F., & Stewart, E. (1994). Category interference in translation and picture naming: Evidence for asymmetric connections between bilingual memory representations. Journal of Memory and Language, 33 (2), 149174.
Lalor, E., & Kirsner, K. (2001). The representation of “false cognates” in the bilingual lexicon. Psychnomic Bulletin & Review, 8, 552559.
Li, P. (1998). Mental control, language tags, and language nodes in bilingual lexical processing. Bilingualism: Language and Cognition, 1 (2), 9293.
Li, P., & Farkas, I. (2002). A self-organized connectionist model of bilingual processing. In Heredia, R. & Altarriba, J. (eds.), Bilingual sentence processing, pp. 5985. Amsterdam: Elsevier.
Li, P., & MacWhinney, B. (2002). PatPho: A phonological pattern generator for neural networks. Behavior Research Methods, Instruments, and Computers, 34, 408415
Li, P., Zhao, X., & MacWhinney, B. (2007). Dynamic Self-Organization and children's word learning. Cognitive Science, 31, 581612.
Loebell, H., & Bock, K. (2003). Structural priming across languages. Linguistics, 41 (5), 791824.
Luce, P. A., & Pisoni, D. B. (1998). Recognizing spoken words: The neighborhood activation model. Ear & Hearing, 19, 136.
Lund, K., & Burgess, C. (1996). Producing high-dimensional semantic spaces from lexical co-occurrence. Behavior Research Methods, 28 (2), 203208.
Mani, N., & Plunkett, K. (2010). In the infant's mind's ear: Evidence for implicit naming in 18-month-olds. Psychological Science, 21 (7), 908913.
Marchman, V. A., Fernald, A., & Hurtado, N. (2010). How vocabulary size in two languages relates to efficiency in spoken word recognition by young Spanish–English bilinguals. Journal of Child Language, 37 (4), 817840.
Marian, V. (2008). Bilingual research methods. In Altarriba, J. & Heredia, R. R. (eds.), An introduction to bilingualism: Principles and processes, pp. 1338. Mahwah, NJ: Laurence Erlbaum.
Marian, V. (2009). Audio-visual integration during bilingual language processing. In Pavlenko, (ed.), pp. 52–78.
Marian, V., & Spivey, M. (2003a). Bilingual and monolingual processing of competing lexical items. Applied Psycholinguistics, 24, 173193.
Marian, V., & Spivey, M. (2003b). Competing activation in bilingual language processing: Within- and between-language competition. Bilingualism: Language and Cognition, 6 (2), 97115.
Marslen-Wilson, W. D. (1987). Functional parallelism in spoken word-recognition. Cognition, 25, 71102.
McClelland, J. L., & Elman, J. L. (1986). The TRACE model of speech perception. Cognitive Psychology, 18, 186.
McClelland, J. L., Mirman, D., & Holt, L. L. (2006). Are there interactive processes in speech perception? Trends in Cognitive Sciences, 10, 363369.
McClelland, J. L., & Rumelhart, D. E. (1981). An interactive activation model of context effects in letter perception, part 1: An account of basic findings. Psychological Review, 88, 375407.
McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264, 746748.
McQueen, J. M., Norris, D., & Cutler, A. (2006). Are there really interactive processes in speech perception? Trends in Cognitive Sciences, 10 (12), 534.
Meuter, R. F. I., & Allport, A. (1999). Bilingual language switching in naming: Asymmetrical costs of language selection. Journal of Memory and Language, 40 (1), 2540.
Meyer, A. S., Belke, E., Telling, A. L., & Humphreys, G. W. (2007). Early activation of objects names in visual search. Psychonomic Bulletin & Review, 14 (4), 710716.
Miikkulainen, R. (1993). Subsymbolic natural language processing: An integrated model of scripts, lexicon, and memory. Cambridge, MA: MIT Press.
Miikkulainen, R. (1997). Dyslexic and category-specific aphasic impairments in a self-organizing feature map model of the lexicon. Brain and Language, 59, 334366.
Miikkulainen, R., & Kiran, S. (2009). Modeling the bilingual lexicon of an individual subject. Lecture Notes in Computer Science, 5629, 191199.
Morton, J. (1969). Interaction of information in word recognition. Psychological Review, 76 (2), 165178.
Navarra, J., & Soto-Faraco, S. (2007). Hearing lips in a second language: Visual articulatory information enables the perception of second language sounds. Psychological Research, 71, 412.
Norris, D. (1994). Shortlist: A connectionist model of continuous speech recognition. Cognition, 52 (3), 189234.
Norris, D., & McQueen, J. M. (2008). Shortlist B: A Bayesian model of continuous speech recognition. Psychological Review, 115 (2), 357395.
Obleser, J., Leaver, A. M., VanMeter, J., & Rauschecker, J. P. (2010). Segregation of vowels and consonants in human auditory cortex: Evidence for distributed hierarchical organization. Frontiers in Psychology, 1 (December), 114.
Pavlenko, A. (ed.) (2009). The bilingual mental lexicon: Interdisciplinary approaches. Tonawanda, NY: Multilingual Matters.
Pavlenko, A., & Driagina, V. (2007). Russian emotion vocabulary in American learners’ narratives. Modern Language Journal, 91 (2), 213234.
Rastle, K., & Brysbaert, M. (2006). Masked phonological priming effects in English: Are they real? Do they matter? Cognitive Psychology, 53 (2), 97145.
Rastle, K., McCormick, S. F., Bayliss, L., & Davis, C. J. (2011). Orthography influences the perception and production of speech. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37 (6), 15881594.
Roelofs, A. (2003). Goal-referenced selection of verbal action: Modeling attentional control in the Stroop task. Psychological Review, 110 (1), 88125.
Roelofs, A., & Verhoef, K. (2006). Modeling the control of phonological encoding in bilingual speakers. Bilingualism: Language and Cognition, 9, 167176.
Samuel, A. G. (1996). Does lexical information influence the perceptual restoration of phonemes? Journal of Experimental Psychology: General, 125, 2851.
Samuel, A. G. (2001). Knowing a word affects the fundamental perception of the sounds within it. Psychological Science, 12, 348351.
Salamoura, A., & Williams, J. N. (2007). Processing verb argument structure across languages: Evidence for shared representations in the bilingual lexicon. Applied Psycholinguistics, 28 (4), 627660.
Sánchez-Casas, R., & García-Albea, J. E. (2005). The representation of cognate and noncognate words on bilingual memory: Can cognate status be characterized as a special kind of morphological relation? In Kroll, & De Groot, (eds.), pp. 226–250.
Schoonbaert, S., Duyck, W., Brysbaert, M., & Hartsuiker, R. J. (2009). Semantic and translation priming from a first language to a second and back: Making sense of the findings. Memory and Cognition, 37, 569586.
Schoonbaert, S., Hartsuiker, R. J., & Pickering, M. J. (2007). The representation of lexical and syntactic information in bilinguals: Evidence from syntactic priming. Journal of Memory and Language, 56 (2), 153171.
Schwartz, A. I., Kroll, J. F., & Diaz, M. (2007). Reading words in Spanish and English: Mapping orthography to phonology in two languages. Language & Cognitive Processes, 22 (1), 106129.
Shaoul, C., & Westbury, C. (2011). A USENET corpus (2005–2010). Edmonton, AB: University of Alberta. http://www.psych.ualberta.ca/~westburylab/downloads/usenetcorpus.download.html (retrieved April 15, 2011).
Shook, A., & Marian, V. (2012). Bimodal bilinguals co-activate both languages during spoken comprehension. Cognition, 124 (3), 314324.
Spivey, M. J., Tanenhaus, M. K., Eberhard, K. M., & Sedivy, J. C. (2002). Eye movements and spoken language comprehension: Effects of visual context on syntactic ambiguity resolution. Cognitive Psychology, 45 (4), 447481.
Sumby, W. H., & Pollack, I. (1954). Visual contributions to speech intelligibility in noise. Journal of the Acoustical Society of America, 26, 212215.
Tanenhaus, M. K., Spivey-Knowlton, M., Eberhard, K., & Sedivy, J. (1995). Integration of visual and linguistic information during spoken language comprehension. Science, 268, 16321634.
Thierry, G., & Wu, Y. J. (2007). Brain potentials reveal unconscious translation during foreign-language comprehension. Proceedings of the National Academy of Sciences, 104, 1253012535.
Thomas, M. S. C., & van Heuven, W. J. B. (2005). Computational models of bilingual comprehension. In Kroll, & De Groot, (eds.), pp. 202–225.
van Heuven, W. J. B., Dijkstra, T., & Grainger, J. (1998). Orthographic neighborhood effects in bilingual word recognition. Journal of Memory and Language, 39, 458483.
Vitevitch, M. S., & Luce, P. A. (1998). When words compete: Levels of processing in spoken word perception. Psychological Science, 9, 325329.
Vitevitch, M. S., & Luce, P. A. (1999). Probabilistic phonotactics and neighborhood activation in spoken word recognition. Journal of Memory and Language, 40 (3), 374408.
Vroomen, J., van Linden, S., de Gelder, B., & Bertelson, P. (2007). Visual recalibration and selective adaptation in auditory-visual speech perception: Contrasting build-up courses. Neuropsychologia, 45, 572577.
Yee, E., & Sedivy, J. (2006). Eye movements reveal transient semantic activation during spoken word recognition. Journal of Experimental Psychology: Learning, Memory, and Cognition, 32, 114.
Yee, E., & Thompson-Schill, S. (2007). Does lexical activation flow from word meanings to word sounds during spoken word recognition? Poster presented at the 48th Annual Meeting of the Psychonomic Society, Long Beach, CA.
Zhao, X., & Li, P. (2007). Bilingual lexical representation in a self-organizing neural network. In McNamara, D. S. & Trafton, J. G. (eds.), Proceedings of the 29th Annual Cognitive Science Society, pp. 759760. Austin, TX: Cognitive Science Society.
Zhao, X., & Li, P. (2010). Bilingual lexical interactions in an unsupervised neural network model. International Journal of Bilingual Education and Bilingualism, 13 (5), 505524.
Ziegler, J. C., & Ferrand, L. (1998). Orthography shapes the perception of speech: The consistency effect in auditory word recognition. Psychonomic Bulletin & Review, 5 (4), 683689.

Keywords

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed