Hostname: page-component-8448b6f56d-c47g7 Total loading time: 0 Render date: 2024-04-17T14:48:08.365Z Has data issue: false hasContentIssue false
  • English
  • Français

In hand and in mind: Effects of gesture production and viewing on second language word learning

Published online by Cambridge University Press:  18 October 2017

LAURA M. MORETT
LAURA M. MORETT*
Affiliation:
University of Alabama
*
ADDRESS FOR CORRESPONDENCE Laura M. Morett, Educational Neuroscience Initiative, Department of Educational Studies in Psychology, Research Methodology, and Counseling, University of Alabama, Box 870231, Tuscaloosa, AL 35487. E-mail: lmorett@ua.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

To date, within the context of second language (L2) word learning, nonspontaneous representational gesture viewing's impact on memory and spontaneous gesture production's impact on communication have been examined separately. Thus, it is unclear whether and how these effects interact, particularly in the same individuals. The present study addresses this question by comparing these effects and by examining their influence on one another. To do so, a dialogic task was employed in which participants learned words from a novel L2 and taught them to other similar participants. The results show that viewing nonspontaneous representational gesture did not affect L2 word recall whereas spontaneous production of different gesture types affected communication and memory of L2 word meanings in varying ways. Furthermore, the results provide evidence that gesture viewing primes production of similar types of gestures, and that the quantity and types of gestures produced differ based on the context of communication. These results indicate that the effects of spontaneous gesture production on communication are stronger than the effects of nonspontaneous gesture viewing on memory, and that these effects influence one another. Together, these results demonstrate that spontaneous gesture production and nonspontaneous gesture viewing play distinct but interrelated roles in L2 acquisition.

Keywords

dialogueembodied cognitiongesturesecond language acquisitionword learning
Type
Original Article
Information
Applied Psycholinguistics , Volume 39 , Issue 2 , March 2018 , pp. 355 - 381
Check for updates
Copyright
Copyright © Cambridge University Press 2017 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Adams, T. W. (1998). Gesture in foreigner talk (Unpublished doctoral dissertation, University of Pennsylvania).Google Scholar
Alibali, M. W., Heath, D. C., & Myers, H. J. (2001). Effects of visibility between speaker and listener on gesture production: Some gestures are meant to be seen. Journal of Memory and Language, 44, 169–188. doi:10.1006/jmla.2000.2752 CrossRefGoogle Scholar
Allen, L. Q. (1995). The effects of emblematic gestures on the development and access of mental representations of French expressions. Modern Language Journal, 79, 521–529. doi:10.1111/j.1540-4781.1995.tb05454.x CrossRefGoogle Scholar
Al-Seghayer, K. (2001). The effect of multimedia annotation modes on L2 vocabulary acquisition: A comparative study. Language Learning and Technology, 5, 202–232.Google Scholar
Asher, J. J. (1969). The total physical response approach to second language learning. Modern Language Journal, 53, 3–17. doi:10.1111/j.1540-4781.1969.tb04552.x Google Scholar
Asher, J. J. (1972). Children's first language as a model for second language learning. Modern Language Journal, 56, 133–139. doi:10.1111/j.1540-4781.1972.tb05031.x Google Scholar
Asher, J. J., & Price, B. S. (1967). The learning strategy of the Total Physical Response: Some age differences. Child Development, 38, 1219–1227. doi:10.1111/j.1540-4781.1995.tb05454.x CrossRefGoogle ScholarPubMed
Atkinson, R. C. (1975). Mnemotechnics in second-language learning. American Psychologist, 30, 821–828. doi:10.1037/h0077029 CrossRefGoogle Scholar
Baddeley, A., Chincotta, D., Stafford, L., & Turk, D. (2002). Is the word length effect in STM entirely attributable to output delay? Evidence from serial recognition. Quarterly Journal of Experimental Psychology: Section A, 55, 353–369. doi:10.1080/02724980143000523 CrossRefGoogle ScholarPubMed
Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychology, 59, 617–645. doi:10.1146/annurev.psych.59.103006.093639 CrossRefGoogle ScholarPubMed
Bergmann, K., & Macedonia, M. (2013). A virtual agent as vocabulary trainer: Iconic gestures help to improve learners’ memory performance. In Aylett, R., Krenn, B., Pelachaud, C., & Shimodaira, H. (Eds.), International workshop on intelligent virtual agents (pp. 139–148). Berlin: Springer.CrossRefGoogle Scholar
Carpenter, S. K., & Olson, K. M. (2011). Are pictures good for learning new vocabulary in a foreign language? Only if you think they are not. Journal of Experimental Psychology: Learning, Memory, and Cognition, 38, 92–101. doi:10.1037/a0024828 Google Scholar
Chu, M., Meyer, A., Foulkes, L., & Kita, S. (2014). Individual differences in frequency and saliency of speech-accompanying gestures: The role of cognitive abilities and empathy. Journal of Experimental Psychology: General, 143, 694–709. doi:10.1037/a0033861 CrossRefGoogle ScholarPubMed
Chun, D. M., & Plass, J. L. (1996). Effects of multimedia annotations on vocabulary acquisition. Modern Language Journal, 80, 183–198. doi:10.1111/j.1540-4781.1996.tb01159.x CrossRefGoogle Scholar
Clark, H. H., & Brennan, S. E. (1991). Grounding in communication. In Resnick, L. B., Levine, J. M., & Teasley, S. D. (Eds.), Perspectives on socially shared cognition (Vol. 13, pp. 127–149). Washington, DC: American Psychological Association.CrossRefGoogle Scholar
Clark, J. M., & Paivio, A. (1991). Dual coding theory and education. Educational Psychology Review, 3, 149–210. doi:10.1007/BF01320076 CrossRefGoogle Scholar
Frick-Horbury, D. (2002). The use of hand gestures as self-generated cues for recall of verbally associated targets. American Journal of Psychology, 115, 1–20. doi:10.2307/1423671 CrossRefGoogle ScholarPubMed
Frick-Horbury, D., & Guttentag, R. E. (1998). The effects of restricting hand gesture production on lexical retrieval and free recall. American Journal of Psychology, 111, 43–62. doi:10.2307/1423536 CrossRefGoogle Scholar
Galati, A., & Brennan, S. E. (2014). Speakers adapt gestures to addressees’ knowledge: Implications for models of co-speech gesture. Language, Cognition and Neuroscience, 29, 435–451. doi:10.1080/01690965.2013.796397 CrossRefGoogle Scholar
Gass, S. M., & Varonis, E. M. (1994). Input, interaction, and second language production. Studies in Second Language Acquisition, 16, 283–302. doi:10.1017/S0272263100013097 CrossRefGoogle Scholar
Gathercole, S. E., Frankish, C. R., Pickering, S. J., & Peaker, S. (1999). Phonotactic influences on short-term memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 25, 84. doi:10.1037/0278-7393.25.1.84 Google ScholarPubMed
Gibbs, R. W. (2006). Embodiment and cognitive science. Cambridge: Cambridge University Press.Google Scholar
Gillespie, M., James, A. N., Federmeier, K. D., & Watson, D. G. (2014). Verbal working memory predicts co-speech gesture: Evidence from individual differences. Cognition, 132, 174–180. doi:10.1016/j.cognition.2014.03.012 CrossRefGoogle ScholarPubMed
Goldin-Meadow, S. (2003). Hearing gesture: How our hands help us think. Cambridge, MA: Harvard University Press.Google Scholar
Goodwin, C. (2003). Pointing as situated practice. In Kita, S. (Ed.), Pointing: Where language, culture and cognition meet (pp. 217–241). Mahwah, NJ: Erlbaum.Google Scholar
Gullberg, M., Roberts, L., & Dimroth, C. (2012). What word-level knowledge can adult learners acquire after minimal exposure to a new language? International Review of Applied Linguistics in Language Teaching, 50, 239–276. doi:10.1515/iral-2012-0010 CrossRefGoogle Scholar
Gullberg, M., Roberts, L., Dimroth, C., Veroude, K., & Indefrey, P. (2010). Adult language learning after minimal exposure to an unknown natural language. Language Learning, 60, 5–24. doi:j.1467-9922.2010.00598.x CrossRefGoogle Scholar
Holler, J., & Wilkin, K. (2009). Communicating common ground: How mutually shared knowledge influences speech and gesture in a narrative task. Language and Cognitive Processes, 24, 267–289. doi:10.1080/01690960802095545 CrossRefGoogle Scholar
Holler, J., & Wilkin, K. (2011). Co-speech gesture mimicry in the process of collaborative referring during face-to-face dialogue. Journal of Nonverbal Behavior, 35, 133–153. doi:10.1007/s10919-011-0105-6 CrossRefGoogle Scholar
Hostetter, A. B., & Alibali, M. W. (2008). Visible embodiment: Gestures as simulated action. Psychonomic Bulletin and Review, 15, 495–514. doi:10.3758/PBR.15.3.495 CrossRefGoogle ScholarPubMed
Hostetter, A. B., & Alibali, M. W. (2010). Language, gesture, action! A test of the gesture as simulated action framework. Journal of Memory and Language, 63, 245–257. doi:10.1016/j.jml.2010.04. 003 CrossRefGoogle Scholar
Jones, L. (2004). Testing L2 vocabulary recognition and recall using pictorial and written test items. Language Learning and Technology, 8, 122–143.Google Scholar
Kelly, S. D., & Lee, A. L. (2012). When actions speak too much louder than words: Hand gestures disrupt word learning when phonetic demands are high. Language and Cognitive Processes, 27, 793–807.CrossRefGoogle Scholar
Kelly, S. D., McDevitt, T., & Esch, M. (2009). Brief training with co-speech gesture lends a hand to word learning in a foreign language. Language and Cognitive Processes, 24, 313–334. doi:10.1080/01690960802365567 CrossRefGoogle Scholar
Kita, S., Özyürek, A., Allen, S., Brown, A., Furman, R., & Ishizuka, T. (2007). Relations between syntactic encoding and co-speech gestures: Implications for a model of speech and gesture production. Language and Cognitive Processes, 22, 1212–1236. doi:10.1080/01690960701461426 CrossRefGoogle Scholar
Kopp, S., Bergmann, K., & Wachsmuth, I. (2008). Multimodal communication from multimodal thinking—Towards an integrated model of speech and gesture production. International Journal of Semantic Computing, 2, 115–136. doi:10.1142/S1793351X08000361 CrossRefGoogle Scholar
Krauss, R. M., Chen, Y., & Chawla, P. (1996). Nonverbal behavior and nonverbal communication: What do conversational hand gestures tell us? In Zanna, M. P. (Ed.), Advances in experimental social psychology (Vol. 28, pp. 389–450). San Diego, CA: Academic Press.Google Scholar
Krönke, K. M., Mueller, K., Friederici, A. D., & Obrig, H. (2013). Learning by doing? The effect of gestures on implicit retrieval of newly acquired words. Cortex, 49, 2553–2568. doi:j.cortex. 2012.11.016 CrossRefGoogle ScholarPubMed
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174. doi:10.2307/2529310 CrossRefGoogle ScholarPubMed
Lantolf, J. P. (2010). Minding your hands: The function of gesture in L2 learning. In Batstone, R. (Ed.), Sociocognitive perspectives on language use and language learning (pp. 130–146). Oxford: Oxford University Press.Google Scholar
Lascarides, A., & Stone, M. (2009). Discourse coherence and gesture interpretation. Gesture, 9, 147–180. doi:10.1075/gest.9.2.01las CrossRefGoogle Scholar
Lazaraton, A. (2004). Gesture and speech in the vocabulary explanations of one ESL teacher: A microanalytic inquiry. Language Learning, 54, 79–117. doi:10.1111/j.1467-9922.2004. 00249.x Google Scholar
Lucero, C., Zaharchuk, H., & Casasanto, D. (2014). Beat gestures facilitate speech production. In Bello, P., Guarini, M., McShane, M., & Scassellati, B. (Eds.), Proceedings of the 36th Annual Meeting of the Cognitive Science Society. Red Hook, NY: Curran Associates.Google Scholar
Macedonia, M. (2014). Bringing back the body into the mind: Gestures enhance word learning in foreign language. In Reevy, G. M. & Bursten, S. N. (Eds.), Pedagogical psychology: Beyond the 21st century. Lausanne: Frontiers Media SA.Google Scholar
Macedonia, M., & Knösche, T. R. (2011). Body in mind: How gestures empower foreign language learning. Mind, Brain, and Education, 5, 196–211. doi:j.1751-228X.2011.01129.x CrossRefGoogle Scholar
Macedonia, M., Müller, K., & Friederici, A. D. (2010). The impact of iconic gestures on foreign language word learning and its neural substrate. Human Brain Mapping, 32, 982–998. doi:10.1002/hbm.21084 CrossRefGoogle ScholarPubMed
Macedonia, M., & von Kriegstein, K. (2012). Gestures enhance foreign language learning. Biolinguistics, 6, 393–416.CrossRefGoogle Scholar
MacWhinney, B. (2000). The CHILDES project: Tools for analyzing talk (Vol. 2). New York: Psychology Press.Google Scholar
Marstaller, L., & Burianová, H. (2013). Individual differences in the gesture effect on working memory. Psychonomic Bulletin and Review, 20, 496–500. doi:10.3758/s13423-012-0365-0 CrossRefGoogle ScholarPubMed
Mayer, K. M., Yildiz, I. B., Macedonia, M., & von Kriegstein, K. (2015). Visual and motor cortices differentially support the translation of foreign language words. Current Biology, 25, 530–535. doi:10.1016/j.cub.2014.11.068 CrossRefGoogle ScholarPubMed
McCafferty, S. G. (2002). Gesture and creating zones of proximal development for second language learning. Modern Language Journal, 86, 192–203. doi:10.1111/1540-4781.00144 CrossRefGoogle Scholar
McNeill, D. (1992). Hand and mind. Chicago: University of Chicago Press.Google Scholar
McNeill, D. (2005). Gesture and thought. Chicago: University of Chicago Press.CrossRefGoogle Scholar
Mol, L., Krahmer, E., Maes, A., & Swerts, M. (2012). Adaptation in gesture: Converging hands or converging minds? Journal of Memory and Language, 66, 249–264. doi:10.1016/j.jml.2011. 07.004 CrossRefGoogle Scholar
Morett, L. M. (2014). When hands speak louder than words: The role of gesture in the communication, encoding, and recall of words in a novel second language. Modern Language Journal, 98, 834–853. doi:10.1111/modl.12125 Google Scholar
Morrel-Samuels, P., & Krauss, R. M. (1992). Word familiarity predicts temporal asynchrony of hand gestures and speech. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18, 615–622. doi:10.1037/0278-7393.18.3.615 Google Scholar
Paivio, A. (1990). Mental representations: A dual coding approach. New York: Oxford University Press.CrossRefGoogle Scholar
Parrill, F., & Kimbara, I. (2006). Seeing and hearing double: The influence of mimicry in speech and gesture on observers. Journal of Nonverbal Behavior, 30, 157–166. doi:10.1007/s10919-006-0014-2 CrossRefGoogle Scholar
Perniss, P., & Vigliocco, G. (2014). The bridge of iconicity: From a world of experience to the experience of language. Philosophical Transactions of the Royal Society Part B, 369, 20130300. doi:10.1098/rstb.2013.0300 CrossRefGoogle Scholar
Porter, A. (2012). A helping hand with language learning: Teaching French vocabulary with gesture. Language Learning Journal, 44, 236–256. doi:10.1080/09571736.2012.750681 CrossRefGoogle Scholar
Pressley, M., Levin, J. R., & Delaney, H. D. (1982). The mnemonic keyword method. Review of Educational Research, 52, 61–91. doi:10.3102/00346543052001061 CrossRefGoogle Scholar
Rauscher, F. H., Krauss, R. M., & Chen, Y. (1996). Gesture, speech, and lexical access: The role of lexical movements in speech production. Psychological Science, 7, 226–231. doi:10.1111/j.1467-9280.1996.tb00364.x CrossRefGoogle Scholar
Rowe, M. L., Silverman, R. D., & Mullan, B. E. (2013). The role of pictures and gestures as nonverbal aids in preschoolers’ word learning in a novel language. Contemporary Educational Psychology, 38, 109–117. doi:10.1016/j.cedpsych.2012.12.001 CrossRefGoogle Scholar
Salaberry, M. R. (2001). The use of technology for second language learning and teaching: A retrospective. Modern Language Journal, 85, 39–56. doi:10.1111/0026-7902.00096 Google Scholar
Sime, D. (2006). What do learners make of teachers’ gestures in the language classroom? International Review of Applied Linguistics in Language Teaching, 44, 211–230. doi:10.1515/IRAL.2006.009 CrossRefGoogle Scholar
So, W. C., Sim Chen-Hui, C., & Low Wei-Shan, J. (2012). Mnemonic effect of iconic gesture and beat gesture in adults and children: Is meaning in gesture important for memory recall? Language and Cognitive Processes, 27, 665–681.CrossRefGoogle Scholar
Sueyoshi, A., & Hardison, D. M. (2005). The role of gestures and facial cues in second language listening comprehension. Language Learning, 55, 661–699. doi:10.1111/j.0023-8333.2005.00320.x CrossRefGoogle Scholar
Swain, M. (1997). Collaborative dialogue: Its contribution to second language learning. Revista Canaria de Estudios Ingleses, 34, 157132.Google Scholar
Tellier, M. (2008). The effect of gestures on second language memorisation by young children. Gesture, 8, 219–235. doi:10.1075/gest.8.2.06tel CrossRefGoogle Scholar
van Compernolle, R. A., & Williams, L. (2011). Thinking with your hands: Speech–gesture activity during an L2 awareness-raising task. Language Awareness, 20, 203–219. doi:10.1080/09658416. 2011.559244 CrossRefGoogle Scholar
Van Hell, J. G., & De Groot, A. (1998). Conceptual representation in bilingual memory: Effects of concreteness and cognate status in word association. Bilingualism: Language and Cognition, 1, 193–211. doi:10.1017/S1366728998000352 CrossRefGoogle Scholar
Wilson, M. (2002). Six views of embodied cognition. Psychonomic Bulletin and Review, 9, 625–636. doi:10.3758/BF03196322 CrossRefGoogle ScholarPubMed