Skip to main content
×
×
Home

Having a different pointing of view about the future: The effect of signs on co-speech gestures about time in Mandarin–CSL bimodal bilinguals

  • YAN GU (a1) (a2), YEQIU ZHENG (a3) and MARC SWERTS (a1)
Abstract

Mandarin speakers often use gestures to represent time laterally, vertically, and sagittally. Chinese Sign Language (CSL) users also exploit signs for that purpose, and can differ from the gestures of Mandarin speakers in their choices of axes and direction of sagittal movements. The effects of sign language on co-speech gestures about time were investigated by comparing spontaneous temporal gestures of late bimodal bilinguals (Mandarin learners of CSL) and non-signing Mandarin speakers. Spontaneous gestures were elicited via a wordlist definition task. In addition to effects of temporal words on temporal gestures, results showed significant effects of sign. Compared with non-signers, late bimodal bilinguals (1) produced more sagittal but fewer lateral temporal gestures; and (2) exhibited a different temporal orientation of sagittal gestures, as they were more likely to gesture past events to their back. In conclusion, bodily experience of sign language can not only impact the nature of co-speech gestures, but also spatio-motoric thinking and abstract space-time mappings.

  • View HTML
    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Having a different pointing of view about the future: The effect of signs on co-speech gestures about time in Mandarin–CSL bimodal bilinguals
      Available formats
      ×
      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      Having a different pointing of view about the future: The effect of signs on co-speech gestures about time in Mandarin–CSL bimodal bilinguals
      Available formats
      ×
      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      Having a different pointing of view about the future: The effect of signs on co-speech gestures about time in Mandarin–CSL bimodal bilinguals
      Available formats
      ×
Copyright
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Corresponding author
Address for correspondence: Yan Gu, Department of Experimental Psychology, Psychology and Language Sciences, Faculty of Brain Sciences, University College London, 26 Bedford Way, London WC1H 0AP, United KingdomYan.gu@ucl.ac.uk
Footnotes
Hide All

*The first author received financial support from The Netherlands Organization for Scientific Research, via NWO Promoties in de Geesteswetenschappen (322-89-007) and the second author received financial support from NWO Research Talent (406-15-298), which are gratefully acknowledged. We thank Yufen Chi, Weidong Zheng, and Qingshi Zhu who have assisted us in running experiments. We are grateful to Marieke Hoetjes, Rein Cozijn, editor Kira Gor and the three anonymous reviewers for their comments on the manuscript, and Arthur van Soest for his advice on statistics. Preliminary results of this study have been presented at the 7th Conference of International Society for Gesture Studies in Paris (ISGS 2016), and “The role of gesture in cognitive and linguistic processes” workshop at the University of Warwick (July, 2017).

Footnotes
References
Hide All
Alibali, M. W. (2005). Gesture in spatial cognition: Expressing, communicating, and thinking about spatial information. Spatial Cognition & Computation, 5 (4), 307331. doi:10.1207/s15427633scc0504_2
Bender, A., & Beller, S. (2014). Mapping spatial frames of reference onto time: A review of theoretical accounts and empirical findings. Cognition, 132 (3), 342382. doi:10.1016/j.cognition.2014.03.016
Boroditsky, L. (2000). Metaphoric structuring: Understanding time through spatial metaphors. Cognition, 75 (1), 128. doi:10.1016/s0010-0277(99)00073-6
Boroditsky, L. (2001). Does language shape thought?: Mandarin and English speakers’ conceptions of time. Cognitive Psychology, 43 (1), 122. doi:10.1006/cogp.2001.0748
Boroditsky, L., & Gaby, A. (2010). Remembrances of times East: Absolute spatial representations of time in an Australian aboriginal community. Psychological Science, 21 (11), 16351639. doi:10.1177/0956797610386621
Bostan, İ., Börütecene, A., Özcan, O., & Göksun, T., (2016). Temporal expressions in speech and gesture. In Papafragou, A., Grodner, D., Mirman, D., & Trueswell, J.C. (Eds.). Proceedings of the 38th Annual Conference of the Cognitive Science Society (pp. 18711876). Austin, TX: Cognitive Science Society.
Brentari, D., Nadolske, M. A., & Wolford, G. (2012). Can experience with co-speech gesture influence the prosody of a sign language? Sign language prosodic cues in bimodal bilinguals. Bilingualism: Language and Cognition, 15 (2), 402412. doi:10.1017/S1366728911000587
Brown, A., & Gullberg, M. (2008). Bidirectional cross-linguistic influence in L1-L2 encoding of manner in speech and gesture. Studies in Second Language Acquisition, 30 (2), 225251. doi:10.1017/S0272263108080327
Brown, A., & Gullberg, M. (2011). Bidirectional cross-linguistic influence in event conceptualization? Expressions of Path among Japanese learners of English. Bilingualism: Language and Cognition, 14 (1), 7994. doi:10.1017/S1366728910000064
Bylund, E., & Athanasopoulos, P. (2017). The Whorfian time warp: Representing duration through the language hourglass. Journal of Experimental Psychology: General, 146 (7), 911916. doi: 10.1037/xge0000314
Cabeza Pereiro, C., & Fernández Soneira, A. (2004). The expression of time in Spanish Sign Language (LSE). Sign Language & Linguistics, 7 (1), 6382. doi:10.1075/sll.7.1.06cab
Casasanto, D., & Boroditsky, L. (2008). Time in the mind: Using space to think about time. Cognition, 106 (2), 579593. doi:10.1016/j.cognition.2007.03.004
Casasanto, D. (2009). Embodiment of abstract concepts: Good and bad in right- and left-handers. Journal of Experimental Psychology: General, 138 (3), 351367. doi:10.1037/a0015854
Casasanto, D., & Jasmin, K. (2012). The hands of time: Temporal gestures in English speakers. Cognitive Linguistics, 23 (4), 643674. doi:10.1515/cog-2012-0020
Casey, S., & Emmorey, K. (2009). Co-speech gesture in bimodal bilinguals. Language and Cognitive Process, 24 (2), 290312. doi: 10.1080/01690960801916188
Casey, S., Emmorey, K., & Larrabee, H. (2012). The effects of learning American Sign Language on co-speech gesture. Bilingualism: Language and Cognition, 15 (4), 677686. doi:10.1017/S1366728911000575P
China Association of the Deaf and Hard of Hearing (2003). Chinese Sign Language (中国手语). Beijing: Huaxia Publishing House.
Choi, S., & Lantolf, J. P. (2008). Representation and embodiment of meaning in L2 communication: Motion events in the speech and gesture of advanced L2 Korean and L2 English speakers. Studies in Second Language Acquisition, 30 (2), 191224. doi:10.1017/S0272263108080315
Chu, M., & Kita, S. (2016). Co-thought and co-speech gestures are generated by the same action generation process. Journal of Experimental Psychology: Learning, Memory, and Cognition, 42 (2), 257270. doi:10.1037/xlm0000168
Chui, K. (2011). Conceptual metaphors in gesture. Cognitive Linguistics, 22 (3), 437458. doi:10.1515/cogl.2011.017
Cienki, A. (1998). Metaphoric gestures and some of their relations to verbal metaphorical expressions. In Koenig, J. (Ed.), Discourse and cognition: Bridging the gap (pp. 189204). Stanford, CA: CSLI Productions.
Cooperrider, K., & Núñez, R. (2009). Across time, across the body: Transversal temporal gestures. Gesture, 9 (2), 181206. doi:10.1075/gest.9.2.02coo
Cooperrider, K., Gentner, D., & Goldin-Meadow, S. (2016). Spatial analogies pervade complex relational reasoning: Evidence from spontaneous gestures. Cognitive Research: Principles and Implications, 1 (1), 28. doi:10.1186/s41235-016-0024-5
Cooperrider, K., & Goldin-Meadow, S. (2017). When gesture becomes analogy. Topics in Cognitive Science, 9 (3), 719737. doi: 10.1111/tops.12276
Croissant, Y. (2012). Estimation of multinomial logit models in R: The mlogit Packages. R package version 0.2-2. URL: http://cran.r-project.org/web/packages/mlogit/vignettes/mlogit.pdf.
De la Fuente, J., Santiago, J., Román, A., Dumitrache, C., & Casasanto, D. (2014). When you think about it, your past is in front of you: How culture shapes spatial conceptions of time. Psychological Science, 25 (9), 16821690. doi:10.1177/0956797614534695
Emmorey, K., Klima, E., & Hickok, G. (1998). Mental rotation within linguistic and non-linguistic domains in users of American Sign Language. Cognition, 68 (3), 221–46.
Emmorey, K. (2001). Space on hand: The exploitation of signing space to illustrate abstract thought. In Gattis, M. (Ed.), Spatial schemas and abstract thought (pp. 147174), Cambridge, MA, US: MIT Press.
Emmorey, K., Borinstein, H. B., Thompson, R., & Gollan, T. H. (2008). Bimodal bilingualism. Bilingualism: Language and Cognition, 11 (1), 4361. doi:10.1017/S1366728907003203
Emmorey, K., Grabowski, T., McCullough, S., Ponto, L. L., Hichwa, R. D., & Damasio, H. (2005). The neural correlates of spatial language in English and American Sign Language: A PET study with hearing bilinguals. Neuroimage, 24 (3), 832840. doi: 10.1016/j.neuroimage.2004.10.008
Emmorey, K., & McCullough, S. (2009). The bimodal bilingual brain: Effects of sign language experience. Brain and Language, 109 (2–3), 124–32. doi:10.1016/j.bandl.2008.03.005
Emmorey, K., Giezen, M. R., & Gollan, T. H. (2016). Psycholinguistic, cognitive, and neural implications of bimodal bilingualism. Bilingualism: Language and Cognition, 19 (2), 223242. doi:10.1017/S1366728915000085
Fischer, S., & Gong, Q. (2010). Variation in East Asian sign language structures. In Brentari, Diane (Ed.), Sign languages, 502521. Cambridge: Cambridge University Press.
Floyd, S. (2016). Modally hybrid grammar?: Celestial pointing for time-of-day reference in Nheengatú. Language, 92 (1), 3164. doi:10.1353/lan.2016.0013
Fuhrman, O., & Boroditsky, L. (2010). Cross-cultural differences in mental representations of time: Evidence from an implicit nonlinguistic task. Cognitive Science, 34 (8), 14301451. doi:10.1111/j.1551-6709.2010.01105.x
Fuhrman, O., McCormick, K., Chen, E., Jiang, H. D., Shu, D. F., Mao, S. M., & Boroditsky, L. (2011). How linguistic and cultural forces shape conceptions of time: English and Mandarin time in 3D. Cognitive Science, 35 (7), 13051328. doi:10.1111/j.1551-6709.2011.01193.x
Gentner, D., Imai, M., & Boroditsky, L. (2002). As time goes by: Evidence for two systems in processing space time metaphors. Language and Cognitive Processes, 17 (5), 537565. doi:10.1080/01690960143000317
Gibbs, R. W. (2003). Embodied experience and linguistic meaning. Brain and Language, 84 (1), 115. doi:10.1016/s0093-934x(02)00517-5
Giezen, M., & Emmorey, K. (2016). Language co-activation and lexical selection in bimodal bilinguals: Evidence from picture–word interference. Bilingualism: Language and Cognition, 19 (2), 264276. doi:10.1017/S1366728915000097
Gu, Y., Hoetjes, M., & Swerts, M. (in preparation). Different hands of time: What can Mandarin-English speakers' and Dutch-English speakers' temporal gestures reveal their conceptions of time?
Gu, Y., Mol, L., Hoetjes, M., & Swerts, M. (2017). Conceptual and lexical effects on gestures: the case of vertical spatial metaphors for time in Chinese. Language, Cognition and Neuroscience, 32 (8), 10481063. doi:10.1080/23273798.2017.1283425
Gu, Y., Zheng, Y., & Swerts, M. (2017). Does Mandarin spatial metaphor for time influence Chinese deaf signers' spatio-temporal reasoning? In Gunzelmann, G., Howes, A., Tenbrink, T., Davelaar, E. J. (Eds). Proceedings of the 39th Annual Conference of the Cognitive Science Society (pp.445450). Austin, TX: Cognitive Science Society.
Gu, Y., Zheng, Y., & Swerts, M. (in revision). Which is in front of Chinese people, Past or Future? The effect of language and culture on temporal gestures and spatial conceptions of time. Cognitive Science.
Hendricks, R., & Boroditsky, L. (2017). New space-time metaphors foster new non-linguistic representations. Topics in Cognitive Science, 9 (3), 800818. doi:10.1111/tops.12279.
Hostetter, A. B., & Alibali, M. W. (2008). Visible embodiment: Gestures as simulated action. Psychonomic Bulletin & Review, 15 (3), 495514. doi:10.3758/pbr.15.3.495
Kita, S. (2000). How representational gestures help speaking. In McNeill, D. (Eds.), Language and gesture (pp. 162185). Cambridge, MA: Cambridge University Press. doi:10.1017/cbo9780511620850.011
Kita, S., Danziger, E., & Stolz, C. (2001). Cultural specificity of spatial schemas, as manifested in spontaneous gestures. In Gattis, M. (Ed.), Spatial schemas and abstract thought (pp. 115146). Cambridge, MA: MIT Press.
Kita, S., Alibali, M. W., & Chu, M. (2017). How do gestures influence thinking and speaking? The Gesture-for-Conceptualization Hypothesis. Psychological Review, 124 (3), 245266. doi:10.1037/rev0000059
Kita, S., & Özyürek, A. (2003). What does cross-linguistic variation in semantic coordination of speech and gesture reveal? Evidence for an interface representation of spatial thinking and speaking. Journal of Memory and Language, 48 (1), 1632. doi:10.1016/s0749-596x(02)00505-3
Lai, V. T., & Boroditsky, L. (2013). The immediate and chronic influence of spatio-temporal metaphors on the mental representations of time in English, Mandarin, and Mandarin-English speakers. Frontiers in Psychology, 4, 110. doi:10.3389/fpsyg.2013.00142
Lakoff, G., & Johnson, M. (1980). Metaphors we live by. Chicago, IL: University of Chicago Press.
Lausberg, H., & Sloetjes, H. (2009). Coding gestural behavior with the NEUROGES-ELAN system. Behavior Research Methods, 41 (3), 841849. doi:10.3758/brm.41.3.841
Le Guen, O., & Balam, L. I. P. (2012). No metaphorical timeline in gesture and cognition among Yucatec Mayas. Frontiers in Psychology, 3. doi:10.3389/fpsyg.2012.00271
Maeder, C., & Loncke, F. (1996). Spatial, temporal and temporo-logical notions in French Sign Language: Comparative study of deaf and hearing subjects. Sign Language Studies, 90 (1), 3851. doi:10.1353/sls.1996.0006
Meade, G., Midgley, K., Sevcikova Sehyr, Z., Holcomb, P., & Emmorey, K. (2017). Implicit co-activation of American Sign Language in deaf readers: An ERP study. Brain and Language, 170, 5061. doi:10.1016/j.bandl.2017.03.004
Morford, J. P., Wilkinson, E., Villwock, A., Piñar, P., & Kroll, J. F. (2011). When deaf signers read English: Do written words activate their sign translations? Cognition, 118 (2), 286292. doi:10.1016/j.cognition.2010.11.006
Moore, K. E. (2011). Ego-perspective and field-based frames of reference: Temporal meanings of FRONT in Japanese, Wolof, and Aymara. Journal of Pragmatics, 43 (3), 759776. doi:10.1016/j.pragma.2010.07.003
Nilsson, A. L. (2016). Embodying metaphors: Signed language interpreters at work. Cognitive Linguistics, 27 (1), 3565. doi:10.1515/cog-2015-0029
Núñez, R., Cooperrider, K., Doan, D., & Wassmann, J. (2012). Contours of time: Topographic construals of past, present, and future in the Yupno valley of Papua New Guinea. Cognition, 124, 2535. doi:10.1016/j.cognition.2012.03.007
Núñez, R., & Cooperrider, K., (2013). The tangle of space and time in human cognition. Trends in Cognitive Sciences, 17 (5), 220229. doi:10.1016/j.tics.2013.03.008
Núñez, R., Motz, B., & Teuscher, U. (2006). Time after time: The psychological reality of the ego- and time-reference-point distinction in metaphorical construals of time. Metaphor and Symbol, 21 (3), 133146. doi:10.1207/s15327868ms2103_1
Núñez, R. E., & Sweetser, E. (2006). With the future behind them: Convergent evidence from Aymara language and gesture in the crosslinguistic comparison of spatial construals of time. Cognitive Science, 30 (3), 401450. doi:10.1207/s15516709cog0000_62
Ortega, G., & Morgan, G. (2015). The effect of sign iconicity in the mental lexicon of hearing non-signers and proficient signers: Evidence of cross-modal priming. Language, Cognition and Neuroscience, 30 (5), 574585. doi:10.1080/23273798.2014.959533.
Özçalışkan, Ş. (2016). Do gestures follow speech in bilinguals’ description of motion? Bilingualism: Language and Cognition, 19 (3), 644653. doi:10.1017/s1366728915000796
Özçalışkan, Ş, Lucero, C., & Goldin-Meadow, S. (2016). Does language shape silent gesture? Cognition, 148, 1018. doi:10.1016/j.cognition.2015.12.001
Pika, S., Nicoladis, E., & Marentette, P. F. (2006). A cross-cultural study on the use of gestures: Evidence for cross-linguistic transfer? Bilingualism: Language and Cognition, 9 (3), 319327. doi:10.1017/S1366728906002665
Pyers, J., & Emmorey, K. (2008). The face of bimodal bilingualism: Grammatical markers in American Sign Language are produced when bilinguals speak to English monolinguals. Psychological Science, 19 (6), 531536. doi:10.1111/j.1467-9280.2008.02119.x
Schermer, T., & Koolhof, C. (1990). The reality of time-lines: Aspects of tense in Sign Language of the Netherlands (SLN). In Prillwitz, S. & Vollhaber, T. (Eds.), Current trends in European sign language research, pp. 295305. Hamburg: Signum.
Sinte, A. (2013). Expression of time in French Belgian Sign Language (LSFB). In Meurant, L. Sinte, A., Van Herreweghe, M., & Vermeerbergen, M. (Eds.), Sign language research, uses and practices: Crossing views on theoretical and applied sign language linguistics (pp. 205235). Berlin: De Gruyter Mouton and Ishara Press. doi: 10.1515/9781614511472.205
Tversky, B. (2011). Visualizing Thought. Topics in Cognitive Science, 3 (3), 499535. doi:10.1111/j.1756-8765.2010.01113.x
Van Hell, J. G., & Dijkstra, T. (2002). Foreign language knowledge can influence native language performance in exclusively native contexts. Psychonomic Bulletin & Review, 9 (4), 780789. doi:10.3758/BF03196335
Walker, E. J., Bergen, B. K., & Núñez, R. (2017). The spatial alignment of time: Differences in alignment of deictic and sequence time along the sagittal and lateral axes. Acta Psychologica, 175, 1320. doi: 10.1016/j.actpsy.2017.02.001
Walker, E., & Cooperrider, K. (2016). The continuity of metaphor: Evidence from temporal gestures. Cognitive Science, 40 (2), 481495. doi:10.1111/cogs.12254
Wilcox, S. (2002). The iconic mapping of space and time in signed languages. In Albertazzi, L. (Ed.), Unfolding perceptual continua, 255281. Amsterdam & Philadelphia: John Benjamins. doi:10.1075/aicr.41.12wil
Wu, L., & Li, H. (2012). Time-space metaphors in Chinese Sign Language (中国手语中的时间空间隐喻). Chinese Journal of Special Education, 150 (12), 2529.
Wu, Y. J., & Thierry, G. (2010). Chinese–English bilinguals reading English hear Chinese. Journal of Neuroscience, 30 (22), 76467651. doi:10.1523/jneurosci.1602-10.2010
Xiao, C., Zhao, M., & Chen, L. (2017). Both earlier times and the future are “Front”: The distinction between time- and ego-reference-points in Mandarin speakers’ temporal representation. Cognitive Science, 42 (3), 10261040. doi:10.1111/cogs.12552
Yang, J. H. (2005). Expressing negation in Chinese sign language. MA Thesis. Rochester Institute of Technology. Accessed from http://scholarworks.rit.edu/theses/4010
Yang, J. H. (2015) Chinese Sign Language. In Jepsen, J.B., De Clerck, G., Lutalo-Kiingi, S., McGregor, W. B. (eds), Sign languages of the world: A comparative handbook (pp. 177194). Berlin: De Gruyter. doi:10.1515/9781614518174-012
Yu, N. (2012). The metaphorical orientation of time in Chinese. Journal of Pragmatics, 44 (10), 13351354. doi:10.1016/j.pragma.2012.06.002
Zheng, X. (2009). A study of signs for non-visual concepts in Shanghai Variety of Chinese Sign Language (上海手语非视觉概念表达研究). PhD Dissertation. Fudan University, China.
Zou, L., Abutalebi, J., Zinszer, B., Yan, X., Shu, H., Peng, D., & Ding, G. (2012). Second language experience modulates functional brain network for the native language production in bimodal bilinguals. NeuroImage, 62 (3), 13671375. doi:10.1016/j.neuroimage.2012.05.062
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Bilingualism: Language and Cognition
  • ISSN: 1366-7289
  • EISSN: 1469-1841
  • URL: /core/journals/bilingualism-language-and-cognition
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Keywords

Metrics

Altmetric attention score