Skip to main content Accessibility help
×
×
Home

Referring strategies in American Sign Language and English (with co-speech gesture): The role of modality in referring to non-nameable objects

  • ZED SEVCIKOVA SEHYR (a1), BRENDA NICODEMUS (a2), JENNIFER PETRICH (a1) and KAREN EMMOREY (a1)
  • Please note a correction has been issued for this article.

Abstract

American Sign Language (ASL) and English differ in linguistic resources available to express visual–spatial information. In a referential communication task, we examined the effect of language modality on the creation and mutual acceptance of reference to non-nameable figures. In both languages, description times reduced over iterations and references to the figures’ geometric properties (“shape-based reference”) declined over time in favor of expressions describing the figures’ resemblance to nameable objects (“analogy-based reference”). ASL signers maintained a preference for shape-based reference until the final (sixth) round, while English speakers transitioned toward analogy-based reference by Round 3. Analogy-based references were more time efficient (associated with shorter round description times). Round completion times were longer for ASL than for English, possibly due to gaze demands of the task and/or to more shape-based descriptions. Signers’ referring expressions remained unaffected by figure complexity while speakers preferred analogy-based expressions for complex figures and shape-based expressions for simple figures. Like speech, co-speech gestures decreased over iterations. Gestures primarily accompanied shape-based references, but listeners rarely looked at these gestures, suggesting that they were recruited to aid the speaker rather than the addressee. Overall, different linguistic resources (classifier constructions vs. geometric vocabulary) imposed distinct demands on referring strategies in ASL and English.

  • View HTML
    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Referring strategies in American Sign Language and English (with co-speech gesture): The role of modality in referring to non-nameable objects
      Available formats
      ×

      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      Referring strategies in American Sign Language and English (with co-speech gesture): The role of modality in referring to non-nameable objects
      Available formats
      ×

      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      Referring strategies in American Sign Language and English (with co-speech gesture): The role of modality in referring to non-nameable objects
      Available formats
      ×

Copyright

Corresponding author

ADDRESS FOR CORRESPONENCE Zed Sevcikova Sehyr, 6495 Alvarado Road, suite #200, San Diego, CA 92120. E-mail: zsevcikova@sdsu.edu

References

Hide All
Attneave, F. (1957). Physical determinants of the judged complexity of shapes. Journal of Experimental Psychology, 55, 221227. doi:10.1037/h0043921
Attneave, F., & Arnoult, M. D. (1956). The quantitative study of shape and pattern perception. Psychological Bulletin, 53, 452471. doi:10.1037/h0044049
Aylett, M., & Turk, A. (2004). The smooth signal redundancy hypothesis: A functional explanation for relationships between redundancy, prosodic prominence, and duration in spontaneous speech. Language and Speech, 47, 3156. doi:10.1177/00238309040470010201
Bard, E. G., Anderson, A. H., Sotillo, C., Aylett, M., Doherty-Sneddon, G., & Newlands, A. (2000). Controlling the intelligibility of referring expressions in dialogue. Journal of Memory and Language, 42, 122. doi:10.1006/jmla.1999.2667
Barr, D. J. (2004). Establishing conventional communication systems: Is common knowledge necessary? Cognitive Science, 28, 937962. doi:10.1016/j.cogsci.2004.07.002
Bellugi, U., & Fisher, S. (1972). A comparison of sign language and spoken language. Cognition, 1, 173200.
Brennan, S. E., & Clark, H. H. (1996). Lexical choice and conceptual pacts in conversation. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22, 14821493. doi:10.1037//0278-7393.22.6.1482
Brentari, D. (2002). Modality differences in sign language phonology and morphomhonemics. In Meier, R. P., Cormier, K., & Quinto-Pozos, D. (Eds.), Modality and structure in signed and spoken languages (pp. 2764). Cambridge: Cambridge University Press.
Clark, H. H., & Brennan, S. E. (1991). Grounding in communication. In Resnick, L. B., Levine, J. M., & Teasley, S. D. (Eds.), Perspectives on socially shared cognitions (pp. 127149). Washington, DC: American Psychological Association.
Clark, H. H., & Wilkes-Gibbs, D. (1986). Referring as a collaborative process. Cognition, 22, 139. doi:10.1016/0010-0277(86)90010-7
de Ruiter, J. P., Bangerter, A., & Dings, P. (2012). The interplay between gesture and speech in the production of referring expressions: Investigating the trade-off hypothesis. Topics in Cognitive Science, 4, 232248. doi:10.1111/j.1756-8765.2012.01183.x
Emmorey, K. (1996). The confluence of space and language in signed languages. In Bloom, P., Peterson, M. A., Nadel, L., & Garrett, M. F. (Eds.), Language and space. Cambridge, MA: MIT Press.
Emmorey, K., & Herzig, M. (2003). Categorical versus gradient properties of classifier constructions in ASL. In Emmorey, K. (Ed.), Perspectives on classifier constructions in sign languages (1st ed., pp. 221246). Mahwah, NJ: Erlbaum.
Emmorey, K., Tversky, B., & Taylor, H. A. (2001). Using space to describe space: Perspective in speech, sign, and gesture. Spatial Cognition and Computation, 2, 157180.
Fowler, C. A. (1988). Differential shortening of repeated content words produced in various communicative contexts. Language and Speech, 31, 307319. doi:10.1177/002383098803100401
Fussell, S. R., & Krauss, R. M. (1992). Coordination of knowledge in communication: Effects of speakers’ assumptions about what others know. Journal of Personality and Social Psychology, 62, 378391. doi:10.1037//0022-3514.62.3.378
Galantucci, B., Garrod, S., & Roberts, G. (2012). Experimental semiotics. Language and Linguistics Compass, 6, 477493. doi:10.1002/lnc3.351
Galati, A., & Brennan, S. E. (2010). Attenuating information in spoken communication: For the speaker, or for the addressee? Journal of Memory and Language, 62, 3551. doi:10.1016/j.jml.2009.09.002
Galati, A., & Brennan, S. E. (2014). Speakers adapt gestures to addressees’ knowledge: Implications for models of co-speech gesture. Language, Cognition and Neuroscience, 29, 435451. doi:10.1080/01690965.2013.796397
Garrod, S., & Anderson, A. (1987). Saying what you mean in dialogue: A study in conceptual and semantic co-ordination. Cognition, 27, 181218. doi:10.1016/0010-0277(87)90018-7
Garrod, S., Fay, N., Lee, J., Oberlander, J., & MacLeod, T. (2007). Foundations of representation: Where might graphical symbol systems come from? Cognitive Science, 31, 961987. doi:10.1080/03640210701703659
Gerwing, J., & Bavelas, J. (2004). Linguistic influences on gesture's form. Gesture, 4, 157195. doi:10.1075/gest.4.2.04ger
Goldin-Meadow, S. (2003). Hearing gesture: How our hands help us think. Cambridge, MA: Harvard University Press.
Goldin-Meadow, S., & Brentari, D. (2017). Gesture, sign, and language: The coming of age of sign language and gesture studies. Behavioral and Brain Sciences, 40, 160. doi:10.1017/S0140525X15001247
Goldin-Meadow, S., & McNeill, D. (1999). The role of gesture and mimetic representation in making language the province of speech. In Corballis, M. C. & Lea, S. (Eds.), The sescent of mind (pp. 155172). Oxford: Oxford University Press.
Gunderson, E. A., Spaepen, E., Gibson, D., Goldin-Meadow, S., & Levine, S. C. (2015). Gesture as a window onto children's number knowledge. Cognition, 114, 1428. doi:10.1016/j.cognition.2015.07.008
Hoetjes, M., Koolen, R. M. F., Goudbeek, M. B., Krahmer, E. J., & Swerts, M. G. J. (2011). GREEBLES greeble greeb. On reduction in speech and gesture in repeated references. Paper presented at the 33rd Annual Conference of the Cognitive Society, Austin, TX.
Hoetjes, M., Koolen, R. M. F., Goudbeek, M. B., Krahmer, E. J., & Swerts, M. G. J. (2015). Reduction in gesture during the production of repeated references. Journal of Memory and Language, 79–80, 117. doi:10.1016/j.jml.2014.10.004
Hoetjes, M., Krahmer, E., & Swerts, M. (2014). Do repeated references result in sign reduction? Sign Language and Linguistics, 17, 5681. doi:10.1075/sll.17.1.03hoe
Hoetjes, M., Krahmer, E. J., & Swerts, M. G. J. (2015). On what happens in gesture when communication is unsuccessful. Speech Communication, 72, 160175. doi:10.1016/j.specom.2015.06.004
Holler, J., & Stevens, R. (2007). The effect of common ground on how speakers use gesture and speech to represent size information. Journal of Language and Social Psychology, 26, 427. doi:10.1177/0261927X06296428
Hostetter, A. (2011). When do gestures communicate? A meta analysis. Psychological Bulletin, 137, 297315. doi:10.1037/a0022128
Hupet, M., & Chantraine, Y. (1992). Changes in repeated references: Collaboration or repetition effects? Journal of Psycholinguistic Research, 21, 485496. doi:10.1007/BF01067526
Jacobs, N., & Garnham, A. (2007). The role of conversational hand gestures in a narrative task. Journal of Memory and Language, 56, 291303. doi:10.1016/j.cognition.2015.07.008
Jordan, K. I., & Battison, R. (1987). A referential communication experiment with foreign sign languages. Sign Language Studies, 1056, 275287. doi:10.1353/sls.1987.0017
Kendon, A. (1980). Gesticulation and speech: Two aspects of the process of utterance. In Key, M. R. (Ed.), The relationship of verbal and nonverbal communication (pp. 207227). The Hague, Netherlands: Mouton.
Kendon, A. (2004). Gesture: Visible action as utterance. Cambridge: Cambridge University Press.
Kita, S. (2000). How representational gestures help speaking. In McNeill, D. (Ed.), Language and gesture (pp. 162185). Cambridge: Cambridge University Press.
Klima, E., & Bellugi, U. (1979). The signs of language. Cambridge, MA: Harvard University Press.
Krauss, R. M., Chen, Y., & Gottesman, R. F. (2000). Lexical gestures and lexical retrieval: A process model. In McNeill, D. (Ed.), Language and gesture (pp. 261283). Cambridge: Cambridge University Press.
Krauss, R. M., & Hadar, U. (1999). The role of speech-related arm/hand gestures in word retrieval. In Campbell, R. & Messing, L. (Eds.), Gesture, speech, and sign (pp. 93116). Oxford: Oxford University Press.
Krauss, R. M., & Weinheimer, S. (1966). Concurrent feedback, confirmation, and the encoding of referents in verbal communication. Journal of Personality and Social Psychology, 4, 343346. doi:10.1037/h0023705
Lam, T. Q., & Watson, D. G. (2014). Repetition reduction: Lexical repetition in the absence of referent repetition. Journal of Experimental Psychology: Learning, Memory, and Cognition, 40, 829843. doi:10.1037/a0035780
McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press.
Meier, R. P., Cormier, K., & Quinto-Pozos, D. (2002). Modality and structure in signed and spoken languages. Cambridge: Cambridge University Press.
Morford, J. (1996). Insights to language from the study of gesture: A review of research on the gestural communication of non-signing deaf people. Language & Communication, 16, 165178. doi:10.1016/0271-5309(96)00008-0
Özyürek, A., Willems, R. M., Kita, S., & Hagoort, P. (2007). On-line integration of semantic information from speech and gesture: Insights from event-related brain potentials. Journal of Cognitive Neuroscience, 19, 605616. doi:10.1162/jocn.2007.19.4.605
Pickering, M. J., & Garrod, S. (2004). Toward a mechanistic psychology of dialogue. Behavioral and Brain Sciences, 27. doi:10.1017/s0140525x04000056
Pickering, M. J., & Garrod, S. (2006). Alignment as the basis for successful communication. Research on Language and Computation, 4, 203228. doi:0.1007/s11168-006-9004-0
Pyers, J. E., Perniss, P., & Emmorey, K. (2015). Viewpoint in the visual-spatial modality: The coordination of spatial perspective. Spatial Cognition and Computation, 15, 143169. doi:10.1080/13875868.2014.1003933
Raaijmakers, J. G. W., Schrijnemakers, J. M. C., & Gremmen, F. (1999). How to deal with “The language-as-fixed-effect fallacy”: Common misconceptions and alternative solutions. Journal of Memory and Language, 41, 416426. doi:10.1006/jmla.1999.2650
Rauscher, F. H., Krauss, R. M., & Chen, Y. (1996). Gesture, speech, and lexical access: The role of lexical movements in speech production. Psychological Science, 7, 226.
Sehyr, S. Z., & Cormier, K. (2016). Perceptual categorization of handling handshapes in British Sign Language. Language and Cognition, 8, 501532. doi:10.1017/langcog.2015.4
Steels, L., & Loetzsch, M. (2012). The grounded naming game. In Steels, L. (Ed.), Experiments in cultural language evolution (pp. 4159). Amsterdam: John Benjamins.
Supalla, T. (1982). Structure and acquisition of verbs of motion and location in American Sign Language (Unpublished doctoral dissertation, University of California at San Diego).
Supalla, T. (1986). The classifier system in American Sign Language. In Craig, C. (Ed.), Noun classification and categorization (pp. 181214). Philadelphia: John Benjamins.
Trofatter, C., Kontra, C., Beilock, S., & Goldin-Meadow, S. (2015). Gesturing has a larger impact on problem-solving than action, even when action is accompanied by words. Language, Cognition and Neuroscience, 30, 251260. doi:10.1080/23273798.2014.905692
Vanderplas, J. M., & Garvin, E. A. (1959). The association value of random shapes. Journal of Experimental Psychology, 57, 147154. doi:10.1037/h0048723
Zwitserlood, I. (1996). Who'll handle the object?: An investigation of the NGT classifier (Unpublished master's dissertation, Utrecht University).
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Applied Psycholinguistics
  • ISSN: 0142-7164
  • EISSN: 1469-1817
  • URL: /core/journals/applied-psycholinguistics
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Keywords

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed