Skip to main content

Predicting sign learning in hearing adults: The role of perceptual-motor (and phonological?) processes


The present study aimed to identify predictors of one aspect of sign language acquisition, sign learning, in hearing nonsigners. Candidate predictors were selected based on the theory that the observed relationship between phonological short-term memory and L2 lexical learning is due in part to common perceptual-motor processes. Hearing nonsigning adults completed a sign learning task, three assessments of short-term memory for movements (movement STM; two of which used sign-like stimuli), and two visuospatial STM tasks. The final sample included 103 adults, ranging between 18 and 33 years of age. All predictors were moderately to strongly correlated with the sign learning task and to each other. A series of regression analyses revealed that both movement and visuospatial STM uniquely contributed to the prediction of sign learning. These results suggest that perceptual-motor processes play a significant role in sign learning and raise questions about the role of phonological processing.

Corresponding author
ADDRESS FOR CORRESPONDENCE David Martinez, School of Psychology, Georgia Institute of Technology, 654 Cherry St., Atlanta, GA 30332. E-mail:
Hide All
Atkins, P. W. B., & Baddeley, A. D. (1998). Working memory and distributed vocabulary learning. Applied Psycholinguistics, 19, 537552. doi:10.1017/S0142716400010353
Baddeley, A. D. (1966). Short-term memory for word sequences as a function of acoustic, semantic and formal similarity. Quarterly Journal of Experimental Psychology, 18, 362365. doi:10.1080/14640746608400055
Baddeley, A. D. (1986). Working memory. New York: Clarendon Press.
Baddeley, A. D. (2012). Working memory: Theories, models, and controversies. Annual Review of Psychology, 63, 129. doi:10.1146/annurev-psych-120710-100422
Baddeley, A. D. (2015). Working memory in second language learning. In Wen, Z., Mota, M. B., McNeill, A., Bunting, M., & Engle, R. (Eds.), Working memory in second language acquisition and processing (pp. 1728). Bristol: Multilingual Matters.
Baddeley, A. D., Gathercole, S., & Papagno, C. (1998). The phonological loop as a language learning device. Psychological Review, 105, 158173. doi:10.1037/0033-295X.105.1.158
Baddeley, A. D., Thomson, N., & Buchanan, M. (1975). Word length and the structure of short-term memory. Journal of Verbal Learning and Verbal Behavior, 14, 575589. doi:10.1016/S0022-5371(75)80045-4
Bates, E., & Goodman, J. C. (1997). On the inseparability of grammar and the lexicon: Evidence from acquisition, aphasia, and real-time processing. Language and Cognitive Processes, 12, 507584. doi:10.1080/016909697386628
Bavelier, D., Newman, A. J., Mukherjee, M., Hauser, P., Kemeny, S., Braun, A., & Boutla, M. (2008). Encoding, rehearsal, and recall in signers and speakers: Shared network but differential engagement. Cerebral Cortex, 18, 22632274. doi:10.1093/cercor/bhm248
Bochner, J. H., Christie, K., Hauser, P. C., & Searls, J. M. (2011). When is a difference really different? Learners' discrimination of linguistic contrasts in American Sign Language. Language Learning, 61, 13021327.
Bochner, J. H., Samar, V. J., Hauser, P. C., Garrison, W. M., Searls, J. M., & Sanders, C. A. (2016). Validity of the American Sign Language Discrimination Test. Language Testing, 33, 473495.
Brentari, D. (1998). A prosodic model of sign language phonology. Cambridge, MA: MIT Press.
Brysbaert, M., & New, B. (2009). Moving beyond Kučera and Francis: A critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English. Behavior Research Methods, 41, 977990.
Campbell, R., MacSweeney, M., & Waters, D. (2008). Sign language and the brain: A review. Journal of Deaf Studies and Deaf Education, 13, 320. doi:10.1093/deafed/enm035
Colle, H. A., & Welsh, A. (1976). Acoustic masking in primary memory. Journal of Verbal Learning and Verbal Behavior, 15, 1731. doi:10.1016/S0022-5371(76)90003-7
Conrad, R., & Hull, A. J. (1964). Information, acoustic confusion and memory span. British Journal of Psychology, 55, 429432. doi:10.1111/j.2044-8295.1964.tb00928.x
Conway, A. R., Kane, M. J., Bunting, M. F., Hambrick, D. Z., Wilhelm, O., & Engle, R. W. (2005). Working memory span tasks: A methodological review and user's guide. Psychonomic Bulletin & Review, 12, 769786.
Cooper, C. J. (1964). Some relationships between paired-associates learning and foreign-language aptitude. Journal of Educational Psychology, 55, 132138. doi:10.1037/h0044223
Cull, A. (2014). Production of movement in users of American Sign Language and its influence on being identified as non-native (Doctoral dossertation, Gallaudet University).
Della Sala, S., Gray, C., Baddeley, A. D., Allamano, N., & Wilson, L. (1999). Pattern span: A tool for unwelding visuo-spatial memory. Neuropsychologia, 37, 11891199. doi:10.1016/S0028-3932(98)00159-6
Ding, X., Zhao, Y., Wu, F., Lu, X., Gao, Z., & Shen, M. (2015). Binding biological motion and visual features in working memory. Journal of Experimental Psychology: Human Perception and Performance, 41, 850865. doi:10.1037/xhp0000061
Engle, R. W., Tuholski, S. W., Laughlin, J. E., & Conway, A. R. A. (1999). Working memory, short-term memory, and general fluid intelligence: A latent-variable approach. Journal of Experimental Psychology: General, 128, 309331. doi:10.1037/0096-3445.128.3.309
Field, A. (2013). Discovering statistics using IBM SPSS statistics. Thousand Oaks, CA: Sage.
Gathercole, S. E. (1995). Is nonword repetition a test of phonological memory or long-term knowledge? It all depends on the nonwords. Memory & Cognition, 23, 8394. doi:10.3758/bf03210559
Gathercole, S. E. (2006). Nonword repetition and word learning: The nature of the relationship. Applied Psycholinguistics, 27, 513543. doi:10.1017/S0142716406060383
Gathercole, S. E., & Baddeley, A. D. (1989). Evaluation of the role of phonological STM in the development of vocabulary in children: A longitudinal study. Journal of Memory and Language, 28, 200213. doi:10.1016/0749-596X(89)90044-2
Gathercole, S. E., Hitch, G. J., Service, E., & Martin, A. J. (1997). Phonological short-term memory and new word learning in children. Developmental Psychology, 33, 966979. doi:10.1037/0012-1649.33.6.966
Gathercole, S. E., Pickering, S. J., Hall, M., & Peaker, S. M. (2001). Dissociable lexical and phonological influences on serial recognition and serial recall. Quarterly Journal of Experimental Psychology A: Human Experimental Psychology, 54A, 130. doi:10.1080/02724980042000002
Gathercole, S. E., Willis, C. S., Baddeley, A. D., & Emslie, H. (1994). The Children's Test of Nonword Repetition: A test of phonological working memory. Memory, 2, 103127. doi:10.1080/09658219408258940
Goldberg, D., Looney, D., & Lusin, N. (2015). Enrollments in languages other than English in United States institutions of higher education, Fall 2013. Modern Language Association. Retrieved June 15, 2016, from
Gupta, P. (2003). Examining the relationship between word learning, nonword repetition, and immediate serial recall in adults. Quarterly Journal of Experimental Psychology A: Human Experimental Psychology, 56A, 12131236. doi:10.1080/02724980343000071
Hilger, A. I., Loucks, T. M., Quinto-Pozos, D., & Dye, M. W. (2015). Second language acquisition across modalities: Production variability in adult L2 learners of American Sign Language. Second Language Research, 31, 375388.
Hulme, C., Maughan, S., & Brown, G. D. A. (1991). Memory for familiar and unfamiliar words: Evidence for a long-term memory contribution to short-term memory span. Journal of Memory and Language, 30, 685701.
Hummel, K. M., & French, L. M. (2016). Phonological memory and aptitude components: Contributions to second language proficiency. Learning and Individual Differences, 51 (Supplement C), 249255. doi:10.1016/j.lindif.2016.08.016
Hundal, P. S., & Horn, J. L. (1977). On the relationships between short-term learning and fluid and crystallized intelligence. Applied Psychological Measurement, 1, 1121. doi:10.1177/014662167700100104
Jones, D. M., Hughes, R. W., & Macken, W. J. (2006). Perceptual organization masquerading as phonological storage: Further support for a perceptual-gestural view of short-term memory. Journal of Memory and Language, 54, 265281. doi:10.1016/j.jml.2005.10.006
Jones, D. M., Hughes, R. W., & Macken, W. J. (2007). The phonological store abandoned. Quarterly Journal of Experimental Psychology, 60, 505511. doi:10.1080/17470210601147598
Jones, D. M., & Macken, W. J. (1993). Irrelevant tones produce an irrelevant speech effect: Implications for phonological coding in working memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 19, 369381. doi:10.1037/0278-7393.19.2.369
Klima, E. S., & Bellugi, U. (1979). The signs of language. Cambridge, MA: Harvard University Press.
Krug, K., Shafer, T., Dardick, W., Magalis, C., & Parenté, R. (2002). A test of foreign language acquisition: Paired-association learning. Applied Cognitive Psychology, 16, 211221. doi:10.1002/acp.781
Kyllonen, P. C., & Tirre, W. C. (1988). Individual differences in associative learning and forgetting. Intelligence, 12, 393421. doi:10.1016/0160-2896(88)90004-9
Kyllonen, P. C., & Woltz, D. J. (1989). Role of cognitive factors in the acquisition of cognitive skill. In Kanfer, R., Cudeck, R., Kanfer, R., & Cudeck, R. (Eds.), Abilities, motivation, and methodology: The Minnesota Symposium on Learning and Individual Differences (pp. 239280). Hillsdale, NJ: Erlbaum.
Li, S. (2015). The construct validity of language aptitude. Studies in Second Language Acquisition, 38, 801842. doi:10.1017/S027226311500042X
Mann, W., Marshall, C. R., Mason, K., & Morgan, G. (2010). The acquisition of sign language: The impact of phonetic complexity on phonology. Language Learning and Development, 6, 6086. doi:10.1080/15475440903245951
Marshalek, B., Lohman, D. F., & Snow, R. E. (1983). The complexity continuum in the radex and hierarchical models of intelligence. Intelligence, 7, 107127. doi:10.1016/0160-2896(83)90023-5
Martin, K. I., & Ellis, N. C. (2012). The roles of phonological short-term memory and working memory in L2 grammar and vocabulary learning. Studies in Second Language Acquisition, 34, 379413. doi:10.1017/S0272263112000125
Masoura, E. V., & Gathercole, S. E. (1999). Phonological short-term memory and foreign language learning. International Journal of Psychology, 34, 383388. doi:10.1080/002075999399738
Masoura, E. V., & Gathercole, S. E. (2005). Contrasting contributions of phonological short-term memory and long-term knowledge to vocabulary learning in a foreign language. Memory, 13, 422429. doi:10.1080/09658210344000323
Masoura, E. V., Gathercole, S. E., & Bablekou, Z. (2004). Contributions of phonological short-term memory to vocabulary acquisition. Psychology, 11, 341355.
Milner, B. (1971). Interhemispheric differences in the localization of psychological processes in man. British Medical Bulletin, 27, 272277.
Mirus, G., Rathmann, C., & Meier, R. P. (2001). Proximalization and distalization of sign movement in adult learners. In Diveley, V., Metzger, M., Taub, S., & Baer, A. M. (Eds.), Signed languages: Discoveries from international research (pp. 103119). Washington, DC: Gallaudet University Press.
Moulton, S. T., & Kosslyn, S. M. (2009). Imagining predictions: Mental imagery as mental emulation. Philosophical Transactions of the Royal Society B: Biological Sciences, 364, 12731280. doi:10.1098/rstb.2008.0314
Neath, I. (2000). Modeling the effects of irrelevant speech on memory. Psychonomic Bulletin & Review, 7, 403423. doi:10.3758/BF03214356
Newman-Norlund, R. D., Frey, S. H., Petitto, L.-A., & Grafton, S. T. (2006). Anatomical substrates of visual and auditory miniature second-language learning. Journal of Cognitive Neuroscience, 18, 19841997.
O'Brien, I., Segalowitz, N., Collentine, J., & Freed, B. (2006). Phonological memory and lexical, narrative, and grammatical skills in second language oral production by adult learners. Applied Psycholinguistics, 27, 377402. doi:10.1017/S0142716406060322
O'Brien, I., Segalowitz, N., Freed, B., & Collentine, J. (2007). Phonological memory predicts second language oral fluency gains in adults. Studies in Second Language Acquisition, 29, 557582. doi:10.1017/S027226310707043X
Orfanidou, E., Adam, R., McQueen, J. M., & Morgan, G. (2009). Making sense of nonsense in British Sign Language (BSL): The contribution of different phonological parameters to sign recognition. Memory & Cognition, 37, 302315. doi:10.3758/MC.37.3.302
Ortega, G., & Morgan, G. (2015). Phonological development in hearing learners of a sign language: The influence of phonological parameters, sign complexity, and iconicity. Language Learning, 65, 660688. doi:10.1111/lang.12123
Pa, J., Wilson, S. M., Pickell, H., Bellugi, U., & Hickok, G. (2008). Neural organization of linguistic short-term memory is sensory modality-dependent: Evidence from signed and spoken language. Journal of Cognitive Neuroscience, 20, 21982210. doi:10.1162/jocn.2008.20154
Papagno, C., Valentine, T., & Baddeley, A. D. (1991). Phonological short-term memory and foreign-language vocabulary learning. Journal of Memory and Language, 30, 331347. doi:10.1016/0749-596X(91)90040-Q
Papagno, C., & Vallar, G. (1992). Phonological short-term memory and the learning of novel words: The effect of phonological similarity and item length. Quarterly Journal of Experimental Psychology A: Human Experimental Psychology, 44A, 4767. doi:10.1080/14640749208401283
Peelen, M. V., & Downing, P. E. (2007). The neural basis of visual body perception. Nature Reviews Neuroscience, 8, 636648. doi:10.1038/nrn2195
Peirce, J. W. (2007). PsychoPy—Psychophysics software in Python. Journal of Neuroscience Methods, 162, 813.
Petitto, L. A. (2005). How the brain begets language. In McGilvray, J. A. (Ed.), The Cambridge companion to Chomsky (pp. 84101). Cambridge: Cambridge University Press.
Petitto, L. A., Berens, M. S., Kovelman, I., Dubins, M. H., Jasinska, K., & Shalinsky, M. (2012). The “Perceptual Wedge Hypothesis” as the basis for bilingual babies' phonetic processing advantage: New insights from fNIRS brain imaging. Brain and Language, 121, 130. doi:10.1016/j.bandl.2011.05.003
Petitto, L. A., Langdon, C., Stone, A., Andriola, D., Kartheiser, G., & Cochran, C. (2016). Visual sign phonology: Insights into human reading and language from a natural soundless phonology. Wiley Interdisciplinary Reviews: Cognitive Science. Advance online publication. doi:10.1002/wcs.1404
Pichler, D. C., & Koulidobrova, H. (2016). Acquisition of sign language as a second language. In Marschark, M., Spencer, P. E., Marschark, M., & Spencer, P. E. (Eds.), The Oxford handbook of deaf studies in language (pp. 218230). New York: Oxford University Press.
Porro, C. A., Francescato, M. P., Cettolo, V., Diamond, M. E., Baraldi, P., Zuiani, C., . . . di Prampero, P. E. (1996). Primary motor and sensory cortex activation during motor performance and motor imagery: A functional magnetic resonance imaging study. Journal of Neuroscience, 16, 76887698.
Rönnberg, J., Rudner, M., & Ingvar, M. (2004). Neural correlates of working memory for sign language. Cognitive Brain Research, 20, 165182. doi:10.1016/j.cogbrainres.2004.03.002
Rosen, R. S. (2004). Beginning L2 production errors in ASL lexical phonology: A cognitive phonology model. Sign Language and Linguistics, 7, 3161.
Rudner, M. (2015). Working memory for meaningless manual gestures. Canadian Journal of Experimental Psychology, 69, 7279. doi:10.1037/cep0000033
Rudner, M., Andin, J., & Rönnberg, J. (2009). Working memory, deafness and sign language. Scandinavian Journal of Psychology, 50, 495505. doi:10.1111/j.1467-9450.2009.00744.x
Salamé, P., & Baddeley, A. D. (1989). Effects of background music on phonological short-term memory. Quarterly Journal of Experimental Psychology A: Human Experimental Psychology, 41, 107122. doi:10.1080/14640748908402355
Seemüller, A., Fiehler, K., & Rösler, F. (2011). Unimodal and crossmodal working memory representations of visual and kinesthetic movement trajectories. Acta Psychologica, 136, 5259. doi:10.1016/j.actpsy.2010.09.014
Seibert, L. C. (1930). An experiment on the relative efficiency of studying French vocabulary in associated pairs versus studying French vocabulary in context. Journal of Educational Psychology, 21, 297314. doi:10.1037/h0070517
Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420428. doi:10.1037/0033-2909.86.2.420
Smyth, M. M., Pearson, N. A., & Pendleton, L. R. (1988). Movement and working memory: Patterns and positions in space. Quarterly Journal of Experimental Psychology A: Human Experimental Psychology, 40, 497514. doi:10.1080/02724988843000041
Smyth, M. M., & Pendleton, L. R. (1989). Working memory for movements. Quarterly Journal of Experimental Psychology A: Human Experimental Psychology, 41, 235250.
Smyth, M. M., & Pendleton, L. R. (1990). Space and movement in working memory. Quarterly Journal of Experimental Psychology A: Human Experimental Psychology, 42, 291304. doi:10.1080/14640749008401223
Stankov, L., Seizova-Cajić, T., & Roberts, R. D. (2001). Tactile and kinesthetic perceptual processes within the taxonomy of human cognitive abilities. Intelligence, 29, 129. doi:10.1016/S0160-2896(00)00038-6
Thorn, A. S. C., & Frankish, C. R. (2005). Long-term knowledge effects on serial recall of nonwords are not exclusively lexical. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31, 729735. doi:10.1037/0278-7393.31.4.729
Thorndike, E. L. (1908). Memory for paired associates. Psychological Review, 15, 122138. doi: 10.1037/h0073570
Urgolites, Z. J., & Wood, J. N. (2013a). Binding actions and scenes in visual long-term memory. Psychonomic Bulletin and Review, 20, 12461252. doi:10.3758/s13423-013-0440-1
Urgolites, Z. J., & Wood, J. N. (2013b). Visual long-term memory stores high-fidelity representations of observed actions. Psychological Science, 24, 403411. doi:10.1177/0956797612457375
Vallar, G. (2006). Memory systems: The case of phonological short-term memory. A festschrift for Cognitive Neuropsychology. Cognitive Neuropsychology, 23, 135155. doi:10.1080/02643290542000012
Vicary, S. A., Robbins, R. A., Calvo-Merino, B., & Stevens, C. J. (2014). Recognition of dance-like actions: Memory for static posture or dynamic movement? Memory & Cognition, 42, 755767. doi:10.3758/s13421-014-0395-0
Vicary, S. A., & Stevens, C. J. (2014). Posture-based processing in visual short-term memory for actions. Quarterly Journal of Experimental Psychology, 67, 24092424. doi:10.1080/17470218. 2014.931445
Williams, J. T. (2017). Modality adaptation hypothesis: Neurocognitive alterations to novel visuospatial components of sign language during initial acquisition in adulthood (Unpublished doctoral dissertation, Indiana University, Bloomington).
Williams, J. T., Darcy, I., & Newman, S. D. (2015). Modality-independent neural mechanisms for novel phonetic processing. Brain Research, 1620, 107115. doi:10.1016/j.brainres.2015.05.014
Williams, J. T., Darcy, I., & Newman, S. D. (2016a). The beneficial role of L1 spoken language skills on initial L2 sign language learning: Cognitive and linguistic predictors of M2L2 acquisition. Studies in Second Language Acquisition Advance online publication. doi:10.1017/S0272263116000322
Williams, J. T., Darcy, I., & Newman, S. D. (2016b). Modality-specific processing precedes amodal linguistic processing during L2 sign language acquisition: A longitudinal study. Cortex, 75, 5667. doi:10.1016/j.cortex.2015.11.015
Williams, J. T., & Newman, S. D. (2016a). Modality-independent effects of phonological neighborhood structure on initial L2 sign language learning. Research in Language, 13, 199213.
Williams, J. T., & Newman, S. D. (2016b). Phonological substitution errors in L2 ASL sentence processing by hearing M2L2 learners. Second Language Research. Advance online publication. doi:10.1177/0267658315626211
Wilson, M. (2001). The case for sensorimotor coding in working memory. Psychonomic Bulletin & Review, 8, 4457. doi:10.3758/BF03196138
Wilson, M., & Emmorey, K. (1997). A visuospatial “phonological loop” in working memory: Evidence from American Sign Language. Memory & Cognition, 25, 313320. doi:10.3758/BF03211287
Wilson, M., & Emmorey, K. (1998). A “word length effect” for sign language: Further evidence for the role of language in structuring working memory. Memory & Cognition, 26, 584590. doi:10.3758/BF03201164
Wilson, M., & Emmorey, K. (2003). The effect of irrelevant visual input on working memory for sign language. Journal of Deaf Studies and Deaf Education, 8, 97103. doi:10.1093/deafed/eng010
Wilson, M., & Fox, G. (2007). Working memory for language is not special: Evidence for an articulatory loop for novel stimuli. Psychonomic Bulletin & Review, 14, 470473.
Wu, Y. C., & Coulson, S. (2014). A psychometric measure of working memory capacity for configured body movement. PLOS ONE, 9.
Zihl, J., & Heywood, C. A. (2015). The contribution of LM to the neuroscience of movement vision. Frontiers in Integrative Neuroscience, 9, 6. doi:10.3389/fnint.2015.00006
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Applied Psycholinguistics
  • ISSN: 0142-7164
  • EISSN: 1469-1817
  • URL: /core/journals/applied-psycholinguistics
Please enter your name
Please enter a valid email address
Who would you like to send this to? *



Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed