Hostname: page-component-76d6cb85b7-92wsb Total loading time: 0 Render date: 2026-07-15T15:54:04.335Z Has data issue: false hasContentIssue false
  • English
  • Français

Linguists should learn to love speech-based deep learning models

Published online by Cambridge University Press:  01 July 2026

Marianne de Heer Kloots Open the ORCID record for Marianne de Heer Kloots [Opens in a new window] ,
Paul Boersma Open the ORCID record for Paul Boersma [Opens in a new window]  and
Willem Zuidema Open the ORCID record for Willem Zuidema [Opens in a new window]
Marianne de Heer Kloots*
Affiliation:
Institute for Logic, Language and Computation (ILLC), University of Amsterdam, Amsterdam, Netherlands m.l.s.deheerkloots@uva.nl
Paul Boersma
Affiliation:
Amsterdam Center for Language and Communication (ACLC), University of Amsterdam, Amsterdam, Netherlands paul.boersma@uva.nl w.h.zuidema@uva.nl
Willem Zuidema
Affiliation:
Institute for Logic, Language and Computation (ILLC), University of Amsterdam, Amsterdam, Netherlands m.l.s.deheerkloots@uva.nl
*
*Corresponding author.
Get access Rights & Permissions [Opens in a new window]

Abstract

Futrell and Mahowald present a useful framework bridging technology-oriented deep learning systems and explanation-oriented linguistic theories. Unfortunately, the target article’s focus on generative text-based Large Language Models (LLMs) fundamentally limits fruitful interactions with linguistics, as many interesting questions on human language fall outside what is captured by written text. We argue that audio-based deep learning models can and should play a crucial role.

Information

Type
Open Peer Commentary
Information
Behavioral and Brain Sciences , Volume 49 , 2026 , e206
Check for updates
Copyright
© The Author(s), 2026. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Alishahi, A., Barking, M., & Chrupała, G. (2017). Encoding of phonology in a recurrent neural model of grounded speech. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017) (pp. 368–378). https://doi.org/10.18653/v1/K17-1037 CrossRefGoogle Scholar
Arora, S., Chang, K. W., Chien, C. M., Peng, Y., Wu, H., Adi, Y., Dupoux, E., Lee, H. Y., Livescu, K., & Watanabe, S. (2025). On the landscape of spoken language models: A comprehensive survey. arXiv preprint. https://doi.org/10.48550/arXiv.2504.08528 CrossRefGoogle Scholar
Aslin, R., & Pisoni, D. (1980). Some developmental processes in speech perception. In Yeni-Komshian, G. H., Kavanagh, J. F., & Ferguson, C. A. (Eds.), Child phonology: Vol 2, Perception (pp. 67–96). Academic Press.Google Scholar
Boersma, P. (2009). Unidirectional optimization of comprehension can achieve bidirectional optimality. Talk presented at 10th Szklarska Poręba Workshop on the Roots of Pragmasemantics, Szklarska Poręba, March 13, 2009.Google Scholar
Boersma, P., Benders, T. & Seinhorst, K. (2020). Neural networks for phonology and phonetics. Journal of Language Modelling, 8(1), 103–177. https://doi.org/10.15398/jlm.v8i1.224 CrossRefGoogle Scholar
Chrupała, G. (2023). Putting Natural in Natural Language Processing. In Findings of the Association for Computational Linguistics: ACL 2023 (pp. 7820–7827). https://doi.org/10.18653/v1/2023.findings-acl.495 CrossRefGoogle Scholar
de Heer Kloots, M., & Zuidema, W. (2024) Human-like linguistic biases in neural speech models: Phonetic categorization and phonotactic constraints in Wav2Vec2.0. Proc. Interspeech 2024, 4593–4597. https://doi.org/10.21437/Interspeech.2024-2490 CrossRefGoogle Scholar
de Seyssel, M., Lavechin, M., Titeux, H., Thomas, A., Virlet, G., Revilla, A.S., Wisniewski, G., Ludusan, B., Dupoux, E. (2023) ProsAudit, a prosodic benchmark for self-supervised speech models. Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH 2023), 2963–2967. https://doi.org/10.21437/Interspeech.2023-438 CrossRefGoogle Scholar
Dupoux, E. (2018). Cognitive science in the era of artificial intelligence: A roadmap for reverse-engineering the infant language-learner. Cognition, 173, 43–59. https://doi.org/10.1016/j.cognition.2017.11.008 CrossRefGoogle ScholarPubMed
Elman, J. L., & Zipser, D. (1988). Learning the hidden structure of speech. The Journal of the Acoustical Society of America, 83(4), 1615–1626. https://doi.org/10.1121/1.395916 CrossRefGoogle ScholarPubMed
Gauthier, J., Breiss, C., Leonard, M. K., & Chang, E. F. (2025). Emergent morpho-phonological representations in self-supervised speech models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (pp. 28055–28074). https://doi.org/10.18653/v1/2025.emnlp-main.1425 CrossRefGoogle Scholar
Grice, H. P. (1975). Logic and conversation. In Cole, P., & Morgan, J.J. (Eds.), Syntax and semantics 3: Speech acts (pp. 41–58). Academic Press.Google Scholar
Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences, 79, 2554–2558. https://doi.org/10.1073/pnas.79.8.2554 CrossRefGoogle ScholarPubMed
Kohonen, T. (1982). Self-organized formation of topologically correct feature maps. Biological Cybernetics, 43, 59–69. https://doi.org/10.1007/BF00337288 CrossRefGoogle Scholar
Lavechin, M., Sy, Y., Titeux, H., Blandón, M.A.C., Räsänen, O., Bredin, H., Dupoux, E., Cristia, A. (2023). BabySLM: Language-acquisition-friendly benchmark of self-supervised spoken language models. Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH 2023), 4588–4592. http://doi.org/10.21437/Interspeech.2023-978 CrossRefGoogle Scholar
Martin, K., Gauthier, J., Breiss, C., Levy, R. (2023) Probing self-supervised speech models for phonetic and phonemic information: A case study in aspiration. Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH 2023), 251–255. http://doi.org/10.21437/Interspeech.2023-2359 CrossRefGoogle Scholar
McClelland, J. L., & Elman, J. L. (1986). The TRACE model of speech perception. Cognitive Psychology, 18(1), 1–86. https://doi.org/10.1016/0010-0285(86)90015-0 CrossRefGoogle ScholarPubMed
Millet, J., & Dunbar, E. (2022). Do self-supervised speech models develop human-like perception biases?. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 7591–7605). https://doi.org/10.18653/v1/2022.acl-long.523 CrossRefGoogle Scholar
McMurray, B., Horst, J. S., Toscano, J. C., & Samuelson, L. K. (2009). Integrating connectionist learning and dynamical systems processing: Case studies in speech and lexical development. In Spencer, J. P., Thomas, M. S. C., & McClelland, J. L. (Eds.), Toward a unified theory of development: Connectionism and dynamic systems theory re-considered (pp. 218–249). Oxford University Press. https://doi.org/10.1093/acprof:oso/9780195300598.003.0011 CrossRefGoogle Scholar
Norris, D. (1994). Shortlist: A connectionist model of continuous speech recognition. Cognition, 52(3), 189–234. https://doi.org/10.1016/0010-0277(94)90043-4 CrossRefGoogle Scholar
Orhan, P., Boubenec, Y., & King, J. R. (2025). The detection of algebraic auditory structures emerges with self-supervised learning. PLOS Computational Biology, 21(9), e1013271. https://doi.org/10.1371/journal.pcbi.1013271 CrossRefGoogle ScholarPubMed
Pasad, A., Chien, C. M., Settle, S., & Livescu, K. (2024). What do self-supervised speech models know about words?. Transactions of the Association for Computational Linguistics, 12, 372–391. https://doi.org/10.1162/tacl_a_00656 CrossRefGoogle Scholar
Poli, M., Schatz, T., Dupoux, E., & Lavechin, M. (2024). Modeling the initial state of early phonetic learning in infants. Language Development Research, 5(1), 1–34. https://doi.org/10.34842/y89t-6q31 Google Scholar
Salakhutdinov, R. R., and Hinton, G. E. (2009). Deep Boltzmann machines. In Proceedings of the 12th International Conference on Artificial Intelligence and Statistics. Clearwater, Florida. https://proceedings.mlr.press/v5/salakhutdinov09a.html Google Scholar
Shen, G., Watkins, M., Alishahi, A., Bisazza, A., & Chrupała, G. (2024). Encoding of lexical tone in self-supervised models of spoken language. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) (pp. 4250–4261). https://doi.org/10.18653/v1/2024.naacl-long.239 CrossRefGoogle Scholar
Smolensky, P. (1990). Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence, 46(1–2), 159–216. https://doi.org/10.1016/0004-3702(90)90007-M CrossRefGoogle Scholar
Smolensky, P. (1999). Grammar-based connectionist approaches to language. Cognitive Science, 23(4), 589–613. https://doi.org/10.1016/S0364-0213(99)00017-8 Google Scholar
Soderstrom, M., Mathis, D., & Smolensky, P. (2006). Abstract genomic encoding of universal grammar in optimality theory. In Smolensky, P., & Legendre, G. (Eds.), The harmonic mind, from neural computation to optimality-theoretic grammar (Volume II: Linguistic and philosophical implications) (pp. 403–471). MIT Press.Google Scholar