Hostname: page-component-76fb5796d-wq484 Total loading time: 0 Render date: 2024-04-26T00:18:26.545Z Has data issue: false hasContentIssue false
  • English
  • Français

On the hazards of relating representations and inductive biases

Published online by Cambridge University Press:  28 September 2023

Thomas L. Griffiths Open the ORCID record for Thomas L. Griffiths [Opens in a new window] ,
Sreejan Kumar  and
R. Thomas McCoy
Thomas L. Griffiths
Affiliation:
Departments of Psychology and Computer Science, Princeton University, Princeton, NJ, USA tomg@princeton.edu http://cocosci.princeton.edu/tom/
Sreejan Kumar
Affiliation:
Neuroscience Institute, Princeton University, Princeton, NJ, USA sreejank@princeton.edu http://sreejankumar.com
R. Thomas McCoy
Affiliation:
Department of Computer Science, Princeton University, Princeton, NJ, USA tom.mccoy@princeton.edu https://rtmccoy.com/
Get access Rights & Permissions [Opens in a new window]

Abstract

The success of models of human behavior based on Bayesian inference over logical formulas or programs is taken as evidence that people employ a “language-of-thought” that has similarly discrete and compositional structure. We argue that this conclusion problematically crosses levels of analysis, identifying representations at the algorithmic level based on inductive biases at the computational level.

Type
Open Peer Commentary
Information
Behavioral and Brain Sciences , Volume 46 , 2023 , e275
Check for updates
Copyright
Copyright © The Author(s), 2023. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Anderson, J. R. (1978). Arguments concerning representations for mental imagery. Psychological Review, 85(4), 249.CrossRefGoogle Scholar
Anderson, J. R. (1990). The adaptive character of thought. Psychology Press.Google Scholar
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., … Fiedel, N. (2022). PaLM: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311.Google Scholar
Dasgupta, I., Wang, J., Chiappa, S., Mitrovic, J., Ortega, P., Raposo, D., … Kurth-Nelson, Z. (2019). Causal reasoning from meta-reinforcement learning. arXiv preprint arXiv:1901.08162.Google Scholar
Feldman, N. H., Griffiths, T. L., & Morgan, J. L. (2009). The influence of categories on perception: Explaining the perceptual magnet effect as optimal statistical inference. Psychological Review, 116(4), 752.CrossRefGoogle ScholarPubMed
Grant, E., Finn, C., Levine, S., Darrell, T., & Griffiths, T. (2018). Recasting gradient-based meta-learning as hierarchical Bayes. In International Conference on Learning Representations. OpenReview.Google Scholar
Griffiths, T. L., Chater, N., Kemp, C., Perfors, A., & Tenenbaum, J. B. (2010). Probabilistic models of cognition: Exploring representations and inductive biases. Trends in Cognitive Sciences, 14(8), 357–364.CrossRefGoogle ScholarPubMed
Jones, M., & Love, B. C. (2011). Bayesian fundamentalism or enlightenment? On the explanatory status and theoretical contributions of Bayesian models of cognition. Behavioral and Brain Sciences, 34(4), 169–188.CrossRefGoogle ScholarPubMed
Kumar, S., Correa, C. G., Dasgupta, I., Marjieh, R., Hu, M., Hawkins, R. D. (2022). Using natural language and program abstractions to instill human inductive biases in machines. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., & Oh, A. (Eds.), Advances in Neural Information Processing Systems (pp. 167–180). Curran Associates.Google Scholar
Marr, D. (1982). Vision: A computational investigation into the human representation and processing of visual information. W.H. Freeman.Google Scholar
McCoy, R. T., Linzen, T., Dunbar, E., & Smolensky, P. (2019). RNNs implicitly implement tensor product representations. In International Conference on Learning Representations. OpenReview.Google Scholar
Mikulik, V., Delétang, G., McGrath, T., Genewein, T., Martic, M., Legg, S., & Ortega, P. (2020). Meta-trained agents implement Bayes-optimal agents. Advances in Neural Information Processing Systems, 33, 18691–18703.Google Scholar
Mitchell, T. M. (1997). Machine learning. McGraw-Hill.Google Scholar
Pinker, S., & Prince, A. (1988). On language and connectionism: Analysis of a parallel distributed processing model of language acquisition. Cognition, 28(1–2), 73–193.CrossRefGoogle ScholarPubMed
Rabinowitz, N., Perbet, F., Song, F., Zhang, C., Eslami, S. A., & Botvinick, M. (2018). Machine theory of mind. In Dy, J. & Krause, A. (Eds.), International Conference on Machine Learning (pp. 4218–4227). PMLR.Google Scholar
Sanborn, A. N., Griffiths, T. L., & Navarro, D. J. (2010). Rational approximations to rational models: Alternative algorithms for category learning. Psychological Review, 117(4), 1144.CrossRefGoogle ScholarPubMed
Santos, R. J. (1996). Equivalence of regularization and truncated iteration for general ill-posed problems. Linear Algebra and its Applications, 236, 25–33.CrossRefGoogle Scholar
Smolensky, P. (1988). On the proper treatment of connectionism. Behavioral and Brain Sciences, 11(1), 1–23.CrossRefGoogle Scholar
Smolensky, P., McCoy, R., Fernandez, R., Goldrick, M., & Gao, J. (2022). Neurocompositional computing: From the central paradox of cognition to a new generation of AI systems. AI Magazine, 43(3), 308–322.CrossRefGoogle Scholar