Over the last few decades probabilistic models of cognition, which explain human behavior in terms of Bayesian inference over a set of hypotheses, have been applied to a wide range of phenomena (Griffiths, Chater, Kemp, Perfors, & Tenenbaum, Reference Griffiths, Chater, Kemp, Perfors and Tenenbaum2010). But what does the success of a particular probabilistic model in capturing human behavior imply? Probabilistic models of cognition are typically defined at Marr's (Reference Marr1982) “computational” level, characterizing the abstract problems human minds have to solve and their ideal solutions. More precisely, they characterize the ideal solutions to inductive problems, where an agent has to draw conclusions that go beyond the available data. The content of a probabilistic model of cognition, expressed via the set of hypotheses and their prior probabilities, is a claim about the inductive biases that guide such inferences – those factors other than the data that influence the hypothesis the agent selects (Mitchell, Reference Mitchell1997). Here, we argue that drawing conclusions that go beyond these inductive biases – and in particular, inferring support for specific cognitive processes and representations – can be problematic.

Inductive biases are at a different level of analysis from cognitive processes and representations, which Marr (Reference Marr1982) located at the “representation and algorithm” level. Representations and algorithms are notoriously underdetermined by observable data (Anderson, Reference Anderson1978). This underdetermination motivated Anderson (Reference Anderson1990) to develop rational analysis, the approach adopted in almost all applications of probabilistic models of cognition. This approach explicitly focuses on abstract problems and their ideal solutions rather than the processes and representations that implement them (inspiring critiques, e.g., Jones & Love, Reference Jones and Love2011). Probabilistic models need some way of representing hypotheses, but such representations do not necessarily guide human behavior. Rather, they are theoretical constructs that help scientists describe inductive biases.

Finding that a particular inductive bias seems to characterize human behavior places constraints on the representations and algorithms that might be involved, but those constraints rarely pick out a unique solution. To give a simple example, consider the problem of learning a linear relationship between two variables. A probabilistic model identifies a set of hypotheses (e.g., all linear functions), defines a prior distribution over those hypotheses, and then performs Bayesian inference. This kind of solution could be implemented by an agent that explicitly represents a set of linear functions and uses an algorithm to update its beliefs about the posterior probability of each hypothesis as new data are observed (see, e.g., Sanborn, Griffiths, & Navarro, Reference Sanborn, Griffiths and Navarro2010). The behavior of this agent will match that of the ideal Bayesian model. However, an agent that seems quite different – a neural network with one hidden layer and a linear output function that updates its weights by a few iterations of gradient descent – will also produce an answer that matches that Bayesian model (assuming a Gaussian prior; see Santos, Reference Santos1996). Two very different representations and algorithms are consistent with the same computational-level account (for a real modeling example, see Feldman, Griffiths, & Morgan, Reference Feldman, Griffiths and Morgan2009).

Quilty-Dunn et al. argue from the success of probabilistic models of cognition based on Bayesian inference over logical formulas and programs to the conclusion that people employ a similarly discrete and compositional “language-of-thought.” This argument crosses levels of analysis in the same problematic way. What we are licensed to conclude from the success of these models is that logical formulas and programs are useful in characterizing human inductive biases for certain problems, not that humans use these representations when solving those problems. Any stronger conclusion seems particularly problematic in light of the recent successes of deep neural networks that Quilty-Dunn et al. mention, because these systems may not require discrete or compositional representations. Metalearning – training a system to perform a set of related tasks – provides a way to create neural networks with specific inductive biases, and has formal connections to learning a prior for Bayesian inference (Grant, Finn, Levine, Darrell, & Griffiths, Reference Grant, Finn, Levine, Darrell and Griffiths2018). Metalearning has been used to train neural networks to perform tasks characterized at the computational level by Bayesian models based on symbolic representations, such as theory-of-mind (Rabinowitz et al., Reference Rabinowitz, Perbet, Song, Zhang, Eslami, Botvinick, Dy and Krause2018) and causal learning (Dasgupta et al., Reference Dasgupta, Wang, Chiappa, Mitrovic, Ortega, Raposo and Kurth-Nelson2019). Analysis of the internal representations of related systems shows that they contain information that can be used to reconstruct appropriate posterior distributions (Mikulik et al., Reference Mikulik, Delétang, McGrath, Genewein, Martic, Legg and Ortega2020). It thus seems plausible that such systems might produce behavior that is just as consistent with Bayesian inference over logical formulas and programs as that of humans.

The possible existence of deep neural networks that can be analyzed at the computational level in terms of Bayesian inference blocks strong conclusions about the language-of-thought, as the representations learned by these networks could emulate the associated behavior without requiring discreteness. Our investigations of networks trained by metalearning show that they can emulate human performance on abstract tasks without explicit representations of the relevant abstractions (Kumar et al., Reference Kumar, Correa, Dasgupta, Marjieh, Hu, Hawkins, Koyejo, Mohamed, Agarwal, Belgrave, Cho and Oh2022). In some cases, deep neural networks succeed on abstract tasks by learning compositionally structured representations (McCoy, Linzen, Dunbar, & Smolensky, Reference McCoy, Linzen, Dunbar and Smolensky2019), but these representations remain continuous, making them importantly different from the inherently discrete ones postulated in the language-of-thought hypothesis (Smolensky, Reference Smolensky1988). Such results align with theoretical work showing that compositional behavior does not require discrete representations (Smolensky, McCoy, Fernandez, Goldrick, & Gao, Reference Smolensky, McCoy, Fernandez, Goldrick and Gao2022). Indeed, the best current models of language itself – which is the prototypical example of a compositional domain (Pinker & Prince, Reference Pinker and Prince1988), as suggested by its use in the name language-of-thought – are deep networks that have continuous internal representations (e.g., Chowdhery et al., Reference Chowdhery, Narang, Devlin, Bosma, Mishra, Roberts and Fiedel2022).

Algorithms and representations may not be identifiable, but we can at least narrow down the equivalence class of possibilities through careful experimentation – behavioral work focused on response times and errors, neuroscientific studies of what the brain might be encoding, and computational simulations – designed to provide strong tests of alternative hypotheses. Until we can definitively do so, the fact that a discrete, compositional language-of-thought is useful as an abstract way of characterizing human inductive biases still allows the possibility that the actual representations and algorithms underlying human cognition may have a very different character.