2.1 Linguistics and the Philosophy of Cognitive Science

A dominant approach in the philosophy of linguistics views linguistics as a cognitive science. There are at least two distinct ways to understand this general position. The first is by far the most prominent within philosophical circles. Generative grammar has inspired a particular way to study the mind and language by introducing the cognitive scientific approach to linguistics.Footnote ⁴ Chomsky (Reference Chomsky1965, Reference Chomsky1957) set the philosophico-linguistic agenda (Freidin, Reference Freidin and Allan2013). Subsequent work laid further foundations in the philosophy of linguistics (Chomsky, Reference Chomsky1966, Reference Chomsky1986, Reference Chomsky2000). The second approach under considerations rejects many of the core tenets of the first while retaining the general aim of studying language via studying cognition. Both approaches provide unique insights into profound philosophical questions at the heart of the cognitive sciences.Footnote ⁵ Before we can appreciate either position, we need to discuss the core tenets of generative linguistics and the philosophical consequences of the view.

2.2 Linguistics and the Philosophy of Social Science

It might be strange to those familiar with the philosophy of linguistics to see a section in a book on the topic that includes discussion of its relevance to the philosophy of social science. The kinds of formalisms adopted in linguistics, at least in syntax and semantics, lend themselves to either an individual psychological interpretation or perhaps an abstract ontology. Yalcin (Reference Yalcin, Ball and Rabern2018 358), in discussing scientific modelling in formal semantics, states unequivocally that semantics is about ‘human linguistic competence—informally, knowledge of meaning’. Glanzberg (Reference Glanzberg, Burgess and Sherman2014, 31) concurs with this picture of semantics, insisting that ‘it is the state of mind, or the semantic aspect thereof (“knowledge of meaning”, understood in the technical sense), that semantics is foremost concerned with modeling’. One criticism of this internalist focus has been the alleged neglect of the communicative dimension of language. Language under certain views is primarily a tool for communication. If there is any kernel of truth in this claim, then linguistics is the study of a distinctly social phenomenon.

Many philosophers of language have recognised the central place of communication with their focus on ordinary language, externalism, reference, and conventionalis. Lewis (Reference Lewis and Gunderson1975) produced an influential picture of the synthesis of a formal language (or grammar qua formal mathematical object) and the conventions of a community of language users. In this sense, a language is both an abstract object and a model of the linguistic patterns of a community of speakers. Still, Lewis’ skepticism of the competence model is somewhat of an exception. In addition, his model remains rather abstract.

Within a broadly generative linguistic framework, Labov’s work stands out as a canonical treatment of the social aspects of grammar. In fact, his research gave birth to modern sociolinguistics. For example, he discovered fascinating regularities within neglected (or even maligned) dialects of English such as African American Vernacular English (AAVE). Chomskyans generally reject dialect studies at peripheral to linguistic theory as they allegedly concern political vagaries of the so-called external or E-languages. However, Labov showed that studying neglected (and oppressed) dialects of English, such as those spoken by working-class African American communities in the United States, provides a theoretical window into phenomena as diverse as structural variation, language contact, and language change. In Labov (Reference Labov1969), he uncovered a particularly intriguing regularity: that when copula contraction is permitted in Standard English, AAVE allows deletion (and the other way around). Furthermore, when contraction is not possible, then neither is deletion. So you can say Sane said she going but not *Clever, that’s what she. The AAVE also allows for negative concord, while most other dialects of English do not. Importantly, investigating these grammatical regularities led Labov towards the tools of social science such as peer group recordings. In a sense, Labov showed a fine-grained example of Lewis’ synthesis by expanding ‘grammar’ to account for inter- and intragroup linguistic regularities.Footnote ¹¹

It is hard to conceive of pragmatics without considering sociality and communication. Pragmatics itself is one of the most productive areas of contemporary linguistics (Nefdt, Reference Nefdt2024). As early as Grice (Reference Grice, Cole and Morgan1975), the logic behind conversational structure was revealed. This structure is rich and implicates multiple cognitive elements such as ‘theory of mind’.Footnote ¹² The idea is that language involves complex inferential patterns which operate over and above semantic or literal meaning to convey momentary conversation dynamics as well as conventional pragmatic effects. Even the seminal Stalnaker (Reference Stalnaker1978, Reference Stalnaker2002) formally models the evolution of conversational dynamics as participants interact within an ever-changing common ground of informational states. He defines the common ground as the set of propositions that participants ‘mutually accept’ for the purposes of conversation, where mutual acceptance involves an iterated acknowledgement between parties of their respective acceptances (A accepts that B accepts that A accepts that $\dots$ and so on, ad infinitum) (Stalnaker, Reference Stalnaker2014). Dynamic semantics exploits these dynamics to produce a model of semantic meaning itself as context-change potential (Dekker, Reference Dekker2012; Veltman, Reference Veltman1996).Footnote ¹³

It might be objected that it is not quite accurate to describe formal pragmatics, the kind of research mentioned in previous paragraph, as a social scientific enterprise for a number of reasons. Here the distinction in Korta and Perry (Reference Korta, Perry, Korta and Perry2008) between ‘near side’ and ‘far side’ pragmatics is useful. The former involves work on indexicality and presupposition which implicates interpersonal relations but remains highly formal and abstract. The far side brings a bridge between formal theory and ordinary usage is built. Here they locate Grice’s (and J.L Austin’s) work, which go beyond saying and approach usage-based accounts of language and communication. As discussed in Keiser (Reference Keiser2022), much of pragmatics remains highly idealised and abstract.

A further connection with social science presents itself in the many game-theoretic approaches to pragmatics in formal linguistics. As effective as game theory has been in economics where rationality, social cognition, cooperation, and similar concepts have been extensively modelled, it has also proven a useful tool in linguistics. Again, Lewis (Reference Lewis1979) explores this game-theoretic approach to communicative presupposition and conversation based on an analogy with baseball scores. In fact, Lewis (Reference Lewis1969) established the relevance of game theory, and signalling games in particular, to the study of linguistic conventions. Game-theoretic pragmatics is now a burgeoning field (see Franke, Reference Franke2013).

The bridge from pragmatics to social science is a short one. Firstly, communicative practices need not assume a monolithic context. Pragmatic phenomena are ubiquitous and are arguable even present at the birth of new languages when language contact is involved (Bickerton, Reference Bickerton1975). Horn and Kecskes (Reference Horn, Kecskes, Anderson, Moeschler and Reboul2013) discuss two types of social pragmatics. They can be grouped under the terms ‘socio-cultural interactional pragmatics’ (SCIP) and ‘intercultural pragmatics’ (IP), respectively. The SCIP broadens the scope of pragmatics to include social and cultural constraints in addition to the linguistic constraints of the mainstream theories. Secondly, it goes beyond the utterance and immediate context level to characterise speaker meaning through the dialogue sequence (sets of utterances) or discourse segment. The IP, on the other hand, incorporates intention-based classical views more centrally while refocusing the field on emergent phenomena within communication. ‘In this approach interlocutors are considered as social beings searching for meaning with individual minds embedded in a sociocultural collectivity’ (Horn & Kecskes, Reference Horn, Kecskes, Anderson, Moeschler and Reboul2013, 365). This is a form of what the authors call ‘sociocognitive pragmatics’.

The social nature of language might go deeper and beyond pragmatics and sociolinguistics. In fact, as far back and Trubetzkoy (Reference Trubetzkoy1958), phonology and its basic unit the ‘phoneme’ have been attributed social-normative force. Speech patterns are certainly affected by conformity, gender, pressure, class, and a number of other social factors. Itkonen (Reference Itkonen2001) investigates the social interpretation of phonetics and phonology while questioning the predominantly mentalist interpretation. In related fashion, in a recent article in Science, Fedorenko, Piantadosi, and Gibson (Reference Fedorenko, Piantadosi and Gibson2024) argue that language is optimised for communication and evolved as a means of cultural transmission of knowledge. They marshal information-theoretic evidence in their attempt to dismount the Chomskyan claim that the primary function of language is for thought.

Viewing linguistics as a social science opens up myriad possibilities for the philosophy of social science. Language has been one of the most studied human phenomena. The tools, results, and insights from research into its social components are surely important for social science in general. For instance, pragmatics provides a window into the inferential underpinnings of communication. This in turn gives us insight into linguistic reasoning and social rationality. One example of the possible cross-fertilisation is Kretzschmar (Reference Kretzschmar2015). In it, he shows via corpus data that an emergent non-linearity characteristic of market economies is omnipresent in language, namely the Pareto or 80/20 principle in which 80 per cent of wealth is concentrated within 20 per cent of the populace. This dovetails on Zipf’s Law, in which the frequency of words in a corpus is inversely proportional to their ranking (i.e. the most common word occurs twice as often as the second most common and three times as often as the next and so on).Footnote ¹⁴ He claims that the Pareto principle shows up all over the data at various levels and that ‘we in language studies can and should make good practical use of the 80/20 Rule on a conceptual basis’ (Kretzschmar, Reference Kretzschmar2015, 85). Pareto equilibrium as a tool is also found in game-theoretic pragmatic settings.

It seems clear that the philosophy of linguistics (and linguistics itself) has a lot to offer the philosophy of social science and vice versa. Despite this, the connections are seldom explored within either field. A more prominent connection is between the philosophy of linguistics and the philosophy of biology given the recent focus on language evolution within linguistic theory.

2.3 Linguistics and the Philosophy of Biology

Evolutionary concerns have never been far removed from questions of language. Darwin and Wallace reflected on it, with the former favouring an adaptionist, gradualist approach which saw some connections with the origin of other species and their communicative systems (Alter, Reference Alter and Ruse2013; Darwin, Reference Darwin1871). In fact, as the story goes, speculation on the origins of language were so rife at some point that in 1866 the Linguistic Society of Paris effectively banned all discussion of it (as did the London Philological Society shortly after in 1872) (see Kenneally, Reference Kenneally2007). It took some time for the conversation to not only gather steam again but also eventually resituate theoretical linguistics and its philosophy.

One explicit move in this direction can be found in the seminal work of Eric Lenneberg, especially Lenneberg (Reference Lenneberg1967). Here, Lenneberg lays the foundations for the biological study of language. Specifically, he defined the ‘critical period hypothesis’. This hypothesis states that there is a critical period (between two and just before puberty) in which a language learner can acquire their first language. He linked this constrained ability to the process of lateralisation in the brain. But similar critical periods had been noted in other species such as songbirds. The important part of his analysis was its influence on the much later biolinguistics movement. His goal was explicitly to ‘reinstate the concept of a biological basis of language capacities and to make the specific assumptions so explicit that they may be subjected to empirical tests’ (Lenneberg, Reference Lenneberg1967, vii). This idea of language as either an actual biological organ or a ‘natural object’ has formed the bedrock of the movement. Anderson and Lightfoot (Reference Anderson and Lightfoot2002) devote an entire book to developing this biological metaphor into a concrete interpretation of a language growing in the mind of speakers. They conclude that ‘there is a biological entity, a finite mental organ, which develops in children along one of a number of paths. $\dots$ The language organ that emerges, the grammar, is represented in the brain and plays a central role in the person’s use of language’ (Anderson & Lightfoot, Reference Anderson and Lightfoot2002, 22).

There are a number of components to this picture of the biology of language. Some such as modularity, innateness, poverty of stimulus, and grammar are inherited from the erstwhile generative programme, while others such as optimality, evolvability, and neurological structure are taken from a mixture of Chomsky (Reference Chomsky1995b) and later work by many others. For example, Bever (Reference Bever, Sanz, Laka and Tanenhaus2013) reviews the development of the biolinguistics programme from its inception to the (then) present day. He specifically investigates in what sense language viewed as an organ corresponds with other senses of the word in biology. This brings up the work on the genetics of language. At some point, scholars were optimistic about the possibility of a FoxP2 or ‘the language gene’ (Gopnik, Reference Gopnik1990). Among other things, such work offers a genuine path to the innateness characteristic of biolinguistics. As Bever (Reference Bever, Sanz, Laka and Tanenhaus2013, 401) notes, ‘the usual method (in principle) is to isolate a particular genetic abnormality and relate it to the selective sparing or selective impairment of language ability, thereby making more specific the claim that language is “innate”.

However, the hype outstripped the hypothesis. While the idea of a grammar gene offered biological grounding for the field, further studies proved that the FoxP2 gene was not human specific, nor did defects associated with it manifest purely linguistically (Sampson, Reference Sampson2005; Vargha-Khadem, Reference Vargha-Khadem, Gadian, Copp and Mishkin2005). Murphy (Reference Murphy2012) insists that the posit is more revealing of linguistic performance than competence in any case. Nevertheless, the work of Lenneberg, Chomsky, Bever, and others aimed at uncovering the biological substrate of linguistics and resetting the agenda. It can be seen as what Boeckx and Grohmann (Reference Boeckx and Grohmann2007), in their launching editorial of the journal Biolinguistics, call ‘the strong sense’ of the term. This idea involves the use of insights from biology as a driving force for linguistic theory. The weaker sense does not constrain generative linguistic theory directly as the minimalist programme would suggest or work on the genetics or neurobiology of language might do. Murphy (Reference Murphy2012) approaches this topic from numerous angles. His exposition of biolinguistics and its relation to philosophy (chapter 1) highlights a number of connected strains, both in terms of weak and strong versions. In it, he makes a potent case for the modularity of language, the organic interpretation of I-language (quite literally growing in the brain), and the ancillarity of communication to language. It certainly seems clear from Murphy (Reference Murphy2012) (and Chomsky, Reference Chomsky2000) that the traditional philosophy of language offers little in terms of correspondence with human biology.

One interesting aspect of this model of biolinguistics (in either the weak or strong form) is its relative isolation from standard biological theory. The most prominent departure presents itself within the saltational account of the evolution of language. Minimalism is the view that the basic mechanism of language (or narrow syntax) is the operation of Merge. It is a bottom-up structure building operation that takes two syntactic objects and combines them with the projected label of the head. Merge acts as our explanans towards the goal of modelling the explanandum or ‘basic property’ of language: ‘each language provides an unbounded array of hierarchically structured expressions that receive interpretations at two interfaces, sensorimotor for externalization and conceptual-intentional for mental processes’ (Chomsky, Reference Chomsky2013, 647). This latter property is usually connected to the linguistic infinity postulate (which we will encounter in Section 4). From an evolutionary perspective, the task of explaining the origins of language becomes a task of explaining the origins of Merge. The biolinguistic story goes something like this:

What results is an explosion of intelligence or a macromutation which set human beings on a path towards cognitive dominance on the planet. But whither natural selection, deep homologies, the tapestry of cross-species adaptations when it comes to language? Many of the standard accounts in evolutionary biology seem to be missing in this project. For instance, on this account, communication is an exaptation. The central evolutionary nexus is the language faculty, represented by its proxy Merge, meaning that ‘communication is merely a possible function of the language faculty, and cannot be equated with it’ (Friederici et al., Reference Friederici, Chomsky, Berwick, Moro and Bolhuis2017, 713).

But of course, every part of the puzzle is up for grabs. One could question the timeline with Dediu and Levinson (Reference Dediu and Levinson2013) and Steedman (Reference Steedman2017). One could question the uniqueness with Everett (Reference Everett2017) or the trigger with Bickerton (Reference Bickerton2014), who adds a niche construction theory on top of the Merge saltation account. Views like Tomasello (Reference Tomasello2008) embrace the communicative and cross-species approach while introducing pragmatics and gesture to the evolutionary fold. Even staying within generative grammar, one could opt for a gradualist (more Darwinian) approach with Progovac (Reference Progovac2015, Reference Progovac2016). Or one could even question whether Merge’s emergence was a single step or broken up into multiple stages (Martins & Boeckx, Reference Martins and Boeckx2019). There are more stark departures and various variations on the theme (Nefdt, Reference Nefdt2023b). The situation perhaps makes one sympathetic to the French academy’s erstwhile injunction.

Many philosophers, and cognitive linguists, reject the demotion of communication in language evolution. Millikan’s work stands out as both philosophically rich and directly inspired by biological theory. Her view draws from an overarching philosophical framework called teleosemantics. Teleosemantics covers more than just linguistic entities like sentences and words but the nature of representation itself. In contrast to truth conditional accounts of meaning and representation, ‘[a]ccording to the teleosemantic program, representations are states whose biological function is to guide behavior in ways appropriate to such-and-such conditions’ (Papineau, Reference Papineau and Smith2016, 97). The states in question are usually mental states like beliefs, desires, perceptions, and so on. However, Shea (Reference Shea2018) presents a compelling telesemantic account of subpersonal representational states. Millikan directs her attention towards linguistic conventions and a uniquely biological account of words.

Millikan’s account starts with the concept of proper function (Millikan, Reference Millikan1984, Reference Millikan2005). The proper function of something is basically what that thing is supposed to do (hence the ‘telos’). The function of my heart is to pump blood, the function of my pen is to write. The rabbit’s stomping of its back legs is a social indication of danger to other rabbits like the bee’s famous waggle dance informs other bees of the direction and distance of food sources. Something may have a proper function even if it malfunctions in some way. Millikan’s extended this to linguistic representations as biological categories with proper functions. Two subsystems comprise this representation system: (1) producer systems, and (2) the consumer systems. Take the bees again. The producer’s function (to produce a dance) and the consumer’s function (to retrieve information on food sources) contribute to the function of the entire system (the colony’s survival). This system only works in normal conditions (which give us the content of the representation). With this in place, and a modification of Lewis (Reference Lewis1969, Reference Lewis and Gunderson1975), the resulting view of language is one in which communication is the primary function and language is ‘a set of speaker-hearer interactions forming lineages roughly in the biological sense’ (Millikan, Reference Millikan, Antony and Hornstein2003). Words on such a view are like species identifiable by their lineages (see Richard (Reference Richard2019) for a similar view of ‘meanings’ of words as biological species).

The intervention of cultural norms and conventions on linguistic evolution is taken up in later work as well. Dennet (Reference Dennett2017) incorporates work on genes and memes to produce a mimetic account of the proliferation and evolution of language, with words at the centre. Planer and Sterelny (Reference Planer and Sterelny2021) start from the point of the messiness of biological kinds to develop a multi-causal, empirically rich picture of the evolution of language, divorced in some ways from the formalised models of linguistic theory. Nefdt (Reference Nefdt2023b) invites complexity science and biological systems theory into the discussion of language evolution while attempting to maintain some connections with more minimalist accounts of biolinguistics.

Once again, the fruitful collaboration between biology and linguistics offers many avenues of insight. In turn, the respective philosophies of these subjects are intertwined in terms of their commingling of culture, genetics, and cognition. And we have not considered the role neurobiology at all so far. Of course, the field has been marshalled to support generative linguistics (Berwick, Reference Berwick, Friederici, Chomsky and Bolhuis2013; Friederici et al., Reference Friederici, Chomsky, Berwick, Moro and Bolhuis2017; Marantz, Reference Marantz2005; Murphy, Reference Murphy2023) and also to question the connection between linguistic theory and biology (Baggio, Reference Baggio, Nefdt, Klippi and Karstens2020; Poeppel & Embick, Reference Poeppel, Embick and Cutler2005). We will return to neurolinguistics in Section 6 as it possibly offers the most promising grounding for biolinguistics and linguistics in general. Another promising line is pursued Balari and colleagues. Balari et al. Balari et al. (Reference Balari, Benítez-Burraco, Longa, Lorenzo, Grohmann and Boeckx2013) offer a palaeontological perspective while Balari and González (Reference Balari and González2013); Balari and Lorenzo (Reference Balari and Lorenzo2009) apply evo-devo thinking to biolinguistics.

From the perspective of the philosophy of biology, two contrasts seem to unfold: either language is truly biologically unique and thus pushes biology itself to adapt to new tools of exploration or philosophers of linguistics (and theoretical linguists) have not explored the full range of options from biology to account.

3.1 Scientific Revolutions

The nature of scientific revolutions and the concomitant sociology of science have been enduring topics within the philosophy of science since, at least, Kuhn’s magnum opus: The Structure of Scientific Revolutions (Kuhn, Reference Kuhn1962). The basic picture is this: normal science operates with relative progress and consistency under a broad paradigm or governing framework, that is, until the problems mount for the paradigm over time, resulting in a crisis. While ‘rationality’ might be attributable to normal science, crises are capricious and often driven by politics, historical accidents, and personal factors. In turn, crises are the only revolutionary mechanisms which can dismount entrenched views and theories. Despite their essential function, revolutions remain disorderly and chaotic. Indeed, Kuhn suggests that this might be a consequence of the fact that scientific evidence itself is under scrutiny in a revolution.

Many, if not most, of these components have been challenged in some form or other. For example, it is not clear whether crises precipitate paradigm shifts in every case (Godfrey-Smith, Reference Godfrey-Smith2003). It is equally unclear whether true incommensurability follows from revolutions as structuralists have pushed for decades (Ladyman, Reference Ladyman1998; Worrall, Reference Worrall1989). But as an idealised model, some core elements of the Kuhnian picture remain useful for understanding revolutions (linguistic or otherwise), such as (1) periods of paradigmatic normal science, (2) accumulation of quandaries leading to revolution, (3) the establishment of a new paradigm, partly inspired by performance on those quandaries, and (4) the fact that (3) is driven by non-rational and sociological factors.

3.2 Justification and Explanation

Two related and central topics in the philosophy of science are justification and explanation, respectively. There are a number of theories and models of how theories and models work in science. What counts as justification or confirmation is a deeply epistemic suite of questions. What makes a good explanation cannot be thoroughly divorced from psychology (for Quine (Reference Quine1969), epistemology was just a form of psychology). In many senses, the trajectory of linguistic theory has attempted to borrow the justificatory methods and explanatory status of the natural sciences. The project of naturalism is rarely seen in such splendour across the other special sciences (Chomsky, Reference Chomsky2000; Johnson, Reference Johnson2007; Nefdt, Reference Nefdt2023a). In one way, this formal rigour and naturalistic aspiration might have inoculated linguistics from the ‘theory crisis’ emerging in the other special sciences like psychology (Markus & Bringmann, Reference Markus and Bringmann2021; van Rooij & Baggio, Reference van Rooij and Baggio2020). On the other hand, the empirical credentials of linguistics has often been called into question due to the kinds of idealisations it celebrates, namely rationalistic mathematical ones. Theoretical linguists have also favoured explanation over prediction, which manifests itself in a general disdain for stochastic methods (see Section 5). This conglomeration of factors produces a thick soup of ingredients for the philosophy of science. Unfortunately, the short space afforded here can only facilitate a guide to the literature but hopefully that will suffice as an entreaty to further investigation.

3.3 Realism versus Antirealism

A book about the philosophy of science that does not talk about the scientific realism would $\dots$ probably be fine and cover other issues. But that doesn’t mean that scientific realism hasn’t commanded significant amounts of conceptual space in the field. It would not be an exaggeration to say that it underlies most of the issues we have encountered so far. Does the predictive successes of science mandate some ontologically committing stance on the objects it posits? What are scientific explanations if not descriptions of some (human) mind independent reality that go beyond our instruments of discovery? Does paradigm shift and theory change affect the veracity of current theories? All of these topics are important and lucky enough to feature prominently in countless textbooks and monographs. So we will restrict ourselves to the ways in which this larger issue manifests in the philosophy of linguistics.

Before we embark on this brief sojourn, some specifics on what exactly scientific realism amounts to. Devitt (Reference Devitt, Curd and Psillos2005), our part-time philosopher of linguistics, is useful here. First he distinguishes between ‘common-sense’ realism and scientific realism. The former seems committed to the existence of observable entities of both common sense and science. But Devitt’s view on scientific realism is that it is only committed to the existence of the unobservables posited by our mature theories. This latter move is meant to soften the blow of pessimistic meta-induction (PMI) to a certain extent.Footnote ²²

We see this form of scientific realism quite ostensibly in the philosophy of linguistics. Theorists often state commitment to movement, hierarchical structure, and Merge but deny the significance of surface forms. Chomsky (Reference Chomsky2000) is clear that our intuitions about language (and mind) are bad indications of the real linguistic value. In fact, the very distinction between E-languages, which are amorphous public entities, and I-languages, the unobservable structure of the mature state of the language faculty, is illuminating. The idea that latter are meant to be the true objects of scientific linguistics rests on the idea that the unobservables reflect a deeper reality. Rey (Reference Rey2020) is an interesting case. He supports mainstream generative linguistic theory but disagrees on the ontology of ‘standard linguistic entities’ like noun phrases and phonemes. He considers such objects to be like Kanizsa triangles or optical illusions that explain intentional contents but do not map onto anything in the world. He calls these entities ‘intentional inexistents’. The position is intriguing. It eschews ontological commitment to the objects of linguistic theory while maintaining a realist stance towards linguistic theory in general.

What about languages themselves? Are they real? Well, this is beautifully complicated in the philosophy of linguistics. For mentalists, language is both real and mind-dependent (there would be no languages without human minds). But public languages, or E-languages, are not scientifically real even if they might be thought of as commonsensically real. Just to add even more flavour. Chomskyans seem to admit their own theoretical posits are proxies for future neurological events and processes. Chomsky states, of Merge and other posits, that ‘if we want a productive theory-constructive [effort], we’re going to have to relax our stringent criteria and accept things that we know don’t make any sense, and hope that some day somebody will make some sense out of them’ (Chomsky & McGilvray, Reference Chomsky and McGilvray2012, 91). Behme (Reference Behme2014), in her very critical review of the book from which the quote comes, asks how it is possible to defend a view that accepts nonsensical things. But the history of quantum mechanics provides ample examples of reluctant realism and proclaimed future resolutions to ontological puzzles.Footnote ²³ We will return to some of these prospects in Section 6.

The standard arguments against scientific realism have unique instantiations in the philosophy of linguistics. And more intriguingly, the most common argument for it has limited scope. As previously mentioned, Lappin, Levine, and Johnson (Reference Lappin, Levine and Johnson2000) argue that generative linguistics itself underwent radical theory change from the early theories (the Standard, Extended, and Government and Binding) to minimalism. Generativists insist that minimalism just marks a shift from descriptive adequacy to explanatory (or beyond explanatory) adequacy. This means that the theory moved from concerns about describing the acquisition data, a task that could marshal significant innate resources, to one that aimed to explain the evolution of language, a task that requires a vast reduction of such resources. This move also confronts the issue of the underdetermination of theory by data. What allegedly makes minimalism a ‘better’ theory than earlier ones (or non-generative rival accounts) is its adherence to simplicity and optimality as biologically constraining features. Chomsky (Reference Chomsky2005) outlines what he calls third factor principles that shape the language faculty. Economy principles, efficient computation, genetic endowment, and a host of others all form part of these language-independent principles that nonetheless have shaped language evolution. Thus, two theories that are descriptively equivalent (or equally empirically adequate in the parlance of the philosophy of science) can be separated by means of third factor considerations. It seems both the PMI and the underdetermination of theory by evidence have special significance in the philosophy of linguistics.

One of the strongest arguments put in favour of realism is the ‘no-miracles’ argument (Putnam, Reference Putnam and Putnam1975). The argument states that realism is the only game in town, philosophically speaking. If we want to explain the successes of science, anti-realist views are simply at a loss, rendering the many and major achievements serendipitous or even miraculous. The argument, however, turns on more than just explanatory success. Science has found vast technological applications from space exploration to smart phone technology. These successes required significant predictive and experimental advancements. This is where theoretical linguistics falters to some extent. Despite the initial promise, language technology, psycho- and neurolinguistics have not greatly benefitted from formal theoretical models. A well-known anecdote attributed to Frederick Jelinek, which made the rounds in computer science for decades, is that ‘Every time I fire a linguist, the performance of the speech recognizer goes up’ (Jelinek, Reference Jelinek1988). Returning to miracles, the good part is that the argument does not entail only one kind of realism about science. Theoretical models might still have a shot at a different sort of realism.

Nefdt (Reference Nefdt2023a, Reference Nefdt2021) offers a structural realist alternative to addressing both the PMI and the underdetermination issue (under the guise of multiple theories problem). Following French (Reference French2014), Ladyman (Reference Ladyman1998), and Ladyman & Ross, Reference Ladyman and Ross2007), he suggests that structural realism in linguistics too presents a ‘best of both worlds’ strategy (Worrall, Reference Worrall1989). For generative linguists, it provides much-needed structural continuity (of which there is much to find), and for the connection with other linguistic theories, unification is offered as a model of explanation (Kitcher, Reference Kitcher, Kitcher and Salmon1989). On this view, a language is like a real pattern, in the sense of Dennett (Reference Dennett1991), and a grammar a special kind of compression algorithm.

Constructive empiricism has not been explicitly applied to linguistics as far as I can tell. This is strange given its dominant status as an anti-realist alternative and its relevance to many views about language. It’s chief proponent is van Fraassen (Reference van Fraassen1980), who laid the groundwork for a complex account of science. The mantra for constructive empiricism is that ‘science gets the observables right’. In other words, empirical adequacy is the goal of science. But along the way, the view makes a few interesting moves. It adds a dose of scepticism to the unobservables (dovetailing with Rey (Reference Rey2020)), a literalism about the language of science, and a reduction of theoretical terms to lists of observation statements. Many linguistic theories, so-called surface grammars, adopt a similar stance. American structuralists and probabilistic grammarians aim to get the observables right and are reticent about what underlies them (at least what entities). Dependency grammars posit no underlying structures or invisible movement operations (Nefdt & Baggio, 2024b). They operate purely on the level of observable words and morphemes. In the literature, this is called a lexicalised grammar. There are many such frameworks to choose from.

The key to constructive empiricism is the rejection of the criterion of truth (or even approximate truth) for our scientific theories. In its place is a modal concept of what the world could be like (without ever telling us exactly which world we live in). This strategy obviates both the PMI and underdetermination to different degrees. Since scientific progress is just the accumulation of observable regularities or results, the unobservables can change as they like. We also don’t have to choose between rival theories which are empirically equivalent unless we have some pragmatic grounds for doing so. In terms of explanation, constructive empiricism was partly responsible for the boom in work on scientific models and their role in scientific explanation and prediction.

For constructive empiricism to help the philosophy of linguistics, we might need a clearer notion of the observable—unobservable distinction. But here, the philosophy of linguistics might be able to provide some assistance itself. Notice how many linguistic concepts are buried within this theory. Besides the pragmatics of theory choice and the semantic literalness of scientific discourse, linguists have been labouring over the observable—unobservable distinction for decades in the form of the competence—performance distinction (assumed in many if not most theories to varying degrees). Reflections on the relationship between ideal competence and actual performance in linguistic theory could shed light on how to draw the distinction elsewhere.

There is more to unpack here. One further curiosity is that platonists, mentalists, and nominalists in the philosophy of linguistics all make use of the term ‘realist’ to characterise their views. So the realism is really about what to be realist about, sets, mental grammars, or splotches of ink on paper. We will have to leave matters here for now as the metaphysics of language is beyond the current scope. In the next section, we explore a case study in the philosophy of linguistics, one that speaks directly to intricate issues in the philosophy of science (and mathematics) but rarely finds representation in those spaces.

4.1 What Is Infinity?

There are many excellent introductory texts on infinity in mathematics.Footnote ²⁵ The notion of the infinite has been prominent in philosophy, theology, and scientific thought for centuries. However, it wasn’t until the work of Georg Cantor in the late 1800s that mathematics truly captured the idea precisely. One particularly central insight or discovery was that there are different sizes (or cardinalities) of infinity. Let us start by distinguishing finitude, countable infinitude, and uncountable infinitude.

Consider the famous thought experiment of Hilbert’s hotel.Footnote ²⁶ If there was a hotel filled to capacity with finitely many rooms, any newly arriving guests would have to be turned away. However, if the hotel was infinite in that it had rooms for every natural number (starting from 0), such that guest 0 occupies room 0, guest 1 occupies room 1, and so on, it could be full and still admit new guests. How is this possible? Well, imagine a new guest named Georg shows up and asks for a room. The hotel manager would merely have to ask every current guest to move up one room from their present location. This procedure will free up room 0 (as the former occupant of room 0 would now be in room 1 and so on). This only works because the axioms of Peano arithmetic guarantee us that every number has a unique successor, so there is no largest natural number. In fact, no matter how large the finite magnitude of guests that arrive at the hotel, our trusty hotel manager can accommodate them by merely relocating guest n to room n + that number. If an infinite number of guests arrive, a similar procedure can be implemented. Simply move guest n to room 2n, thus freeing up an infinite number of rooms for occupancy. The question then looms, are there any magnitudes of guests that our hotel cannot accommodate according to this procedure? To answer this question, we need to introduce some terminology.

The cardinality of a set (represented as $\mid A\mid$ for set $A$) tells us how many members the set contains. In set theory, the standard mechanism for determining the relative size or cardinality of sets is bijection. This is an equivalence relation (that is a reflexive, symmetric, and transitive relation) that ensures that between two sets A and B, there is a function that maps each member of A to a different member of B, and no member of B is left behind in the pairing. In finite sets, this means that two sets have the same of cardinality iff there is a bijection between them.

A number of infinite sets can be shown to have the same cardinality by establishing bijections between their members. Consider the set of natural numbers $\mathbb{N}$ and the set of integers $\mathbb{Z}$ (containing the natural numbers and their additive inverses). The procedure for showing these sets to be of the same size is relatively straightforward. One way of doing it is by assigning all the even numbers from $\mathbb{N}$ to all the positive integers and the odd numbers to the negative ones. Showing that there is a bijection from $\mathbb{N}$ to $\mathbb{Q}$ is also relatively simple (although it involves ignoring some identical fractions with different labels like 1/2 and 2/4). This brings us to the definition of a countable infinity, that is, any set that can be shown to have a bijection with $\mathbb{N}$.

Returning to our question of whether there are any infinite sets of guests that would have to be turned away from Hilbert’s Hotel, a result known as Cantor’s theorem sheds the necessary light. Basically, Cantor’s theorem states that sets have more subsets than members or ‘for any set A, $\mid A\mid <\mid \mathcal{P}(A)\mid$’ (Rayo, Reference Rayo2019, 11).Footnote ²⁷ This results in an iterative infinity of infinite cardinalities. More importantly for our purposes, it distinguishes the real numbers $\mathbb{R}$ from $\mathbb{N}$. In fact, it turns out that $\mid \mathbb{R}\mid =\mid \mathcal{P}(\mathbb{N})\mid$. That means that the set of real numbers is strictly larger than that of the natural numbers, it is uncountable. In other words, if a busload of guests show up with the cardinality of real numbers, our hotel manager will be in real trouble.

An important fact to note is that once we move past finite magnitudes, intuition begins to fail us. Specifically, our intuitions on relative sizes of sets seem to imply what Rayo calls the ‘proper subset principle’: if A is a proper subset of B, then B is larger (has more members) than A. When we reach infinite sets, this principle contradicts the bijection principle, that is, that two sets are the same size if there is a bijection between them. For instance, the set of natural numbers is a proper subset of the set of rational numbers ($\mathbb{N}\subset \mathbb{Q}$) but, according to Cantor’s conception of sameness of size, is of equal cardinality (or even more starkly, the even numbers are a proper subset of the natural numbers yet have the same cardinality). The set of real numbers can be constructed by defining Dedekind cuts on the rational numbers. But it isn’t the fact that the set of natural numbers (or rationals) is contained in the set of reals which makes the latter uncountable but rather what seals its infinite fate is that lack of a bijection.

4.2 Linguistic Infinity

When generative linguists talk about infinity, they often use the term ‘discrete infinity’ to mean a countable infinity. Recently, Chomsky et al. (Reference Chomsky, Seely and Berwick2023, 1) have once again placed the concept at the centre of the field:

Chomsky himself has repeatedly stated that infinity marks a unique departure from other biological phenomena and systems in nature.Footnote ²⁸

Let us call this the linguistic infinity postulate (lip). There are a number of popular avenues for establishing lip (although it is often just stated without argument). From the literature, there is one usual strategy for establishing this result. It attempts a proof-theoretic reconstruction of an insight, attributed to Willem von Humboldt, that language is ‘infinite use of finite means’. This strategy allegedly produces the requisite analogy with the natural numbers.

The literature on recursion in linguistics is complicated. Specifically, it is unclear at which level (or even which object) the property is meant to be interpreted. I cannot embark on a mission to disentangle the many vines of this particularly thorny issue here (see Tomalin (Reference Tomalin2006) and Lobina (Reference Lobina2017) for some perspective). Basically, the mathematical definition involves some property of self-reference, usually in two steps: one that specifies the condition of termination of the recursion (or the base case) and the recursive step which reduces all other cases to the base. For example, addition in Peano arithmetic can be defined recursively as $x+0=x$, $x+Sy=S(x+y)$ (where ‘S’ denotes the ‘successor of’ one place relation). Similarly, the rewrite rules for a context-free grammar are recursive: $S\to ab$ or $S\to aSb$ (where ‘S’ is now a unique start symbol or category and ‘a’ and ‘b’ are terminal elements like words).

This brings us to the first argument in favour of discrete infinity, that is, that generative grammars output infinite expressions or grammatical structures. If a grammar G is a set of finite rules for defining stringsets (or sequences of symbols), then a language is the set of all the strings that G generates. In post-production system style, ‘G will be said to generate a string w consisting of symbols from $\mathrm{\Sigma }$ [the alphabet] if and only if it is possible to start with S and produce w through some finite sequence of rule applications’ (Jäger & Rogers, 1957, Reference Jäger and Rogers2012). What does this mean for lip? Well, it comes down to what is known as the ‘membership problem’. Generative grammars have sharp boundaries. Some string w is either generated (or derived) by the rules of G or it isn’t. It is either in the language or it is not. There is no in between. This means that the language that the grammar generates ‘has an exact cardinality: it contains either some finite number of objects or a countable infinity of them’ (Pullum, Reference Pullum2013, 496). Furthermore, the reason for opting for a countable infinity (i.e. lip) allegedly comes from the data of natural language itself.

In order to establish a connection between the set of natural language expressions and the set of natural numbers, we need a few things. Firstly, we need an analogy with the successor function in Peano arithmetic. Remember, the successor function applies to an object in the set and produces another object in that same set endlessly. To mount an analogy, we would need a similar kind of operation in language. The usual candidates come from syntax.

The idea is something like: English syntax is ‘closed under adverbial modification’ (or coordination or subordination, etc.). In other words, if ‘Noam is brilliant’ is in the set of English sentences or, if you prefer, generated by the rules of English, then so is ‘Noam is very brilliant’, ‘Noam is very very brilliant’ so on ad infinitum. The same strategy can be employed for other common examples involving subordination, centre embedding, conjunction, and so on. Pullum and Scholz (Reference Pullum, Scholz and van der Hulst2010) call this the ‘no maximal length claim’ (nml) or the claim that there is no longest English sentence (i.e. you can always create a longer one). The problem they point out is that to establish this analogy, one would have to assume both that every English expression has a successor, and that this function is injective somehow. But doing so would essentially be begging the question.

It should be noted that Chomsky (Reference Chomsky, Freidin, Otero, Zubizarreta and Keyser2008) does attempt to make a direct argument concerning the Merge operation of Minimalism and arithmetic. He outlines the following procedure for mimicking the successor function within linguistics.

The idea is that Merge applied over a lexicon with a single entry results in the successor function. Mukerji (Reference Mukerji2022) and Hinzen and Uriagereka (Reference Hinzen and Uriagereka2006) too make the connection between arithmetic and language. However, the problem is that Merge, or the set-theoretic operation of taking two objects and creating a new labelled unit with the two objects as (unordered) members, has nothing to do with language in and of itself. Making the argument that natural language syntax can be characterised with this simple operation is a separate task (one that minimalists have taken up for twenty years; see C. Collins and Stabler (Reference Collins and Stabler2016) for a clear formalisation). But merely showing that you can model the successor function with a recursive set-theoretic operation does not settle the matter.

This brings us to an important assumption about the nature of generative grammars imbued with recursive rules, namely that they necessarily yield or entail a discrete infinity of linguistic outputs. Again, Pullum and Scholz (Reference Pullum, Scholz and van der Hulst2010) show that this entailment doesn’t hold by means of a toy example using a context-sensitive grammar. The grammar contains a recursive rule ($VP\to VPVP$) which is only capable of a single application given the specified lexicon, but nevertheless remains non-trivially recursive. Thus, the recursive element of the grammar does not ensure infinite output (even if it can create infinite structures).

More recently, Huybregts (Reference Huybregts2019), a firm adherent of lip, insists that infinite cardinality is the default or null hypothesis of formal grammars. His target is evolutionary accounts of Merge as a step-wise or gradualist approach to infinite language along a cultural or evolutionary path (starting with some finite magnitude). He reiterates nml as a datum. What is interesting is where he locates the infinitude. He states ‘[w]e should be careful, however, not to confuse the infinite productivity of the generative procedure (a function in intension) with the cardinality of the decidable set of structures it generates (the well-formed formulas defined in extension)’ (Huybregts, Reference Huybregts2019, 3). Appreciating this caveat apparently allows for explanations of languages with finite output or no overt recursive constructions, without giving up on lip. This relates back to our discussion of standards of evidence in Section 3.2. Hence, the counterexample provided by Pullum and Scholz (Reference Pullum, Scholz and van der Hulst2010) is apparently neutralised since the grammar ‘may generate an infinite language but only produce a finite subset of it’ (Huybregts, Reference Huybregts2019, 3). If this claim holds, it is because linguistics is about structures not stringsets. Everaert et al. (Reference Everaert, Huybregts, Chomsky, Berwick and Bolhuis2015) make this position explicit. They argue that it is a mischaracterisation to associate linguistic theory (and natural languages) with the generation of stringsets. The computations of the mind can only ‘see’ hierarchically structured, recursive phrases. This is enough to establish an infinity without postulating actual sets of collections of sentences. The early distinction between weak, involving the generation of strings, and strong generative capacity, involving the additional association with structures (like trees), might be helpful here.Footnote ²⁹

Not only does Chomsky provide us with arguments as to the adequacy of different formal language families but in his landmark Chomsky (Reference Chomsky1959a), he takes one step further to define a nested hierarchy of computational complexity showing the relationships between these formalisms. At the bottom or inner-most ring are the regular languages (Type-3). They are easy to parse but not powerful enough to capture various patterns of natural language. Context-free languages do better but are more complex as a result (and contain the regular languages as a proper subset) (Type-2). Context-sensitive languages can capture patterns that are missing from CFGs (such as $a^n{b}^n{c}^n$) but do so at the cost of computational tractability (Type-1). Lastly, the class of languages that comprise all of the other families and constitute all the computably enumerable languages are Type-0 grammars. A useful way of understanding this level is by appreciating their associated accepting automata. In this case, the automata they map onto are unrestricted turing machines. Chomsky claims that unrestricted turing machines offer us nothing of interest in terms of linguistics. ‘We learn nothing about a natural language from the fact that its sentences can be effectively displayed, i.e. that they constitute a recursively enumerable set’ (Chomsky, Reference Chomsky1959a, 138). Importantly for him, for the study of formal languages to be relevant, it needs to not only describe stringsets (‘weak generative capacity’) but also provide structural information (‘strong generative capacity’). Type 0 grammars are unrestricted and thus do not perform that latter task. Chomsky thinks that the question of where natural language lies in the hierarchy of formal languages is a significant one.Footnote ³⁰ The reason he provides is interesting. He insists that an adequate theory must serve as a basis for an account of how speakers understand and use a particular grammar, that is, they must be empirically motivated. But this opens up the possibility that some speakers do not make use of languages with recursive structure at all. The possibility is chased up by Langendoen (Reference Langendoen and van der Hulst2010). He describes the situation in the following way.

This formalisation allows for some languages to be modelled as infinite and others not (based on the evidence of the respective linguistic communities). This kind of modelling makes linguistics somewhat unique. Highly formal concepts usually resigned to a priori mathematics receive empirical justification and real-world instantiation not just application (as in the natural sciences). The philosopher of mathematics, Resnik (Reference Resnik1982, 101), describes the situation well.

According to this view, linguists, like mathematicians, study patterns (Hellman, Reference Hellman1989; Nefdt, Reference Nefdt, Behme and Neef2018, Reference Nefdt2023a; Shapiro, Reference Shapiro1997). These patterns are formally characterisable. But their formal properties stand in a modelling relation to the world. Careful study of the actual requirements of individual language communities (or individual cognisers) will reveal the extent to which the formal properties are instantiated in the real-world languages under study. Of course, if corpora decide the empirical details, then infinite generalisation is unlikely to be found in the natural systems in the same way that phase transitions do not support the existence of infinite particles in a finite universe. However, if the abstract rules of competence ultimately decide the remit of the grammar, then some sort of potential infinity might be more realisable.

4.3 Lessons for the Philosophy of Science

There are a number of lessons to be drawn from the case study discussed in Section 4. Some are negative. Formal linguists have been accused of possibly conflating aspects or artefacts of their formal models with elements of the target system. If generative grammars are models, then they might indeed entail infinite output. But this does not hold immediately for the natural languages that they model (ignoring the technicalities of whether it holds in general). It might just be more convenient to talk as if natural languages were infinite (Appiah, Reference Appiah2017).

On the positive side, the linguistic infinity postulate does seem to account for the data on iterative structures in competence, not to mention the phenomenology of linguistic creativity. Perhaps human language is biologically unique and the philosophy of science (and biology) should pay due attention to the possibility of a genuinely infinite structure in the human brain (and the universe). In many ways, linguistic infinity or infinite generalisation provides a rich set of arguments and evidence for the existence of infinite sets or objects in the physical world, whether or not you agree that language is infinite itself. The situation can be well characterised in terms of the concept of neutral analogy in Hesse (Reference Hesse1963). For Hesse (Reference Hesse1963), neutral analogies (as opposed to positive or negative) involve cases in which it is unknown whether the model and the target share certain properties. Thus, they offer a route to possibly new hypotheses. A classic example is Dirac’s discovery of the positron, in which negative energy solutions were initially considered to be mere features of the mathematical model (or ‘artefacts’). However, after finding physical interpretations of these solutions, he was led to revise his entire theory and predict the existence of a novel particle (Bueno & Colyvan, Reference Bueno and Colyvan2011b).

The concept of linguistic infinity might not answer the puzzle concerning the effectiveness of set theory or mathematics in general within the sciences but it does provide fertile ground for further exploration of the topic of infinite generalisations and idealisations.

5.1 Why They Matter

It might argued that LLMs are learners of human language. However, this does not imply that they learn languages as we do (see Dupre, Reference Dupre2024; Lappin, Reference Lappin2024). That is a stronger claim which comprises the core of most of the current debates.

Why think that LLMs are relevant to understanding human language in the first place? One of the most common arguments in the literature is that LLMs offer us a testing bed for cognitive hypotheses, especially with relation to the issue of innateness. A. Clark and Lappin (Reference Clark, Lappin, Kempson, Fernando and Asher2012) claim that predessors of LLMs show us that only weak induction biases are necessary for language learning. Similar arguments have been made across the board. The idea that language is simply not learnable without strong (innate) structural biases is simply false. The standard counterarguments, however, point to the fact that LLMs are trained on significantly (in fact multiple orders of magnitude) more data than the average human child when acquiring their first language. To this, initiatives, such as BabyLM, which aim to train models on data more closely aligned to what we know about children’s available stimulus were spawned. But, of course, unlike children, LLMs are generally trained on flattened text (or tokens). In response to this restriction, Vong et al. (Reference Vong, Wang, Orhan and Lake2024) have attempted to develop a multi-modal training set involving innovative use of visual input (using head-mounted cameras on a young infant). The Zero speech challenge also aims to address this issue (www.zerospeech.com/).

The next reason for optimism comes from the success of LLMs on various linguistic tasks, even ones involving hidden structure such as wh-movement and other filler-gap dependencies (Linzen & Baroni, Reference Linzen and Baroni2021; Wilcox et al., Reference Wilcox, Levy, Morita, Futrell, Linzen, Chrupała and Alishahi2018). The fact that LLMs are able to pick up on subtle patterns in text only is quite remarkable. This is even possible in cases of relatively rare constructions such as English Preposing constructions in prepositional phrases or PiPPs such as ‘happy as we were’ or ‘brilliant though they seemed’, where the corpora do not exhibit many examples (see Potts, Reference Potts2024).

Hasson, Nastase, and Goldstein (Reference Hasson, Nastase and Goldstein2020) turn the issue of over-parameterised models like LLMs (or more broadly artificial neural networks, ANNs) around to reflect an alleged analogy with what cognitive neuroscientists have discovered about brain processing. They argue that modern ANNs embrace complexity in a manner distinct from traditional experimental designs with straightforward interpretability. Rather than viewing this as a disadvantage for cognitive science, they leverage Big Data and over-parameterisation to argue that these models operate in similar ways to naturally evolved human brain networks. Specifically, they claim that both brain networks and ANNs are ‘direct fit’ models in which interpolation or an interpolation space is key. This process requires ‘dense and broad sampling of the parameter space for reliable interpolation’ (428). They contrast this with the standard assumptions in cognitive psychology of poverty of stimulus and limited computational resources leading to extrapolation criteria and ‘ideal fit’ models. Direct fit optimisation in brains and ANNs, in contrast, do not extrapolate from external data, but interpolate from high-dimensional dense sampling. Interpolation involves local computations which usually produce weaker generalisations. However, given enough data, that is, large training sets or Big Data, interpolation can produce the same level of predictive power as extrapolation. Furthermore, and in line with our evolutionary discussion, ‘[s]imilar to natural selection, the family of models to which both ANNs and BNNs [brain neural networks] belong optimizes parameters according to objective functions to blindly fit the task-relevant structure of the world, without explicitly aiming to learn its underlying generative structure’ (Hasson, Nastase, & Goldstein, Reference Hasson, Nastase and Goldstein2020, 425).Footnote ³³ Chemla and Nefdt (Reference Chemla and Nefdt2024) marshall this natural selection metaphor to caution against the idea of a general learner, pervasive in the NLP community, and suggest that both sides of the impossible languages debate are missing a step.

One core aspect of human language processing which does seem to be different in LLMs is efficiency. As we have seen in Section 2.2, prominent examples include Zipf’s law, where frequency of words is efficiently associated with shorter strings, but generally dependency length minimisation (DLM) and minimum description length (MDL) are essential aspects of human linguistic cognition and analysis, respectively. In other words, in terms of the former, we tend to minimise the length between words to increase efficiency (Gibson et al., Reference Gibson, Futrell and Piantadosi2019). While MDL operates at the grammar or corpus level, such that optimality of analysis is defined in ‘virtue of providing both the most compact representation of the data and the most compact means of extracting that compression from the original data’ (Goldsmith, Reference Goldsmith2001, 2). Thus, information-theoretic concepts of compression and efficiency can explain many aspects of human language (Chater et al., Reference Chater, Clark, Goldsmith and Perfors2015; Gibson et al., Reference Gibson, Futrell and Piantadosi2019; Nefdt, Reference Nefdt2023a). If human language is indeed shaped by such efficiency considerations, then LLMs should follow suit. However, research has indicated the opposite tendency. Chaabouni et al. (Reference Chaabouni, Kharitonov, Dupoux and Baroni2019) ran experiments to show that LSTMs incorporate a preference for anti-efficient encoding rendering the most frequent expressions to the longest strings. Adding a cost function which penalises long messages moves the models towards efficiency contra its initial tendencies (see also Lian, Bisazza, & Verhoef, Reference Lian, Bisazza and Verhoef2021).

Lappin (Reference Lappin2024) provides a longer list of pros and cons for LLMs and linguistic theory. One of the cons that has received quite a bit of attention is metasemantic in nature. Bender and Koller (Reference Bender, Koller, Jurafsky, Chai, Schluter and Tetreault2020) argue that LLMs trained only on (textual) form cannot access semantic meaning, that is, are not semantically grounded. In addition, they lack the communicative intent required for understanding language. Their claims have been tested and challenged by using the resources of semantics and the philosophy of language. Mandelkern and Linzen (Reference Mandelkern and Linzen2024) argue that LLMs can access meaning indirectly via a linguistic division of labour argument from Putnam (Reference Putnam1981). The causal externalist metasemantics they offer is in line with contemporary philosophy of language but at odds with internalists views associated with syntax. Baggio and Murphy (Reference Baggio and Murphy2024) make this case forcefully against Mandelkern and Linzen (Reference Mandelkern and Linzen2024). Lappin (Reference Lappin2024) focuses more on multimodal LLMs which are largely immune to Bender and Koller’s Chinese room revival.

But again, models do not need to reflect every aspect of the target system nor do they have to be made of the same stuff or even operate via the same mechanisms. The bar is relatively low for their similarity in accordance with explanatory value, depending on our scientific aims.

5.2 What It Means

The take home message is that the philosophy of linguistics might forever be altered by the introduction of LLMs. Despite the terminology, it is not clear that LLMs are actual models in the scientific sense. But this issue certainly needs further investigation. Without the resources of the philosophy of science, such an endeavour is more liable to cause conflation and confusion.

I think one of the clearest and most useful interpretations of the role LLMs can play in linguistic theory is that they are model organisms (Ankeny & Leonelli, Reference Ankeny and Leonelli2011, Reference Ankeny and Leonelli2020). This perspective is endorsed in Tuckute, Kanwisher, and Fedorenko (Reference Tuckute, Kanwisher and Fedorenko2024) and suggested elsewhere as well. It should even satisfy biolinguists who favour the strong interpretation of their field. The idea is that only limited comparative work was possible with our primate cousins and animals further up (or down) the phylogenetic tree. Their capacities for anything resembling the complex structures of natural language are impoverished at best. For the first time in our history, another system seems able to reproduce (or mimic) of linguistic behaviour in much of its complexity. Probing and prodding this system might not yield direct homologies with the human language system but it could tell us something interesting and useful about linguistic representation itself. We have learned much from comparative work in vision. Language until now has be more refractory given its relative uniqueness within the biological world. The analogy goes potentially deeper. Different LLMs incorporate different architectures and process language is distinct ways. Recurrent networks include a memory element and were generally the standard models for NLP before transformers with their attention heads and positional encoding proved more robust on most linguistic tasks (Vaswani et al., Reference Vaswani, Shazeer and Parmar2023). Studying how different models operate, instead of just moving on to newer ones based on their successes on performance benchmarks, would enrich this avenue for linguistic enlightenment further and push the model organisms perspective to its limits.

One might object that model organisms only provide useful information in so far as they are biological, organic creatures linked to us by genetics, natural selection, and evolution. Searle (Reference Searle1980) seemed to suggest that machines are just not made of the right kinds of stuff to be considered conscious or capable of understanding. Oddly, this biochemical chauvinism is a difficult position to maintain in principle, especially for those who prescribe to a genuinely computationalist perspective on mind and language. The LLMs are nothing if not computational, albeit in the non-classical, connectionist sense of the term. It is still true that LLMs are certainly not occupants of the phylogenetic tree of life that humans and other non-human animals both find their respective branches. Nevertheless, as mentioned in Section 5.1, Hasson, Nastase, and Goldstein (Reference Hasson, Nastase and Goldstein2020) and Chemla and Nefdt (Reference Chemla and Nefdt2024) do find analogies with natural selection. In fact, Yax, Oudeyer, and Palminteri (Reference Yax, Oudeyer and Palminteri2024) push this metaphor to its extreme, as they propose a framework for constructing phylogenetic trees of LLMs (primarily in terms of ‘traits’, that is their behaviour as to what tokens they generate and how they fare with respect to various benchmarks, rather than direct structural or historical considerations). It might be useful to note that many model organisms are often carefully bred to elicit the comparative properties we want to model, resulting in significant departures from their wild counterparts.

Despite this possible positive, one can sympathise with those who emphasise the successes of LLMs, where traditional symbolic models have failed. If natural language is indeed based on discrete rule systems somehow embedded in our neural pathways, then achievements of LLMs which do not rely on these kinds of mathematical properties are even more puzzling. But, of course, statistics and Big Data can go a long way towards approximating intelligence without ever reaching it. The issues are thorny. Is this a case of the detachment of prediction from explanation in the philosophy of science (discussed in Section 3.2)? Or are LLMs telling us something important about ourselves? Here, the honed work of philosophers of linguistics, steeped in the philosophy of science, can help clarify confusion and chart new paths forward. Or at least, one hopes it can.