This chapter highlights a few problems of serial order that may pose problems to a biologically realistic model, problems best explained in the context of theories designed to solve them. Therefore, the following paragraphs sketch examples of linguistic descriptions of syntactic regularities. In addition, technical terms are introduced. Occasionally, a sketch of what a neuron-based approach to syntax might look like may intrude but is developed systematically only in Chapters 9–12.

There are many different approaches to syntaxin theoretical linguistics, and it is not necessary to give an overview of them in the present context. This chapter highlights examples, their choice being primarily motivated by the historical development. So-called phrase structure grammars and their offspring rooted in the work of Harris (1951, 1952) and Chomsky (1957) are in the focus, because the superiorityof phrase structure grammars to a model of serial order in the McCulloch–Pitts (McCulloch & Pitts, 1943) tradition was one of the main reasons in the 1940s and later to base syntax theories on these more abstract algorithms rather than on neuronal algorithms. Apartfrom approaches related to and building on phrase structure grammars, an alternative framework whose roots also date back to the 1940s is mentioned occasionally. This framework, or family of theories, goes back to Tesnière and is called dependency grammar. Clearly, phrase structure grammars anddependency grammars have been modified and much developed, but, as isargued later in this chapter, some of their underlying main ideas persist.

The neuroscience of language is a multidisciplinary field. The reader's primary interest may therefore lie in various classical disciplines, including psychology, neuroscience, neurology, linguistics, computational modeling, or even philosophy. Because readers with different backgrounds may be interested in different parts of this book, Chapter 1, Section 1.3 gives an overview of the book contents and the gist of each chapter. In Section 1.1, the general structure of the book is explained; subsequently, paths through the book are recommended for readers with different backgrounds and interests in Section 1.2.

Structure and Function of the Book

The fourteen chapters of this book are mainly designed to convey one single message: It is a good idea to think about language in terms of brain mechanisms – to spell out language in the language of neurons, so to speak. Making this point is not a new proposal. One can find similar statements in classical writings; for example, in Freud's monograph on aphasia (Freud, 1891) and other publications by neurologists in the late nineteenth century, and, of course, in modern brain-theoretical and linguistic publications (Braitenberg, 1980; Mesulam, 1990; Schnelle, 1996a). However, a systematic model of language at the level of neurons as to date is not available, at least, not an approach that would be both grounded in empirical research while at the same time attacking a wide range of complex linguistic phenomena.

Models are only as good as the tools available for testing them. Is there any perspective on testing the proposals about syntactic brain processes discussed to this point?

Making Predictions

One obvious prediction of the neuronal grammar framework is that the cortical processes following a sentence in the input are of different types. Such process types include the following:

• Ignitions of word webs

• Ignitions of sequence sets

• Downward or upward regulation of activity by threshold control

• Reverberation and priming of neuronal sets

It may be possible to investigate these putative processes using neurophysiological recording techniques. Because the relevant activation and deactivation processes likely occur within tenths or hundredths of a second, it is advisable to use fast neuroimaging techniques in the investigation of serial-order information processing.

At first glance, one straightforward way of transforming any of the mentioned simulations into predictions about neurophysiological processes appears to be by calculating sum activity values at any time step of a simulation, and by thereby obtaining a series of activity values.

However, this strategy is not so straightforward as it appears at first glance because several parameters must be chosen beforehand. The relative weighting of the activity states is one set of parameters that appears to distinguish between pronounced activity changes – those induced by ignitions and regulation processes – and smaller ones – as, for example, those caused by the spreading of priming or reverberation. As an additional set of parameters, the time constants of individual processes must be specified.

This chapter complements Chapter 3 in providing neuroimaging and neuropsychological data about language. Here, the focus is on words. It is asked which brain areas become active during, and are relevant for, the processing of words in general, and that of specific word categories in particular.

An aim of this chapter is to show that the neuroscientific principles discussed in Chapters 2 and 3 give rise to new ideas about the representation and processing of words in the brain. The cortex, a neuroanatomically defined associative memory obeying the correlation learning principle, allows for the formation of distributed functional webs. During language acquisition, the neurobiological principles governing the cortex interact to yield the neuron machinery underlying language. Distributed functionally coupled neuronal assemblies, functional webs, are proposed to represent meaningful language units. These distributed but functionally coupled neuronal units are proposed to exhibit different topographies. Their cortical distribution is proposed to relate to word properties. It is asked how this idea fits into evidence collected with modern neuroimaging techniques.

Word-Form Webs

Early babbling and word production are likely caused by neuron activity in cortical areas in the inferior frontal lobe, including the inferior motor cortex and adjacent prefrontal areas. The articulations cause sounds, which activate neurons in the auditory system, including areas in the superior temporal lobe. The fiber bundles between the inferior frontal and superior temporal areas provide the substrate for associative learning between neurons controlling specific speech motor programs and neurons in the auditory cortical system stimulated by the self-produced language sounds.

The cortex may be an associative memory in which correlation learning establishes discrete distributed functional webs. It is important to ask how this view relates to clinical observations, in particular to the neuropsychological double dissociations seen in aphasic patients. Here is an example of such a double dissociation. Patient A exhibits severe deficits in producing oral language (Task 1), but much less difficulty in understanding oral language (Task 2). Patient B, however, presents with the opposite pattern of deficits – that is, only relatively mild language production deficits, but substantial difficulty in comprehending spoken language. Briefly, Patient A is more impaired on Task 1 than Task 2, whereas Patient B is more impaired on Task 2 than Task 1.

Clearly, the explanation of neuropsychological syndromes and, in particular, of double dissociations is an important issue for any model of brain function and, therefore, a brief excursus may be appropriate here. It was once argued that the existence of a double dissociation demonstrates, or strongly suggests, the presence of modules differentially contributing to specific aspects of the tasks involved (Shallice, 1988). A module would be conceptualized as a largely autonomous information processor (see also Section 6.2). A standard explanation of a double dissociation, therefore, is that two modules are differentially involved in the two tasks (1 and 2), and that one of them is selectively damaged in each of the patients (A and B).

What does the investigation of artificial models of networks of neurons contribute to the investigation of brain function in general, and of language mechanisms in particular? There are at least two possible answers to this question.

One answer is that an artificial neural modelcan be used to prove that a circuit of a certain type can solve a given problem. When thinking about complex interactions of functional neuronal units, one is in danger of losing track of what a network actually can, and cannot, do. Here, a simulation may help by providing anexistence proof that a desired result can be obtained using a given circuitry or circuit structure. A successful simulation never proves that the network it is based on is actually realized in the nervous system; it shows only that the network type the simulation is based on is one candidate. To call it a realistic simulation, other criteria must be met: in particular, that the network structurecan be likened to brain structure and that the functional principles governing the neurons' behavior and their actual behavior haveanalogs in reality as well.

A second possible answer to the question about the significance of neuron models is that they can serve as illustrations of one's ideas about brain function. In the same way as a detailed verbal description or a description in terms of algorithms, a neuron circuit can help to make an idea more plastic.

How is language organized in the human brain? This book provides results of brain activation studies, facts from patients with brain lesions, and hints from computer simulations of neural networks that help answer this question. Great effort was spent to spell out the putative neurobiological basis of words and sentences in terms of nerve cells, or neurons. The neuronal mechanisms – that is, the nerve cell wiring of language in the brain – are actually in the focus. This means that facts about the activation of cortical areas, about the linguistic deficits following brain disease, and the outcome of neural network simulations will always be related to neuronal circuits that could explain them, or, at least, could be their concrete organic counterpart in the brain.

In cognitive neuroscience, the following questions are commonly asked with regard to various higher brain functions, or cognitive processes, including language processes:

1. Where? In which areas of the brain is a particular process located?

2. When? Before and after which other processes does the particular process occur?

3. How? By which neuron circuit or which neuron network type is the particular process realized?

4. Why? On the basis of which biological or other principles is the particular process realized by this particular network, at this particular point in time, and at these particular brain loci?

The ultimate answer to the question of language and the brain implies answers to these questions, with respect to all aspects of language processing.

Chapter 4 offers a neurobiological perspective on word processing in the brain. The time course and topography of cortical activation during word processing, in particular during the processing of words of different categories, is discussed in some detail. The proposal is that there is a cell ensemble or functional web for each and every word, and that words with different referential meaning may have functional webs characterized by different topographies.

Looking at words more closely, more questions arise, for example, regarding complex form–meaning relationship. Two words may share their form (e.g., “plane,” meaning “aircraft” or “flat surface”), or may sound differently but have largely the same meaning (e.g., “car” and “automobile”). There are word forms that include other word forms (e.g., the letter sequence “nor,” including “no” and “or”), and there are words whose meaning includes, so to speak, the meaning of others (e.g., “animal” and “dog”). These relationships of homophony (or polysemy), homonymy (or synonymy), inclusion of word forms in other word forms, and hyperonymy (vs. hyponymy) may pose problems to an approach relying on cortical neuron webs. How might a neurobiologically realistic model of complex form-meaning relationships look? Tentative answers are discussed in the section on overlap of representations.

Other aspects largely ignored in earlier chapters relate to the information about written language and to aspects of meaning that have been characterized as emotional or affective.

Large, strongly connected groups of neurons were proposed to form the neurobiological substrate of higher cognitive processes in general and language in particular. If the reader wishes, the ultimate answer to the question of language, the brain, and everything was suggested to be neuronal ensemble. Different authors define terms such as neuron ensemble, cell assembly, and neuronal group in different ways, and therefore a new term, functional web, was proposed and its meaning clarified (see Chapters 2, 5, and 8). There is support for the concept of functional webs from neurophysiological and neuroimaging experiments on language and memory (see Chapters 2 and 4). In this chapter, the notion of a functional web is used as a starting point for a serial-order model. The elements of this model are called neuronal sets. Neuronal sets are functional webs with additional special properties that are relevant for serial-order processing. Neuronal sets can represent sequences of words and are then called sequence sets (or alternatively, sequencing units, or sequence detectors). New terms are introduced to distinguish the entity that has empirical support (functional web) from the theoretical concept developed (neuronal set).

In this chapter, the notions neuronal set and sequence set are explained and applied to introduce a putative basic mechanism of grammar in the brain. Properties of functional webs are first briefly summarized, and then the concept of a neuronal set is defined as a special type of functional web.

In this chapter, an introduction to facts known from neurological, neuropsychological, and neuroimaging research on language is given. How the obtained results can be explained and how they fit together is discussed.

Aphasiology

The scientific study of language proliferated in the second half of the nineteenth century. Apart from linguists and psychologists, neurologists focused on language in the context of the then new findings about “affections of speech from disease of the brain” (Jackson, 1878). In adults who had been fully able to speak and understand their native language, a stroke, tumor, trauma, or encephalitis was sometimes found to severely and specifically reduce their language abilities. Such language disturbances were called aphasias. There was, and still is some discussion as to whether there are subtypes of aphasia, and a good deal of the research on aphasia was dedicated to developing new classification schemes and arguing why one scheme should be a better reflection of what appeared to be the truth than another. The research effort resulted in numerous classification schemes (Caplan, 1987) and also in what appeared to be an extreme position expressed by the claim that there is only one type of aphasia (Marie, 1906).

This last view can be based on the common features of all – or at least the large majority of – aphasias. All aphasics have difficulty speaking, although their ability to move their articulators – lips, tongue, pharynx, larynx, and other muscles in the mouth-nose region – may be well preserved.

This chapter takes the concept of a neuronal grammar developed earlier as a starting point. The earlier proposal is partly revised and extended to cope with problems put forth by natural language phenomena.

The three phenomena considered here are as follows:

• The distinction between a constituent's obligatory complements and its optional adjuncts.

• The multiple occurrence of the same word form in a string.

• The embedding of sentences into other sentences.

All three issues have been addressed occasionally before; however, they were not treated systematically in the context of neuronal grammar. This requires its own discussion because the necessary extensions lead to a substantial revision of the previous concept of neuronal grammar.

The gist of the revision briefly in advance is as follows: In earlier chapters, the relationship between sequence detectors and words in the input was assumed to be static. The word web ignites and then, successively, one set of its sequence detectors is recruited according to relationships the word exhibits to words in its context, as they are manifest in regular co-occurrence of lexical category members. Each word would recruit one of its connected lexical category representations. However, if each word form were attributed to one lexical category, it would be impossible to model the situation in which one word occurs twice in different grammatical roles in a sentence.

Linguistics is the study of language. Language is a system of brain circuits. To qualify the latter statement, one may cite the father of modern linguistics, Ferdinand de Saussure, who claimed that language (i.e., the language system, or langue) is a “concrete natural object seated in the brain” (de Saussure, 1916). If linguistics is the study of language and language is in one sense a system of brain circuits, one would expect linguists to be open to the study of brain circuits. However, the dominating view in linguistics appears to be that language theories must be formulated in an abstract manner, not in terms of neuron circuits. Section 14.1 asks why linguists favor abstract rather than neuron-based formulations of language mechanisms. Section 14.2 discusses a few thoughts about how an abstract theory of language may profit from a brain basis.

Why Are Linguistic Theories Abstract?

As mentioned, the dominating view in linguistics is that language theories must be formulated in an abstract way. This could be a trivial claim because it is clear that every scientific theory must include abstract concepts. However, this is not the point. The point is that abstract in this context excludes explicit reference to the organic basis of the processes described in an abstract fashion. Linguistic theory is abstract in the sense that it does not refer to neurons. Why is this so?

This excursus presents and discusses a very complex sentence, which is actually a word chain whose status as a grammatical sentence may be questioned. The interested reader may nevertheless find it relevant to glance at the processes, because of the prominent role sentences of this type played in the history of language science. The present circuit proves that a string with center embeddings can be processed by neuronal grammar. There is nothing in the neuronal algorithm that would restrict the number of embeddings possible. Clearly, however, such restrictions apply for biological systems.

Consider sentence (1).

1. (1) Betty who John who Peter helps loves gets up.

Figure E5.1 presents the network representations of the elements of this sentence, word webs and sequence sets, and their mutual connections, whereas Table E5.1 shows the derivation. It becomes obvious from the table that this derivation draws heavily on multiple activation of sequence sets. As in Excursus E4, the table lists multiple activity states of each neuronal set at each time step, with the highest activity state listed at the top. Note that, according to the present proposal, each neuronal set can be considered a store in which multiple entries can be placed on each other. A similar proposal was made by Schnelle (1996b). The little “stacks” of symbols represent the entries in the pushdown memory characterizing each set. The maximum “height” of a stack is three levels.

How does neuronal grammar operate? The following examples further illustrate the activation processes taking place in a network of neuronal sets during perception of congruent or grammatically well-formed and incongruent or ill-formed strings. This excursus aims to illustrate the principled difference in network dynamics between the processing of congruent and incongruent word strings, and further aims to introduce illustration schemes for network dynamics that are used in later sections of the book (see E3–E5 Chapters 11, 13).

Although the general mechanism of serial-order detection, mediated sequence detection by sequence sets, is simple, the interaction of several neuronal sets can become quite complex. To make their activity dynamics easy to overlook, two strategies are used to illustrate processes in grammar networks. One strategy is to list activity states of all sets contributing to the processing of a string at each point in time when a string element is present in the input and shortly thereafter. Activity dynamics are therefore presented in the form of tables. A second strategy is to present the simulations as animations. The animations, including illustrations of the three examples presented in this excursus, are available on the Internet at this book's accompanying web-page (http://www.cambridge.org).

Examples, Algorithms, and Networks

Strings such as (1), (2), or (3) could be taken as examples for illustrating the function of a simple grammar network.

The synfire model discussed in Chapter 8 is one example of a realistic model of serial order in the brain. One may call it realistic because it has strong footings in neuroscientific research. Which alternative mechanisms for establishing serial order exist in the nervous system? This chapter reviews a class of serial-order mechanisms different from the synfire chain. It is argued that this type of mechanism may be important for organizing grammar in the brain, and an attempt is undertaken to apply the mechanism to the processing of a simple sentence.

Movement Detection

As emphasized in Chapter 8, the synfire model realizes a sequence of elementary events “A then B” by direct connections between their neuronal representations, α and α. As an alternative, it is possible to connect a third element to both representations of elementary events. The third element, γ, would become active if sequence AB occurs. The third element would be a mediator serving the sequence detection process, which could otherwise be performed by a synfire mechanism as well.

The basic idea for this mechanism of mediated sequence processing has been formulated by McCulloch and Pitts (Kleene, 1956; McCulloch & Pitts, 1943). In Chapter 6, the cardinal cells responding specifically to strings of events were called string detectors. In modern neuroscientific research, several lines of research have proved similar mechanisms in the nervous system of animals.

Many animals respond specifically to stimuli that move. Therefore, they must be equipped with a mechanism for movement detection.

This excursus illustrates circuits motivated by the proposals discussed in Chapter 12. The abbreviations used here and the representation of activity dynamics in table form are the same as those used in Excursuses E2 and E3.

With the extensions of the grammar circuits proposed in Chapter 12, it now becomes possible to treat sentences in which the same word occurs twice and as member of different lexical categories, such as sentence (1).

1. (1) Betty switches the switch on.

In Chapter 10, Figures 10.3 and 10.4 are used to sketch a putative neuronal correlate of the syntactic category representations that may be connected to the representation of the word form switch. Two of the lexical categories, transitive particle verb, here abbreviated as V, and accusative noun, here abbreviated as N, are relevant for the processing of the syntactically ambiguous word used in sentence (1). The homophonous words and their lexical categories are characterized by the assignment formulas (2) and (3) and the valence formulas (4) and (5).

1. (2) switch ↔ V (transitive particle verb)

2. (3) switch ↔ N (accusative noun)

3. (4) V (p /*/ f1, f2, f3)

4. (5) N (p1, p2 /*/)

Figure E4.1 shows the entire network used for sentence processing. The representation of the ambiguous word form is doubled for ease of illustration. This figure is almost identical to Figure E3.1, which dealt with a similar sentence. Table E4.1 presents activity dynamics of the sets involved in processing the ambiguous word and its two lexical categories.