2. Two kinds of neuroplasticity

The most familiar kind of neuroplasticity is Hebbian learning, also known as spike-timing dependent plasticity (Song et al. Reference Song, Miller and Abbott2000). Hebbian learning is a crucial developmental process for tuning local neural interactions and helping determine the functional bias of local networks. Neural reuse, whereby individual neural elements are put to use for multiple cognitive and behavioral ends, involves an additional kind of neuroplasticity that I have called neural search. Neural search is a process that places neural elements into new functional partnerships with one another. During the course of learning and development, each element will come to be a member of multiple functional coalitions.

The first two chapters of After Phrenology are dedicated to marshaling the evidence for neural reuse in general and neural search in particular. I will just gesture at a few key pieces of evidence, here. If individual regions of the brain are in fact used and reused in multiple circumstances (as posited by neural reuse), then they should be functionally diverse, active in support of multiple tasks from different task domains. If variety of function is a matter of putting the same neural elements into different functional coalitions (supported by neural search), then we should see different patterns of functional cooperation across the brain under different psychological circumstances.

In one recent study (Anderson et al. Reference Anderson, Kinnison and Pessoa2013), my coauthors and I borrowed a technique from ecology and measured functional diversity in various regions of the brain in much the same way as one measures ecological diversity. We asked: How many individual tasks (i.e., animals) in how many different task categories (i.e., species) does each region of the brain support? Using Shannon entropy (Shannon Reference Shannon1948) as our metric, and a large collection of more than 2,000 functional neuroimaging experiments, we measured functional diversity voxel-by-voxel using a spherical searchlight of various sizes. The unequivocality of the results surprised even us: Individual regions of the brain, even small regions, are highly diverse. Figure 1 reproduces the histogram of results from one run, using a 10 mm spherical searchlight and 11 task categories, and scaling the diversity metric from zero to one, such that zero diversity indicates that every observed activation is in a single category, and one indicates equal numbers of activations from each task category.

Figure 1.

Histogram of whole-brain, voxel-wise functional diversity measurements.

As can be easily observed from the leftward skew of the data, there are very few specialists in the brain, supporting only tasks from a single task category such as semantics or visual perception. Most regions of the brain are active during multiple tasks in different task categories. Regions nevertheless achieve their functional diversity in different ways. Figure 2 illustrates the functional fingerprints of three different voxels from the same data run described above: a voxel from left auditory cortex, a relative specialist with a diversity value of 0.41; a voxel from left anterior insula with a high diversity value of 0.88; and a voxel from left thalamus with a diversity value of 0.76, equal to the population median. Functional fingerprints display the relative degree of activity observed in each task category for the region pictured. Hence, the portion of auditory cortex shown is most frequently active during auditory tasks, and only occasionally in somesthesis, action, and language tasks. By contrast, the pictured region of anterior insula is active at least some of the time during tasks in every category, albeit not uniformly. Functional fingerprints represent the likelihood that an active region is active during, or being activated by, a given type of task or stimulus, and thus offer a way to capture the different functional biases or underlying causal dispositions of individual regions.

Figure 2.

Functional fingerprints representing the relative amount of activity across 11 task categories for three voxels from left thalamus, left anterior insula, and left auditory cortex (counterclockwise from top right).

This brings us to the question of whether we can observe regions of the brain cooperating with different partners under different circumstances. To illuminate this question, the technique of choice is a functional connectivity analysis. Using the same collection of neuroimaging experiments, we searched for deviations from statistical independence in the activity of individual regions. That is, we looked to see whether regions are more likely to be active during the same experimental task than would be predicted by chance. The results of such analyses can be represented as a graph, where the nodes of the graph represent regions of the brain, and edges between the nodes indicate that the connected regions are statistically likely to be co-active, and are therefore functionally connected. As was reported in a number of recent studies (Anderson Reference Anderson2008a; Reference Anderson2010; Anderson & Penner-Wilger Reference Anderson and Penner-Wilger2013), it does indeed appear to be the case that regions of the brain – variously defined in the different analyses – have different functional partners during different functional circumstances. By way of illustration, Figure 3 depicts the functional connectivity graphs observed during emotion, attention, and semantics tasks. The functional partners of left precentral gyrus are highlighted. As can be easily seen (and can be confirmed quantitatively), individual regions of the brain are active in multiple task circumstances, but have different functional partners in each.

Figure 3.

Functional connectivity graphs during semantics, emotion, and attention tasks. The functional partners of left precentral gyrus are highlighted. Nodes are placed in a projected three-dimensional space at the approximate center of each brain region from the Harvard-Oxford atlas; the figure shows the brain from above, front toward the top of the page.

These are just two pieces of suggestive evidence for neural reuse, both rooted in the neuroimaging literature, which is of course limited in various ways. After Phrenology also surveys electrophysiological studies of single neurons that highlight the importance and prevalence of mixed selectivity (Cisek Reference Cisek2007; Cisek & Kalaska Reference Cisek and Kalaska2005; Reference Cisek and Kalaska2010; Rigotti et al. Reference Rigotti, Barak, Warden, Wang, Daw, Miller and Fusi2013); cognitive interference and neural attenuation studies that demonstrate the activation of individual cells by multiple different tasks and stimuli (Glenberg & Kaschak Reference Glenberg and Kaschak2002; Glenberg et al. Reference Glenberg, Sato and Cattaneo2008; Roux et al. Reference Roux, Boetto, Sacko, Chollet and Trémoulet2003; Rusconi et al. Reference Rusconi, Walsh and Butterworth2005); work with sensory substitution devices that suggests that many regions of the brain are (and remain throughout life) capable of receiving and processing inputs from multiple sensory modalities (Merabet et al. Reference Merabet, Hamilton, Schlaug, Swisher, Kiriakapoulos, Pitskel, Kauffman and Pascual-Leone2008); and work demonstrating the importance and ubiquity of neuromodulation at multiple spatial scales (Bargmann Reference Bargmann2012; Hermans et al. Reference Hermans, van Marle, Ossewaarde, Henckens, Qin, van Kesteren, Schoots, Cousijn, Rijpkema, Oostenveld and Fernández2011). Overall, the evidence is far more consistent with neural reuse than with competing, modular accounts of brain organization.

The developmental framework advocated in After Phrenology is an extension of the Interactive Specialization framework (Johnson Reference Johnson2001; Reference Johnson2011). As with interactive specialization, and unlike the maturational viewpoint championed by Kanwisher (Reference Kanwisher2010) and others (e.g., Atkinson Reference Atkinson1984), neural reuse emphasizes the importance of experience in shaping the functional biases of local neural elements. It will only rarely, if ever, be the case that the functional properties of a region of the brain are shaped primarily by genetic factors. Similarly, neural reuse emphasizes that the functional properties of local regions both partly determine and are partly determined by the regions with which they interact. The multiple functional coalitions that are set up during development and learning depend on the functional biases of their constituent regions, but these coalitions also help shape those functional biases as the behaviors the coalitions support are refined.

Neural reuse departs from interactive specialization by emphasizing the participation of neural elements in multiple coalitions. Consequently, it also departs from interactive specialization on the issue of whether and to what degree we should expect neural elements to be functionally specialized. That there is functional differentiation across the brain is abundantly clear (and illustrated in Fig. 2). But there is apparently not functional specialization. Hence, the developmental framework advocated in After Phrenology is called interactive differentiation and search.

Although the evidence surveyed in After Phrenology does not appear to be consistent with the idea of functional specialization, it might nevertheless be the case that there exists some alternate taxonomy of function and level of description in terms of which brain regions could be assigned specific, dedicated functions (Price & Friston Reference Price and Friston2005). In my own view, the apparent ubiquity of neuromodulation, and the prevalence of mixed selectivity in individual neurons, will make true functional specialization rare. But it is certainly an open question, one that is treated at length in After Phrenology.

3. Neural reuse, evolution, and modularity

As I hope is clear even from the brief discussion above, neural reuse is not consistent with the notion that the brain is composed largely of segregated, functionally dedicated, specialized neural modules. Different networks share parts, and the parts may do different things for each of the networks in which they participate, as a result of the constraints imposed by the network interactions (Anderson Reference Anderson, Metzinger and Windt2015). The brain is functionally differentiated but also deeply integrated in ways that make modularity very unlikely. Yet, the modularity assumption remains pervasive, despite the mounting evidence for reuse in the cognitive neurosciences, and the scant evidence for mosaic evolution in evolutionary biology (Aboitiz Reference Aboitiz1996; Finlay & Darlington Reference Finlay and Darlington1995; Finlay et al. Reference Finlay, Darlington and Nicastro2001; Stephan et al. Reference Stephan, Baron, Frahm, Steklis and Erwin1988; Yopak et al. Reference Yopak, Lisney, Darlington, Collin, Montgomery and Finlay2010). What accounts for this tenacity? In short: modularity appears to offer an answer to the paired questions of how behavior is heritable and how brains are evolvable. To break the hold of modularity, then, requires offering better answers to these questions.

Although a critique of evolutionary psychology is not central to After Phrenology, a few words about that approach to understanding the psychological and neural legacy of our evolutionary history will highlight some of the reasons modularity can seem attractive, and throw into relief the alternative account I am offering. Evolutionary psychology (Buss Reference Buss2005; Confer et al. Reference Confer, Easton, Fleischman, Goetz, Lewis, Perilloux and Buss2010) rests on two problematic assumptions. It assumes, first, that the environment of selection is different from the current environment and can be adequately described and, second, that the solutions to the adaptive challenges posed by that environment are individually genetically encoded. The first assumption is problematic not just because of the inherent uncertainty in identifying and accurately describing ancient environments, but also because of what might be called the evidentiary dilemma for evolutionary psychology. Insofar as the environment of selection is very different from our own environment, evidence for the persistence of psychological mechanisms optimized for that environment is always simultaneously evidence for (an) adaptation, but against adaptivity (because the mechanism is tuned to the “wrong” environment). Likewise, insofar as the environment is relevantly similar to our own, then the identification of psychological mechanisms appropriate to that environment is simply evidence for adaptivity, and not for an adaptation.

Hence, as it is currently conceived, evolutionary psychology is hard-pressed to do justice to both adaptation and adaptivity, and a fully adequate evolutionary science of the mind must of course do both. This issue is related to the second assumption driving evolutionary psychology: because it assumes that solutions to environmental challenges must be encoded genetically (and result in dedicated neural modules), it is forced to conclude that the timescale of change will be quite long. I see little evidence for this latter assumption, but there is one important consideration that, at least on its face, seems to favor it. The idea is this: If psychological processes and the neural structures that support them are to be viewed as heritable adaptations, then they must be separately modifiable, for otherwise there is no available target for selection pressures. If this is correct, a nearly decomposable, modular brain consisting of separately modifiable subsystems appears to be required by evolution.

The mistake that evolutionary psychology makes here is subtle and twofold. The first mistake is to forget that not just genes but also environments are generally inherited, and the second is to suppose that a cognitive process is separately modifiable if, and only if, its supporting components are separately modifiable. In fact, the key to understanding how organisms inherit species-typical behaviors is seeing how genetics, environment, and developmental processes all work together (Anderson & Finlay Reference Anderson and Finlay2014). According to the interactive differentiation and search framework developed in After Phrenology, learning is a matter of finding and consolidating the right neural partnerships to support the acquisition of the target behaviors, where the “right” partners are those with the particular functional biases that together serve the behavioral ends. The functional biases are in turn shaped by learning and experience, all of the way back to and including very early experience. It is here that genetic and environmental factors have their most important initial impact. If we assume highly stereotyped projections from sensory afferents to specific regions of the developing brain, and an environment largely conserved between generations, then early experience will be sufficiently similar between individuals to induce neural structures with conserved, species-typical functional biases.

Given a similar stock of functional elements, and a species-typical developmental trajectory for skill acquisition, the processes of neural reuse – of the discovery and consolidation of functional coalitions – will tend to produce similar networks, and hence similar, species-typical solutions to the challenges posed by the largely conserved environment. On this model, selection pressures would tend to target not specific cognitive processes, but rather developmental mechanisms for ensuring the robust availability of neural elements with a wide range of functional biases. Note that this model also accounts for the possibility of psychological adaptations, and for the persistence of rapid adaptability to changing environments. One can inherit a psychological adaptation in virtue of inheriting the environmental challenge along with the neural elements that can be put together to meet it; and one can adjust a cognitive process to a new situation by changing the mix of elements in the neural coalition that implements it.

4. Networks of the brain

The brain is a network. So far, this is to say very little, for who would deny it? What is different about the neural reuse framework is not that it insists the brain is a network, but rather that it supposes the brain is a network with some very important architectural and functional properties. These include multiscale dynamics, multidirectional feedback, noncomponentiality, and action-orientation. I will treat each of these properties in turn.

The brain is a dynamic network that remodels itself at multiple spatial and temporal scales. In addition to the two types of neuroplasticity detailed earlier (sect. 2) that cooperate to remodel the synaptic (or “wired”) network, there are modulatory processes that change the effective connectivity of the synaptic network. Mechanisms include genetic expression that serves to activate and inactivate individual synapses, thereby changing the functional properties of local networks (Bargmann Reference Bargmann2012); dendritic spine motility (Holtmaat & Svoboda Reference Holtmaat and Svoboda2009) that can make synaptic connections more or less reliable; extra-synaptic diffusion neurotransmission involving the release from non-synaptic sites of neurotransmitters that diffuse through the extracellular matrix and change the firing likelihood of the neurons to which the transmitters bind (Agnati et al. Reference Agnati, Guidolin, Guescini, Genedani and Fuxe2010); and various hormonal mechanisms and systems that modulate brain activity at long temporal and broad spatial scales (Bauer et al. Reference Bauer, Heinz and Whybrow2001; Pfaff Reference Pfaff2002). Hence, function in the brain depends upon, at least: a neural network, an underlying genetic network, and an overlaid chemical gradient. Each of these elements is only partially understood, and their dynamic interactions even less so.

At any given moment in a quiescent network, the current effective connectivity would dictate the evolution of any induced pattern of activity. But the brain is of course never quiescent. It is always active to some degree, whether as a result of the purposeful activity of the agent or the endogenous activity of the brain at “rest” (Raichle et al. Reference Raichle, MacLeod, Snyder, Powers, Gusnard and Shulman2001). The effect of externally induced (e.g., perceptual) neural activation will depend not just on the effective connectivity of the network, but also on the ongoing activity resulting from past patterns. The brain is decidedly not a primarily feed-forward system. Instead, interactions between feed-forward, feed-back, bottom-up, and top-down processes both determine how the activation patterns evolve and also induce further changes in the effective connectivity of the network (Cole et al. Reference Cole, Reynolds, Power, Repovs, Anticevic and Braver2013). Moreover, in driving the ongoing behavior of the organism, these evolving patterns influence the nature of the externally induced activations; organisms are perception seeking, not passive recipients of environmental stimulation.

In a brain marked by such multidirectional feedback, understanding the interactions between parts becomes a significant challenge. Indeed, even defining the functional parts becomes difficult, as the relevant functional parts will themselves apparently change over time. For these reasons, we must move beyond componential computational models of the brain. Different neural patterns indexing different perceptual states, action choices, preferences, reward estimations, other predictions, and so forth, do not combine syntactically in the manner of compositional linguistic structures. Neither are the functional parts of the brain always best understood as components with stable, intrinsic input-output mappings and well-defined interfaces supporting the exchange of content-carrying symbols. Instead, patterns superpose in the brain and interact through the process of biased pattern competition (Cole et al. Reference Cole, Reynolds, Power, Repovs, Anticevic and Braver2013; Desimone & Duncan Reference Desimone and Duncan1995; Miller & Cohen Reference Miller and Cohen2001; Platt Reference Platt2002). Ongoing perception and evolving reward estimates reinforce some patterns and disrupt others, changing the trajectory of the evolving neural state and thereby the behavioral (and perceptual) trajectory of the organism. Similarly, local function emerges from the complex, dynamic interactions between large- and small-scale structures in the brain. Sometimes the function of larger structures can be understood by understanding the intrinsic functions of its parts and the nature of their interaction (Craver Reference Craver2007); but as is illustrated by the case of direction-selectivity in the dendrites of Starburst Amacrine Cells (SACs), other times the functions of the low-level parts appear to be determined by the constraints imposed by the larger structures with which they interact (Anderson Reference Anderson, Metzinger and Windt2015).

In the brain sciences, we need to develop models of explanation that allow for the possibility of top-down and bottom-up mutual constraint, in which both local and global function are synchronically co-determined by the dynamic coupling between elements at various spatial levels of organization. In After Phrenology, I therefore introduce the idea of Transiently Assembled Local Neural Subsystems (TALoNS). TALoNS are the temporary, reproducibly assembled functional parts (large- and small-scale networks and other elements) of the brain. TALoNS have intrinsic causal properties or dispositions determined by their internal structure and effective connectivity, but their functional selectivity (e.g., direction selectivity in SAC dendrites) emerges from the way these dispositions are constrained by the other functional structures with which they interact.

All of the above serve to underscore the following: The brain is a highly dynamic, adaptive system, in which structure and function are constantly adjusting to the changing circumstances of the organism. This is as it should be. The brain evolved to control action. It is a crucial mediator and modulator of the sensory-motor coupling that governs an organism's fit to its environment. Given this job, it had to be adaptive at multiple temporal scales, and capable of naturally managing the multiple simultaneous demands that are imposed by the complex interactions between an organism's needs and its perceived opportunities for action. The brain is an action-oriented, and not a perception-oriented, system. It is crucial to understand the implications of this fact for the nature of the brain and for the science that purports to study it. These matters are taken up in the next section.

5. Embodiment and cognitive processing

Traditional cognitive science is captured by a particular picture of our fundamental epistemic situation. According to that picture, sense organs are conduits for inputs called “sensations,” on the basis of which the individual organism generates a representation of the causes of that input –internally reconstructing the objects and properties in the external world. Cognition, in this picture, consists of the targeted internal manipulations of this reconstruction in service of the agent's goals – ultimately, deciding what to do next. Perception is induction, and cognition is calculation.

Acceptance of this framework accounts for the fact that one of the fundamental jobs of cognitive neuroscience has been to discover what is represented where in the brain, and how each representation is transformed into or impacts the others. Acceptance of the framework accounts for the abiding interest in specifying the innate “knowledge” or stored assumptions that guide perceptual reconstruction, whether that involves solving the (otherwise apparently intractable) problem of inverse optics (Edelman Reference Edelman2008; Marr Reference Marr1982) or inducing the grammar of natural language (Chomsky Reference Chomsky1957), for it is readily apparent that “sensations” are impoverished and unreliable – and need to be supplemented. Acceptance of this framework even accounts indirectly for the componential assumption that is built into most theories of the functional structure of the brain, for insofar as the challenges of perceptual reconstruction and cognitive calculation require specialized knowledge, it is natural to imagine specialized neural systems for solving those problems. Moreover, insofar as cognition is a matter of representation transformation, it must also involve information communication among these systems, which requires conduits and interfaces, and naturally leads to a modular architecture of stable, specialized, relatively isolated, nearly decomposable, message-passing components (van Gelder Reference van Gelder1995). The framework, this is to say, is deeply embedded in the cognitive sciences. But it is time to abandon it.

Perhaps the most fundamental problem is with the very concept of a sensation. To make a point that is at least as old as James (Reference James1890): “Sensation” is a theoretical construct, an abstraction away from actual experience. As with some fundamental particles of physics, sensations do theoretical work, but no sensation has yet been observed. If perception is reconstructive, then it needs building blocks, and sensations are the hypothesized blocks. If perception is reconstructive, it needs a starting point, and sensations are the hypothesized points. If perception is reconstructive, there is a definite order of events: sense, think, act. But perception is not reconstructive; representing the environment is not what our brains evolved to do. Our brains evolved to control action. Experience is not composed of atomic units, nor does it have a definite starting point; it is a continuous stream. Action does not come after thinking, which comes after perceiving; thinking, perceiving, and acting are synchronous and co-determining.

The alternative, action-oriented framework developed in After Phrenology consists of the following tenets: perception is active; perception is relational; the brain is a control system. Thinking – cognition – involves harnessing the mechanisms of sensory-motor coordination and environmental interaction to more abstract ends, but the character of the underlying mechanism remains what it has always been.

Perceiving is always acting because to know the world is to move about in it. Consider the case of olfaction, which is largely useless without the ability to move. All of the useful information about chemicals lies in the distribution in the environment, and picking up this information requires moving around. Put differently, chemical detection is not chemical perception unless and until it is chemotaxis. Touch, too, is quintessentially active: We feel the support offered by a surface, or the hardness of a material by pressing, the heft of a thing by lifting, and texture and shape by brushing and grasping. Naturally, one can be touched, just as one can subject to a chemical impingement, and such events may well convey information without movement, but these are degenerate cases for perceptual systems that normally function via movement. The same is true of vision: The passive reception of reflected light is the degenerate case for what is an active perceptual system. The data of visual perception are not the momentary impacts of reflected light in the retina, but rather the changes in the retinal projection as our posture and position changes. The problem of visual perception is not one of constructing a three-dimensional model of the world from passive two-dimensional stills; it is rather a matter of picking up on the world-specifying information available in the actively gathered stream of experience (Gibson Reference Gibson1966; Reference Gibson1979). The processes whereby we do this are of course still poorly understood; the point is that perception poses a different problem from what has been traditionally supposed. And it is this latter problem that our brains evolved to solve.

Because perception is both active and in the service of action, much of the information to which organisms are attuned is not objective information of the sort one might need for model-building, but rather relational information that is more immediately useful for guiding action in the world. It is the overall job of the organism's brain and nervous system to manage various organism–environment relationships. Perceptual systems keep the organism in touch with the values of these relationships: the closeness of the obstacle, the support of the surface, the passability of the gap. When we think otherwise, we can make scientific errors of an interesting sort, underestimating the accuracy of our perceptual systems. Consider the matter of weight perception. Humans are notoriously poor weight estimators and are liable to such errors as the size–weight illusion: given two objects of the same weight but different sizes, the smaller object will be judged heavier (Murray et al. Reference Murray, Ellis, Bandomir and Ross1999). On the traditional view, this fallibility is unsurprising. After all, the torque imposed on our arm as we hold an object in the hand will depend on the length of one's arm, the angle of the shoulder and the elbow, and other variables, and will change as we move about. Extracting any stable, objective property of the object would naturally be very difficult in light of such variation. But this is not how perception works. In fact, the information is in the variation, and the relational property that the information specifies in this case appears to be the throwability of the object. Humans turn out to be very accurate estimators of throwability (Zhu & Bingham Reference Zhu and Bingham2011). The position defended in After Phrenology is that most of perception should be understood on this relational model.

The last tenet that makes up the embodied framework outlined here is that the brain evolved to be the control system for an active, environmentally situated organism. The fundamental cognitive problem facing the organism – deciding what to do next – is best understood not as choosing the right response to a given stimulus, but rather as choosing the right stimulus – the right experience to seek – in light of a goal. Knowledge of sensorimotor contingencies (Noë Reference Noë2004) – of how perceptions change with action – and the perception of affordances (relationships between an organism's abilities and objects in the world that indicate opportunities for action) work together to allow an organism to follow chains or sequences of experiences to achieve its ends, whether that be a feeling of satiety, the experience of safety, or the perception of a finished nest. As Paul Cisek (Reference Cisek1999) has pointed out, all living things have homeostatic mechanisms that keep biologically relevant variables such as temperature, pH, or chemical concentrations within some acceptable range. Some of these mechanisms are metabolic or physiological, but others are behavioral: moving, eating, manipulating, and so on. The fundamental function of behavior, then, is to maintain organism-relevant variables within some desired range, and the fundamental function of the brain is to manage such behavior. The brain is a dynamic control system that modulates the sensorimotor coupling at multiple spatial and temporal scales.

In After Phrenology, I follow Cisek (Reference Cisek2007; Cisek & Kalaska Reference Cisek and Kalaska2010) in arguing that the biased pattern competition observed in the brain should be understood psychologically as biased affordance competition. What an organism's brain is fundamentally doing is managing the relationship between the organism and the environment, and its perceptual apparatus is specially suited for facilitating that task. An organism perceives the values of salient organism–environment relationships and, in light of some goal(s), acts so as to perceive the right changes in those relationships. The brain that manages this behavior is organized in such a way that its various parts have different dispositions to manage the values of the perceived relationships. Interaction with an environment offering multiple affordances causes regions of the brain to be differentially activated in accordance with their functional biases. A situation posing several possible courses of action will cause multiple distributed patterns of neural activation across the brain, and the behavior of the organism in this situation will be ultimately determined by competition among the patterns. I argue that this competition should be understood to reflect tension among the various behavioral control loops that could be enacted; loosely speaking: Pattern competition in the brain is affordance competition in the mind. The summed cooperation and competition among the active dispositions in the brain both determines the course of action and structures the control loop that facilitates the required behavior.

6. Function–structure mapping in an interactive brain

Over the past several sections, I have been advocating for a picture of the functional structure of the brain that illuminates its evolutionary and developmental origins, and does justice to the significant functional complexity of its individual working parts. I also advocated for functional fingerprinting as an appropriate tool for capturing and quantifying functional complexity. In fact, functional fingerprints and the style of thinking they promote may help point the sciences of the mind in a new and fruitful direction.

To see how and why, we need to appreciate the epistemic situation we are in. A scientific experiment is a deliberate intervention into the causal structure of the world. We intentionally manipulate – vary the value of – some physical condition and record the value of another. The signal that this intervention produces is generally mixed – that is, dependent on numerous causal factors that we would ideally like to disentangle. A simple example is the measurement of weight (or force more generally), which physics teaches us is in fact the product of two more fundamental properties, mass and acceleration. This realization gave us a better purchase on the underlying causal structure of physical reality. Similarly, the varying price of a stock over time is a mixed signal driven by multiple economic factors including the money supply, corporate profits, and perceived innovation, whereas the price of 100 stocks is a set of mixed signals all being driven by the same causal factors but to different degrees. The price of a tech stock might be relatively less sensitive to earnings and more to innovation than the price of an energy stock, for example.

Given this situation, we need to ask: When we measure the activity of 1 or 100 or 1,000 different parts of the brain, what is the underlying nature of this set of mixed signals? What are the psychological factors that contribute to the changing values we record from brain and behavior? In After Phrenology, I argue that the central guiding scientific quest for the cognitive neurosciences should not be determining what the basic cognitive operations implemented in individual regions of the brain are. The functional complexity of the brain suggests that this approach will offer at best an incomplete and at worst deeply misleading account of brain function. There should nevertheless be detectable regularities in the patterns we record from brain and behavior; there should be some underlying structure in the signal. Hence, I advocate for a science that asks: What are the psychological factors that best capture and account for the differential activity of the brain in various circumstances?

One reason functional fingerprinting can be so powerful is that it offers an avenue toward an answer. In the same way that analysis of people's responses to a variety of interventions can reveal a common set of factors defining individual personalities, so too the analysis of multidimensional functional fingerprints of brain regions and networks may reveal a set of primitive psychological factors (Barrett & Satpute Reference Barrett and Satpute2013; Gold et al. Reference Gold, Havasi, Anderson and Arnold2011; Lindquist & Barrett Reference Lindquist and Barrett2012; Lindquist et al. Reference Lindquist, Wager, Kober, Bliss-Moreau and Barrett2012; Poldrack Reference Poldrack2010; Poldrack et al. Reference Poldrack, Halchenko and Hanson2009). I call these neuroscientifically relevant psychological (NRP) factors. According to this approach, psychological states such as anger and fear, as well as processes such as attention and cognitive control, involve different mixtures of many of the same domain-general ingredients. These factors would map to the brain such that more than one part of the brain would support each factor, and more than one factor would load on each part. That is, brain regions and networks will differ not necessarily in terms of their component operations, but rather according to their loadings on a set of primitive NRP factors.

This scientific approach appears to better respect three organizational features of the brain emphasized here: (1) the functional diversity of individual regions of the brain, (2) the functional differentiation of individual regions of the brain, and (3) the frequent functional overlap between the constituents of different networks. It will also help us think our way beyond the functional model of linearly interacting components that we inherited from seventeenth-century mechanism and nineteenth-century engineering practices. In the brain, function emerges from structure in ways more complex than that model can capture. But we are developing tools adequate to the task.

What exactly are NRP factors? What is their best psychological construal? That is, of course, an open question, one that will be answered as part of doing the science described in After Phrenology, not in advance of it. In my view, NRP factors index basic dispositions to help manage the value of some organism-relevant environmental variable or relationship (see sect. 5). Because neural reuse has both an anatomical and a behavioral aspect, we should expect to see these dispositions manifest in multiple circumstances. Hence, there might be basic factors for managing closeness and warmth, and these might manifest in both physical and interpersonal contexts (Bargh & Shalev Reference Bargh and Shalev2012; Xiao & Van Bavel Reference Xiao and Van Bavel2012). Therefore, we would also expect the regions of the brain that load on the relevant factor to be active across these different contexts.

It is of course an implication of the approach that the fundamental NRP factors that we are seeking have generally not been already identified, and will cross-cut the current taxonomy of psychology. In After Phrenology, I marshal the evidence for this claim; here, I will simply note the following: given that cognitive neuroscience (and, indeed, psychology more generally) has yet to be deeply influenced by evolutionary biology, and that it adopted wholesale the psychological taxonomy of cognitive psychology that, as I have argued above, is organized around a faulty framework (and was initially devised to be a science autonomous from the neurosciences besides), then it would be something of a miracle if the right set of concepts had already been formulated. I believe that following the path laid out in After Phrenology will lead to a new and better vocabulary for understanding mind, brain, and behavior. Moreover, I argue that this vocabulary will better reflect the evolutionary history of human beings, and the action-orientation of cognition, if it is organized not around the concepts of sensation and representation, but rather around the notion of an affordance.

7. Reuse, interaction, and “higher-order” cognition

As I noted in the introduction, thinking and acting are mutually supporting and intertwined at multiple timescales. We think with and through our interactions with objects and one another. We routinely act to help us see and think: we spin puzzle pieces to make their fit easier to perceive, rearrange playing cards and Scrabble tiles to make patterns easier to detect, and label our environments with signs to aid memory and ease navigation (Clark Reference Clark1997). And just as we create physical tools such as hammers, knives, and levers to augment our physical capacities, so too we invent cognitive artifacts to augment our mental ones. Among the most important of these are the cultural practices of speaking, writing, and calculating, and the symbol systems that support them. And what is deeply fascinating, and helps illuminate the true nature of human intelligence, is that we treat these cognitive artifacts just like physical ones, reusing our finely honed abilities for interacting with objects in the service of improving our thinking.

Consider mathematical symbols: People point at them, gesture over them, move them, and strike them out. These actions serve myriad purposes: They direct spatial attention, they aid memory, they keep one's place in the problem-solving procedure, and they make a solution easier to reach. These actions are not peripheral to knowing and doing mathematics, but part and parcel of it. Mathematical symbols have the character that they do so that perception–action loops can be brought to bear on – be harnessed to – the practice of calculating. Equations have affordances that invite us to act on and with them to achieve the task they were designed for. To learn algebra is to acquire a sensorimotor skill, and acting in accord with the rules of algebra is a matter of learning to see and act in accord with the transformations that the equations afford (Landy & Goldstone Reference Landy, Goldstone, Taatgen and van Rijn2009).

Does this mean that doing math is mindless and noncognitive, that it does not involve thinking? Of course not! I hope it is clear by now that the rigid distinctions between sensing and thinking and doing are among the many bad ideas that need to be jettisoned in our reformed science of the mind. Seeing and touching and interacting with and manipulating things are partly constitutive of thinking. We have achieved our cognitive capacities in part because we have found ways to reuse our physical capacities to augment our mental ones; in a process supported by neural reuse, we repurpose our behavioral routines in multiple circumstances for myriad cognitive ends.

To drive this point home, and to preempt the argument that the embodied, embedded, evolutionary developmental account of cognition that is developed in After Phrenology can never account for our capacity for natural language, I outline a theory of language according to which language is an interactive social practice. It is both a form of joint action (Clark Reference Clark1996; Sebanz et al. Reference Sebanz, Bekkering and Knoblich2006) and a coordinating structure for facilitating cognitive and social interactions (Tomasello Reference Tomasello1999). Language works by presenting and manipulating cultural affordances that will cause one's dialog partner(s) to see and do what the speaker intends to be seen and done. Language works because it has developed to take advantage of and is fitted to our interactive sociality (and not because we evolved specialized, dedicated, modular neural machinery to support it). Like all successful artifacts – physical and cognitive both – it has the right two-way fit: It suits both our abilities and its purpose. And like all successful cognitive artifacts, it enhances our capacities in various ways: It aids memory, improves self-control, biases attention, and more. There is, of course, much more to the argument, and much more to the story, and for that I hope you will turn to After Phrenology.

8. Psychology after phrenology

As I hope is clear in this précis, and as I hope is compelling in the book it introduces, I am calling for the development of a new functionalism as the basis of a unified science of mind that respects its biological bases, its evolutionary history, and its environmental and cultural embeddedness. Among its tenets are the claims that the functional architecture of the brain has been established by natural selection through a process marked by both differentiation and continuity, that our complex and diverse behavioral repertoire is supported primarily by the ability to dynamically establish multiple different functional coalitions coordinating both neural partnerships and extra-neural resources, and that the brain is fundamentally action-oriented, with its primary purpose to coordinate the organism's ongoing interactions with the world and adjustments to external circumstances. What might psychology and neuroscience look like if the framework I advocate in After Phrenology were widely adopted? In an appendix to the book, I lay out the theoretical challenges and a specific research agenda. Here, I will end with a broad-strokes characterization of the science to come.

1. 1. We will represent the functional activity of the brain in a multidimensional manner that captures the underlying functional and dispositional properties, and we will give up the notion that the neural responses we observe and measure must reflect the engagement of a single unified function.

2. 2. We will expect not just local, but also distributed contributions to overall function, determined by the interactions between top-down and bottom-up, feed-forward and feed-back processes. Structurally, we will attend to the interactions between regions – how these change and how they map onto changes in behavior. We will develop better noncomponential models of functional integration that can capture the myriad ways that function emerges from interacting structure. Developmentally, we will work to establish the mechanisms whereby potential functional partnerships in the brain are discovered, tested, and maintained. Evolutionarily, we will seek to capture the adaptivity of the organism in all its forms and to understand that natural selection targets not just structures but also processes.

3. 3. We will deeply rethink the vocabulary of cognition, ideally giving the brain a voice in the process. In discerning what the brain cares about, we will remember that it evolved to be an action-control system, specializing in managing the values of salient organism-environment relationships. Hence, many of the properties to which the brain is attuned will be action-relevant and relational; throwability and climbability will likely be more important to the brain than weight and slope.

4. 4. We will recognize that cognition does not take place in the brain alone. We think with and through artifacts and one another. Although it will always be tempting (and occasionally necessary) to bracket off the natural and social worlds to focus on the brain in isolation, we will work to develop experimental paradigms that include robust social and environmental interactions, and we will develop techniques for measuring the details of the interactions among brain, body, and world.

5. 5. We will embrace the empirical tools offered us by machine learning, graph theory, independent component analysis, multidimensional scaling, linear algebra, dynamic systems theory, and so forth, that promise to help us do justice to the dynamic complexity of the brain. We will realize that the focus on local, linear correlations between brain activity and simple stimuli will never be by itself sufficient to capture the complexity of the brain and its interacting parts. We will turn to empirical tools better suited to measuring distributed information and able to disentangle the psychological mixtures that brain activity reflects.

I believe that this is the most exciting time in the history of the neurosciences. We have at our disposal phenomenal technological tools allowing us to measure and analyze function in ways unimaginable even just a few short years ago. If we can manage to match the quality of our conceptual and experimental tools to the quality of our technology, the scientific future is very bright. I hope After Phrenology can help illuminate the path.