In May 2011, Wired magazine published a paper under the provocative title “Spies, meet Shakespeare: intel geeks build metaphor motherlode.” This title generates curiosity, as spies and Shakespeare belong to two different “semantic fields.” Although some notable writers, particularly British writers such as W. Somerset Maugham and Graham Greene, belonged to the intelligence community, Shakespeare was probably not one of them.

The author of the article, Lena Groeger (2011), introduces the Intelligence Advanced Research Projects Activity (IARPA) – “the mad science unit of the intelligence community” – and its metaphor project, which aims to “exploit the use of metaphors by different cultures to gain insight into their cultural norms.”

In fact this project has drawn wide attention; The Economist joined the celebration by publishing the article “Metaphors we do everything by?” (R.L.G. 2011), and other newspapers have published similar stories.

IARPA is a relatively new American agency that was established to develop “the next generation of spycrat technologies” (Bhattacharjee 2009).

In the context of “spycraft,” metaphor identii cation isn’t the most salient tool. Our images of the spy world are nurtured by Hollywood rather than by the real world. We can imagine agent 007 – James Bond – driving a car that turns into a war machine. We can imagine the tech geeks who operate the most sophisticated technology to locate the place where Jason Bourne, the hero of The Bourne Identity, is calling from. However, we would hardly imagine a spy movie in which a secret agent bursts into the technology geeks’ oi ce shouting, “Urgent! I need the metaphors used by the Iranians to describe the United States.”

Building on the previous chapters, this final chapter addresses the preeminently multidisciplinary question of how human cognitive architecture might be implemented in the brain's neural architecture. Focusing on perceptual organization, this question is addressed in a pluralist way which — sustained by neuroscientific evidence and metatheoretical considerations — combines complementary insights from representational, connectionist, and dynamic systems approaches to cognition. To this end, I expand on van der Helm (2012) and I revisit several things from the previous chapters to discuss them in the light of this cognitive architecture question. Next, I briefly introduce the main ingredients.

Cognitive architecture. The term cognitive architecture (or unified theory of cognition) refers to computational models of not only resulting behavior but also structural properties of intelligent systems (Anderson, 1983; Newell, 1990). These structural properties can be physical properties as well as more abstract properties implemented in physical systems such as computers and brains. There is no consensus about what these structural properties should be, and indeed, many different cognitive architecture models have been proposed (for extensive reviews, see, e.g., Langley, Laird, & Rogers, 2009; Sun, 2004). These models differ, for instance, in whether they involve fixed or flexible architectures, in what forms of processing they allow, and in the extent to which they are based on a set of symbolic information-processing rules applied by one central processor or rely on emergent properties of many interacting processing units.

The germ of the ideas presented in this book took root in the minds of the early twentieth-century Gestalt psychologists. They argued that perceptual organization involves a complex interaction between parts to arrive at wholes, and they proposed the Law of Prägnanz as governing principle. This law expresses the idea that the brain, like any dynamic physical system, tends to settle in relatively stable neural states characterized by cognitive properties such as symmetry, harmony, and simplicity. In the 1960s, this holistic idea was overshadowed by the rise of single-cell recording (which marks the beginning of modern neuroscience), but in the 1970s, it started to return to the mainstream of cognitive neuroscience. Nowadays, not only representational approaches like structural information theory (SIT) but also connectionism and dynamic systems theory (DST) tend to trace their origins back to this idea — even though they use quite different tools to implement it in formal models.

In this book, I made a case for a multidisciplinary approach to perceptual organization, precisely because different tools are needed to address the different questions of (a) what the nature is of the mental representations of percepts; (b) how cognitive processes proceed to yield these representations; and (c) how these processes and representations are neurally realized. To address these questions, I used SIT as operating base, but if one looks beyond the differences in tools, then the conceptual parallels between SIT, connectionism, and DST seem to prevail.

Preface

For Lord Byron, who was an adventurer and probably a thrill seeker, the pleasure in the “pathless woods” may have been the result of the thrill associated with the uncertainty, promise, and danger of traveling in an unknown territory. I am neither an adventurer nor a thrill seeker. However, from a very young age, I have found great pleasure in traveling the pathless woods of knowledge. A kind of a wimpy adventurer, you may call me. These travels have led me to very varied territories of human knowledge, from experimental psychology (Neuman and Weitzman 2003) and psychoanalysis (Neuman 2009a) to mathematical modeling (Neuman et al. 2012); from theoretical biology (Neuman 2008) to semiotics (Neuman 2009b); and from discourse analysis (Neuman et al. 2001) to innovative information technologies (Neuman et al. 2013b). What is interpreted by some of my colleagues as a symptom of an academic multiple personality disorder is for me a natural and legitimate expression of a deep intellectual passion.

Knowledge is not naturally demarcated by borders, and wherever borders exist in our minds they indicate our tendency to force order in a simplistic way and to follow the social power relations of academic politics. Physicists who study the semantic networks of language are not linguists, but their contribution to our understanding of human language is indispensable. Should this work be appreciated despite the fact that it transcends disciplinary boundaries? My answer is: yes!

You can’t beat the system – unless you try

In 2008, during the great fall of the stock market, I was on my way to my university coffee shop. One of my colleagues, a graduate of an Ivy League university and an internationally recognized economist, met me and we started talking (how surprisingly) about the stock market and the failure of economic models to predict its chaotic behavior. “Everyone would like to predict whether a stock price is going to increase or decrease,” he said, “but unfortunately it is impossible. You cannot beat the system.”

As a researcher deeply interested in the psychology of human beings in their complex sociocultural context, I was not surprised by this pessimistic conclusion. Complex systems, specifically complex interactive psychocultural systems, are difficult to understand and to predict. The question is what constructive and optimistic vision we may provide for those who mess their hands while trying to understand these systems.

The idea of the theoretical cycle of research is to formalize assumptions, to see if they can be underpinned by derivations from first principles. This method is characteristic of mathematics, in which a theorem usually starts as a conjecture that calls for a proof. The search for a proof may be successful, but may also lead to the conclusion that the conjecture is false or has to be adjusted to be provable. A successful proof means that the correctness of the conjecture can be derived logically from facts proved earlier, and hence, from first principles.

In this first part, Chapter 1 sets the stage by presenting an overview of visual information-processing ideas adhered in structural information theory (SIT). The central idea in SIT is the simplicity principle, which holds that the visual process yields simplest organizations of stimuli. An implicit assumption then is that such organizations have evolutionary survival value in that they are sufficiently veridical to guide us through the world. In Chapter 2, this assumption is addressed in a historical and multidisciplinary setting, using findings from the mathematical domain of algorithmic information theory (AIT). SIT and AIT developed independently of each other, but provide similar modern alternatives for Shannon's (1948) classical selective-information theory.

Notice that Chapter 2 contains mathematical proofs which, however, do not pinpoint the exact degree of veridicality of simplest perceptual organizations (which is probably impossible; see also the Prologue).

In the 1960s, SIT began as a theory about the nature of the mental representations produced by the perceptual organization process, that is, as a theory at what Marr (1982/2010) called the computational level of description (see the Prologue). In this chapter, I discuss a formal process model, thereby including the algorithmic level of description.

As indicated in Chapter 1, the application of SIT's formal coding model in the empirical practice involves three steps. First, for a given proximal stimulus, candidate distal stimuli are represented by symbol strings; second, coding rules are employed to determine simplest codes for these strings; and third, the overall simplest code is taken to reflect the percept in terms of a hierarchical organization of a distal stimulus that fits the proximal stimulus. As also indicated in Chapter 1, the experimenter is expected to make psychologically plausible choices to perform the first step, but after that, strictly formal rules take over. That is, the second and third steps reflect SIT's ideas about the perceptual organization process, in that the actual process is assumed to rely on the same information-processing principles as those which SIT's formal coding model applies to strings.

In SIT's formal coding model, however, the selection of simplest codes of strings poses a computing problem that seems intractable by traditionally considered forms of processing. This has raised questions about the practical feasibility of the simplicity principle (cf. Hatfield & Epstein, 1985).

In the previous chapter, I specified perceptual organization as an autonomous process that enables us to perceive scenes as structured wholes consisting of objects arranged in space. Because any scene can be interpreted in numerous ways, it is amazing not only that the visual system usually has a clear preference for only one interpretation, but also that this interpretation usually is sufficiently veridical to guide us through the world. Indeed, it is true that visual illusions show that what we see is not always what we look at, but a fair degree of veridicality seems necessary — otherwise, our visual system would probably not have survived during evolution. In this chapter, expanding on van der Helm (2000), I assess what degree of veridicality vision might achieve by aiming at simplicity. To this end, I elaborate on three related issues, which I next introduce briefly.

Simplicity versus likelihood. The main issue is whether the perceptual organization process is guided by the likelihood principle (von Helmholtz, 1909/1962) or by the simplicity principle (Hochberg & McAlister, 1953). Both principles take this process as a form of unconscious inference yielding interpretations which persons subjectively believe are most likely to be true. The question, however, is what drives this unconscious inference, and as I indicate next, the two principles differ fundamentally in this respect.

The likelihood principle, on the one hand, aims explicitly at a high degree of veridicality in the external world.

The paradigmatic starting point in this book is that, for a given proximal stimulus, the distal organization with the simplest descriptive code is predicted to be perceived. This starting point as such, however, does not yet prescribe which specific coding scheme is to be employed. This has been pointed out by Simon (1972). He compared a number of perceptual coding models (including SIT's), and he found that these models perform about equally well. In the 1960s, a similar finding in mathematics (i.e., the Invariance Theorem; see Chapter 2) brought mathematicians to conclude that the descriptive simplicity paradigm is sufficiently robust to modeling variations to form the basis of promising research. Simon's (1972) equally rightful conclusion, however, was:

Hence, Simon called for a foundation of the specifics of the coding model to be used in perception. These specifics comprise (a) the coding rules that are applied to capture regularity, and (b) the information measure that is applied to quantify complexity. In vision, the regularities to be captured are visual regularities and the information to be measured is structural information. This, however, evokes profound questions:

1. • How to distinguish between visual and nonvisual regularities?

2. • How to measure amounts of structural information?

The idea of the tractability cycle of research is to assess if models allow for practically feasible process implementations. This method stems from computer science, is characteristic of artificial intelligence research, and is also a fruitful method in cognitive neuroscience (van Rooij, 2008). Various models, including SIT's coding model, base their predictions on the selection of one outcome from among a highly combinatorial number of candidate outcomes, so that a naive selection method could easily require more time than is available in this universe. Such a naive selection method then is said to be intractable, no matter whether it is to be performed by computers or by brains.

As I discuss in Chapter 5, SIT's coding model does allow for a tractable implementation, however. Simplest organizations of strings can be computed via a combination of feature extraction, feature binding, and feature selection – where the binding mechanism is special in that it allows for transparallel processing by hyperstrings. This form of processing – which is feasible on classical computers – is as powerful as quantum computing promises to be, and does justice to the high combinatorial capacity and speed of perceptual organization.

Scientific research is an endeavor to enable us — via metaphors, theories, and models — to understand and thereby control reality. We may never be able to fully understand reality, however, because the main tool we use to understand reality, namely our brain, is an inextricable part of reality. At best, so it seems, scientific research may arrive at an understanding of reality as we experience it subjectively, that is, acknowledging the workings of the brain. Understanding the latter is the objective of cognitive neuroscience. Even so, cognitive neuroscientists too use their brain as a tool to understand data. Just as in daily life, this is a potential pitfall because, as an abundance of visual illusions are proof of, what you think you see is not always what you look at.

In this context, human vision research is special in that it takes vision not only as mediating instrument but also as the very topic of study (cf. Rock, 1983). It recognizes that vision is a complex yet fast process that organizes meaningless patches of light on the retina into the objects we perceive, that is, objects with potentially meaningful properties such as shape and spatial arrangement of parts (see Fig. P.1). In other words, when we look at a scene, the objects we perceive constitute the output of vision — not its input.