Overview

Two aspects of the what, or ventral pathway, are colour and texture. In the biological world colour and texture are properties of surfaces which help in identifying objects, often rapidly. This realisation that colour did little in defining the boundaries in an image (§8.5.2) and the shape of things is relatively recent, the last few decades, whereas interest in colour itself goes back to the time of Newton and before.

Colour is essentially concerned with what things are, the nature of surfaces and the objects they represent. In mammals it plays a minimal role in form analysis, separating objects from one another or in analysing their shape. It travels exclusively along the P/K-cell pathways from the retina (§6.6.2) to the area V4 where colour, as opposed to wavelength of incoming light, is extracted (§8.4.2).

Colour does not carry anywhere near as much information as the monochromatic channels and subserves nowhere near as many functions. Yet it fascinates us, with writings from the ancient Greeks onwards, which Wade discusses in his book on the history of vision (Wade, 1998). It seems to generate emotional connotations. We talk of red with anger or green with envy. Kandinsky based his abstract art on elaborate theories of colour, while some of the great abstract expressionists, such as Mark Rothko, relied almost entirely on subtle shades of colour to generate a deep impact out of pictures with little figural content.

Introduction

The importance of the chemical senses varies throughout the animal kingdom. Humans are dominated by audiovisual stimuli. Not only is this reflected in the amount of brain capacity devoted to chemical senses, but it is also reflected in language. In a wide-ranging study of languages across the globe, two-thirds to three-quarters of words denote sensory experience or function referring to hearing or vision (Wilson, 1998).

We think of our senses as quite distinct, although there are cross-over effects, referred to as synaesthesia. People with synaesthesia get strong percepts of another sense from the stimulus of one. Thus a particular smell or musical tone may invoke a distinct colour (§12.3.2).

Chemical senses are at their most developed in olfaction, yet there are chemical sensors throughout the skin and internal organs. In fact chemical sensing might be considered the first to evolve of all the senses, being present in simple unicellular organisms such as bacteria and protozoans. Some are able to sense and move along chemical gradients. Taste and smell have obvious similarities and synergies, but the detector systems are not confined to the tongue and nose. Even bacteria emit a range of chemicals which impact across eukayrotes, plants, fungi and animals (Dunkel et al., 2009).

Our subjective experience of smell tends to be one of gathering environmental information. Think how different it is for many other mammals. Think about how dogs sniff each other when they meet. Think about the way cats are dominated by the smells in their territory, particularly the highly specific urine markers of other cats. There is another lifestyle out there, a lifestyle in which molecules, semiomolecules, are synthesised to be passed by one animal or plant to another as a signal. It might be a signal to mate, a signal that fruit is ripe and ready to eat, a signal of where other members of the group have found food.

Vision is the sense with the greatest bandwidth, but our sense of hearing provides distinctive, equally vital information. Apart from communication through speech, tone voice conveys a full gamut of emotions. Moreover sound, with quite different properties to light, carries quite different environmental information, and is less directional than vision.

Human hearing is good, but by no means the most sensitive within the animal kingdom, or that with the greatest frequency range. But again, animals get very close to the limits imposed by physics. As computing power and miniaturisation have grown dramatically in the last decade, knowledge of hearing has become essential to determining the standards for the capture and storage of audio information from telephone conversations to games. In 2010 the directional information from sound is still not very well captured and utilised in artificial systems. Chapter 7 covers the processing of directional information.

Having got the theoretical building blocks behind us, sound is a good place to start in the study of the senses themselves. It embodies all the ideas of the chapters on information theory and Fourier Analysis, but, being one-dimensional, is somewhat simpler than vision. Moreover, the idea of frequency in sound is commonplace, and thus easier to discuss, whereas spatial frequency in vision is a less everyday concept.

But whereas the theoretical framework of Fourier Analysis is pivotal to understanding vision and hearing, the sonic transduction elements have a great deal in common with touch and we shall come back to them in Chapter 10. In fact these transduction elements are some of the oldest in evolution.

Introduction

This chapter takes an overview of sensory systems and some of the general principles needed to understand them. It also takes a very brief look at the brain itself. Although the focus of the book is really at the input and encoding end of the brain, a knowledge of its basic structure is helpful in understanding how the various information processing pathways fit together.

Before looking at each of the principle sensory modalities possesed by humans it is intriguing to ask what senses might exist. Have animals learned to exploit every possible known physical force or interaction? On the other hand are there senses yet to be discovered, where there are no established physical mechanisms? The first of these questions is tractable, at least in principle, and §2.2 offers a framework. The second lies outside the scope of the book.

By starting with the physics the limits to information processing (§2.3) become apparent (and in fact animals are pretty good). Two overarching methodologies arise out of the physics viewpoint: the representation of signals, the subject of Chapter 3, a key idea in Horace Barlow's early work, and information theory, the subject of Chapter 4. They form the theoretical core of the book. But perception is not a simple one way transmission of information from eye or hand to brain. It is highly conditioned by what we know and what we expect. Thus what happens at the periphery depends to some extent on what happens deep inside the brain.

In the last decade a lot of growth has occurred in our understanding of the integration of sensory systems. In fact entire conferences are now devoted to integration, both biological and robotic. The individual senses usually reinforce one another, enhancing feature detection and object recognition. But in some cases there may be contradictions and all sorts of strange experiences result.

From the perspective of the themes of this book, integration seems contrary to the fundamental strategy of streaming. So what we are really interested in is the way these streams are used to cross-check each other and how they are used to make decisions and guide behaviour. There are some deep questions of information theory, again beyond the scope of this book in any quantitative treatment. The data from one sensory stream can act as a cross-reference or prior, in effect reducing noise. As a rapidly expanding area, this chapter is thus a selection rather than comprehensive overview of integration. The topics it covers are:

• System integration exemplified by the optokinetic system (§12.1). The vestibular system provides feedback to the eye muscles to control gaze direction independently of head movement. This is the most complex integration system discussed and is fundamental to visual processing.

• Cross-calibration,where one sense is used to calibrate or adjust the inputs from another (§12.2). Unlike the opto-kinetic system, which operates continually, calibration systems may operate intermittently, e.g. at dusk.

• Integration areas in the brain where data are collated from more than one sense (§12.4.3). This is obviously a huge topic, where at best a few key ideas can be discussed.

• Unusual effects arising from sensory conflicts, such as various kinds of touch illusion and out-of-body experiences (§12.1.8).

• Consciousness is a philosophical grand challenge. What is it, what animals have it, what are its brain correlates (if any) and many other questions are still earnestly debated. Needless to say this book cannot really enter into these arguments. But our impression of the sensory world is a unitary one, and there are some sensory discoveries which shed a little light on these most difficult of questions (§12.5).