Introduction

This chapter takes an overview of sensory systems and some of the general principles needed to understand them. It also takes a very brief look at the brain itself. Although the focus of the book is really at the input and encoding end of the brain, a knowledge of its basic structure is helpful in understanding how the various information processing pathways fit together.

Before looking at each of the principle sensory modalities possesed by humans it is intriguing to ask what senses might exist. Have animals learned to exploit every possible known physical force or interaction? On the other hand are there senses yet to be discovered, where there are no established physical mechanisms? The first of these questions is tractable, at least in principle, and §2.2 offers a framework. The second lies outside the scope of the book.

By starting with the physics the limits to information processing (§2.3) become apparent (and in fact animals are pretty good). Two overarching methodologies arise out of the physics viewpoint: the representation of signals, the subject of Chapter 3, a key idea in Horace Barlow's early work, and information theory, the subject of Chapter 4. They form the theoretical core of the book. But perception is not a simple one way transmission of information from eye or hand to brain. It is highly conditioned by what we know and what we expect. Thus what happens at the periphery depends to some extent on what happens deep inside the brain.

In the last decade a lot of growth has occurred in our understanding of the integration of sensory systems. In fact entire conferences are now devoted to integration, both biological and robotic. The individual senses usually reinforce one another, enhancing feature detection and object recognition. But in some cases there may be contradictions and all sorts of strange experiences result.

From the perspective of the themes of this book, integration seems contrary to the fundamental strategy of streaming. So what we are really interested in is the way these streams are used to cross-check each other and how they are used to make decisions and guide behaviour. There are some deep questions of information theory, again beyond the scope of this book in any quantitative treatment. The data from one sensory stream can act as a cross-reference or prior, in effect reducing noise. As a rapidly expanding area, this chapter is thus a selection rather than comprehensive overview of integration. The topics it covers are:

• System integration exemplified by the optokinetic system (§12.1). The vestibular system provides feedback to the eye muscles to control gaze direction independently of head movement. This is the most complex integration system discussed and is fundamental to visual processing.

• Cross-calibration,where one sense is used to calibrate or adjust the inputs from another (§12.2). Unlike the opto-kinetic system, which operates continually, calibration systems may operate intermittently, e.g. at dusk.

• Integration areas in the brain where data are collated from more than one sense (§12.4.3). This is obviously a huge topic, where at best a few key ideas can be discussed.

• Unusual effects arising from sensory conflicts, such as various kinds of touch illusion and out-of-body experiences (§12.1.8).

• Consciousness is a philosophical grand challenge. What is it, what animals have it, what are its brain correlates (if any) and many other questions are still earnestly debated. Needless to say this book cannot really enter into these arguments. But our impression of the sensory world is a unitary one, and there are some sensory discoveries which shed a little light on these most difficult of questions (§12.5).