Introduction and overview

Vertebrates have two eyes, ears and nostrils, in fact they are pretty much bilaterally symmetrical (although we have just one liver, for example). We can think of several reasons why this might be the case. Animals fight and get injured, or they get injured in other ways, so having two eyes or two ears (vets do a lot of business sewing cats' ears back up) provides some degree of redundancy. Having an eye on each side of the head makes it possible to see a large portion of the world, a more panoramic view. But in the sensory domain two channels enable the extraction of directional information. In the case of olfaction, the information, the differential arrival of odours at the two nostrils gives some indication of the direction of a source, but it remains imprecise. But for hearing and vision it is very precise indeed.

This chapter covers an important theoretical idea, the correspondence problem which underlies several aspects of sensory processing. When the information is collected by two spatially separate detectors, the signals coming in to each do not necessarily match perfectly. The differences can be used to infer something about the spatial properties of the signal. The two primary sonic differences are a variation in time of arrival and a difference in intensity arising because the signals have travelled slightly different routes.

What are the commonalities of information gathering and processing in all living creatures? This is the implicit question that underpins Terry Bossomaier's ambitious book The Senses. His is a Herculean task and one to be greatly applauded.

Bossomaier addresses the senses using the tools of contemporary information science, in an attempt to provide a unifying perspective, one that allows for quantitative comparison of senses between the species.

This fascinates me. It is now nearly 35 years since Simon Laughlin, Doekele Stavenga and I introduced information theory to understand the design of eyes, both compound eyes of insects as well as the simple eyes of humans. We recognised that the fundamental limitations to resolving the power of eyes are the wave (diffraction) and particle (photon noise) nature of light. By appreciating their interrelation we derived insight into the design and limitations of eyes, especially between the optical image quality and the visual photoreceptor mosaic. The capacity of the eye to perceive its spatial environment was quantified by determining the number of different pictures that can be reconstructed by its array of visual cells. We were then able to decide on the best compromise between an animal's capacity for fine detail and contrast sensitivity. In a series of papers, including those with Bossomaier and A. Hughes, we went on to use the tools of information theory to study various aspects of eye design. It was a rewarding and rich endeavour, but one at that time limited to vision.

Introduction

This chapter collects together two quite different systems. The first is the sensor system distributed throughout the body which monitors what is going on inside or outside, the temperature, chemical and tactile (pressure and vibration).

The second is the vestibular system which monitors the position and movement of the body in space. It interacts quite strongly with the tactile system. So, for example, the receptors in the skin in the feet contribute to maintaining our balance and can take over if the vestibular system gets damaged. The vestibular system is also tightly coupled to the visual system, through the opto-kinetic control of eye movements and gaze, a topic which §12.1 takes up in detail. Yet it is located inside the ear. Unsurprisingly, these systems share common principles and some transduction mechanisms with the senses considered earlier in the book.

These two systems are at the forefront of developments in virtual reality and computer games. Haptic interfaces are fast growing in importance (§10.8.4) and monitoring gaze has numerous evolving applications in human computer interface design The vestibular system appears obliquely in its role in physically interactive games and total immersion (§12.1.8).

Collectively these sensors comprise the the somatosensory system, from the Greek word soma for body. The sensors embedded in the skin monitor the impact of the external world and gather information about it. Hearing and vision are active senses in that we turn our heads or move our eyes to attend to a stimulus, and we work from hypotheses of what we might be seeing or hearing.

Overview

This theoretical chapter examines core ideas for the whole book, the ideas of linear systems, vector spaces and functional representation. All the sensory modalities sample the incoming signal in some way, producing a discrete representation. They then transform and split this signal up into streams. Along the way the data is usually compressed. This chapter deals with how to sample a signal and represent it in different ways. The section on information theory (§3.9.3) then takes on the topic of compression.

One fundamental representational example is Fourier decomposition, crucial to vision and hearing. It is one of the most important and powerful ideas in the whole of engineering and communications and it is essential to understanding sensory processing. The mathematics is rather complicated, but the emphasis herein is on the ideas rather than the detailed formalism. For the interested reader, excellent books describing the mathematical content in much more detail are Linear Systems, Fourier Transforms and Optics by Gaskill (1978) and The Fourier Transform and its Applications by Bracewell (1999).

We already have an intuitive feel for these ideas. We know how somebody's voice changes if they have a cold, when they talk over the telephone; we know how pictures can be blurred by a dirty lens, or can vary in contrast from soft portraits to sharp lithographs. All these common phenonema are examples of filtering, of changing the frequency balance in a signal.

A visitor from the mild climate of the UK to Rochester, New York State, in the middle of summer, receives a sensory shock. Apart from being much, much warmer, the visceral impact is huge. The light is brighter, the colouring of the birds is dramatic and the scent of the trees and plants is just so strong. At night the circadias are almost deafening. The information about the world gathered by sensory systems is the core idea this book will explore.

It so happens that Rochester is the home of Eastman Kodak and other major imaging companies such as Xerox and Bausch and Lomb, the place where a lot of important research on image physics, capture and storage took place. Images have played a part in human culture since the earliest cave paintings, but as computers have got faster, dynamic images, from mobile phones to giant plasma displays increasingly dominate our lives. Reproduced sound has gone beyond the radio to the ubiquitous MP3 player, seen on countless commuters, runners and diverse workers. But there are other senses, not yet so widespread in the artificial world. This book lies at the interface of the sensory biological world and man-made systems. It also projects forward to new computer interfaces and virtual environments not far down the track.

The senses of many animals, especially human beings, are very powerful general purpose information-seeking systems. A cat soon learns to recognise the sound of metal on metal of tin opener on tin or the sheepdog the distinctive whistle from which he receives his instructions.

As human beings we are aware of just five senses – vision, hearing, touch, taste and smell, which we have looked at in some detail in the previous chapters. Are there any others? As we noted in Chapter 9 we do have a subsidiary olfactory sense, the vomeronasal organ, known to be sensitive to pheromones in other mammals. There is still argument about whether human beings do indeed detect pheromones, but the evidence is mounting that we do (§9.3.9). Thus, as we come to the end of the book, we look at other senses which human beings either do not have or senses which have atrophied to the extent that they are no longer accessible to consciousness.

But are there other senses of which we are not aware? If we leave out paranormal phenomena, the only possibility might be some sort of innate navigational system. But in other animals there are documented electrical, magnetic and infrared image sensors. Compared to the other senses discussed previously, these are far less studied. Thus the information estimates in this chapter are considerably less accurate and more speculative than before.

Electrical sense

If you have the misfortune to be chased by a shark, don't bleed! Sharks have an exceptionally acute sense of smell. But even if, wrapped up tight in a wet suit, from whence you omit no detectable odour, the worst may still not be over. Sharks have an electrical sense, able to pick up the electrical signals from animal neural and other activity (Kalmijn, 1971). It gets worse if you bleed – injuries release electrolytes into the water enhancing the electrical activity and the shark uses its electrical sense to home in on the victim at close range (Fields, 2007b).

Overview

Information must be one of the most used words of today's world. Computers, media and the internet dominate many of our lives. The volume of information we collect and store increases annually, so much so that energy efficiency for data storage and manipulation is becoming a serious issue (Ranganathan, 2010). This omnipresence has given us everyday measures of information, bits, nips, nibbles, bytes, kilobytes, megabytes, gigabtyes terabytes, petabytes, exabytes … But what do these quantities really mean?

Entropy is effectively average information, and the quantity which defines the capacity of a transmission channel or storage medium. It is this second of the two fairly mathematical chapters addressing three interrelated concepts: information and entropy (§4.2, §4.2.1); noise (§4.4); and bandwidth, which links back to the previous chapter on Fourier Analysis. The emphasis is, again, on the concepts and how to visualise what happens rather than detailed physics and mathematics. Bialek (2002) gives a very detailed and thorough account of the mathematical framework.

Information has a companion concept, entropy (§4.2). Whereas anybody would offer you a definition of information, fewer would hazard a guess at entropy. Yet together they are amongst the most powerful of all concepts in science and engineering. In statistical mechanics, entropy is a measure of the disorder of a system and corresponds to the number of microstates a system can occupy. Without delving too deeply into the physics, which space does not permit, the relationship to computer systems can be explained fairly simply.

Overview

Vision in the animal world is unbelievably diverse. The compound eyes of insects have a resolution of a few cycles per degree (cpd), whereas eagle resolution is almost a hundred times greater at almost 200 cpd. Insects frequently have colour vision and some have ultra-violet sensitivity, whereas many mammals have weak colour vision and few are trichromats like man. Most mammals are dichromats (Ahnelt & Kolb, 2000). Stereopsis is present in mammals, but not all animals. On the other hand the flicker fusion frequency (§7.5), the frequency at which images blend together as in the cinema, is about three times greater in flies than in man.

In the face of such diversity, what are the useful information processing principles in common? They are as discussed in Chapters 1 and 2: the nature of the physical stimulus and its fundamental limitations; transduction, the conversion of light to an electrical signal; noise and information; and information streaming. These principles are what the designer of robots or virtual reality systems needs to take away. They find use in human computer interaction and the compression of sensory data for optimal trade-offs between bandwidth and perceptual quality. This chapter deals with the overall architecture, later chapters look at colour and object properties, movement and binocular vision. So the plan is to start right at the optics of the eye and track upwards as visual data is streamed into the brain.