This chapter transitions from the physiology of human vision to perception. How do our brains interpret the world around us so effectively in spite of our limited biological hardware? To understand how we may be fooled by visual stimuli presented by a display, you must first understand how we perceive or interpret the real world under normal circumstances. It is not always clear what we will perceive. We have already seen several optical illusions. VR itself can be considered as a grand optical illusion. Under what conditions will it succeed or fail? Section 6.1 covers perception of the distance of objects from our eyes, which is also related to the perception of object scale. Section 6.2 explains how we perceive motion. An important part of this is the illusion of motion that we perceive from videos, which are merely a sequence of pictures. Section 6.3 covers the perception of color, which may help explain why displays use only three colors (red, green, and blue) to simulate the entire spectral power distribution of light. Finally, Section 6.4 presents a statistically based model of how information is combined from multiple sources to produce a perceptual experience.

This chapter introduces interaction mechanisms that may not have a counterpart in the physical world. Section 10.1 introduces general motor learning and control concepts. The most important concept is remapping, in which a motion in the real world may be mapped into a substantially different motion in the virtual world. This enables many powerful interaction mechanisms. The task is to develop ones that are easy to learn, easy to use, effective for the task, and provide a comfortable user experience. Section 10.2 discusses how the user may move himself in the virtual world, while remaining fixed in the real world. Section 10.3 presents ways in which the user may interact with other objects in the virtual world. Section 10.4 discusses social interaction mechanisms, which allow users to interact directly with each other. Section 10.5 briefly considers some additional interaction mechanisms, such as editing text, designing 3D structures, and Web browsing.

If you ask the average person what “magnetism” is, you will probably be told about refrigerator decorations, compass needles, and the North Pole – none of which has any obvious connection with moving charges or current-carrying wires. And yet, in classical electrodynamics all magnetic phenomena are due to electric charges in motion; if you could examine a piece of magnetic material on an atomic scale you would find tiny currents: electrons orbiting around nuclei and spinning about their axes.

In the real world, audio is crucial to art, entertainment, and oral communication. Audio recording and reproduction can be considered a VR experience by itself, with both a CAVE-like version (surround sound) and a headset version (wearing headphones). When combined consistently with the visual component, audio helps provide a compelling and comfortable VR experience. Each section of this chapter is the auditory (or audio) complement to one of Chapters 4 through 7. The progression again goes from physics to physiology, and then from perception to rendering. Section 11.1 explains the physics of sound in terms of waves, propagation, and frequency analysis. Section 11.2 describes the parts of the human ear and their function. This naturally leads to auditory perception, which is the subject of Section 11.3. Section 11.4 concludes by presenting auditory rendering, which can produce sounds synthetically from models or reproduce captured sounds.

In this chapter we study conservation of energy, momentum, and angular momentum, in electrodynamics. But I want to begin by reviewing the conservation of charge, because it is the paradigm for all conservation laws. What precisely does conservation of charge tell us? That the total charge in the Universe is constant? Well, sure – that’s global conservation of charge. But local conservation of charge is a much stronger statement: if the charge in some region changes, then exactly that amount of charge must have passed in or out through the surface. The tiger can’t simply rematerialize outside the cage; if it got from inside to outside it must have slipped through a hole in the fence.

What is a “wave”? I don’t think I can give you an entirely satisfactory answer – the concept is intrinsically somewhat vague – but here’s a start: A wave is a disturbance of a continuous medium that propagates with a fixed shape at constant velocity. Immediately I must add qualifiers: in the presence of absorption, the wave will diminish in size as it moves; if the medium is dispersive, different frequencies travel at different speeds; in two or three dimensions, as the wave spreads out, its amplitude will decrease; and of course standing waves don’t propagate at all. But these are refinements; let’s start with the simple case: fixed shape, constant speed, one dimension (Fig. 9.1).

We now want to model motions more accurately because the physics of both real and virtual worlds impact VR experiences. The accelerations and velocities of moving bodies impact simulations in the VWG and tracking methods used to capture user motions in the physical world. Section 8.1 introduces fundamental concepts from math and physics, including velocities, accelerations, and the movement of rigid bodies. Section 8.2 presents the physiology and perceptual issues from the human vestibular system, which senses velocities and accelerations. Section 8.3 then describes how motions are described and produced in a VWG. This includes numerical integration and collision detection. Section 8.4 focuses on vection, which is a source of VR sickness that arises due to sensory conflict between the visual and vestibular systems: the eyes may perceive motion while the vestibular system is not fooled. This can be considered as competition between the physics of the real and virtual worlds.

If you walk 4 miles due north and then 3 miles due east (Fig. 1.1), you will have gone a total of 7 miles, but you’re not 7 miles from where you set out – only 5. We need an arithmetic to describe quantities like this, which evidently do not add in the ordinary way. The reason they don’t, of course, is that displacements (straight line segments going from one point to another) have direction as well as magnitude (length), and it is essential to take both into account when you combine them.

In this chapter, we shall study electric fields in matter. Matter, of course, comes in many varieties – solids, liquids, gases, metals, woods, glasses – and these substances do not all respond in the same way to electrostatic fields. Nevertheless, most everyday objects belong (at least, in good approximation) to one of two large classes: conductors and insulators (or dielectrics).