Matchmoving, also known as camera tracking, is a major aspect of modern visual effects. It's the key underlying process that allows visual effects artists to convincingly insert computer-generated elements and characters into a live-action plate, so that everything appears to “live in” a consistent three-dimensional world. In every modern action movie (and even many non-action movies), the first step after acquiring live footage is to track the camera to enable the addition of spatially accurate visual effects.

The basic problem is to determine, using a given video sequence as input, the three-dimensional location and orientation of the camera at every frame with respect to landmarks in the scene. Depending on the situation, we may have some prior information – such as estimates of the focal length from the camera's lens barrel or labeled landmarks with known 3D coordinates-or the video may come from an entirely unknown camera and environment.

Matchmoving is fundamentally the same as a computer vision problem called structure from motion. In fact, several of the main matchmoving software packages for visual effects grew directly out of academic research discussed in this chapter. In turn, structure from motion is closely related to photogrammetry, mathematical techniques used by surveyors to estimate the shape of buildings and terrain from multiple images. Many structure from motion techniques “discovered” by computer vision researchers in the 1990s share key steps with photogrammetric techniques developed by cartographers and geodesists in the 1950s or earlier. Finally, structure from motion is closely related to the problem of simultaneous location and mapping or SLAM from robotics, in which a mobile robot must self-localize by taking measurements of its environment.