Anyone who has had to move about in the dark recognizes the importance of vision to human navigation. Tasks that are fraught with difficulty and danger in the dark become straightforward when the lights are on. Given that humans seem to navigate effortlessly with vision, it seems natural to consider vision as a sensor for mobile robots. Visual sensing has many desirable potential features, including that it is passive and has high resolution and a long range.

There are many exercises included at the ends of chapters in Parts I and II of book. This appendix provides brief solutions or at least answers to most of these exercises.

We begin our journey into state estimation by considering systems that can be modelled using linear equations corrupted by Gaussian noise. While these linear-Gaussian systems are severe approximations of real robots, the mathematics are very amenable to straightforward analysis. We discuss the difference between Bayesian estimation and maximum a posteriori estimation in the context of batch trajectory estimation; these two approaches are effectively the same for linear systems, but this contrast is crucial to understanding the results for nonlinear systems later on. After introducing batch trajectory estimation, we show how the structure of the problem gives rise to sparsity in our equations that can be exploited to provide a very efficient solution. Indeed, the famous Rauch-Tung-Striebel smoother (whose forward pass is the Kalman filter) is equivalent to solving the batch trajectory problem. Several other avenues to the Kalman filter are also explored. Although much of the book focusses on discrete-time motion models for robots, we show how to begin with continuous-time models as well; in particular, we make the connection that batch continuous-time trajectory is an example of Gaussian process regression, a popular tool from machine learning.

This appendix contains a few extra derivations relating to rotations and poses that may be of interest to some enthusiastic readers. In particular, the eigen/Jordan decomposition of rotations and poses provides some deeper insight into these quantities that are ubiquitous in robotics.

Typical robots not only translate in the world but also rotate. This chapter serves a primer on three-dimensional geometry introducing such important geometric concepts as vectors, reference frames, coordinates, rotations, and poses (rotation and translation). We introduce kinematics, how geometry changes over time, with an eye towards describing robot motion models. We also present several common sensor models using our three-dimensional tools: camera, stereo camera, lidar, and inertial measurement unit.

The final technical chapter returns to the idea of representing a robot trajectory as a continuous function of time, only now in three-dimensional space where the robot may translate and rotate. We provide a method to adapt our earlier continuous-time trajectory estimation to Lie groups that is practical and efficient. The chapter serves as a final example of pulling together many of the key ingredients of the book into a single problem: continuous time estimation as Gaussian process regression, Lie groups to handle rotations, and simultaneous localization and mapping.

Nonlinear systems provide additional challenges for robotic state estimation. We provide a derivation of the famous extended Kalman filter (EKF) and then go on to study several generalizations and extensions of recursive estimation that are commonly used: the Bayes filter, the iterated EKF, the particle filter, and the sigmapoint Kalman filter. We return to batch estimation for nonlinear systems, which we connect more deeply to numerical optimization than in the linear-Gaussian chapter. We discuss the strengths and weaknesses of the various techniques presented and then introduce sliding-window filters as a compromise between recursive and batch methods. Finally, we discuss how continuous-time motion models can be employed in batch trajectory estimation for nonlinear systems.

Rotational state variables are a problem for our estimation tools from earlier chapters, which all assumed the state to be estimated was a vector in the sense of linear algebra. Rotations cannot be globally described as vectors and as such must be handled with care. This chapter re-examines rotations as an example of a Lie group, which has many useful properties despite not being a vector space. The main takeaway of the chapter is that in estimation we can use the Lie group structure to adapt our estimation tools to work with rotations, and by association poses. The key is to consider small perturbations to rotations in the group's Lie algebra in order to make two tasks easier to handle: performing numerical optimization and representing uncertainty. The chapter can also serve as a useful reference for readers already familiar with the content.

Following on the heels of the chapter on nonlinear estimation, this chapter focusses on some of the common pitfalls and failure modes of estimation techniques. We begin by discussing some key properties that we would like healthy estimators to have (i.e., unbiased, consistent) and how to measure these properties. We delve more deeply into biases and discuss how in some cases we can fold bias estimation right into our estimator, while in other cases we cannot. We touch briefly on data association (matching measurements to the right parts of models) and how to mitigate the effect of outlier measurements using robust estimation. We close with some methods to determine good measurement covariances for use in our estimators.

This chapter opens with a brief history of estimation from astronomy, navigation at sea, and space exploration. It defines the problem of estimation and gives some modern sensor fusion examples. A description of how the book is organized and how to read it is provided. The book is compared to other great volumes on estimation and robotics in order to understand how it fits into the larger landscape.