In most of the models in this book, the observed data are treated as continuous. Hence, for generative models the data likelihood is usually based on the normal distribution. In this chapter, we explore generative models that treat the observed data as discrete. The data likelihoods are now based on the categorical distribution; they describe the probability of observing the different possible values of the discrete variable.

As a motivating example for the models in this chapter, consider the problem of scene classification (Figure 20.1). We are given example training images of different scene categories (e.g., office, coastline, forest, mountain) and we are asked to learn a model that can classify new examples. Studying the scenes in Figure 20.1 demonstrates how challenging a problem this is. Different images of the same scene may have very little in common with one another, yet we must somehow learn to identify them as the same. In this chapter, we will also discuss object recognition, which has many of the same characteristics; the appearance of an object such as a tree, bicycle, or chair can vary dramatically from one image to another, and we must somehow capture this variation.

The key to modeling these complex scenes is to encode the image as a collection of visual words, and use the frequencies with which these words occur as the substrate for further calculations. We start this chapter by describing this transformation.

In this chapter we discuss a family of models that explain observed data in terms of several underlying causes. These causes can be divided into three types: the identity of the object, the style in which it is observed, and the remaining variation.

To motivate these models, consider face recognition. For a facial image, the identity of the face (i.e., whose face it is) obviously influences the observed data. However, the style in which the face is viewed is also important. The pose, expression, and illumination are all style elements that might be modeled. Unfortunately, many other things also contribute to the final observed data: the person may have applied cosmetics, put on glasses, grown a beard, or dyed his or her hair. These myriad contributory elements are usually too difficult to model and are hence explained with a generic noise term.

In face recognition tasks, our goal is to infer whether the identities of face images are the same or different. For example, in face verification, we aim to infer a binary variable ω ϵ {0;1}, where ω=0 indicates that the identities differ and ω=1 indicates that they are the same. This task is extremely challenging when there are large changes in pose, illumination, or expression; the change in the image due to style may dwarf the change due to identity (Figure 18.1).

The models in this chapter are generative, so the focus is on building separate density models over the observed image data cases where the faces do and don't have the same identity.

I was very pleased to be asked to write this foreword, having seen snapshots of the development of this book since its inception. I write this having just returned from BMVC 2011, where I found that others had seen draft copies, and where I heard comments like “What amazing figures!”, “It's so comprehensive!”, and “He's so Bayesian!”.

But I don't want you to read this book just because it has amazing figures and provides new insights into vision algorithms of every kind, or even because it's “Bayesian” (although more on that later). I want you to read it because it makes clear the most important distinction in computer vision research: the difference between “model” and “algorithm.” This is akin to the distinction that Marr made with his three-level computational theory, but Prince's two-level distinction is made beautifully clear by his use of the language of probability.

Why is this distinction so important? Well, let us look at one of the oldest and apparently easiest problems in vision: separating an image into “figure” and “ground.” It is still common to hear students new to vision address this problem just as the early vision researchers did, by reciting an algorithm: first I'll use PCA to find the dominant color axis, then I'll generate a grayscale image, then I'll threshold that at some value, then I'll clean up the holes using morphological operators.

This chapter concerns models for 2D and 3D shape. The motivation for shape models is twofold. First, we may wish to identify exactly which pixels in the scene belong to a given object. One approach to this segmentation problem, is to model the outer contour of the object (i.e., the shape) explicitly. Second, the shape may provide information about the identity or other characteristics of the object: it can be used as an intermediate representation for inferring higher-level properties.

Unfortunately, modeling the shape of an object is challenging; we must account for deformations of the object, the possible absence of some parts of the object and even changes in the object topology. Furthermore, the object may be partially occluded, making it difficult to relate the shape model to the observed data.

One possible approach to establishing 2D object shape is to use a bottom-up approach; here, a set of boundary fragments are identified using an edge detector (Section 13.2.1) and the goal is to connect these fragments to form a coherent object contour. Unfortunately, achieving this goal has proved surprisingly elusive. In practice, the edge detector finds extraneous edge fragments that are not part of the object contour and misses others that are part of the true contour. Hence it is difficult to connect the edge fragments in a way that correctly reconstructs the contour of an object.

In the final part of this book, we discuss four families of models. There is very little new theoretical material; these models are straight applications of the learning and inference techniques introduced in the first nine chapters. Nonetheless, this material addresses some of the most important machine vision applications: shape modeling, face recognition, tracking, and object recognition.

In Chapter 17 we discuss models that characterize the shape of objects. This is a useful goal in itself as knowledge of shape can help localize or segment an object. Furthermore, shape models can be used in combination with models for the RGB values to provide a more accurate generative account of the observed data.

In Chapter 18 we investigate models that distinguish between the identities of objects and the style in which they are observed; a prototypical example of this type of application would be face recognition. Here the goal is to build a generative model of the data that can separate critical information about identity from the irrelevant image changes due to pose, expression and lighting.

In Chapter 19 we discuss a family of models for tracking visual objects through time sequences. These are essentially graphical models based on chains such as those discussed in Chapter 11. However, there are two main differences. First, we focus here on the case where the unknown variable is continuous rather than discrete. Second, we do not usually have the benefit of observing the full sequence; we must make a decision at each time based on information from only the past.

Mathematical explanation is a hot topic in current work in the philosophy of mathematics. We have already seen one reason for this: the close connection between the indispensability argument for mathematical realism and the scientific realist's reliance on inference to the best explanation. This connection is even tighter if it can be established that there are mathematical explanations of empirical phenomena. As a result, a great deal of recent work on realism-anti-realism issues in mathematics has focused on mathematical explanations in science. Irrespective of such issues, the question of mathematical explanation is important in its own right and deserves closer attention.

We start by making a distinction between two different senses of mathematical explanation. The first we call intra-mathematical explanations. These are mathematical explanations of mathematical facts. Such explanations can take the form of an explanatory proof – a proof that tells us why the theorem in question is true – or perhaps a recasting of the mathematical fact in question in terms of another area of mathematics. There is also the issue of whether mathematics can explain empirical facts. Call this extra-mathematical explanation. A full account of mathematical explanation will provide both a philosophically satisfying account of intra-mathematical explanation and an account that coheres with our account of explanation elsewhere in science.

You know the old question about which 20 books, 20 albums, 20 movies, or whatever you'd like to have with you if you were stranded on a desert island? Well, in this chapter I'll give you my top 20 mathematical theorems for desert island-bound philosophers. We look at a number of mathematical results that have some philosophical interest, or in some cases are just very cool pieces of mathematics. (Alternatively, you might think of this chapter as 20 theorems you should come to terms with before you die.) Of course, this is just my top 20 theorems. If you don't like my choices, feel free to construct your own list. For good measure I throw in a few famous open problems and interesting numbers to round out my desert-island survival kit.

Philosophers' favourites

The theorems in this section are well known by philosophers and rightly get a great deal of attention in philosophical circles. These are the obvious choices for desert island theorems, but in some cases you'd be disappointed to be stuck with just these. You wouldn't be disappointed because they are uninteresting or trivial; you'd be disappointed because they are just a bit too obvious. Everybody would have these! In any case, the theorems below are the classics – the obvious ones that almost anyone would put high on their list. (These are the Citizen Kanes and Vertigos of the maths world.)

In the last chapter we saw one of the main cases for Platonism, namely, the indispensability argument. In this chapter we look at a few anti-realist philosophies of mathematics. Each of these positions can be understood as a response to the indispensability argument. They are also motivated by the Benacerraf epistemic challenge to Platonism and the hope that it's easier to be rid of troublesome mathematical entities than it is to provide a Platonist epistemology.

Fictionalism

Fictionalism in the philosophy of mathematics is the view that mathematical statements, such as ‘7+5 = 12’ and ‘πis irrational’, are to be interpreted at face value and, thus interpreted, are false. Fictionalists are typically driven to reject the truth of such mathematical statements because these statements imply the existence of mathematical entities, and, according to fictionalists, there are no such entities. Fictionalism is a nominalist (or antirealist) account of mathematics in that it denies the existence of a realm of abstract mathematical entities. It should be contrasted with mathematical realism (or Platonism), where mathematical statements are taken to be true and, moreover, are taken to be truths about mathematical entities. Fictionalism should also be contrasted with other nominalist philosophical accounts of mathematics that propose a reinterpretation of mathematical statements, according to which the statements in question are true but no longer about mathematical entities.

Most mathematics is concerned with proving theorems, developing new mathematical theories, and finding axioms for theories. But there are very important questions about the mathematical theories themselves. For example, it would be nice to know whether a particular mathematical theory is consistent. That is, we'd like to be able to prove that the mathematical theory in question will not deliver a contradiction, as naïve set theory did. We'd also like to know whether a mathematical theory is capable of answering any question thrown at it. That is, we'd like to be able to show that for any mathematical statement of the theory, either we can prove it or prove its negation. This is called completeness. The study of such higher-level questions about mathematics is known as Metamathematics and can be thought of as the mathematical study of mathematics. Not surprising, this is an area of great interest for philosophers, especially in light of a number of key results that place limitation on what mathematics can do. These results are intriguing (and often surprising) in their own right, but they are also supposed to have consequences for philosophy of mathematics and beyond – to areas such as philosophy of mind and metaphysics. In this chapter we consider some of these results and discuss their significance for philosophy.

One often hears the claim that mathematics is ‘the language of science’. This is meant as a compliment to mathematics. But mathematics is not the language of science in the way that French is the language of love. The latter is surely conventional, perhaps driven by aesthetic preferences for ‘amour’ over ‘love’ and ‘belle’ over ‘beautiful’ and the like. In any case, mathematics, if it is the language of science, is not like this. It's not as though science looks or sounds sexier when it's written mathematically (actually, perhaps it does, but that's by the by). The point of the slogan is to emphasise that a great deal of science – especially physics, but many other branches of science as well – is typically highly mathematical. Moreover, a great deal of science could not even be formulated without mathematics.

In this chapter I will argue that although there is undoubtedly something right about the view of mathematics as the language of science, it seriously undersells mathematics. To think of mathematics as merely the language of science fails to appreciate the variety of roles mathematics plays in many diverse branches of science. Thinking of mathematics as a language is useful in appreciating the significance of, and the difficulties encountered in developing, a good notational system. Good notation is far from trivial. So let’s start by looking at some of the benefits of good notation. Along the way, we will see the role good notation can play in prompting new ideas and new developments in mathematics and science.

Mathematical realism or Platonism is the philosophical position that mathematical statements such as ‘there are infinitely many prime numbers’ are true and that these statements are true by virtue of the existence of mathematical objects – prime numbers, in this case. That all seems fine until you think about the nature of the objects being posited. Where are these numbers? What are they like? How do we know about them? What about all the other mathematical objects: sets, functions, Hilbert spaces, and the like? Do all these exist as well? Are all mathematical objects made up of the same basic ingredients – sets, perhaps – or are they each a distinct kind of thing? Are these mathematical objects abstract or do they have causal powers and space-time locations? In any case, what is their relationship to the physical world? And most difficult of all: if mathematical knowledge is knowledge of these mathematical entities how do we come by such knowledge? Negotiating a set of answers to these questions, unsurprisingly, leads to a variety of different realist positions. In this chapter we will very briefly consider a few of the realist positions on offer, before looking in more detail at an influential argument for mathematical realism.

Mathematics occupies a unique and privileged position in human inquiry. It is the most rigorous and certain of all of the sciences, and it plays a key role in most, if not all, scientific work. It is for such reasons that the great German mathematician Carl Friedrich Gauss (1777–1855) pronounced mathematics to be the queen of the sciences. But the subject matter of mathematics is unlike that of any of the other branches of science. Mathematics seems to be the study of mathematical entities – such as numbers, sets, and functions – and the structural relationships between them. Mathematical entities, if there are such things, are very peculiar. They are abstract: they do not have spatiotemporal location and do not have causal powers. Moreover, the methodology of mathematics is apparently unlike the methodology of other sciences. Mathematics seems to proceed via a-priori means using deductive proof, as opposed to the a-posteriori methods of experimentation and induction found in the rest of science. And, on the face of it at least, mathematics is not revisable in the way that the rest of our science is. Once a mathematical theorem is proven, it stands forever. Mathematics may well be the queen of the sciences, but she would seem to be an eccentric and obstinate queen.