1. SECTION 1 offers some reasons for why anyone who uses probability should know about the measure theoretic approach.

2. SECTION 2 describes some of the added complications, and some of the compensating benefits that come with the rigorous treatment of probabilities as measures.

3. SECTION 3 argues that there are advantages in approaching the study of probability theory via expectations, interpreted as linear functionals, as the basic concept.

4. SECTION 4 describes the de Finetti convention of identifying a set with its indicator function, and of using the same symbol for a probability measure and its corresponding expectation.

5. SECTION *5 presents a fair-price interpretation of probability, which emphasizes the linearity properties of expectations. The interpretation is sometimes a useful guide to intuition.

Why bother with measure theory?

Following the appearance of the little book by Kolmogorov (1933), which set forth a measure theoretic foundation for probability theory, it has been widely accepted that probabilities should be studied as special sorts of measures. (More or less true—see the Notes to the Chapter.) Anyone who wants to understand modern probability theory will have to learn something about measures and integrals, but it takes surprisingly little to get started.

For a rigorous treatment of probability, the measure theoretic approach is a vast improvement over the arguments usually presented in undergraduate courses. Let me remind you of some difficulties with the typical introduction to probability.

Independence

There are various elementary definitions of independence for random variables.

1. SECTION 1 defines measures and sigma-fields.

2. SECTION 2 defines measurable functions.

3. SECTION 3 defines the integral with respect to a measure as a linear functional on a cone of measurable functions. The definition sidesteps the details of the construction of integrals from measures.

4. SECTION *4 constructs integrals of nonnegative measurable functions with respect to a countably additive measure.

5. SECTION 5 establishes the Dominated Convergence theorem, the Swiss Army knife of measure theoretic probability.

6. SECTION 6 collects together a number of simple facts related to sets of measure zero.

7. SECTION *7 presents a few facts about spaces of functions with integrable pth powers, with emphasis on the case p=2, which defines a Hilbert space.

8. SECTION 8 defines uniform integrability, a condition slightly weaker than domination. Convergence in £1 is characterized as convergence in probability plus uniform integrability.

9. SECTION 9 defines the image measure, which includes the concept of the distribution of a random variable as a special case.

10. SECTION 10 explains how generating class arguments, for classes of sets, make measure theory easy.

11. SECTION *11 extends generating class arguments to classes of functions.

Measures and sigma-fields

As promised in Chapter 1, we begin with measures as set functions, then work quickly towards the interpretation of integrals as linear functionals. Once we are past the purely set-theoretic preliminaries, I will start using the de Finetti notation (Section 1.4) in earnest, writing the same symbol for a set and its indicator function.

1. SECTION 1 collects together some facts about stochastic processes and the normal distribution, for easier reference.

2. SECTION 2 defines Brownian motion as a Gaussian process indexed by a subinterval T of the real line. Existence of Brownian motions with and without continuous sample paths is discussed. Wiener measure is defined.

3. SECTION 3 constructs a Brownian motion with continuous sample paths, using an orthogonal series expansion of square integrable functions.

4. SECTION *4 describes some of the finer properties—lack of differentiability, and a modulus of continuity—for Brownian motion sample paths.

5. SECTION 5 establishes the strong Markov property for Brownian motion. Roughly speaking, the process starts afresh as a new Brownian motion after stopping times.

6. SECTION *6 describes a family of martingales that can be built from a Brownian motion, then establishes Lévy's martingale characterization of Brownian motion with continuous sample paths.

7. SECTION *7 shows how square integrable functions of the whole Brownian motion path can be represented as limits of weighted sums of increments. The result is a thinly disguised version of a remarkable property of the isometric stochastic integral, which is mentioned briefly.

8. SECTION *8 explains how the result from Section 7 is the key to the determination of option prices in a popular model for changes in stock prices.

Prerequisites

Broadly speaking, Brownian motion is to stochastic process theory as the normal distribution is to the theory for real random variables.

1. SECTION 1 presents a few of the basic properties of Fourier transforms that make them such a valuable tool of probability theory.

2. SECTION 2 exploits a mysterious coincidence, involving the Fourier transform and the density function of the normal distribution, to establish inversion formulas for recovering distributions from Fourier transforms.

3. SECTION *3 explains why the coincidence from Section 2 is not really so mysterious.

4. SECTION 4 shows that the inversion formula from Section 2 has a continuity property, which explains why pointwise convergence of Fourier transforms implies convergence in distribution.

5. SECTION *5 establishes a central limit theorem for triangular arrays of martingale differences.

6. SECTION 6 extends the theory to multivariate distributions, pointing out how the calculations reduce to one-dimensional analogs for linear combinations of coordinate variables—the Cramér and Wold device.

7. SECTION *7 provides a direct proof (no Fourier theory) of the fact that the family of (one-dimensional) distributions for all linear combinations of a random vector uniquely determines its multivariate distribution.

8. SECTION *8 illustrates the use of complex-variable methods to prove a remarkable property of the normal distribution—the Lévy-Cramér theorem.

Definitions and basic properties

Some probabilistic calculations simplify when reexpressed in terms of suitable transformations, such as the probability generating function (especially for random variables taking only positive integer values), the Laplace transform (especially for random variables taking only nonnegative values), or the moment generating function (for random variables with rapidly decreasing tail probabilities).

This book began life as a set of handwritten notes, distributed to students in my one-semester graduate course on probability theory, a course that had humble aims: to help the students understand results such as the strong law of large numbers, the central limit theorem, conditioning, and some martingale theory. Along the way they could expect to learn a little measure theory and maybe even a smattering of functional analysis, but not as much as they would learn from a course on Measure Theory or Functional Analysis.

In recent years the audience has consisted mainly of graduate students in statistics and economics, most of whom have not studied measure theory. Most of them have no intention of studying measure theory systematically, or of becoming professional probabilists, but they do want to learn some rigorous probability theory—in one semester.

Faced with the reality of an audience that might have neither the time nor the inclination to devote itself completely to my favorite subject, I sought to compress the essentials into a course as self-contained as I could make it. I tried to pack into the first few weeks of the semester a crash course in measure theory, with supplementary exercises and a whirlwind exposition (Appendix A) for the enthusiasts. I tried to eliminate duplication of mathematical effort if it served no useful role. After many years of chopping and compressing, the material that I most wanted to cover all fit into a one-semester course, divided into 25 lectures, each lasting from 60 to 75 minutes.

1. SECTION 1 illustrates the usefulness of coupling, by means of three simple examples.

2. SECTION 2 describes how sequences of random elements of separable metric spaces that converge in distribution can be represented by sequences that converge almost surely.

3. SECTION *3 establishes Strassen's Theorem, which translates the Prohorov distance between two probability measures into a coupling.

4. SECTION *4 establishes Yurinskii's coupling for sums of independent random vectors to normally distributed random vectors.

5. SECTION 5 describes a deceptively simple example (Tusnády's Lemma) of a quantile coupling, between a symmetric Binomial distribution and its corresponding normal approximation.

6. SECTION 6 uses the Tusnády Lemma to couple the Haar coefficients for the expansions of an empirical process and a generalized Brownian Bridge.

7. SECTION 7 derives one of most striking results of modern probability theory, the KMT coupling of the uniform empirial process with the Brownian Bridge process.

What is coupling?

A coupling of two probability measures, P and Q, consists of a probability space (Ω, ℱ, ℙ) supporting two random elements X and F, such that X has distribution P and Y has distribution Q. Sometimes interesting relationships between P and Q can be coded in some simple way into the joint distribution for X and Y. Three examples should make the concept clearer.

Example. Let Pα denote the Bin(n,α) distribution. As α gets larger, the distribution should “concentrate on bigger values.” More precisely, for each fixed x, the tail probability Pα[x, n] should be an increasing function of α. A coupling argument will give an easy proof.