1. SECTION 1 considers the elementary case of conditioning on a map that takes only finitely many different values, as motivation for the general definition.

2. SECTION 2 defines conditional probability distributions for conditioning on the value of a general measurable map.

3. SECTION 3 discusses existence of conditional distributions by means of a slightly more general concept, disintegration, which is essential for the understanding of general conditional densities.

4. SECTION 4 defines conditional densities. It develops the general analog of the elementary formula for a conditional density: (joint density)/(marginal density).

5. SECTION *5 illustrates how conditional distributions can be identified by symmetry considerations. The classical Borel paradox is presented as a warning against the misuse of symmetry.

6. SECTION 6 discusses the abstract Kolmogorov conditional expectation, explaining why it is natural to take the conditioning information to be a sub-sigma-field.

7. SECTION *7 discusses the statistical concept of sufficiency.

Conditional distributions: the elementary case

In introductory probability courses, conditional probabilities of events are defined as ratios, ℙ(A∣B) = ℙAB/ℙB, provided ℙB ≠ 0. The division by ℙB ensures that ℙ(· ∣ B) is also a probability measure, which puts zero mass outside the set B, that is, ℙ(Bc ∣ B) = 0. The conditional expectation of a random variable X is defined as its expectation with respect to ℙ(· ∣ B), or, more succinctly, ℙ(X ∣ B) = ℙ(XB)/ℙB. If ℙB = 0, the conditional probabilities and conditional expectations are either left undefined or are extracted by some heuristic limiting argument.

1. SECTION 1 defines the concepts of weak convergence for sequences of probability measures on a metric space, and of convergence in distribution for sequences of random elements of a metric space and derives some of their consequences. Several equivalent definitions for weak convergence are noted.

2. SECTION 2 establishes several more equivalences for weak convergence of probability measures on the real line, then derives some central limit theorems for sums of independent random variables by means of Lindeberg's substitution method.

3. SECTION 3 explains why the multivariate analogs of the methods from Section 2 are not often explicitly applied.

4. SECTION 4 develops the calculus of stochastic order symbols.

5. SECTION *5 derives conditions under which sequences of probability measures have weakly convergent subsequences.

Definition and consequences

Roughly speaking, central limit theorems give conditions under which sums of random variable have approximate normal distributions. For example:

The traditional way to formalize approximate normality requires, for each real x, that ℙ{∑i ξi ≤ x) ≈ ℙ{Z ≤ x} where Z has a N(0, 1) distribution. Of course the variable Z is used just as a convenient way to describe a calculation with the N(0, 1) probability measure; Z could be replaced by any other random variable with the same distribution.

1. SECTION 1 explains why the traditional split of introductory probability courses into two segments—the study of discrete distributions, and the study of “continuous” distributions—is unnecessary in a measure theoretic treatment. Absolute continuity of one measure with respect to another measure is defined. A simple case of the Radon-Nikodym theorem is proved.

2. SECTION *2 establishes the Lebesgue decomposition of a measure into parts absolutely continuous and singular with respect to another measure, a result that includes the Radon-Nikodym theorem as a particular case.

3. SECTION 3 shows how densities enter into the definitions of various distances between measures.

4. SECTION 4 explains the connection between the classical concept of absolute continuity and its measure theoretic generalization. Part of the Fundamental Theorem of Calculus is deduced from the Radon-Nikodym theorem.

5. SECTION *5 establishes the Vitali covering lemma, the key to the identification of derivatives as densities.

6. SECTION *6 presents the proof of the other part of the Fundamental Theorem of Calculus, showing that absolutely continuous functions (on the real line) are Lebesgue integrals of their derivatives, which exist almost everywhere.

Densities and absolute continuity

Nonnegative measurable functions create new measures from old.

Let (X, A, µ) be a measure space, and let Δ(·) be a function in M+(X, A). The increasing, linear functional defined on M+(X, A) by vf ≔ µ(fΔ) inherits from µ the Monotone Convergence property, which identifies it as an integral with respect to a measure on A.

1. SECTION 1 introduces independence as a property that justifies some sort of factorization of probabilities or expectations. A key factorization Theorem is stated, with proof deferred to the next Section, as motivation for the measure theoretic approach. The Theorem is illustrated by a derivation of a simple form of the strong law of large numbers, under an assumption of bounded fourth moments.

2. SECTION 2 formally defines independence as a property of sigma-fields. The key Theorem from Section 1 is used as motivation for the introduction of a few standard techniques for dealing with independence. Product sigma-fields are defined.

3. SECTION 3 describes a method for constructing measures on product spaces, starting from a family of kernels.

4. SECTION 4 specializes the results from Section 3 to define product measures. The Tonelli and Fubini theorems are deduced. Several important applications are presented.

5. SECTION *5 discusses some difficulties encountered in extending the results of Sections 3 and 4 when the measures are not sigma-finite.

6. SECTION 6 introduces a blocking technique to refine the proof of the strong law of large numbers from Section 1, to get a version that requires only a second moment condition.

7. SECTION *7 introduces a truncation technique to further refine the proof of the strong law of large numbers, to get a version that requires only a first moment condition for identically distributed summands.

8. SECTION *8 discusses the construction of probability measures on products of countably many spaces.

1. SECTION 1 explains why you will not learn from this Chapter everything there is to know about the multivariate normal distribution.

2. SECTION 2 introduces Fernique's inequality. As illustration, Sudakov's lower bound for the expected value of a maximum of correlated normals is derived.

3. SECTION *3 proves Fernique's inequality.

4. SECTION 4 introduces the Gaussian isoperimetric inequlity. As an application, Borell's tail bound for the distribution of the maximum of correlated normals is derived.

5. SECTION *5 proves the Gaussian isoperimetric inequlity.

Introduction

Of all the probability distributions on multidimensional Euclidean spaces the multivariate normal is the most studied and, in many ways, the most tractable. In years past, the statistical subject known as “Multivariate Analysis” was almost entirely devoted to the study of the multivariate normal. The literature on Gaussian processes—stochastic processes whose finite dimensional distributions are all multivariate normal—is vast. It is important to know a little about the multivariate normal.

As you saw in Section 8.6, the multivariate normal is uniquely determined by its vector of means and its matrix of covariances. In principle, everything that one might want to know about the distribution can be determined by calculation of means and covariances, but in practice it is not completely straightforward. In this Chapter you will see two elegant examples of what can be achieved: Fernique's (1975) inequality, which deduces important information about the spread in a multivariate normal distribution from its covariances; and Borell's (1975) Gaussian isoperimetric inequality, with a proof due to Ehrhard (1983a, 1983b).

1. SECTION 1 gives some examples of martingales, submartingales, and supermartingales.

2. SECTION 2 introduces stopping times and the sigma-fields corresponding to “information available at a random time.” A most important Stopping Time Lemma is proved, extending the martingale properties to processes evaluted at stopping times.

3. SECTION 3 shows that positive supermartingales converge almost surely.

4. SECTION 4 presents a condition under which a submartingale can be written as a difference between a positive martingale and a positive supermartingale (the Krickeberg decomposition). A limit theorem for submartingales then follows.

5. SECTION *5 proves the Krickeberg decomposition.

6. SECTION *6 defines uniform integrability and shows how uniformly integrable martingales are particularly well behaved.

7. SECTION *7 show that martingale theory works just as well when time is reversed.

8. SECTION *8 uses reverse martingale theory to study exchangeable probability measures on infinite product spaces. The de Finetti representation and the Hewitt-Savage zero-one law are proved.

What are they?

The theory of martingales (and submartingales and supermartingales and other related concepts) has had a profound effect on modern probability theory. Whole branches of probability, such as stochastic calculus, rest on martingale foundations. The theory is elegant and powerful: amazing consequences flow from an innocuous assumption regarding conditional expectations. Every serious user of probability needs to know at least the rudiments of martingale theory.

A little notation goes a long way in martingale theory. A fixed probability space (Ω, ℱ, ℙ) sits in the background.