Although more detailed and formal than the presentation in §1.1, this appendix does not claim to provide a complete, rigorous presentation of axiomatic set theory (there are several entire books devoted to the subject, some of them listed in the references). Although axioms for set theory will be stated in detail, some definitions, such as linear ordering and well-ordering, will be assumed to be known (from Chapter 1).

Mathematical Logic

Around 300 b.c., Euclid's geometry presented “a strictly logical deduction of theorems from a set of definitions, postulates and axioms” (Struik, 1948, p. 59). Euclid went a long way, although not all the way, to the modern ideal of the axiomatic method, where, when the proof of a theorem is written out in detail, it can be checked mechanically and precisely to ascertain that it is (or is not) a proof. From a modern point of view, perhaps the least strictly logical part of Euclid's system is his definitions—for example, “a point is that which has no extension,” “a line is a length, without width …” As was noted in §1.1, a truly precise mathematical system, or ‘formal system’, begins with some basic undefined terms. Then other terms can be defined from the basic ones.

The most widely accepted formal systems, giving a foundation for modern mathematics, are based on propositional calculus and first-order predicate logic. Only a very brief introduction to these topics will be given here. For more details see, for example, Kleene (1967).

So far we have dealt with convergence of laws mainly on finite-dimensional Euclidean spaces ℝk, for the central limit theorem (§§9.3–9.5). Now we'll treat converging laws on more general, possibly infinite-dimensional spaces. Here are some cases where such spaces and laws can be helpful.

Let x(t, Ω) be the position of a randomly moving particle at time t, where Ω ∈ Ω, for some probability space (Ω, , P). For each Ω, we then have a continuous function t↦ x(t, Ω). Suppose we consider times t with 0 ≤ t ≤ 1 and that x is real-valued (the particle is moving along a line, or we just consider one coordinate of its position). Then x(·, Ω) belongs to the space C[0, 1] of continuous real-valued functions on [0, 1]. The space C[0, 1] has a usual norm, the supremum norm |f| ≔, sup|f(t)|: 0 < t < 1. Then C[0, 1] is a complete separable metric space for the metric d defined as usual by d(f, g) ≔, |f – g|. It may be useful to approximate the process x, for example, by a process yn such that for each Ω and each k = 1, …, n, yn (·, Ω) is linear on the interval [(k – 1)/n, k/n]. Thus it may help to define yn converging to x in law (or in probability or a.s.) in C[0, 1].

Introduction to Measures

A classical example of measure is the length of intervals. In the modern theory of measure, developed by Émile Borel and Henri Lebesgue around 1900, the first task is to extend the notion of “length” to very general subsets of the real line. In representing intervals as finite, disjoint unions of other intervals, it is convenient to use left open, right closed intervals. The length is denoted by λ((a, b]) ≔ b – a for a ≤ b. Now, in the extended real number system [−∞, ∞] ≔ {−∞} ∪ ℝ ∪ {+∞}, −∞ and +∞ are two objects that are not real numbers. Often +∞ is written simply as ∞. The linear ordering of real numbers is extended by setting −∞ < x < ∞ for any real number x. Convergence to ±∞ will be for the interval topology, as defined in §2.2; for example, xn → +∞ iff for any K < ∞ there is an m with xn > K for all n < m. If a sequence or series of real numbers is called convergent, however, and the limit is not specified, then the limit is supposed to be in ℝ, not ±∞. For any real x, x + (–∞) ≔, −∞ and x + ∞ ≔, +∞, while ∞ – ∞, or ∞ + (−∞), is undefined, although of course it may happen that an → + ∞ and bn → −∞ while an + bn approaches a finite limit.

Stochastic processes have been treated so far mainly in connection with martingales, although a general definition was given: a stochastic process is a function X of two variables t and Ω, t ∈ T, Ω ∈ Ω, where (Ω, , P) is a probability space and for each t, X(t,·) is measurable on Ω. Taking T to be the set of positive integers, any sequence of random variables is a stochastic process. In much of the more classical theory of processes, T is a subset of the real line. But by the 1950s, if not before, it began to be realized that there are highly irregular random processes, useful in representing or approximating “noise,” for example, which are in a sense defined over the line but which do not have values at points t. Instead, “integrals” W(f)= ∫ W(t)f(t)dt are defined only if f has some smoothness and/or other regularity properties. Thus an appropriate index set T for the process may be a set of functions on ℝ rather than a subset of ℝ. Such processes are also useful where we may have random functions not only changing in time but defined also on space, so that T may be a set of smooth functions of space as well as, or instead of, time variables. At any rate, the beginnings of the theory of stochastic processes, and a basic existence theorem, hold for an arbitrary index set T without any structure.

Nearly every measure used in mathematics is defined on a space where there is also a topology such that the domain of the measure is either the Borel σ-algebra generated by the topology, its completion for the measure, or perhaps an intermediate σ-algebra. Defining the integrals of real-valued functions on a measure space did not involve any topology as such on the domain space, although structures on the range space ℝ (order as well as topology) were used. Section 7.1 will explore relations between measures and topologies.

The derivative of one measure with respect to another, dγ/dμ = f, is defined by the Radon-Nikodym theorem (§5.5) in case γ is absolutely continuous with respect to μ. A natural question is whether differentiation is valid in the sense that then γ(A)/μ(A) converges to f(x) as the set A shrinks down to x. For this, it is clearly not enough that the sets A contain x and their measures approach 0, as most of the sets might be far from x. One would expect that the sets should be included in neighborhoods of x forming a filter base converging to x. In ℝ, for the usual differentiation, the sets A are intervals, usually with an endpoint at x. It turns out that it is not enough for the sets A to converge to x.

Functional analysis is concerned with infinite-dimensional linear spaces, such as Banach spaces and Hilbert spaces, which most often consist of functions or equivalence classes of functions. Each Banach space X has a dual space X′ defined as the set of all continuous linear functions from X into the field ℝ or ℂ.

One of the main examples of duality is for Lp spaces. Let (X, S, μ) be a measure space. Let 1 < p < ∞ and 1/p + 1/q = 1. Then it turns out that Lp and Lq are dual to each other via the linear functional f ↦ ∫ f g d μ for f in p and g in q. For p = q = 2, L2 is a Hilbert space, where it was shown previously that any continuous linear form on a Hilbert space H is given by inner product with a fixed element of H (Theorem 5.5.1).

Other than linear subspaces, some of the most natural and frequently applied subsets of a vector space S are the convex subsets C, such that for any x and y in C, and 0 < t < 1, we have tx + (1 – t)y ∈ C. These sets are treated in §§6.2 and 6.6. A function for which the region above its graph is convex is called a convex function. §6.3 deals with convex functions. Convex sets and functions are among the main subjects of modern real analysis.

In constructing a building, the builders may well use different techniques and materials to lay the foundation than they use in the rest of the building. Likewise, almost every field of mathematics can be built on a foundation of axiomatic set theory. This foundation is accepted by most logicians and mathematicians concerned with foundations, but only a minority of mathematicians have the time or inclination to learn axiomatic set theory in detail.

To make another analogy, higher-level computer languages and programs written in them are built on a foundation of computer hardware and systems programs. How much the people who write high-level programs need to know about the hardware and operating systems will depend on the problem at hand.

In modern real analysis, set-theoretic questions are somewhat more to the fore than they are in most work in algebra, complex analysis, geometry, and applied mathematics. A relatively recent line of development in real analysis, “nonstandard analysis,” allows, for example, positive numbers that are infinitely small but not zero. Nonstandard analysis depends even more heavily on the specifics of set theory than earlier developments in real analysis did.

This chapter will give only enough of an introduction to set theory to define some notation and concepts used in the rest of the book. In other words, this chapter presents mainly “naive” (as opposed to axiomatic) set theory.

General topology has to do with, among other things, notions of convergence. Given a sequence xn of points in a set X, convergence of xn to a point x can be defined in different ways. One of the main ways is by a metric, or distance d, which is nonnegative and real-valued, with xn → x meaning d(xn, x) → 0. The usual metric for real numbers is d(x, y) = ∣x - y∣. For the usual convergence of real numbers, a function f is called continuous if whenever xn → x in its domain, we have f(xn) → f(x).

On the other hand, some interesting kinds of convergence are not defined by metrics: if we define convergence of a sequence of functions fn “pointwise,” so that fn → f means fn(x) → f(x) for all x, it turns out that (for a large enough class of functions defined on an uncountable set) there may be no metric e such that fn→ f is equivalent to e(fn, f) → 0.

Given a sense of convergence, we can call a set F closed if whenever xi ∈ F for all i and xi → x we have x ∈ F also. Any closed interval [a, b] ≔ x: a ≤ x ≤ b is an example of a closed set. The properties of closed sets F and their complements U ≔ X∖F, which are called open sets, turn out to provide the best and most accepted way of extending the notions of convergence, continuity, and so forth to nonmetric situations.

Recall that a topological space is called locally compact iff every point in it is contained in some open set with compact closure. For some time, locally compact spaces were considered the most natural spaces on which to study general measures and integrals. The regularity extension, treated in §7.3, appeared as a primary advantage of measures on compact or locally compactspaces. Local compactness also fits well with group structures. A topological group is a group G with a Hausdorff topology  for which the group operations 〈g, h〉 ↦ gh and g ↦ g−1 are continuous. A locally compact Hausdorff topological group has left and right Haar measures, which are Radon measures (finite on compact sets) invariant under all left or right translations g ↦ hg or g ↦ gh, respectively. The theory of measures on locally compact spaces, particularly groups, occupied the last three of the twelve chapters of the classic text of Halmos (1950). Bourbaki (1952–1969) put even more emphasis on locally compact spaces. The main purpose of this appendix is to indicate why locally compact spaces have had less attention in this book than in those of Halmos and Bourbaki.

Another class of spaces for measures theory is complete separable metric spaces, or topological spaces which can be metrized to be separable and complete (Polish spaces). Among such spaces, for example, are separable, infinite-dimensional Banach spaces, none of which is locally compact.

This is a text at the beginning graduate level. Some study of intermediate analysis in Euclidean spaces will provide helpful background, but in this edition such background is not a formal prerequisite. Efforts to make the book more self-contained include inserting material on the real number system into Chapter 1, adding a treatment of the Stone-Weierstrass theorem, and generally eliminating references for proofs to other books except at very few points, such as some complex variable theory in Appendix B.

Chapters 1 through 5 provide a one-semester course in real analysis. Following that, a one-semester course on probability can be based on Chapters 8 through 10 and parts of 11 and 12. Starred paragraphs and sections, such as those found in Chapter 6 and most of Chapter 7, are called on rarely, if at all, later in the book. They can be skipped, at least on first reading, or until needed.

Relatively few proofs of less vital facts have been left to the reader. I would be very glad to know of any substantial unintentional gaps or errors. Although I have worked and checked all the problems and hints, experience suggests that mistakes in problems, and hints that may mislead, are less obvious than errors in the text. So take hints with a grain of salt and perhaps make a first try at the problems without using the hints.

Probabilities are easiest to define on finite sets. For example, consider a toss of a fair coin. Here “fair” means that heads and tails are equally likely. The situation may be represented by a set with two points H and T where H= “heads” and T= “tails.” The total probability of all possible outcomes is set equal to 1. Let “P(…)” denote “the probability of.…” If two possible outcomes cannot both happen, then one assumes that their probabilities add. Thus P(H) + P(T) = 1. By assumption P(H) = P(T), so P(H) = P(T)= 1/2.

Now suppose the coin is tossed twice. There are then four possible outcomes of the two tosses: HH, HT, TH, and TT. Considering these four as equally likely, they must each have probability 1/4. Likewise, if the coin is tossed n times, we have 2n possible strings of n letters H and T, where each string has probability 1/2n.

Next let n go to infinity. Then we have all possible infinite sequences of H's and T's. Each individual sequence has probability 0, but this does not determine the probabilities of other interesting sets of possible outcomes, as it did when n was finite. To consider such sets, first let us replace H by 1 and T by 0, precede the sequence by a “binary point” (as in decimal point), and regard the sequence as a binary expansion.