The focus of this book is on applied structured regression modeling for categorical data. Therefore, it is concerned with the traditional problems of regression analysis: finding a parsimonious but adequate model for the relationship between response and explanatory variables, quantifying the relationship, selecting the influential variables, and predicting the response given explanatory variables.

The objective of the book is to introduce basic and advanced concepts of categorical regressions with the focus on the structuring constituents of regressions. The term “categorical” is understood in a wider sense, including also count data. Unlike other texts on categorical data analysis, a classical analysis of contingency tables in terms of association analysis is considered only briefly. For most contingency tables that will be considered as examples, one or more of the involved variables will be treated as the response. With the focus on regression modeling, the generalized linear model is used as a unifying framework whenever possible. In particular, parametric models are treated within this framework.

In addition to standard methods like the logit and probit models and their extensions to multivariate settings, more recent developments in flexible and high-dimensional regressions are included. Flexible or non-parametric regressions allow the weakening of the assumptions on the structuring of the predictor and yield fits that are closer to the data. High-dimensional regression has been driven by the advance of quantitative genetics with its thousands of measurements. The challenge, for example in gene expression data, is in the dimensions of the datasets. The data to be analyzed have the unusual feature that the number of variables is much higher than the number of cases.

In a first course on probability one typically works with a sequence of random variables X1, X2, … For stochastic processes, instead of indexing the random variables by the positive integers, we index them by t ∈ [0, ∞) and we think of Xt as being the value at time t. The random variable could be the location of a particle on the real line, the strength of a signal, the price of a stock, and many other possibilities as well.

We will also work with increasing families of σ-fields {ℱt}, known as filtrations. The σ-field ℱt is supposed to represent what we know up to time t.

Processes and σ-fields

Let (Ω, ℱ, ℙ) be a probability space. A real-valued stochastic process (or simply a process) is a map X from [0,∞) × Ω to the reals. We write Xt = Xt(ω) = X (t, ω). We will impose stronger measurability conditions shortly, but for now we require that the random variables Xt be measurable with respect to ℱ for each t ≥ 0.

A collection of σ-fields ℱt such that ℱt ⊂ ℱ for each t and ℱs ⊂ ℱt if s ≤ t is called a filtration. Define ℱt+ = ∩∈ > 0ℱt+∈. A filtration is right continuous if ℱt+ = ℱt for all t ≥ 0. The σ-field ℱt+ is supposed to represent what one knows if one looks ahead an infinitesimal amount. Most of the filtrations we will come across will be right continuous, but see Exercise 1.1.

A null set N is one that has outer probability 0.

Why study stochastic processes? This branch of probability theory offers sophisticated theorems and proofs, such as the existence of Brownian motion, the Doob–Meyer decomposition, and the Kolmogorov continuity criterion. At the same time stochastic processes also have far-reaching applications: the explosive growth in options and derivatives in financial markets throughout the world derives from the Black–Scholes formula, while NASA relies on the Kalman–Bucy method to filter signals from satellites and probes sent into outer space.

A graduate student taking a year-long course in probability theory first learns about sequences of random variables and topics such as laws of large numbers, central limit theorems, and discrete time martingales. In the second half of the course, the student will then turn to stochastic processes, which is the subject of this text. Topics covered here are Brownian motion, stochastic integrals, stochastic differential equations, Markov processes, the Black–Scholes formula of financial mathematics, the Kalman–Bucy filter, as well as many more.

The 42 chapters of this book can be grouped into seven parts. The first part consists of Chapters 1–8, where some of the basic processes and ideas are introduced, including Brownian motion. The next group of chapters, Chapters 9–15, introduce the theory of stochastic calculus, including stochastic integrals and Itô's formula. Chapters 16–18 explore jump processes. This requires a study of the foundations of stochastic processes, which is also known as the general theory of processes. Next we take up Markov processes in Chapters 19–23. A formidable obstacle to the study of Markov processes is the notation, and I have attempted to make this as accessible as possible. Chapters 24–29 involve stochastic differential equations.

Suppose we have a sequence of probabilities on a metric space S and we want to define what it means for the sequence to converge weakly. Alternately, we may have a sequence of random variables and want to say what it means for the random variables to converge weakly. We will apply the results we obtain here in later chapters to the case where S is a function space such as C[0, 1] and obtain theorems on the convergence of stochastic processes.

For now our state space is assumed to be an arbitrary metric space, although we will soon add additional assumptions on S. We use the Borelσ-field on S, which is the σ-field generated by the open sets in S.We write A0, Ā, and δA for the interior, closure, and boundary of A, respectively.

The portmanteau theorem

Clearly the de?nition of weak convergence of real-valued random variables in terms of distribution functions (see Section A. 12) has no obvious analog. The appropriate generalization is the following; cf. Proposition A. 41.

Definition 30.1 A sequence of probabilities {ℙn} on a metric space S furnished with the Borel σ-field is said to converge weakly to ℙ if ʃ f dℙn → ʃ fdℙ for every bounded and continuous function f on S. A sequence of random variables {Xn} taking values in S converges weakly to a random variable X taking values in S if E f (Xn) → Ef(X) whenever f is a bounded and continuous function.