In this chapter we introduce and motivate the statistical models that will be considered in this book. Some of the materials depend on basic facts developed in subsequent chapters – mostly the basic Gaussian process and Hilbert space theory. This will be hinted at when necessary.

Very generally speaking, a statistical model for a random observation Y is a family

{Pf : f ∈ F}

of probability distributions Pf, each of which is a candidate for having generated the observation Y. The parameter f belongs to the parameter space F. The problem of statistical inference on f, broadly speaking, can be divided into three intimately connected problems of using the observation Y to

1. (a) Estimate the parameter f by an estimator T(Y),

2. (b) Test hypotheses on f based on test functions Ψ(Y) and/or

3. (c) Construct confidence sets C(Y) that contain f with high probability.

To interpret inferential results of these kinds, we will typically need to specify a distance, or loss function on F, and for a given model, different loss functions may or may not lead to very different conclusions.

The statistical models we will introduce in this chapter are, on the one hand, conceptually closely related to each other in that the parameter space F is infinite or high dimensional and the loss functions relevant to the analysis of the performance of statistical procedures are similar. On the other hand, these models are naturally divided by the different probabilistic frameworks in which they occur – which will be either a Gaussian noise model or an independent sampling model. These frameworks are asymptotically related in a fundamental way (see the discussion after Theorem 1.2.1). However, the most effective probabilistic techniques available are based on a direct, nonasymptotic analysis of the Gaussian or product probability measures that arise in the relevant sampling context and hence require a separate treatment.

Thus, while many of the statistical intuitions are common to both the sampling and the Gaussian noise models and in fact inform each other, the probabilistic foundations of these models will be laid out independently.