Introduction

Game theory and statistical decision theory, as treated here, are closely connected. Although the origins of most theories are hard to trace, the original major contributions to these theories are von Neumann & Morgenstern's book, Theory of games and economic behaviour, 1944 and Wald's book Statistical decision functions, 1950. Here we shall give a short introduction to those parts of game theory which have been particularly useful for decision theory. The author learned about most of the results which are presented here from the lectures and writings of Bickel, Blackwell and LeCam.

Sections 3.2 and 3.3 are intended as an introduction to the general theory of two-person, zero-sum games. Some of the standard tools for establishing minimaxity and admissibility are described in section 3.4. We have found it convenient to phrase these results in terms of a general functional rather than in terms of Bayes risk. The principles are elementary as well as useful. We have therefore organized the material so that this section may be read independently of the previous two sections as well as of chapter 1.

Values and strategies

A two-person, zero-sum game is a triple Γ = (A, B, M) where A and B are arbitrary sets and M is a function from A × B to [– ∞, ∞]. The game involves two players, player I and player II. The elements of A and B are called the (pure) strategies of player I and player II, respectively.

Introduction

We saw in chapter 6 that a variety of natural and apparently different criteria for comparison actually were equivalent. Thus we found equivalent criteria in terms of overall comparison of risk functions, in terms of Bayes risk for a fixed prior distribution, in terms of sublinear functional, in terms of performance functions of decision rules and in terms of randomizations.

The idea that the underlying distributions which constitute a statistical experiment should be proper probability distributions was of course essential for the interpretation of these results. In particular the interpretation of risk as expected loss depended on this assumption. However if we look over the proofs we see that the arguments do not rely on this. Thus the formal expression of the risk of a decision procedure, as the corresponding bilinear functional evaluated for the underlying distribution and the loss function, remains well defined within a much more general set-up.

We shall now see that such a generalization, or extension, to ‘non-proper’ experiments actually yields interesting results on ‘proper’ experiments. Another benefit is a unified approach to several results concerning systems of inequalities which are frequently encountered in mathematical statistics. The theory we shall formulate here, partly based on Torgersen (1969, 1985), may be considered as a generalization of the theory of majorization as it is described in Marshall & Olkin (1979) and in later generalizations of Dahl (1983) and Karlin & Rinott (1983).

At this point let us briefly indicate some directions of applications.

Example 9.1.1 (Comparison for different losses). Assume that all losses are bounded by 1 and let a, b and e be given functions of the unknown parameter θ.

Introduction

A vector lattice (Riesz space) is a linear space equipped with a lattice ordering which is ‘compatible’ with the linear structure. An ordering is here called a lattice ordering if finite sets possess greatest lower bounds and smallest upper bounds. The theory of statistical experiments abounds with linear spaces which are vector lattices for their ‘natural orderings’. Without specifying the structures we mention a few examples of such spaces.

The space of (equivalence classes) of bounded variables.

The space of (equivalence classes) of real valued variables.

The space of (bounded) continuous real valued functions on a topological space.

The space of differences of sublinear functional on a linear space.

The M-space M(ℰ) of an experiment ℰ.

The space of bounded additive set functions on a given algebra of sets.

The space of finite measures on a given σ-algebra of sets.

The L-space L(ℰ) of an experiment ℰ.

The industrious reader may define the ‘natural’ structures in these examples and check that they really deserve to be called vector lattices.

In this chapter we will give a short introduction to the theory of vector lattices. The theory is illustrated throughout by various spaces which are important to the theory of statistical experiments.

The nuts and bolts are collected in sections 5.2–5.6. Actually it might suffice at a first reading to study sections 5.2, 5.3, theorem 5.4.1 and example 5.4.2, and then look over the contents of sections 5.5 and 5.6. The reader might then proceed and return to this chapter when the need arises.

Introduction

The single most important concept in this work is the concept of a deficiency

Let ℰ and ℱ be two experiments having the same parameter set Θ. Then we shall say that ℰ is at least as informative as ℱ, if to any decision problem and any decision rule in ℱ there is a decision rule in ℰ which is at most as risky for any θ in Θ.

In this way we arrive at the partial ordering ‘being at least as informative as’ for experiments having the same parameter set Θ. The concept of sufficiency corresponds to the case where ℰ is a subexperiment of ℱ.

With this kind of a definition it is to be expected that the ordering is not total. In fact in general we may expect that two experiments having the same parameter set Θ are not comparable with respect to this ordering. Thus we are led to the following generalization of the fundamental question: How much do we lose, under the worst possible circumstances, by using ℰ instead of ℱ?

As we shall see, an answer to this problem may be given by a non-negative number; the deficiency of ℰ with respect to ℱ.

Closely associated with the notion of deficiency is a distance for experiments or, equivalently, for the (undefined) amounts of information carried by the experiments.

Finally we may restrict our attention to certain types of decision problems. This leads to deficiencies and distances relative to the relevant type of decision problems.

This is a self-contained exposition of the theory of comparison of statistical experiments. The idea of comparing experiments by comparing the risks they produce for varying losses, developed gradually after the appearance of Wald's works on statistical decision functions. Contributions by Blackwell, Bohnenblust, Shapley, Sherman and Stein in the years 1949–1953 provided criteria for one experiment being more informative than another. In 1955 Boll showed how the comparison problem may be reduced when invariance conditions are satisfied. In 1965 Strassen generalized the dilation criterion and also provided a variety of other related and interesting results.

In the same period a considerable effort was made to clarify basic statistical concepts such as sufficiency and invariance within the framework introduced by Kolmogorov in 1933. Building on earlier works of Fisher, Neyman, Pearson and Wald these concepts were explored by Bahadur, Blackwell, Halmos and Savage, for example, and, although it was not well known at the time, by Dynkin.

Substantially broadening the scope, LeCam (1959) (see LeCam, 1964) addressed himself to the following question.

Given two experiments ℰ and ℱ, when are we justified in claiming that only so much information is lost by basing ourselves on the experiment ℰ rather than on the experiment ℱ?

It turned out that a variety of different approaches to this problem led to the same numerical quantity, the deficiency of ℰ with respect to ℱ. Thus the deficiency has natural interpretations in terms of pointwise comparison of risks, in terms of maximum risks, in terms of performance functions of decision rules, in terms of Bayes risk and in terms of randomizations.

Introduction

Convex analysis is an indispensible tool of mathematical statistics. The purpose of this chapter is to provide a self-contained body of central results which are useful for decision theory in general and which are particularly useful for the topic covered in this book.

Several excellent textbooks are available in this area. Some well known books in decision theory, e.g. Blackwell & Girshick (1954), Ferguson (1967) and LeCam (1986), also contain quite an amount of convex analysis. Another highly relevant work is the book by Marshall & Olkin (1979).

References to original sources, with a few exceptions, are not given. They may be found either in the cited textbooks or in some of the other textbooks which are included in the references. The references are mainly books which have been particularly useful to the author.

The basic concepts and notions are introduced within the framework of a general real linear space, in section 2.1. However results which do not follow fairly directly from the definitions are usually only given in the finite dimensional case. An exception is the Hahn-Banach theorem. The spadework contained in the ‘standard’ proof of that theorem is finite dimensional. As we shall see, an inspection of this proof provides criteria for measurability of the guaranteed extension and this yields a substantial part of an important decomposition theorem in Strassen (1965). The complete proof of this theorem is provided in complement 22 in chapter 10.

Introduction

This chapter is intended to serve several purposes. Firstly it contains tools which will be used throughout this work. Thus several of the basic concepts are introduced. Some of the definitions are repeated in later chapters, but others are only given here. It is not necessary to have acquired an understanding of everything here in order to proceed. Indeed some of the material in this chapter anticipates later developments and may not be fully comprehended before these developments have been studied. Thus the reader may just look over the chapter and later return to this material when the need arises.

Another purpose is to provide a self-contained introduction to some ideas which later will play a central role. Thus it is intended to be possible to go through this chapter before (or whilst) reading the later chapters.

A third purpose is to present some ‘classical’ material which is important as background for the theory of statistical experiments as it is presented here. In particular some of this material indicates the need of extending, as LeCam did, the measure theoretical framework of decision theory.

The notion of a statistical experiment (statistical model) is introduced in section 2. It is argued there that the usual notion of a statistic as a random variable is too narrow and that it should be extended to the more general concept of a consistent family. The optimistic goal is to make everything as simple (or difficult) as it is in the ‘finite’ case. Difficulties beyond the finite case may then be considered as being of technical nature.

Introduction

In this chapter we will consider comparison of some of the most familiar linear models. Here ‘linear’ means essentially that the expectations of the observations depend linearly on the unknown parameter. The main emphasis will be on linear models with covariances which are either completely known or are known up to a common positive factor. The observations which realize our models may not be normally distributed. However, if we assume they are normally distributed, the conclusions may be drastically strengthened. In this case the Hellinger transform is a very useful tool.

As we do not wish to stretch the generality too far, we shall restrict our attention to models where the dimensions of the parameter spaces and the sample spaces are finite. We shall find it convenient to use a coordinate-free approach and we shall assume that the underlying spaces are finite dimensional inner product spaces, i.e. finite dimensional Hilbert spaces.

One advantage of using this framework rather than the ℝn-approach is that the various related linear spaces need not be re-parametrized in order to be in the appropriate form. This framework also appears to make it easier to envisage generalizations to infinite dimensional spaces.

Before proceeding we should state some basic facts on random vectors. Firstly a finite dimensional inner product space, as any metric space, has a natural topology. The Borel class is of course the σ-algebra generated by the open sets (closed sets, compact sets) and this is the smallest σ-algebra making the linear functional measurable. Representing the vectors by their coordinates with respect to a fixed orthonormal basis we obtain an isomorphism between H and some space ℝm equipped with the usual scalar product.