Introduction

In layman's terms, a sufficient statistic for guessing M based on the observable Y is a random variable or a collection of random variables that contains all the information in Y that is relevant for guessing M. This is a particularly useful concept when the sufficient statistic is more concise than the observables. For example, if we observe the results of a thousand coin tosses Y1, …, Y1000 and we wish to test whether the coin is fair or has a bias of ¼, then a sufficient statistic turns out to be the number of “heads”among the outcomes Y1, …, Y1000. Another example was encountered in Section 20.12. There the observable was a two-dimensional random vector, and the sufficient statistic summarized the information that was relevant for guessing H in a scalar random variable; see (20.69).

In this chapter we provide a formal definition of sufficient statistics in the multihypothesis setting and explore the concept in some detail. We shall see that our definition is compatible with Definition 20.12.2, which we gave for the binary case. We only address the case where the observations take value in the d-dimensional Euclidean space ℝd. Extensions to observations consisting of a stochastic process are discussed in Section 26.3. Also, we only treat the case of guessing among a finite number of alternatives.