Introduction

In layman's terms, a sufficient statistic for guessing M based on the observable Y is a random variable or a collection of random variables that contains all the information in Y that is relevant for guessing M. This is a particularly useful concept when the sufficient statistic is more concise than the observables. For example, if we observe the results of a thousand coin tosses Y1, …, Y1000 and we wish to test whether the coin is fair or has a bias of 1/4, then a sufficient statistic turns out to be the number of “heads”among the outcomes Y1, …, Y1000. Another example was encountered in Section 20.12. There the observable was a two-dimensional random vector, and the sufficient statistic summarized the information that was relevant for guessing H in a scalar random variable; see (20.69).

In this chapter we provide a formal definition of sufficient statistics in the multihypothesis setting and explore the concept in some detail. We shall see that our definition is compatible with Definition 20.12.2, which we gave for the binary case. We only address the case where the observations take value in the d-dimensional Euclidean space Rd. Also, we only treat the case of guessing among a finite number of alternatives. We thus consider a finite set of messages

whereM ≥ 2, and we assume that associated with each message is a density on Rd, i.e., a nonnegative Borel measurable function that integrates to one.

The concept of sufficient statistics is defined for the family of densities

it is unrelated to a prior. But when we wish to use it in the context of hypothesis testing we need to introduce a probabilistic setting. If, in addition to the family, we introduce a prior, then we can discuss the pair (M,Y), where Pr[M = m] = πm, and where, conditionally on M = m, the distribution of Y is of density.