We concluded the previous chapter by introducing two methods of inference concerning the parameter vector. Since the Bayesian approach was one of them, we focus here on the competing frequentist or classical approach in its attempt to draw conclusions about the value of this vector. We introduce hypothesis testing, test statistics and their critical regions, size, and power. We then introduce desirable properties (lack of bias, uniformly most powerful test, consistency, invariance with respect to some class of transformations, similarity, admissibility) that help us find optimal tests. The Neyman–Pearson lemma and extensions are introduced. Likelihood ratio (LR), Wald (W), score and Lagrange multiplier (LM) tests are introduced for general hypotheses, including inequality hypotheses for the parameter vector. Monotone LR and the Karlin–Rubin theorem are studied, as is Neyman's structure and its role in finding optimal tests. The exponential family features prominently in the applications. Finally, distribution-free (nonparametric) tests are studied and linked to results in earlier chapters.

We introduce variates (random variables), defining their cumulative distribution function (c.d.f.) and probability density function (p.d.f.). We give the two main decomposition theorems of c.d.f.s, Jordan and Lebesgue, and the unifying Stieltjes integral approach for continuous and discrete variates. We study properties and descriptions such as symmetry, median, quantiles, and the quantile function, and modes of a distribution. The mixing of variates is also studied, including the famous example of Student's t, which can be represented as a mixed-normal variate.

We introduce principles of point estimation, that is, the estimation of a value for the vector of unknown parameters of the density of a variate. The chapter starts by considering some desirable properties of point estimators, a sort of “the good, the bad, and the ugly” classification! The topics covered include bias, efficiency, mean-squared error (MSE), consistency, robustness, invariance, and admissibility. We then introduce methods of summarizing the data via statistics that retain the relevant sample information about the parameter vector, and we see how they achieve the desirable properties of estimators. We discuss sufficiency, Neyman's factorization, ancillarity, Rao-Blackwellization, completeness, the Lehmann–Scheffé theorem and the minimum-variance unbiasedness of an estimator, and Basu's theorem. We consider the exponential family and special cases and conclude by introducing the most common model in statistics, the linear model, which is used for illustrations in this chapter and is covered more extensively in the following chapters.

Up to now, we have dealt with various foundational aspects of random variables and their distributions. We have occasionally touched on how these variates can arise in practice. In the second part of this book, we start analyzing in more detail how the variates are connected with sampling situations, how we can estimate the parameters of their distributions (which are typically unknown in practice), and how to conduct inference regarding these estimates and their magnitudes. This chapter starts with the first of these three aims. We study the sample mean and variance and their sampling properties, but also the sample's order statistics and extremes. The empirical distribution function (EDF) is defined and analyzed. In the case of multivariate normality, the Wishart distribution arises as a generalization of the chi-squared. We study the properties of matrix Wishart variates. We also show how Hotelling's T² arises as the counterpart of Student's t in the case of multivariate samples. The density of the correlation coefficient is also derived. We introduce rank and sign correlations, known as Spearman's rho and Kendall's tau, respectively.