Summary. Approximate forms of inference based on local approximations of the log likelihood in the neighbourhood of its maximum are discussed. An initial discussion of the exact properties of log likelihood derivatives includes a definition of Fisher information. Then the main properties of maximum likelihood estimates and related procedures are developed for a one-dimensional parameter. A notation is used to allow fairly direct generalization to vector parameters and to situations with nuisance parameters. Finally numerical methods and some other issues are discussed in outline.

General remarks

The previous discussion yields formally exact frequentist solutions to a number of important problems, in particular concerning the normal-theory linear model and various problems to do with Poisson, binomial, exponential and other exponential family distributions. Of course the solutions are formal in the sense that they presuppose a specification which is at best a good approximation and which may in fact be inadequate. Bayesian solutions are in principle always available, once the full specification of the model and prior distribution are established.

There remain, however, many situations for which the exact frequentist development does not work; these include nonstandard questions about simple situations and many models where more complicated formulations are unavoidable for any sort of realism. These issues are addressed by asymptotic analysis. That is, approximations are derived on the basis that the amount of information is large, errors of estimation are small, nonlinear relations are locally linear and a central limit effect operates to induce approximate normality of log likelihood derivatives.