Given some data our problem is to discover the best point estimate of a magnitude. For this purpose a good estimate will be what I have called a well-supported one: an estimate which the data give good reason to suppose is close to the true value. We assume that a scale of closeness is implied in the statement of the problem. We shall be concerned with the excellence of individual estimates, and of estimators making estimates for particular problems, rather than with comparing the average properties of estimators. We seek estimators which give well-supported estimates.

Only rather limited and feeble solutions are offered here. Perhaps there is no general answer to fundamental questions about estimation. Maybe estimation, as I have understood it in the preceding chapter, is only barely relevant to statistical inference. One ought, perhaps, to search for other, related, notions which are more fruitful. But first we must try to do our best with well-supported estimates of statistical parameters.

An historical note

A little history will hint at the flavour of chaos in estimation theory. Unfortunately few students have distinguished the property of being a well-supported estimate, from the property of being an estimate made by an estimator which is in some way close on the average. So the traditional estimators I am about to describe were seldom aimed at one or the other of these two properties, but if anything, at both at once.

Estimation in statistics is relatively recent; it followed two centuries of estimation in other fields. Most work did, however, make use of the doctrine of chances. The usual problem concerned measurement. Some quantity, such as the distance of a star or the melting point of a chemical, is measured several times; there are small discrepancies; now use the measurements to estimate the true value of the physical quantity. Another problem is curve fitting; try to fit a smooth curve to a number of different points all based on observations. This is both a problem of guessing the true curve, and of estimating the characteristics of that curve.

There were forerunners, but Laplace and Gauss are the great pioneers of what came to be called the theory of errors. Despite their many theoretical insights, the whole science remained pretty ad hoc until the 1920's. Many issues had to be settled which now are taken for granted.