The general problem of statistical inference is one in which, given observations of some random phenomenon, we try to make an inference about the probability distribution describing it. Much of statistics is devoted to the problem of inference. Usually we will suppose that the distribution is one of a family of distributions f(t|θ) parameterized by θ, and we try to make an assessment of the likely values taken by θ. An example is the exponential distribution f(t|λ) = λ exp(−λt), but also the joint distribution of n independent samples from the same exponential, f(t1, …, tn|λ) = λn exp(−λ(t1 + … + tn)), falls into the same category and is relevant when making inference on the basis of n independent samples.

Unfortunately, statisticians are not in agreement about the ways in which statistical inference should be carried out. There is a plethora of estimation methods which give rise to different estimates. Statisticians are not even in agreement about the principles that should be used to judge the quality of estimation techniques. The various creeds of statistician, of which the most important categories are Bayesian and frequentist, differ largely in the choice of principles to which they subscribe. (An entertaining guide to the differences is given in the paper of Bradley Efron ‘Why isn't everyone a Bayesian?’ and the heated discussion that follows, [Efron, 1986].) To some extent the question is whether one thinks that statistical inference should be inductive or deductive.