Summary. First a number of distinct situations are given in which significance tests may be relevant. The nature of a simple significance test is set out and its implications explored. The relation with interval estimation is emphasized. While most of the discussion is from a frequentist perspective, relations with Bayesian theory are outlined in the final section.

General remarks

So far, in our frequentist discussion we have summarized information about the unknown parameter ψ by finding procedures that would give in hypothetical repeated applications upper (or lower) bounds for ψ a specified proportion of times in a long run of repeated applications. This is close to but not the same as specifying a probability distribution for ψ; it avoids having to treat ψ as a random variable, and moreover as one with a known distribution in the absence of the data.

Suppose now there is specified a particular value ψ0 of the parameter of interest and we wish to assess the relation of the data to that value. Often the hypothesis that ψ = ψ0 is called the null hypothesis and conventionally denoted by H0. It may, for example, assert that some effect is zero or takes on a value given by a theory or by previous studies, although ψ0 does not have to be restricted in that way.