The consequences of multiple tests within a study have received much attention in recent years. In spite of many detailed considerations, there remains controversy concerning the seriousness of these consequences.

To appreciate fully the arguments that abound in this controversy, it is necessary to be aware of several different definitions of errors that can occur in hypothesis testing. The definitions of Type I error (rejection of a true null hypothesis) and Type II error (failure to reject a false null hypothesis) remain basic. Consider initially the Type I error, which receives the major emphasis of arguments concerning the effects of multiple testing. Its usual definition is based on the fact that for a single variable and a single comparison, there is a probability (α) that the hypothesis of no effect erroneously can be rejected. If all studies consisted of only one variable and if only one comparison were of interest (for example, only two treatments or two groups were studied), there would be no multiple testing controversy. Of course, few, if any, studies are so limited in their intent. Thus, the consequences of multiple testing apply to nearly all studies.