Focus on the Sampling Distribution

At the core of parametric testing is the assumption that the sampling distribution of the test statistic, rather than the raw sample data, is normally distributed. ^{Reference Tsagris and Pandis2} Normality has to be established for the populations under consideration (or at least its sampling distribution), not for the sample of data being analyzed. ^{Reference Rochon, Gondan and Kieser3} This theoretical foundation clarifies that testing the normality of observed data is not a direct verification of the parametric assumptions. The emphasis lies on the distribution of the mean over repeated sampling, a concept that renders preliminary normality tests on raw data conceptually misplaced.

Normality Tests Do Not Confirm Normality

It is a theoretical fallacy to interpret a non-significant normality test as evidence that the data are normally distributed. Such tests can only suggest a lack of evidence to reject the null hypothesis of normality, not confirm it. A non-significant P value in a normality test means “we don’t have enough evidence to reject normality,” not “the data are normal.” Suppose a researcher has n = 12 measurements of a biomarker used to detect susceptibility to excess bleeding in blunt force trauma. Imagine that the true underlying population is a heavily right-skewed exponential distribution. The Shapiro–Wilk test is applied:

• Test result: W = 0.95; P = .18.

• Interpretation: P >.05 → “fail to reject” normality.

But here’s the catch: with such a small sample, the test doesn’t have the statistical power to pick up the strong skewness of the underlying distribution. If we simulated thousands of values from the same process, the skew would be obvious. The “non-significant” result is simply telling us this small dataset doesn’t provide enough evidence to say it isn’t normal, not that it truly is normal. This theoretical limitation underscores the unreliability of using normality tests as a basis for methodological decisions.

Robustness of Parametric Tests

From a theoretical and practical perspective, parametric tests are known to be robust to moderate deviations from normality. This is particularly true with larger sample sizes: although these tests remain valid in very small samples when the outcome variable is normally distributed, their principal strength lies in the fact that in every moderately large sample, they retain validity regardless of the underlying distribution. ^{Reference Lumley, Diehr, Emerson and Chen4} For instance, when using linear regression where the number of observations per variable is more than 10, violations of the normality assumption often do not noticeably impact results. ^{Reference Schmidt and Finan5} This robustness implies that a strict preliminary normality test is not required for determining whether parametric methods can be applied, as these tests can tolerate non-normality to a reasonable extent. ^{Reference Knief and Forstmeier6}

Sample Size Sensitivity

In practical terms, normality tests exhibit paradoxical behavior depending on sample size. They often fail to detect real deviations in small samples and over-emphasize trivial deviations in large samples. ^{Reference Fagerland7} In small samples, where non-parametric testing is most useful, normality tests suffer from low statistical power and frequently fail to identify substantial departures from normality. In contrast, in large samples, normality tests become excessively sensitive, almost inevitably detecting statistically significant deviations since no real-world dataset is perfectly normal. This may prompt researchers to choose a less powerful non-parametric test unnecessarily, thereby diminishing the ability to detect true effects. As a result, researchers may be misled into using non-parametric methods when parametric ones would suffice, or vice versa, based solely on sample size artifacts.

Distortion of Error Rates

Finally, the most practical concern is that conditioning the choice of analysis on a normality test can distort Type I error rates. From a theoretical standpoint, the practice of double-testing inflates Type I error rates because the preliminary normality test and the main hypothesis test are not independent—they are both conducted on the same data, and the choice of test is conditional on the outcome of the first. As a result, the nominal significance level no longer reflects the true probability of a false positive, and the actual Type I error rate for the overall procedure exceeds the pre-specified threshold. By making methodological decisions contingent on a preliminary test, researchers risk inflating their overall error rate and undermining the integrity of their statistical inference.

Rational Use of Normality Testing

Although this editorial argues strongly against the routine use of normality testing as a gatekeeper for choosing between parametric and non-parametric methods, there are cases where tests for normality are appropriate. Specifically, when the primary hypothesis of the study concerns the distribution of the data itself—for example, to determine whether a biological measurement follows a normal distribution or to compare the empirical distribution of a sample against a theoretical one—formal tests of normality are both relevant and justified. In such circumstances, normality tests serve as the object of inquiry rather than as a preliminary diagnostic tool, and their use can meaningfully advance scientific understanding.

Finally, a point about style in academic writing: most journals, including Prehospital and Disaster Medicine (PDM), require that every statistical test described in the Methods section be accompanied by corresponding results in the Results section. Therefore, if authors choose to use normality testing, they must ensure that the outcomes of these tests are explicitly and accurately presented in the Results section, rather than implied or omitted.