Research Design in Clinical Psychology

Null hypothesis significance testing is the dominant method to analyze the results of research. Statistical tests use probability levels to make the decision to accept or reject the null hypothesis (there is no difference statistically). Issues critical to statistical evaluation were discussed, including significance levels (alpha), power, sample size, and significance and magnitude of effects. Statistical power has received extensive discussion in research in part because repeated evaluations have shown that the majority of studies are designed in such a way as to have weak power. Multiple ways of increasing power were presented. There have been many sources of dissatisfaction with statistical significance testing including the fact that null hypothesis and statistical significance testing give us arbitrary cutoff points to make binary decisions (accept or reject the null hypothesis), and most importantly do not provide the critical information we would like (e.g., direct tests of our hypotheses and information about the strengths of our interventions). Null hypothesis statistical testing is not the only way of approaching the data and data analyses. Bayesian data analyses were highlighted as an alternative to null hypothesis statistical tests.