The lessons of this book can be distilled into the thirty recommendations listed below. The numbers in brackets refer to the relevant chapters in this book.

Before doing the study:

1. Quantify your expectations regarding the effect size. Ask yourself, what results do I expect to see in this study? Be explicit. Develop a rationale for doing another study given extant results. If there is no past relevant research, ask: How big an effect would I need to see to make this study worthwhile? Would the rejection of the null hypothesis of no effect be sufficiently interesting? (1)

2. Identify the range of effect sizes observed in prior studies. When reviewing past research, do not be distracted by the conclusions of others that may have been mistakenly drawn from p values. Rather, examine the evidence and draw your own conclusions. The relevant evidence includes the size and direction of the estimated effect, the precision of the estimate, and the reliability of the measurement procedures. To minimize the threat of the availability bias make every effort to examine the evidence from unpublished, as well as published, research. (5,6)

3. Look for meta-analyses that are relevant to the effect you are interested in or consider doing one yourself. Meta-analyses are useful for providing non-zero benchmarks that may be more meaningful than testing the null hypothesis of no effect. A good meta-analysis will also reveal unexplored avenues for further research. (5)

When designing the study:

1. 4. Conduct a prospective power analysis to determine the minimum sample sizes needed to detect the expected effect size. Carefully assess the trade-off between sample size and power. Ask yourself, do the anticipated benefits of detecting an effect of this magnitude exceed the costs required to detect it? (3, Appendix 1)

2. […]