Science is invariably based on some sort of data collection and further treatment of the data gathered. Data can come from pure observations, from structured observations (‘natural experiments’) or from experiments. The central importance of models in science is mentioned. It is discussed how the choice of statistics reflects the philosophy of science adopted by the scientist. Different research programmes use different statistics, in particular, depending on when and how they deal with variation. The relationship between falsificationism and the rejection of null hypotheses as a workaround for the Duhem-Quine thesis is discussed, as well as the role of significance thresholds and their associated problems. It is argued that predicted results are more reliable than chance findings. The pros and cons of having alternative hypotheses are discussed, and a short introduction to Bayesian statistics as an alternative to frequentist approaches is given. Systematic reviews and metaanalyses of data from several studies are introduced, and an example is given on how different types of evidence from many studies are combined to form the current consensus of rational opinion regarding a particular hypothesis.