Goodness of fit tests

In addition to graphical methods one can also perform numerical tests of goodness of fit (GoF). The advantages of such tests are that they are objective (except for chi-square tests – see below) and provide a single number; they can thus be used in automated methods for ranking a number of distributions.

In the following discussion of the most common GoF tests we use the statistical concept of ‘power’ to describe the effciency of the test. A GoF test evaluates how likely it is that the observed sample could have been generated from the distribution in question. The power of the test is the proportion of correctly rejected tests. If a test has low power it means that it is bad at rejecting distributions even if the fit is poor. Conversely, a test with high power is good at identifying distributions that do not match the sample. In practice this means that if a distribution fails a test with low power (such as, for example, chi-square or Kolmogorov–Smirnov (KS)) then this is a good indication that the distribution is not appropriate.

The chi-square test

The chi-square test can be used as a GoF test with any distribution, either continuous or discrete. However, if the distribution is continuous it must first be discretised by dividing the sample space into intervals and binning the observations into these intervals. For a discrete distribution the bins are simply the different outcomes in the sample.