Our aim in this book is to explain and illustrate the fundamental statistical concepts required for designing efficient experiments to answer real questions. This book has evolved from a previous book written by the first author. That book was based on 25 years of experience of designing experiments for research scientists and of teaching the concepts of statistical design both to statisticians and to experimenters. The present book is based on approximately a combined 100 years of experience of designing experiments for research scientists, and of teaching the concepts of statistical design both to statisticians and to experimenters.

The development of statistical philosophy about the design of experiments has always been dominated by mathematical theory. In contrast the influence of the availability of vastly improved computing facilities on teaching, textbooks and, most crucially, practical experimentation has been relatively small. The existence of statistical programs capable of analysing the results from any designed experiment does not imply any changes in the main statistical concepts of design. However, developments from these concepts have often been restricted by the earlier need to develop mathematical theory for design in such a way that the results from the designs could be analysed without recourse to computers. The fundamental concepts continually require reexamination and reinterpretation outside the limits implied by classical mathematical theory so that the full range of design possibilities may be considered. The result of the revolution in computing facilities is that the design of experiments should become a much wider and more exciting subject. We hope that this book will display that breadth and excitement.

Preliminary examples

(a) In an animal feeding experiment, six dietary treatments are to be compared. The diets are all possible combinations of three different levels of molasses at two energy levels. Twenty-four sheep are available and each sheep can be fed a different diet in each of two time periods. It is expected that there will be large differences in nutritional performances between sheep and some systematic differences between the results for the first and second periods. The structure of the experimental units therefore has two blocking classifications, giving a 24 × 2 row-and-column structure. The treatments have a 3 × 2 structure and main effect comparisons, particularly between the three levels of molasses, are the principal area of interest.

(b) An industrial experiment is to be planned to investigate the effects of varying seven factors in a chemical process. Eight treatment combinations can be tested using the same batch of basic material. Eight different batches of material will be available, and it is expected that theremay be substantial differences in output for sample units from the different batches. If it is decided to use two levels of each factor how shall the 64 treatment combinations to be included in the experiment be chosen, and how shall they be allocated to the eight blocks, or batches?

(c) An experiment on absorption of sugar by rabbits is to be designed to compare eight experimental treatments.

Preliminary examples

(a) An experiment to examine the pattern of variation over time of a particular chemical constituent of blood involved sampling the blood of nine chickens on 25 weekly occasions. The principal interest is in the variation of the chemical over the 25 times, the nine chickens being included to provide replication. The chemical analysis is complex and long and a set of at most ten blood samples can be analysed concurrently. It is known that there may be substantial differences in the results of the chemical analysis between different sets of samples. How should the 225 samples (25 times for nine chickens) be allocated to sets of ten (or fewer) so that comparisons between the 25 times are made as precise as possible?

(b) In an experiment to compare diets for cows the experimenter has five diet treatments that he wishes to compare. The diets have to be fed to fistulated (surgically prepared) cows so that the performance of the cows can be monitored throughout the application of each diet. Nine such cows are available for sufficient time that four periods of observation can be used for each cow, providing a set of 9 × 4 = 36 observations. Concerned that there will be differences between cows, so that cows have to be treated as a blocking structure, the experimenter had already decided before consulting a statistician that he could only test four of the diet treatments, each cow receiving each diet once.

Preliminary examples

(a) In an experiment to investigate the effect of training on human-computer-human interactions, six subjects were randomly allocated to each of four training programmes. Subjects were then paired into 12 blocks using two replicates of an unreduced balanced incomplete block design. Each pair carried out a conversation through a computer ‘chat’ program. In addition to several response variables measured on each subject individually, each pair was given a score by an independent observer, for the success of their interaction.

We have only a single response representing each block. Can we use this information and, if so, how? If we can, do the block totals contain useful information about the effects of treatments on other responses? Does this affect how we should design the experiment? In particular, for this response, should we have used some blocks with both subjects getting the same treatment?

(b) Eight feeds are to be compared for their effects on the growth of young chickens. The experiment will be carried out using 32 cages, arranged in four brooders, with each brooder having four tiers of two cages. Should the experiment be designed to ensure that each treatment appears once in each brooder and once in each tier, or should we consider the brooder×tier combinations as blocks of size 2 and choose a good design for this setup? Can we do both simultaneously?

Identifying multiple levels in data

In Section 7.3 we considered the analysis for general block–treatment designs. However, in that analysis only the information about treatments from comparisons within blocks was considered.

Controlling variation between experimental units

In an experiment to compare different treatments, each treatment must be applied to several different units. This is because the responses from different units vary, even if the units are treated identically. If each treatment is only applied to a single unit, the results will be ambiguous – we will not be able to distinguish whether a difference in the responses from two units is caused by the different treatments, or is simply due to the inherent differences between the units.

The simplest experimental design to compare t treatments is that in which treatment 1 is applied to n1 units, treatment 2 to n2 units,… and treatment t to nt units. In many experiments the numbers of units per treatment, n1, n2, …, nt, will be equal, but this is not necessary, or even always desirable. Some treatments may be of greater importance than others, in which case more information will be needed about them, and this will be achieved by increasing the replication for these treatments. This design, in which the only recognisable difference between units is the treatments which are applied to those units, is called a completely randomised design.

However, the ambiguity, when each treatment is only applied to a single unit, is not always removed by applying each treatment to multiple units. One treatment might be ‘lucky’ in the selection of units to which it is to be applied, and another might be ‘unlucky’.

The need for additional variation

Thus far in this book we have considered the design of individual experiments and have been concerned to ensure that each experiment should provide answers to the questions which motivated the experiment as efficiently as possible. In general this has required that the variation in the experimental units be controlled so that the answers provided from each experiment should be as precise as possible. This will frequently require that there should be relatively little variation between the experimental units, i.e. the population from which the units are drawn will be narrowly defined.

However, if the population from which the units are taken is narrowly defined, then it follows logically that the results from the experiment would apply only to that narrowly defined population. This would usually be quite unacceptable to an experimenter who would hope to convince the wider world that the results from the experiment would apply for a much wider population. For example determining which variety of rice gives the best results in a highly controlled experiment on the paddy fields of a research institute is only going to be useful if that variety of rice is going to be the best for a large region within which the institute is located, and if farmers in that region believe in the results. Similarly if a new drug is shown to be an improvement on current practice, through a rigorously controlled clinical trial, the pharmaceutical company which has produced the drug will wish to promote the use of the drug across the whole population of the country, or even of several countries.

Choice of treatments

It might seem that the choice of treatments is not a statistical matter, being a question for the experimenter alone. However, there are very many examples of research programmes where the objectives of the programme either have been, or could have been, achieved much more efficiently by using statistical theory. In some situations, there are several intuitively appealing alternative sets of treatments, and the choice between these alternatives can be assisted by a statistical assessment of the efficiency with which the alternative sets of treatments satisfy the objectives of the experiment. In other situations statistical methods can allowmore objectives to be achieved from the same set of experimental resources. In yet other situations statistical arguments can lead to adding experimental treatments which avoid the necessity of making possibly unjustifiable assumptions in the interpretation of results from the experimental data. The early theory of the design of experiments was less concerned with the choice of treatments than with the control of variation between units, but there is a large body of knowledge about treatment structure. Later, much of the statistical theory of experimental design was concerned with the optimal choice of treatments, and while some of this theory is very mathematical and some has little relevance to real experiments, the general principles are important and will be considered in Chapter 16.

The first point to clarify is that there are many different forms of objective for an experiment. Some experiments are concerned with the determination of optimal operating conditions for some chemical or biological process. Others are intended to make very precise comparisons between two or more methods of controlling or encouraging the growth of a biological organism. Some experiments are intended to provide information about the effects of increasing amounts of some stimulus on the output of the experimental units to which the stimulus is applied. And other experiments are planned to screen large numbers of possible contestants for a job, for example, drugs for treating cancer or athletes for a place in the Olympics. And most experiments have not one but several objectives.

Preliminary example

Suppose eight treatments comprising all combinations of three two-level factors P, Q and R are compared and the resulting treatment means are shown in Table 13.1. What is the best estimate of the difference between the two treatment combinations p1q1r1 and p1q1r0? Or, in practical terms, if you have decided to recommend the upper levels of factors P and Q, what is the benefit of using r1 rather than r0? If you think there is only one possible answer, then you have not yet fully grasped the benefits of factorial structure.With a little effort, you should be able to find at least six different conceivable answers.

Factors with two levels only

In Chapters 3 and 12, we started to examine the general advantages of using experimental treatments with a factorial structure. In this chapter, we look in more detail at special forms of factorial structure to examine how the general advantages manifest themselves in particular cases, and we develop methods for using further advantages of these special forms.

The first special form of factorial to consider is that in which all factors have two levels. There are many reasons why this is an interesting form to examine in detail. First, it is a sensible practical solution for many scientific situations. We can think of one level of each factor as being the normal mode and the other level as a new, alternative mode.

Preliminary example

An experiment to compare four varieties of tomato is to be run in a greenhouse at a horticultural research station. The greenhouse has 16 compartments in a 4 × 4 array. The initial proposal is to use a Latin square design, such as that shown in Figure 11.1(a), which is randomised, as in Chapter 8 by randomly permuting rows and columns, to give the square shown in Figure 11.1(b). On seeing the randomised design, the experimenter recalls that previous experiments in this greenhouse showed unusually high yields along the top-right to bottom-left diagonal and is concerned that this might bias the results in favour of variety D. She also notes that the situation is even worse in the unrandomised design.

One possible solution is to abandon the natural seeming 4 × 4 row-and-column structure and define blocks according to the distance from the top-right to bottom-left diagonal. However, this is not satisfactory either, since the experimenter has also previously observed row and column trends. Another possible solution is to restrict the randomisation more than usual to ensure that the design has all of the required properties.

Time-trend resistant run orders and designs

In industrial experiments, the experimental units are often sequential runs of the same process. The possibility of time trends needs to be allowed for, but resources are often too scarce to benefit from using very small blocks.

Some more real problems

Many of the problems considered in previous chapters have occurred during statistical consultancy sessions and are appropriate to motivate this chapter. Typical examples are the peppers experiment in Chapter 7 and the rabbits experiment in Chapter 15. Three more problems occurred in the work of one of us (RM) in quick succession within three weeks in early 1984 and illustrate a wide range of practical design problems.

(a) Problem 1: the moving cups. In assaying the chemical concentration of liquid samples, a set of about 24 samples can be automatically assayed in a single batch. The samples are placed in cups which are held in position on a circular disc, and the disc rotates so that each cup in turn appears beneath the assay machinery. It is required to investigate changes in the concentration of chemicals over time, and therefore the 24 samples are assayed over a time period of between one and two hours. Two shapes of cup are available, and two sizes of sample are possible within each cup. Three chemicals, sodium (Na), chlorine (Cl) and potassium (K) are of interest, and it is proposed to have two levels (zero and some) for each chemical. The cups may be covered during the run of 24 samples until each is sampled, but the covers, if used, must be used for all 24 samples in the run. […]

Preliminary examples

Both of the following examples come from situations where the experimenter designed an experiment without consulting a statistician. When the experimenter came to consult the statistician, with the experimental data, the first problem was to identify the structure of the design and then to provide a suitable analysis.

(a) The first experiment was concerned with the influences on the production of glasshouse tomatoes of differing air and soil temperatures. Eight glasshouse compartments were available, and these were paired in four ‘blocks’. Each compartment contained two large troughs in which the tomatoes were grown and in each half of each trough the soil temperature could be heated to a required level or left unheated. In one compartment of each block the minimum air temperature was kept at 55 °F and in the other the minimum air temperature was 60 °F. In each trough, one half was maintained at the control soil temperature while the other half was at an increased temperature, the increased temperature being 65 °F for one trough, and 75 °F for the other trough. The design layout for one pair of compartments is shown in Figure 18.1. Yields of tomatoes from each half-trough were recorded.

(b) The second experiment was also concerned with heating, this time the heating characteristics of different forms of plastic pot situated in cold frames and of different forms of covers for the frames. Six wooden frames were divided into four quarters and four different forms of pot were allocated to the four quarters randomly in each frame.