By rejecting a statistical hypothesis I shall mean concluding that it is false. On what statistical data should this be done? Braithwaite thought the matter so crucial that he tried to state the very meaning of ‘probability statements’ in terms of rules for their rejection. We shall examine his ideas later. First we must establish when evidence does justify rejection. To do so, it need not entail that the hypothesis is false. But what relations must it bear to the hypothesis?

Perhaps rejection covers two distinct topics. There have been many debates on this point, and it cannot be settled before further analysis. But a warning may be useful. An hypothesis may be rejected because of the evidence against it. This is my main subject. But situations can arise in which it is wise to reject an hypothesis even though there is little evidence against it. Suppose a great many hypotheses are under test. A good strategy for testing is one which rejects as many false and as few true hypotheses as possible. The best strategy might occasionally entail rejecting hypotheses even though there is little evidence against them. This sounds implausible, but examples will be given.

There is no general agreement on whether rejection should be studied in terms of evidence or strategies. I do not want to prejudge the issue. But I shall begin with examples in which an hypothesis should be rejected because of the evidence against it. I shall not begin with examples in which a great many similar hypotheses are under test. The logic of the two may be the same, for all that has been proved. But I shall not begin by assuming it.

The forthcoming discussion is, as usual, very academic. It concerns the relation of statistical hypotheses to statistical data. Generally one has all sorts of data bearing on an interesting statistical hypothesis, far more than merely statistical data. Hence one's problem is generally more complex than any to be discussed in this chapter. Here I deal only with data which may be precisely evaluated, and whose evaluation is peculiar to statistics.

Estimation theory is the most unsatisfactory branch of every school on the foundations of statistics. This is partly due to the unfinished state of our science, but there are general reasons for expecting the unhappy condition to continue. The very concept of estimation is ill adapted to statistics.

The theory of statistical estimation includes estimating particular characteristics of distributions and also covers guessing which possible distribution is the true one. It combines point estimation, in which the estimate of a magnitude is a particular point or value, and interval estimation, in which the estimate is a range of values which, it is hoped, will include the true one. Many of the questions about interval estimation are implicitly treated in the preceding chapter. A suitable interval for an interval estimate is one such that the hypothesis, that the interval includes the true values, is itself well supported. So problems about interval estimation fall under questions about measuring support for composite hypotheses. Thus the best known theories on interval estimation have been discussed already. The present chapter will treat some general questions about estimation, but will chiefly be devoted to defining one problem about point estimation. It is not contended that it is the only problem, but it does seem an important one, and one which has been little studied. When the problem has been set, the succeeding chapter will discuss how it might be solved. But neither chapter aims at a comprehensive survey of estimation theory; both aim only at clarifying one central problem about estimation.

Guesses, estimates, and estimators

An estimate, or at least a point estimate, is a more or less soundly based guess at the true value of a magnitude. The word ‘guess’ has a wider application than ‘estimate’, but we shall not rely on the verbal idiosyncracies of either. It must be noted, however, that in the very nature of an estimate, an estimate is supposed to be close to the true value of what it estimates. Since it is an estimate of a magnitude, the magnitude is usually measured along some scale, and this scale can be expected to provide some idea of how close the estimate is.

The problem of the foundation of statistics is to state a set of principles which entail the validity of all correct statistical inference, and which do not imply that any fallacious inference is valid. Much statistical inference is concerned with a special kind of property, and a good deal of the foundations depends upon its definition. Since no current definition is adequate, the next several chapters will present a better one.

Among familiar examples of the crucial property, a coin and tossing device can be so made that, in the long run, the frequency with which the coin falls heads when tossed is about 3/4. Overall, in the long run, the frequency of traffic accidents on foggy nights in a great city is pretty constant. More than 95% of a marksman's shots hit the bull's eye. No one can doubt that these frequencies, fractions, ratios, and proportions indicate physical characteristics of some parts of the world. Road safety programmes and target practice alike assume the frequencies are open to controlled experiment. If there are sceptics who insist that the frequency in the long run with which the coin falls heads is no property of anything, they have this much right on their side: the property has never been clearly defined. It is a serious conceptual problem, to define it.

The property need not be static. It is the key to many dynamic studies. In an epidemic the frequency with which citizens become infected may be a function of the number ill at the time, so that knowledge of this function would help to chart future ravages of the disease. Since the frequency is changing, we must consider frequencies over a fairly short period of time; perhaps it may even be correct to consider instantaneous frequencies but such a paradoxical conception must await further analysis.

First the property needs a name. We might speak of the ratio, proportion, fraction or percentage of heads obtained in coin tossing, but each of these words suggests a ratio within a closed class. It is important to convey the fact that whenever the coin is tossed sufficiently often, the frequency of heads is about 3/4. So we shall say, for the present, that the long run frequency is about 3/4.

Of what kind of thing is chance, or frequency in the long run, a property? Early writers may have conceived chances as properties of things like dice. Von Mises defines probability as the property of a sequence, while Neyman applies it to sets called fundamental probability sets. Fisher has an hypothetical infinite population in mind. But a more naïve answer stands out. The frequency in the long run of heads from a coin tossing device seems to be a property of the coin and device; the frequency in the long run of accidents on a stretch of highway seems to be a property of, in part, the road and those who drive upon it. We have no general name in English for this sort of thing. I shall use ‘chance set-up’. We also need a term corresponding to the toss of the coin and observing the outcome, and equally to the passage of a day on which an accident may occur. For three centuries the word ‘trial’ has been used in this sense, and I shall adopt it.

A chance set-up is a device or part of the world on which might be conducted one or more trials, experiments, or observations; each trial must have a unique result which is a member of a class of possible results.

A piece of radium together with a recording mechanism might constitute a chance set-up. One possible trial consists in observing whether or not the radium emits radiation in a small time interval. Possible results are ‘radiation’ and ‘none’. A pair of mice may provide a chance set-up, the trial being mating and the possible results the possible genetic make-ups of the offspring. The notion of a chance set-up is as old as the study of frequency. For Cournot, frequencies are properties of parts of the world, though he is careful not to say exactly what parts, in general. Venn's descriptions make it plain that he has a chance set-up in mind, and that it is this which leads him to the idea of an unending series of trials. Von Mises’ probability is a property of a series, but it is intended as a model of a property of what he calls an experimental set-up—I have copied the very word ‘set-up’ from his English translators.

This book analyses, from the point of view of a philosophical logician, the patterns of statistical inference which have become possible in this century. Logic has traditionally been the science of inference, but although a number of distinguished logicians have contributed under the head of probability, few have studied the actual inferences made by statisticians, or considered the problems specific to statistics. Much recent work has seemed unrelated to practical issues, and is sometimes veiled in a symbolism inscrutable to anyone not educated in the art of reading it. The present study is, in contrast, very much tied to current problems in statistics; it has avoided abstract symbolic systems because the subject seems too young and unstable to make them profitable. I have tried to discover the simple principles which underlie modern work in statistics, and to test them both at a philosophical level and in terms of their practical consequences. Technicalities are kept to a minimum.

It will be evident how many of my ideas come from Sir Ronald Fisher. Since much discussion of statistics has been coloured by purely personal loyalties, it may be worth recording that in my ignorance I knew nothing of Fisher before his death and have been persuaded to the truth of some of his more controversial doctrines only by piecing together the thought in his elliptic publications. My next debt is to Sir Harold Jeffreys, whose Theory of Probability remains the finest application of a philosophical understanding to the inferences made in statistics. At a more personal level, it is pleasant to thank the Master and Fellows of Peterhouse, Cambridge, who have provided and guarded the leisure in which to write. I have also been glad of a seminar consisting of Peter Bell, Jonathan Bennett, James Cargile and Timothy Smiley, who, jointly and individually, have helped to correct a great many errors. Finally I am grateful to R. B. Braithwaite for his careful study of the penultimate manuscript, and to David Miller for proof-reading.

Much of chapter 4 has appeared in the Proceedings of the Aristotelian Society for 1963–4, and is reprinted by kind permission of the Committee. The editor of the British Journal for the Philosophy of Science has authorized republication of some parts of my paper ‘On the Foundations of Statistics’, from volume xv.

Introduction. There is a fascinating interplay and overlap between recursion theory and descriptive set theory. A particularly beautiful source of such interaction has been Martin's conjecture on Turing invariant functions. This longstanding open problem in recursion theory has connected to many problems in descriptive set theory, particularly in the theory of countable Borel equivalence relations.

In this paper, we shall give an overview of some work that has been done on Martin's conjecture, and applications that it has had in descriptive set theory. We will present a long unpublished result of Slaman and Steel that arithmetic equivalence is a universal countable Borel equivalence relation. This theorem has interesting corollaries for the theory of universal countable Borel equivalence relations in general. We end with some open problems, and directions for future research.

Martin's conjecture. Martin's conjecture on Turing invariant functions is one of the oldest and deepest open problems on the global structure of the Turing degrees. Inspired by Sacks’ question on the existence of a degree-invariant solution to Post's problem [Sac66], Martin made a sweeping conjecture that says in essence, the only nontrivial definable Turing invariant functions are the Turing jump and its iterates through the transfinite.

Our basic references for descriptive set theory and effective descriptive set theory are the books of Kechris [Kec95] and Sacks [Sac90]. Let ≤T be Turing reducibility on the Cantor space ω2, and let ≡T be Turing equivalence. Given x ∈ω2, let x′ be the Turing jump of x. The Turing degree of a real x∈ω2 is the ≡T equivalence class of x. A Turing invariant function is a function such that for all reals x, y ∈ ω2, if x ≡T y, then f(x) ≡T f(y). The Turing invariant functions are those which induce functions on the Turing degrees.

With the axiom of choice, we can construct many pathological Turing invariant functions. Martin's conjecture is set in the context of ZF+DC+AD, where AD is the axiom of determinacy. We assume ZF+DC+AD for the rest of this section. The results we will discuss all “localize” so that the assumption of AD essentially amounts to studying definable functions assuming definable determinacy, for instance, Borel functions using Borel determinacy.