We now look more deeply into the matter of learning from experience, where a pair of probability assignments represents your judgments before and after you change your mind in response to the result of experiment or observation. We start with the simplest, most familiar mode of updating, which will be generalized in sec. 3.2 and applied in sec. 3.3 and 3.4 to the problem of learning from other people's experience-driven probabilistic updating.

Conditioning

Suppose your judgments were once characterized by an assignment (“old”) which describes all of your conditional and unconditional probabilities as they were then. And suppose you became convinced of the truth of some data statement D. That would change your probabilistic judgments; they would no longer corrrespond to your prior assignment old, but to some posterior assignment new, where you are certain of D's truth.

Certainty:new(D) = 1

How should your new probabilities for hypotheses H other than D be related to your old ones?

The simplest answer goes by the name of “conditioning” (or “conditionalization”) on D.

Conditioning:new(H) = old(H|D)

This means that your new unconditional probability for any hypothesis H will simply be your old conditional probability for H given D.

When is it appropriate to update by conditioning on D? It is easy to see–once you think of it–that certainty about D is not enough.

Here is an account of basic probability theory from a thoroughly “subjective” point of view, according to which probability is a mode of judgment. From this point of view probabilities are “in the mind”–the subject's, say, yours. If you say the probability of rain is 70% you are reporting that, all things considered, you would bet on rain at odds of 7:3, thinking of longer or shorter odds as giving an unmerited advantage to one side or the other. A more familiar mode of judgment is flat, “dogmatic” assertion or denial, as in “It will rain” or “It will not rain”. In place of this “dogmatism”, the probabilistic mode of judgment offers a richer palate for depicting your state of mind, in which the colors are all the real numbers from 0 to 1. The question of the precise relationship between the two modes is a delicate one, to which I know of no satisfactory detailed answer.

Chapter 1, “Probability Primer,” is an introduction to basic probability theory, so conceived. The object is not so much to enunciate the formal rules of the probability calculus as to show why they must be as they are, on pain of inconsistency.

Chapter 2, “Testing Scientific Theories,” brings probability theory to bear on vexed questions of scientific hypothesis-testing. It features Jon Dorling's “Bayesian” solution of Duhem's problem (and Quine's), the dreaded holism.

Where Do Probabilities Come from?

Your “subjective” probability is not something fetched out of the sky on a whim; it is what your actual judgment should be, in view of your information to date and of your sense of other people's information, even if you do not regard it as a judgment that everyone must share on pain of being wrong in one sense or another.

But of course you are not always clear about what your judgment is, or should be. The most important questions in the theory of probability concern ways and means of constructing reasonably satisfactory probability assignments to fit your present state of mind. (Think: trying on shoes.) For this, there is no overarching algorithm. Here we examine two answers to these questions that were floated by Bruno de Finetti in the decade from (roughly) 1928 to 1938. The second of them, “Exchangeability”, postulates a definite sort of initial probabilistic state of mind, which is then updated by conditioning on statistical data. The first (“Minimalism”) is more primitive: The input probability assignment will have large gaps, and the output will not arise via conditioning.

Probabilities from Statistics: Minimalism

Statistical data are a prime determinant of subjective probabilities; that is the truth in frequentism. But that truth must be understood in the light of certain features of judgmental probabilizing.