Elementary rules of probability theory

It is possible to begin a discussion of probability from different starting points. But because, in the end, comparison with real empirical data is the only test of a theory or model, we adopt only one, the empirical definition of probability based upon the law of large numbers. Given an event with possible outcomes A1, A2, …, AN, the probability for Ak is pk ≈ nk/N, where N is the number of repeated identical experiments or observations and nk is the number of times that the event Ak is observed to occur.We point out in the next section that the empirical definition of probability agrees with the formal measure theoretic definition. For equally probable events p = 1/N. For mutually exclusive events (Gnedenko, 1967; Gnedenko and Khinchin, 1962) A and B probabilities add, P(A or B) = P(A) + P(B). For example, the probability that a coin lands heads plus the probability that it does not land heads adds to unity (total probability is normalized to unity in this text). For a complete (i.e. exhaustive) set of mutually exclusive alternatives {Ak}, we have Σ P(Ak) = 1. For example, in die tossing, if pk is the probability for the number k to show, where 1 ≤ k ≤ 6, then the p1 + p2 + p3 + p4 + p5 + p6 = 1. For a fair die tossed fairly, pk = 1/6.

For statistically independent events A and B the probabilities multiply, p(A and B) = p(A)p(B). For example, for two successive fair tosses of a fair coin (p = 1/2) the probability to get two heads is p2 = (1/2)2.