A key matter in many (most?) practical applications of Markov theory is the ability to simulate Markov chains on a computer. This chapter deals with how that can be done.

We begin by stating a lie:

This is a lie for at least two reasons:

1. (A) The numbers U0, U1, … obtained from random number generators are not uniformly distributed on [0, 1]. Typically, they have a finite binary (or decimal) expansion, and are therefore rational. In contrast, it can be shown that a random variable which (truly) is uniformly distributed on [0, 1] (or in fact any continuous random variable) is irrational with probability 1.

2. (B) U0, U1, … are not even random! Rather, they are obtained by some deterministic procedure. For this reason, random number generators are sometimes (and more accurately) called pseudo-random number generators.

The most important of these objections is (B), because (A) tends not to be a very big problem when the number of binary or decimal digits is reasonably large (say, 32 bits). Over the decades, a lot of effort has been put into constructing (pseudo-)random number generators whose output is as indistinguishable as possible from a true i.i.d. sequence of uniform [0, 1] random variables.