This book is about setting up and analyzing economic models for large collections of interacting agents. We describe models in terms of states: as stationary distribution of states, or dynamics for time evolution of states. We may speak informally also of patterns of partitions of the set of agents by types or categories, or configurations, and how they or some functions of them evolve with time. Our models are usually specified as jump Markov processes, that is, Markov processes with a finite or at most countable number of states, and time running continuously.

In setting up a stochastic model for a collection of agents, then, we first choose a set of variables as a state vector for the model. The state vector should carry enough information about the model for the purpose at hand, so that we can, in principle, calculate the conditional distributions of future state vectors, given the current one. Put differently, we must be able to calculate the conditional probability distributions of the model state vector at least for a small step forward in time, given current values of the state vector and time paths of exogenous variables.

This dynamic aspect of the model is described by the master equation, which is introduced in Chapter 3. Briefly, the master equation is the differential (or difference) equation that indicates how the probability of the model being at some specific state at a point in time is changed by the inflows and outflows of the probability fluxes.