One of the most tattered volumes on my bookshelf is the Symposium on Stochastic Processes, reprinted from the 1949 volume of the Journal of the Royal Statistical Society, Series B. This consists of the three substantial and highly personal survey articles contributed to the symposium by Joseph Moyal, Maurice Bartlett and David Kendall. Together they provided magnificent reading and reference at a time when the stochastic process literature was a meagre scattered one, and continued to do so for many readers over many years. In doing so, they confirmed, if they did not indeed found, the recognisable British tradition in stochastic processes.

David Kendall's article in this volume, “Stochastic processes and population growth”, is only one of many he has written in the course of a distinguished career, and may well not be the one he prizes highest. I single it out, however, both for the seminal effect it had upon me personally, and because it gives an early example of his fascination with the capturing of an applied problem, having both intrinsic interest and an aspect of intangibility, in the meshes of a theory which is light but strong. David's trophies in this benign hunt are legion and living.

Introduction

There have been many models of competition between species, deterministic and stochastic. The most satisfactory of these are those in which the mechanism of competition is made explicit, as competition for limited resources. Deterministic versions of such models lead to the conclusion which O.M. Phillips (1973, 1974) terms the principle of competitive exclusion.

Question for a demy

I hope this salute to my old friend David Kendall will remind him of his Oxford days, for it commemorates an excellent scholarship question that he once set in the 1950s when he was the mathematics tutor at Magdalen. The gist of the question ran as follows. A spherical ball of unit radius rests on an infinite horizontal table. You may imagine that it is a globe with a map of the world painted on its surface to distinguish its spatial orientations. The state of the ball is specified by specifying both its spatial orientation and its position on the table. You have to transfer the ball from a given initial state to an arbitrary final state via a sequence of moves. Each move consists of rolling the ball along some straight line on the table: the length and direction of any move are at your disposal, but the rolling must be pure in the sense that the axis of rotation must be horizontal and there must be no slipping between ball and table. How many moves, N, will be necessary and sufficient to reach any final state?

The original version of the question, set for 18–year–old schoolboys, invited candidates to investigate how two moves, each of length π, would change the ball's orientation; and to deduce in the first place that N ≤ 11, and in the second place that N ≤ 7. Candidates scored bonus marks for any improvement on 7 moves.

Introduction

This is the first paper in a series devoted to Green's and Dirichlet spaces. In the next publications we shall study the spaces associated with fine Markov processes and with a certain class of multiparameter processes.

For the Brownian motion with exponential killing, the Dirichlet space is Sobolev's space H1 and Green's space is the dual space H−1. Both spaces are widely used in the theory of the free field (arising in quantum field theory). General Dirichlet and Green's spaces can be applied in an analogous way to Gaussian random fields associated with Markov processes [2].

Axiomatic theory of Dirichlet spaces was developed by Beurling and Deny [1]. Silverstein [5] and Fukushima [3] investigated the relation between Dirichlet spaces and Markov processes.

We start from a symmetric Markov transition function and we deal simultaneously with a pair: the Dirichlet space H and Green's space K. They are in a natural duality and they play symmetric roles but, in some respects, K is simpler than H. We consider several models for K and H. In particular, we represent them by L-valued functions of time t where L is a functional Hilbert space. We get the conventional representation of H by passage to the limit as t → ∞. Analogously, letting t → 0, we arrive at a representation of K by distributions (generalized functions).

Abstract

This paper provides a necessary and sufficient condition for a measure to be invariant for a Markov process. The condition is expressed in terms of the q-matrix assumed to generate the process.

Introduction

Let Q = (qij, i, j ∈ S) be a stable, conservative, regular and irreducible q-matrix over a countable state space S, and let P(t) = (Pij(t), i, j, ∈ S) be the matrix of transition probabilities of the Markov process determined by Q. If(the Markov process determined by) Q is recurrent then the relations

have a solution m = (mi, i ∈ S), unique up to constant multiples. Call m an invariant measure for P(t) if

When Q is positive recurrent it is known (Doob [5], Kendall and Reuter [13]) that a solution m to (1) is an invariant measure for P(t). This conclusion also holds when Q is null recurrent, but may not when Q is transient. When Q is transient the set of solutions to (1) may be empty or it may contain linearly independent elements: we obtain a necessary and sufficient condition for a given element of the set to be an invariant measure for P(t).

The basic properties of Markov processes which will be needed are taken from Kendall [11] and are briefly stated in Section 2: they can also be found in [3], [6], [10], [12], [13] and [17]. Section 3 contains the main result of the present paper. Here it is shown that a solution to (1) is an invariant measure for P(t) if and only if a time-reversed q-matrix, defined in terms of m and Q, is regular. It is convenient to obtain the result assuming only that Q is stable and conservative, with P(t) the minimal (Feller) transition matrix determined by Q.

Most of the papers compiled in this volume have been published in Uspekhi Matematicheskikh Nauk and translated into English in the Russian Mathematical Surveys. The core consists of the series [IV], [V], [VI], [VII] presenting a new approach to Markov processes (especially to the Martin boundary theory and the theory of duality) with the following distinctive features:

1. The general non-homogeneous theory precedes the homogeneous one. This is natural because non-homogeneous Markov processes are invariant with respect to all monotone transformations of time scale – a property which is destroyed in the homogeneous case by the introduction of an additional structure: a one-parameter semi-group of shifts. In homogeneous theory, the probabilistic picture is often obscured by the technique of Laplace transforms.

2. All the theory is invariant with respect to time reversion. We consider processes with random birth and death times and we use on equal terms the forward and backward transition probabilities, i.e., the conditional probability distributions of the future after t and of the past before t given the state at time t. (This is an alternative to introducing a pair of processes in duality defined on different sample spaces.)

3. […]

Introduction.

In 1941 R. S. Martin [13] published a paper where positive harmonic functions in a domain D of a Euclidean space were investigated. Let H stand for the class of all such functions subject to condition f(a) < ∞ where a is a fixed point of D. Martin has proved that:

1. (a) each element of H can be decomposed in a unique way into minimal elements normalized by the condition f(a) = 1;

2. (b) if the Green function of the Laplacian in D is known, then all minimal elements can be computed by a certain limit process.

J. L. Doob [2] has discovered that the Martin decomposition of harmonic functions is closely related to the behaviour of Brownian paths at the first exit time from D. G. A. Hunt [9] has shown that, using these relations, it is possible to get Martin's results by probabilistic considerations. Actually only discrete Markov chains were treated in [1] and [5], however, the methods are applicable to Brownian motion as well.

The intimate connection between Markov processes and problems in analysis has been apparent ever since the theory of the former began to develop. It is not without reason that A. N. Kolmogorov's paper [39] (Russian translation [38]) of 1931, which is of fundamental importance in this domain, was entitled “On analytical methods in probability theory”. The investigation of these connections also forms, to a large extent, the subject matter of A. Ya. Khinchin's book of 1933 on “Asymptotic laws of the theory of probability” [52] (Russian translation [51]).

In the fifties, and more particularly during the last five years, the theory of Markov processes entered a new period of intense growth. If previously the connections between probability theory and analysis were somewhat one-sided, probability theory applying results and methods of analysis, now the opposite tendency increasingly asserts itself, and probabilistic methods are applied to the solution of problems of analysis. Methods belonging to the theory of probability not only suggest a heuristic approach, but also, in many cases, yield rigourous proofs of analytic results. Applications of the methods of the theory of semigroups of linear operators have led to far-reaching advances in the classification of wide classes of Markov processes. New and deep connections between the theory of Markov processes and potential theory have been discovered. The foundations of the theory have been critically re-examined; the new concept of a strongly Markovian process has acquired a crucial importance in the whole theory of Markov processes.

This article is concerned with the foundations of the theory of Markov processes. We introduce the concepts of a regular Markov process and the class of such processes. We show that regular processes possess a number of good properties (strong Markov character, continuity on the right of excessive functions along almost all trajectories, and so on). A class of regular Markov processes is constructed by means of an arbitrary transition function (regular re-construction of the canonical class). We also prove a uniqueness theorem.

We diverge from tradition in three respects:

1. a) we investigate processes on an arbitrary random time interval;

2. b) all definitions and results are formulated in terms of measurable structures without the use of topology (except for the topology of the real line);

3. c) our main objects of study are non-homogeneous processes (homogeneous ones are discussed as an important special case).

In consequence of a), the theory is highly symmetrical: there is no longer disparity between the birth time α of the process, which is usually fixed, and the death time β, which is considered random.

Principle b) does not prevent us from introducing, when necessary, various topologies in the state space (as systems of coordinates are introduced in geometry). However, it is required that the final statements should be invariant with respect to the choice of such a topology.

Finally, the main gain from c) is simplification of the theory: discarding the “burden of homogeneity” we can use constructions which, generally speaking, destroy this homogeneity.

Similar questions have been considered (for the homogeneous case) by Knight [8], Doob [2], [3] and other authors.