Most of the papers compiled in this volume have been published in Uspekhi Matematicheskikh Nauk and translated into English in the Russian Mathematical Surveys. The core consists of the series [IV], [V], [VI], [VII] presenting a new approach to Markov processes (especially to the Martin boundary theory and the theory of duality) with the following distinctive features:

1. The general non-homogeneous theory precedes the homogeneous one. This is natural because non-homogeneous Markov processes are invariant with respect to all monotone transformations of time scale – a property which is destroyed in the homogeneous case by the introduction of an additional structure: a one-parameter semi-group of shifts. In homogeneous theory, the probabilistic picture is often obscured by the technique of Laplace transforms.

2. All the theory is invariant with respect to time reversion. We consider processes with random birth and death times and we use on equal terms the forward and backward transition probabilities, i.e., the conditional probability distributions of the future after t and of the past before t given the state at time t. (This is an alternative to introducing a pair of processes in duality defined on different sample spaces.)

3. […]

Introduction.

In 1941 R. S. Martin [13] published a paper where positive harmonic functions in a domain D of a Euclidean space were investigated. Let H stand for the class of all such functions subject to condition f(a) < ∞ where a is a fixed point of D. Martin has proved that:

1. (a) each element of H can be decomposed in a unique way into minimal elements normalized by the condition f(a) = 1;

2. (b) if the Green function of the Laplacian in D is known, then all minimal elements can be computed by a certain limit process.

J. L. Doob [2] has discovered that the Martin decomposition of harmonic functions is closely related to the behaviour of Brownian paths at the first exit time from D. G. A. Hunt [9] has shown that, using these relations, it is possible to get Martin's results by probabilistic considerations. Actually only discrete Markov chains were treated in [1] and [5], however, the methods are applicable to Brownian motion as well.

The intimate connection between Markov processes and problems in analysis has been apparent ever since the theory of the former began to develop. It is not without reason that A. N. Kolmogorov's paper [39] (Russian translation [38]) of 1931, which is of fundamental importance in this domain, was entitled “On analytical methods in probability theory”. The investigation of these connections also forms, to a large extent, the subject matter of A. Ya. Khinchin's book of 1933 on “Asymptotic laws of the theory of probability” [52] (Russian translation [51]).

In the fifties, and more particularly during the last five years, the theory of Markov processes entered a new period of intense growth. If previously the connections between probability theory and analysis were somewhat one-sided, probability theory applying results and methods of analysis, now the opposite tendency increasingly asserts itself, and probabilistic methods are applied to the solution of problems of analysis. Methods belonging to the theory of probability not only suggest a heuristic approach, but also, in many cases, yield rigourous proofs of analytic results. Applications of the methods of the theory of semigroups of linear operators have led to far-reaching advances in the classification of wide classes of Markov processes. New and deep connections between the theory of Markov processes and potential theory have been discovered. The foundations of the theory have been critically re-examined; the new concept of a strongly Markovian process has acquired a crucial importance in the whole theory of Markov processes.

This article is concerned with the foundations of the theory of Markov processes. We introduce the concepts of a regular Markov process and the class of such processes. We show that regular processes possess a number of good properties (strong Markov character, continuity on the right of excessive functions along almost all trajectories, and so on). A class of regular Markov processes is constructed by means of an arbitrary transition function (regular re-construction of the canonical class). We also prove a uniqueness theorem.

We diverge from tradition in three respects:

1. a) we investigate processes on an arbitrary random time interval;

2. b) all definitions and results are formulated in terms of measurable structures without the use of topology (except for the topology of the real line);

3. c) our main objects of study are non-homogeneous processes (homogeneous ones are discussed as an important special case).

In consequence of a), the theory is highly symmetrical: there is no longer disparity between the birth time α of the process, which is usually fixed, and the death time β, which is considered random.

Principle b) does not prevent us from introducing, when necessary, various topologies in the state space (as systems of coordinates are introduced in geometry). However, it is required that the final statements should be invariant with respect to the choice of such a topology.

Finally, the main gain from c) is simplification of the theory: discarding the “burden of homogeneity” we can use constructions which, generally speaking, destroy this homogeneity.

Similar questions have been considered (for the homogeneous case) by Knight [8], Doob [2], [3] and other authors.

A great deal of research into the theory of random processes is concerned with the problem of constructing a process that has certain properties of regularity of the trajectories and has the same finite-dimensional probability distribution as a given stochastic process xt. It is a complicated theory and one that is difficult to apply to those properties that we most need for the study of Markov processes (the strong Markov property, quasi-left-continuity, and the like.)

The problem can be usefully reformulated. In an actual experiment we do not observe the state xt at a fixed instant t, but rather events that occupy certain time intervals. This is the motivation behind the Gel'fand-Itô theory of generalized random processes. Kolmogorov, in 1972, proposed an even more general concept of a stochastic process as a system of σ-algebras ℱ(I) labelled by time intervals I. Developing this approach, we introduce the concept of a Markov representation xt of the stochastic system ℱ (I) and prove the existence of regular representations. We construct two dual regular representations (the right and the left), which we then combine into a single Markov process by two methods, the “vertical” and the “horizontal” method. We arrive at a general duality theory, which provides a natural framework for the fundamental results on entrance and exit spaces, excessive measures and functions, additive functionals, and others. The initial steps in the construction of this theory were taken in [6]. The note [5] deals with applications to additive functionals (detailed proofs are in preparation). We consider random processes defined in measurable spaces without any topology: the introduction of a reasonable topology allows of a certain arbitrariness.

These notes grew out of M. Sc. lecture courses given by the first named author at the Mathematics Institute (University of Warwick) in 1976 and 1977. To be more precise, the material presented here is concerned only with those parts of the courses which are not to be found in well known texts, together with additional material, organised by the second named author, which was not presented in the above courses.

The M. Sc. audience was expected to be familiar with measure and integration theory and it was assumed that students had had at least some contact with elementary functional analysis including Hilbert space theory. The foundations of the course, which were discussed in an informal tutorial fashion, consisted roughly of the following topics: measure-preserving transformations, recurrence, Birkhoff's and von Neumann's ergodic theorems, conditional expectation, increasing and decreasing sequences of σ-algebras and the associated Martingale theorems, information, entropy and the Shannon-McMillan-Breiman theorem. Students were expected to read these topics as an integral part of the course and were advised to refer to the relevant sections of [H], [R. 2], [W. l] (to which we would now add [P. l]). Readers of these notes who are unfamiliar with these foundations are similarly advised.

We offer our thanks to M. Keane and M. Smorodinsky for a number of consultations concerning Chapter III. We would also like to record our gratitude to Klaus Schmidt and Peter Walters for helpful critical comments concerning an earlier draft of these notes.