2.1 Notation

Throughout this paper, we consider the standard measurable space $(\mathbb {X}^{\mathbb {Z}},\mathcal {X}^{\mathbb {Z}})$ of two-sided infinite sequences over a finite alphabet $\mathbb {X}=\left \{ a_1,\dots ,a_D \right \}$ , where $D\ge 2$ . (Occasionally, we also apply the space of one-sided infinite sequences $(\mathbb {X}^{\mathbb {N}},\mathcal {X}^{\mathbb {N}})$ .) The points of the space are (infinite) sequences $x=(x_i)_{i\in \mathbb {Z}}\in \mathbb {X}^{\mathbb {Z}}$ . We also denote (finite) strings $x_j^k=(x_i)_{j\le i\le k}$ , where $x_j^{j-1}=\lambda $ equals the empty string. By $\mathbb {X}^*=\bigcup _{n\ge 0}\mathbb {X}^n$ we denote the set of strings of an arbitrary length including the singleton $\mathbb {X}^0=\left \{ \lambda \right \}$ . We use random variables $X_k((x_i)_{i\in \mathbb {Z}}):=x_k$ . Having these, the $\sigma $ -field $\mathcal {X}^{\mathbb {Z}}$ is generated by cylinder sets $(X_{-|\sigma |+1}^{|\tau |}=\sigma \tau )$ for all $\sigma ,\tau \in \mathbb {X}^*$ . We tacitly assume that P and R denote probability measures on $(\mathbb {X}^{\mathbb {Z}},\mathcal {X}^{\mathbb {Z}})$ . For any probability measure P, we use the shorthand notations $P(x_1^n):=P(X_1^n=x_1^n)$ and $P(x_j^n|x_1^{j-1}):=P(X_j^n=x_j^n|X_1^{j-1}=x_1^{j-1})$ . Notation $\log x$ denotes the binary logarithm, whereas $\ln x$ is the natural logarithm.

2.2 Stationary and ergodic measures

Let us denote the measurable shift operation $T((x_i)_{i\in \mathbb {Z}}):=(x_{i+1})_{i\in \mathbb {Z}}$ for two-sided infinite sequences ${(x_i)_{i\in \mathbb {Z}}\in \mathbb {X}^{\mathbb {Z}}}$ .

The class of stationary ergodic probability measures has various nice properties guaranteed by the collection of fundamental results called ergodic theorems. Typically stationary ergodic measures are not computable (e.g., consider independent biased coin tosses with a common uncomputable bias) but they allow for computable universal coding and computable universal prediction schemes can achieve optimal error rates, as it will be explained in Section 3.

2.3 Sorts of randomness

Now let us discuss some computability notions. In the following, computably enumerable is abbreviated as c.e. Given a real r, the set $\left \{ q\in \mathbb {Q}:q<r \right \}$ is called the left cut of r. A real function f with arguments in a countable set is called computable or left-c.e. respectively if the left cuts of $f(\sigma )$ are uniformly computable or c.e. given an enumeration of $\sigma $ . For an infinite sequence $s\in \mathbb {X}^{\mathbb {Z}}$ , we say that real functions f are s-computable or s-left-c.e. if they are computable or left-c.e. with oracle s. Similarly, for a real function f taking arguments in $\mathbb {X}^{\mathbb {Z}}$ , we will say that f is s-computable or s-left-c.e. if left cuts of $f(x)$ are uniformly computable or c.e. with oracles $x\oplus s:=(\ldots ,x_{-1},s_{-1},x_0,s_0,x_1,s_1,\ldots )$ . This induces in effect s-computable and s-left-c.e. random variables and stochastic processes on $(\mathbb {X}^{\mathbb {Z}},\mathcal {X}^{\mathbb {Z}})$ , where the values of an s-computable (s-left-c.e.) variable on a point x are $(x\oplus s)$ -computable (s-left-c.e.) uniformly in x and the values of s-computable (s-left-c.e.) process $X_{i}$ (with natural or integer i) are s-computable (s-left-c.e.).

For stationary ergodic measures, we need a definition of algorithmically random points with respect to an arbitrary, i.e., not necessarily computable probability measure on $(\mathbb {X}^{\mathbb {Z}},\mathcal {X}^{\mathbb {Z}})$ . A simple definition thereof was proposed by Reimann [Reference Reimann42] and Reimann and Slaman [Reference Reimann and Slaman43]. This definition is equivalent to earlier approaches by Levin [Reference Levin32–Reference Levin34] and Gács [Reference Gács23] as shown by Day and Miller [Reference Day and Miller13] and we will use it since it leads to straightforward generalizations of the results in Section 2.4. The definition is based on measure representations. Let $\mathcal {P}(\mathbb {X}^{\mathbb {Z}})$ be the space of probability measures on $(\mathbb {X}^{\mathbb {Z}},\mathcal {X}^{\mathbb {Z}})$ . A measure $P\in \mathcal {P}(\mathbb {X}^{\mathbb {Z}})$ is called s-computable if real function $(\sigma ,\tau )\mapsto P(X_{-|\sigma |+1}^{|\tau |}=\sigma \tau )$ is s-computable. Similarly, a representation function is a function $\rho :\mathbb {X}^{\mathbb {Z}}\rightarrow \mathcal {P}(\mathbb {X}^{\mathbb {Z}})$ such that real function $(\sigma ,\tau ,s)\mapsto \rho (s)(X_{-|\sigma |+1}^{|\tau |}\,{=}\,\sigma \tau )$ is computable. Subsequently, we say that an infinite sequence $s\in \mathbb {X}^{\mathbb {Z}}$ is a representation of measure P if there exists a representation function $\rho $ such that $\rho (s)=P$ . We note that any measure P is s-computable for any representation s of P.

We will consider two important sorts of algorithmically random points: Martin-Löf or 1-random points and weakly 2-random points with respect to an arbitrary stationary ergodic measure P on $(\mathbb {X}^{\mathbb {Z}},\mathcal {X}^{\mathbb {Z}})$ . Note that the following notions are typically defined for one-sided infinite sequences over the binary alphabet and computable measures P. In the following parts of this paper, let an infinite sequence $s\in \mathbb {X}^{\mathbb {Z}}$ be a representation of measure P.

Definition 2.3. A collection of events $U_1,U_2,\ldots \in \mathcal {X}^{\mathbb {Z}}$ is called uniformly s-c.e. if and only if there is a collection of sets $V_1,V_2,\ldots \subset \mathbb {X}^*\times \mathbb {X}^*$ such that

$$ \begin{align*}U_i=\left\{ x\in\mathbb{X}^{\mathbb{Z}}:\exists (\sigma,\tau)\in V_i: x_{-|\sigma|+1}^{|\tau|}=\sigma\tau \right\}\end{align*} $$

and sets $V_1,V_2,\ldots $ are uniformly s-c.e.

Subsequently, an event $C\in \mathcal {X}^{\mathbb {Z}}$ is called a $\Sigma ^0_2(s)$ event if there exists a uniformly s-c.e. sequence of events $U_1,U_2,\ldots $ such that $\mathbb {X}^{\mathbb {Z}}\setminus C=\bigcap _{i\ge 1} U_i$ .

The sets of weakly $2$ -random points are strictly smaller than the respective sets of $1$ -random points (see [Reference Downey and Hirschfeldt18]).

In general, there is a whole hierarchy of algorithmically random points, such as (weakly) n-random points, where n runs over natural numbers. For our purposes, however, only $1$ -random points and weakly $2$ -random points matter since the following proposition sets the baseline for effectivization:

The above proposition is obvious since the set of points on which limit $\lim _{n\to \infty }Y_n$ exists is a $\Sigma ^0_2(s)$ event. The effectivization program aims to strengthen the above claim to $1$ -P-random points (or even weaker notions such as Schnorr randomness) but this need not always be feasible. In particular, one can observe that:

This fact is a simple consequence of the existence of $\Delta ^0_2$ $1$ -P-random sequences (for a computable P) and may be also interpreted in terms of learning theory (cf. [Reference Osherson and Weinstein41] and the upcoming paper [Reference Steifer52]).

2.4 Known effectivizations

Many probabilistic theorems have been effectivized so far. Usually they were stated for computable measures but their generalizations for uncomputable measures follow easily by relativization, i.e., putting a representation s of measure P into the oracle. In this section, we list several known effectivizations of almost sure theorems which we will use further.

As shown by Solovay (unpublished; see [Reference Downey and Hirschfeldt18]), we have this effective version of the Borel–Cantelli lemma:

By the effective Borel–Cantelli lemma, Proposition 2.9 follows the effective version of the Barron lemma [Reference Barron5, Theorem 3.1]:

Proposition 2.10 Effective Barron lemma

For any probability measure P and any s-computable probability measure R, on $1$ - $(s,P)$ -random points we have

(2) $$ \begin{align} \lim_{n\to\infty} \left[ -\log R(X_1^n)+\log P(X_1^n)+2\log n \right]=\infty. \end{align} $$

In the following, we make an easy but important observation—probabilities conditioned on an infinite past are defined on random points. First, we need to recall the notion of a martingale process and prove an effective version of Doob’s martingale convergence.

The proof of Doob’s martingale convergence can be easily made effective. This was already observed by Takahashi [Reference Takahashi54], who stated the effective martingale convergence for a specific filtration generated by cylinders $X_1^n$ . The following upcrossing inequality can be used to define a test which enforces convergence.

Proposition 2.12 Doob upcrossing inequality

Let $(X_i)_{i\in \mathbb {N}}$ be a martingale process and let $C_n$ will be the random variable denoting the number of upcrossings of interval $[a,b]\ ($ with $a,b\in \mathbb {R})$ by time n and suppose that $\sup _n\operatorname {\mathrm {\textbf {E}}}(|X_n|)<\infty $ . Then for each n, we have

$$\begin{align*}\operatorname{\mathrm{\textbf{E}}}\left( \sup_n C_n \right)\leq \frac{|a|+\sup_n\operatorname{\mathrm{\textbf{E}}}(|X_n|)}{b-a} .\end{align*}$$

Proof Suppose that process $(X_i)_{i\in \mathbb {N}}$ does not converge on some random point x. Then there exist rational $a,b$ such that the number of upcrossings of the interval $[a,b]$ by $X_i(x)$ is infinite. Let $C_n$ be the random variable denoting the number of upcrossings of interval $[a,b]$ by the process $(X_i)_{i\in \mathbb {N}}$ by the time n. Let $C_{\infty }$ denote $\sup _n C_n$ and let $f:\mathbb {N}\rightarrow \mathbb {N}$ be a monotonic function. Consider a collection of sets $U_1,U_2,\ldots $ such that for all $i>0$

$$\begin{align*}U_i=\{\omega: C_{\infty}(\omega)>f(n)\}.\end{align*}$$

By Proposition 2.12 (Doob upcrossing inequality) and the Markov inequality, we have

$$\begin{align*}P(U_i)\leq \frac{|a|+\sup_{n}\operatorname{\mathrm{\textbf{E}}}(|X_n|)}{f(n)(b-a)}.\end{align*}$$

Note that if f grows sufficiently fast, then $\sum ^{\infty }_{i=1}P(U_i)$ converges. Moreover, the collection of sets $U_1,U_2,\ldots $ is uniformly s-c.e. It follows by Proposition 2.9 (effective Borel–Cantelli lemma) that $C_{\infty }(x)<\infty $ for every $1$ - $(s,P)$ -random point x, which is a contradiction.

It remains to observe that the limit of $(X_i)_{i\in \mathbb {N}}$ is finite. This follows easily if one considers the collection of sets $V_1,V_2,\ldots $ with

$$\begin{align*}V_i=\{x:\sup_{n}X_n(x)>2^n\},\end{align*}$$

which are uniformly s-c.e. By the Markov inequality and the monotone convergence theorem, we have $P(V_i)\leq 2^{-i}\sup _{n}\operatorname {\mathrm {\textbf {E}}}(X_n)$ . We apply Proposition 2.9 to conclude that $X_n$ are bounded on every $1$ - $(s,P)$ -random point.

Random variables $P(x_0|X_{-n}^{-1})$ for $n\ge 1$ form a uniformly s-computable martingale process with respect to the filtration generated by cylinder sets $X_{-n}^{-1}$ for any representation s of P. Thus applying the effective Doob martingale convergence, we obtain an effective version of the Lévy law in particular. In this work, our attention is limited to the following form.

Proposition 2.14 Effective Lévy law

On $1$ -P-random points there exist limits

(3) $$ \begin{align} P(x_0|X_{-\infty}^{-1}):=\lim_{n\to\infty} P(x_0|X_{-n}^{-1}). \end{align} $$

Now let us proceed to a celebrated result of the algorithmic randomness theory, which is the effective Birkhoff ergodic theorem [Reference Bienvenu, Day, Hoyrup, Mezhirov and Shen6, Reference Franklin, Greenberg, Miller and Ng21, Reference Hoyrup and Rojas28, Reference Hoyrup and Rojas29, Reference Nandakumar39, Reference V’yugin55]. In the following, $\operatorname {\mathrm {\textbf {E}}} X:=\int X dP$ stands for the expectation of a random variable X with respect to measure P.

Proposition 2.15 Effective Birkhoff ergodic theorem [Reference Bienvenu, Day, Hoyrup, Mezhirov and Shen6, Theorem 10]

For a stationary ergodic probability measure P and an s-left-c.e. real random variable G such that $G\ge 0$ and $\operatorname {\mathrm {\textbf {E}}} G<\infty $ , on $1$ - $(s,P)$ -random points we have

(4) $$ \begin{align} \lim_{n\to\infty}\frac{1}{n}\sum_{i=0}^{n-1} G\circ T^i = \operatorname{\mathrm{\textbf{E}}} G. \end{align} $$

We note in passing that if a point is not $1$ -random for a computable P then (4) fails on this point for some computable real random variable G and some computable transformation T [Reference Franklin and Towsner22].

The proof of the next proposition is an easy application of Proposition 2.15 and properties of left-c.e. functions.

Proposition 2.16 Effective Breiman ergodic theorem [Reference Steifer51]

For a stationary ergodic probability measure P and uniformly s-computable real random variables $(G_i)_{i\ge 0}$ such that $G_n\ge 0$ , $\operatorname {\mathrm {\textbf {E}}} \sup _n G_n<\infty $ , and limit $\lim _{n\to \infty } G_n$ exists P-almost surely, on $1$ - $(s,P)$ -random points we have

(5) $$ \begin{align} \lim_{n\to\infty}\frac{1}{n}\sum_{i=0}^{n-1} G_i\circ T^i = \operatorname{\mathrm{\textbf{E}}}\lim_{n\to\infty} G_n. \end{align} $$

Proof Let $H_k:=\sup _{t>k}G_t\ge 0$ . Then $G_t\le H_k$ for all $t>k$ and consequently,

(6) $$ \begin{align} \limsup_{n\to\infty}\frac{1}{n}\sum_{i=0}^{n-1}G_i\circ T^i\le \limsup_{n\to\infty}\frac{1}{n}\sum_{i=0}^{n-1}H_k\circ T^i. \end{align} $$

Observe that the supremum $H_k$ of uniformly s-computable functions $G_{k+1},G_{k+2},\ldots $ is s-left-c.e. Indeed, to enumerate the left cut of the supremum $H_k(x)$ we simultaneously enumerate the left cuts of $G_{k+1}(x),G_{k+2}(x),\ldots $ . This is possible since every s-computable function is also s-left-c.e. Moreover, we are considering only countably many functions, and hence we can guarantee that an element of each left cut appears in the enumeration at least once.

Now, since for all $k\ge 0$ random variables $H_k$ are s-left-c.e. then by Theorem 2.15 (effective Birkhoff ergodic theorem), on $1$ - $(s,P)$ -random points we have

(7) $$ \begin{align} \limsup_{n\to\infty}\frac{1}{n}\sum_{i=0}^{n-1}G_i\circ T^i\le \operatorname{\mathrm{\textbf{E}}} H_k. \end{align} $$

Since $H_k\ge 0$ and $\operatorname {\mathrm {\textbf {E}}} \sup _k H_k<\infty $ then by the dominated convergence,

(8) $$ \begin{align} \inf_{k\ge 0} \operatorname{\mathrm{\textbf{E}}} H_k= \lim_{k\to\infty}\operatorname{\mathrm{\textbf{E}}} H_k= \operatorname{\mathrm{\textbf{E}}}\lim_{k\to\infty}H_k= \operatorname{\mathrm{\textbf{E}}}\lim_{n\to\infty}G_n. \end{align} $$

Thus,

(9) $$ \begin{align} \limsup_{n\to\infty}\frac{1}{n}\sum_{i=0}^{n-1}G_i\circ T^i\le \operatorname{\mathrm{\textbf{E}}}\lim_{n\to\infty}G_n. \end{align} $$

For the converse inequality, consider a natural number M and put random variables $\bar H_k:=M-\inf _{t>k}\min \left \{ G_t,M \right \}\in [0,M]$ . We observe that $\bar H_k$ are also s-left-c.e. since $G_t$ are uniformly s-computable by the hypothesis. By Theorem 2.15 (effective Birkhoff ergodic theorem), on $1$ - $(s,P)$ -random points we have

(10) $$ \begin{align} M-\liminf_{n\to\infty}\frac{1}{n}\sum_{i=0}^{n-1}\min\left\{ G_i,M \right\}\circ T^i\le \operatorname{\mathrm{\textbf{E}}} \bar H_k. \end{align} $$

Since $0\le \bar H_k\le M$ then by the dominated convergence,

(11) $$ \begin{align} \inf_{k\ge 0} \operatorname{\mathrm{\textbf{E}}} \bar H_k= \lim_{k\to\infty}\operatorname{\mathrm{\textbf{E}}} \bar H_k= \operatorname{\mathrm{\textbf{E}}}\lim_{k\to\infty}\bar H_k= M-\operatorname{\mathrm{\textbf{E}}}\min\left\{ \lim_{n\to\infty}G_n,M \right\}. \end{align} $$

Hence, regrouping the terms we obtain

(12) $$ \begin{align} \liminf_{n\to\infty}\frac{1}{n}\sum_{i=0}^{n-1}G_i\circ T^i &\ge \liminf_{n\to\infty}\frac{1}{n}\sum_{i=0}^{n-1}\min\left\{ G_i,M \right\}\circ T^i \nonumber\\ &\ge \operatorname{\mathrm{\textbf{E}}}\min\left\{ \lim_{n\to\infty}G_n,M \right\} \xrightarrow[M\to\infty]{} \operatorname{\mathrm{\textbf{E}}}\lim_{n\to\infty}G_n, \end{align} $$

where the last transition follows by the monotone convergence. By (9) and (12) we derive the claim.

The almost sure versions of Propositions 2.15 and 2.16 concern random variables which need not be nonnegative [Reference Breiman7].

An important result for universal prediction is the Azuma inequality [Reference Azuma3], whose following corollary will be used in Sections 3.2 and 3.4.

Theorem 2.17 Effective Azuma theorem

For a probability measure P and uniformly s-computable real random variables $(Z_n)_{n\ge 1}$ such that $Z_n=g(X_1^n,s)$ and $\left | Z_n \right |\le \epsilon _n\sqrt {n/\ln n}$ with $\lim _{n\to \infty } \epsilon _n=0$ , on $1$ - $(s,P)$ -random points we have

(13) $$ \begin{align} \lim_{n\to\infty}\frac{1}{n}\sum_{i=1}^{n} \left[ Z_i-\operatorname{\mathrm{\textbf{E}}}\left( Z_i\middle|X_1^{i-1} \right) \right] = 0. \end{align} $$

Proof Define

(14) $$ \begin{align} Y_n:=\sum_{i=1}^{n} \left[ Z_i-\operatorname{\mathrm{\textbf{E}}}\left( Z_i\middle|X_1^{i-1} \right) \right]. \end{align} $$

Process $(Y_n)_{n\ge 1}$ is a martingale with respect to process $(X_n)_{n\ge 1}$ with increments bounded by inequality

(15) $$ \begin{align} \left| Z_n-\operatorname{\mathrm{\textbf{E}}}\left( Z_n\middle|X_1^{n-1} \right) \right|\le c_n := 2\epsilon_n\sqrt{n/\ln n}. \end{align} $$

By the Azuma inequality [Reference Azuma3] for any $\epsilon>0$ we obtain

(16) $$ \begin{align} P(\left| Y_n \right|\ge n\epsilon) &\le 2\exp\left( -\frac{\epsilon^2n^2}{2\sum_{i=1}^{n} c_i^2} \right) \le n^{-\alpha_n} , \end{align} $$

where

(17) $$ \begin{align} \alpha_n&:=\frac{\epsilon^2n}{8\sum_{i=1}^{n} \epsilon_i^2}. \end{align} $$

Since $\alpha _n\to \infty $ , we have $\sum _{n=1}^\infty P(\left | Y_n \right |\ge n\epsilon )<\infty $ and by Proposition 2.9 (effective Borel–Cantelli lemma), we obtain (13) on $1$ - $(s,P)$ -random points.

3.1 Universal coding

Let us begin our considerations with the problem of universal measures, which is related to the problem of universal coding. Suppose that we want to compress losslessly a typical sequence generated by a stationary probability measure P. We can reasonably ask what is the lower limit of such a compression, i.e., what is the minimal ratio of the encoded string length divided by the original string length. In information theory, it is well known that the greatest lower bound of such ratios is given by the entropy rate of measure P. For a stationary probability measure P, we denote its entropy rate as

(18) $$ \begin{align} h_{P} &:=\lim_{n\to\infty}\frac{1}{n}\operatorname{\mathrm{\textbf{E}}}\left[ -\log P(X_1^n) \right] =\lim_{k\to\infty}\operatorname{\mathrm{\textbf{E}}}\left[ -\log P(X_{k+1}|X_1^k) \right] , \end{align} $$

which exists for any stationary probability measure.

The entropy rate has the interpretation of the minimal asymptotic rate of lossless encoding of sequences emitted by measure P in various senses: in expectation, almost surely, or on algorithmically random points, where the last interpretation will be pursued in this subsection.

To furnish some theoretical background for universal coding let us recall the Kraft inequality $\sum _{w\in A} 2^{-\left | w \right |}\le 1$ , which holds for any prefix-free subset of strings $A\subset \left \{ 0,1 \right \}^*$ . The Kraft inequality implies in particular that lossless compression procedures, called prefix-free codes, can be mapped one-to-one to semi-measures. In particular, if we are seeking for a universal code, i.e., a prefix-free code $w\mapsto C(w)\in \left \{ 0,1 \right \}^*$ which is optimal for some class of stochastic sources P, we can equivalently seek for a universal semi-measure of form $w\mapsto R(w):=2^{-\left | C(w) \right |}$ . (Similar correspondence holds also for uniquely decodable codes [Reference McMillan36].) Consequently, the problem of universal coding will be solved if we point out such a semi-measure R that

(19) $$ \begin{align} \lim_{n\to\infty}\frac{1}{n}\left[ -\log R(X_1^n) \right]= \lim_{n\to\infty}\frac{1}{n}\left| C(X_1^n) \right|=h_{P} \end{align} $$

for some points that are typical of P.

As it is well established in information theory, some initial insight into the problem of universal coding or universal measures is given by the Shannon–McMillan–Breiman (SMB) theorem, which states that function $\frac {1}{n}\left [ -\log P(X_1^n) \right ]$ tends P-almost surely to the entropy rate $h_{P}$ . The classical proofs of this result were given by Algoet and Cover [Reference Algoet and Cover2] and Chung [Reference Chung9]. An effective version of the SMB theorem was presented by Hochman [Reference Hochman26] and Hoyrup [Reference Hoyrup27] (cf. [Reference Nakamura38, Reference V’yugin55] for related partial and weaker results).

Theorem 3.1 Effective SMB theorem [Reference Hochman26, Reference Hoyrup27]

For a stationary ergodic probability measure P, on $1$ -P-random points we have

(20) $$ \begin{align} \lim_{n\to\infty}\frac{1}{n}\left[ -\log P(X_1^n) \right]=h_{P}. \end{align} $$

The essential idea of Hoyrup’s proof, which is a bit more complicated, can be retold using tools developed in Section 2.4. Observe first that we have

(21) $$ \begin{align} \frac{1}{n}\left[ -\log P(X_1^n) \right]= \frac{1}{n}\sum_{i=1}^n\left[ -\log P(X_i|X_{1}^{i-1.}) \right]. \end{align} $$

Moreover, we have the uniform bound

(22) $$ \begin{align} \operatorname{\mathrm{\textbf{E}}} \sup_{n\ge 0}\left[ -\log P(X_0|X_{-n}^{-1}) \right] \le \operatorname{\mathrm{\textbf{E}}} \left[ -\log P(X_0) \right]+\log e \le \log e D <\infty \end{align} $$

(see [Reference Smorodinsky49, Lemma 4.26])—invoked by Hoyrup as well. Consequently, the effective SMB theorem follows by Proposition 2.16 (effective Breiman ergodic theorem) and Proposition 2.14 (effective Lévy law). In contrast, the reasoning by Hoyrup was more casuistic and his effective version of the Breiman ergodic theorem is weaker than the one proven here.

We note in passing that it could be also interesting to check whether one can effectivize the textbook sandwich proof of the SMB theorem by Algoet and Cover [Reference Algoet and Cover2] using the decomposition of conditionally algorithmically random sequences by Takahashi [Reference Takahashi54]. However, this step would require some novel theoretical considerations about conditional algorithmic randomness for uncomputable measures. We mention this only to point out a possible direction for future research.

As a direct consequence of the effective SMB theorem and Proposition 2.10 (effective Barron lemma), we obtain this effectivization of another well-known almost sure statement:

Theorem 3.2 Effective source coding

For any stationary ergodic measure P and any s-computable probability measure R, on $1$ - $(s,P)$ -random points we have

(23) $$ \begin{align} \liminf_{n\to\infty}\frac{1}{n}\left[ -\log R(X_1^n) \right] \ge h_{P}. \end{align} $$

In the almost sure setting, relationship (23) holds P-almost surely for any stationary ergodic measure P and any (not necessarily computable) probability measure R.

Now we can define universal measures.

Definition 3.3 Universal measure

A computable (not necessarily stationary) probability measure R is called (weakly) n-universal if for any stationary ergodic probability measure P, on (weakly) n-P-random points we have

(24) $$ \begin{align} \lim_{n\to\infty}\frac{1}{n}\left[ -\log R(X_1^n) \right]= h_{P}. \end{align} $$

In the almost sure setting, we say that a probability measure R is almost surely universal if (24) holds P-almost surely for any stationary ergodic probability measure P. By Proposition 2.7, there are only two practically interesting cases of computable universal measures: weakly $2$ -universal ones and $1$ -universal ones, since every computable almost surely universal probability measure is automatically weakly $2$ -universal. We stress that we impose computability of (weakly) n-universal measures by definition since it simplifies statements of some theorems. This should be contrasted with universal prediction à la Solomonoff for left-c.e. semimeasures where the universal element belongs to the class and is not computable [Reference Solomonoff50].

Computable almost surely universal measures exist if the alphabet $\mathbb {X}$ is finite. An important example of an almost surely universal and, as we will see in Section 3.5, also $1$ -universal measure is the Prediction by Partial Matching (PPM) measure [Reference Cleary and Witten10, Reference Ryabko44, Reference Ryabko47]. As we have mentioned, universal measures are closely related to the problem of universal coding (data compression) and more examples of universal measures can be constructed from universal codes, for instance given in [Reference Charikar, Lehman, Lehman, Liu, Panigrahy, Prabhakaran, Sahai and Shelat8, Reference Dębowski14, Reference Kieffer and Yang31, Reference Ziv and Lempel56], using the normalization by Ryabko [Reference Ryabko45]. This normalization is not completely straightforward, since we need to forge semi-measures into probability measures.

3.2 Universal prediction

Universal prediction is a problem similar to universal coding. In this problem, we also seek for a single procedure that would be optimal within a class of probabilistic sources but we apply a different loss function, namely, we impose the error rate being the density of incorrect guesses of the next output given previous ones. In spite of this difference, we will try to state the problem of universal prediction analogously to universal coding. A predictor is an arbitrary total function $f:\mathbb {X}^*\rightarrow \mathbb {X}$ . The predictor induced by a probability measure P will be defined as

(25) $$ \begin{align} f_{\kern-1pt P}(x_1^n):=\operatorname*{\mbox{arg max}}_{x_{n+1}\in\mathbb{X}} P(x_{n+1}|x_1^n), \end{align} $$

where $\operatorname *{\mbox {arg max}}_{x\in \mathbb {X}} g(x):=\min \left \{ a\in \mathbb {X}: g(a)\ge g(x) \text { for all }x\in \mathbb {X} \right \}$ for the total order $a_1<\cdots <a_D$ on $\mathbb {X}=\left \{ a_1,\dots ,a_D \right \}$ . Moreover, for a stationary measure P, we define the unpredictability rate

(26) $$ \begin{align} u_{P}:=\lim_{n\to\infty}\operatorname{\mathrm{\textbf{E}}}\left[ 1-\max_{x_0\in\mathbb{X}} P(x_0|X_{-n}^{-1}) \right]. \end{align} $$

It is natural to ask whether the unpredictability rate can be related to entropy rate. Using the Fano inequality [Reference Fano19], a classical result of information theory, and its converse [Reference Dębowski16], both independently brought to computability theory by Fortnow and Lutz [Reference Fortnow and Lutz20], yields this bound:

Theorem 3.4. For a stationary measure P over a D-element alphabet,

(27) $$ \begin{align} \frac{D}{D-1}\eta\left( \frac{1}{D} \right) u_P \le h_P \le \eta(u_P)+u_P\log(D-1) , \end{align} $$

where $\eta (p):=-p\log p-(1-p)\log (1-p)$ .

Moreover, Fortnow and Lutz [Reference Fortnow and Lutz20] found out some stronger inequalities, sandwich-bounding the unpredictability of an arbitrary sequence in terms of its effective dimension. The effective dimension turns out to be a generalization of the entropy rate to arbitrary sequences [Reference Hoyrup27], which are not necessarily random with respect to stationary ergodic measures.

In the less general framework of stationary ergodic measures, using the Azuma theorem, we can show that no predictor can beat the induced predictor and the error rate committed by the latter equals the unpredictability rate $u_{P}$ . The following proposition concerning the error rates effectivizes the well-known almost sure proposition (the proof in the almost sure setting is available in [Reference Algoet1]).

Theorem 3.5 Effective source prediction

For any stationary ergodic measure P and any s-computable predictor f, on $1$ - $(s,P)$ -random points we have

(28) $$ \begin{align} \liminf_{n\to\infty}\frac{1}{n}\sum_{i=0}^{n-1}\mathbf{1}{\left\{ X_{i+1}\neq f(X_1^i) \right\}} &\ge u_{P}. \end{align} $$

Moreover, if the induced predictor $f_{\kern-1pt P}$ is s-computable then (28) holds with the equality for $f=f_{\kern-1pt P}$ .

Proof Let measure P be stationary ergodic. In view of Theorem 2.17 (effective Azuma theorem), for any s-computable predictor f, on $1$ - $(s,P)$ -random points we have

(29) $$ \begin{align} \lim_{n\to\infty}\frac{1}{n}\sum_{i=0}^{n-1} \left[ \mathbf{1}{\left\{ X_{i+1}\neq f(X_1^i) \right\}}-P(X_{i+1}\neq f(X_1^i)|X_1^i) \right]=0. \end{align} $$

Moreover, we have

(30) $$ \begin{align} P(X_{i+1}\neq f(X_1^i)|X_1^i)\ge 1-\max_{x_{i+1}\in\mathbb{X}}P(x_{i+1}|X_1^i). \end{align} $$

Subsequently, we observe that limits $\lim _{n\to \infty } P(x_0|X_{-n}^{-1})$ exist on $1$ - $(s,P)$ -random points by Proposition 2.14 (effective Lévy law). Thus by Proposition 2.16 (effective Breiman ergodic theorem) and the dominated convergence, on $1$ - $(s,P)$ -random points we obtain

(31) $$ \begin{align} \lim_{n\to\infty}\frac{1}{n}\sum_{i=0}^{n-1} \left[ 1-\max_{x_{i+1}\in\mathbb{X}}P(x_{i+1}|X_1^i) \right] &= \operatorname{\mathrm{\textbf{E}}}\lim_{n\to\infty}\left[ 1-\max_{x_0\in\mathbb{X}} P(x_0|X_{-n}^{-1}) \right] \nonumber\\ &= u_{P}. \end{align} $$

Hence inequality (28) follows by (29)–(31). Similarly, the equality in (28) for $f=f_{\kern-1pt P}$ follows by noticing that inequality (30) turns out to be the equality in this case.

In the almost sure setting, relationship (28) holds P-almost surely for any stationary ergodic measure P and any (not necessarily computable) predictor f.

We can see that there can be some problem in the effectivization of relationship (28) caused by the induced predictor $f_{\kern-1pt P}$ possibly not being s-computable for certain representations s of measure P—since sometimes testing the equality of two real numbers cannot be done in a finite time. However, probabilities $P(X_{i+1}\neq f_{\kern-1pt P}(X_1^i)|X_1^i)$ are always s-computable. Thus, we can try to define universal predictors in the following way.

Definition 3.6 Universal predictor

A computable predictor f is called (weakly) n-universal if for any stationary ergodic probability measure P, on (weakly) n-P-random points we have

(32) $$ \begin{align} \lim_{n\to\infty}\frac{1}{n}\sum_{i=0}^{n-1}\mathbf{1}{\left\{ X_{i+1}\neq f(X_1^i) \right\}} = u_{P}. \end{align} $$

In the almost sure setting, we say that a predictor f is almost surely universal if (32) holds P-almost surely for any stationary ergodic probability measure P. Almost surely universal predictors exist if the alphabet $\mathbb {X}$ is finite [Reference Algoet1, Reference Bailey4, Reference Györfi, Lugosi, Dror, L’Ecuyer and Szidarovszky24, Reference Györfi, Lugosi and Morvai25, Reference Ornstein40]. In [Reference Steifer51] it was proved that the almost sure predictor by [Reference Györfi, Lugosi and Morvai25] is also $1$ -universal.

3.3 Predictors induced by backward estimators

The almost surely universal predictors by [Reference Algoet1, Reference Bailey4, Reference Györfi, Lugosi, Dror, L’Ecuyer and Szidarovszky24, Reference Györfi, Lugosi and Morvai25, Reference Ornstein40] were constructed without a reference to universal measures. Nevertheless, these constructions are all based on estimation of conditional probabilities. For a stationary ergodic process one can consider two separate problems: backward and forward estimation. The first problem is naturally connected to prediction. We want to estimate the conditional probability of $(n+1)$ -th bit given the first n bits. Is it possible that, as we increase n, our estimates converge to the true value at some point? To be precise, we ask whether there exists a probability measure R such that for every stationary ergodic measure P we have P-almost surely

(33) $$ \begin{align} \lim_{n\to\infty}\sum_{x_{n+1}\in\mathbb{X}}\left| R(x_{n+1}|X_{1}^{n-1})-P(x_{n+1}|X_{1}^{n-1}) \right|=0. \end{align} $$

It was shown by Bailey [Reference Bailey4] that this is not possible. As we are about to see, we can get something a bit weaker, namely, the convergence in Cesaro averages. But to get there, it will be helpful to consider a bit different problem.

Suppose again that we want to estimate a conditional probability but the bit that we are interested in is fixed and we are looking more and more into the past. In this scenario, we want to estimate the conditional probability $P(x_0|X_{-\infty }^{-1})$ and we ask whether increasing the knowledge of the past can help us achieve the perfect guess. Precisely, we ask if there exists a probability measure R such that for every stationary ergodic measure P we have P-almost surely

(34) $$ \begin{align} \lim_{n\to\infty} \sum_{x_0\in\mathbb{X}}\left| R(x_0|X^{-1}_{-n})-P(x_0|X_{-\infty}^{-1}) \right| =0. \end{align} $$

It was famously shown by Ornstein that such estimators exist. (Ornstein proved this for binary-valued processes but the technique can be generalized to finite-valued processes.)

Theorem 3.7 Ornstein theorem [Reference Ornstein40]

Let the alphabet be finite. There exists a computable measure R such that for every stationary ergodic measure P we have P-almost surely that

(35) $$ \begin{align} \lim_{n\to\infty} \sum_{x_0\in\mathbb{X}}\left| R(x_0|X^{-1}_{-n})-P(x_0|X_{-\infty}^{-1}) \right| =0. \end{align} $$

One can come up with a naive idea: What if we take a universal backward estimator and use it in a forward fashion? Surprisingly, this simple trick gives us almost everything we can get, i.e., a forward estimator that converges to the conditional probability on average. Bailey [Reference Bailey4] showed that for an almost surely universal backward estimator R and for every stationary ergodic measure P we have P-almost surely

(36) $$ \begin{align} \lim_{n\to\infty}\frac{1}{n}\sum_{i=0}^{n-1}\sum_{x_{i+1}\in\mathbb{X}} \left| R(x_{i+1}|X_{1}^{i})-P(x_{i+1}|X_{1}^{i}) \right|=0. \end{align} $$

The proof of this fact is a direct application of the Breiman ergodic theorem. Since we have a stronger effective version of the Breiman theorem (Theorem 2.16), we can strengthen Bailey’s result to an effective version as well. It turns out that even if we take a backward estimator that is good only almost surely (possibly failing on some random points), then the respective result for the forward estimation will hold in the strong sense—on every $1$ -P-random point.

Proof Let R be a computable almost surely universal backward estimator. Fix an $x\in \mathbb {X}$ . By Proposition 2.14 (effective Lévy law), for every stationary ergodic probability measure P we have P-almost surely

(37) $$ \begin{align} \lim_{n\to\infty}\left| R(x|X^{-1}_{-n})-P(x|X_{-n}^{-1}) \right|=0. \end{align} $$

Note that the bound $0\le \left | R(x|X^{-1}_{-n})-P(x|X_{-n}^{-1}) \right |\le 1$ holds uniformly. Moreover, variables $R(x|X^{-1}_{-n})-P(x|X_{-n}^{-1})$ are uniformly s-computable for any representation s of P. Hence, we can apply Theorem 2.16 (effective Breiman ergodic theorem) to obtain

(38) $$ \begin{align} \lim_{n\to\infty}\frac{1}{n}\sum_{i=0}^{n-1} \left| R(x|X_{1}^{i})-P(x|X_{1}^{i}) \right|=\operatorname{\mathrm{\textbf{E}}} 0=0 \end{align} $$

for $1$ -P-random points. The claim follows from this immediately.

One can expect that the predictor $f_R$ induced by a universal forward estimator R in the sense of Definition 3.10 is also universal in the sense of Definition 3.6. This is indeed true. To show this fact, we will first prove a certain inequality for induced predictors, which generalizes the result from [Reference Devroye, Györfi and Lugosi17, Theorem 2.2] for binary classifiers. This particular observation seems to be new.

Proposition 3.11 Prediction inequality

Let p and q be two probability distributions over a countable alphabet $\mathbb {X}$ . For $x_p=\operatorname *{\mbox {arg max}}_{x\in \mathbb {X}} p(x)$ and $x_q=\operatorname *{\mbox {arg max}}_{x\in \mathbb {X}} q(x)$ , we have inequality

(39) $$ \begin{align} 0&\le p(x_p)-p(x_q)\le \sum_{x\in\mathbb{X}} \left| p(x)-q(x) \right|. \end{align} $$

Proof Without loss of generality, assume $x_p\neq x_q$ . By the definition of $x_p$ and $x_q$ , we have $p(x_p)-p(x_q)\ge 0$ and $q(x_q)-q(x_p)\ge 0$ . Hence we obtain

(40) $$ \begin{align} 0&\le p(x_p)-p(x_q) \le p(x_p)-p(x_q) -q(x_p)+q(x_q) \nonumber\\ &\le \left| p(x_p)-q(x_p) \right| +\left| p(x_q)-q(x_q) \right| \le \sum_{x}\left| p(x)-q(x) \right|. \end{align} $$

Now we can show a general result about universal predictors induced by forward estimators of conditional probabilities.

Proof Let R be $1$ -universal forward estimator. By the definition, for every stationary ergodic measure P and all $1$ -P-random points

(41) $$ \begin{align} \lim_{n\to\infty} \frac{1}{n}\sum_{i=0}^{n-1}\sum_{x_{i+1}\in\mathbb{X}} \left| P(x_{i+1}|X_1^i)-R(x_{i+1}|X_1^i) \right| = 0. \end{align} $$

Consequently, combining this with Proposition 3.11 (prediction inequality) yields on $1$ -P-random points

(42) $$ \begin{align} \lim_{n\to\infty} \frac{1}{n}\sum_{i=0}^{n-1} \left[ P(X_{i+1}\neq f_R(X_1^i)|X_1^i)- P(X_{i+1}\neq f_{\kern-1pt P}(X_1^i)|X_1^i) \right] = 0. \end{align} $$

Now, we notice that by (29), we have on $1$ -P-random points

(43) $$ \begin{align} \lim_{n\to\infty} \frac{1}{n}\sum_{i=0}^{n-1} \left[ \mathbf{1}{\left\{ X_{i+1}\neq f_R(X_1^i) \right\}}-P(X_{i+1}\neq f_R(X_1^i)|X_1^i) \right] &= 0 , \end{align} $$

(44) $$ \begin{align} \lim_{n\to\infty} \frac{1}{n}\sum_{i=0}^{n-1} \left[ \mathbf{1}{\left\{ X_{i+1}\neq f_P(X_1^i) \right\}}-P(X_{i+1}\neq f_P(X_1^i)|X_1^i) \right] &= 0. \end{align} $$

Combining the three above observations completes the proof.

Interestingly, it suffices for a measure to be a computable almost surely universal backward estimator to yield a $1$ -universal forward estimator and, consequently, a $1$ -universal predictor. In contrast, we can easily see that a computable almost surely universal forward estimator does not necessarily induce a $1$ -universal predictor.

Proof Let us take $\mathbb {X}=\left \{ 0,1 \right \}$ and restrict ourselves to one-sided space $\mathbb {X}^{\mathbb {N}}$ without loss of generality. Fix a computable almost surely universal forward estimator Q. Let $P_0$ be the computable measure of a Bernoulli( $\theta $ ) process, i.e., $P_0(x_1^n)=\prod _{i=1}^n\theta ^{x_i}(1-\theta )^{1-x_i}$ , where $\theta>1/2$ is rational. Observe that by Proposition 2.8 there exists a point $y\in \mathbb {X}^{\mathbb {N}}$ which is $1$ - $P_0$ -random and a computable function $g:\mathbb {X}^*\rightarrow \left \{ 0,1 \right \}$ such that $P_0(A)=0$ and $A=\left \{ y \right \}$ for event

(45) $$ \begin{align} A:=(\#\left\{ i\in\mathbb{N}:g(X_1^i)=1 \right\}=\infty). \end{align} $$

In other words, there is a computable method to single out some $1$ - $P_0$ -random point y out of the set of sequences $\mathbb {X}^{\mathbb {N}}$ . In particular, we can use function g to spoil measure Q on that point y while preserving the property of an almost surely universal forward estimator. We will denote the spoilt version of measure Q by R. Conditional distributions $R(X_{m+1}|X_1^m)$ will differ from $Q(X_{m+1}|X_1^m)$ for infinitely many m on point y and for finitely many m elsewhere.

Let $K(x_1^n):=\#\left \{ i\le n:g(x_1^i)=1 \right \}$ . The construction of measure R proceeds by induction on the string length together with an auxiliary counter U. We let $R(x_1):=Q(x_1)$ and $U(x_1):=0$ . Suppose that $R(x_1^n)$ and $U(x_1^n)$ are defined but $R(x_1^{n+1})$ is not. If $U(x_1^n)\ge K(x_1^n)$ then we put $R(x_{n+1}|x_1^n):=Q(x_{n+1}|x_1^n)$ and $U(x_1^{n+1}):=U(x_1^n)$ . Else, if $U(x_1^n)< K(x_1^n)$ then we put $R(x_{n+1}^{n+N}|x_1^n):=\prod _{i=n+1}^{n+N}\theta ^{1-x_i}(1-\theta )^{x_i}$ (reverted compared to the definition of $P_0$ !) and $U(x_{n+N}):=K(x_1^n)$ where N is the smallest number such that

(46) $$ \begin{align} \frac{1}{n+N} \left[ \sum^{n-1}_{i=0} P_0(X_{i+1}\neq f_R(x_1^{i})|X_1^i=x_1^{i})+N\theta \right] \ge \frac{1}{2}. \end{align} $$

Such number N exists since $P_0(X_{i+1}\neq f_R(x_1^{i})|X_1^i=x_1^{i})> 1-\theta $ . This completes the construction of R.

The sets of $1$ -P-random sequences are disjoint for distinct stationary ergodic P by Theorem 2.15 (effective Birkhoff ergodic theorem). Hence $K(X_1^n)$ is bounded P-almost surely for any stationary ergodic P. Consequently, since $U(X_1^n)$ is non-decreasing then P-almost surely there exists a random number $M<\infty $ such that for all $m>M$ we have $R(X_{m+1}|X_1^m)=Q(X_{m+1}|X_1^m)$ . Hence R inherits the property of an almost surely universal forward estimator from Q.

Now let us inspect what happens on y. Since $K(X_1^n)$ is unbounded on y then by the construction of R, we obtain on y that $U(X_1^n)<K(X_1^n)$ holds infinitely often and

(47) $$ \begin{align} \limsup_{n\to\infty}\frac{1}{n}\sum^{n}_{i=0} P_0(X_{i+1}\neq f_R(X_1^{i})|X_1^{i})\ge \frac{1}{2}>u_{P_0}=1-\theta. \end{align} $$

Hence predictor $f_{R}$ is not $1$ -universal.

3.4 Predictors induced by universal measures

Following the work of Ryabko [Reference Ryabko45] (see also [Reference Ryabko, Astola and Malyutov46]), we can ask a natural question whether predictors induced by some universal measures in the sense of Definition 3.3, such as the PPM measure [Reference Cleary and Witten10, Reference Ryabko44, Reference Ryabko47] to be discussed in Section 3.5, are also universal. Ryabko was close to demonstrate the analogous implication in the almost sure setting but did not provide the complete proof. He has shown this proposition:

Theorem 3.14 Theorem 3.3 in [Reference Ryabko44]

Let R be an almost surely universal measure and P be a stationary ergodic measure. We have that

(48) $$ \begin{align} \lim_{n\to\infty}\operatorname{\mathrm{\textbf{E}}}\frac{1}{n}\sum_{i=0}^{n-1} \left| P(X_{i+1}|X_0^i)-R(X_{i+1}|X_0^i) \right|=0. \end{align} $$

At the first glance, condition (48) may seem close to condition (36), i.e., the universal forward estimator, which—as we have shown in Theorem 3.12—implies universality of the induced predictor. However, this average-case result is too weak for our needs as we seek the almost-sure and effective version thereof. If we tried to derive universality of the induced predictor directly from (48), there are two problems on the way (in the following, $Y_n\,{\ge}\, 0$ stands for the expression under the expectation): Firstly, $\lim _{n\to \infty } \operatorname {\mathrm {\textbf {E}}} Y_n=0$ does not necessarily imply $\operatorname {\mathrm {\textbf {E}}}\lim _{n\to \infty } Y_n=0$ since the limit may not exist almost surely and, secondly, if $\operatorname {\mathrm {\textbf {E}}}\lim _{n\to \infty } Y_n=0$ then $Y_n=0$ holds almost surely but this equality may fail on some $1$ -random points.

In this section, we will show that each $1$ -universal measure, under a relatively mild condition (1), satisfied by the PPM measure, is a $1$ -universal forward estimator and hence, in the light of the previous section, it induces a $1$ -universal predictor. We do not know yet whether this condition is necessary. We will circumvent Theorem 3.14 by applying Proposition 2.16 (effective Breiman ergodic theorem) and Theorem 2.17 (effective Azuma theorem). The first stage of our preparations includes two statements which can be called the effective conditional SMB theorem and the effective conditional universality.

Proposition 3.15 Effective conditional SMB theorem

Let the alphabet be finite and let P be a stationary ergodic probability measure. On $1$ -P-random points we have

(49) $$ \begin{align} \lim_{n\to\infty} \frac{1}{n}\sum_{i=0}^{n-1} \left[ -\sum_{x_{i+1}\in\mathbb{X}}P(x_{i+1}|X_1^i) \log P(x_{i+1}|X_1^i) \right]=h_{P}. \end{align} $$

Proof Let us write the conditional entropy

(50) $$ \begin{align} W_i:=\left[ -\sum_{x_{i+1}\in\mathbb{X}}P(x_{i+1}|X_1^i) \log P(x_{i+1}|X_1^i) \right]. \end{align} $$

We have $0\le W_i\le \log D$ with D being the cardinality of the alphabet. Moreover by Proposition 2.14 (effective Lévy law), on $1$ -P-random points there exists limit

(51) $$ \begin{align} \lim_{n\to\infty} W_n\circ T^{-n-1}= \left[ -\sum_{x_0\in\mathbb{X}}P(x_0|X_{-\infty}^{-1}) \log P(x_0|X_{-\infty}^{-1}) \right]. \end{align} $$

Hence by Proposition 2.16 (effective Breiman ergodic theorem), on $1$ - $(s,P)$ -random points

(52) $$ \begin{align} \lim_{n\to\infty} \frac{1}{n}\sum_{i=0}^{n-1} W_i= \operatorname{\mathrm{\textbf{E}}} \left[ -\sum_{x_0\in\mathbb{X}}P(x_0|X_{-\infty}^{-1}) \log P(x_0|X_{-\infty}^{-1}) \right]=h_{P} \end{align} $$

since $\operatorname {\mathrm {\textbf {E}}}\left [ -\log P(X_0|X_{-\infty }^{-1}) \right ]=\lim _{n\to \infty }\left [ -\log P(X_1^n) \right ]/n=h_{P}$ .

Proposition 3.16 Effective conditional universality

Let the alphabet be finite and let P be a stationary ergodic probability measure. If measure R is $1$ -universal and satisfies

(53) $$ \begin{align} -\log R(x_{n+1}|x_1^n)&\le\epsilon_n\sqrt{n/\ln n}, \quad \lim_{n\to\infty} \epsilon_n=0, \end{align} $$

then on $1$ -P-random points we have

(54) $$ \begin{align} \lim_{n\to\infty} \frac{1}{n}\sum_{i=0}^{n-1} \left[ -\sum_{x_{i+1}\in\mathbb{X}}P(x_{i+1}|X_1^i) \log R(x_{i+1}|X_1^i) \right]=h_{P}. \end{align} $$

Proof Let us write the conditional pointwise entropy $Z_i:=-\log R(X_{i+1}|X_1^i)$ . Now suppose that measure R is $1$ -universal and satisfies (53). Then by Theorem 2.17 (effective Azuma theorem), on $1$ -P-random points we obtain

(55) $$ \begin{align} \lim_{n\to\infty}\frac{1}{n}\sum_{i=0}^{n-1} \operatorname{\mathrm{\textbf{E}}}\left( Z_i\middle|X_1^i \right) =\lim_{n\to\infty}\frac{1}{n}\sum_{i=0}^{n-1} Z_i = \lim_{n\to\infty}\frac{1}{n}\left[ -\log R(X_1^n) \right]=h_{P} , \end{align} $$

which is the claim of Proposition 3.16.

In the second stage of our preparations, we recall the famous Pinsker inequality used by Ryabko [Reference Ryabko45] to prove Theorem 3.14.

Proposition 3.17 Pinsker inequality [Reference Csiszár and Körner11]

Let p and q be probability distributions over a countable alphabet $\mathbb {X}$ . We have

(56) $$ \begin{align} \left[ \sum_{x\in\mathbb{X}} \left| p(x)-q(x) \right| \right]^2\le (2\ln 2) \sum_{x\in\mathbb{X}} p(x)\log\frac{p(x)}{q(x)}. \end{align} $$

Now we can show the main result of this section, namely, that every universal measure which satisfies a mild condition induces a universal predictor.

Proof Let R be a $1$ -universal measure, whereas P be the stationary ergodic measure. By Propositions 3.15 (effective conditional SMB theorem) and 3.16 (effective conditional universality), on $1$ -P-random points we obtain

(57) $$ \begin{align} \lim_{n\to\infty} \frac{1}{n}\sum_{i=0}^{n-1} \left[ \sum_{x_{i+1}}P(x_{i+1}|X_1^i) \log\frac{P(x_{i+1}|X_1^i)}{R(x_{i+1}|X_1^i)} \right] =0. \end{align} $$

Hence by Proposition 3.17 (Pinsker inequality), we derive on $1$ -P-random points

(58) $$ \begin{align} \lim_{n\to\infty} \frac{1}{n}\sum_{i=0}^{n-1} \left[ \sum_{x_{i+1}}\left| P(x_{i+1}|X_1^i)-R(x_{i+1}|X_1^i) \right| \right]^2 =0. \end{align} $$

Subsequently, the Cauchy–Schwarz inequality $\operatorname {\mathrm {\textbf {E}}} Y^2\ge (\operatorname {\mathrm {\textbf {E}}} Y)^2$ yields on $1$ -P-random points

(59) $$ \begin{align} 0 &\ge \lim_{n\to\infty} \left[ \frac{1}{n}\sum_{i=0}^{n-1}\sum_{x_{i+1}} \left| P(x_{i+1}|X_1^i)-R(x_{i+1}|X_1^i) \right| \right]^2 \nonumber\\[6pt] &= \left[ \lim_{n\to\infty} \frac{1}{n}\sum_{i=0}^{n-1}\sum_{x_{i+1}} \left| P(x_{i+1}|X_1^i)-R(x_{i+1}|X_1^i) \right| \right]^2 \ge 0. \end{align} $$

Consequently, R is a $1$ -universal forward estimator.

Combining Theorems 3.18 and 3.12, we obtain that predictor $f_R$ is $1$ -universal provided measure R is $1$ -universal and satisfies condition (53)—if predictor $f_R$ is computable itself. Condition (53) does not seem to have been discussed in the literature of universal prediction.

3.5 PPM measure

In this section, we will discuss the Prediction by Partial Matching (PPM) measure. The PPM measure comes in several flavors and was discovered gradually. Cleary and Witten [Reference Cleary and Witten10] coined the name PPM, which we prefer since it is more distinctive, and considered the adaptive Markov approximations $\operatorname {\mathrm {PPM}}_k$ defined roughly in Equation (63). Later, Ryabko [Reference Ryabko44, Reference Ryabko47] considered the infinite series $\operatorname {\mathrm {PPM}}$ defined in Equation (64), called it the measure R, and proved that it is a universal measure. Precisely, Ryabko used the Krichevsky–Trofimov smoothing ( $+1/2$ ) rather than the Laplace smoothing ( $+1$ ) applied in (63). This difference does not affect universality. As we will show now, the series $\operatorname {\mathrm {PPM}}$ provides an example of a $1$ -universal measure that satisfies condition (53) and thus yields a natural $1$ -universal predictor.

Upon the first reading, the definition of the PPM measure may appear cumbersome but it is roughly a Bayesian mixture of all Markov chains of all orders. Its universality can be then motivated by the fact that Markov chains with rational transition probabilities are both countable and dense in the class of stationary ergodic measures [Reference Ryabko48]. Our specific definition of the $\operatorname {\mathrm {PPM}}$ measure is as follows.

Definition 3.19 PPM measure

Let the alphabet be $\mathbb {X}=\left \{ a_1,\ldots ,a_D \right \}$ , where $D\ge 2$ .

Define the frequency of a substring $w_1^k$ in a string $x_1^n$ as

(60) $$ \begin{align} N(w_1^k|x_1^n):=\sum_{i=1}^{n-k+1}\mathbf{1}{\left\{ x_i^{i+k-1}=w_1^k \right\}}. \end{align} $$

Adapting the definitions by [Reference Cleary and Witten10, Reference Dębowski15, Reference Ryabko44, Reference Ryabko47], the PPM measure of order $k\ge 0$ is defined as

(61) $$ \begin{align} \operatorname{\mathrm{PPM}}_k(x_1^{k+1})&:=D^{-k-1}, \end{align} $$

(62) $$ \begin{align} \operatorname{\mathrm{PPM}}_k(x_{n+1}|x_1^n)&:= \frac{N(x_{n+1-k}^{n+1}|x_1^n)+1}{N(x_{n+1-k}^n|x_1^{n-1})+D}, \quad n\ge k+1, \end{align} $$

(63) $$ \begin{align} \operatorname{\mathrm{PPM}}_k(x_1^n)&:=\operatorname{\mathrm{PPM}}_k(x_1^{k+1})\prod_{i=k+1}^{n}\operatorname{\mathrm{PPM}}_k(x_{i+1}|x_1^i). \end{align} $$

Subsequently, we define the total PPM measure

(64) $$ \begin{align} \operatorname{\mathrm{PPM}}(x_1^n) &:= \sum_{k=0}^\infty \left[ \frac{1}{k+1}-\frac{1}{k+2} \right]\operatorname{\mathrm{PPM}}_k(x_1^n). \end{align} $$

The infinite series (64) is computable since $\operatorname {\mathrm {PPM}}_k(x_1^n)=D^{-n}$ for $k\,{\ge}\, n-1$ . The almost sure universality of the total PPM measure follows by the Stirling approximation and the Birkhoff ergodic theorem (see [Reference Dębowski15, Reference Ryabko44, Reference Ryabko47]). Since the Birkhoff ergodic theorem can be effectivized for $1$ -random points in the form of Proposition 2.15, we obtain in turn this effectivization.

Proof As we have mentioned, computability of the PPM measure follows since series (64) can be truncated with the constant term $\operatorname {\mathrm {PPM}}_k(x_1^n)=D^{-n}$ for $k\ge n-1$ and thus values $\operatorname {\mathrm {PPM}}(x_1^n)$ are rational.

To show $1$ -universality of the PPM measure, we first observe that

(65) $$ \begin{align} \operatorname{\mathrm{PPM}}_k(X_1^n) &=D^{-k}\prod_{w_1^k} \frac{\prod_{w_{k+1}}1\cdot 2\cdot\,\cdots\,\cdot N(w_1^{k+1}|x_1^n) }{D\cdot(D+1)\cdot\,\cdots\,\cdot(N(w_1^k|x_1^{n-1})+D-1)} \nonumber\\ &=D^{-k}\prod_{w_1^k} \frac{(D-1)!\prod_{w_{k+1}} N(w_1^{k+1}|x_1^n)! }{(N(w_1^k|x_1^{n-1})+D-1)!}. \end{align} $$

In contrast, the empirical (conditional) entropy of string $x_1^n$ of order $k\ge 0$ is defined as

(66) $$ \begin{align} h_k(x_1^n) &:= \sum_{w_1^{k+1}} \frac{N(w_1^{k+1}|x_1^n)}{n-k} \log \frac{N(w_1^k|x_1^{n-1})}{N(w_1^{k+1}|x_1^n)}. \end{align} $$

Using the Stirling approximation for the factorial function, the PPM measure of order $k\ge 0$ can be related to the empirical entropy. In particular, by Theorem A4 in [Reference Dębowski15], we have the bound

(67) $$ \begin{align} 0\le -\log\operatorname{\mathrm{PPM}}_k(x_1^n) - k\log D - (n-k)h_k(x_1^n) \le D^{k+1}\log[e^2n]. \end{align} $$

Subsequently, by Proposition 2.15 (effective Birkhoff ergodic theorem), on $1$ -P-random points we have

(68) $$ \begin{align} \lim_{n\to\infty} \frac{N(w_1^{k+1}|X_1^n)}{n-k}=P(w_1^{k+1}). \end{align} $$

Then by (67),

(69) $$ \begin{align} \lim_{n\to\infty}\frac{1}{n}\left[ -\log\operatorname{\mathrm{PPM}}_k(X_1^n) \right]=h_{k,P}:=\operatorname{\mathrm{\textbf{E}}}\left[ -\log P(X_{k+1}|X_1^k) \right]. \end{align} $$

Since

(70) $$ \begin{align} -\log\operatorname{\mathrm{PPM}}(x_1^n)\le 2\log (k+2)-\log\operatorname{\mathrm{PPM}}_k(x_1^n), \end{align} $$

then

(71) $$ \begin{align} \limsup_{n\to\infty}\frac{1}{n}\left[ -\log\operatorname{\mathrm{PPM}}(X_1^n) \right]\le \inf_{k\ge 0} h_{k,P}=h_{P} \end{align} $$

on $1$ -P-random points, whereas the reverse inequality for the lower limit follows by Proposition 2.10 (effective Barron lemma) and Theorem 3.1 (effective SMB theorem).

Finally, we can show that predictor $f_{\operatorname {\mathrm {PPM}}}$ induced by the PPM measure is $1$ -universal. First, we notice explicitly these bounds:

Theorem 3.21 PPM bounds

We have

(72) $$ \begin{align} -\log \operatorname{\mathrm{PPM}}(x_1^n)&\le 2\log (n+1)+n\log D, \end{align} $$

(73) $$ \begin{align} \phantom{\hspace{-70pt}}-\log \operatorname{\mathrm{PPM}}(x_{n+1}|x_1^n)&\le 3\log (n+D). \end{align} $$

Proof Observe that $\operatorname {\mathrm {PPM}}_k(x_1^n)=D^{-n}$ for $k\ge n-1$ . Hence by (70), we obtain claim (72). The derivation of claim (73) is slightly longer. First, by the definition of $\operatorname {\mathrm {PPM}}_k$ , we have

(74) $$ \begin{align} -\log \operatorname{\mathrm{PPM}}_k(x_{n+1}|x_1^n)\le \log \left[ N(x_{n-k+1}^{n}|x_1^{n-1})+D \right] \le \log(n+D). \end{align} $$

Now let us denote

(75) $$ \begin{align} G:=\operatorname*{\mbox{arg max}}_{k\ge 0} \operatorname{\mathrm{PPM}}_k(x_1^n). \end{align} $$

We have $G\le n-1$ , since $\operatorname {\mathrm {PPM}}_k(x_1^n)=D^{-n}$ for $k\ge n-1$ . Moreover, we have a bound reverse to (70), namely

(76) $$ \begin{align} -\log\operatorname{\mathrm{PPM}}(x_1^n)\ge -\log\operatorname{\mathrm{PPM}}_{G}(x_1^n). \end{align} $$

Combining the above with (70) yields

(77) $$ \begin{align} &-\log \operatorname{\mathrm{PPM}}(x_{n+1}|x_1^n) = -\log \operatorname{\mathrm{PPM}}(x_1^{n+1})+\log \operatorname{\mathrm{PPM}}(x_1^n) \nonumber\\ &\le 2\log (G+2)-\log \operatorname{\mathrm{PPM}}_{G}(x_1^{n+1}) +\log \operatorname{\mathrm{PPM}}_{G}(x_1^n) \nonumber\\ &= 2\log (G+2) +\log \operatorname{\mathrm{PPM}}_{G}(x_{n+1}|x_1^n) \le 3\log (n+D). \end{align} $$

Now comes the main theorem.

We think that $1$ -universality of the predictor $f_{\operatorname {\mathrm {PPM}}}$ is quite expected and intuitive. But as we can see, the PPM measure satisfies condition (53) with a large reserve. It is an open question whether there are $1$ -universal measures such that conditional probabilities $R(x_{n+1}|x_1^n)$ converge to zero much faster than for the PPM measure but they still induce $1$ -universal predictors. It would be interesting to find such measures. Maybe they have some other desirable properties, also from a practical point of view.