2.1 Measure-theoretic versus topological typicality

One prominent way to think about typicality—the one on which Bayesian convergence-to-the-truth theorems capitalize—is measure-theoretic. Recall that the Borel $\sigma $ -algebra $\mathfrak{B}$ on ${\{ 0,1\} ^\mathbb{N}}$ is the smallest $\sigma $ -algebra containing all open sets in Cantor space. The elements of $\mathfrak{B}$ are called Borel sets. A probability measure $\mu $ on $\mathfrak{B}$ assigns to each Borel subset of ${\{ 0,1\} ^\mathbb{N}}$ a value in $\left[ {0,1} \right]$ in such a way that $\mu(\{0, 1\}^\mathbb{N}) = 1$ and, for any countable collection ${\{ {{\cal A}_n}\} _{n \in \mathbb{N}}}$ of pairwise disjoint Borel sets, $\mu(\bigcup_{n\in\mathbb{N}}\mathcal{A}_n)=\sum_{n\in\mathbb{N}}\mu(\mathcal{A}_n)$ . Every probability measure on $\mathfrak{B}$ can be identified with a function $\mu $ that maps cylinders to real numbers in $\left[ {0,1} \right]$ and satisfies the following two conditions: (1) $\mu \left( {\left[ \varepsilon \right]} \right) = 1$ (where $\varepsilon $ denotes the empty string) and (2) $\mu \left( {\left[ \sigma \right]} \right) = \mu \left( {\left[ {\sigma 1} \right]} \right) + \mu \left( {\left[ {\sigma 0} \right]} \right)$ for all $\sigma \in {\{ 0,1\} ^{ \lt \mathbb{N}}}$ (where $\sigma 1$ is the string consisting of $\sigma $ followed by $1$ and $\sigma 0$ the string consisting of $\sigma $ followed by $0$ ). By Carathéodory’s Extension Theorem (see Williams Reference Williams1991, Theorem 1.7), any such function can in fact be uniquely extended to a full probability measure on $\mathfrak{B}$ .

A probability measure of which we will often make use is the uniform measure $\lambda $ : the probability measure that results from tossing a fair coin infinitely many times, given by $\lambda \left( {\left[ \sigma \right]} \right) = {2^{ - \left| \sigma \right|}}$ for all $\sigma \in {\{ 0,1\} ^{ \lt \mathbb{N}}}$ (where $\left| \sigma \right|$ denotes the length of $\sigma $ ).

Although probability measures admit various interpretations, here we will take them to represent the subjective priors of Bayesian reasoners. So, a probability measure $\mu $ on $\mathfrak{B}$ will always stand for a specific agent’s initial degrees of belief, or credences, about the events in $\mathfrak{B}$ , and it will be understood as capturing that agent’s background knowledge and inductive assumptions at the beginning of the learning process.

Given a probability measure $\mu $ , a set is measure-theoretically typical relative to $\mu $ if it has $\mu $ -measure one and is measure-theoretically atypical relative to $\mu $ if it has $\mu $ -measure zero. A $\mu $ -measure-one-set corresponds to an event of which the agent with prior $\mu $ is essentially certain, whereas a $\mu $ -measure-zero set corresponds to an event that the agent considers negligible. Measure-theoretic typicality is therefore inextricably tied to the underlying prior: distinct probability measures may disagree wildly as to which sets count as measure-theoretically typical. ^{Footnote 3}

Topological typicality (the type of typicality on which Belot’s criticism is grounded) is instead defined as follows. Recall that a set is nowhere dense if its closure has empty interior—intuitively, if it corresponds to a hypothesis that, no matter what evidence has been observed so far, can always be refuted by further evidence. Equivalently, a set ${\cal S} \subseteq {\{ 0,1\} ^\mathbb{N}}$ is nowhere dense if, for every open set ${\cal U} \subseteq {\{ 0,1\} ^\mathbb{N}}$ , ${\cal S} \cap {\cal U}$ is not dense in the subspace topology on ${\cal U}$ —where a set is dense if it has a nonempty intersection with every nonempty open set (intuitively, a dense set corresponds to a hypothesis that cannot be refuted by any finite amount of evidence). A subset of a topological space is topologically atypical if it is meager, that is, if it is expressible as a countable union of nowhere dense sets. ^{Footnote 4} On the other hand, a set is topologically typical if it is co-meager: if it is the complement of a meager set.

Measure-theoretic and topological (a)typicality have several features in common. For instance, the class of measure-zero sets and the class of meager sets are both $\sigma $ -ideals: measure-zero and meager sets are both closed under subsets and countable unions. However, although they both aim at capturing notions of “largeness” and “smallness,” these concepts often diverge. For a well-trodden example, consider the collection of sequences that satisfy the Strong Law of Large Numbers relative to the uniform measure $\lambda $ (the set of sequences with limiting relative frequency 1/2 for $1$ ): this is a set with $\lambda $ -measure one, and yet it is also meager (see Oxtoby Reference Oxtoby1980, 85).

2.2 Lévy’s Upward Theorem

Bayesian convergence to the truth is epitomized by Lévy’s Upward Theorem (Lévy Reference Lévy1937), which establishes that, given some quantity that a Bayesian agent is trying to measure, the probability of observing a data stream that will lead the agent’s successive estimates to asymptotically align with the truth is one. In other words, a Bayesian reasoner conducting repeated experiments to gauge some quantity expects that almost every sequence of observations will bring about inductive success.

Let $f:{\{ 0,1\} ^\mathbb{N}} \to \mathbb{R}$ be a random variable relative to some probability measure $\mu $ on $\mathfrak{B}$ . Think of the values of $f$ as quantities that the agent with prior $\mu $ wishes to estimate; for instance, $f$ could record the value of some unknown physical parameter that may vary between possible worlds. The unconditional expectation of $f$ with respect to $\mu $ (the average value of $f$ weighted by $\mu $ , given by $\int_{\{0,1\}^\mathbb{N}} f\,d\mu$ ) is abbreviated as ${\mathbb{E}_\mu }\left[ f \right]$ . If ${\mathbb{E}_\mu }\left[ {\left| f \right|} \right] \lt \infty $ , then $f$ is integrable. For each $n \in \mathbb{N}$ , let ${\mathfrak{F}}_n$ be the sub- $\sigma $ -algebra of $\mathfrak{B}$ generated by the cylinders $\left[ \sigma \right]$ centered on strings $\sigma \in {\{ 0,1\} ^{ \lt \mathbb{N}}}$ of length $n$ . This collection of algebras has an especially natural epistemic interpretation: each ${\mathfrak{F}}_n$ intuitively captures the possible information that the agent may obtain at the $n$ th stage of the learning process—any string of outcomes that could result from $n$ experiments. The conditional expectation ${\mathbb{E}_\mu }[f|{\mathfrak{F}}_n]:{\{ 0,1\} ^\mathbb{N}} \to \mathbb{R}$ of $f$ given ${\mathfrak{F}}_n$ is itself a random variable that, on input $\omega \in {\{ 0,1\} ^\mathbb{N}}$ , returns the best estimate of $f$ ’s value, from the perspective of $\mu $ , conditional on the first $n$ digits $\omega\restriction n$ of $\omega $ . More suggestively, when $\omega $ is the true state of the world, ${\mathbb{E}_\mu }[f|{\mathfrak{F}}_n]\left( \omega \right)$ can be seen as encoding the agent’s beliefs regarding the true value of $f$ (namely, $f\left( \omega \right)$ ) after having observed the outcomes $\omega \restriction n$ of the first $n$ experiments. We use throughout the following version of the conditional expectation (because it is unique only up to $\mu $ -measure zero)—though, as will become clear, this choice is immaterial for our results. For all $\omega \in {\{ 0,1\} ^\mathbb{N}}$ ,

$${\mathbb{E}_\mu }[{\mkern 1mu} f\mid {_n}](\omega ) = \left\{ {\frac{1}{{\mathop \mu \limits_0 ([\omega n])}}{\mkern 1mu} \int_{[\omega n]} {\mkern 1mu} f{\mkern 1mu} d\mu {\rm{ }}\begin{array}{*{20}{c}} {{\rm{if}}\mu ([\omega n])} \\ {{\rm{otherwise}}} \\ \end{array}{\rm{ \gt }}\,0,\,{\mkern 1mu} {\rm{and }}} \right.$$

Lévy’s Upward Theorem is the following result (see Williams Reference Williams1991, sec. 14.2):

Call the set of sequences $\{ \omega \in {\{ 0,1\} ^\mathbb{N}}:{\rm{\;li}}{{\rm{m}}_{n \to \infty }}{\mathbb{E}_\mu }[f|{\mathfrak{F}}_n]\left( \omega \right) = f\left( \omega \right)\} $ that satisfy Lévy’s Upward Theorem the success set of agent $\mu $ with respect to $f$ and its complement the failure set. Lévy’s Upward Theorem says that, from the agent’s viewpoint, their failure set is negligible: the agent assigns probability one to the hypothesis that their beliefs will eventually converge to the truth about the value of $f$ (i.e., $\mu (\{ \omega \in {\{ 0,1\} ^\mathbb{N}}:{\rm{\;li}}{{\rm{m}}_{n \to \infty }}{\mathbb{E}_\mu }[\,f|{\mathfrak{F}}_n]\left( \omega \right) = f\left( \omega \right)\} ) = 1$ ).

The philosophical literature on Lévy’s Upward Theorem generally focuses on the special case in which the integrable random variable being estimated is the indicator function ${{\mathbb {1}}_{\mathcal A}}$ of some Borel set ${\cal A}$ (as we shall see, this is the setting within which Belot frames his objection). This restriction corresponds to the case in which the inductive problem faced by the agent is a binary decision problem: does the true world—the observed data stream—belong to ${\cal A}$ ? Or, put differently, does the true world possess the property corresponding to ${\cal A}$ ? In this setting, the quantity that the agent is trying to estimate is the truth value of ${\cal A}$ , and learning proceeds by standard Bayesian conditioning. Whenever $\mu([\omega\restriction n]) \gt 0$ , we in fact have that

$${\mathbb{E}_\mu }[{\mathbb {1}_{\cal A}}|{\mathfrak{F}}_n]\left( \omega \right) = \frac{1}{{\mu ([\omega |n])}}\int_{[\omega |n]} {\mathbb {1}_{\cal A}d\mu } = \frac{{\mu ({\cal A} \cap [\omega |n])}}{{\mu ([\omega |n])}} = \mu ({\cal A}|[\omega |n]).$$

So, because the support $\text{supp}(\mu)=\{\omega\in\{0,1\}^\mathbb{N}:\, (\forall {\it{n}})\, \mu([\omega \restriction {\it n}]) \gt 0\}$ of $\mu $ has $\mu $ -probability one, Lévy’s Upward Theorem entails that $\lim_{n\to\infty} \mu(\mathcal{A}\mid [\omega\restriction n]) = \mathbb {1}_\mathcal{A}(\omega)$ for $\mu $ -almost every $\omega \in {\{ 0,1\} ^\mathbb{N}}$ . An agent with prior $\mu $ expects their beliefs, given by the preceding sequence of posterior probabilities, to converge almost surely to the truth about whether ${\cal A}$ is the case with increasing information.

The almost-sure convergence to the truth achieved via Lévy’s Upward Theorem is always relative to the agent’s prior. Before performing any experiments or measurements, the agent is essentially certain that, with increasing information, their beliefs will eventually converge to the truth. Yet, there is no objective or external guarantee that this will indeed be the case. Thus Lévy’s Upward Theorem does not establish the universal reliability of Bayesian learning methods from an objective, third-person standpoint. Its epistemic significance stems from the fact that it establishes that a certain kind of skepticism about induction is impossible: if an agent is independently committed to probabilistic coherence, then, by Lévy’s Upward Theorem, that agent cannot be a skeptic about the possibility of learning from experience. The agent’s independent commitment to the Bayesian framework implies that, by their own light, their recourse to inductive reasoning is justified. As Skyrms (Reference Skyrms1984, 62) observes, from the perspective of a Bayesian agent, it is “inappropriate for you to ask the standard question, ‘Why should I believe that the real situation is not in that set of measure zero?’ The measure in question is your degree of belief. You do believe that the real situation is not in that set, with degree of belief one.”

2.3 Belot’s argument

Precisely because of its barring a certain type of skepticism about induction, Lévy’s Upward Theorem has been accused of being a drawback of the Bayesian approach, rather than serving in its favor. In particular, Belot (Reference Belot2013) argues that, because of Lévy’s Upward Theorem (and other convergence-to-the-truth results), Bayesian reasoners are forced to be epistemically immodest in an especially pernicious way. Belot’s worry is that

Bayesian reasoners are, in a sense, incapable of entertaining the possibility of inductive failure, and this is so even when “their desire to reach the truth” is thwarted along many data streams—in fact, on a topologically typical collection of data streams.

To make his point,^{Footnote 6} Belot (Reference Belot2013) considers a specific class of priors, which he calls open-minded. The notion of an open-minded prior is, by definition, always relative to a particular Borel set—more suggestively, to a particular hypothesis under consideration: given some ${\cal S} \in \mathfrak{B}$ , $\mu $ is open-minded with respect to ${\cal S}$ if, no matter what string $\sigma \in {\{ 0,1\} ^{ \lt \mathbb{N}}}$ has been observed so far, there are always two possible distinct extensions $\tau ,\rho \in {\{ 0,1\} ^{ \lt \mathbb{N}}}$ of $\sigma $ (i.e., $\sigma \sqsubset \tau $ and $\sigma \sqsubset \rho $ ) such that $\mu ({\cal S}|\left[ \tau \right]) \ge 1/2$ and $\mu ({\cal S}|\left[ \rho \right]) \lt 1/2$ . If $\mu $ is open-minded with respect to ${\cal S}$ , then no finite number of observations will ever suffice for $\mu $ to settle on whether the data stream being observed belongs to ${\cal S}$ .

Now, suppose the hypothesis under consideration corresponds to a countable dense Borel set ${\cal D}$ (e.g., ${\cal D}$ could be the set of sequences that are eventually $0$ ). Given a Bayesian agent with prior $\mu $ , what do the success set and the failure set of $\mu $ with respect to the binary estimation problem encoded by ${\mathbb{1}_{\mathcal D}}$ look like? The answer to this question depends, of course, on the particular prior adopted by the agent. Because ${\cal D}$ and its complement are both dense, any finite sequence of observations is compatible with the true data stream being in ${\cal D}$ but also with it not being in ${\cal D}$ . Hence, according to Belot, in this case, it is reasonable to adopt a prior $\mu $ that is open-minded with respect to ${\cal D}$ . Yet, Belot shows, if $\mu $ is open-minded with respect to ${\cal D}$ , then its failure set—the set of sequences $\omega \in {\{ 0,1\} ^\mathbb{N}}$ along which the conditional probabilities $\mu(\mathcal{D}\mid[\omega\restriction n])$ fail to converge to $\mathbb {1}{_{\cal D}}\left( \omega \right)$ in the limit—is co-meager, despite being a $\mu $ -measure-zero set by Lévy’s Upward Theorem. Equivalently, the success set of $\mu $ relative to ${\mathbb {1}_{\cal D}}$ is meager (and, so, topologically negligible), despite having probability one, according to the agent. Probabilistic and topological typicality are thus orthogonal notions in this setting.

In light of these (and other analogous) observations, Belot (Reference Belot2013, 484) concludes that the Bayesian approach is irremediably flawed: the account of rationality it yields “renders a certain sort of arrogance rationally mandatory, requiring agents to be certain that they will be successful at certain tasks, even in cases where the task is so contrived as to make failure the typical outcome.”

3.1 The Borel hierarchy and the arithmetical hierarchy

The events in $\mathfrak{B}$ can be classified in terms of their rank, or descriptive complexity, within the Borel hierarchy (see, e.g., Kechris Reference Kechris1995, sec. 11B):

For instance, the ${\bf{\Pi }}_1^0$ sets are the closed sets, the ${\bf{\Delta }}_1^0$ sets are the clopen sets, the ${\bf{\Sigma }}_2^0$ sets are countable unions of closed sets, and the ${\bf{\Pi }}_2^0$ sets are countable intersections of open sets.

The Borel hierarchy has an effective counterpart called the arithmetical hierarchy, which allows to classify certain Borel sets in terms of their arithmetical complexity (see, e.g., Soare Reference Soare2016, chap. 14; Downey and Hirschfeldt Reference Downey and Hirschfeldt2010, sec. 2.19):

For instance, the $\Pi _1^0$ sets are the effectively closed sets, the $\Delta _1^0$ sets are the (effectively) clopen sets, the $\Sigma _2^0$ sets are effective countable unions of effectively closed sets, and the $\Pi _2^0$ sets are effective countable intersections of effectively open sets. A set with a classification within the arithmetical hierarchy is said to be arithmetical (or arithmetically definable).

The levels of the arithmetical hierarchy can also be characterized in terms of the complexity of the formulas in the language of first-order arithmetic that define the sets belonging to those levels (hence the name “arithmetical hierarchy”). A set ${\cal S} \in \mathfrak{B}$ is in ${\rm{\Sigma }}_n^0$ if and only if it is definable by a ${\rm{\Sigma }}_n^0$ formula, namely, if $\mathcal{S} = \{\omega\in \{0,1\}^\mathbb{N}:\, (\exists k_1)(\forall k_2) ... (Q k_n)\, R(\omega \restriction k_1, \omega \restriction k_2, ..., \omega \restriction k_n)\}$ for some computable relation $R$ , with $Q = \exists $ if $n$ is odd and $Q = \forall $ if $n$ is even. On the other hand, a set ${\cal S} \in \mathfrak{B}$ is in ${\rm{\Pi }}_n^0$ if and only if it is definable by a ${\rm{\Pi }}_n^0$ formula, namely, if there is a computable relation $R$ such that $\mathcal{S} = \{\omega\in\{0,1\}^\mathbb{N}:\, (\forall k_1)(\exists k_2) ... (Q k_n)\, R(\omega\restriction k_1, \omega\restriction k_2, ..., \omega\restriction k_n)\}$ , with $Q = \forall $ if $n$ is odd and $Q = \exists $ if $n$ is even.

3.2 Algorithmic randomness and effective genericity

As notions of “largeness” go, having measure one and being co-meager are rather coarse-grained. There are many sets that, while measure-theoretically or topologically (a)typical, seem to intuitively differ in “size.” For instance, the concept of Hausdorff dimension, which is a generalization of the uniform measure, was introduced to formalize the intuition that certain subsets of a metric space differ in size, even though, from the viewpoint of the uniform measure, they all have measure zero. In what follows, we will consider some more refined notions of typicality whose definitions rely on the machinery of computability theory. These notions of effective typicality allow one to make more fine-grained distinctions between intuitively “large” sets. Here we will use them to provide a more detailed analysis of the collections of data streams along which convergence to the truth holds for several classes of inductive problems.

Let us start with algorithmic randomness: a branch of computability theory that offers an account of effective measure-theoretic typicality (see Nies Reference Nies2009; Downey and Hirschfeldt Reference Downey and Hirschfeldt2010). According to algorithmic randomness, given a probability measure $\mu $ fixed in the background, a sequence is random relative to $\mu $ if it is a representative outcome of $\mu $ . One naive idea is that an outcome is representative of $\mu $ if it satisfies every property that, according to $\mu $ , “most” sequences possess, that is, if it belongs to all $\mu $ -measure-one sets. Given that being representative in this sense is not possible in general, ^{Footnote 9} algorithmic randomness instead identifies representativeness—and, so, randomness—with membership in certain countable collections of $\mu $ -measure-one sets: more precisely, $\mu $ -measure-one sets of a certain arithmetical complexity that correspond to natural statistical laws (such as the set of sequences that satisfy the Strong Law of Large Numbers relative to $\mu $ ). In a nutshell, given such a countable collection of arithmetically definable properties that hold with $\mu $ -measure one (of arithmetically definable statistical laws), a sequence is algorithmically $\mu $ -random relative to that collection if and only if it possesses all of the corresponding properties (if and only if it satisfies all of the corresponding statistical laws).

The field of algorithmic randomness is teeming with notions of differing logical strength, each determined by the particular family of measure-one arithmetically definable properties a sequence must satisfy to count as random. In what follows, we focus on two such notions.

Arguably, the simplest algorithmic randomness notion is Kurtz randomness (Kurtz Reference Kurtz1981):

In other words, to qualify as $\mu $ -Kurtz random, a sequence has to possess all the properties that correspond to $\mu $ -measure-theoretically typical effectively open subsets of Cantor space (such as the property of having at least one prime-numbered $0$ entry when $\mu $ is a nontrivial Bernoulli measure). Given that there are only countably many ${\rm{\Sigma }}_1^0$ sets, there are only countably many of them that have $\mu $ -measure one. Hence, for every $\mu $ , the collection of $\mu $ -Kurtz random sequences is itself a $\mu $ -measure-one set.

Another fundamental algorithmic randomness notion is Martin-Löf randomness (Martin-Löf Reference Martin-Löf1966), which can easily be seen to entail Kurtz randomness:

The requirement that, for a $\mu $ -Martin-Löf test ${\{ {{\cal U}_n}\} _{n \in \mathbb{N}}}$ , $\mu \left( {{{\cal U}_n}} \right) \le {2^{ - n}}$ for all $n \in \mathbb{N}$ ensures that $\bigcap_{n\in\mathbb{N}}\mathcal{U}_n$ is a set of effective $\mu $ -measure zero: it is a $\mu $ -measure-zero set whose measure can be approximated at a computable rate ( ${2^{ - n}}$ ) using the measures of the components ${{\cal U}_n}$ of the test. And because the intersection of a computable sequence of ${\rm{\Sigma }}_1^0$ sets is a ${\rm{\Pi }}_2^0$ set, a sequence $\omega\in\{0, 1\}^\mathbb{N}$ is $\mu $ -Martin-Löf random if and only if it does not possess any ${\rm{\Pi }}_2^0$ properties of effective $\mu $ -measure zero—equivalently, if and only if it possesses all ${\rm{\Sigma }}_2^0$ properties of effective $\mu $ -measure one. Once again, seeing that there are only countably many ${\rm{\Sigma }}_2^0$ properties of (effective) $\mu $ -measure one, the collection of $\mu $ -Martin-Löf random sequences is itself a $\mu $ -measure-one set.

Since $\mu $ -Martin-Löf randomness entails $\mu $ -Kurtz randomness, while the reverse implication does not hold in general, $\mu $ -Martin-Löf randomness yields a more fine-grained notion of measure-theoretic typicality than $\mu $ -Kurtz randomness does; in turn, $\mu $ -Kurtz randomness provides a more fine-grained notion of measure-theoretic typicality than simply having $\mu $ -measure one.

The second family of effective typicality notions that will be relevant for our discussion falls under the umbrella of effective genericity: a theory of effective topological typicality (see Downey and Hirschfeldt Reference Downey and Hirschfeldt2010, sec. 2.24). Just as algorithmic randomness is defined in terms of membership in every measure-one set from some prespecified countable collection of sets, effective genericity essentially amounts to membership in every co-meager set from some prespecified countable collection of sets. Although there are many notions of effective genericity in the literature, here we only consider the $n$ -genericity hierarchy: a linearly ordered family of canonical genericity notions. We begin by defining $1$ -genericity (the first level of the hierarchy) and discuss the rest of the hierarchy in the last section of the article.

Given $\omega \in {\{ 0,1\} ^\mathbb{N}}$ and $S \subseteq {\{ 0,1\} ^{ \lt \mathbb{N}}}$ , $\omega $ is said to meet $S$ if ${\omega\in[S]=\bigcup_{\sigma\in S}[\sigma]}$ . A set $S \subseteq {\{ 0,1\} ^{ \lt \mathbb{N}}}$ is dense along $\omega \in {\{ 0,1\} ^\mathbb{N}}$ if $\omega $ is in the closure of $\left[ S \right]$ —in other words, if, for every $n \in \mathbb{N}$ , there is some $\sigma \in {\{ 0,1\} ^{ \lt \mathbb{N}}}$ with $\omega\restriction n\sqsubseteq \sigma$ such that $\left[ \sigma \right] \subseteq \left[ S \right]$ . ^{Footnote 11} Then, $1$ -genericity is defined as follows:

Equivalently, a sequence is $1$ -generic if and only if it is not on the boundary of any ${\rm{\Sigma }}_1^0$ set. Intuitively, a $1$ -generic sequence $\omega $ is such that, for any ${\rm{\Sigma }}_1^0$ hypothesis ${\cal S}$ , if no imprecise measurement of $\omega $ can rule ${\cal S}$ out, then $\omega $ is in ${\cal S}$ (and, so, satisfies the hypothesis).

It is not difficult to see that every $1$ -generic sequence belongs to every dense ${\rm{\Sigma }}_1^0$ set and that every such set is co-meager. Moreover, the following well-known facts will be important for our discussion (see Kurtz Reference Kurtz1981; Nies Reference Nies2009; Downey and Hirschfeldt Reference Downey and Hirschfeldt2010).

When $\mu $ is a probability measure with full support, the set of $\mu $ -Kurtz random sequences is thus itself a co-meager set. So, both $1$ -genericity and $\mu $ -Kurtz randomness provide more fine-grained notions of topological typicality than simply being co-meager.

3.3 Continuous functions

With these taxonomic tools at our disposal, we are ready to consider some specific families of inductive problems for which Bayesian convergence to the truth is not susceptible to Belot’s charge of epistemic immodesty.

One way to achieve co-meager success is, of course, to be inductively successful no matter what data stream is observed, that is, for convergence to the truth to occur everywhere, not just almost everywhere according to the agent’s prior. So, an immediate question is whether there are any ordinary learning situations in which convergence to the truth can be achieved everywhere. In what follows, we highlight one such class of learning situations that will guide the rest of our discussion.

One simple type of inductive problem consists in estimating a continuous quantity. Quantities of this kind are naturally modeled in terms of continuous random variables. First, recall that the standard topology on $\mathbb{R}$ is the topology generated by the open intervals, namely, by sets of the form $\left( {a,b} \right) = \{ r \in \mathbb{R}:a,b \in \mathbb{R}\;{\rm{and\;}}a \lt r \lt b\} $ . Earlier, we introduced the Borel hierarchy of subsets of ${\{ 0,1\} ^\mathbb{N}}$ (Definition 3.1). The very same taxonomy in terms of ${\bf{\Sigma }}_n^0,{\bf{\Pi }}_n^0$ , and ${\bf{\Delta }}_n^0$ sets also applies to the Borel subsets of $\mathbb{R}$ , where the Borel subsets of $\mathbb{R}$ are the elements of the Borel $\sigma $ -algebra on $\mathbb{R}$ : the smallest $\sigma $ -algebra containing all open intervals. Continuous functions from ${\{ 0,1\} ^\mathbb{N}}$ to $\mathbb{R}$ are defined as follows:

Suppose an experiment is being conducted that involves measuring some real-valued physical parameter, such as the temperature at a given location or the concentration of some substance in a fluid. Such a learning situation may be modeled via the function $f:{\{ 0,1\} ^\mathbb{N}} \to \mathbb{R}$ that maps each sequence in ${\{ 0,1\} ^\mathbb{N}}$ to the real number in $\left[ {0,1} \right]$ of which that sequence is the binary expansion. Let the true parameter be given by $f\left( \omega \right)$ . Then, at each finite stage $n$ of the learning process, the observed data $\omega\restriction n$ provides an approximation of $f\left( \omega \right)$ . The map $f$ is a continuous function. For another simple example of a continuous function, let ${\cal U}$ be a clopen subset of ${\{ 0,1\} ^\mathbb{N}}$ and take its indicator function ${\mathbb{1}_\mathcal{U}}$ . Intuitively, ${\mathbb{1}_\mathcal{U}}$ represents a binary decision problem that can be settled with a finite amount of data (such as the question of whether the first $n$ patients from a given sample all recovered after being treated for a certain disease).

The following observation is entirely straightforward, but it is a useful starting point. First, note that, because Cantor space is compact, ^{Footnote 13} every continuous function on it is bounded both below and above; ^{Footnote 14} hence, every continuous random variable on Cantor space is integrable. When the quantity to be estimated is a continuous random variable, it is easy to see that Lévy’s Upward Theorem holds for every sequence in the support of the agent’s prior. So, when the agent’s prior has full support, Lévy’s Upward Theorem holds everywhere.

Proof. Let $\omega \in {\rm{supp}}\left( \mu \right)$ . Then, for all $n \in \mathbb{N}$ , $\mathbb{E}_\mu[f\mid {\mathfrak{F}}_n](\omega)={\int_{[\omega\restriction n]} f\, d\mu} /{\mu([ \omega\restriction n])}$ . Let $\epsilon \gt 0$ . By continuity, there is $m \in \mathbb{N}$ such that, for all $\alpha\in[\omega\restriction m]$ , $\left| {f\left( \omega \right) - f\left( \alpha \right)} \right| \lt \epsilon $ . Hence, for all $n \ge m$ ,

$\Bigg|{\frac{\int_{[\omega\restriction n]} f\,d\mu} \over {\mu([\omega\restriction n])}} - f(\omega)\Bigg| \leq {\frac{\int_{[\omega\restriction n]} |f - f(\omega)|\,d\mu} \over {\mu([\omega\restriction n])}} \lt {\frac{\int_{[\omega\restriction n]} \epsilon\,d\mu} \over {\mu([\omega\restriction n])}} = \epsilon,$

where the first inequality holds because $f\left( \omega \right)$ is a constant. This establishes the claim.

When the agent’s prior $\mu $ does not have full support, convergence to the truth is not guaranteed to happen on a co-meager set. To see this, let $\mu $ be the probability measure that results from first flipping a coin that lands heads with probability one and then flipping a fair coin forever after. ^{Footnote 15} Take the indicator function ${\mathbb{1}_{\left[ 0 \right]}}$ of the cylinder $\left[ 0 \right]$ , which, as noted earlier, is continuous. Lévy’s Upward Theorem fails everywhere on $\left[ 0 \right]$ , so the success set of $\mu $ with respect to ${\mathbb{1}_{\left[ 0 \right]}}$ is not co-meager, as the cylinder $\left[ 1 \right]$ is not co-meager.

3.4 Baire class $n$ functions

Continuity, though natural, is a strong condition. What we will consider next is a family of functions that, relying on the classifications afforded by the Borel hierarchy, provides a broad generalization of the class of continuous functions:

Clearly the collection of Baire class $0$ functions coincides with the collection of continuous functions. Moreover, for each $n \in \mathbb{N}$ , every Baire class $n$ function is also a Baire class $\left( {n + 1} \right)$ function (while the converse does not hold).

For each $n \ge 1$ , the indicator functions of the ${\bf{\Delta }}_n^0$ , ${\bf{\Sigma }}_n^0$ , ${\bf{\Pi }}_n^0$ , and ${\bf{\Delta }}_{n + 1}^0$ subsets of Cantor space are straightforward examples of Baire class $n$ functions. These functions have natural epistemic interpretations. For instance, the indicator functions of ${\bf{\Sigma }}_1^0$ sets intuitively capture binary decision problems membership in which can be verified with a finite amount of data, while the indicator functions of ${\bf{\Pi }}_1^0$ sets intuitively capture binary decision problems membership in which can be refuted with a finite amount of data. Similarly, the indicator functions of ${\bf{\Sigma }}_2^0$ sets correspond to binary decision problems membership in which can be verified in the limit, the indicator functions of ${\bf{\Pi }}_2^0$ sets correspond to binary decision problems membership in which can be refuted in the limit, and the indicator functions of ${\bf{\Delta }}_2^0$ sets correspond to binary decision problems membership in which can be decided in the limit. ^{Footnote 16}

Another family of functions that are of Baire class $1$ (in addition to the indicator functions of ${\bf{\Delta }}_1^0$ , ${\bf{\Sigma }}_1^0$ , ${\bf{\Pi }}_1^0$ , and ${\bf{\Delta }}_2^0$ sets) is the class of semicontinuous functions, which includes the lower semicontinuous and the upper semicontinuous functions:

Semicontinuity is a weaker form of continutiy, and it is not difficult to see that a function is continuous if and only if it is both lower semicontinuous and upper semicontinuous. The collection of lower semicontinuous functions includes the indicator functions of ${\bf{\Sigma }}_1^0$ sets, while the collection of upper semicontinuous functions includes the indicator functions of ${\bf{\Pi }}_1^0$ sets. For another simple example of a lower semicontinuous function (and, so, of a Baire class $1$ function), let $f$ be given by $f\left( \omega \right) = 1$ if $\omega $ ’s prime-numbered entries feature a $1$ infinitely often and $f\left( \omega \right) = 1 - {2^{ - n}}$ if $\omega $ has exactly $n \in \mathbb{N}$ prime-numbered entries featuring a $1$ (basically, for any $\omega $ , $f$ is a normalized function that counts the number of $1$ s in $\omega $ that occur at prime-numbered positions). For one last example, take a bounded function $c:{\{ 0,1\} ^{ \lt \mathbb{N}}} \to \mathbb{R}$ recording the daily values of some stock market share. Then, the function $f$ given by $f(\omega) = \sup_{n,m \in\mathbb{N}}|c(\omega\restriction n) - c({\omega\restriction m})|$ , which tracks the greatest spread between this share’s values over its history, is lower semicontinuous. Similarly, the function $g$ given by $g(\omega) = \inf_{n,m\in\mathbb{N}}|c(\omega\restriction n) - c({\omega\restriction m})|$ , which tracks the lowest spread between the share’s values over its history, is upper semicontinuous.

The following is a classical result due to Baire (see Kechris Reference Kechris1995, Theorem 24.14; Oxtoby Reference Oxtoby1980, Theorem 7.3) that will help us shed light on Lévy’s Upward Theorem in the context of Baire class $1$ functions:

With Theorem 3.12 at hand, the following can easily be seen to hold:

Proof. Let $\omega $ be a point of continuity of $f$ . Because $\mu $ has full support, $\mathbb{E}_\mu[f\mid{\mathfrak{F}}_n](\omega)={\int_{[\omega\restriction n]}f\, d\mu}/{\mu([ \omega\restriction n])}$ for all $n \in \mathbb{N}$ . By the very same argument used in the proof of Proposition 3.9, ${\rm{li}}{{\rm{m}}_{n \to \infty }}{\mathbb{E}_\mu }[f|{\mathfrak{F}}_n]\left( \omega \right) = f\left( \omega \right)$ . So, the set of points of continuity of $f$ is a subset of the set of sequences along which Lévy’s Upward Theorem holds. By Theorem 3.12, the former set is co-meager. Hence, so is the set of sequences along which Lévy’s Upward Theorem holds.

When the underlying prior $\mu $ has full support, the success set of $\mu $ relative to a Baire class $1$ integrable random variable is a co-meager set. We thus have another class of inductive problems relative to which convergence to the truth is not only probabilistically typical but also topologically typical.

There are priors that do not have full support for which Corollary 3.13 does not hold. By virtue of being continuous, the indicator function ${\mathbb{1}_{\left[ 0 \right]}}$ of $\left[ 0 \right]$ is also of Baire class $1$ , and we have already mentioned an example of a probability measure that does not have full support whose success set with respect to ${\mathbb{1}_{\left[ 0 \right]}}$ fails to be co-meager.

For $n \ge 2$ , it is not in general true that the points of discontinuity of a Baire class $n$ function form a meager set. Consider once again the set of all sequences that are eventually $0$ , that is, the set ${\cal Z} = \{ \omega \in {\{ 0,1\} ^\mathbb{N}}:{\rm{\;}}\left( {\exists n} \right)(\forall m \gt n){\rm{\;}}\omega \left( m \right) = 0\} $ . The indicator function ${\mathbb{1}_{\cal Z}}$ of this set is of Baire class $2$ , yet ${\mathbb{1}_{\cal Z}}$ is discontinuous everywhere. Hence, the points of continuity of ${\mathbb{1}_{\cal Z}}$ not only fail to form a co-meager set; they form a meager set (because this set is empty).

Of course, this remark does not preclude the possibility that an analogue of Corollary 3.13 may hold for Baire class $n$ integrable random variables in general. The following proposition establishes that Corollary 3.13 does not generalize:

Proof. Let $\mu{-MLR}$ denote the set of $\mu $ -Martin-Löf random sequences (Definition 3.4). The computability of $\mu $ ensures the existence of a universal $\mu $ -Martin-Löf test: a $\mu $ -Martin-Löf test ${\{ {{\cal V}_n}\} _{n \in \mathbb{N}}}$ such that, to determine whether a sequence is $\mu $ -Martin-Löf random, it suffices to check whether that sequence belongs to $\bigcap_{n\in\mathbb{N}}\mathcal{V}_n$ (if it does, then the sequence is not $\mu $ -Martin-Löf random; if it does not, then it is) (see Downey and Hirschfeldt Reference Downey and Hirschfeldt2010, Theorem 6.2.5). Then, $\mu\textsf{-MLR}= \overline{\bigcap_{n\in\mathbb{N}}\mathcal{V}_n}$ . Because $\bigcap_{n\in\mathbb{N}}\mathcal{V}_n$ is a ${\rm{\Pi }}_2^0$ set, $\mu\textsf{-MLR}$ is a ${\Sigma }_2^0$ set. A fortiori, $\mu\textsf{-MLR}$ is in ${\bf{\Sigma }}_2^0$ . The set $\bigcap_{n\in\mathbb{N}}\mathcal{V}_n$ is also dense. For, suppose not. Then, there is some $\sigma \in {\{ 0,1\} ^{ \lt \mathbb{N}}}$ with $\big(\bigcap_{n\in\mathbb{N}}\mathcal{V}_n\big)\cap[\sigma]=\emptyset$ . Hence, $\left[ \sigma \right] \subseteq \mu\textsf{-MLR}$ . Take a computable sequence in $\left[ \sigma \right]$ that is not a $\mu $ -atom. We can find such a sequence as follows. Because $\mu $ is computable, we can computably find ${\tau _1} \in {\{ 0,1\} ^{ \lt \mathbb{N}}}$ with $\mu \left( {\left[ {\sigma {\tau _1}} \right]} \right) \lt {{1} \over {2}}$ by dovetailing through the cylinders contained in $\left[ \sigma \right]$ , approximating their respective measures from above. And, given ${\tau _1},...,{\tau _n} \in {\{ 0,1\} ^{ \lt \mathbb{N}}}$ with $\mu \left( {\left[ {\sigma {\tau _1}...{\tau _n}} \right]} \right) \lt {2^{ - n}}$ , we can computably find ${\tau _{n + 1}} \in {\{ 0,1\} ^{ \lt \mathbb{N}}}$ with $\mu \left( {\left[ {\sigma {\tau _1}...{\tau _{n + 1}}} \right]} \right) \lt {2^{ - \left( {n + 1} \right)}}$ by following the same procedure inside $\left[ {\sigma {\tau _1}...{\tau _n}} \right]$ . Then, $\omega = \sigma {\tau _1}{\tau _2}...$ is a computable sequence with $\mu \left( {\left\{ \omega \right\}} \right) = 0$ . Since the only way for a computable sequence to be $\mu $ -Martin-Löf random is to be a $\mu $ -atom, $\omega $ is not $\mu $ -Martin-Löf random. But this contradicts the fact that $\left[ \sigma \right] \subseteq \mu\textsf{-MLR}$ . Hence, $\bigcap_{n\in\mathbb{N}}\mathcal{V}_n$ is indeed dense. Clearly $\mu\textsf{-MLR} = \bigcup_{n\in\mathbb{N}} \overline{\mathcal{V}_n}$ , where each set $\overline {{{\cal V}_n}} $ is ${\rm{\Pi }}_1^0$ and, so, closed. By the density of $\bigcap_{n\in\mathbb{N}}\mathcal{V}_n$ , no $\overline {{{\cal V}_n}} $ contains any nonempty open sets. Therefore each $\overline {{{\cal V}_n}} $ is such that its closure has empty interior; that is, each $\overline {{{\cal V}_n}} $ is nowhere dense. Hence $\mu\textsf{-MLR}$ is meager. The indicator function ${\mathbb{1}_{\mu\textsf{-MLR}}}$ of $\mu\textsf{-MLR}$ is a Baire class $2$ integrable random variable. And because $\mu \left( {\mu\textsf{-MLR}} \right) = 1$ and $\mu $ has full support, the set of sequences $\alpha \in {\{ 0,1\} ^\mathbb{N}}$ with ${\rm{li}}{{\rm{m}}_{n \to \infty }}{\mathbb{E}_\mu }[{\mathbb{1}_{\mu\textsf{-MLR}}}|{\mathfrak{F}}_n]\left( \alpha \right) = {\mathbb{1}_{\mu\textsf{-MLR}}}\left( \alpha \right)$ coincides with $\mu{-MLR}$ . Therefore the set of sequences along which Lévy’s Upward Theorem holds for ${\mathbb{1}_{\mu\textsf{-MLR}}}$ is meager. For $n \gt 2$ , we can then reason as follows. ^{Footnote 18} The notion of $\mu $ -Kurtz randomness (Definition 3.3) can be generalized to arbitrary levels of the arithmetical hierarchy: for any $n \ge 1$ , a sequence is $\mu $ -weakly $n$ -random if and only if it belongs to every ${\rm{\Sigma }}_n^0$ set of $\mu $ -measure one (so, $\mu $ -Kurtz randomness coincides with $\mu $ -weak $1$ -randomness). The set of $\mu $ -weakly $n$ -random sequences is in ${\bf{\Pi }}_{n + 1}^0$ and has $\mu $ -measure one. Moreover, $\mu $ -weak $\left( {n + 1} \right)$ -randomness entails $\mu $ -weak $n$ -randomness, and $\mu $ -weak $2$ -randomness entails $\mu $ -Martin-Löf randomness (see Downey and Hirschfeldt Reference Downey and Hirschfeldt2010, sec. 7.2). Hence, for each $n \gt 2$ , by the same argument used for ${\mathbb{1}_{\mu\textsf{-MLR}}}$ , the indicator function of the set of $\mu $ -weakly $\left( {n - 1} \right)$ -random sequences is an example of a Baire class $n$ integrable random variable for which convergence to the truth occurs on a meager set.

Proposition 3.9 and Corollary 3.13 circumscribe the reach of Belot’s objection: they establish that, at least for relatively simple inductive problems, convergence to the truth is topologically typical, in addition to being probabilistically typical. Proposition 3.14 pulls in the opposite direction. Not only does it show that, for more complex classes of inductive problems, co-meager success is not always achievable, but it also reveals that the dichotomy problematized by Belot is perhaps more pervasive than one might have initially thought. Past the level of Baire class $1$ integrable random variables, co-meager failure can easily be found at every level of the Borel hierarchy.

3.5 Computable and almost everywhere computable functions

Though well behaved, all of the functions considered so far were allowed to be arbitrarily computationally complex. We will now concentrate on effective functions—functions whose values are in some sense calculable or approximable—and provide several examples of effective random variables for which convergence to the truth is topologically typical.

The very same taxonomy we discussed in the context of the arithmetical hierarchy of subsets of Cantor space also applies to the arithmetical subsets of $\mathbb{R}$ . Here the ${\rm{\Sigma }}_1^0$ sets are those that can be expressed as a computably enumerable union of open intervals with rational endpoints, while the other levels of the hierarchy are defined in the same way as in the Cantor space setting. Computable functions from ${\{ 0,1\} ^\mathbb{N}}$ to $\mathbb{R}$ are defined as follows:

It is not difficult to see that the computable functions are precisely those functions whose values can be computably approximated to any degree of precision (via a computable sequence of rational-valued step functions) (see, e.g., Li and Vitányi Reference Li and Vitányi2019, 35–36).

Every open set in the standard topology on the reals can be expressed as a (countable) union of ${\rm{\Sigma }}_1^0$ sets. Moreover, every ${\rm{\Sigma }}_1^0$ subset of Cantor space is open. Therefore every computable function is continuous. Consequently, Proposition 3.9 holds for computable random variables as well. And when $\mu $ has full support, the set of sequences along which Lévy’s Upward Theorem holds is co-meager. So, a Bayesian agent with a prior with full support trying to estimate a computable quantity is guaranteed to be inductively successful on a topologically typical collection of data streams (in fact, along every data stream).

Much like its classical counterpart—the concept of a continuous function—the notion of a computable function is rather demanding. The following example, taken from Ackerman, Freer, and Roy (Reference Ackerman, Freer and Roy2019), nicely illustrates this point. Let $\theta \in \left[ {0,1} \right]$ . A $\left\{ {0,1} \right\}$ -valued random variable $f:{\{ 0,1\} ^\mathbb{N}} \to \mathbb{R}$ relative to a probability measure $\mu $ is a $\theta $ -Bernoulli random variable if $\mu (\{ \omega \in \{ {0,1{\} ^\mathbb{N}}:{\rm{\;}}f\left( \omega \right) = 1} \}) = \theta $ (i.e., if the probability that $f$ takes value $1$ is $\theta $ ). Ackerman, Freer, and Roy show that, for any $\theta \in \left[ {0,1} \right]$ that is not a dyadic rational, every $\theta $ -Bernoulli random variable fails to be continuous and, as a result, is not computable in the sense of Definition 3.15. At the same time, for any computable $\theta \in \left[ {0,1} \right]$ , there are many $\theta $ -Bernoulli random variables that, though not computable, are “very close” to being computable. The following, more permissive notion—almost everywhere computability—is thus generally regarded as the more natural one on which to focus in the context of (computable) probability theory (see Hoyrup Reference Hoyrup2008; Hoyrup and Rojas Reference Hoyrup and Rojas2009; Ackerman, Freer, and Roy Reference Ackerman, Freer and Roy2019).Footnote ²⁰

The argument from the proof of Proposition 3.9 can once again be employed to establish the following fact (because almost everywhere computable random variables are integrable):

Proof. Let ${\omega \in {\rm{supp}}\left( \mu \right) \cap {\cal D}}$ . Then, for all $n \in \mathbb{N}$ , $\small{\mathbb{E}_\mu[\,f\mid{\mathfrak{F}}_n](\omega)={\mathop \smallint \nolimits_{[\omega\restriction n]}f\,d\mu}/{\mu([\omega\restriction n])}}$ . Let ${\epsilon} \gt 0$ . Because $f$ is computable on ${\cal D}$ , $f$ is continuous on ${\cal D}$ . Hence, there is $m \in \mathbb{N}$ such that, for all $\alpha \in {\cal D}$ , if $\alpha\in[\omega\restriction m]$ , then $\left| {f\left( \omega \right) - f\left( \alpha \right)} \right| \lt {\epsilon} $ . Therefore, for all $n \ge m$ ,

$${\left| {{\frac{{\mathop \smallint \nolimits_{\left[ {\omega \restriction n} \right]} f{\rm{\;}}d\mu }} \over {{\mu \left( {\left[ {\omega \restriction n} \right]} \right)}}} - f\left( \omega \right)} \right|} { \le {\frac{{\mathop \smallint \nolimits_{\left[ {\omega \restriction n} \right]} \left| {f - f\left( \omega \right)} \right|{\rm{\;}}d\mu }} \over {{\mu \left( {\left[ {\omega \restriction n} \right]} \right)}}}} { = {\frac{{\mathop \smallint \nolimits_{\left[ {\omega \restriction n} \right] \cap {\cal D}} \left| {f - f\left( \omega \right)} \right|{\rm{\;}}d\mu }} \over {{\mu \left( {\left[ {\omega \restriction n} \right]} \right)}}}} { \lt {\frac{{\mathop \smallint \nolimits_{\left[ {\omega \restriction n} \right] \cap {\cal D}} \epsilon {\rm{\;}}d\mu }} \over {{\mu \left( {\left[ {\omega \restriction n} \right]} \right)}}}} { = \epsilon ,}$$

where both identities follow from the fact that $\mu \left( {\cal D} \right) = 1$ . This establishes the claim.

For any probability measure $\mu $ with full support, the ${\rm{\Pi }}_2^0$ collection of sequences over which a $\mu $ -almost everywhere computable function is computable is co-meager:

Proof. Let ${\{ {{\cal D}_n}\} _{n \in \mathbb{N}}}$ be a computable sequence of ${\rm{\Sigma }}_1^0$ sets with $\mathcal{D} = \bigcap_{n\in\mathbb{N}}\mathcal{D}_n$ . Then, $\mu \left( {{{\cal D}_n}} \right) = 1$ for all $n \in \mathbb{N}$ , and so each ${{\cal D}_n}$ is dense. For, suppose not. Then, there is some $n \in \mathbb{N}$ and $\sigma \in {\{ 0,1\} ^{ \lt \mathbb{N}}}$ such that ${{\cal D}_n} \cap \left[ \sigma \right] = \emptyset $ . Because $\mu \left( {\left[ \sigma \right]} \right) \gt 0$ , $\mu \left( {{{\cal D}_n}} \right) \le 1 - \mu \left( {\left[ \sigma \right]} \right) \lt 1$ , which yields a contradiction. Let ${\cal U} \subseteq {\{ 0,1\} ^\mathbb{N}}$ be an arbitrary open set. Because each ${{\cal D}_n}$ is dense, ${{\cal D}_n} \cap {\cal U} \ne \emptyset $ for all $n$ . Moreover, given that each ${{\cal D}_n}$ is open, each set ${{\cal D}_n} \cap {\cal U}$ is open in the subspace topology on ${\cal U}$ . For all $n$ , given that $\left( {\overline {{{\cal D}_n}} \cap {\cal U}} \right) \cap \left( {{{\cal D}_n} \cap {\cal U}} \right) = \emptyset $ , $\overline {{{\cal D}_n}} \cap {\cal U}$ is not dense in ${\cal U}$ . Hence, each $\overline {{{\cal D}_n}} $ is nowhere dense. Therefore $\overline{\mathcal{D}}= \bigcup_{n\in\mathbb{N}}\overline{\mathcal{D}_n}$ is meager and ${\cal D}$ is co-meager.

Propositions 3.17 and 3.18 entail the following:

Hence, a Bayesian reasoner whose prior has full support has a co-meager success set for any random variable whose values are computably approximable.

3.6 Randomness and genericity at work

Though in and of themselves significant, convergence to the truth with probability one and convergence to the truth on a co-meager set remain somewhat elusive notions. In its classical form, Lévy’s Upward Theorem is silent as to which data streams belong to the probability-one set of sequences along which convergence to the truth provably occurs. And, more generally, proving that convergence to the truth happens with probability one or on a co-meager set provides little information about the composition of the success set. It also does not tell us how the composition of this set varies depending on the particular quantity the agent is trying to estimate, nor does it indicate whether the data streams that ensure eventual convergence to the truth share any property that might explain their conduciveness to learning—that is, any significant property that sets them apart from the data streams along which learning fails. In what follows, we address these worries from the vantage point of computability theory. In particular, we will see that the theories of algorithmic randomness and effective genericity can be put to use to identify specific topologically typical collections of data streams along which convergence to the truth in the sense of Lévy’s Upward Theorem is achieved for several classes of effective random variables.

We begin by having a second look at the class of almost everywhere computable random variables.

Recall the definition of $\mu $ -Kurtz randomness (Definition 3.3). Given $\mu $ , let $\mu\textsf{-KR}$ denote the set of $\mu $ -Kurtz random sequences.

Proof. Let $\omega \in {\{ 0,1\} ^\mathbb{N}}$ . First, suppose that $\mu([\omega\restriction n])=0$ for some $n \in \mathbb{N}$ . Then, there is a ${\rm{\Pi }}_1^0$ set of $\mu $ -measure zero, $[\omega\restriction n]$ , to which $\omega $ belongs, which entails that $\omega\ \notin \mu\textsf{-KR}$ . Now, suppose that $\omega \in \overline {\cal D}$ . Because $\overline {\cal D}$ is a ${\rm{\Sigma }}_2^0$ set of $\mu $ -measure zero, $\overline{\mathcal{D}}= \bigcup_{n\in\mathbb{N}}{\mathcal{A}_n}$ , where each ${{\cal A}_n}$ is in ${\rm{\Pi }}_1^0$ and has $\mu $ -measure zero. Therefore, there is once again a ${\rm{\Pi }}_1^0$ set of $\mu $ -measure zero to which $\omega $ belongs. Hence $\omega\ \notin \mu\textsf{-KR}$ .

By combining Proposition 3.20 with Proposition 3.17, we can immediately conclude that, for any $\mu $ -almost everywhere computable random variable (and, a fortiori, any computable random variable), observing a $\mu $ -Kurtz random data stream is a sufficient condition for converging to the truth in the limit:

When $\mu $ has full support, the set $\mu\textsf{-KR}$ is co-meager. Hence, when the agent’s prior has full support, there is a precisely identifiable collection of data streams—one that is typical both probabilistically and topologically—membership in which guarantees convergence to the truth for any inductive problem that can be modeled as a $\mu $ -almost everywhere computable random variable.

Just as the computable functions are the effective analogue of the continuous functions, Baire class $n$ functions can be naturally effectivized as follows:

For each $n \ge 1$ , the indicator functions of ${\rm{\Delta }}_n^0$ , ${\rm{\Sigma }}_n^0$ , ${\rm{\Pi }}_n^0$ , and ${\rm{\Delta }}_{n + 1}^0$ sets are simple examples of effective Baire class $n$ functions (this, of course, is the effective counterpart of the fact that the indicator functions of ${\bf{\Delta }}_n^0$ , ${\bf{\Sigma }}_n^0$ , ${\bf{\Pi }}_n^0$ , and ${\bf{\Delta }}_{n + 1}^0$ sets are Baire class $n$ functions in the classical setting). For instance, the indicator functions of ${\rm{\Sigma }}_1^0$ sets, which intuitively correspond to binary decision problems that can be effectively verified with a finite amount of data, and the indicator functions of ${\rm{\Pi }}_1^0$ sets, which correspond to binary decision problems that can be effectively refuted with a finite amount of data, are all effective Baire class $1$ functions. For the first level of the hierarchy, another natural example is the collection of semicomputable functions:

Semicomputability is the effective analogue of semicontinuity: a function is computable if and only if it is both lower semicomputable and upper semicomputable. The lower semicomputable functions are those whose values can be computably approximated from below, whereas the upper semicomputable functions are those whose values can be computably approximated from above (see Li and Vitányi Reference Li and Vitányi2019, 35–36). The collection of lower semicomputable functions includes the indicator functions of ${\rm{\Sigma }}_1^0$ sets, while the collection of upper semicomputable functions includes the indicator functions of ${\rm{\Pi }}_1^0$ sets.

Given that every effective Baire class $1$ function is a Baire class $1$ function in the classical sense, Theorem 3.12 and Corollary 3.13 also apply to effective Baire class $1$ integrable random variables: when the agent’s prior has full support and the quantity to be estimated is an effective Baire class $1$ integrable random variable, Lévy’s Upward Theorem holds on a co-meager set of data streams.

But, as with the almost everywhere computable functions, adding effectivity into the mix allows us to go beyond the mere observation that we can attain co-meager success. For effective Baire class $1$ functions, $1$ -genericity (Definition 3.28) can be used to provide a more in-depth analysis of convergence to the truth via the following effectivization of Theorem 3.12:

It then immediately follows that, for any agent whose prior has full support, observing a $1$ -generic data stream is conducive to learning no matter which effective Baire class $1$ integrable random variable that agent is trying to estimate:

Because the collection of $1$ -generic sequences is co-meager, Corollary 3.25 allows one to pinpoint a single co-meager collection of data streams that guarantee convergence to the truth for all inductive problems that can be modeled as effective Baire class $1$ integrable random variables.

3.7 ${\emptyset ^{\left( k \right)}}$ -Computable functions and ${\emptyset ^{\left( k \right)}}$ -effective Baire class $n$ functions

We conclude our discussion of co-meager convergence to the truth with a more technical note: by considering two collections of functions that rely on oracle computation—the ${\emptyset ^{\left( k \right)}}$ -computable functions and the ${\emptyset ^{\left( k \right)}}$ -effective Baire class $n$ functions.

The arithmetical hierarchy can be relativized to sequences $\omega \in {\{ 0,1\} ^\mathbb{N}}$ (taken to represent the indicator function of a set of natural numbers) by letting the relation $R$ be $\omega $ -computable (i.e., computable with oracle $\omega $ ). In this way, one obtains the notions of ${\rm{\Sigma }}_n^{0,\omega }$ , ${\rm{\Pi }}_n^{0,\omega }$ , and ${\rm{\Delta }}_n^{0,\omega }$ sets. From the perspective of computability theory, an especially useful collection of oracles is the class of Turing jumps of the empty set $\emptyset $ . The zeroth jump ${\emptyset ^{\left( 0 \right)}}$ of $\emptyset $ is simply $\emptyset $ itself, which of course does not provide any additional computational power. The first jump ${\emptyset ^{\left( 1 \right)}}$ of $\emptyset $ is the halting set, namely, the set $\left\{ {n \in \mathbb{N}:{\varphi _n}\left( n \right) \downarrow } \right\}$ of all natural numbers $n$ such that the $n$ th partial computable function ${\varphi _n}$ (equivalently, $\varphi _n^{{\emptyset ^{\left( 0 \right)}}}$ ) is defined on $n$ (the Turing machine computing ${\varphi _n}$ halts on input $n$ ). For $k \gt 1$ , the $k$ th jump ${\emptyset ^{\left( k \right)}}$ of $\emptyset $ is the halting set relativized to ${\emptyset ^{\left( {k - 1} \right)}}$ , that is, the set $\{ {n \in \mathbb{N}:\varphi _n^{{\emptyset ^{\left( {k - 1} \right)}}}\left( n \right) \downarrow } \}$ of all natural numbers $n$ such that the $n$ th ${\emptyset ^{\left( {k - 1} \right)}}$ -partial computable function $\varphi _n^{{\emptyset ^{\left( {k - 1} \right)}}}$ is defined on $n$ . Using sets of the form ${\emptyset ^{\left( k \right)}}$ (or, rather, the infinite sequences corresponding to these sets) as oracles, one obtains the classes ${\rm{\Sigma }}_n^{0,{\emptyset ^{\left( k \right)}}}$ , ${\rm{\Pi }}_n^{0,{\emptyset ^{\left( k \right)}}}$ , and ${\rm{\Delta }}_n^{0,{\emptyset ^{\left( k \right)}}}$ . For instance, a set ${\cal S}$ is ${\rm{\Sigma }}_1^{0,{\emptyset ^{\left( k \right)}}}$ if and only if there is a ${\rm{\Sigma }}_{k + 1}^0$ set $S \subseteq {\{ 0,1\} ^{ \lt \mathbb{N}}}$ —namely, a set of strings $S$ that is computably enumerable relative to the $k$ th jump ${\emptyset ^{\left( k \right)}}$ of $\emptyset $ —such that ${\cal S} = \left[ S \right]$ (see Soare Reference Soare2016, chaps. 3 and 4).

The notion of a ${\emptyset ^{\left( k \right)}}$ -computable function is defined as follows:

The indicator functions of ${\rm{\Delta }}_1^{0,{\emptyset ^{\left( k \right)}}}$ sets are simple instances of ${\emptyset ^{\left( k \right)}}$ -computable functions. For $k \ge 1$ , functions of this type can be thought of as binary decision problems that are not in themselves decidable but become decidable having access to the background information encapsulated by ${\emptyset ^{\left( k \right)}}$ (or any other problem of the same complexity). More generally, a ${\emptyset ^{\left( k \right)}}$ -computable function is one whose values can be computably approximated to any degree of precision with background information ${\emptyset ^{\left( k \right)}}$ .

Crucially, for all $k \in \mathbb{N}$ , the ${\rm{\Sigma }}_1^{0,{\emptyset ^{\left( k \right)}}}$ subsets of ${\{ 0,1\} ^\mathbb{N}}$ are open. Hence, just like the computable functions, the ${\emptyset ^{\left( k \right)}}$ -computable functions are continuous. By Proposition 3.9, the set of data streams that satisfy Lévy’s Upward Theorem relative to a ${\emptyset ^{\left( k \right)}}$ -computable random variable is thus co-meager, as long as the agent’s prior has full support. And because, for any probability measure $\mu $ , every $\mu $ -Kurtz random sequence is in the support of $\mu $ , observing a $\mu $ -Kurtz random data stream suffices to converge to the truth for any ${\emptyset ^{\left( k \right)}}$ -computable random variable. So, the collection of $\mu $ -Kurtz random sequences once again provides a crisp example of a set of data streams along which convergence to the truth is guaranteed to occur.

A ${\emptyset ^{\left( k \right)}}$ -effective Baire class $n$ function is instead defined as follows:

The indicator functions of ${\rm{\Delta }}_n^{0,{\emptyset ^{\left( k \right)}}}$ , ${\rm{\Sigma }}_n^{0,{\emptyset ^{\left( k \right)}}}$ , ${\rm{\Pi }}_n^{0,{\emptyset ^{\left( k \right)}}}$ , and ${\rm{\Delta }}_{n + 1}^{0,{\emptyset ^{\left( k \right)}}}$ sets are all of ${\emptyset ^{\left( k \right)}}$ -effective Baire class $n$ . For instance, the indicator functions of ${\rm{\Sigma }}_1^{0,{\emptyset ^{\left( k \right)}}}$ sets, which correspond to binary decision problems that can be effectively verified having access to ${\emptyset ^{\left( k \right)}}$ , are all of ${\emptyset ^{\left( k \right)}}$ -effective Baire class $1$ , whereas the indicator functions of ${\rm{\Sigma }}_2^{0,{\emptyset ^{\left( k \right)}}}$ sets, which correspond to binary decision problems that can be effectively verified in the limit having access to ${\emptyset ^{\left( k \right)}}$ , are all of ${\emptyset ^{\left( k \right)}}$ -effective Baire class $2$ .

Once again, because every ${\emptyset ^{\left( k \right)}}$ -effective Baire class $1$ function $f$ is a Baire class $1$ function, Theorem 3.12 and Corollary 3.13 both apply: the points of continuity of $f$ form a co-meager ${\bf{\Pi }}_2^0$ set, and, for any prior with full support, the collection of data streams along which Lévy’s Upward Theorem holds with respect to $f$ (when $f$ is an integrable random variable) is co-meager. As with effective Baire class $1$ functions, we can, however, say more.

First, note that the notion of $1$ -genericity introduced earlier can be generalized as follows to any positive natural number:

For every $n \ge 1$ , $\left( {n + 1} \right)$ -genericity entails $n$ -genericity, but the reverse implication does not hold. Moreover, the fact that the set of $1$ -generic sequences is topologically typical generalizes to the entire hierarchy (see Kurtz Reference Kurtz1981; Nies Reference Nies2009; Downey and Hirschfeldt Reference Downey and Hirschfeldt2010):

The following is proven analogously to Theorem 3.24:

Proof. Let ${{\cal S}_0},{{\cal S}_1},{{\cal S}_2},...$ be a fixed effective enumeration of the ${\rm{\Sigma }}_1^0$ subsets of $\mathbb{R}$ . Recall that $f$ being discontinuous at $\omega \in {\{ 0,1\} ^\mathbb{N}}$ means that there is an open subset ${\cal O}$ of $\mathbb{R}$ with $f\left( \omega \right) \in {\cal O}$ , but, for all open ${\cal U} \subseteq {f^{ - 1}}\left( {\cal O} \right)$ , $\omega\ \notin \,\,{\cal U}$ . In other words, $\omega \in {f^{ - 1}}\left( {\cal O} \right)$ , but $\omega\ \notin \,\,{\rm{Int}}\left( {{f^{ - 1}}\left( {\cal O} \right)} \right)$ . Every open set is a union of ${\rm{\Sigma }}_1^0$ sets, so ${\{\omega\in\{0,1\}^\mathbb{N}:\, f\ {\rm is\ discontinuous\ at\ }\omega\}=\bigcup_{n\in\mathbb{N}}\big(\,{f^{-1}(\mathcal{S}_n)\setminus \text{Int}(\,{{\it{f}}^{-1}(\mathcal{S}_{\it n}))}}\big)}$ . Let $\omega \in {\{ 0,1\} ^\mathbb{N}}$ be such that $f$ is discontinuous at $\omega $ . Then, there is $m \in \mathbb{N}$ with $\omega \in {f^{ - 1}}\left( {{{\cal S}_m}} \right)\backslash {\rm{Int}}\left( {{f^{ - 1}}\left( {{{\cal S}_m}} \right)} \right)$ . Given that $f$ is of ${\emptyset ^{\left( {k - 1} \right)}}$ -effective Baire class $1$ , by definition, ${f^{ - 1}}\left( {{{\cal S}_m}} \right)$ is a ${\rm{\Sigma }}_2^{0,{\emptyset ^{\left( {k - 1} \right)}}}$ subset of ${\{ 0,1\} ^\mathbb{N}}$ . Hence $f^{-1}(\mathcal{S}_m)=\bigcup_{i\in\mathbb{N}}\mathcal{C}_i$ , where each ${{\cal C}_i}$ is a ${\rm{\Pi }}_1^{0,{\emptyset ^{\left( {k - 1} \right)}}}$ set. So, ${f^{ - 1}}({{\cal S}_m}) \setminus {\rm{Int}}({\mkern 1mu} {f^{ - 1}}({{\cal S}_m})) = { \cup _{i \in }}{{\cal C}_i} \setminus {\rm{Int}}({ \cup _{i \in }}{{\cal C}_i}) \subseteq { \cup _{i \in }}({{\cal C}_i}\backslash {\rm{Int(}}{{\cal C}_i}{\rm{))}}$ . Thus there is $j \in \mathbb{N}$ with $\omega \in {{\cal C}_j}\backslash {\rm{Int}}\left( {{{\cal C}_j}} \right)$ , which means that $\omega \in {{\cal C}_j} \cap {\rm{Cl}}\left( {\overline {{{\cal C}_{j}}} } \right)$ . Because $\overline {{{\cal C}_j}} $ is a ${\rm{\Sigma }}_1^{0,{\emptyset ^{\left( {k - 1} \right)}}}$ set, $\overline {{{\cal C}_j}} = \left[ C \right]$ for some ${\rm{\Sigma }}_k^0$ set $C \subseteq {\{ 0,1\} ^{ \lt \mathbb{N}}}$ . And because $\omega \in {\rm{Cl}}\left( {\left[ C \right]} \right)$ , $C$ is dense along $\omega $ . However, $\omega $ does not meet $C$ , as $\omega \,\notin \,\left[ C \right] = \overline {{{\cal C}_j}} $ . Therefore $\omega $ is not $k$ -generic.

Theorem 3.30 entails that, for any Bayesian reasoner whose prior has full support, observing a $k$ -generic data stream leads to inductive success for any inductive problem corresponding to a ${\emptyset ^{\left( {{k} - 1} \right)}}$ -effective Baire class $1$ integrable random variable:

As noted earlier, the set of $k$ -generic sequences is co-meager. Hence Corollary 3.31 reveals that, for priors with full support, we can once again single out a specific co-meager collection of data streams along which convergence to the truth happens for all ${\emptyset ^{\left( {k - 1} \right)}}$ -effective Baire class $1$ integrable random variables.