2.1 Sofic groups

A sofic approximation to a countable group $\Gamma $ is a sequence $\Sigma =\{\sigma _n\}_{n\in \mathbb {N}}$ of maps

$$ \begin{align*}\sigma_n: \Gamma \to \operatorname{\mathrm{Sym}}(V_n),\end{align*} $$

where $V_n$ are finite sets, $\operatorname {\mathrm {Sym}}(V_n)$ is the symmetric group on $V_n$ , and the sequence is required to be asymptotically homomorphic and free in the following sense: For every $g,h \in \Gamma $ , we require that the homomorphism equation $\sigma _n(gh)v=\sigma _n(g)\sigma _n(h)v$ holds for asymptotically all $v\in V_n$ :

$$ \begin{align*}1 = \lim_{n\to\infty} |V_n|^{-1} |\{ v\in V_n:~ \sigma_n(gh)v = \sigma_n(g)\sigma_n(h)v\}|.\end{align*} $$

For every non-identity element $g \in \Gamma \setminus \{1_\Gamma \}$ , we require that the percentage of points fixed by $\sigma _n(g)$ tends to zero:

$$ \begin{align*} 0= \lim_{n\to\infty} |V_n|^{-1} |\{ v\in V_n:~ \sigma_n(g)v = v\}|. \end{align*} $$

A group $\Gamma $ is sofic if it admits a sofic approximation.

If $\Gamma $ admits a finite generating set S, then it is common to visualize a map $\sigma _n$ as above in terms of the labeled and directed graph $G(\sigma _n)=(V_n,E_n)$ it induces. The edges of this graph are pairs of the form $(v, \sigma _n(s)v)$ for $s\in S$ and $v\in V_n$ , and the label of this pair is s. Then $\Sigma $ is a sofic approximation precisely when this sequence of graphs $G(\sigma _n)$ Benjamini-Schramm converges to the (labeled and directed) Cayley graph of $\Gamma $ with respect to S [Reference Elek and Szabó27].

It is an exercise to check that all amenable groups and all residually finite groups are sofic. Because finitely generated linear groups are sofic and direct unions of sofic groups are also sofic, it follows that all linear groups are sofic. It is an open problem whether all countable groups are sofic.

The class of sofic groups was introduced implicitly by Gromov [Reference Gromov35] and explicitly by Weiss [Reference Weiss73]. For further background on sofic groups, see [Reference Pestov59, Reference Capraro and Lupini21].

2.2 Sofic entropy

This section defines sofic entropy for subshifts using the formulation from [Reference Bowen14]. See also [Reference Bowen16] or [Reference Kerr and Li40] for more comprehensive references.

Let $\Gamma $ denote a countable group and let $\mathtt {A}$ be a finite set (called the alphabet). Let $\mathtt {A}^\Gamma $ be the set of all functions $x:\Gamma \to \mathtt {A}$ . We write either $x_g$ or $x(g)$ for the value of x on $g\in \Gamma $ , whichever is most convenient. We endow $\mathtt {A}^\Gamma $ with the pointwise convergence topology, under which it is compact and metrizable.

Let $\operatorname {\mathrm {Prob}}(\mathtt {A}^\Gamma )$ denote the space of all Borel probability measures on $\mathtt {A}^\Gamma $ , which we endow with the weak* topology. In this topology, a sequence of Borel probability measures $(\mu _n)_{n \in \mathbb {N}}$ converges to a measure $\mu _\infty $ if and only if for every continuous function $f:\mathtt {A}^\Gamma \to \mathbb {R}$ , $\lim _{n\to \infty } \int f~d \mu _n= \int f~d \mu _\infty $ . An equivalent characterization uses cylinder sets which are defined as follows. Given a finite subset $F \subset \Gamma $ and $x:F \to \mathtt {A}$ , let $C(x,F)$ be the set of all functions $y:\Gamma \to \mathtt {A}$ such that $y(f)=x(f)$ for all $f\in F$ . Then $(\mu _n)_{n \in \mathbb {N}}$ converges to $\mu _\infty $ if and only if for every such F and x, $\lim _{n\to \infty } \mu _n(C(x,F)) = \mu _\infty (C(x,F))$ . This is because the cylinder sets $C(x,F)$ form a sub-basis for the topology on $\mathtt {A}^\Gamma $ .

Let $T = (T^g)_{g\in \Gamma }$ be the shift action on $\mathtt {A}^\Gamma $ defined by $T^gx(f)= x(g^{-1}f)$ for $x \in \mathtt {A}^\Gamma $ . This induces an action on $\operatorname {\mathrm {Prob}}(\mathtt {A}^\Gamma )$ by pushforwards. The set of all shift-invariant Borel probability measures on $\mathtt {A}^\Gamma $ will be denoted $\operatorname {\mathrm {Prob}}^\Gamma (\mathtt {A}^\Gamma )$ . If $\mu $ is a shift-invariant Borel probability measure on $\mathtt {A}^\Gamma $ , then the system $(\mathtt {A}^\Gamma ,\mu ,T)$ is called a shift $\Gamma $ -system. We define here the sofic entropy of such systems.

Given $\sigma :\Gamma \to \textrm {Sym}(V)$ , $v \in V$ and $x:V \to \mathtt {A}$ , the pullback name of x at v is the labeling $\Pi ^\sigma _v(x) \in \mathtt {A}^\Gamma $ defined by

$$\begin{align*}\Pi^\sigma_v(x)(g) = x_{\sigma(g^{-1})v} \quad \forall g \in \Gamma. \end{align*}$$

For the sake of building some intuition, note that when $\sigma $ is a homomorphism, the map $v \mapsto \Pi ^\sigma _{v}(x)$ is $\Gamma $ -equivariant (in the sense that $\Pi ^\sigma _{\sigma (g)v}(x) = T^g \Pi ^\sigma _{v}(x)$ ). In particular, ${\Pi ^\sigma _{v}(x) \in \mathtt {A}^\Gamma} $ is periodic. In general, we think of $\Pi ^\sigma _v(x)$ as an approximate periodic point.

The empirical measure of $x:V\to \mathtt {A}$ is

$$ \begin{align*}P^\sigma_x = |V|^{-1} \sum_{v\in V} \delta_{\Pi^\sigma_v(x)} \in \operatorname{\mathrm{Prob}}(\mathtt{A}^\Gamma),\end{align*} $$

where, for $y \in \mathtt {A}^\Gamma $ , $\delta _y \in \operatorname {\mathrm {Prob}}(\mathtt {A}^\Gamma )$ is the Dirac measure concentrated on $\{y\}$ . For example, if $\sigma $ is a homomorphism, then $P^\sigma _x$ is a $\Gamma $ -invariant measure supported on the $\Gamma $ -orbits of the pullback names $\Pi ^\sigma _v(x)$ .

Given an open set $\mathcal {O} \subset \operatorname {\mathrm {Prob}}(\mathtt {A}^\Gamma )$ , a map $x:V \to \mathtt {A}$ is called an $(\mathcal {O},\sigma )$ -microstate if $P^\sigma _x \in \mathcal {O}$ . Typically, we take $\mathcal {O}$ to be a small neighborhood of $\mu $ , in which case we consider $(\mathcal {O},\sigma )$ -microstates to be ‘good microstates for $\mu $ ’. Let $\Omega (\sigma ,\mathcal {O}) \subset \mathtt {A}^V$ denote the set of all $(\mathcal {O},\sigma )$ -microstates.

Let $\mu \in \operatorname {\mathrm {Prob}}(\mathtt {A}^\Gamma )$ be $\Gamma $ -invariant and let $\Sigma =(\sigma _i:\Gamma \to \textrm {Sym}(V_i))_{i \in \mathbb {N}}$ be a sofic approximation to $\Gamma $ . We say that the system $(\mathtt {A}^\Gamma ,\mu ,T)$ has microstates along $\Sigma $ if for every neighbourhood $\mathcal {O}$ of $\mu $ ,

$$\begin{align*}\Omega(\sigma_n,\mathcal{O}) \ne \emptyset \quad \mbox{for all sufficiently large }n.\end{align*}$$

More generally, an arbitrary measure-preserving $\Gamma $ -system has microstates along $\Sigma $ if every shift-system factor of it has microstates along $\Sigma $ .

The $\Sigma $ -entropy of the action $(\mathtt {A}^\Gamma , \mu , T)$ is defined by

(2) $$ \begin{align} h_\Sigma(\mathtt{A}^\Gamma, \mu, T):= \inf_{\mathcal{O} \ni \mu} \limsup_{i\to\infty} |V_i|^{-1} \log {\lvert {\Omega(\sigma_i,\mathcal{O})} \rvert}, \end{align} $$

where the infimum is over all open neighborhoods of $\mu $ in $\operatorname {\mathrm {Prob}}(\mathtt {A}^\Gamma )$ . We abbreviate this to $h_\Sigma (\mu )$ if the other data are clear from the context. This number depends on the action $(\mathtt {A}^\Gamma , \mu , T)$ only up to measure conjugacy [Reference Bowen11]. It therefore defines an invariant for any abstract measure-preserving system that can be represented up to measure conjugacy by a shift system with a finite alphabet or, equivalently, that has a finite generating partition. If the system does not have microstates along any subsequence of $\Sigma $ , then we declare that the $\Sigma $ -entropy is $-\infty $ .

2.3 Factor maps and inherited entropy

The second part of Theorem A concerns the entropy not only of $(X,m_X,T)$ but also of all its factors. Since X is a shift-invariant subset of $\mathbb {Z}_2^\Gamma $ , the $\Sigma $ -entropy of $(X,m_X,T)$ is an instance of formula (2). But this system may have factors that are not measure conjugate to shift systems, so that formula (2) does not apply.

Sofic entropy can be generalized to measure-preserving systems on standard measurable spaces in various ways: see, for instance, [Reference Bowen16, Subsection 2.4] and [Reference Austin3, Subsection 3.1] for formulations and proofs of their equivalence. However, rather than repeat these in detail here, we need only recall how they are controlled by another entropy notion that does permit us to reduce our work to the study of shift systems.

Towards defining this, consider a factor map between two shift systems on finite alphabets, say

(3) $$ \begin{align} \Phi:(\mathtt{A}^\Gamma,\mu)\to (\mathtt{B}^\Gamma,\nu), \end{align} $$

where $\mu $ and $\nu $ are both invariant under the shift actions of $\Gamma $ on their respective spaces. Rather than counting good microstates for $\mu $ or $\nu $ separately, we can ask how many of the good microstates for $\nu $ can be lifted to good microstates for $\mu $ – that is, how many good microstates $\nu $ ‘inherits’ through the map $\Phi $ . To make this precise, consider the graphical joining

(4) $$ \begin{align} \lambda := \int_{\mathtt{A}^\Gamma} \delta_{(x,\Phi(x))}\ d\mu(x). \end{align} $$

This is invariant under the shift action of $\Gamma $ on $(\mathtt {A}\times \mathtt {B})^\Gamma $ . Let $\mathrm {proj}_i$ be the coordinate projection from $(\mathtt {A}\times \mathtt {B})^{V_i}$ to $\mathtt {B}^{V_i}$ . Finally, define the inherited $\Sigma $ -entropy of $\Phi $ to be

(5) $$ \begin{align} h_\Sigma(\mu,T\,;\,\Phi) := \inf_{\mathcal{O}\ni \lambda}\limsup_{i\to\infty}|V_i|^{-1}\log|\mathrm{proj}_i[\Omega(\sigma_i,\mathcal{O})]|. \end{align} $$

If $\Phi $ is the identity, then this is easily checked to coincide with $h_\Sigma (\mu )$ .

Like sofic entropy itself, inherited sofic entropy can be generalized to factor maps between arbitrary measure-preserving systems. This general notion is not new, but our use of the term ‘inherited’ is new. Indeed, the idea behind this quantity is already implicit in Kerr’s original approach to defining sofic entropy itself for general measure-preserving systems (see [Reference Kerr39] or [Reference Bowen16, Subsubsection 2.4.2]). It was formulated and studied explicitly by Hayes in [Reference Hayes38, Reference Hayes37], who refers to it as ‘entropy in the presence’ in recognition of a parallel usage in Voiculescu’s theory of free entropy [Reference Voiculescu70]. It has also been studied by other authors under various names, including ‘outer sofic entropy’ and ‘extension sofic entropy’. It is reviewed in [Reference Bowen16, Subsection 11.1], which gives more complete references. Despite this history, we do propose the new name ‘inherited’ entropy because this seems to capture the idea behind the definition better.

Starting from (5), one can define entropy for a factor map between general systems by carefully inserting a supremum over generating partitions of the lower system and an infimum over generating partitions of the upper system. However, the proof that sofic entropy itself is invariant under measure conjugacy can be adapted directly to the quantity in (5), showing that it is an invariant of $\Phi $ , where we consider two factor maps to be equivalent if they appear downwards in a commuting square whose horizontal arrows are conjugacies. As a corollary, (5) coincides with the abstract inherited entropy in the case of two shift systems. A more general version of this argument can be found in [Reference Hayes38, Proposition 2.9].

One crucial advantage to working with inherited sofic entropy is its monotonicity under factor maps. The following lemma is a special case of parts (iii) and (iv) of [Reference Hayes38, Proposition 2.10].

Lemma 2.1. If

$$\begin{align*}(X,\mu,T) \stackrel{\Pi}{\to} (Y,\nu,S) \stackrel{\Phi}{\to} (Z,\theta,R)\end{align*}$$

are factor maps, then

$$\begin{align*}h_\Sigma(\mu,T\,;\,\Phi\circ\Pi) \le \min\big\{h_\Sigma(\nu,S\,;\,\Phi),\ h_\Sigma(\mu,T\,;\,\Pi)\big\}.\end{align*}$$

In particular, taking $\Pi $ (resp. $\Phi $ ) to be the identity, we obtain

$$\begin{align*}h_\Sigma(\mu,T\,;\,\Phi) \le h_\Sigma(\mu,T) \quad (\mbox{resp.}\quad h_\Sigma(\mu,T\,;\,\Pi) \le h_\Sigma(\mu,S)).\end{align*}$$

Inspired by this property, one defines the outer $\Sigma $ -Pinsker factor of a measure-preserving system to be the largest factor whose inherited $\Sigma $ -entropy is zero. A routine argument shows that a unique maximal such factor exists. Hayes’ paper [Reference Hayes38] develops this story as well, although it had been considered in unpublished work previously. As a result, the assertion that every nontrivial factor of a measure-preserving system has positive inherited $\Sigma $ -entropy is equivalent to the assertion that the outer $\Sigma $ -Pinsker factor of that system is trivial. In addition, by the last inequality in Lemma 2.1, if a system has this property, then it also has completely positive $\Sigma $ -entropy. The observations together explain our formulation of the second part of Theorem A.

Starting from Lemma 2.1, Hayes develops enough properties of the outer Pinsker factor to turn it into a valuable tool in the study of sofic entropy in general. For instance, these properties are crucial to his proof that a certain large class of systems of algebraic origin all have completely positive sofic entropy [Reference Hayes37]. Our reasons for using inherited entropy in the proof of Theorem A are similar, but the details of our proof are essentially disjoint from those of Hayes, and our LDPC system is not among the systems of algebraic origin that he considers in that reference.

Among its useful consequences, Lemma 2.1 gives the following:

Proof. Let $\Pi $ be a factor map to another nontrivial system $(Y,\nu ,S)$ . Then $(Y,\nu )$ has a nontrivial partition into two measurable subsets. By acting on this partition using S, we define a further factor map of the form

$$\begin{align*}\Phi:(Y,\nu,S)\to (\{0,1\}^\Gamma,\theta,\textrm{shift}),\end{align*}$$

where $\theta $ is not a Dirac measure. Now our hypothesis gives that $h_\Sigma (\mu ,T\,;\,\Phi \circ \Pi )> 0$ , and this implies that $h_\Sigma (\mu ,T\,;\,\Pi )> 0$ by Lemma 2.1.

Corollary 2.2 can simplify many of the technicalities involved in a proof of completely positive inherited entropy by letting us restrict our attention to factor maps between shift spaces. Given such a map, say

$$\begin{align*}\Phi:(\mathtt{A}^\Gamma,\mu)\to (\mathtt{B}^\Gamma,\nu),\end{align*}$$

it is uniquely determined by its coordinate at the identity of $\Gamma $ , which is an arbitrary measurable map $\phi :\mathtt {A}^\Gamma \to \mathtt {B}$ . Starting from $\phi $ , we write $\phi ^\Gamma $ for the factor map that it induces, which is given by

(6) $$ \begin{align} \phi^\Gamma(x)(\gamma) = \phi(\gamma^{-1} \cdot x). \end{align} $$

When working with such a map $\Phi $ , a further simplification is often necessary. If D is a finite subset of $\Gamma $ , then the map $\phi $ above is called D -local if the image $\phi (x)$ depends only on the coordinates of x indexed by D – equivalently, if $\phi $ factorizes into maps

$$\begin{align*}\mathtt{A}^\Gamma \to \mathtt{A}^D\to \mathtt{B},\end{align*}$$

where the first map is the coordinate projection. We say that $\phi $ is local if it is D-local for some finite set D,and apply the same terminology to the whole of $\phi ^\Gamma $ . This is equivalent to $\phi ^\Gamma $ being a continuous map for the product topologies on our shift spaces.

If $\mathbf {x} \in \mathtt {A}^{V}$ is some good microstate for $\mu $ over a map $\sigma \colon \Gamma \to \operatorname {\mathrm {Sym}}(V)$ , we might attempt to send it to a good microstate for $\nu $ using the map $\phi ^\sigma \colon \mathtt {A}^V \to \mathtt {B}^V$ defined by

(7) $$ \begin{align} \phi^\sigma(\mathbf{x})(v) = \phi( \Pi_v^\sigma \mathbf{x}) , \end{align} $$

since the empirical distribution of $\phi ^\sigma (\mathbf {x})$ would then be $\phi ^\Gamma _* P_{\mathbf {x}}^\sigma $ . Since $P_{\mathbf {x}}^\sigma $ is close to $\mu $ , one would hope this would be close to $\nu $ . This argument is correct in case $\phi ^\Gamma _*$ acts continuously on measures, which in turn holds if $\phi $ is local. In that case, $\phi ^\sigma $ also has the following form of quantitative continuity.

Equivalently, this asserts that $\phi ^\sigma $ is $|D|$ -Lipschitz for the ‘normalized Hamming metrics’ on $\mathtt {A}^V$ and $\mathtt {B}^V$ . This point of view is introduced and used for an application of Lemma 2.3 in Subsection 5.1.

The arguments about $\phi ^\sigma $ above can fail for general measurable factor maps, but this problem can be overcome by approximating these in measure by local factor maps. We say that a sequence of maps

$$\begin{align*}\psi_m:\mathtt{A}^\Gamma \to \mathtt{B} \quad (m=1,2,\dots)\end{align*}$$

is a local approximating sequence to $\phi $ if each $\psi _m$ is local and

(8) $$ \begin{align} \mu\{\psi_m \ne \phi\} \to 0. \end{align} $$

(Note that this notion implicitly also depends on the measure $\mu $ .) These are the special case for shift spaces of the ‘almost Lipschitz approximating sequences’ introduced and used in [Reference Austin3] to study factor maps between more general measure-preserving systems. At a few points in the sequel, we refer to [Reference Austin3] for properties that we need in our special case. For instance, informally, if $\psi $ is a good enough local approximation to $\phi $ , then $\psi ^\sigma $ sends very good microstates for $\mu $ to fairly good microstates for $\phi ^\Gamma _*\mu $ [Reference Austin3, Lemma 4.10].

3.1 Random sofic approximations

Instead of directly constructing sofic approximation sequences $\sigma _n: \Gamma \to \operatorname {\mathrm {Sym}}(V_n)$ , we will construct probability measures on the space of homomorphisms from $\Gamma $ to $\operatorname {\mathrm {Sym}}(kn)$ . These measures were used for the same purpose in [Reference Airey, Bowen and Lin2].

Let V be a finite set whose size is divisible by k. Let us say that a permutation of V is k -uniform if it consists entirely of k-cycles: that is, its cycle type is $[k^{n/k}]$ . Consider a d-tuple of k-uniform permutations $(\sigma (s_1),\dots ,\sigma (s_d))$ . Then $\sigma (s_i^k)$ equals the identity permutation for each i, and so the tuple $(\sigma (s_i),\dots ,\sigma (s_d))$ is the image of the generators $(s_1,\dots ,s_d)$ under a homomorphism $\sigma :\Gamma \to \operatorname {\mathrm {Sym}}(V)$ . We call such a homomorphism k -uniform if the images of these generators are all k-uniform permutations. Denote the set of k-uniform homomorphisms into $\operatorname {\mathrm {Sym}}(V)$ by $\operatorname {\mathrm {Hom}}_{\mathrm {unif}}(\Gamma , \operatorname {\mathrm {Sym}}(V))$ . Note that for arbitrary members of $\operatorname {\mathrm {Hom}}(\Gamma , \operatorname {\mathrm {Sym}}(V))$ , the cycle sizes of the images of the generators of $\Gamma $ must be factors of k.

Given a homomorphism $\sigma \in \operatorname {\mathrm {Hom}}_{\mathrm {unif}}(\Gamma , \operatorname {\mathrm {Sym}}(V))$ , consider the collection of all the orbits of the individual maps $\sigma _i = \sigma (s_i)$ – that is, all the subsets of the form

(9) $$ \begin{align} \{ \sigma(s_i^j)(v) \,:\, 0 \leq j < k \} \end{align} $$

for $v \in V$ and $i \in [d]$ . Taken together, these may be regarded as a hyper-graph on V. Because we consider a k-uniform homomorphism, this hyper-graph is k-uniform in the usual sense in combinatorics: this is the origin of the terminology. However, it could happen that two $\sigma _i$ and $\sigma _j$ give some vertex the same orbit. In this eventuality, it is better to think of the family of sets (9) as a multi-hyper-graph, in that we count each hyper-edge with this multiplicity.

For each n, we set $V_n := \{1,2,\dots ,n\}$ . If k divides n, then we let $\mathbb {P}_n$ be the uniform distribution on $\operatorname {\mathrm {Hom}}_{\mathrm {unif}}(\Gamma ,\operatorname {\mathrm {Sym}}(V_n))$ . The sofic approximations that appear in Theorem A are obtained at random from these distributions and are shown to have all the desired properties with high probability.

In order to take this approach, Proposition 1.2 above is a basic prerequisite. It allows us to focus on typical properties of the microstate spaces, knowing that random homomorphisms will be good sofic approximation maps with high probability. It is implied by the more precise estimate in [Reference Airey, Bowen and Lin2, Lemma 3.1], where our $\mathbb {P}_n$ is called ‘ $\mathbb {P}_n^u$ ’. Note that attention is restricted to even n in that reference, but that restriction is unnecessary for the case of $\mathbb {P}_n^u$ ; it is included there only for the sake of the other probability distribution $\mathbb {P}_n^p$ that is covered by the same lemma.

3.2 Formula for Kikuchi entropy

Let $\mathtt {A}$ be a finite set and $\Gamma = (\mathbb {Z}_k)^{\ast d}$ as above. Given a $\Gamma $ -invariant measure $\mu \in \operatorname {\mathrm {Prob}}(\mathtt {A}^\Gamma )$ , we define the edge weight $W_\mu ( \cdot; \cdot ) \colon \mathtt {A}^{\mathbb {Z}_k} \times [d] \to [0,1]$ by

$$\begin{align*}W_\mu (\mathbf{a}; i) = \mu \{ \mathbf{x} \in \mathtt{A}^\Gamma \,:\, \mathbf{x}(s_i^j) = \mathbf{a}(j) \ \forall 0 \leq j < k \}. \end{align*}$$

Each $W(\cdot; i)$ is a probability measure on $\mathtt {A}^{\mathbb {Z}_k}$ which records the statistics of $\mu $ on the hyper-edge $\{s_i, s_i^2, \ldots , s_i^{d-1}\}$ . Also define the vertex weight

$$\begin{align*}W_\mu(\mathtt{a}) = \mu \{ \mathbf{x} \in \mathtt{A}^\Gamma \,:\, \mathbf{x}(e) = \mathtt{a} \} \end{align*}$$

for $\mathtt {a} \in \mathtt {A}$ . This probability measure on $\mathtt {A}$ records the single-site statistics of $\mu $ . It is determined by the edge weight $W_\mu (\cdot; \cdot )$ .

More generally, an abstract weight W is a d-tuple of probability vectors $W(\cdot; 1), \ldots , W(\cdot; d)$ on $\mathtt {A}^{\mathbb {Z}_k}$ such that there is a probability vector $W(\cdot )$ on $\mathtt {A}$ satisfying the consistency condition

$$\begin{align*}W(\mathtt{a}_0) = \sum_{\mathbf{a} \in \mathtt{A}^{\mathbb{Z}_k} \,:\, \mathbf{a}(j) = \mathtt{a}_0 } W(\mathbf{a}; i)\end{align*}$$

for every $i \in [d]$ , $j \in \mathbb {Z}_k$ and $\mathtt {a}_0 \in \mathtt {A}$ .

For any weight W, we define the Kikuchi entropy of W by

$$\begin{align*}\mathrm{H_K}(W) := (1-d) \operatorname{\mathrm{H}}( W(\cdot) ) + \frac{1}{k} \sum_{i \in [d]} \operatorname{\mathrm{H}}( W(\cdot; i) ), \end{align*}$$

where $\operatorname {\mathrm {H}}(\cdot )$ denotes Shannon entropy, and for a $\Gamma $ -invariant measure $\mu $ on $\mathtt {A}^\Gamma $ , we abbreviate $\mathrm {H_K}(W_\mu )$ to $\mathrm {H_K}(\mu )$ . The functional $\mathrm {H_K}$ appears in our work because it gives the upper exponential growth rate of the expected number of microstates whose averaged hyper-edge marginals are approximately specified by $\mu $ . This is an analog of [Reference Bowen9, Theorem 1.4]. Given a microstate $\mathbf {x} \in \mathtt {A}^V$ and $\sigma \in \operatorname {\mathrm {Hom}}(\Gamma , \operatorname {\mathrm {Sym}}(V))$ , let $W_{\mathbf {x}, \sigma }$ be the weight corresponding to $P_{\mathbf {x}}^\sigma \in \operatorname {\mathrm {Prob}}(\mathtt {A}^\Gamma )$ , the empirical distribution of $\mathbf {x}$ over $\sigma $ . Also, given two weights $W,W'$ , let

$$ \begin{align*}{\| {W - W'}\|} := \max_{1\le i \le d} \max_{\mathbf{a} \in \mathtt{A}^{\mathbb{Z}_k}} |W( \mathbf{a};i) - W'(\mathbf{a} ;i)|.\end{align*} $$

Proposition 3.1. We have

$$\begin{align*}\mathrm{H_K}(\mu) = \lim_{\varepsilon \to 0} \limsup_{n \to \infty} \frac{1}{nk} \log \mathbb{E}_{\sigma_n \sim \mathbb{P}_{nk}} {\lvert {\{ \mathbf{x} \in \mathtt{A}^{nk} \,:\, {\| {W_{\mathbf{x}, \sigma_n} - W_\mu} \|} < \varepsilon \}}\rvert}. \end{align*}$$

The proof of this proposition shows that we also obtain $\mathrm {H_K}(\mu )$ if $\limsup $ is replaced with $\liminf $ on the right-hand side.

Proof. Let $\mathbb {Z}_k$ act on $\mathtt {A}^{\mathbb {Z}_k}$ in the usual way: $\pi \mathbf {a}(j) = \mathbf {a}(j-\pi )$ for $\pi \in \mathbb {Z}_k$ , $\mathbf {a} \in \mathtt {A}^{\mathbb {Z}_k}$ and $j \in \mathbb {Z}_k$ . We will say that two elements $\mathbf {a}, \mathbf {b} \in \mathtt {A}^{\mathbb {Z}_k}$ are equivalent if they are in the same $\mathbb {Z}_k$ -orbit. Let $[\mathbf {a}] \subset \mathtt {A}^{\mathbb {Z}_k}$ denote the equivalence class of $\mathbf {a}$ . For a weight W, write

A weight W is cyclically invariant if $W(\mathbf {a}; i)=W(\mathbf {b}; i)$ for every equivalent pair $\mathbf {a},\mathbf {b}$ in $\mathtt {A}^{\mathbb {Z}_k}$ . For example, since $\mu $ is $\Gamma $ -invariant, for each $i \in [d]$ , the probability measure $W_\mu (\cdot; i) \in \operatorname {\mathrm {Prob}}(\mathtt {A}^{\mathbb {Z}_k})$ is cyclically invariant.

We first calculate $\mathbb {E} {\lvert {\{ \mathbf {x} \in \mathtt {A}^{kn} \,:\, W_{\mathbf {x}, \sigma _n} = W\}}\rvert }$ for an arbitrary cyclically invariant weight W with denominator $kn$ (i.e., such that $kn \cdot W([\mathbf {a}]; i) \in \mathbb {Z}$ for each $i, \mathbf {a}$ ). Letting

$$\begin{align*}N_n = {\lvert {\operatorname{\mathrm{Hom}}_{\mathrm{unif}}(\Gamma, \operatorname{\mathrm{Sym}}(kn))} \rvert} = \left( \frac{(kn)!}{n! k^n} \right)^d, \end{align*}$$

we have

$$ \begin{align*} \mathbb{E} {\lvert {\{ \mathbf{x} \in \mathtt{A}^{kn} \,:\, W_{\mathbf{x}, \sigma_n} = W\}} \rvert} &= \frac{1}{N_n} \sum_{\sigma \in \operatorname{\mathrm{Hom}}_{\mathrm{unif}}(\Gamma, \operatorname{\mathrm{Sym}}(kn))} {\lvert {\{ \mathbf{x} \in \mathtt{A}^{kn} \,:\, W_{\mathbf{x}, \sigma} = W \}}\rvert} \\ &= \frac{1}{N_n} \sum_{\mathbf{x} \in \mathtt{A}^{kn}} {\lvert {\{ \sigma \in \operatorname{\mathrm{Hom}}_{\mathrm{unif}}(\Gamma, \operatorname{\mathrm{Sym}}(kn)) \,:\, W_{\mathbf{x}, \sigma} = W\}} \rvert}. \end{align*} $$

Now a labeling $\mathbf {x} \in \mathtt {A}^{kn}$ admits at least one $\sigma $ that gives the correct weight W if and only if it has the correct ‘vertex statistics’; that is, $\frac {1}{kn}{\lvert {\{ i \in [kn] \,:\, \mathbf {x}(i) = \mathtt {a} \}} \rvert } = W(\mathtt {a})$ for all $\mathtt {a} \in \mathtt {A}$ . There are $\exp \{ kn ( \operatorname {\mathrm {H}}(W(\cdot )) + o(1))\}$ such $\mathbf {x}$ , and each admits the same number of $\sigma $ . From now on, fix one such $\mathbf {x}$ .

For $i \in [d]$ , let $G_i$ be the set of k-uniform permutations $\pi \in \operatorname {\mathrm {Sym}}(kn)$ such that if $W(\cdot; \pi )$ is the probability vector on $\mathtt {A}^{\mathbb {Z}_k}$ given by

$$ \begin{align*}W(\mathbf{a}; \pi) = (kn)^{-1} |\{v \in [kn]:~ \mathbf{a}(t) = \mathbf{x}(\pi^t(v))~\forall 0\le t < k\}|,\end{align*} $$

then $W(\cdot; \pi ) \equiv W(\cdot; i)$ . Any k-uniform homomorphism $\sigma $ with $W_{\mathbf {x}, \sigma } = W$ is determined by the permutations $\sigma (s_i)$ which must be in $G_i$ . So the number of such homomorphisms is $\prod _{i=1}^d |G_i|$ .

Let $G(\mathbf {x})$ be the set of permutations $g\in \operatorname {\mathrm {Sym}}(kn)$ which fix $\mathbf {x}$ in the sense that $\mathbf {x}(v)=\mathbf {x}(gv)$ for all $v\in [kn]$ . Observe that $G(\mathbf {x})$ acts on $G_i$ by conjugation. This means that if $\pi _i$ is a fixed permutation in $G_i$ and $g\in G(\mathbf {x})$ , then $g \pi _i g^{-1} \in G_i$ . Moreover, this action is transitive. So

$$ \begin{align*}|G_i| = \frac{ |G(\mathbf{x})|}{|\textrm{Stab}(\pi_i)|}\end{align*} $$

where $\textrm {Stab}(\pi _i)$ is the set of $g \in G(\mathbf {x})$ with $g \pi _i g^{-1} = \pi _i$ . Observe

$$ \begin{align*}|G(\mathbf{x})| = \prod_{\mathtt{a} \in \mathtt{A}} \left( kn \cdot W(\mathtt{a}) \right) !.\end{align*} $$

There are two mechanisms by which a $g \in G(\mathbf {x})$ can stabilize $\pi _i$ . Either g can permute k-cycles with the same labels or it can rotate a given labeled k-cycle. Therefore,

$$\begin{align*}|\textrm{Stab}(\pi_i)| = \prod_{[\mathbf{a}] \in \mathtt{A}^{\mathbb{Z}_k}/\mathbb{Z}_k} \left( n \cdot W([\mathbf{a}]; i) \right)! \left( \frac{k}{{\lvert {[\mathbf{a}]} \rvert} } \right)^{n \cdot W([\mathbf{a}]; i)}. \end{align*}$$

Putting everything together, we get

$$\begin{align*}\mathbb{E} {\lvert {\{ \mathbf{x} \in \mathtt{A}^{kn} \,:\, W_{\mathbf{x}, \sigma_n} = W\}} \rvert} = \frac{ e^{kn (\operatorname{\mathrm{H}}(W(\cdot)) + o(1))} \left( n! k^n \prod_{\mathtt{a} \in \mathtt{A}} \left( kn \cdot W(\mathtt{a}) \right) ! \right)^d}{(kn)!^d \prod_{i \in [d]} \prod_{[\mathbf{a}] \in \mathtt{A}^{\mathbb{Z}_k}/\mathbb{Z}_k} \left( n \cdot W([\mathbf{a}]; i) \right)! \left( \frac{k}{{\lvert {[\mathbf{a}]} \rvert}} \right)^{n \cdot W([\mathbf{a}]; i)} }. \end{align*}$$

Applying Stirling’s approximation $\log n! = n \log n - n + o(n)$ , the logarithm of this is

$$\begin{align*}kn \left( \operatorname{\mathrm{H}}(W(\cdot)) + d\sum_{\mathtt{a} \in \mathtt{A}} W(\mathtt{a}) \log W(\mathtt{a}) - \frac{1}{k} \sum_{i \in [d]} \sum_{[\mathbf{a}]} W([\mathbf{a}]; i) \log \frac{W([\mathbf{a}]; i)}{{\lvert {[\mathbf{a}]} \rvert} } \right) + o(n). \end{align*}$$

By cyclic invariance of each $W(\cdot; i)$ , we have $W(\mathbf {a}; i) = \frac {W([\mathbf {a}]; i)}{{\lvert {[\mathbf {a}]} \rvert }}$ , so this gives

$$\begin{align*}\mathbb{E} {\lvert {\{ \mathbf{x} \in \mathtt{A}^{kn} \,:\, W_{\mathbf{x}, \sigma_n} = W\}}\rvert} = e^{nk F(W) + o(n)} ,\end{align*}$$

where

Now, since the number of denominator-n weights W grows polynomially in n, it follows that for any $\varepsilon>0$ ,

$$\begin{align*}\lim_{n \to \infty} \frac{1}{kn} \log \mathbb{E} {\lvert {\{ \mathbf{x} \in \mathtt{A}^{kn} \,:\, {\| {W_{\mathbf{x}, \sigma_n} - W_\mu}\|} < \varepsilon\}}\rvert} = \sup \{ F(W) \,:\, {\| {W - W_\mu} \|} < \varepsilon\}, \end{align*}$$

and taking $\varepsilon $ to 0 gives the claimed formula, by continuity of F.

The annealed entropy of $\mu $ is defined by

$$\begin{align*}\mathrm{h_{ann}}(\mu) = \inf_{\mathcal{O} \ni \mu} \limsup_{n \to \infty} \frac{1}{nk} \log \mathbb{E}_{\sigma_n \sim \mathbb{P}_{nk}} {\lvert {\Omega(\sigma_n, \mathcal{O})} \rvert}, \end{align*}$$

where the infimum is over all open neighborhoods of $\mu $ .

To emphasize the relationship between $\mathrm {H_K}(\mu )$ and $\mathrm {h_{ann}}(\mu )$ , let

$$ \begin{align*}\mathcal{O}_\epsilon(\mu) = \{\nu \in \operatorname{\mathrm{Prob}}(\mathtt{A}^\Gamma):~ {\| {W_\nu - W_\mu} \|} <\epsilon\}.\end{align*} $$

Then $\mathcal {O}_\epsilon (\mu )$ is an open neighborhood of $\mu $ and Proposition 3.1 becomes

$$\begin{align*}\mathrm{H_K}(\mu) = \inf_{\epsilon>0} \limsup_{n \to \infty} \frac{1}{nk} \log \mathbb{E}_{\sigma_n \sim \mathbb{P}_{nk}} {\lvert {\Omega(\sigma_n, \mathcal{O}_\epsilon(\mu))} \rvert}. \end{align*}$$

In particular, $\mathrm {h_{ann}} (\mu ) \leq \mathrm {H_K}(\mu )$ .

The next proposition shows that $\mathrm {H_K}(\mu )$ is an upper bound for sofic entropy with respect to ‘most’ sofic approximations.

Proposition 3.2. Let $\mu $ be a $\Gamma $ -invariant Borel probability measure on $\mathtt {A}^\Gamma $ . Then there are subsets $\Omega _n' \subseteq \Omega _n^{\mathrm {sofic}}$ with

$$ \begin{align*}\lim_{n\to\infty} \mathbb{P}_n(\Omega_n')= 1\end{align*} $$

and such that if $\Sigma =\{\sigma _n\}_{n=1}^\infty $ satisfies $\sigma _n \in \Omega ^{\prime }_{i_n}$ for some increasing sequence $(i_n)_n$ with $k \mid i_n$ for all n, then

$$ \begin{align*}h_\Sigma(\mathtt{A}^\Gamma,\mu,T)\le \mathrm{H_K}(\mu).\end{align*} $$

Proof. Given a positive integer n divisible by k, $\varepsilon>0$ , and $\sigma :\Gamma \to \operatorname {\mathrm {Sym}}(n)$ , let $N_{n,\varepsilon }(\sigma )$ be ${\lvert {\Omega (\sigma , \mathcal {O}_\epsilon (\mu ))} \rvert }$ . Think of $N_{n,\varepsilon }$ as a random variable with respect to the uniform measure on the space of k-uniform homomorphisms from $\Gamma $ to $\operatorname {\mathrm {Sym}}(n)$ . In addition, let

$$ \begin{align*} \mathrm{H}_{\mathrm{K},\varepsilon}(\mu) := \limsup_{n \to \infty} \frac{1}{n} \log \mathbb{E}[N_{n,\varepsilon}], \end{align*} $$

where here and below we always restrict n to be a multiple of k.

For each $\varepsilon $ and n as above, let $\Omega _{n,\epsilon }'$ be the set of those $\sigma \in \Omega _n^{\mathrm {sofic}}$ that satisfy $N_{n,\epsilon }(\sigma ) \le e^{\sqrt {n}}\mathbb {E}[N_{n,\varepsilon }]$ , and let $\Omega _n' := \Omega _{n,1}'\cap \cdots \cap \Omega _{n,1/n}'$ . Then Markov’s inequality gives

$$ \begin{align*}\mathbb{P}_n(\Omega_n') \ge 1 - \sum_{m=1}^n\mathbb{P}_n\big(N_{n,1/m}(\sigma)> e^{\sqrt{n}}\mathbb{E}[N_{n,1/m}]\big) \ge 1 - ne^{-\sqrt{n}} \to 1.\end{align*} $$

Finally, suppose that $\Sigma =\{\sigma _n\}_{n=1}^\infty $ satisfies $\sigma _n \in \Omega ^{\prime }_{i_n}$ for some increasing sequence $(i_n)_n$ with $k \mid i_n$ for all n. Then we also have $\sigma _n \in \Omega ^{\prime }_{i_n,1/m}$ whenever $i_n\ge m$ . Fixing m and letting $n\to \infty $ , it follows that

$$ \begin{align*}h_\Sigma(\mathtt{A}^\Gamma,\mu,T) \le \limsup_{n \to \infty} \frac{1}{i_n} \log N_{i_n,1/m}(\sigma_n) \le \limsup_{n \to \infty} \frac{1}{i_n} \log \mathbb{E}[N_{i_n,1/m}] \le \mathrm{H}_{\mathrm{K},1/m}(\mu),\end{align*} $$

where the first inequality uses again the fact that $\mathcal {O}_{1/m}(\mu )$ is an open neighborhood of $\mu $ for every positive integer m. Now letting $m\to \infty $ , Proposition 3.1 gives $\mathrm {H_K}(\mu ) = \lim _{m\to \infty }\mathrm {H}_{\mathrm {K},1/m}(\mu )$ , so this completes the proof.

3.3 A brief history

The most basic method for analyzing the behaviour of a random sofic approximation is the first moment method. Our first indication of the typical number of microstates for $(X,m_X,T)$ over $\sigma _n$ chosen from $\mathbb {P}_n$ is given by the expectation of that number.

In the analogous setting of actions of free groups, such averages have been studied intensely in recent years. In [Reference Bowen9], the exponential growth rate of the expected number of good microstates was shown to coincide with an invariant of systems previously introduced by the second author in [Reference Bowen10], where it was used to solve the isomorphism problem for finite-state Bernoulli actions of free groups. In those and several subsequent papers, this invariant was called the ‘f-invariant’. Here, we propose a new term instead: we refer to this quantity as ‘annealed entropy’.

In work of the second author, the f-invariant was obtained as a limit of functionals referred to as F, which are annealed entropies of Markov approximations. As explained farther below, this quantity first appeared in refinements of work of Kikuchi [Reference Kikuchi42]. For this reason, we call it Kikuchi entropy. In later sections, it is used to prove Theorem 1.3.

The reason for the name ‘annealed entropy’ is a connection to statistics and statistical physics. During the last forty years, very similar first-moment calculations for various configurations over large sparse random graphs have become a central feature of the analysis of ‘graphical models’ in those disciplines. Often, the use of such averages can be seen as a first attempt to find the value for a ‘typical’ random graph. In such settings, the first moment is referred to as an annealed average: see, for instance, the usage in [Reference Mézard, Parisi and Virasoro50, Section IV.1] or [Reference Mézard and Montanari49, Section 5.4]. Its use as a prediction of typical behaviour is called the ‘Bethe ansatz’ (or sometimes the ‘replica symmetric’ approximation in reference to a phenomenology in the study of spin glasses that we do not explain here: see, for instance, [Reference Mézard, Parisi and Virasoro50, Chapter I] or [Reference Mézard and Montanari49, Chapter 8]Footnote ¹ ).

In fact, the origins of these quantities lie even farther back in the statistical physics literature. In the general setting for random graphical models studied in statistics, the leading order exponents in first moment calculations are given by quantities called ‘Bethe’ or ‘Kikuchi’ entropy.

The first of these terms refers to foundational work by Bethe [Reference Bethe8]. He estimated the free energy of a certain model of an alloy on a two-dimensional lattice by a recursive expansion that retained nearest-neighbour interactions but ignored the effect of loops in the lattice graph. A more mathematical description is that the two-dimensional lattice is approximated by an infinite regular tree, and this is why such trees are now often called ‘Bethe lattices’ in statistical physics.

In [Reference Kikuchi42], Kikuchi expanded on Bethe’s ideas by proposing a more careful expansion that respects slightly more of the lattice structure. In modern terms, this can be understood as an approximation to the lattice by a hyper-tree rather than a simple tree. While Bethe argued mostly in terms of free energies, Kikuchi’s paper includes various explicit formulae for entropy estimates, and these evolved over time into the quantities studied in statistical inference today. See [Reference Kikuchi42, Equations (A.7) and (C1.6)] for early intimations of these modern formulae. Bethe’s and Kikuchi’s approximation methods can also be found in physics surveys from closer to that time such as Section III in Burley’s contribution [Reference Domb and Green26, Chapter 9].

These formulae were brought explicitly into statistical theory around 2000 by Yedidia and various co-authors in a series of technical reports: see, in particular, [Reference Yedidia77, Reference Yedidia, Freeman and Weiss78] and the further references given there. While these references continued to emphasize free energy more than entropy, they do cover both: the explicit formula for Bethe entropy is [Reference Yedidia77, Formula (1.32)], for example.

The term ‘annealed entropy’ (instead of ‘f-invariant’) emphasizes the connection between these two fields. To be more precise, the annealed entropy of a measure-preserving action of a free group is defined as an infimum of the values of a more elementary quantity over Markov approximations to the action. This more elementary quantity, denoted by ‘F’ in [Reference Bowen10, Reference Bowen9], has precisely the same formula as Bethe entropy. Thus, the same formula for an entropy-like quantity was discovered independently and then used for very similar first-moment calculations in both fields.

Bethe entropy has by now become a textbook topic in statistical inference with graphical models: see, for instance, [Reference Wainwright and Jordan71, Section 4] or [Reference Mézard and Montanari49, Chapter 14, especially Subsection 14.2.4]. Some of the theory of these models has also been analyzed rigorously in the probability literature. For example, [Reference Dembo, Montanari, Sly and Sun24] proves that a first-moment quantity asymptotically agrees with the typical free energy for ferromagnetic Potts models over sparse graph sequences, justifying the ‘Bethe ansatz’ for these models.

Whereas Bethe entropy can be understood as a functional of a probability distribution over a tree, the extension to Kikuchi entropy allows an underlying graph that is a hyper-tree, which is a hyper-graph $G=(V,E)$ such that there is some tree with vertex set V whose subgraphs induced by hyper-edges of G are all connected.

For the first-moment calculations we need below, the exponent is given by the analogous annealed entropy for an action of $\Gamma $ , the d-fold free power of $\mathbb {Z}_k$ , rather than a free group. It turns out that this could again be defined as an infimum of a more elementary quantity over Markov approximations, where now the more elementary quantity is the Kikuchi entropy associated to the Cayley hyper-tree of $\Gamma $ . We do not work this out completely here. Instead, we give a more direct formula for the annealed entropy and only show that it is bounded above by the Kikuchi entropy (see discussion following Proposition 3.1).

4.1 Property M

To define notions of model mixing, we will impose distance functions on the finite sets V which form a given sofic approximation. For this purpose, we will assume $\Gamma $ is finitely generated and let $E \subset \Gamma $ be a finite symmetric generating subset. For $g\in \Gamma $ , let $|g|$ be the word-length of g, which is the length of the shortest word in E representing g. Given $\sigma :\Gamma \to \operatorname {\mathrm {Sym}}(V)$ , define distance in V by

$$ \begin{align*}d_\sigma(v,w) = \min\{ |g|:~ g \in \Gamma, \sigma(g)v=w\}.\end{align*} $$

If there does not exist g with $\sigma (g)v=w$ , then we set $d_\sigma (v,w)=+\infty $ . If $\sigma $ is not a homomorphism, then $d_\sigma $ may fail to satisfy the triangle inequality.

A subset $S \subset V$ is r -separated if $d_\sigma (v,w)> r$ for every pair of distinct $v,w \in S$ .

Suppose a model measure sequence $(\mu _n)_n$ converges locally and empirically to $\mu $ . We say the sequence is uniformly model mixing (umm) if for every finite $F \subset \Gamma $ and every $\epsilon>0$ , there is some $r<\infty $ and a sequence of finite subsets $W_n \subset V_n$ such that

$$ \begin{align*}|W_n| = (1-o(1))|V_n|,\end{align*} $$

and if $S \subset W_n$ is r-separated, then

$$ \begin{align*}\operatorname{\mathrm{H}}( (\mu_n)_{\sigma_n^{F}(S)} ) \ge |S| (\operatorname{\mathrm{H}}(\mu_F) - \epsilon),\end{align*} $$

where

• • $\mu _F$ is the probability measure on $\mathtt {A}^F$ which is the pushforward of $\mu $ under the projection map $\mathtt {A}^\Gamma \to \mathtt {A}^F$ ;

• • $\sigma _n^F(S) = \{\sigma _n(f)s:~f \in F, s\in S\}$ ;

• • $ (\mu _n)_{\sigma _n^{F}(S)} $ is the probability measure on $\mathtt {A}^{\sigma _n^{F}(S)}$ which is the pushforward of $\mu _n$ under the projection map $\mathtt {A}^{V_n} \to \mathtt {A}^{\sigma _n^{F}(S)}$ .

This is a microstates analog of uniform mixing, introduced by Rudolph and Weiss in [Reference Rudolph and Weiss62] for actions of an amenable group; see also [Reference Weiss74, Definition 10], where the name ‘uniform mixing’ appears for the first time. The main result of [Reference Austin and Burton7] is that if $(\mu _n)_n$ locally and empirically converges to $\mu $ and is uniformly model mixing with respect to a fixed sofic approximation $\Sigma $ , then the system $(\mathtt {A}^\Gamma ,\mu ,T)$ has completely positive entropy with respect to $\Sigma $ , in analogy with a corresponding result of [Reference Rudolph and Weiss62].

Unfortunately, we do not know whether the parity check sub-shifts of Theorem 1.3 are uniformly model mixing. Instead, we define a weaker version of model mixing which suffices.

Definition 4.2 (Property M).

Suppose $(\mu _n)_n$ is a model measure sequence and $\mu \in \operatorname {\mathrm {Prob}}^\Gamma (\mathtt {A}^\Gamma )$ is an invariant measure. We say the sequence has property M if for every $\epsilon>0$ and $0<r<\infty $ , there is a sequence of subsets $S_n \subset V_n$ such that

$$ \begin{align*}\liminf \frac{|S_n|}{|V_n|}>0\end{align*} $$

and

(11) $$ \begin{align} \operatorname{\mathrm{H}}( (\mu_n)_{\sigma_n^{B_r}(S_n)} ) \ge |S_n| (\operatorname{\mathrm{H}}(\mu_{B_r}) - \epsilon) \end{align} $$

for all n, where $B_r = {\mathrm {B}({r}, {e})} \subset \Gamma $ denotes the ball of radius r centered at the identity. In applications, $\mu $ will be a limit of the sequence $(\mu _n)_n$ , but we do not impose any such requirement for the definition.

In contrast with uniform model-mixing, we only require $S_n$ to have asymptotically positive density in $V_n$ , and there is no uniform lower bound on this density across different choices of $\epsilon , r$ . This density could be much smaller than $|B_r|^{-1}$ , for example. We also do not require the sets $S_n$ to be separated, although the lower bound (11) does usually imply a kind of approximate separation anyway.

We suspect that other variants of uniform model mixing could be used in a similar way to prove completely positive entropy, so Definition 4.2 is not an attempt at optimal generality. This is why we have chosen a rather bland name for Property M, although it is convenient in our work below.

Here is the main result of this section:

Proof. By Corollary 2.2, it suffices to consider factor maps to other shift systems. For these, the proof is based on the proof of [Reference Austin and Burton7, Theorem 1.2].

Step 1. We start by establishing some general entropy inequalities. For two or more jointly distributed random variables $X_1, \ldots , X_k$ , define the total correlation

$$\begin{align*}\operatorname{\mathrm{TC}}(X_1; \cdots; X_k) = \left(\sum_{i=1}^k \operatorname{\mathrm{H}}(X_i)\right) - \operatorname{\mathrm{H}}(X_1, \ldots, X_k). \end{align*}$$

This is a generalization of mutual information to more than two random variables, introduced in [Reference Watanabe72]. It can also be recursively defined by setting $\operatorname {\mathrm {TC}}(X_1; X_2) = \operatorname {\mathrm {I}}(X_1; X_2)$ and for $k \geq 3$

$$\begin{align*}\operatorname{\mathrm{TC}}(X_1; \cdots; X_k) = \operatorname{\mathrm{TC}}(X_1; \cdots; X_{k-1}) + \operatorname{\mathrm{I}}(X_1, \ldots, X_{k-1}; X_k). \end{align*}$$

Using this recursion and the data-processing inequality [Reference Cover and Thomas23, Theorem 2.8.1], it can be shown by induction on k that if f is any function and $Y_i = f(X_i)$ for each i, then

(12) $$ \begin{align} \operatorname{\mathrm{TC}}(Y_1; \cdots; Y_k) \leq \operatorname{\mathrm{TC}}(X_1; \cdots; X_k). \end{align} $$

This inequality has also appeared in [Reference Austin6, Lemma 4.3]. Note that the total correlation does not depend on the order in which the random variables are listed. Below, we will refer to the total correlation of a collection of random variables $\{X_i \,:\, i \in I\}$ indexed by a finite set I using the notation $\operatorname {\mathrm {TC}}(\{ X_i \,:\, i \in I\})$ , since fixing an ordering would unnecessarily complicate notation.

The Rokhlin distance between random variables $\alpha ,\beta $ which are defined on the same probability space is defined by $\textrm {d}^{\mathrm {Rok}}_\mu (\alpha , \beta ) = \operatorname {\mathrm {H}}_\mu (\alpha | \beta ) + \operatorname {\mathrm {H}}_\mu (\beta | \alpha )$ . This satisfies the triangle inequality, and it equals zero if and only if $\alpha $ and $\beta $ generate the same partition up to null sets. This distance can be used to control total correlation via the bound

$$ \begin{align*} &\hspace{-1in} {\lvert {\operatorname{\mathrm{TC}}(X_1; \cdots; X_k) - \operatorname{\mathrm{TC}}(Y_1; \cdots; Y_k)} \rvert} \\ &\leq {\lvert {\operatorname{\mathrm{H}}(X_1, \ldots, X_k)- \operatorname{\mathrm{H}}(Y_1, \ldots, Y_k)}\rvert} + \sum_{i=1}^k {\lvert {\operatorname{\mathrm{H}}(X_i) - \operatorname{\mathrm{H}}(Y_i)} \rvert} \\ &\leq 2 \sum_{i=1}^k \big( \operatorname{\mathrm{H}}(X_i | Y_i) + \operatorname{\mathrm{H}}(Y_i | X_i) \big) = 2 \sum_{i=1}^k \textrm{d}^{\mathrm{Rok}}(X_i,Y_i). \end{align*} $$

Step 2. Since $(\mu _n)_n$ locally and empirically converges to $\mu $ , if $S_n \subset V_n$ satisfies $\liminf \frac {{\lvert {S_n}\rvert } }{{\lvert {V_n} \rvert }}> 0$ , then

$$\begin{align*}\frac{1}{{\lvert {S_n}\rvert}} \sum_{v \in S_n} \operatorname{\mathrm{H}} \big( (\mu_n)_{\sigma_n^{B_r}(v)} \big) = \operatorname{\mathrm{H}}(\mu_{B_r}) + o(1). \end{align*}$$

So property M implies that for every $r,\varepsilon>0$ , there is a sequence of subsets $S_n \subset V_n$ such that $\liminf \frac {{\lvert {S_n}\rvert }}{{\lvert {V_n}\rvert }}> 0$ and

$$\begin{align*}\frac{1}{{\lvert {S_n}\rvert} } \operatorname{\mathrm{TC}}\big( \{ (\mathbf{x}_n)^{\sigma_n^{B_r}(v)} \,:\, v \in S_n \} \big) \leq \varepsilon \end{align*}$$

for all large n, where $\mathbf {x}_n$ is a random sample of $\mu _n$ and the projections $\{ (\mathbf {x}_n)^{\sigma _n^{B_r}(v)} \,:\, v \in S_n \}$ are jointly distributed in the natural way (from a common sample of $\mathbf {x}_n$ ).

Step 3. Now let $\phi \colon \mathtt {A}^\Gamma \to \mathtt {B}$ be a measurable map into a finite set $\mathtt {B}$ generating a factor map $\phi ^\Gamma $ as in (6). If this factor is nontrivial, then $\operatorname {\mathrm {H}}_\mu (\phi )> 0$ . We want to show that the property M assumption on $\mu _n$ implies that

$$\begin{align*}h_\Sigma(\mu\,;\,\phi^\Gamma)> 0.\end{align*}$$

Let $\lambda $ be the graphical joining of the factor map $\phi ^\Gamma $ as in (4).

Fix $r \in \mathbb {N}$ and a ${\mathrm {B}({r}, {e})}$ -local function $\psi \colon \mathtt {A}^\Gamma \to \mathtt {B}$ which approximates $\phi $ closely enough in measure that $\textrm {d}^{\mathrm {Rok}}_\mu (\psi , \phi ) < \frac {1}{8}\operatorname {\mathrm {H}}_\mu (\phi )$ .

Now with $\varepsilon = \frac {1}{8}\operatorname {\mathrm {H}}_\mu (\phi )$ and this r, let $(S_n)_n$ be the sequence of subsets of $V_n$ given by property M. Since $\psi $ is ${\mathrm {B}({r}, {e})}$ -local, the data-processing inequality (12) above implies that

$$\begin{align*}\operatorname{\mathrm{TC}}\big( \{ {\psi^{\sigma_n}}(\mathbf{x}_n)(v) \,:\, v \in S_n \} \big) \leq \operatorname{\mathrm{TC}}\big( \{ (\mathbf{x}_n)^{\sigma_n^{B_r}(v)} \,:\, v \in S_n \} \big) \leq \varepsilon {\lvert {S_n}\rvert}, \end{align*}$$

where $\mathbf {x}_n$ is a random sample of $\mu _n$ and $\psi ^{\sigma _n} \colon \mathtt {A}^{V_n} \to \mathtt {B}^{V_n}$ is defined as in (7).

Step 4. Let $(\psi _m)_m$ be a local approximating sequence sequence to $\phi $ , meaning that (8) holds, and hence, $\textrm {d}^{\mathrm {Rok}}_\mu (\phi , \psi _m)$ also converges to 0. Since the Rokhlin distance satisfies the triangle inequality, there is some $M \in \mathbb {N}$ such that if $m \geq M$ , then $\textrm {d}^{\mathrm {Rok}}_\mu (\psi , \psi _m) < \frac {1}{8} \operatorname {\mathrm {H}}_\mu (\phi )$ .

Given an open neighborhood $\mathcal {O} \ni \lambda $ , by [Reference Austin3, Prop. 4.10], there is some open neighborhood $\mathcal {U} \ni \mu $ and some $m \geq M$ such that, for all large enough n, the map $(\mathrm {id}_{\mathtt {A}^{V_n}},\psi _m^{\sigma _n})$ sends $\mathcal {U}$ -microstates to $\mathcal {O}$ -microstates. Let us also assume m is large enough that $\operatorname {\mathrm {H}}_\mu (\psi _m) \geq \frac {1}{2} \operatorname {\mathrm {H}}_\mu (\phi )$ .

Now fix some R such that $\psi $ and $\psi _m$ are both ${\mathrm {B}({R}, {e})}$ -local. By local convergence of $\mu _n$ to $\mu $ , for any $\delta>0$ , the fraction of $v \in S_n$ for which the local marginal ${\mathrm {Loc}({\mu _n}, {v})}_{B_R}$ is within total variation distance $\delta $ of $\mu _{B_R}$ is $1-o(1)$ . For the rest of the $v \in S_n$ , the term in the sum below has the upper bound $2 \log {\lvert {\mathtt {B}} \rvert } $ :

$$ \begin{align*} &\hspace{-1in} \tfrac{1}{{\lvert {S_n}\rvert} } {\lvert {\operatorname{\mathrm{TC}}\big( \{ {\psi^{\sigma_n}}(\mathbf{x}_n)(v) \,:\, v \in S_n \} \big) - \operatorname{\mathrm{TC}}\big( \{ {\psi_m^{\sigma_n}}(\mathbf{x}_n)(v) \,:\, v \in S_n \} \big)} \rvert} \\ &\leq \frac{2}{{\lvert {S_n}\rvert} } \sum_{v \in S_n} \left( \operatorname{\mathrm{H}}_{{\mathrm{Loc}({\mu_n}, {v})}} (\psi | \psi_m) + \operatorname{\mathrm{H}}_{{\mathrm{Loc}({\mu_n}, {v})} } (\psi_m | \psi) \right) \\ &\leq 2 \textrm{d}^{\mathrm{Rok}}_\mu(\psi_{m},\psi) + o(1). \end{align*} $$

Hence,

(13) $$ \begin{align} \tfrac{1}{{\lvert {S_n}\rvert} } \operatorname{\mathrm{TC}}\big( \{ {\psi_m^{\sigma_n}}(\mathbf{x}_n)(v) \,:\, v \in S_n \} \big) \leq \frac{3}{8} \operatorname{\mathrm{H}}_\mu(\phi) + o(1). \end{align} $$

By empirical convergence, for large n, the model measure $\mu _n$ is mostly supported on $\mathcal {U}$ -microstates. So $(\mathrm {id}_{\mathtt {A}^{V_n}},\psi _m^{\sigma _n})_*\mu _n$ is mostly supported on $\mathcal {O}$ -microstates, and using Fano’s inequality, we see that

$$\begin{align*}\frac{1}{{\lvert {V_n}\rvert} } \operatorname{\mathrm{H}}( {\psi_{m}^{\sigma_n}}_*\mu_n) \leq \frac{1}{{\lvert {V_n}\rvert} } \log {\lvert {\mathrm{proj}_n[\Omega(\sigma_n, \mathcal{O})]} \rvert} + o(1). \end{align*}$$

The total correlation bound (13) gives

$$ \begin{align*} \operatorname{\mathrm{H}}( {\psi_{m}^{\sigma_n}}_*\mu_n) = \operatorname{\mathrm{H}}( \psi_{m}^{\sigma_n}(\mathbf{x}_n)) &\ge \operatorname{\mathrm{H}}( \{ (\psi_{m}^{\sigma_n}(\mathbf{x}_n)(v) \,:\, v \in S_n \}) \\ &\geq \sum_{v \in S_n} \operatorname{\mathrm{H}}( (\psi_{m}^{\sigma_n}(\mathbf{x}_n)(v) \}) -{\lvert {S_n}\rvert} \big( \tfrac{3}{8} \operatorname{\mathrm{H}}_\mu(\phi) + o(1) \big). \end{align*} $$

Since $(\mu _n)_n$ converges locally to $\mu $ and $\psi _m$ is a local function,

$$\begin{align*}\frac{1}{{\lvert {S_n}\rvert} } \sum_{v \in S_n} \operatorname{\mathrm{H}}( (\psi_{m}^{\sigma_n}(\mathbf{x}_n)(v) ) = \operatorname{\mathrm{H}}_\mu(\psi_m) + o(1) \geq \tfrac{1}{2} \operatorname{\mathrm{H}}_\mu(\phi) + o(1). \end{align*}$$

So

$$\begin{align*}\frac{1}{{\lvert {V_n}\rvert} } \log {\lvert {\mathrm{proj}_n[\Omega(\sigma_n, \mathcal{O})]}\rvert} \geq \frac{{\lvert {S_n}\rvert} }{{\lvert {V_n}\rvert} } \left( \tfrac{1}{8}\operatorname{\mathrm{H}}_\mu(\phi) + o(1) \right) + o(1) , \end{align*}$$

and for every $\mathcal {O} \ni \lambda $ ,

$$\begin{align*}\liminf_{n \to \infty} \frac{1}{{\lvert {V_n}\rvert} } \log {\lvert {\mathrm{proj}_n[\Omega(\sigma_n, \mathcal{O})]}\rvert} \geq \left(\liminf_{n \to \infty} \frac{{\lvert {S_n}\rvert} }{{\lvert {V_n}\rvert} }\right) \tfrac{1}{8} \operatorname{\mathrm{H}}_\mu(\phi). \end{align*}$$

Since we chose $(S_n)_n$ independently of $\mathcal {O}$ , and $\liminf _{n \to \infty } \frac {{\lvert {S_n}\rvert } }{{\lvert {V_n}\rvert } }> 0$ , taking the infimum over $\mathcal {O}$ completes the proof.

5.1 Consequences of shattering

In this section, fix a sofic approximation $\Sigma = (\sigma _n)_n$ by homomorphisms, and assume that $(\mathtt {A}^\Gamma ,\mu ,T)$ has totally shattered microstate spaces along $\Sigma $ .

Theorem 5.2. For every $\varepsilon> 0$ , there is a neighbourhood $\mathcal {O}$ of $\mu $ for which the following holds. Let $(\mathtt {B}^\Gamma ,\nu ,T)$ be another shift system that has microstates along $\Sigma $ , let $(\mathtt {K}^\Gamma , \kappa ^\Gamma ,T)$ be a Bernoulli shift, and let

$$\begin{align*}\phi:\mathtt{B}^\Gamma \times \mathtt{K}^\Gamma \to \mathtt{A}\end{align*}$$

be a measurable map. If

(14) $$ \begin{align} \phi^\Gamma_\ast (\nu\times \kappa^\Gamma) \in \mathcal{O}, \end{align} $$

then

(15) $$ \begin{align} (\nu\times \kappa^\Gamma\times \kappa^\Gamma)\{(y,z,z'):\ \phi(y,z) \ne \phi(y,z')\} < \varepsilon. \end{align} $$

Intuitively, the conclusion is that if $\phi ^\Gamma $ is approximately a factor map onto $(\mathtt {A}^\Gamma ,\mu ,T)$ , then it must be approximately independent from the second coordinate in $\mathtt {B}^\Gamma \times \mathtt {K}^\Gamma $ . There are many ways to capture the latter assertion precisely, but (15) turns out to be convenient during the proof.

The proof of Theorem 5.2 has much in common with the main proof in [Reference Gamarnik and Sudan33]. In that paper, Gamarnik and Sudan used a relative of shattering to prove an a.a.s. upper bound on the maximum size of an independent set on a random regular graph that can be constructed using a local algorithm. The property they use there is now called the ‘overlap gap property’ and is actually a little weaker than being totally shattered. See [Reference Gamarnik32] for a recent survey. The reference [Reference Lyons46, Section 4] explains how the absence of an approximating local algorithm for certain combinatorial problems implies that a resulting limit process is not weakly contained in a Bernoulli shift.

Our proof of Theorem 5.2 needs a couple of known facts about microstate spaces for a product in which one factor is Bernoulli. We recall these as separate lemmas before starting the proof.

Lemma 5.3. Let $(\mathtt {B}^\Gamma ,\nu ,T)$ be a shift system and let $(\mathtt {L}^\Gamma ,\lambda ^\Gamma ,T)$ be a Bernoulli shift. Assume that $\mathbf {y}_n \in \mathtt {B}^{V_n}$ is a sequence such that $P^{\sigma _n}_{\mathbf {y}_n} \to \nu $ . Then

$$\begin{align*}\lambda^{V_n}\big\{\mathbf{z}:\ P^{\sigma_n}_{(\mathbf{y}_n,\mathbf{z})}\in \mathcal{O}\big\} \to 1\end{align*}$$

for every neighbourhood $\mathcal {O}$ of $\nu \times \lambda ^\Gamma $ . symbol

Lemma 5.3 is well known as folklore in the study of sofic entropy, and a full proof can be found inside the proofs of some of its consequences in the literature. The earliest and perhaps easiest to extract is inside the proof of the lower bound in [Reference Bowen11, Theorem 8.1].

The next lemma is more specialized but was also used in the second author’s previous counterexample to the weak Pinsker conjecture for some sofic groups: it is a special case of [Reference Bowen17, Proposition 7.9]. It refers to ‘hereditary’ neighbourhoods of a shift-invariant measure. We do not repeat the definition of these here; the only property we need is that they form a basis for the weak $^\ast $ topology.

Proof of Theorem 5.2.

Let $\delta $ be a shatter distance for the system along $\Sigma $ . Let $\varepsilon> 0$ be small enough that $2\varepsilon < \delta $ , and now let $\mathcal {O}_1$ and $n_0$ be a neighbourhood and positive integer as promised by Definition 5.1 for this choice of $\varepsilon $ . Lastly, choose a smaller neighbourhood $\mathcal {O}$ of $\mu $ whose closure is contained in $\mathcal {O}_1$ .

Let $\phi :\mathtt {B}^\Gamma \times \mathtt {K}^\Gamma \to \mathtt {A}$ be a measurable map such that $\phi _\ast ^\Gamma (\nu \times \kappa ^\Gamma ) \in \mathcal {O}$ . In the rest of the proof, we show that (15) holds for this $\mathcal {O}$ and with $3\varepsilon $ in place of $\varepsilon $ .

Step 1. Since $\mathtt {A}$ is finite and $\phi $ is measurable, for any $\eta> 0$ , there is an approximating map

$$\begin{align*}\psi:\mathtt{B}^\Gamma\times \mathtt{K}^\Gamma \to \mathtt{A}\end{align*}$$

such that

(16) $$ \begin{align} (\nu\times \kappa^\Gamma)\{\psi \ne \phi\} < \eta \end{align} $$

and such that (i) $\psi $ is a local map and (ii) $\psi $ depends on the coordinates in $\mathtt {K}^\Gamma $ only through some finite measurable partition $\mathcal {P}$ of $\mathtt {K}$ . If we choose $\eta $ sufficiently small in terms of $\mathcal {O}\subset \mathcal {O}_1$ and $\varepsilon $ , then (14) and (16) imply that $\psi ^\Gamma _\ast (\nu \times \kappa ^\Gamma )$ still lies in $\mathcal {O}$ , and also (16) implies that the desired conclusion (15) holds for $\phi $ with error $3\varepsilon $ if it holds for $\psi $ with error $2\varepsilon $ .

By replacing $\mathtt {K}$ with the set of cells $\mathcal {P}$ , we have therefore reduced our work to the case when $\mathtt {K}$ is finite and $\phi $ is F-local for some finite subset F of $\Gamma $ . We assume this for the rest of the proof and do not refer to $\psi $ again.

Having made these assumptions, let us note that the set

$$\begin{align*}\Delta:= \{(y,z,z'):\ \phi(y,z) \ne \phi(y,z')\}\end{align*}$$

is closed and open in $\mathtt {B}^\Gamma \times \mathtt {K}^\Gamma \times \mathtt {K}^\Gamma $ .

Step 2. Since $\phi $ is local, pushing forward by the equivariant map $\phi ^\Gamma $ acts continuously on probability measures. We may therefore choose a neighbourhood $\mathcal {V}$ of $\nu \times \kappa ^\Gamma $ such that $\phi ^\Gamma _\ast [\mathcal {V}] \subset \mathcal {O}$ . Since $\sigma _n$ is a homomorphism, we have $P_{\phi ^{\sigma _n}(\mathbf {x})}^{\sigma _n} = \phi ^\Gamma _* P_{\mathbf {x}}^{\sigma _n}$ , so this implies that

(17) $$ \begin{align} \phi^{\sigma_n}[\Omega(\sigma_n,\mathcal{V})] \subset \Omega(\sigma_n,\mathcal{O}) \quad \mbox{for every}\ n. \end{align} $$

In addition, since hereditary neighbourhoods form a basis, we may let $\mathcal {U}$ be a hereditary neighbourhood of $\nu \times \kappa ^\Gamma $ whose closure is contained in $\mathcal {V}$ .

Step 3. By assumption, there is a sequence $\mathbf {y}_n \in \mathtt {B}^{V_n}$ such that $P^{\sigma _n}_{\mathbf {y}_n} \to \nu $ . Fix this for the rest of the proof.

For each n, consider $\mathbf {z}_n$ and $\mathbf {z}_n' \in \mathtt {K}^{V_n}$ drawn independently at random according to $\kappa ^{V_n}$ , and consider the event

$$\begin{align*}E := \big\{(\mathbf{z}_n,\mathbf{z}_n') \in \mathtt{K}^{V_n}\times \mathtt{K}^{V_n}:\ (\mathbf{y}_n,\mathbf{z}_n),(\mathbf{y}_n,\mathbf{z}_n')\in \Omega(\sigma_n,\mathcal{U})\big\}.\end{align*}$$

By Lemma 5.3, we have

(18) $$ \begin{align} (\kappa^{V_n}\times \kappa^{V_n})(E) \to 1. \end{align} $$

When E occurs, we can draw the following additional conclusions:

• • The points $(\mathbf {y}_n,\mathbf {z}_n)$ and $(\mathbf {y}_n,\mathbf {z}_n')$ are $(\delta /|F|)$ -connected within $\Omega (\sigma _n,\mathcal {V})$ for all sufficiently large n (not depending on the specific values of $\mathbf {y}_n$ , $\mathbf {z}_n$ or $\mathbf {z}_n'$ ), by Lemma 5.4.

• • Since $\phi $ is F-local, the map $\phi ^{\sigma _n}$ is $|F|$ -Lipschitz for the normalized Hamming metrics (Lemma 2.3), so $\phi ^{\sigma _n}(\mathbf {y}_n,\mathbf {z}_n)$ and $\phi ^{\sigma _n}(\mathbf {y}_n,\mathbf {z}_n')$ both lie in $\Omega (\sigma _n,\mathcal {O})$ and are $\delta $ -connected within that set, by the previous conclusion and (17).

• • By total shattering and our choice of $\mathcal {O}$ , we can now deduce that

  (19) $$ \begin{align} \textrm{d}^{(V_n)}\big(\phi^{\sigma_n}(\mathbf{y}_n,\mathbf{z}_n),\phi^{\sigma_n}(\mathbf{y}_n,\mathbf{z}_n')\big) < \varepsilon \end{align} $$
  for all sufficiently large n. Indeed, if n is large enough and
  $$\begin{align*}\phi^{\sigma_n}(\mathbf{y}_n,\mathbf{z}_n) = \mathbf{w}_1,\ \dots,\ \mathbf{w}_l = \phi^{\sigma_n}(\mathbf{y}_n,\mathbf{z}_n')\end{align*}$$
  is a sequence in $\Omega (\sigma _n,\mathcal {O})$ with all consecutive distances less than $\delta $ , then total shattering implies that all of these distances are actually less than $\varepsilon $ . Then the triangle inequality implies that $\textrm {d}^{(V_n)}(\mathbf {w}_1,\mathbf {w}_3) < 2\varepsilon $ , which is still less than $\delta $ . We may therefore invoke total shattering again to conclude that in fact ${\textrm {d}^{(V_n)}(\mathbf {w}_1,\mathbf {w}_3) < \varepsilon }$ . Now a simple induction shows that in fact $\textrm {d}^{(V_n)}(\mathbf {w}_1,\mathbf {w}_i) < \varepsilon $ for every i, giving (19) when $i=l$ .

Unpacking the definitions of normalized Hamming metric and empirical distribution, the left-hand side of (19) is equal to

$$\begin{align*}\frac{1}{|V_n|}\sum_{v \in V_n} 1_{\left\{\phi(\Pi^{\sigma_n}_v\mathbf{y}_n,\Pi^{\sigma_n}_v\mathbf{z}_n)\ne \phi(\Pi^{\sigma_n}_v\mathbf{y}_n,\Pi^{\sigma_n}_v\mathbf{z}_n')\right\}} = P^{\sigma_n}_{(\mathbf{y}_n,\mathbf{z}_n,\mathbf{z}_n')}(\Delta).\end{align*}$$

Therefore, in view of the conclusions above, (18) implies that

(20) $$ \begin{align} (\kappa^{V_n}\times \kappa^{V_n})\big\{P^{\sigma_n}_{(\mathbf{y}_n,\mathbf{z}_n,\mathbf{z}_n')}(\Delta) < \varepsilon\big\} \to 1. \end{align} $$

However, since $\Delta $ is a closed and open set, another appeal to Lemma 5.3 (this time for the larger product $\nu \times \kappa ^\Gamma \times \kappa ^\Gamma $ ) shows that

(21) $$ \begin{align} (\kappa^{V_n}\times \kappa^{V_n}) \big\{P^{\sigma_n}_{(\mathbf{y}_n,\mathbf{z}_n,\mathbf{z}_n')}(\Delta)> (\nu\times \kappa^\Gamma\times \kappa^\Gamma)(\Delta) - \epsilon\big\} \to 1. \end{align} $$

The limits (20) and (21) can hold simultaneously only if

$$\begin{align*}(\nu \times \kappa^\Gamma\times \kappa^\Gamma)(\Delta) < 2\varepsilon.\end{align*}$$

Since $\varepsilon $ was arbitrary, this completes the proof.

Theorem 5.2 has several more streamlined consequences.

Proof. Part 1. Let the factor map in question be $\xi ^\Gamma $ for some measurable map ${\xi :Y\times \mathtt {K}^\Gamma \to \mathtt {A}}$ . Let $\varepsilon> 0$ , and let $\mathcal {O}$ be the neighbourhood of $\mu $ given by Theorem 5.2 for this $\varepsilon $ .

By approximating the level sets of $\xi $ by measurable rectangles, for any $\eta> 0$ , we can choose (i) a factor map $\pi $ from $(Y,\theta ,S)$ to a finite-alphabet shift system $(\mathtt {B}^\Gamma ,\nu ,T)$ and (ii) a map $\phi :\mathtt {B}^\Gamma \times \mathtt {K}^\Gamma \to \mathtt {A}$ such that

(22) $$ \begin{align} (\theta\times \kappa^\Gamma)\{(y,z):\ \phi(\pi(y),z) \ne \xi(y,z)\} < \eta. \end{align} $$

Provided we choose $\eta $ small enough, the inequality (22) implies that

$$ \begin{align*}\phi^\Gamma_\ast(\nu\times \kappa^\Gamma) \in \mathcal{O}.\end{align*} $$

By shrinking further if necessary, we may also assume that $\eta < \varepsilon $ .

Fix a choice of $\eta $ with these properties, and consider the resulting maps $\pi $ and $\phi $ . Since $(Y,\theta ,S)$ has microstates along $\Sigma $ , so does its factor $(\mathtt {B}^\Gamma ,\nu ,T)$ . By our choice of $\eta $ , we have arranged that this factor system and the map $\phi $ satisfy (14). Therefore, Theorem 5.2 tells us that they also satisfy (15). Combined with (22), this gives

$$ \begin{align*} &(\theta\times \kappa^\Gamma\times \kappa^\Gamma)\{(y,z,z'):\ \xi(y,z)\neq \xi(y,z')\} \\ &\qquad\qquad\qquad\le 2\eta + (\nu\times \kappa^\Gamma\times \kappa^\Gamma)\{(y_1,z,z'):\ \phi(y_1,z)\neq \phi(y_1,z')\} < 2\eta + \varepsilon < 3\varepsilon. \end{align*} $$

Since $\varepsilon $ is arbitrary, the left-hand side here must actually equal $0$ , so in fact,

$$\begin{align*}\xi(y,z) = \xi(y,z') \quad \mbox{for}\ (\theta\times \kappa^\Gamma\times \kappa^\Gamma)\mbox{-a.e.}\ (y,z,z').\end{align*}$$

Therefore, possibly after adjusting on a set of measure zero, $\xi $ depends on only the first coordinate in $Y\times \mathtt {K}^\Gamma $ .

Part 2. If

$$\begin{align*}\xi^\Gamma:(Y,\theta,S)\times (\mathtt{K}^\Gamma,\kappa^\Gamma,T)\to (\mathtt{A}^\Gamma,\mu,T)\end{align*}$$

is an isomorphism, then the system $(Y,\theta ,S)$ is a factor of $(\mathtt {A}^\Gamma ,\mu ,T)$ , and so it inherits the property of having microstates along $\Sigma $ . Therefore, Part 1 shows that, up to agreement a.e., $\xi $ depends on only the first coordinate in $Y\times \mathtt {K}^\Gamma $ . Since $\xi ^\Gamma $ is an isomorphism, this is possible only if the Bernoulli factor $(\mathtt {K}^\Gamma ,\kappa ^\Gamma ,T)$ is trivial.

Part 3. Towards a contradiction, suppose that $(\mathtt {K}^\Gamma ,\kappa ^\Gamma ,T)$ is a Bernoulli shift and that $\phi _m:\mathtt {K}^\Gamma \to \mathtt {A}$ is a sequence of measurable maps such that

(23) $$ \begin{align} \phi^\Gamma_{m\ast}\kappa^\Gamma \to \mu. \end{align} $$

Apply Theorem 5.2 by inserting a trivial one-point system in the place of $(\mathtt {B}^\Gamma ,\nu ,T)$ . The conclusion is that

$$\begin{align*}(\kappa^\Gamma\times \kappa^\Gamma)\{(z,z'):\ \phi_m(z) \ne \phi_m(z')\} \to 0.\end{align*}$$

However, this implies that the distribution of $\phi _m$ is converging to $\delta _a$ for some $a\in \mathtt {A}$ . Combined with (23), it follows that $\mu = \delta _{(\dots ,a,a,\dots )}$ .

5.2 Shattering and Bernoulli splittings

Suppose the shift system $(\mathtt {A}^\Gamma , \mu , T)$ has totally shattered microstate spaces along $\Sigma $ . Note as soon as $\varepsilon < \delta /2$ , if $\mathcal {O}$ is the neighborhood of $\mu $ given by the definition, then the relation ‘ $\textrm {d}^{(V_n)} < \delta $ ’ restricted to the microstate space $\Omega (\sigma _n, \mathcal {O})$ is an equivalence relation, so it partitions the microstate space into small-diameter, well-separated clusters.

In this section, we give a second proof of Corollary 5.5(2) and sketch a third. Both of these approaches are based on the clusters of microstates: one uses the number of clusters, which is one of the sofic homological invariants introduced in [Reference Bowen17], and the other uses the sizes of clusters. Both can be compared to the use of the ‘overlap gap property’ introduced in [Reference Gamarnik and Sudan33] to prove an a.a.s. upper bound on the size of an independent set on a random regular graph, when the independent set is required to be constructed from a local algorithm.

The size of clusters can be used as follows: if a direct Bernoulli factor exists, it can be shown that its entropy rate is a uniform lower bound for the exponential size of all microstate clusters, while if a system has totally shattered microstate spaces, then all its clusters are of subexponential size. This implies Corollary 5.5(2).

In the rest of the section, we give a proof using the number of clusters. First, we give relevant definitions.

If $\mathcal {O}$ is a subset of a metric space and $x,y \in \mathcal {O}$ , we say $x,y$ are $\delta $ -connected within $\mathcal {O}$ if there is a sequence of points $z_1, \ldots , z_l \in \mathcal {O}$ with $z_1 = x$ , $z_l = y$ , and $\textrm {d}(z_i, z_{i+1}) < \delta $ for each i. This defines an equivalence relation on $\mathcal {O}$ , which we denote $\mathrm {cl}(\mathcal {O},\delta )$ .

Given a map $\sigma \colon \Gamma \to \operatorname {\mathrm {Sym}}(V)$ , a labeling $\mathbf {x} \in \mathtt {A}^{V}$ , an open set $\mathcal {O} \subset \operatorname {\mathrm {Prob}}(\mathtt {A}^\Gamma )$ , and $\delta>0$ , let

$$\begin{align*}\mathrm{cl}(\sigma, \mathbf{x}, \mathcal{O}, \delta) = [x]_{\mathrm{cl}(\Omega(\sigma,\mathcal{O}),\delta)} = \{ \mathbf{z} \in \mathtt{A}^{V} \,:\, \mathbf{z} \text{ is }\delta\text{-connected to } \mathbf{x} \text{ within } \Omega(\sigma,\mathcal{O}) \} , \end{align*}$$

where $\Omega (\sigma ,\mathcal {O}) \subset \mathtt {A}^{V}$ has the normalized Hamming metric $\textrm {d}^{(V)}$ . In particular, $\mathrm {cl}(\sigma ,\mathbf {x}, \mathcal {O}, \delta ) \subset \Omega (\sigma ,\mathcal {O})$ .

Now if in addition to the above we have some $\mathcal {O}' \subset \mathcal {O}$ , the quotient

is the set of clusters of $\mathcal {O}'$ -microstates that are $\delta $ -connected within $\Omega (\sigma _n, \mathcal {O})$ . We define

Informally, we first set coarseness parameters $\mathcal {O}, \delta $ which divide microstate spaces into clusters. We consider the exponential growth rate of the number of these clusters which contain $\mathcal {O}'$ -good microstates for $\mu $ . At first, it might seem more natural to consider directly the growth rate of the number of $\delta $ -connected clusters within a single $\Omega (\sigma _n, \mathcal {O})$ :

then take $\delta $ to $0$ and in some sense $\mathcal {O}$ to $\mu $ . But there is no monotonicity in $\mathcal {O}$ : shrinking the neighborhood of $\mu $ removes some microstates from $\Omega (\sigma _n, \mathcal {O})$ , but this can both remove some clusters and/or break a cluster into multiple pieces. Considering pairs $\mathcal {O}' \subset \mathcal {O}$ is one natural way around this. See also the discussion of a related definition of connected model spaces in [Reference Austin4, Section 2.2].

In [Reference Bowen17], $b_{0,\Sigma }(\mu )$ is called the $0$ th Betti number of $\mu $ . If X is totally disconnected and $\mu \in \operatorname {\mathrm {Prob}}^\Gamma (X^\Gamma )$ , then $b_{0,\Sigma }(\mu )$ is a measure-conjugacy invariant [Reference Bowen17, Corollary 4.2].

It follows directly from the definition that $b_{0,\Sigma }(\mu ) \le \operatorname {\mathrm {h}}_\Sigma (\mu )$ (a similar inequality holds for higher-dimensional sofic homology theories [Reference Bowen17, Lemma 7.13]).

Proof. Recall that in general $b_{0,\Sigma }(\mu ) \leq \operatorname {\mathrm {h}}_\Sigma (\mu )$ , so we only have to prove that having totally shattered microstate spaces implies the reverse inequality.

Let $\delta>0$ be as in the definition of totally shattered microstate spaces. Given $\varepsilon < \delta /2$ , there exists a neighborhood U of $\mu $ such that for all large n, every $\mathbf {x},\mathbf {y} \in \Omega (\sigma _n, U)$ have $\textrm {d}^{(V_n)}(\mathbf {x},\mathbf {y}) \in [0,\varepsilon ) \cup [\delta , \infty )$ . In particular, for every $\mathbf {x} \in \Omega (\sigma _n, U)$ ,

$$\begin{align*}\mathrm{cl}(\sigma_n, \mathbf{x}, U, \varepsilon) \subseteq {\mathrm{B}({\varepsilon}, {\mathbf{x}})} , \end{align*}$$

where ${\mathrm {B}({\varepsilon }, {\mathbf {x}})}$ is the radius- $\varepsilon $ ball around $\mathbf {x}$ . Thus,

$$\begin{align*}{\lvert {\mathrm{cl}(\sigma_n, \mathbf{x}, U, \varepsilon)}\rvert} \leq {\lvert {{\mathrm{B}({\varepsilon}, {\mathbf{x}})} }\rvert} \le \exp\big( {\lvert {V_n}\rvert} (\operatorname{\mathrm{H}}(\varepsilon) + \varepsilon \log {\lvert {\mathtt{A}}\rvert} + o(1) )\big). \end{align*}$$

This leads to a lower bound on the number of clusters for all $\mathcal {O}' \subset U$ :

Hence,

and taking the supremum over $\varepsilon>0$ and $U \ni \mu $ completes the proof.

Alternate proof of Corollary 5.5(2).

To obtain a contradiction, suppose that $(\mathtt {A}^\Gamma ,\mu ,T)$ is measurably conjugate to the direct product of a nontrivial Bernoulli shift $(\mathtt {K}^\Gamma ,\kappa ^\Gamma ,T)$ with another shift system $(\mathtt {B}^\Gamma ,\nu ,S)$ where $\mathtt {B}$ is a compact metrizable space. We may assume $\mathtt {B}$ is totally disconnected without loss of generality because any dynamical system is measurably conjugate to a system of this form; this assumption is used in results of [Reference Bowen17] cited below.

Sofic entropy is additive under taking direct products with a Bernoulli shift [Reference Bowen11, Theorem 8.1], so

(24) $$ \begin{align} \operatorname{\mathrm{h}}_\Sigma(\mathtt{A}^\Gamma,\mu,T) = \operatorname{\mathrm{H}}(\mathtt{K},\kappa) + \operatorname{\mathrm{h}}_\Sigma(\mathtt{B}^\Gamma,\nu,S)> \operatorname{\mathrm{h}}_\Sigma(\mathtt{B}^\Gamma,\nu,S). \end{align} $$

Theorem 7.8 of [Reference Bowen17] implies that the $0$ -dimensional sofic homology theories of $(\mathtt {A}^\Gamma ,\mu ,T)$ and $(\mathtt {B}^\Gamma ,\nu ,S)$ are equivalent. In particular, this implies that the exponential rate of growth of the number of clusters in the microstate spaces of the two actions are the same. In the notation of [Reference Bowen17], this means

$$ \begin{align*}b_{0,\Sigma}(\mathtt{A}^\Gamma,\mu,T) = b_{0,\Sigma}(\mathtt{B}^\Gamma,\nu,S)\le \operatorname{\mathrm{h}}_\Sigma(\mathtt{B}^\Gamma,\nu,S),\end{align*} $$

where the last inequality holds by [Reference Bowen17, Lemma 7.13].

Because $(\mathtt {A}^\Gamma ,\mu ,T)$ has totally shattered microstate spaces, by Lemma 5.6, we have $b_{0,\Sigma }(\mathtt {A}^\Gamma ,\mu ,T) = \operatorname {\mathrm {h}}_\Sigma (\mathtt {A}^\Gamma ,\mu ,T)$ . Combined with the previous inequality, this implies

$$ \begin{align*}\operatorname{\mathrm{h}}_\Sigma(\mathtt{A}^\Gamma,\mu,T) = b_{0,\Sigma}(\mathtt{A}^\Gamma,\mu,T) \le \operatorname{\mathrm{h}}_\Sigma(\mathtt{B}^\Gamma,\nu,S),\end{align*} $$

which contradicts (24). This contradiction finishes the proof.

PART II

Parity check subshifts

In this part, we fix natural numbers $d,k$ with $k> d \geq 3$ and let $\Gamma =\Gamma _{d,k}$ be the d-fold free product of order-k cyclic groups:

$$\begin{align*}\Gamma := \langle s_1,\dots,s_d:\ s_1^k = \dots = s_d^k = e\rangle = \underbrace{\mathbb{Z}_k\ast \cdots \ast \mathbb{Z}_k}_{d}.\end{align*}$$

Let $X \le \mathbb {Z}_2^\Gamma $ be the closed subgroup defined by

$$\begin{align*}X = \left\{x \in \mathbb{Z}_2^\Gamma:\ \sum_{j=0}^{k-1}x_{gs_i^j} = 0 \ \forall g\in \Gamma,\ i = 1,\dots,d\right\},\end{align*}$$

and let $\mu = m_X$ be the Haar probability measure on X.

If F is any subset of $\Gamma $ , let $X_F$ be the image of X under the coordinate projection map $\mathbb {Z}_2^\Gamma \to \mathbb {Z}_2^F$ , and let $m_F$ be the pushforward of $m_X$ under this projection. We mostly use these notations when F is $B_r$ , the ball of radius r centered at the identity in the Cayley graph of $\Gamma $ , where we use $\{s_i^j:~ 1\le i \le d, 1\le j \le k-1\}$ for the generating set.

6.1 The use of LDPC codes

Our proofs of the main theorems are considerably simplified by using the special structure of the system as a subgroup of $\mathbb {Z}_2^\Gamma $ . We largely do so via the corresponding finitary codes constructed over the sofic approximations. Given a k-uniform homomorphism $\sigma :\Gamma _{d,k}\to \operatorname {\mathrm {Sym}}(V)$ , let

$$\begin{align*}X_\sigma := \left\{\mathbf{x} \in \mathbb{Z}_2^V:\ \sum_{j=0}^{k-1}x_{\sigma_i^j(v)} = 0\ \forall v\in V,\ i=1,\dots,d\right\}.\end{align*}$$

Let $\mu _\sigma $ be the uniform distribution on $X_\sigma $ .

These are the obvious finitary analogs of X and $m_X$ themselves. It turns out that these finitary constructs can be used as better and better approximations to the infinitary system and measure, and their linear structure makes them easier to analyze than the ‘looser’ sets $\Omega (\sigma ,U)$ that are used to define sofic entropy.

In fact, sets such as $X_\sigma $ are classical objects in coding theory. They are linear codes over the field $\mathbb {Z}_2$ , each defined by a collection of parity-check constraints. Since each of those parity-checks involves only k vertices, and k is fixed as n grows, these are examples of low-density parity-check (‘LDPC’) codes. Such codes were introduced in Gallager’s PhD thesis [Reference Gallager31, Reference Gallager30]. After many years of relative neglect, they were rediscovered independently by MacKay in the late 1990s [Reference MacKay47], and they are now a textbook family of codes with desirable properties. A good basic reference for their theory is [Reference MacKay48, Chapter 47], and a more dedicated treatment is [Reference Richardson and Urbanke61]. In fact, the essence of the parity-check subshift X itself already appears in those sources too, playing the role of an ‘idealized limit code’ on which to investigate the performance of local decoding algorithms: see, for instance, [Reference MacKay48, Figure 47.11] and the discussion around it.

6.2 Outline of the rest of the paper

Our use of the measures $\mu _\sigma $ to analyze the system $(X,m_X,T)$ rests on the following main results. As before, we confine the index n to multiples of k. Abbreviate $X_{\sigma _n}$ to $X_n$ and $\mu _{\sigma _n}$ to $\mu _n$ .

Recall that $\mathbb {P}_n$ is the uniform probability measure on $\operatorname {\mathrm {Hom}}_{\mathrm {unif}}(\Gamma , \operatorname {\mathrm {Sym}}(n))$ which is the set of k-uniform homomorphisms $\sigma :\Gamma \to \operatorname {\mathrm {Sym}}(n)$ .

Theorem 6.1. There are subsets

$$\begin{align*}\Omega^{\prime}_n \subseteq \operatorname{\mathrm{Hom}}_{\mathrm{unif}}(\Gamma,\operatorname{\mathrm{Sym}}(V_n))\end{align*}$$

such that $\mathbb {P}_n(\Omega ^{\prime }_n) \to 1$ and the following holds. If $\sigma _n\in \Omega _n'$ for each n, then

1. 1. $(\mu _n)_n$ has property M;

2. 2. $(\mu _n)_n$ converges locally and empirically to $m_X$ ;

3. 3. $(X,m_X,T)$ has totally shattered microstate spaces along $\Sigma =(\sigma _n)_n$ .

Our proof of Theorem 6.1 relies on the linear structure of the codes $X_\sigma $ .

In the next few sections, we introduce notation to formulate Proposition 7.1 which, roughly speaking, rules out near-cancellations among parity-check words of a typical $\sigma _n \sim \mathbb {P}_n$ . Section §7.1 proves Theorem 6.1(1) from Proposition 7.1. The rest of §7 proves Proposition 7.1.

Item (2) of Theorem 6.1 is proven in §8. Its proof relies on item (1). Item (3) is proven in §10. Its proof does not refer to items (1) or (2). Theorem B is an immediate consequence of item (3) and Corollary 5.5(2).

Theorem 1.3 is proven in §9. Part (b) of that theorem follows readily from items (1) and (2) of Theorem 6.1 together with Theorem 4.3. Part (a) (which computes the sofic entropy value) uses item (2) and the Bethe-Kikuchi entropy theory of §3.

6.3 Random factor graphs

In this subsection, fix a size n that is divisible by k and suppress it from the notation: thus, for instance, $\mathbb {P}$ stands for $\mathbb {P}_n$ . Most of our work towards Theorem 6.1 consists of estimates of various probabilities under $\mathbb {P}$ .

Several of these estimates involve a sum or union bound over possible subfamilies of the set of all hyper-edges of the form (9). We need to be able to move the sum outside an expectation, and for this purpose, the sum must be over a range which is fixed, not random. For this reason, it is convenient to augment the information in $\sigma _n$ with a labeling of the family of hyper-edges (9) by a fixed index set.

Let $E_1$ , …, $E_d$ be disjoint sets, each of size $n/k$ , and let $E := E_1\cup \cdots \cup E_d$ . Taking some terminology from coding theory, we refer to the elements of E as check nodes: this is explained further in Subsection 6.4 below. Let $\sigma \in \operatorname {\mathrm {Hom}}_{\mathrm {unif}}(\Gamma ,\operatorname {\mathrm {Sym}}(V))$ with $|V|=n$ . Fix $i \in [d]$ , and consider that V is partitioned into the orbits of the generator $\sigma (s_i)$ . Each orbit corresponds to a hyper-edge as in (9). Since each hyper-edge has size k, there are $n/k$ of them, and so there exists a bijection between $E_i$ and this family of hyper-edges. Let us choose such a bijection uniformly and independently at randomly for each i, and record the result as a subset $H\subseteq E\times V$ : a pair $(e,v) \in E_i\times V$ lies in H if e is attached by the $i^{\mathrm {th}}$ bijection to the hyper-edge that contains v.

We regard H as a bipartite graph on the disjoint union of E and V. As such, each check node in E has exactly k neighbours in V, and each vertex in V has exactly one neighbour in each of the subsets $E_i$ (and thus d neighbours in total). It follows that each intersection $H\cap (E_i\times V)$ is equivalent to a partition of V into parts labeled by $E_i$ . In the sequel, we borrow some more terminology from coding theory and refer to any such bipartite graph H on E and V as a factor graph: see, for instance, [Reference MacKay48, Sections 26.1 and 47.2]. Beware that this is actually a slight deviation from standard usage, which would not insist that H be a union of the partitions $H\cap (E_i\times V)$ , but here we do take this as part of the definition of a ‘factor graph’.

More generally, if $F \subseteq E$ , then a partial factor graph on F and V is a bipartite graph $M \subseteq F\times V$ such that every check node in F has precisely k neighbours in V and every vertex in V is joined to at most one check node in each intersection $F\cap E_i$ . Equivalently, there exists a factor graph H such that $M = H \cap (F\times V)$ . In particular, if $F = E_i$ , then a partial factor graph is simply a partition of V into k-sets that are labeled by $E_i$ .

If H is a factor graph on E and V and $F\subseteq E$ , then the vertex neighbourhood of F is the set

$$\begin{align*}\mathtt{Vert}(H;F) := \{v \in V:\ (e,v) \in H\ \mbox{for some}\ e \in F\}.\end{align*}$$

We also use the notation $\mathtt {Vert}(M;F) $ for a partial factor graph M in the same way.

Similarly, the check-node neighbourhood of $U\subseteq V$ is the set

$$\begin{align*}\mathtt{Check}(H;U) := \{e \in E:\ (e,v) \in H\ \mbox{for some}\ v \in V\}.\end{align*}$$

We may iterate these definitions to define neighbourhoods with larger radii. In general, for $F \subseteq E$ or $U\subseteq V$ , we define

$$ \begin{align*} \mathtt{Vert}_1(H;F) &:= \mathtt{Vert}(H;F),\\ \mathtt{Check}_1(H;U) &:= \mathtt{Check}(H;U),\\ \mathtt{Check}_1(H;F) &:= \mathtt{Check}_1\big(\mathtt{Vert}_1(H;F)\big) \\ \mbox{and} \quad \mathtt{Vert}_1(H;U) &:= \mathtt{Vert}_1\big(\mathtt{Check}_1(H;U)\big). \end{align*} $$

Then for integers $r> 1$ , we make the recursive definitions

$$\begin{align*}\mathtt{Vert}_r(H;F) := \mathtt{Vert}_1(H;\big( \mathtt{Vert}_{r-1}(H;F)\big),\end{align*}$$

and the same with U in place of F or $\mathtt {Check}$ in place of $\mathtt {Vert}$ . In graph theoretic terms, if $F \subseteq E$ , then $\mathtt {Vert}_r(H;F)$ is the set of all vertices whose graph distance in H is at most $2r-1$ from F, $\mathtt {Check}_r(H;F)$ is the set of all check nodes whose graph distance in H is at most $2r$ from F, and similarly for subsets of V.

These neighbourhoods are compatible with the Cayley graph neighbourhoods $B_r = {\mathrm {B}({e}, {r})}$ induced by the word metric on $\Gamma $ (with respect to the generating set $\{s_i^j:~ 1 \le i \le d, 1\le j \le k-1\})$ . Specifically, if H arises from $\sigma $ through the construction above, and $U\subseteq V$ , then

$$\begin{align*}\sigma^{B_r}(U) := \{\sigma^g(u):\ g\in B_r,\ u \in U\} = \mathtt{Vert}_r(H;U).\end{align*}$$

Now consider $\sigma \sim \mathbb {P}$ and generate H from it as described above. Then H results from two random steps: the choice of $\sigma $ and then the choice of a bijection for each i. Let $\tilde {\mathbb {P}}$ be the joint distribution of $(\sigma ,H)$ after this construction. This is a probability measure on

$$\begin{align*}\tilde{\Omega} := \operatorname{\mathrm{Hom}}_{\mathrm{unif}}(\Gamma,\operatorname{\mathrm{Sym}}(V)) \times \{0,1\}^{E\times V}.\end{align*}$$

It is a coupling of $\mathbb {P}$ to the law of H described above, and it has the following simple properties:

• • Given $\sigma $ , the conditional distribution of H is uniform over $((n/k)!)^d$ choices of bijections.

• • Given H, the conditional distribution of $\sigma $ is uniform over all choices of cyclic orderings for each hyper-edge of the multi-hyper-graph: there are ${((k-1)!)^{dn/k}}$ such choices for any H.

• • Under the marginal distribution of H, the intersections $H\cap (E_i\times V)$ are independent as i varies.

For $(\sigma ,H)$ drawn from $\tilde {\mathbb {P}}$ , the underlying multi-hyper-graph may be read off from either coordinate: it is the multi-set of all $\sigma (s_i)$ orbits for $i\in [d]$ , and it is also the collection of all vertex neighbourhoods of the check nodes according to H. The labeling of these hyper-edges by the fixed set E that is given by H is convenient for union bounds and other forms of counting. Most of our probabilistic estimates concerning these hyper-graphs later refer to $\tilde {\mathbb {P}}$ rather than $\mathbb {P}.$

6.4 Parity-check matrices

To prove the desired properties of the random measures $\mu _{\sigma _n}$ , we make careful use of the linear structure of the codes $X_n$ . As is standard in coding theory, this structure is conveniently summarized by a parity-check matrix.

To introduce this point of view, we start with some more notation. If a ground set A is understood and $B \subseteq A$ , then $\mathbf {e}_B$ denotes the mod- $2$ indicator function of B: this is the element of $\mathbb {Z}_2^A$ with entries equal to $\mathtt {1}$ precisely at the indices in B.

Now consider a vertex set V of size n which is a multiple of k, and let $E = E_1\cup \cdots \cup E_d$ as in Subsection 3.1. Let H be a factor graph on E and V, and turn it into the $(E\times V)$ -matrix $\mathbf {H} = \mathbf {e}_H$ , which also takes values in $\mathbb {Z}_2$ . If this generates the same hyper-graph as $\sigma \in \operatorname {\mathrm {Hom}}_{\mathrm {unif}}(\Gamma , \operatorname {\mathrm {Sym}}(V))$ , then our code is given by

$$\begin{align*}X_\sigma = \{\mathbf{x} \in \mathbb{Z}_2^V:\ \mathbf{H}\mathbf{x} = \boldsymbol{0}\} = \ker \mathbf{H}.\end{align*}$$

In this role, $\mathbf {H}$ is called the parity-check matrix (for an introduction, see, for instance, [Reference Cameron and van Lint20, Chapter 9], [Reference Cover and Thomas23, Section 7.11] or [Reference MacKay48, Chapter 1], especially the discussion of Exercise 1.9). This is why we refer to the elements of E as ‘check nodes’: each corresponds to a row of $\mathbf {H}$ , which every codeword $\mathbf {x}$ must be orthogonal to. The representation of a code using a set of connections between vertices and check nodes is a common example of a ‘factor graph representation’ [Reference MacKay48, Section 47.2] – hence our use of the term ‘factor graph’ for H.