2.1 Connecting the spectra of A and B

Understanding the spectrum of B turns out to be a challenging question. A useful result in this direction is the following theorem proved by [Reference Bass6], and subsequently in [Reference Watanabe and Fukumizu63] and [Reference Kotani and Sunada40]; see also Theorem 3.3 in [Reference Angel, Friedman and Hoory3].

Theorem 2.1 (Ihara–Bass formula). Let $G=(V,E)$ be any finite graph and B be its non-backtracking matrix. Then

\begin{equation*}\textrm{det}(B-\lambda I) = \left(\lambda^2-1\right)^{|E|-|V|} \textrm{det}\left(D-\lambda A+\lambda^2 I\right),\end{equation*}

where D is the diagonal matrix with $D_{vv}=d_v - 1$ and A is the adjacency matrix of G.

We use the Ihara–Bass formula to analyse the relationship of the spectrum of B to the spectrum of A in the case of a bipartite biregular graph. It will turn out that this relationship can be completely unpacked. From Theorem 2.1, we get that

\begin{equation*}\sigma(B)=\{\pm 1\}\bigcup\{\lambda: D-\lambda A+\lambda^2 I\ \textrm{is not invertible}\}.\end{equation*}

Note that there are precisely $2(m+n)$ eigenvalues of B that are determined by A, and that $\lambda = 0$ is not in the spectrum of B, since the graph has no isolated vertices ( $\det(D) \neq 0$ ).

We use the special structure of G to get a more precise description of $\sigma(B)$ . The matrices A and D are equal to:

\begin{equation*}A=\left(\begin{array}{c@{\quad}c} 0 & X \\ X^* & 0 \end{array}\right ),\ \ D=\left(\begin{array}{c@{\quad}c} (d_1-1)I_n & 0 \\0 & (d_2-1)I_m \end{array}\right ),\end{equation*}

where $I_k$ is the $k\times k$ identity matrix. Let $\lambda\in \sigma(B)\backslash\{-1,1\}$ . Then there exists a nonzero vector v such that

\begin{equation*}(D-\lambda A+\lambda^2 I)v = 0 .\end{equation*}

Writing $v=(v_1,v_2)$ with $v_1\in \mathbb{C}^n$ , $v_2\in \mathbb{C}^m$ , we obtain:

(4) \begin{eqnarray}Xv_2 & =& \frac{d_1-1+\lambda^2}{\lambda}v_1, \end{eqnarray}

(5) \begin{eqnarray}X^*v_1 & = &\frac{d_2-1+\lambda^2}{\lambda}v_2 .\end{eqnarray}

The above imply that, provided that the right-hand side is non-zero,

(6) \begin{eqnarray}\xi^2=\frac{(d_1-1+\lambda^2)(d_2-1+\lambda^2)}{\lambda^2}\end{eqnarray}

is a nonzero eigenvalue of both $XX^*$ with eigenvector $v_1$ and $X^*X$ , with eigenvector $v_2$ . We can rewrite equation (6) as

(7) \begin{eqnarray}\lambda^4 - (\xi^2 - d_1 - d_2+2) \lambda^2 + (d_1-1)(d_2-1) = 0 .\end{eqnarray}

We will now detail how the eigenvalues of A (denoted $\xi$ here) map to eigenvalues of B and vice-versa. Let us examine the special case $\xi = 0$ . Assume $n \leq m$ for simplicity. Assume that the rank of X is r. Then X has $m-r$ independent vectors in its nullspace. Let u be one such vector. Now, if we pick $v_2 = u$ , $v_1 = 0$ , and $\lambda = \pm i \sqrt{d_2-1}$ , equations (4) and (5) are satisfied. Hence, $\pm i \sqrt{d_2-1}$ are eigenvalues of B, both with multiplicity $m-r$ .

Since the rank of X is r, it follows that the nullity of $X^*$ is $n-r$ , so there are $n-r$ independent vectors w for which $X^*w = 0$ . Now, note that picking $v_1 = w$ , $v_2 = 0$ , and $\lambda = \pm i \sqrt{d_1 -1}$ , we satisfy equations (4) and (5). Thus, $\pm i \sqrt{d_1-1}$ are eigenvalues of B, both with multiplicity $n-r$ .

The remaining 4r eigenvalues of B determined by A come from nonzero eigenvalues of A. For each $\xi^2$ with $\xi$ a nonzero eigenvalue of A, we will have precisely 4 complex solutions to equation (6). Since there are 2r such eigenvalues, coming in pairs $\pm \xi$ , they determine a total of 4r eigenvalues of B, and the count is complete. To summarise the discussion above, we have the following Lemma:

4.1. The configuration model

The configuration or permutation model is a practical procedure to sample random graphs with a given degree distribution. Let us recall its definition for bipartite biregular graphs. Let $V_1=\{v_1,v_2,\dots,v_n\}$ and $V_2=\{w_1,w_2,\dots, w_m\}$ be the vertices of the graph. We define the set of half edges out of $V_1$ to be the collection of ordered pairs

\begin{equation*}\vec{E}_1=\{(v_i,j) \mbox{for $1\leq i\leq n$ and $1\leq j\leq d_1$}\}\end{equation*}

and analogously the set of half edges out of $V_2$ :

\begin{equation*}\vec{E}_2=\{(w_i,j) \mbox{for $1\leq i\leq m$ and $1\leq j\leq d_2$}\},\end{equation*}

see Figure 1. Note that $|\vec{E}_1|=|\vec{E}_2|=nd_1=md_2 = |E|$ . To sample a graph, we choose a random permutation $\pi$ of $[n d_1]$ . We put an edge between $v_i$ and $w_j$ in the G whenever

\begin{equation*}\pi((i-1)d_1+s)=(j-1)d_2+t\end{equation*}

for any pair of values $1\leq s\leq d_1$ , $1\leq t\leq d_2$ . For specific half edges $e = (v_i, j)$ and $f = (w_s, t)$ , we use the notation $\pi(e) = f$ as shorthand for $\pi((i-1)d_1 + j) = (s-1)d_2 + t$ and say that “e matches to f.”

The graph obtained may not be simple, since multiple half edges may be matched between any pair of vertices. However, conditioning on a simple graph outcome, the distribution is uniform in the set of all simple bipartite biregular graphs. Furthermore, for fixed $d_1, d_2$ and $n,m \to \infty$ , the probability of getting a simple graph is bounded away from zero [Reference Bollobás11].

Consider the random $B \in \mathbb{R}^{2 |E|\times 2 |E|}$ whose first $|E|$ rows are indexed by the elements of $\vec{E}_1$ and the last $|E|$ rows are indexed by those of $\vec{E}_2$ , in lexicographic order. Columns are indexed in the same way. Entry $B_{ef}$ with $e=(v_i,j) \in \vec{E}_1$ and $f=(w_s,t) \in \vec{E}_2$ is defined as

\begin{equation*}B_{ef}=\begin{cases} 1,& \mbox{if $\pi(e) = f' = (w_s, t')$ and $t'\neq t$};\\ 0,& \textrm{otherwise}.\end{cases}\end{equation*}

This defines the upper half of B. We define the lower half similarly, by putting

\begin{equation*}B_{fe}=\begin{cases} 1,&\textrm{if $\pi^{-1}(f) = e' = (v_i, j')$ and $j'\neq j$};\\ 0,& \textrm{otherwise}.\end{cases}\end{equation*}

This is the same definition used in [Reference Bordenave12]. In words, it says that the directed edge given by e followed by the directed edge given by f are connected by some half edge $f' = (w_s, t')$ , and the path they form does not backtrack. This is therefore the same matrix introduced in Section 2, ordered according to the half edges. Notice that the randomness comes from the matching only.

We consider two symmetric matrices $M = M(\pi)$ and N, indexed the same as B, and defined by:

\begin{equation*}M_{ef} = 1_{ \{ \pi(e) = f \} }\mbox{ and }M_{fe} = 1_{ \{ \pi^{-1} (f) = e \} } \mbox{ for $e\in \vec{E}_1$ and $f\in \vec{E}_2$}\end{equation*}

and

\begin{equation*}N_{gh} = 1_{ \{u = v \mbox{ and } i\neq j \}} \mbox{ for $g = (u,i), h = (v,j) \in \vec{E}_1 \cup \vec{E}_2$} .\end{equation*}

We see that a term like $M_{eg} N_{gf}$ corresponds to M matching the directed edge e to g by $\pi$ , and N taking us out of the vertex of g along the directed edge f, which is different from g. Thus, the rule of matrix multiplication means that

(9) \begin{align}B=MN.\end{align}

This equality will be useful in Section 5.2 when working with products of the matrix B.

4.2. Tangle-free paths

Sparse random graphs, including bipartite graphs, have the important property of being ‘tree-like’ in the neighbourhood of a typical vertex. Formally, consider a vertex $v \in V_1 \cup V_2$ . For a natural number $\ell$ , we define the ball of radius $\ell$ centered at v to be:

\begin{equation*}B_{\ell}(v) = \{w \in V_1 \cup V_2 : d_{G}(v,w) \leq \ell\}\end{equation*}

where $d_G(\cdot ,\cdot )$ is the graph distance.

The next lemma says that most bipartite biregular graphs are $\ell$ -tangle-free up to logarithmic sized neighbourhoods.

Proof. This is essentially the proof given in [Reference Lubetzky and Sly47], Lemma 2.1. Fix a vertex v. We will use the so called exploration process to discover the ball $B_{\ell}(v)$ . More precisely, we order the set $\vec{E}_1$ lexicographically: $(v_i,j) < (v_{i'},j')$ if $i\leq i'$ and $j\leq j'$ . The exploration process reveals $\pi$ one edge at the time, by doing the following:

• A uniform element is chosen from $\vec{E}_2$ , and it is declared equal to $\pi(1)$ .

• A second element is chosen uniformly, now from the set $\vec{E}_2\backslash\{\pi(1)\}$ and set equal to $\pi(2)$ .

• Once we have determined $\pi(i)$ for $i\leq k$ , we set $\pi(k+1)$ equal to a uniform element sampled from the set $\vec{E}_2\backslash\{\pi(1),\pi(2),\dots,\pi(k)\}$ .

We use the final $\pi$ to output a graph as we did in the configuration model. The law of these graphs is the same. With the exploration process, we expose first the neighbours of v, then the neighbours of these vertices, and so on. This breadth-first search reveals all vertices in $B_k (v)$ before any vertices in $B_{j > k} (v)$ . Note that, although our bound is for the family $\mathcal{G}(n,m,d_1,d_2)$ , the neighbourhood sizes are bounded above by those of the d-regular graph with $d=\max(d_1,d_2)$ .

Consider the matching of half edges attached to vertices in the ball $B_i(v)$ at depth i (thus revealing vertices at depth $i+1$ ). In this process, we match a maximum $m_i \leq d^{i+1}$ pairs of half edges total. Let $\mathcal{F}_{i,k}$ be the filtration generated by matching up to the kth half edge in $B_i(v)$ , for $1 \leq k \leq m_i$ . Denote by $A_{i,k}$ the event that the kth matching creates a cycle at the current depth. For this to happen, the matched vertex must have appeared among the $k-1$ vertices already revealed at depth $i+1$ . The number of unmatched half edges is at least $nd - 2 d^{i+1}$ . We then have that:

\begin{equation*}\mathbb{P}(A_{i,k})\leq \frac{(k-1) (d-1)}{nd - 2 d^{i+1}}\leq \frac{(d-1) m_i}{(1 - 2 d^{i+1} n^{-1}) nd}\leq \frac{m_i}{n}.\end{equation*}

So, we can stochastically dominate the sum

\begin{equation*}\sum_{i=1}^{\ell-1}\sum_{k=1}^{m_i} A_{i,k}\end{equation*}

by $Z \sim \textrm{Bin} \left( d^{\ell+1},\ n^{-1} d^{\ell} \right)$ . So the probability that $B_{\ell}(v)$ is $\ell$ -tangle-free has the bound:

\begin{equation*}\mathbb{P} ( \mbox{$B_{\ell}(v)$ is not $\ell$-tangle-free} ) =\mathbb{P} \left( \sum_{i=1}^{\ell-1}\sum_{k=1}^{m_i} A_{i,k}> 1 \right) \leq \mathbb{P} (Z>1) =O\left( \frac{d^{4\ell+1}}{n^2} \right) =O\left( n^{-3/2} \right) ,\end{equation*}

which follows using that $\ell = c \log_d n$ with $c < 1/8$ . The Lemma follows by taking a union bound over all vertices.

5.1. Outline

We are now prepared to explain the main result. To study the second largest eigenvalue of the non-backtracking matrix, we examine the spectral radius of the matrix obtained by subtracting off the dominant eigenspace. We use for this:

Lemma 5.1 ([Reference Bordenave, Lelarge and Massoulié14], Lemma 3). Let T and R be matrices such that $\textrm{Im}(T)\subset \textrm{Ker}(R)$ , $\textrm{Im}(T^*)\subset \textrm{Ker}(R)$ . Then all eigenvalues $\lambda$ of $T+R$ that are not eigenvalues of T satisfy:

\begin{equation*}|\lambda|\leq \max_{x\in \textrm{Ker}(T)}\frac{\|(T+R)x\|}{\|x\|} .\end{equation*}

Throughout the text, $\| \cdot \|$ is the spectral norm for matrices and $\ell^2$ -norm for vectors. Recall that the leading eigenvalues of B, in magnitude, are $\lambda_1 = \sqrt{(d_1 -1)(d_2-1)}$ and $\lambda_{2|E|} = -\lambda_1$ with corresponding eigenvectors $\textbf{1}_{\alpha}$ and $\textbf{1}_{-\alpha}$ . Applying Lemma 5.1 with $T=\frac{\lambda_1^{\ell}}{\textbf{1}_{\alpha}^* \textbf{1}_{\alpha}}(\textbf{1}_{\alpha} \textbf{1}_{\alpha}^* +(-1)^\ell \textbf{1}_{-\alpha} \textbf{1}_{-\alpha}^*)$ and $R = B^\ell - T$ , we get that

(10) \begin{align}\lambda_2(B)\leq \max_{\begin{array}{c}x \in \textrm{Ker}(T) \\\|x\| = 1\end{array}}\left( \| B^{\ell}x \| \right)^{1/\ell}.\end{align}

It will be important later to have a more precise description of the set $\textrm{Ker}(T)$ . It is not hard to check that

\begin{align*}\textrm{Ker}(T) &= \{x: \langle x, \textbf{1}_{\alpha} \rangle = \langle x, \textbf{1}_{-\alpha} \rangle=0\} \\& =\{(v,w)\in \mathbb{R}^{2|E|}: \langle v, \textbf{1} \rangle = \langle w, \textbf{1} \rangle=0\}.\end{align*}

In the last line, the vectors v, w and $\textbf{1}$ are $|E|-$ dimensional, and $\textbf{1}$ is the vector of all ones.

In order to use equation (10), we must bound $\| B^\ell x \|$ for large powers $\ell$ and $x \in \textrm{Ker}(T)$ . This amounts to counting the contributions of certain non-backtracking walks. We will use the tangle-free property in order to only count $\ell$ -tangle-free walks. We break up $B^\ell$ into two parts in Section 5.2, an ‘almost’ centered matrix $\bar{B}^\ell$ and the remainder $\sum_j R^{\ell,j}$ and we bound each term independently.

To compute these bounds, we need to count the contributions of many different non-backtracking walks. We will use the trace technique, so only circuits which return to the starting vertex will contribute. In Section 5.3, we compute the expected contribution of products of B along such circuits, employing a result from [Reference Bordenave12].

Section 5.4 covers the combinatorial component of the proof. The total contributions $\| B^\ell x \|$ come from many non-backtracking circuits of different flavours, depending on their number of vertices, edges, cycles, etc. Each circuit is broken up into 2k segments of tangle-free walks of length $\ell$ . We need to compute not only the expectation along the circuit, but also upper-bound the number of circuits of each flavor. We introduce an injective encoding of such circuits that depends on the number of vertices, length of the circuit, and, crucially, the tree excess of the circuit. An important part of these calculations is to keep track of the imbalance between left and right vertices visited in the circuit, since this controls the powers of $d_1$ and $d_2$ in the result.

Finally, in Section 5.8, we put all of these ingredients together and use Markov’s inequality to bound each matrix norm with high probability. We find that $\| \bar{B}^\ell \|$ contributes a factor that goes as $((d_1 - 1)(d_2 - 1))^{\ell/4}$ , whereas $\| R^{\ell,j} \|$ contributes only a factor of $(d-1)^\ell/n$ , up to polylogarithmic factors in n. Thus, the main contribution to the circuit counts comes from the mean and, in fact, comes from circuits which are exactly trees traversed forwards and backwards. Interestingly, this is analogous to what happens when using the trace method on random matrices of independent entries.

In the proof, we are forced to consider tangled paths but which are built up of tangle-free components. This delicate issue was first made clear by [Reference Friedman28] who introduced the idea of tangles and a ‘selective trace.’ Bordenave et al. [Reference Bordenave, Lelarge and Massoulié14], who we follow closely in this part of our analysis, also has a good discussion of these issues and their history. We use the fact that

(11) \begin{equation}\mathbb{E}\left(\|\bar{B}^{\ell}\|^{2k}\right)\leq \mathbb{E}\left(\textrm{Tr}\left[\left( (\bar{B}^{\ell}) (\bar{B}^{\ell})^* \right)^{k}\right]\right),\end{equation}

and so deal with circuits built up of 2k segments which are $\ell$ -tangle-free. Notice that the first segment comes from $\bar{B}^\ell$ , the second from $(\bar{B}^{\ell})^*$ , etc. Because of this, the directionality of the edges along each segment alternates. See Figure 4 for an illustration of a path which contributes for $k=2$ and $\ell=2$ . Also, while each segment is $\ell$ -tangle-free, the overall circuit may be tangled.

Figure 4. An example circuit that contributes to the trace in equation (11), for $k = 2$ and $\ell = 2$ . Edges are numbered as they occur in the circuit. Each segment $\{ \gamma_i \}_{i=1}^4$ is of length $\ell + 1 = 3$ and made up of edges $3(i-1)+1$ through 3i. The last edge of each $\gamma_i$ is the first edge of $\gamma_{i+1}$ , and these are shown in purple. Every path $\gamma_{i}$ with i even follows the edges backwards due to the matrix transpose. However, this detail turns out not to make any difference since the underlying graph is undirected. Our example has no cycles in each segment for clarity, but, in general, each segment can have up to one cycle, and the overall circuit may be tangled.

5.2. Matrix decomposition

We start this section by defining the set of paths that will be relevant to bound the norm of $\|\bar{B}^{\ell}\|$ . We closely follow [Reference Bordenave12].

Each path in $\Gamma^\ell_{ef}$ uses $2 \ell + 1$ half edges, corresponding to $\ell + 1$ edges in the graph. To be clear, the above definition counts all possible non-backtracking sequences of half edges. These are different than the usual non-backtracking paths and do not necessarily exist in the graph. Some of these paths might backtrack along a duplicate edge which utilizes a different half edge.

We now have

(12) \begin{equation}(B^{\ell})_{ef} =\sum_{\gamma \in \Gamma^\ell_{ef}}\prod_{t=1}^{\ell} B_{e_{2t-1} e_{2t+1}}=\sum_{\gamma \in \Gamma^\ell_{ef}}\prod_{t=1}^{\ell} M_{e_{2t-1} e_{2t}}N_{e_{2t}e_{2t+1}}=\sum_{\gamma \in \Gamma^\ell_{ef}}\prod_{t=1}^{\ell} M_{e_{2t-1} e_{2t}}\end{equation}

where we used equation (9) and the fact that $\Gamma^{\ell}_{ef}$ is non-backtracking, so $N_{e_{2t}e_{2t+1}}=1$ .

Recall that we will use equation (10) and Lemma 5.1 to bound $\lambda_2$ . Denote by $\bar{B}$ the matrix with entries equal to $\bar{B} = B - S$ , where

\begin{equation*} S = \frac{1}{|E|} \left( \begin{array}{c@{\quad}c} 0 & (d_2 - 1) \textbf{1} \textbf{1}^* \\[4pt] (d_1 - 1) \textbf{1} \textbf{1}^* & 0 \end{array} \right ).\end{equation*}

Note that $\bar{B}$ is an almost centred version of B, and $\textrm{Ker} ( S ) = \textrm{Ker}(T) = \textrm{span} (\textbf{1}_{\alpha}, \textbf{1}_{-\alpha} ) $ , where T is the matrix from Lemma 5.1. To apply the lemma, we wish to get an expression like equation (12) for $\bar{B}^{\ell}$ . To do so, we write:

\begin{equation*}\bar{B} = B - S = M N - S = (M - S') N\end{equation*}

the above matrix equation in the unknown S’ can be solved by simple manipulations. We get

\begin{equation*}S'=\frac{1}{|E|} \left( \begin{array}{c@{\quad}c} 0 & \textbf{1} \textbf{1}^* \\[4pt] \textbf{1} \textbf{1}^* & 0 \end{array} \right ).\end{equation*}

Using again that N is identically one over the elements of the set $\Gamma^{\ell}_{ef}$ , we find a similar formula to equation (12):

(13) \begin{align}(\bar{B}^\ell)_{ef}=\sum_{\gamma \in \Gamma^\ell_{ef}}\prod_{t=1}^{\ell}\left( B - S \right)_{e_{2t-1} e_{2t+1}}=\sum_{\gamma \in \Gamma^\ell_{ef}}\prod_{t=1}^{\ell}\bar{M}_{e_{2t-1} e_{2t}},\end{align}

where $\bar{M}=M-S'$ .

The following telescoping sum formula is a simple algebraic manipulation and appears in [Reference Massoulie50] and [Reference Bordenave, Lelarge and Massoulié14]:

\begin{equation*}\prod_{s=1}^\ell x_s = \prod_{s=1}^\ell y_s +\sum_{j=1}^\ell \prod_{s=1}^{j-1} y_s (x_j - y_j) \prod_{t=j+1}^\ell x_t .\end{equation*}

Using this, with $x_s = B_{e_{2s-1} e_{2s+1}}$ and $y_s = \bar{B}_{e_{2s-1} e_{2s+1}}$ , we obtain the following relation:

(14) \begin{align}( B^{\ell} )_{ef}=( \bar{B}^{\ell} )_{ef}+\sum_{\gamma\in \Gamma^{\ell}_{ef}}\sum_{j=1}^{\ell}\prod_{s=1}^{j-1} \bar{B}_{e_{2s-1} e_{2s+1}}S_{e_{2j-1} e_{2j+1}}\prod_{t=j+1}^{\ell} B_{e_{2t-1} e_{2t+1}}.\end{align}

This decomposition breaks the elements in $\Gamma^{\ell}_{ef}$ into two subpaths, also non-backtracking, of length j and $\ell-j$ , respectively.

We will take the parameter $\ell$ to be small enough so that the path $\gamma$ is tangle-free with high probability. Thus, the sums in equations (12) or (13) need only be over the paths $\gamma \in F_{ef}^\ell$ . However, to recover the matrices B and $\bar{B}$ by rearranging equation (14), we need to also count those tangle-free subpaths that arise from splitting tangled paths. While breaking a tangle-free path will necessarily give us two new tangle-free subpaths, the converse is not always true. This extra term generates a remainder that we define now.

Set the remainder

(15) \begin{align} R^{\ell,j}_{ef} &= \sum_{\gamma\in T^{\ell,j}_{ef}} \sum_{j=1}^{\ell} \prod_{s=1}^{j-1} \bar{B}_{e_{2s-1} e_{2s+1}} S_{e_{2j-1} e_{2j+1}} \prod_{t=j+1}^{\ell} B_{e_{2t-1} e_{2t+1}} \nonumber \\ &= \sum_{\gamma\in T^{\ell,j}_{ef}} \sum_{j=1}^{\ell} \prod_{s=1}^{j-1}\ \bar{M}_{e_{2s-1} e_{2s}} S_{e_{2j-1} e_{2j+1}} \prod_{t=j+1}^{\ell} M_{e_{2t-1} e_{2t}}.\end{align}

Since the paths are non-backtracking, the N terms are all unity.

Adding and subtracting $\sum_{j=1}^{\ell} R^{\ell,j}_{ef}$ to equation (14) and rearranging the sums, we obtain

(16) \begin{align}B^{(\ell)} =\bar{B}^{(\ell)} +\sum_{j=1}^{\ell}\bar{B}^{(j)} S B^{(\ell-j)}-\sum_{j=1}^{\ell} R^{\ell,j},\end{align}

where the matrices $B^{(\ell)}$ and $\bar{B}^{(\ell)}$ are tangle-free versions of $B^\ell$ and $\bar{B}^\ell$ , i.e. element ef in both matrices only counts paths $\gamma \in F_{ef}^\ell$ . Multiplying equation (16) on the right by $x \in \textrm{Ker} (T)$ and using that $B^{(\ell - j)} x$ is also within $\textrm{Ker}(S)$ , since it is just the space spanned by the leading eigenvectors, we find that the middle term is identically zero. Thus, for $x \in \textrm{Ker}(T)$ ,

(17) \begin{equation}\| B^{(\ell)}x \|\leq \| \bar{B}^{(\ell)} x \|+\left\|\sum_{j=1}^{\ell}R^{\ell,j} x\right\|\leq \| \bar{B}^{(\ell)} \|+\sum_{j=1}^{\ell} \| R^{\ell,j} \| .\end{equation}

5.3 Expectation bounds

Our goal is to find a bound on the expectation of certain random variables which are products of $\bar{B}_{ef}$ along a circuit. To do this, we will need to bound the probabilities of different subgraphs when exploring G. This requires us to introduce the concept of consistent edges and their multiplicity.

Lemma 5.6. Let $\gamma = (e_1, \ldots, e_{2k})$ be a sequence of half edges of even length, with M and $\bar{M}$ the matching matrix and its centred version generated by a uniform matching in the configuration model. Then for $1 \leq k \leq \sqrt{|E|}$ and $0 \leq t_0 \leq k$ we have that

\begin{equation*}\left|\mathbb{E}\prod_{t=1}^{t_0} \bar{M}_{e_{2t - 1} e_{2t}}\prod_{t=t_0+1}^{k} M_{e_{2t - 1} e_{2t}}\right|\leq C \cdot 2^b \cdot \left( \frac{1}{|E|} \right)^\mathcal{E}\left( \frac{3 k}{\sqrt{|E|}} \right)^{\mathcal{E}_1}\end{equation*}

where $b = $ number of inconsistent edges of multiplicity one occurring before $t_0$ , $\mathcal{E}_1 = $ number of consistent edges with multiplicity one occurring before $t_0$ , $\mathcal{E} = |E(\gamma)|$ , and C is a universal constant.

Proof. Recall the form of the matrices

\begin{equation*} M = \left( \begin{array}{c@{\quad}c} 0 & M_1 \\[4pt] M_1^* & 0 \end{array} \right ) \quad \mbox{and} \quad \bar{M} = M - \frac{1}{|E|} \left( \begin{array}{cc} 0 & \textbf{1} \textbf{1}^* \\[4pt] \textbf{1} \textbf{1}^* & 0 \end{array} \right ).\end{equation*}

Matrix $M_1 \in \mathbb{R}^{|E| \times |E|}$ is a random permutation matrix between $n d_1 = |E|$ and $m d_2 = |E|$ half edges. Therefore, $M_1$ is distributed exactly the same as a matching matrix of a random $|E|$ -lift of a single edge, and the same holds for its centered version $M_1 - \frac{1}{|E|} \textbf{1} \textbf{1}^*$ . The only paths $\gamma$ that contribute in this bipartite setting must alternate between the bipartite sets and avoid the 0 blocks, otherwise the bound holds trivially. For one of these paths $\gamma$ assume, without loss of generality, that the path starts in set $V_1$ . Then define the transformed path $\gamma' = (e^{\prime}_1, \ldots, e^{\prime}_{2k}) = (e_1, e_2, e_4, e_3, e_5, \ldots)$ , i.e. with every other pair in $\gamma$ in reverse order. Note that

(18) \begin{equation}\prod_{t=1}^{t_0} \bar{M}_{e_{2t - 1} e_{2t}}\prod_{t=t_0+1}^{k} M_{e_{2t - 1} e_{2t}}=\prod_{t=1}^{t_0} (\bar{M}_1)_{e^{\prime}_{2t - 1} e^{\prime}_{2t}}\prod_{t=t_0+1}^{k} (M_1)_{e^{\prime}_{2t - 1} e^{\prime}_{2t}} .\end{equation}

Then the Lemma holds by [Reference Bordenave12], Proposition 28.

5.4. Path counting

This section is devoted to counting the number of ways non-backtracking walks can be concatenated to obtain a circuit as in Section 5.2. We will follow closely the combinatorial analysis used in [Reference Brito, Dumitriu, Ganguly, Hoffman and Tran16]. In that paper, the authors needed a similar count for self-avoiding walks. We make the necessary adjustments to our current scenario.

Our goal is to find a reasonable bound for the number of circuits that contribute to the trace bound, equation (11) and shown graphically in Figure 4. Define $\mathcal{C}_{\mathcal{V}, \mathcal{E}}^\mathcal{R}$ as those circuits which visit exactly $\mathcal{V} = |V(\gamma)|$ different vertices, $\mathcal{R} = |V(\gamma) \cap V_2|$ of them in the right set, and $\mathcal{E} = |E(\gamma)|$ different edges. Note, these are undirected edges in E(G). This is a set of circuits of length $2 k \ell$ obtained as the concatenation of 2k non-backtracking, tangle-free walks of length $\ell$ . We denote such a circuit as $\gamma= ( \gamma_1,\gamma_2,\cdots, \gamma_{2k} )$ , where each $\gamma_j$ is a length $\ell$ walk.

To bound $C_{\mathcal{V}, \mathcal{E}}^\mathcal{R} = | \mathcal{C}_{\mathcal{V}, \mathcal{E}}^\mathcal{R} |$ , we will first choose the set of vertices and order them. The circuits that contribute are indeed directed non-backtracking walks. However, by considering undirected walks along a fixed ordering of vertices, that ordering sets the orientation of the first and thus the rest of the directed edges in $\gamma$ . Thus, we are counting the directed walks which contribute to equation (11). We relabel the vertices as $1,2, \ldots, \mathcal{V}$ as they appear in $\gamma$ . Denote by $\mathcal{T}_{\gamma}$ the spanning tree of those edges leading to new vertices as induced by the path $\gamma$ . The enumeration of the vertices tells us how we traverse the circuit and thus defines $\mathcal{T}_{\gamma}$ uniquely.

We encode each walk $\gamma_j$ by dividing it into sequences of subpaths of three types, which in our convention must always occur as type 1 $\to$ type 2 $\to$ type 3, although some may be empty subpaths. Each type of subpath is encoded with a number, and we use the encoding to upper bound the number of such paths that can occur. Given our current position on the circuit, i.e. the label of the current vertex, and the subtree of $\mathcal{T}_{\gamma}$ already discovered (over the whole circuit $\gamma$ not just the current walk $\gamma_j$ ), we define the types and their encodings:

1. Type 1: These are paths with the property that all of their edges are edges of $\mathcal{T}_{\gamma}$ and have been traversed already in the circuit. These paths can be encoded by their end vertex. Because this is a path contained in a tree, there is a unique path connecting its initial and final vertex. We use 0 if the path is empty.

2. Type 2: These are paths with all of their edges in $\mathcal{T}_{\gamma}$ but which are traversed for the first time in the circuit. We can encode these paths by their length, since they are traversing new edges, and we know in what order the vertices are discovered. We use 0 if the path is empty.

3. Type 3: These paths are simply a single edge, not belonging to $\mathcal{T}_\gamma$ , that connects the end of a path of type 1 or 2 to a vertex that has been already discovered. Given our position on the circuit, we can encode an edge by its final vertex. Again, we use 0 if the path is empty.

Now, we decompose $\gamma_j$ into an ordered sequence of triples to encode its subpaths:

\begin{equation*}(p_1,q_1,r_1) (p_2,q_2,r_2) \cdots (p_t,q_t,r_t),\end{equation*}

where each $p_i$ characterises subpaths of type 1, $q_i$ characterises subpaths of type 2 and $r_i$ characterizes subpaths of type 3. These subpaths occur in the order given by the triples. We perform this decomposition using the minimal possible number of triples.

Now, $p_i$ and $r_i$ are both numbers in $\{0,1,...,\mathcal{V} \}$ , since our cycle has $\mathcal{V}$ vertices. On the other hand, $q_i \in \{ 0,1,...,\ell \}$ since it represents the length of a subpath of a non-backtracking walk of length $\ell$ . Hence, there are $(\mathcal{V} + 1)^2 (\ell+1)$ possible triples. Next, we want to bound how many of these triples occur in $\gamma_j$ . We will use the following lemma.

Proof. We can check this case by case. Assume that for some $i<t$ we have $(p_i,q_i,0)$ and consider the concatenation with $(p_{i+1},q_{i+1},r_{i+1})$ . First, notice that both $p_{i+1}$ and $q_{i+1}$ cannot be zero since then we will have $(p_i,q_i,0)(0,0,v^*)$ which can be written as $(p_i,q_i,v^*)$ . If $q_i\neq 0$ , then we must have $p_{i+1} \neq 0$ . Otherwise, we split a path of new edges (type 2), and the decomposition is not minimal. This implies that we visit new edges and move to edges already visited, hence we need to go through a type 3 edge, implying that $r_i \neq 0$ . Finally, if $p_i \neq 0$ and $q_i = 0$ , then we must have $p_{i+1}=0$ ; otherwise, we split a path of old edges (type 1). We also require $q_{i+1} \neq 0$ , but $(p_i,0,0)(0,q_{i+1},r_{i+1})$ is the same as $(p_i,q_{i+1},r_{i+1})$ , which contradicts the minimality condition. This covers all possibilities and finishes the proof.

Using the lemma, any encoding of a non-backtracking walk $\gamma_j$ has at most one triple with $r_i=0$ . All other triples indicate the traversing of a type 3 edge. We now give a very rough upper bound for how many of such encodings there can be. To do so, we will use the tangle-free property and slightly modify the encoding of the paths with cycles. Consider the two cases:

1. Case 1: Path $\gamma_j$ contains no cycle. This implies that we traverse each edge within $\gamma_j$ once. Thus, we can have at most $\chi= \mathcal{E} - \mathcal{V} + 1$ many triples with $r_i\neq 0$ . This gives a total of at most

   \begin{equation*} \left((\mathcal{V}+1)^2 (\ell+1) \right)^{\chi+1}\end{equation*}
   many ways to encode one of these paths.

2. Case 2: Path $\gamma_j$ contains a cycle. Since we are dealing with non-backtracking, tangle-free walks, we enter the cycle once, loop around some number of times, and never come back. We change the encoding of such paths slightly. Let $\gamma_j^{a}$ , $\gamma_j^{b}$ , and $\gamma_j^{c}$ be the segments of the path before, during, and after the cycle. We mark the start of the cycle with $|$ and its end with $\|$ . The new encoding of the path is

   \begin{equation*}(p^a_1,q^a_1,r^a_1) \cdots (p^a_{t^a},q^a_{t^a},r^a_{t^a})\, | \,(p^b_1,q^b_1,r^b_1) \cdots (p^b_{t^b},q^b_{t^b},r^b_{t^b})\, \| \,(p^c_1,q^c_1,r^c_1) \cdots (p^c_{t^c},q^c_{t^c},r^c_{t^c}),\end{equation*}
   where we encode the segments separately. Observe that each a subpath is connected and self-avoiding. The above encoding tells us all we need to traverse $\gamma_j$ , including how many times to loop around the cycle: since the total length is $\ell$ , we can back out the number of circuits around the cycle from the lengths of $\gamma_j^{a}$ , $\gamma_j^{b}$ , and $\gamma_j^{c}$ , see Figure 5. Following the analysis made for Case 1, the subpaths $\gamma_j^{a}$ , $\gamma_j^{b}$ , $\gamma_j^{c}$ are encoded by at most $\chi + 1$ triples, but we also have at most $\ell$ choices each for our marks $|$ and $\|$ . We are left with at most
   \begin{equation*} \ell^2 \left((\mathcal{V}+1)^2 (\ell+1) \right)^{\chi+1}\end{equation*}
   ways to encode any path of this kind.

Figure 5. Encoding an $\ell$ -tangle-free walk, in this case the first walk in the circuit $\gamma_1$ , when it contains a cycle. The vertices and edges are labeled in the order of their traversal. The segments $\gamma^a$ , $\gamma^b$ , and $\gamma^c$ occur on edges numbered (1, 2, 3); $(4 + 6i, 5+6i, 6+6i, 7+6i, 8+6i, 9+6i)$ for $i = 0, 1, \ldots c$ ; and $(10+6c)$ , respectively. The encoding is $(0,3,0) | (0,4,3)(4,0,0) \| (0,1,0)$ . Suppose $c = 1$ . Then $\ell = 22$ and the encoding is of length $3 + (4+1+1)(c+1) + 1$ , we can back out c to find that the cycle is repeated twice. The encodings become more complicated later in the circuit as vertices see repeat visits.

Together, these two cases mean there are less than $2 \ell^2 \left((\mathcal{V}+1)^2 (\ell+1) \right)^{\chi+1}$ such paths.

Now we conclude by encoding the entire circuit $\gamma = (\gamma_1, \ldots, \gamma_{2k})$ . We first choose $\mathcal{V}$ vertices, $\mathcal{R}$ in the set $V_2$ , and order them, which can occur in $(m)_\mathcal{R} (n)_{\mathcal{V}-\mathcal{R}} \leq m^\mathcal{R} n^{\mathcal{V} - \mathcal{R}}$ different ways. Finally, in the whole path $\gamma$ we are counting concatenations of 2k paths which are $\ell$ -tangle-free. Therefore, we conclude with the following Lemma:

Lemma 5.8. Let $\mathcal{C}_{\mathcal{V}, \mathcal{E}}^\mathcal{R}$ be the set of circuits $\gamma = (\gamma_1, \ldots, \gamma_{2k})$ of length $2 k \ell$ obtained as the concatenation of 2k non-backtracking, tangle-free walks of length $\ell$ , i.e. $\gamma_s \in F^\ell$ for all $s \in [2k]$ , which visit exactly $\mathcal{V} = |V(\gamma)|$ different vertices, $\mathcal{R} = |V(\gamma) \cap V_2|$ of them in the right set, and $\mathcal{E} = |E(\gamma)|$ different edges. If $C_{\mathcal{V}, \mathcal{E}}^\mathcal{R} = |\mathcal{C}_{\mathcal{V}, \mathcal{E}}^\mathcal{R}|$ , then

(19) \begin{equation}C_{\mathcal{V}, \mathcal{E}}^\mathcal{R}\leq m^\mathcal{R} n^{\mathcal{V} - \mathcal{R}} (2 \ell)^{4k}\left( (\mathcal{V}+1)^2 (\ell+1) \right)^{2k (\chi + 1)},\end{equation}

where $\chi = \mathcal{E} - \mathcal{V} + 1$ .

The circuits that contribute to the remainder term $R^{\ell, j}$ are slightly different. In this case, each length $\ell$ segment is an element of $T^{\ell, j}$ rather than $F^{\ell}$ . We have to slightly modify the previous argument for this case.

Lemma 5.9. Let $\mathcal{D}_{\mathcal{V},\mathcal{E}}^\mathcal{R}$ be the set of circuits $\gamma = (\gamma_1, \ldots, \gamma_{2k})$ of length $2k\ell$ obtained as the concatenation of 2k elements $\gamma_s \in T^{\ell, j}$ for $s = 1, \ldots, 2k$ , that visit exactly $\mathcal{V}$ vertices, $\mathcal{R}$ of which are in $V_2$ , and $\mathcal{E}$ different edges. Then for $D_{\mathcal{V},\mathcal{E}}^\mathcal{R} = | \mathcal{D}_{\mathcal{V},\mathcal{E}}^\mathcal{R} |$ , we have

(20) \begin{align}D_{\mathcal{V},\mathcal{E}}^\mathcal{R}\leq m^\mathcal{R} n^{\mathcal{V}-\mathcal{R}}(2 \ell)^{6k}( (\mathcal{V} + 1)^2 (\ell + 1) )^{6k(\chi + 1)} .\end{align}

Proof. Since each $\gamma^s = (e_1, \ldots, e_{2\ell + 1}) \in T^{\ell,j}$ , we have that $\gamma' = (e_1, \ldots, e_{2j-1}) \in F^{j-1}$ , $\gamma'' = (e_{2j-1}, e_{2j}, e_{2j+1}) \in F^{1}$ , and $\gamma''' = (e_{2j+1}, \ldots, e_{2\ell + 1}) \in F^{\ell - j}$ . Encoding $\gamma'$ , $\gamma''$ , and $\gamma'''$ as before, we have the generous upper bound of at most

\begin{equation*}\left[ (2\ell) ((\mathcal{V} + 1)^2(\ell+1))^{\chi+1} \right]^3\end{equation*}

many encodings for each $\gamma_s$ . Choosing and ordering the vertices, then concatenating 2k of these paths gives the final result.

5.5. Half edge isomorphism counting

We have constructed the circuits in $\mathcal{C}_{\mathcal{V}, \mathcal{E}}^\mathcal{R}$ and $\mathcal{D}_{\mathcal{V}, \mathcal{E}}^\mathcal{R}$ by choosing the vertices and edges that participate in them. However, the expectation bound applies to matchings of half-edges in the configuration model. Since there are multiple ways to configure the half edges into such a circuit, this must be taken into account in the combinatorics.

Lemma 5.10. Let $I_{\mathcal{V}, \mathcal{E}}^\mathcal{R}$ be the number of half edge choices for the graph induced by $\gamma \in \mathcal{C}_{\mathcal{V}, \mathcal{E}}^\mathcal{R}\cup \mathcal{D}_{\mathcal{V}, \mathcal{E}}^\mathcal{R}$ . Then,

(21) \begin{equation} I_{\mathcal{V}, \mathcal{E}}^\mathcal{R} \leq d_1^{\mathcal{V} - \mathcal{R}} (d_1 - 1)^{\mathcal{E} - \mathcal{V} + \mathcal{R}} d_2^{\mathcal{R}} (d_2 - 1)^{\mathcal{E} - \mathcal{R}} . \end{equation}

Proof. For every left vertex v, with degree $g_v$ on the graph induced by $\gamma$ , the number of choices of half edges is $d_1(d_1-1)\dots(d_1- g_v+1)\leq d_1 (d_1-1)^{g_v-1}$ . Note that the choice of half edges are independent for the left vertices. We then get that there are $d_1^{\mathcal{V}-\mathcal{R}} (d_1-1)^{\mathcal{E} - \mathcal{V}+\mathcal{R}}$ many choices, where we used that the sum of all the degrees on one component of a bipartite graph equals the number of edges: $\mathcal{E} = \sum_v g_v$ . Similarly, for right vertices we get $d_2^{\mathcal{R}}(d_2-1)^{(\mathcal{E}-\mathcal{R})}$ .

Corollary 5.11. We have that

(22) \begin{equation} I_{\mathcal{V}, \mathcal{E}}^\mathcal{R} \leq (d_1 (d_1 - 1))^{\mathcal{V} - \mathcal{R}} (d_2 (d_2 - 1))^\mathcal{R} (d - 1)^{2 (\chi - 1)}.\end{equation}

5.6. Bounding the imbalance $\psi$

We focus now on the quantity defined as $\psi = \mathcal{R} - \mathcal{E} / 2 $ . Informally, $\psi$ captures the imbalance between the number of vertices on each partition of the bipartite graph visited by the circuit $\gamma$ . We show that this imbalance is not too large.

Proof. For any subgraph H define $\psi(H) = \mathcal{R}(H)-\mathcal{E}(H)/2$ . We set $\psi=\psi(\gamma)$ . To bound this quantity, we analyse the subgraph $\gamma_{\leq i}$ , obtained by the concatenation of the first i walks in $\gamma$ , i.e. the union of the graphs induced by $\gamma_1, \ldots, \gamma_i$ . Our choice of $\ell$ implies that every neighbourhood of radius $4\ell$ is tangle-free with high probability. Hence, every non-backtracking walk $\gamma_j$ is either a path or a path with exactly one loop. It is not hard to conclude that $\psi(\gamma_j) \leq 2$ for all j and $\psi(\gamma_{\leq 1}) = \psi(\gamma_1) \leq 2$ . We now proceed inductively to add walks to our graph, one by one, as they appear on the circuit. We will upper bound the increment $\psi(\gamma_{\leq i+1}) - \psi(\gamma_{\leq i})$ by looking at how the addition of $\gamma_{i+1}$ changes the imbalance.

To analyse this, consider the intersection of $\gamma_{i+1}$ and each $\gamma_j$ , $1\leq j\leq i$ . Notice that $\psi$ may increase only if there are vertices at which the two walks split apart. We claim that there are at most two such vertices. Assume that at $v_1,v_2$ and $v_3$ the two walks split. Then there are two disjoint cycles in the union of $\gamma_{i+1}$ and $\gamma_j$ , obtained by following each first from $v_1$ to $v_2$ and then from $v_2$ to $v_3$ . But this is a contradiction, since the diameter of this union is less than $2 \ell < \frac{1}{8} \log(n)$ , which implies that their union is tangle-free. We conclude that $\psi(\gamma_{i+1} \cup \gamma_j) \leq 8$ since there are at most two splits and each split contributes at most four to the imbalance. Then $\psi(\gamma_{\leq i+1}) \leq \psi(\gamma_{\leq i}) + 8i + 2$ , which implies that $\psi(\gamma)\leq 16k^2$ , as desired.

5.7. Bounding the inconsistent edges

We will need a bound on the number of inconsistent edges of multiplicity one, which we get in the following lemma. Recall Definition 5.5, which introduced inconsistent edges.

Proof. Let $\{e,f\}$ be an inconsistent edge of multiplicity one, where e and f are its half edges. For inconsistency and without loss of generality, there must exist another edge $\{e,f'\}$ in $\gamma$ , so that $m_\gamma (e) \neq 1$ . We may assume that $\{e,f\}$ is traversed before $\{e,f'\}$ . Let v be the vertex of e and consider the two possible scenarios:

1. Case 1: There is no cycle containing v in $\gamma$ . Then the edge $\{e,f\}$ may only be inconsistent if v is visited at the end of one of the 2k non-backtracking walks $\gamma_i$ and $\{e,f'\}$ is at the beginning of $\gamma_{i+1}$ . Hence, in this case, we have at most two inconsistent edges of multiplicity one. This yields at most 4k such edges.

2. Case 2: There is a cycle passing through v. For each such cycle, there is an edge that does not belong to the tree $T_{\gamma}$ (defined in Section 5.4). Furthermore, each cycle creates at most four inconsistent edges. Combining these two facts we get at most $4\chi$ and the proof follows.

Proof. The argument is similar to the above; however, now there are 4k segments in $\bar\gamma = \bigcup_{s=1}^{2k} (\gamma^{\prime}_s \cup \gamma^{\prime\prime}_s)$ , counting $\gamma^{\prime}_s$ and $\gamma^{\prime\prime}_s$ separately. As above, each of these 4k segments may yield at most 2 inconsistent edges. Furthermore, the graph induced by $\bar\gamma$ may not be connected; let C be the number of connected components. Each edge that creates a cycle may yield at most 4 inconsistent edges, and there are at most $\mathcal{E} - \mathcal{V} + C$ non-tree edges. Then, we have that

\begin{equation*}b_\mathcal{D}\leq 8 k + 4 (\mathcal{E} - \mathcal{V} + C)\leq 8k + 4 (\mathcal{E} - \mathcal{V} + 2k)\leq 16 k + 4 \chi,\end{equation*}

as claimed.

5.8. Bounds on the norm of $\bar{B}^{\ell}$ and $R^{\ell,j}$

All of the ingredients are gathered to bound the matrix norms.

Theorem 5.15. Let $\ell = \lfloor c\log(n) \rfloor$ where $c < \frac{1}{32}$ is a universal constant. It holds that

\begin{equation*}\|\bar{B}^{(\ell)}\|\leq \left((d_1-1)(d_2-1)\right)^{\ell/4}\exp( \log^{3/4} n )\end{equation*}

asymptotically almost surely.

Proof. The following holds for any natural number k, but for our proof, we will take

(23) \begin{equation} k = \lfloor \log (n)^{1/3} \rfloor \quad \mbox{ and } \quad \ell = \lfloor c \log n \rfloor \mbox{ for some } c < \frac{1}{32}.\end{equation}

We have

(24) \begin{align}\mathbb{E} \left(\| \bar{B}^{(\ell)} \|^{2k} \right)\leq \mathbb{E}\left(\textrm{Tr} \left[ \left( \bar{B}^{(\ell)} \bar{B}^{(\ell)^*} \right)^{k} \right]\right)=\mathbb{E}\left(\sum_{\gamma \in \mathcal{C}}\prod_{t=1}^{2 k \ell} \bar{B}_{e_{2t-1} e_{2t+1}}\right).\end{align}

The sum is taken over the set $\mathcal{C}$ of all circuits $\gamma$ of length $2k\ell$ , where $\gamma= ( \gamma_1, \gamma_2, \ldots, \gamma_{2k} )$ is formed by concatenation of 2k tangle-free segments $\gamma_s \in F^{\ell}$ , with the convention $e^{s+1}_1=e^s_{\ell+1}$ . Again, refer to Figure 4 for clarification.

As in Section 5.4, we will break these into circuits which visit exactly $\mathcal{V} = |V(\gamma)|$ different vertices, $\mathcal{R} = |V(\gamma) \cap V_2|$ of them in the right set, and $\mathcal{E} = |E(\gamma)|$ different edges. We define three disjoint sets of circuits:

\begin{align*} \mathcal{C}_1 & = \{\gamma \in \mathcal{C}: \mbox{all edges in $\gamma$ are traversed at least twice} \}, \\ \mathcal{C}_2 & = \{\gamma \in \mathcal{C}: \mbox{at least one edge in $\gamma$ is traversed exactly once and $\mathcal{V} \leq kl+1$} \}, \mbox{and} \\ \mathcal{C}_3 & = \{\gamma \in \mathcal{C}: \mbox{at least one edge in $\gamma$ is traversed exactly once and $\mathcal{V} > kl+1$} \} .\end{align*}

Define the quantities

\begin{equation*}I_j=\left|\mathbb{E} \sum_{\gamma\in \mathcal{C}_j} \prod_{t=1}^{2k\ell} \bar{B}_{e_{2t-1} e_{2t+1}}\right|\leq \sum_{\gamma \in \mathcal{C}_j}\left|\mathbb{E}\prod_{t=1}^{2k\ell}\bar{M}_{e_{2t-1} e_{2t}}\right|\end{equation*}

for $j = 1, 2$ and 3, so that (24) can be bounded as

(25) \begin{align}\mathbb{E}\left(\|\bar{B}^{(\ell)}\|^{2k}\right)\leq I_1+I_2+I_3.\end{align}

We will bound each term on the right-hand side above. The reason for this division is that, by Theorem 5.6, when we have any two-path traversed exactly once, the expectation of the corresponding circuit is smaller, because the matrix $\bar{B}$ is nearly centered. We will see that the leading order terms in equation (24) will come from circuits in $\mathcal{C}_1$ . From Lemmas 5.6 and 5.8 and Corollary 5.11, we get that

(26) \begin{align} I_j&\leq \sum_{\mathcal{V},\mathcal{E},\mathcal{R}}\sum_{\gamma\in \mathcal{C}_j \cap \mathcal{C}_{\mathcal{V},\mathcal{E}}^\mathcal{R}}\left|\mathbb{E}\prod_{t=1}^{2k\ell} \bar{M}_{e_{2t-1} e_{2t}}\right|\nonumber\\ &\leq \sum_{\mathcal{V},\mathcal{E},\mathcal{R}}C_{\mathcal{V},\mathcal{E}}^\mathcal{R} \,I_{\mathcal{V}, \mathcal{E}}^\mathcal{R} \,\left|\mathbb{E}\prod_{t=1}^{2k\ell} \bar{M}_{e_{2t-1} e_{2t}}\right|\nonumber\\ &\leq \sum_{\mathcal{V},\mathcal{E},\mathcal{R}}n^{\mathcal{V} - \mathcal{R}}m^{\mathcal{R}}(2\ell)^{4k} ((\mathcal{V} + 1)^2 (\ell + 1))^{2k(\chi + 1)} \nonumber \\ & \qquad\qquad\cdot (d_1 (d_1 - 1))^{\mathcal{V} - \mathcal{R}}(d_2 (d_2 - 1))^\mathcal{R}(d - 1)^{2(\chi - 1)} \nonumber \\& \qquad\qquad\cdot C2^{b_\mathcal{C}} \left( \frac{1}{|E|} \right)^\mathcal{E}\left( \frac{6 k \ell}{\sqrt{|E|}} \right)^{\mathcal{E}_1} \nonumber \\&\leq C\sum_{\mathcal{V},\mathcal{E},\mathcal{R}}(2\ell)^{4k} ((\mathcal{V} + 1)^2 (\ell + 1))^{2k(\chi + 1)}(d_1 - 1)^{\mathcal{V} - \mathcal{R}}(d_2 - 1)^\mathcal{R}(d - 1)^{2(\chi - 1)} \nonumber \\& \qquad\qquad\cdot 2^{b_\mathcal{C}}\left( \frac{1}{|E|} \right)^{\mathcal{E} - \mathcal{V}}\left( \frac{6 k \ell}{\sqrt{|E|}} \right)^{\mathcal{E}_1} \nonumber \\&\leq C\sum_{\mathcal{V},\mathcal{E},\mathcal{R}}(2\ell)^{4k} ((\mathcal{V} + 1)^2 (\ell + 1))^{2k(\chi + 1)}\left( (d_1 - 1)(d_2 - 1) \right)^{\mathcal{E} / 2}\nonumber \\& \qquad\qquad\cdot \left( \frac{d_2 - 1}{d_1 - 1} \right)^\psi(d_1 - 1)^{1-\chi}(d - 1)^{2(\chi - 1)}2^{b_\mathcal{C}}\left( \frac{1}{|E|} \right)^{\mathcal{E} - \mathcal{V}}\left( \frac{6 k \ell}{\sqrt{|E|}} \right)^{\mathcal{E}_1}\nonumber \\&\leq c_0 n\ell^{4k}c_1^{k^2}c_2^k\sum_{\mathcal{V},\mathcal{E}}\alpha^{2\mathcal{E}}\mathcal{V}((\mathcal{V} + 1)^2 (\ell + 1))^{2k(\chi + 1)}\left( \frac{c_3}{n} \right)^\chi\left( \frac{c_4 k \ell}{\sqrt{n}} \right)^{\mathcal{E}_1}\end{align}

We use $C, c_0, c_1, c_2, c_3, c_4$ to denote constant terms and set $\alpha = ((d_1-1)(d_2-1))^{1/4}$ . In the last line, we used Lemmas 5.12 and 5.13 to bound $\psi$ and $b_\mathcal{C}$ in terms of k and $\chi$ and remove the sum over $\mathcal{R}$ , which contains at most $\mathcal{V}$ terms. We will use equation (26) to bound each $I_j$ .

5.8.1 Bounding $I_1$

Here, $\mathcal{E}_1 = 0$ since every edge is traversed twice. We then have that $\mathcal{V} - 1 \leq \mathcal{E} \leq k\ell$ . Since $\gamma$ is connected, we have $1 \leq \mathcal{V} \leq k\ell+1$ . Thus, on the right-hand side of equation (26) we get

\begin{align*}I_1&\leq c_0 n\ell^{4k}c_1^{ k^2}c_2^k\sum_{\mathcal{V} = 1}^{k\ell+1}\sum_{\mathcal{E}=\mathcal{V}-1}^{k\ell}\alpha^{2\mathcal{E}}\mathcal{V}((\mathcal{V} + 1)^2 (\ell + 1))^{2k(\chi + 1)}\left( \frac{c_3}{n} \right)^\chi\\&=c_0 n\ell^{4k}c_1^{ k^2}c_2^k\sum_{\mathcal{V} = 1}^{k\ell+1}\alpha^{2(\mathcal{V} - 1)}\mathcal{V}((\mathcal{V} + 1)^2 (\ell + 1))^{2k}\sum_{\chi= 0}^{k\ell - \mathcal{V} + 1}\left( \alpha^2 ((\mathcal{V} + 1)^2 (\ell + 1))^{2k} \frac{c_3}{n}\right)^\chi\end{align*}

The second sum is upper bounded by $\sum_{\chi=0}^\infty\left( C \frac{ ((\mathcal{V}+1)^2 (\ell+1) )^{2k} }{n} \right)^\chi$ , where C is some constant, which we show next is bounded by a common constant for our choices of k and $\ell$ and all $\mathcal{V} \in [k\ell + 1]$ . To see this, it will suffice to show that $((\mathcal{V}+1)^2 (\ell+1) )^{2k}=o(n)$ . But $((\mathcal{V}+1)^2 (\ell+1) ) = O(k^2 \ell^3)$ which, for our choices of k and $\ell$ , yields

\begin{equation*}2k \log( (\mathcal{V}+1)^2 (\ell+1) )=O({\log(n)^{1/3}}\log(\log(n)))=o(\log(n))\end{equation*}

as desired.

Finally, the first summand is maximised for $\mathcal{V} = k\ell + 1$ and there are at $k\ell + 1$ many terms in that sum. Therefore, modifying the constant $c_0$ yields

(27) \begin{align}I_1\leq c^{\prime}_0 n\ell^{4k}c_1^{ k^2}c_2^k(k \ell + 1)^2\left( (k\ell+2)^2 (\ell+1) \right)^{2k}\alpha^{2 k \ell}.\end{align}

5.8.2 Bounding $I_2$

Here there is at least one edge traversed exactly once, so we have $\mathcal{E} \geq \mathcal{V}$ for $\gamma\in \mathcal{C}_2$ . Taking $\mathcal{E}_1 = 0$ only increases the right-hand side on equation (26); it becomes

\begin{align*}I_2&\leq c_0 n\ell^{4k}c_1^{ k^2}c_2^k\sum_{\mathcal{V}=1}^{k\ell+1} \sum_{\mathcal{E}=\mathcal{V}}^{2k\ell}\alpha^{2\mathcal{E}}\mathcal{V}((\mathcal{V} + 1)^2 (\ell + 1))^{2k(\chi + 1)}\left( \frac{c_3}{n} \right)^\chi\\&=c_0 n\ell^{4k}c_1^{ k^2}c_2^k\sum_{\mathcal{V} = 1}^{k\ell+1}\alpha^{2(\mathcal{V}-1)}\mathcal{V}((\mathcal{V} + 1)^2 (\ell + 1))^{2k}\sum_{\chi= 1}^{2k\ell - \mathcal{V} + 1}\left( \alpha^2 ((\mathcal{V} + 1)^2 (\ell + 1))^{2k} \frac{c_3}{n}\right)^\chi\end{align*}

Notice that this last term is almost identical to the one in the bound of $I_1$ , except that now we start the second sum at $\chi=1$ , which leads to an extra factor of $O( ((\mathcal{V} + 1)^2 (\ell + 1))^{2k} / n )$ . This allow us to factor out another geometric series and proceed as we did for $I_1$ . This yields

(28) \begin{align}I_2\leq c^{\prime}_0\ell^{4k}c_1^{ k^2}c_2^k(k \ell+1)^2\left( (k\ell+2)^2 (\ell+1) \right)^{4k}\alpha^{2k\ell},\end{align}

since there are $k\ell+1$ terms in the first sum.

5.8.3 Bounding $I_3$

This set will require more delicate treatment, since circuits in $\mathcal{C}_3$ visit potentially many vertices and edges, yet we need to keep the power of $\alpha$ at most $2k\ell$ .

We first show that, in this case, $\mathcal{E}_1$ is also large. We have $\mathcal{E} \geq \mathcal{V}$ , and let $\mathcal{V} = k\ell + t$ . Define $\mathcal{E}^{\prime}_1$ as the number of edges traversed once in $\gamma$ , so that $\mathcal{E}^{\prime}_1 = b + \mathcal{E}_1$ . Since $\gamma$ has length $2k \ell$ , we deduce that $2 (\mathcal{E} - \mathcal{E}^{\prime}_1) + \mathcal{E}^{\prime}_1 \leq 2k\ell$ , which implies that $\mathcal{E}^{\prime}_1 \geq 2t$ . Finally, Lemma 5.13 yields $\mathcal{E}_1 \geq (2t - 4 (\chi + k))_+$ . equation (26) then gives,

\begin{align*}I_3&\leq c_0 n\ell^{4k}c_1^{ k^2}c_2^k\sum_{\mathcal{V}=k\ell+1}^{2k\ell}\sum_{\mathcal{E}=\mathcal{V}}^{2k\ell}\alpha^{2\mathcal{E}}\mathcal{V}((\mathcal{V} + 1)^2 (\ell + 1))^{2k(\chi + 1)}\left( \frac{c_3}{n} \right)^\chi\left( \frac{(c_4 k \ell)^2}{n} \right)^{(\mathcal{V} - k\ell - 2(\chi+k))_+}\nonumber \\&=c_0 n\ell^{4k}c_1^{ k^2}c_2^k\sum_{t=1}^{k\ell}\sum_{\chi=1}^{k\ell - t+1}\alpha^{2 (k\ell + \chi + t-1)}(k\ell + t)((k\ell + t + 1)^2 (\ell + 1))^{2k(\chi + 1)}\\& \qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad \cdot \left( \frac{c_3}{n} \right)^{\chi}\left( \frac{(c_4 k \ell)^2}{n} \right)^{(t - 2(\chi+k))_+}\nonumber \\&=c_0n\ell^{4k}c_1^{ k^2}c_2^k\alpha^{2 (k\ell-1)}\sum_{t=1}^{k\ell}(k\ell + t)((k\ell + t + 1)^2 (\ell + 1))^{2k}\alpha^{2t}\\& \qquad\qquad\qquad\qquad\qquad\qquad \cdot \sum_{\chi=1}^{k\ell - t+1}\left( \alpha^2 ((k\ell + t + 1)^2 (\ell + 1))^{2k} \frac{c_3}{n} \right)^{\chi}\left( \frac{(c_4 k \ell)^2}{n} \right)^{(t - 2(\chi+k))_+}.\end{align*}

To simplify our notation, we will write

\begin{equation*}F(k,\ell)=c_0n\ell^{4k}c_1^{ k^2}c_2^k\alpha^{2 (k\ell-1)}(2k\ell)((2k\ell + 1)^2 (\ell + 1))^{2k}.\end{equation*}

Observe now that we can write

\begin{equation*}I_3\leq F(k,\ell)\sum_{t=1}^{k\ell} \alpha^{2t}\sum_{\chi=1}^{k\ell-t+1} \left(\frac{c(k,\ell)}{n}\right)^{\chi} \left(\frac{(c_4k\ell)^2}{n}\right)^{(t-2k-2\chi)_+}\end{equation*}

where $c(k,\ell)=c_3\alpha^2((2k\ell+1)^2(\ell+1))^{2k}$ . To bound the double sum on the right-hand side above, we start by removing a factor of $\frac{c(k,\ell)}{n}$ , which leaves

(29) \begin{equation}I_3\leq F(k,\ell)\frac{c(k,\ell)}{n} \sum_{t=1}^{k\ell} \alpha^{2t}\sum_{\chi=0}^{k\ell-t} \left(\frac{c(k,\ell)}{n}\right)^{\chi} \left(\frac{(c_4k\ell)^2}{n}\right)^{(t-2k-2-2\chi)_+}\end{equation}

The n in the denominator is crucial to cancel the linear term in $F(k,\ell)$ , keeping the upper bound for $I_3$ small. We focus on bounding the double sum. We split the sum in t in two parts.

Case 1: $t< 2k+2$ . For these values of t, we have $(t-2k-2-2\chi)_+=0$ , hence

(30) \begin{equation}\sum_{t=1}^{2k+1} \alpha^{2t}\sum_{\chi=0}^{k\ell-t} \left(\frac{c(k,\ell)}{n}\right)^{\chi} \left(\frac{(c_4k\ell)^2}{n}\right)^{(t-2k-2-2\chi)_+}=\sum_{t=1}^{2k+1} \alpha^{2t}\sum_{\chi=0}^{k\ell-t} \left(\frac{c(k,\ell)}{n}\right)^{\chi}=O(\alpha^{4k})\end{equation}

where the last equality uses again the same geometric upper bound for the sum over $\chi$ .

Case 2: $t\geq 2k+2$ . We split the second sum, from $\chi=0$ to $N=\lfloor t/2-k-1\rfloor$ and the terms with $\chi > N$ and analyse the two separately. The first can be upper bounded by

\begin{equation*}\sum_{t=2k+2}^{k\ell} \alpha^{2t}\sum_{\chi=0}^{N}\left(\frac{c(k,\ell)}{n}\right)^{\chi}\left(\frac{c_4 k \ell}{\sqrt{n}}\right)^{(t/2-k-1-\chi)}\leq \alpha^{4k+4} \sum_{t=0}^{k\ell-2k-2}\alpha^{2t} (N+1)\left( \frac{c_4 k \ell}{\sqrt{n}} \right)^{\frac{t}{2}}.\end{equation*}

The last inequality can be checked in two steps: We first factor out the power of $\alpha^{4k+4}$ , and then use that $\frac{c(k,\ell)}{n}\leq \frac{c_4 k \ell}{\sqrt{n}}$ , which holds for large enough n, to simplify the second sum to the addition of $N+1$ equal terms. To bound the right hand side, we one more time upper bound by a geometric series of ratio less than one to get

(31) \begin{equation}\sum_{t=2k+2}^{k\ell} \alpha^{2t}\sum_{\chi=0}^{N}\left(\frac{c(k,\ell)}{n}\right)^{\chi}\left(\frac{c_4 k\ell}{\sqrt{n}}\right)^{(t/2-k-1-\chi)}\leq O(k\ell\alpha^{4k}).\end{equation}

We are left with the terms $\chi > N$ . In this case, we get $(t-2k-2-2\chi)_+=0$ , so

\begin{equation*}\sum_{t=2k+2}^{k\ell} \alpha^{2t}\sum_{\chi=N+1}^{k\ell-t}\left(\frac{c(k,\ell)}{n}\right)^{\chi}\left(\frac{c_4 k \ell}{\sqrt{n}}\right)^{(t/2-k-1-\chi)_+}=\sum_{t=2k+2}^{k\ell} \alpha^{2t}\sum_{\chi=N+1}^{k\ell-t} \left(\frac{c(k,\ell)}{n}\right)^{\chi}.\end{equation*}

The sum over $\chi$ is of the order of $\left(\frac{c(k,\ell)}{n}\right)^{N+1}\leq \left(\frac{c(k,\ell)}{n}\right)^{t/2-k-1}$ . Substituting this into the above, we are left with

\begin{equation*}\sum_{t=2k+2}^{k\ell} \alpha^{2t}\sum_{\chi=N+1}^{k\ell-t} \left(\frac{c(k,\ell)}{n}\right)^{\chi}=C\sum_{t=2k+2}^{k\ell} \alpha^{2t}\left(\frac{c(k,\ell)}{n}\right)^{t/2-k-1}\end{equation*}

for some universal constant C. After factoring $\alpha^{4k+4}$ and changing variables in the summation, we conclude that

(32) \begin{equation}\sum_{t=2k+2}^{k\ell} \alpha^{2t}\sum_{\chi=N+1}^{k\ell-t} \left(\frac{c(k,\ell)}{n}\right)^{\chi}=C\alpha^{4k+4}\sum_{t=0}^{k\ell-2k-2} \alpha^{2t}\left(\frac{c(k,\ell)}{n}\right)^{t/2}=O(\alpha^{4k}).\end{equation}

Using (29) and the results for case 1 (30) and case 2 (31) and (32), we conclude that

(33) \begin{equation}I_3\leq F(k,\ell)\frac{c(k,\ell)}{n}O(k\ell \alpha^{4k})=c^{\prime}_0\ell^{4k}c_1^{ k^2}(c^{\prime}_2)^k(k\ell)^2((2k\ell + 1)^2 (\ell + 1))^{4k}\alpha^{2k\ell}\end{equation}

5.8.4 Finishing the proof of Theorem 5.15

We have bounded the three pieces we need to prove the theorem. From (27), (28), and (33), with n sufficiently large, we get

\begin{align*}\mathbb{E} \left( \| \bar{B}^{(\ell)} \|^{2k}\right)&\leq I_1 + I_2 + I_3 \\&\leq \alpha^{2k\ell}n\cdot Cc_1^{ k^2}c_5^k\ell^{4k}(k \ell)^{4}\left(k^2\ell^3 \right)^{4k}\\&\leq \alpha^{2k\ell}n\cdot C'c_1^{(\log n)^{2/3}}c_6^{(\log n)^{1/3}}(\log n)^{4 (\log n)^{1/3}}(\log n)^{16/3}\,((\log n)^{11/3})^{4 (\log n)^{1/3}}\\&\leq \alpha^{2k\ell}n\cdot C''c_1^{(\log n)^{2/3}}c_6^{(\log n)^{1/3}}(\log n)^{20 (\log n)^{1/3} + 6}\\& :=\alpha^{2k\ell} n \cdot f(n) ,\end{align*}

where $c_5, c_6, C, C', C''$ are universal constants.

Take any $\epsilon > 0$ . It can be checked that $(\log n)^{20 (\log n)^{1/3} + 6}= o(n^{\epsilon})$ , and $f(n) = o(n^{\epsilon})$ as well. Let $g(n) = \exp((\log n)^{3/4})$ ; then $g(n) = o(n^\epsilon)$ but $g(n)^{2k} \gg n^\epsilon$ . We apply Markov’s inequality, so that

(34) \begin{align}\mathbb{P} \left[\| B^{(\ell)} \|\geq\alpha^{\ell} g(n)\right]&\leq \frac{ \mathbb{E} \left( \| \bar{B}^{(\ell)} \|^{2k} \right) }{ \alpha^{2k\ell} g(n)^{2k}}\leq \frac{n f(n)}{g(n)^{2k}}=o(1),\end{align}

which is the statement of the theorem.

Theorem 5.16. Let $1\leq j\leq \ell = \lfloor c \log(n) \rfloor$ where $c < \frac{1}{32}$ is a universal constant. Then

\begin{equation*}\|R^{\ell,j}\|\leq \frac{(d-1)^\ell}{n} \exp((\log n)^{3/4})\end{equation*}

asymptotically almost surely.

Proof. The proof is analogous to the proof of Theorem 5.15. Recall the definition of $R^{\ell,j}$ in equation (15). For any integer k, we have that

(35) \begin{align}\mathbb{E}\left(\|R^{\ell,j}\|^{2k}\right)&\leq \mathbb{E}\left(\textrm{Tr}\left[ \left( (R^{\ell,j}) (R^{\ell,j})^* \right)^{k}\right]\right)\nonumber \\&\leq \sum_{ \gamma \in \mathcal{D} }\prod_{s=1}^{2k}\left|\mathbb{E}\prod_{i=1}^{j-1} \bar{B}_{e^s_{2i-1} e^s_{2i+1}}S_{e^s_{2j-1} e^s_{2j+1}}\prod_{i=j+1}^{\ell} B_{e^s_{2i-1} e^s_{2i+1}}\right|\nonumber \\&=\sum_{ \gamma \in \mathcal{D} }\prod_{s=1}^{2k}\left|\mathbb{E}\prod_{i=1}^{j-1} \bar{M}_{e^s_{2i-1} e^s_{2i}}S_{e^s_{2j-1} e^s_{2j+1}}\prod_{i=j+1}^{\ell} M_{e^s_{2i-1} e^s_{2i}}\right|\end{align}

Now, the sum is over the set $\mathcal{D}$ of circuits $\gamma= (\gamma_1,\gamma_2,\dots,\gamma_{2k})$ of length $2k\ell$ formed from 2k elements of $T^{\ell,j}$ , $\gamma_s = (e^s_1, e^s_2, \ldots, e^s_{2\ell+1} )$ for $s \in [2k]$ , with the convention $e^{s+1}_1 = e^s_{2\ell+1}$ (see Definition 5.4).

Proceeding as before, using Lemma 5.9 and Corollary 5.11, we get that

\begin{align*} \mathbb{E}\left(\|R^{\ell,j}\|^{2k}\right) &\leq \sum_{\mathcal{V}, \mathcal{E}, \mathcal{R}} \sum_{\gamma \in \mathcal{D}_{\mathcal{V}, \mathcal{E}}^\mathcal{R} } \prod_{s=1}^{2k} \left| \mathbb{E} \prod_{i=1}^{j-1} \bar{M}_{e^s_{2i-1} e^s_{2i}} S_{e^s_{2j-1} e^s_{2j+1}} \prod_{i=j+1}^{\ell} M_{e^s_{2i-1} e^s_{2i}} \right| \\ &\leq \sum_{\mathcal{V}, \mathcal{E}, \mathcal{R}} D_{\mathcal{V}, \mathcal{E}}^\mathcal{R} I_{\mathcal{V}, \mathcal{E}}^\mathcal{R} \prod_{s=1}^{2k} \left| \mathbb{E} \prod_{i=1}^{j-1} \bar{M}_{e^s_{2i-1} e^s_{2i}} S_{e^s_{2j-1} e^s_{2j+1}} \prod_{i=j+1}^{\ell} M_{e^s_{2i-1} e^s_{2i}} \right| \\ &\leq \sum_{\mathcal{V}, \mathcal{E}, \mathcal{R}} m^\mathcal{R} n^{\mathcal{V}-\mathcal{R}} (2 \ell)^{6k} ( (\mathcal{V} + 1)^2 (\ell + 1) )^{6k(\chi + 1)} \\ & \qquad \qquad \cdot (d_1 (d_1 - 1))^{\mathcal{V} - \mathcal{R}} (d_2 (d_2 - 1))^\mathcal{R} (d - 1)^{2 (\chi - 1)} \\ & \qquad \qquad \cdot \prod_{s=1}^{2k} \left( \frac{d-1}{|E|} \right) C 2^{b_s} \left(\frac{1}{|E|} \right)^{\mathcal{E}_s} \left( \frac{6\ell}{\sqrt{|E|}} \right)^{\mathcal{E}_{1,s}} .\end{align*}

The last inequality uses Lemma 5.6 on each path $\gamma_s$ , with $\mathcal{E}_s$ , $b_s$ , and $\mathcal{E}_{1,s}$ defined analogous to the quantities $\mathcal{E}$ , b, and $\mathcal{E}_1$ in the same lemma. Note that $S_{ef} \leq \frac{d-1}{|E|}$ for any e,f and $\sum_{s=1}^{2k} b_s \leq b_\mathcal{D} \leq 16 (k + \chi)$ by Lemma 5.14. Setting $\mathcal{E} = \sum_{s=1}^{2k} \mathcal{E}_s$ , taking $\mathcal{E}_1 = \sum_{s=1}^{2k} \mathcal{E}_{1,s} \geq 0$ , using $d_1, d_2 \leq d$ , and combining terms, we find that

\begin{align*} \mathbb{E}\left(\|R^{\ell,j}\|^{2k}\right) & \leq \sum_{\mathcal{V}, \mathcal{E}, \mathcal{R}} (2 \ell)^{6k} ( (\mathcal{V} + 1)^2 (\ell + 1) )^{6k(\chi + 1)} \\ & \qquad \qquad \cdot (d_1 - 1)^{\mathcal{V} - \mathcal{R}} (d_2 - 1)^\mathcal{R} (d - 1)^{2 (\chi - 1)} \\ & \qquad \qquad \cdot \left( \frac{C'}{|E|} \right)^{2k} 2^{b_\mathcal{D}} \left(\frac{1}{|E|} \right)^{\mathcal{E} - \mathcal{V}} \\ &\leq c_0 \ell^{6k} c_1^k n^{1 - 2k} \sum_{\mathcal{V}, \mathcal{E}} \mathcal{V} ( (\mathcal{V} + 1)^2 (\ell + 1) )^{6k(\chi + 1)} (d - 1)^{\mathcal{V}} \left( \frac{c_2}{n} \right)^{\chi} \\ &\leq c_0 \ell^{6k} c_1^k n^{1 - 2k} \sum_{\mathcal{V} = 1}^{2k\ell} \sum_{\mathcal{E} = \mathcal{V} - 1}^{2k\ell} \mathcal{V} ( (\mathcal{V} + 1)^2 (\ell + 1) )^{6k(\chi + 1)} (d - 1)^{\mathcal{V}} \left( \frac{c_2}{n} \right)^{\chi} \\ &\leq c_0 \ell^{6k} c_1^k n^{1 - 2k} \sum_{\mathcal{V} = 1}^{2k\ell} \mathcal{V} ( (\mathcal{V} + 1)^2 (\ell + 1) )^{6k} (d - 1)^{\mathcal{V}} \sum_{\chi = 0}^{2k\ell} \left( ( (\mathcal{V} + 1)^2 (\ell + 1) )^{6k} \frac{c_2}{n} \right)^{\chi} , \end{align*}

for some constants $c_0, c_1, c_2$ . Note that the important $n^{-2k}$ comes from the $|E|^{-2k}$ deterministic term. Now, like before we have that $( (\mathcal{V} + 1)^2 (\ell + 1) )^{6k} = o(n)$ , so for sufficiently large n we can bound the sum over $\chi$ by a constant. The sum over $\mathcal{V}$ is easily bounded as before, leading to

\begin{align*} \mathbb{E}\left(\|R^{\ell,j}\|^{2k}\right) &\leq (d-1)^{2k\ell} n^{1-2k} \cdot c^{\prime}_0 \ell^{6k} c_1^k (2k\ell)^2 ((2k\ell + 1)^2 (\ell + 1))^{6k} \\ &\leq (d-1)^{2k\ell} n^{1-2k} \cdot c^{\prime\prime}_0 c^{\prime(\log n)^{1/3}}_1 (\log n)^{6(\log n)^{1/3}} (\log n)^{8/3} ((\log n)^{11/3})^{6(\log n)^{1/3}} \\ &:= (d-1)^{2k\ell} n^{1-2k} \cdot f(n).\end{align*}

To finish the proof, we note that $f(n) = o(n^\epsilon)$ for any $\epsilon > 0$ . Take $g(n) = \exp((\log n)^{3/4})$ . Then, applying Markov’s inequality,

(36) \begin{equation} \mathbb{P} \left[ \| R^{\ell,j} \| \geq \frac{(d - 1)^{\ell} }{n} g(n) \right] \leq \frac{\mathbb{E} \left( \| R^{\ell,j} \|^{2k} \right) n^{2k}} { (d-1)^{2k\ell} g(n)^{2k} } \leq \frac{n f(n)}{g(n)^{2k}} = o(1).\end{equation}

5.9. Proof of the main result, Theorem 3.1

We will again take $\ell = \lfloor c \log n \rfloor$ , with c chosen so that the graph is $\ell$ -tangle-free with high probability. By equations (10) and (17),

\begin{equation*}|\lambda_2|^{\ell}\leq \| \bar{B}^{(\ell)}\|+\sum_{k=1}^{\ell} \| R^{\ell,j}\|.\end{equation*}

Notice that $(d-1)^\ell \leq (d-1)^{c \log n} \leq n^{c \log d}$ , so take $c < \min \left(\frac{1}{32}, \frac{1}{\log d} \right)$ . Then $(d-1)^\ell = O(n^\epsilon)$ for some $0 < \epsilon < 1$ , and recall that $\exp((\log n)^{3/4}) = o(n^{\epsilon'})$ for any $\epsilon' > 0$ . We apply Theorems 5.15 and 5.16 to get

\begin{eqnarray}\nonumber|\lambda_2|& \leq &\left(\exp((\log n)^{3/4}) \left( (d_1-1)(d_2-1) \right)^{\ell/4}+\ell\exp((\log n)^{3/4})\frac{(d-1)^\ell}{n}\right)^{1/\ell} \\\nonumber& = & \left( (d_1-1)(d_2-1) \right)^{1/4}+\epsilon_n,\end{eqnarray}

where $\epsilon_n \to 0$ as $n \to \infty$ .

6.1. The frame model

We define the random regular frame graph distribution $\mathcal{G}(n,H)$ as a distribution of simple graphs on n vertices parametrised by the ‘frame’ H. The frame $H=(V,E,p,D)$ is a weighted, directed graph. Here, V is the vertex set, $E \subseteq \{(i,j): i,j \in V\}$ is the directed edge set, the vertex weights are p, and the edge weights are D. Note that we drop the arrows on the edge set in this section, since it will always be directed. The vertex weight vector $p \in \mathbb{R}^{|V|}$ , where $\sum_{i \in V} p_i = 1$ , sets the relative sizes of the classes. The edge weights are a matrix of degrees $D \in \mathbb{N}^{|V| \times |V|}$ . These assign the number of edges between each class in the random graph: $D_{ij}$ is the number of edges from each vertex in class i to vertices in class j. The degrees must satisfy the balance condition

(37) \begin{equation} p_i D_{ij} = p_j D_{ji}\end{equation}

for all $i,j \in V$ where (i,j) or (j,i) are in E. This requires that, for every edge $e \in E$ , its reverse orientation also exists in H. We also require that $n_i = n p_i \in \mathbb{N}$ for every $i \in V$ , so that the number of vertices in each type is integer.

Given the frame H, a random regular frame graph $G \sim \mathcal{G}(n,H)$ is a simple graph on n vertices with $n_i$ vertices in class i. It is chosen uniformly among graphs with the constraint that each vertex in class i makes $D_{ij}$ connections among the vertices in class j. In other words, if $i = j$ , we sample that block of the adjacency matrix as the adjacency matrix of a $D_{ii}$ -regular random graph on $n_i$ vertices. For off-diagonal blocks $i \neq j$ , these are sampled as bipartite, biregular random graphs $\mathcal{G} (n_i, n_j, D_{ij}, D_{ji})$ .

Sampling from $\mathcal{G}(n,H)$ can be performed similar to the configuration model, where each node is assigned as many half-edges as its degree, and these are wired together with a random matching [Reference Newman55]. The detailed balance condition equation (37) ensures that this matching is possible. Practically, we often have to generate many candidate matchings before the resulting graph is simple, but the probability of a simple graph is bounded away from zero for fixed D.

An example of a random regular frame graph is the bipartite, biregular random graph. The family $\mathcal{G}(n, m, d_1, d_2)$ is a random regular frame graph $\mathcal{G}(n + m, H)$ , where the frame H is the directed path on two vertices: $V= \{ 1, 2 \}$ and $E = \{ (1,2), (2,1) \}$ . The weights are taken as $p_1 = n/(n+m)$ , $p_2 = m/(n+m)$ , $D_{12} = d_1$ , and $D_{21} = d_2$ .

Another example random regular frame graph is shown in Figure 6. In this case, the frame H has $V= \{ A, B, C \}$ and $E = \{ (A,B), (A,C), (B,A), (B,C), (C,A), (C,B) \}$ with weights p and D as shown in Figure 6A. We see that this generates a random tripartite graph with regular degrees between vertices in different independent sets, shown in Figure 6B.

Figure 6. Schematic and realisation of a random regular frame graph. A, The frame graph. The vertices of the frame (red = A, green = B, blue = C) are weighted according to their proportions p in the random regular frame graph. The edge weights $D_{ij}$ set the between-class vertex degrees in the random regular frame graph. This frame will yield a random tripartite graph. B, Realisation of the graph on 72 vertices. In this instance, there are $1/8 \times 72=9$ green and red vertices and $3/4 \times 72=54$ blue vertices. Each blue vertex connects to $k_{CA}=1$ red vertex and $k_{CB}=2$ green vertices. This is actually a multigraph; with so few vertices, the probability that the configuration model algorithm yields parallel edges is high.

6.2. Markov and related matrices of frame graphs

Now, we define a number of matrices associated with the frame and the sample of the random regular frame graph.

Let G be a simple graph. Define $D_G = \textrm{diag}(d_G)$ , the diagonal matrix of degrees in G. The Markov matrix $P=P(G)$ is defined as

\begin{equation*}P = D_G^{-1} A,\end{equation*}

where $A=A(G)$ is the adjacency matrix. The Markov matrix is the row-normalized adjacency matrix, and it contains the transition probabilities of a random walker on the graph G. Let $L = I - \mathcal{L} = D_G^{-1/2} A \, D_G^{-1/2}$ be a matrix simply related to the normalized Laplacian. We call this the symmetrized Markov matrix. Then P and L have the same eigenvalues, but L is symmetric, since $L_{ij} = \frac{A_{ij}}{ \sqrt{ d_i d_j } }$ .

Suppose $G \sim \mathcal{G}(n, H)$ , where the frame $H = (V,E,p, D)$ . Another matrix that will be useful is what we call the Markov matrix of the frame R, where $R_{ij} = \frac{ D_{ij} }{\sum_j D_{ij}}$ . Thus, R is a row-normalized D, in the same way that the Markov matrix P is the row-normalized adjacency matrix A. Furthermore, R is invariant under any uniform scaling of the degrees. Because of this equitable partition property of random regular frame graphs, eigenvectors of the frame matrices $D=D(H)$ or $R=R(H)$ lift to eigenvectors of $A = A(G)$ or $P=P(G)$ , respectively. Suppose $D x = \lambda x$ , then it is a straightforward exercise to check that $A \tilde{x} = \lambda \tilde{x}$ for the piecewise constant vector

\begin{equation*}\tilde{x} =\left[\begin{array}{c}\textbf{1}_{n_1} x_1 \\\textbf{1}_{n_2} x_2 \\\vdots\end{array}\right] .\end{equation*}

Using the same procedure, we can lift any eigenpair of R to an eigenpair of P with the same eigenvalue.

6.2.1 Bounds on the eigenvalues of frame graphs in terms of blocks

The following result is due to [Reference Wan, Meilă, Cortes, Lawrence, Lee, Sugiyama and Garnett62]:

Proposition 6.1. Let G be a random regular frame graph G(n,H), P its Markov matrix, and L the Laplacian with vertices ordered by class in both cases. Let R be the Markov matrix of the frame $H=(V,E,p,D)$ , with $|V(H)| = K$ classes. Define the matrices $L^{(kl)}$ as the (k,l) block of L with respect to the clustering of vertices by class. For $l \neq k$ , let

\begin{equation*}M^{(kl)} =\left(\begin{array}{cc}0 & L^{(kl)} \\L^{(kl)} & 0\end{array}\right)=\left(\begin{array}{cc}0 & L^{(kl)} \\L^{(lk)*} & 0\end{array}\right).\end{equation*}

For $l=k$ , let $M^{(kk)} = L^{(kk)}$ . Assume that all eigenvalues of D are nonzero and pick a constant C such that

\begin{equation*} \frac{| \lambda_2^{(kl)} |}{\lambda_1^{(kl)}} \le C < 1\end{equation*}

for every $k,l = 1, \ldots K$ , where $\lambda_1^{(kl)}$ and $\lambda_2^{(kl)}$ are the leading and second eigenvalues of $M^{(kl)}$ . Under these conditions, the eigenvalues of P which are not eigenvalues of R are bounded by

\begin{equation*}C \max_{k=1,\ldots, K} \left( R_{kk} + \sum_{l \neq k} \sqrt{R_{kl} R_{lk}} \right)\leq \frac{C}{2} \left(1 + \max_{k=1,\ldots, K} \sum_{l=1}^K R_{lk} \right) .\end{equation*}

The spectrum of the Markov matrix $\sigma(P)$ enjoys a simple connection to $\sigma(A)$ when A is the adjacency matrix of a graph drawn from $G(n,m,d_1,d_2)$ . In this case, $L = \frac{ A }{ \sqrt{d_1 d_2} }$ , so the eigenvalues of P are just the scaled eigenvalues of A. This and the spectral gap for bipartite, biregular random graphs, Theorem 3.2, lead to the following remark:

Remark. For a random regular frame graph, $M^{(kl)}$ corresponds to the symmetrized Markov matrix L of a bipartite biregular graph $G(n_k, n_l, D_{kl}, D_{lk})$ . Thus,

\begin{equation*}\frac{| \lambda_2^{(kl)} |}{\lambda_1^{(kl)}}\leq \frac{\sqrt{D_{kl} - 1} +\sqrt{D_{lk} - 1}}{\sqrt{D_{kl} D_{lk}}} + \epsilon .\end{equation*}

Suppose we are given a frame that fits the conditions of Proposition 6.1; namely, D cannot have any zero eigenvalues. Then we can uniformly grow the degrees, which leaves R invariant, but allows us to reach an arbitrarily small C. This ensures that the leading K eigenvalues of P are equal to the eigenvalues of R. Note that this actually means that the entire random regular frame graph satifsfies a weak Ramanujan property. We now show that this guarantees spectral clustering.

6.3. Spectral clustering

Spectral clustering is a popular method of community detection. Because some eigenvectors of P, the Markov matrix of a random regular frame graph, are piecewise constant on classes, we can use them to recover the communities so long as those eigenvectors can be identified. Suppose there are K total classes in our random regular frame graph. Then, given the eigenvectors $x^1, x^2, \ldots, x^K$ , which are piecewise constant across classes, we can cluster vertices by class. For each vertex $v \in V(G)$ , associate the vector $y^v \in \mathbb{R}^K$ where $y^v_j = x^j_v$ . Then if $y^v = y^u$ for $u, v \in V(G)$ , vertices u and v belong to the same class.Footnote a It is simple to recover these piecewise constant vectors $x^1, x^2, \ldots, x^K$ when they are the leading eigenvectors. These facts lead to the following theorem:

Remark. The conditions of Theorem 6.2, while very general, are also weaker than may be expected using more sophisticated methods tailored to the specific frame model. We illustrate this with the following example.

6.3.1 Example: The regular stochastic block model

Brito et al. [Reference Brito, Dumitriu, Ganguly, Hoffman and Tran16] and Barucca [Reference Barucca5] studied a regular stochastic block model, which can be seen as a special case of our frame model. Let the frame H be the complete directed graph on two vertices, including self loops, where

\begin{equation*}D = \left(\begin{array}{cc}d_1 & d_2\\d_2 & d_1\end{array}\right)\end{equation*}

and $p = (1/2, 1/2)$ . Define the regular stochastic block model as $\mathcal{G} (2n, H)$ . This is a graph with two classes of equal size, representing two communities of vertices, with within-class degree $d_1$ and between-class degree $d_2$ . We assume $d_1 > d_2$ , since communities are more strongly connected within. Brito et al. [Reference Brito, Dumitriu, Ganguly, Hoffman and Tran16] proved the following theorem:

Theorem 6.3. If $(d_1-d_2)^2 > 4 (d_1 + d_2 -1)$ , then there is an efficient algorithm for strong recovery, i.e. recovery of the exact communities with high probability as $n \to \infty$ .

Theorem 6.3 gives a sharp bound on the degrees for recovery, which we can compare to our spectral clustering results. The eigenvalues of D are $d_1 + d_2$ and $d_1 - d_2$ , and the Markov matrix of the frame R has eigenvalues 1 and $(d_1 - d_2) / (d_1 + d_2)$ . The diagonal blocks $L^{(11)}$ and $L^{(22)}$ each correspond to the Laplacian matrix of a $d_1$ -regular random graph on n vertices, whereas the off-diagonal block term $M^{(12)}$ corresponds to the Laplacian of a $d_2$ -regular bipartite graph on 2n vertices. Using our results and the previously known results for regular random graphs [Reference Friedman27, Reference Friedman28, Reference Bordenave, Lelarge and Massoulié14], we can pick some $C > 2 \sqrt{d_2 - 1}/d_2$ since $d_1 > d_2$ and we will eventually take the degrees to be large. Using Proposition 6.1, we find that the spurious eigenvalues of P come after the leading 2 eigenvalues if

\begin{equation*}\frac{2 \sqrt{d_2 - 1}}{d_2} < \frac{d_1 - d_2}{d_1 + d_2},\end{equation*}

to leading order in the degrees. Rearranging, we obtain the condition

\begin{equation*}(d_1 - d_2)^2 > 4 (d_2 - 1) \left( \frac{d_1 + d_2}{d_2} \right)^2 .\end{equation*}

Assuming $d_2/d_1 = \beta < 1$ fixed, and taking the limit $d_1, d_2 \to \infty$ , we find that the result of [Reference Brito, Dumitriu, Ganguly, Hoffman and Tran16] becomes

\begin{equation*}d_1 > 4 \frac{1+\beta}{(1-\beta)^2} + o(1),\end{equation*}

whereas our result becomes

\begin{equation*}d_1 > \frac{4}{\beta} \left( \frac{1+\beta}{1-\beta} \right)^2 + o(1),\end{equation*}

illustrating that the spectral threshold is a factor of $(1+\beta)/\beta$ weaker.

7.1 Example: An unbalanced code based on a (14,9)-regular bipartite graph

We illustrate the applicability of our distance bound with an example. Let $\mathcal{C}_1 = [14, 8, 7]$ and $\mathcal{C}_2 = [9, 4, 6]$ . These can be achieved by using a Reed–Salomon code on the common field $\mathbb{F}_{q}$ for any $q > 14$ [Reference Richardson and Urbanke58]. We take $q = 2^4 = 16$ for inputs that are actually binary, and this means each edge in the graph actually contains 4 bits of information. Employing Corollary 7.2, the Tanner code $\mathcal{C}$ will have relative minimum distance $\delta / (n d_1) \geq 0.0014$ and rate at least $0.016$ . Taking $n = 216$ and $m = 336$ gives the code a minimum distance of at least 4.

8.1. Matrix norms as measures of complexity and their relationships

We will employ a number of different matrix and vector norms in this section. These are all related by the properties of the underlying Banach spaces. The complexity of Y is measured using a factorisation norm (also called the max-norm):

\begin{equation*} \gamma_2 (Y) = \min_{UV^* = Y} \|U\|_{\ell_2 \to \ell_\infty^n} \|V\|_{\ell_2 \to \ell_\infty^m}.\end{equation*}

The minimum is taken over all possible factorisations of $Y = UV^*$ , and the norm $\| X \|_{\ell_2 \to \ell_\infty^n} = \max_i \sqrt{\sum_j X^2_{ij}}\ $ returns the largest $\ell_2$ norm of a row. So, equivalently,

\begin{equation*} \gamma_2 (Y) = \min_{U V^* = Y} \max_{i,j} \|u_i \|_2 \, \|v_j \|_2 ,\end{equation*}

where $u_i$ and $v_i$ are the rows of U and V. See [Reference Linial, Mendelson, Schechtman and Shraibman44] for a number of results about the norm $\gamma_2$ . In particular, note that

(38) \begin{align}\gamma_2(A \circ B) \leq \gamma_2(A) \gamma_2(B) \end{align}

(39) \begin{align} \frac{1}{\sqrt{n m}} \| Y \|_{\textrm{Tr}} \leq \gamma_2(Y) \end{align}

(40) \begin{align} \gamma_2(Y) \leq \sqrt{\textrm{rank} (Y) } \| Y \|_{\infty}.\end{align}

Property (38) says that $\gamma_2$ is sub-multiplicative under the Hadamard product [Reference Lee, Shraibman and Špalek42, Reference De Winter, Schillewaert and Verstraete23] and will be used in our proof. Properties (39) and (40) relate $\gamma_2$ to two common complexity measures of matrices, the trace norm (sum of singular values, i.e. the $\ell^m_2 \to \ell^n_2$ nuclear norm) and rank. Note also the well-known fact that

\begin{equation*}\| Y \|_{\textrm{Tr}} = \min_{UV^* = Y} \| U \|_F \|V \|_F ,\end{equation*}

where $\| X \|_F = \sqrt{\sum_{ij} X_{ij}^2}$ is the Frobenius norm. We see that the trace norm constrains factors U and V to be small on average via $\| \cdot \|_F$ , whereas the norm $\gamma_2$ is similar but constrains factors uniformly via $\| \cdot \|_{\ell_2 \to \ell_\infty^n}$ . However, we should note that computing $\gamma_2(Y)$ is more costly than the trace norm, which can be performed with just the singular value decomposition, although still possible in polynomial time with convex programming [Reference Heiman, Schechtman and Shraibman33].

8.2. Matrix completion generalisation bounds

The method of matrix completion that we study is to return the matrix X which is the solution to:

(41) \begin{equation}\begin{aligned}& \underset{X}{\text{minimise}}& & \gamma_2(X) \\& \text{subject to}& & X_{ij} = Y_{ij}, \; (i,j) \in E .\end{aligned}\end{equation}

The $\gamma_2$ norm was first proposed by [Reference Srebro and Shraibman60, Reference Srebro, Rennie, Jaakkola, Saul, Weiss and Bottou59] as a robust complexity measure, and it was shown to be an effective and practical regularisation on real data sets [Reference Lee, Recht, Srebro, Tropp and Salakhutdinov41, Reference Recht and Ré56].

Heiman et al. [Reference Heiman, Schechtman and Shraibman33] analyse the performance of the convex program (41) for a square matrix Y using an expander argument, assuming that E is the edge set of a d-regular graph with second eigenvalue $\eta$ . They obtain the following theorem:

Theorem 8.1 ([Reference Heiman, Schechtman and Shraibman33]). Let E be the set of edges of a d-regular graph with second eigenvalue bound $\eta$ . For every $Y \in \mathbb{R}^{n \times n}$ , if $\hat{Y}$ is the output of the optimization problem (38), then

\begin{equation*} \frac{1}{n^2} \| \hat{Y} - Y \|_F^2 \leq c \gamma_2(Y)^2 \frac{\eta}{d},\end{equation*}

where $c = 8 K_G \leq 14.3$ is a universal constant and $\| \cdot \|_F$ is the Frobenius norm.

Considering sampling following the biadjacency matrix of a bipartite graph, we find a similar result which also applies to rectangular matrices. If $n=m$ and $d_1 = d_2 = d$ , our bound is equivalent to that of Theorem 8.1, but with constants improved by a factor of two due to stronger mixing in bipartite graphs. Intuitively, using a biadjacency matrix is a ‘more random’ way of sampling than using an adjacency matrix, since it is not symmetric.

Theorem 8.2. Let E be the set of edges of a $(d_1, d_2)$ -regular graph with second eigenvalue bound $\eta$ . For every $Y \in \mathbb{R}^{n \times m}$ , if $\hat{Y}$ is the output of the optimization problem (41), then

\begin{equation*} \frac{1}{n m} \| \hat{Y} - Y \|_F^2 \leq c \gamma_2(Y)^2 \frac{\eta}{\sqrt{d_1 d_2}} ,\end{equation*}

where $c = 4 K_G \leq 7.13$ .

Proof. We start by considering a rank-1 sign matrix $S=u v^*$ , where $u\in\{-1,1\}^{n}, v\in\{-1,1\}^{m}$ . Let $S' = \frac{1}{2} (S + J)$ , where J is the all-ones matrix, so that S’ has the entries of -1 in S replaced by zeros. Then $S' = 1_A 1_B^* + 1_{A^c} 1_{B^c}^*$ for subsets $A \subset V_1 = [n]$ and $B \subset V_2 = [m]$ , where $A = \{ i:\, u_i = 1 \} $ and $B = \{ j :\, v_j = 1 \}$ . Consider the expression

\begin{align*}\left| \frac{1}{n m} \sum_{i,j} s_{ij} -\frac{1}{|E|} \sum_{(i,j) \in E} s_{ij} \right|&=\left| \frac{1}{n m} \sum_{i,j} (2 s^{\prime}_{ij} - 1) -\frac{1}{|E|} \sum_{(i,j) \in E} (2 s^{\prime}_{ij} - 1) \right| \\&=2 \left| \frac{1}{n m} \sum_{i,j} s^{\prime}_{ij} -\frac{1}{|E|} \sum_{(i,j) \in E} s^{\prime}_{ij} \right| \\&= 2 \left| \frac{|A||B| + |A^c| |B^c|}{n m} -\frac{E(A,B) + E(A^c, B^c)}{|E|} \right| \\&\leq 2 \left| \frac{|A||B|}{nm} - \frac{E(A,B)}{|E|} \right|+2 \left| \frac{|A^c| |B^c|}{n m} -\frac{E(A^c, B^c)}{|E|} \right|.\end{align*}

The following is a bipartite version of the expander mixing lemma [Reference De Winter, Schillewaert and Verstraete23]:

\begin{equation*}\left| \frac{E(A,B)}{|E|} - \frac{|A||B|}{n m} \right|\leq \frac{\eta}{\sqrt{d_1 d_2}}\sqrt{\frac{|A||B|}{nm}\left(1- \frac{|A|}{n} \right) \left( 1- \frac{|B|}{m} \right) } \\=\frac{\eta}{\sqrt{d_1 d_2}}\sqrt{\frac{|A||B||A^c||B^c|}{(nm)^2}} .\end{equation*}

We find that

\begin{align*}\left| \frac{1}{n m} \sum_{i,j} s_{ij} -\frac{1}{|E|} \sum_{(i,j) \in E} s_{ij} \right|& \leq \frac{4 \eta}{\sqrt{d_1 d_2}}\sqrt{\frac{|A||B||A^c||B^c|}{(nm)^2}}\\& = \frac{4 \eta}{\sqrt{d_1 d_2}}\sqrt{ x y (1-x) (1-y) }\\& \leq \frac{\eta}{\sqrt{d_1 d_2}},\end{align*}

since $x y (1-x) (1-y)$ attains a maximal value of $2^{-4}$ for $0 \leq x,y \leq 1$ .

The rest of the proof develops identical to the results of [Reference Heiman, Schechtman and Shraibman33], which we include for completeness. We apply the result for rank-1 sign matrices to any matrix R. Let $R = \sum_i \alpha_i S^i$ , where $S^i$ is a rank-1 sign matrix and $\alpha_i \in \mathbb{R}$ . For a general matrix R, this might require many rank-1 sign matrices. Define the sign nuclear norm $\nu (R) = \sum_i |\alpha_i|$ . Then,

\begin{equation*}\left| \frac{1}{n m} \sum_{i,j} r_{ij} -\frac{1}{|E|} \sum_{(i,j) \in E} r_{ij} \right|\leq \nu(R) \frac{\eta}{\sqrt{d_1 d_2}} .\end{equation*}

It is a consequence of Grothendieck’s inequality, a well-known theorem in functional analysis, that there exists a universal constant $1.5 \leq K_G \leq 1.8$ so that $\gamma_2 (X) \leq \nu(X) \leq K_G \gamma_2(X)$ for any real matrix X; see [Reference Heiman, Schechtman and Shraibman33].

Now, let the matrix of residuals $R = (\hat{Y}-Y) \circ (\hat{Y} - Y)$ , where $\circ$ is the Hadamard entry-wise product of two matrices, so that $R_{ij} = (\hat{Y}_{ij} - Y_{ij})^2$ . Since

\begin{equation*}\frac{1}{|E|} \sum_{(i,j) \in E} r_{ij} = 0 ,\end{equation*}

we conclude that

\begin{equation*}\frac{1}{n m} \sum_{i,j} r_{ij}\leq \nu(R) \frac{\eta}{\sqrt{d_1 d_2}}\leq K_G \gamma_2(R) \frac{\eta}{\sqrt{d_1 d_2}}.\end{equation*}

Furthermore, $\gamma_2 (R) \leq \gamma_2 (\hat{Y}-Y)^2 \leq ( \gamma_2(\hat{Y}) + \gamma_2(Y) ) ^2 $ by (38) and the triangle inequality. Since $\hat{Y}$ is the output of the algorithm and Y is a feasible solution, $\gamma_2 (\hat{Y}) \leq \gamma_2 (Y)$ . Thus, $\gamma_2(R) \leq 4 \gamma_2(Y)^2$ and the proof is finished.

8.3. Noisy matrix completion bounds

Furthermore, our analysis easily extends to the case where the matrix we observe is corrupted with noise. As mentioned in the above remark, similar results will hold for the trace norm. In the noisy case, we solve the problem

(42) \begin{equation}\begin{aligned}& \underset{X}{\text{minimize}}& & \gamma_2(X) \\& \text{subject to}& & \frac{1}{|E|} \sum_{(i,j) \in E} ( X_{ij} - Z_{ij})^2 \leq \delta^2\end{aligned}\end{equation}

and obtain the following theorem:

Theorem 8.3. Suppose we observe $Z_{ij} = Y_{ij} + \epsilon_{ij}$ with bounded error

\begin{equation*} \frac{1}{|E|} \sum_{(i,j) \in E} \epsilon_{ij}^2 \leq \delta^2 .\end{equation*}

Then solving the optimization problem (42) will yield a bound of

\begin{equation*}\frac{1}{n m} \| \hat{Y} - Y \|_F^2 \leq c \gamma_2(Y)^2 \frac{\eta}{\sqrt{d_1 d_2}} + 4 \delta^2,\end{equation*}

where $c = 4 K_G \leq 7.13$ .

Proof. Denote $\hat{Y}$ the solution to P42. It will be useful to introduce the sampling operator $\mathcal{P}_E : \mathbb{R}^{n \times m} \to \mathbb{R}^{n \times m}$ , where $(\mathcal{P}_E (X) )_{ij} = X_{ij}$ if $(i,j) \in E$ and 0 otherwise. Again let $R = (\hat{Y} - Y) \circ (\hat{Y} - Y)$ be the matrix of squared errors, then

\begin{align*}\left|\frac{1}{nm} \| \hat{Y} - Y \|_F^2 -\frac{1}{|E|} \| \mathcal{P}_E (\hat{Y} - Y) \|_F^2\right|&=\left| \frac{1}{n m} \sum_{i,j} (\hat{Y}_{ij} - Y_{ij})^2 -\frac{1}{|E|} \sum_{(i,j) \in E} (\hat{Y}_{ij} - Y_{ij})^2 \right| \\& \leq K_G \gamma_2(R) \frac{\eta}{\sqrt{d_1 d_2}}.\end{align*}

However, since Y is a feasible solution to P42, we have

\begin{equation*}\gamma_2 (\hat{Y}) \leq \gamma_2 (Y).\end{equation*}

Applying (38) and the triangle inequality,

\begin{equation*}\gamma_2 (R) \leq \left( \gamma_2(\hat{Y} - Y) \right)^2\leq \left( \gamma_2(\hat{Y}) + \gamma_2(Y) \right)^2\leq 4 \gamma_2(Y)^2 .\end{equation*}

Using the triangle inequality again gives

\begin{equation*}\| \mathcal{P}_E (\hat{Y} - Y) \|_F\leq \| \mathcal{P}_E (\hat{Y} - Z) \|_F+\| \mathcal{P}_E (Z - Y) \|_F\leq 2 \delta \sqrt{|E|},\end{equation*}

taking into account the bound on the observation errors. Because

\begin{equation*}\frac{1}{nm} \| \hat{Y} - Y \|_F^2\leq \left| \frac{1}{nm} \| \hat{Y} - Y \|_F^2 - \frac{1}{|E|} \| \mathcal{P}_E (\hat{Y} - Y) \|_F^2\right|+\frac{1}{|E|} \| \mathcal{P}_E (\hat{Y} - Y) \|_F^2\end{equation*}

we get the final bound

\begin{equation*}\frac{1}{n m} \| \hat{Y} - Y \|_F^2\leq 4 K_G \gamma_2(Y)^2 \frac{\eta}{\sqrt{d_1 d_2}} + 4 \delta^2.\end{equation*}

8.4. Application of the spectral gap

Theorem 8.2 provides a bound on the mean squared error of the approximation X. Directly applying Theorem 3.2, we obtain the following bound on the generalisation error of the algorithm using a random biregular, bipartite graph:

Corollary 8.4. Let E be sampled from a $\mathcal{G}(n,m, d_1, d_2)$ random graph. For every $Y \in \mathbb{R}^{n \times m}$ , if $\hat{Y}$ is the output of the optimization problem (38), then

\begin{equation*} \frac{1}{n m} \| \hat{Y} - Y \|_F^2 \leq c \gamma_2(Y)^2 \frac{\sqrt{d_1 - 1} + \sqrt{d_2 - 1} + \epsilon_n }{\sqrt{d_1 d_2}} ,\end{equation*}

where $c = 4 K_G \leq 7.13$ is a universal constant.