1.1 Outline of the spectral approach

Given a finite abelian group $\operatorname {\mathrm {Z}}$ and a function $f:\operatorname {\mathrm {Z}}\to \mathbb {C}$ , a first key step in this approach consists in turning f into a matrix in $\mathbb {C}^{\operatorname {\mathrm {Z}}\times \operatorname {\mathrm {Z}}}$ (or a $\operatorname {\mathrm {Z}}$ -matrix, as we shall call it; see Definition 2.1). We consider various choices for how to do this, all of which proceed by applying a (typically nonlinear) transformation $K:\mathbb {C}^{\operatorname {\mathrm {Z}}}\to \mathbb {C}^{\operatorname {\mathrm {Z}}}$ to every so-called $\operatorname {\mathrm {Z}}$ -diagonal function of the rank-1 matrix

$$\begin{align*}f\otimes\overline{f}:\operatorname{\mathrm{Z}}\times \operatorname{\mathrm{Z}}\to\mathbb{C},\, (x,y)\mapsto f(x)\overline{f(y)}, \end{align*}$$

a $\operatorname {\mathrm {Z}}$ -diagonal being a set of entries whose indices have fixed difference in $\operatorname {\mathrm {Z}}$ (see Definition 2.1).

The matrix $f\otimes \overline {f}$ is a natural and convenient object to consider in higher-order Fourier analysis, since its $\operatorname {\mathrm {Z}}$ -diagonal function with entry-difference t is the multiplicative derivative

$$\begin{align*}\Delta_t f(x):=f(x+t)\overline{f(x)}, \end{align*}$$

which plays a key role in this theory (notably in proofs of inverse theorems). The matrix $f\otimes \overline {f}$ thus encapsulates in a useful way all the multiplicative derivatives of f.

In general, given a map (or operator) $K:\mathbb {C}^{\operatorname {\mathrm {Z}}}\to \mathbb {C}^{\operatorname {\mathrm {Z}}}$ and a matrix $M\in \mathbb {C}^{\operatorname {\mathrm {Z}}\times \operatorname {\mathrm {Z}}}$ , we denote by $\mathcal {K}(M)$ the matrix obtained by applying K to every $\operatorname {\mathrm {Z}}$ -diagonal function of M. To ensure that $\mathcal {K}$ is well-behaved, we require the operator K to commute with complex conjugation and the shift action of $\operatorname {\mathrm {Z}}$ on functions in $\mathbb {C}^{\operatorname {\mathrm {Z}}}$ (in which case we call K an invariant operator; see Definition 2.10). If these conditions are satisfied, then $\mathcal {K}$ preserves the property of being a self-adjoint (or Hermitian) matrix. In particular $\mathcal {K}(f\otimes \overline {f})$ is self-adjoint, so it has real eigenvalues and orthogonal eigenvectors. (In this paper, the linear-algebraic notions pertaining to $\operatorname {\mathrm {Z}}$ -matrices, including their eigenvalues, are normalized in coherence with the probability Haar measure on $\operatorname {\mathrm {Z}}$ and the corresponding inner-product on the space $\mathbb {C}^{\operatorname {\mathrm {Z}}}$ , a space on which $\operatorname {\mathrm {Z}}$ -matrices act as kernels of linear integral operators; see Definition 2.3 and the discussion preceding it.)

It turns out that the spectral data associated with $\mathcal {K}(f\otimes \overline {f})$ carries important information to perform k-th order Fourier analysis on f. Let us formulate this as the following principle, which captures in general terms a phenomenon that is central to this approach.

Order increment principle: If $K:\mathbb {C}^{\operatorname {\mathrm {Z}}}\to \mathbb {C}^{\operatorname {\mathrm {Z}}}$ maps functions in $\mathbb {C}^{\operatorname {\mathrm {Z}}}$ to their k-th order structured parts, then for $f\in \mathbb {C}^{\operatorname {\mathrm {Z}}}$ the spectral decomposition of $\mathcal {K}(f\otimes \overline {f})$ can be used to obtain the $(k+1)$ -th order structured part (and corresponding useful decomposition) of f.

This principle can be used to turn k-th order Fourier analysis into $(k+1)$ -th order Fourier analysis. A precise form of this principle was observed in the ultralimit setting in [Reference Szegedy59].

Figure 1

Sketch of the spectral approach to higher-order Fourier analysis.

Figure 1 Long description

The diagram consists of two large rounded rectangles labeled function on the left in light green and operator on the right in light red.

In the function panel, the top line shows f maps Z to C. Below this, it states For every t in Z, Delta sub t f maps to K open parenthesis Delta sub t f close parenthesis. This is followed by eigenfunctions f sub 1, f sub 2, and eigenvalues lambda sub 1 is greater than or equal to lambda sub 2. A downward arrow points to two bullet points: regularization of f of order k plus 1, and correlating characters of order k plus 1.

In the operator panel, the top line shows f tensor f-bar maps Z cross Z to C. A downward arrow labeled order k regularization of diagonals points to script K open parenthesis f tensor f-bar close parenthesis maps Z cross Z to C.

Four blue arrows connect the panels. A solid arrow points from the function f to the tensor product in the operator panel. A solid arrow points from the script K operator back to the eigenvalues and eigenfunctions in the function panel. Two dotted arrows form a cross-pattern in the center, connecting the Delta sub t f expression to both the tensor product and the script K operator.

A simple choice for K is the averaging operator (mapping f to the constant functionFootnote ² $\mathbb {E}_{x\in \operatorname {\mathrm {Z}}} f(x)$ ), which pertains to $0$ -th order Fourier analysis. In this case the eigenvectors of $\mathcal {K}(f\otimes \overline {f})$ are the Fourier characters of f, and the eigenvalues are the squares of the absolute values of the corresponding Fourier coefficients. This illustrates how $0$ -th order Fourier analysis is turned into first order, classical Fourier analysis (we discuss this in more detail in Subsection 2.2).

The next logical step is to progress from first-order to quadratic Fourier analysis using the above principle. This step requires an operator K that extracts the Fourier-structured component of a function f in an adequate way. Finding such an operator is challenging because of a lack of uniqueness. Indeed, while in an appropriate limit setting (such as the setting of [Reference Szegedy59]), there exists a unique, and even linear, operator that isolates the structured part (consisting of conditional expectation relative to an appropriate $\sigma $ -algebra), the counterparts in finite settings exist only in an approximate and nonlinear way. There are various possible choices, many of which depend on additional parameters. In this work, we introduce such an operator with especially useful analytic properties, which we call the Fourier denoising operator, and we develop several tools for its application.

To define this operator, recall first that for a character $\chi $ in the dual group $\widehat {\operatorname {\mathrm {Z}}}$ (where $\operatorname {\mathrm {Z}}$ is a finite abelian group), we define the corresponding Fourier coefficient of f by

(1.1) $$ \begin{align} \widehat{f}(\chi)=\mathbb{E}_{x\in \operatorname{\mathrm{Z}}} f(x)\overline{\chi(x)}. \end{align} $$

Then, for a fixed (typically small) constant $\varepsilon>0$ , our denoising operator is defined by

(1.2) $$ \begin{align} K_{\varepsilon}(f)=\sum_{\chi\in \widehat{\operatorname{\mathrm{Z}}}~:~|\widehat{f}(\chi)|\ge \varepsilon}\tfrac{|\widehat{f}(\chi)|-\varepsilon}{|\widehat{f}(\chi)|}\, \widehat{f}(\chi)\, \chi. \end{align} $$

Note that there is a simpler operator (that we can call the Fourier cut-off operator) which sets Fourier coefficients with absolute value smaller than $\varepsilon $ to $0$ and keeps the remaining terms in the Fourier decomposition unchanged. The main problem with the Fourier cut-off operator is that it is not continuous and has inconvenient properties for calculations. Instead, the operator $K_\varepsilon $ applies the continuous function $x\mapsto \mathrm { ReLU}(x-\varepsilon )$ to the magnitudes of the Fourier coefficients while keeping their phases. Here, ReLU is the function $x\mapsto (x+|x|)/2=\max (x,0)$ known from machine learning. It turns out that $K_\varepsilon $ is a contraction in $L^2$ and has other pleasant properties while being sufficiently similar to the cut-off operator. Related to the operator $K_\varepsilon $ we introduce the operator $\mathcal {K}_\varepsilon :\mathbb {C}^{\operatorname {\mathrm {Z}}\times \operatorname {\mathrm {Z}}}\to \mathbb {C}^{\operatorname {\mathrm {Z}}\times \operatorname {\mathrm {Z}}}$ which applies $K_\varepsilon $ to all $\operatorname {\mathrm {Z}}$ -diagonals of a matrix in $\mathbb {C}^{\operatorname {\mathrm {Z}}\times \operatorname {\mathrm {Z}}}$ . A more detailed discussion of the Fourier denoising operator begins in Subsection 2.5.

1.2 Main results with algorithmic consequences

Let us begin by recalling the formula for the Gowers norms from [Reference Gowers22, Lemma 3.9] (see also [Reference Tao and Vu66, Definition 11.2]). For each integer $k\geq 2$ , the $U^k$ -norm of a function $f\in \mathbb {C}^{\operatorname {\mathrm {Z}}}$ is defined by

(1.3) $$ \begin{align} \|f\|_{U^k}:=\Big(\mathbb{E}_{x,t_1,\dots,t_k\in \operatorname{\mathrm{Z}}}~ \prod_{v\in\{0,1\}^k}\mathcal{C}^{|v|}f(x+v(1)\, t_1+v(2)\,t_2+\cdots+v(k)\,t_k)\Big)^{1/2^k}, \end{align} $$

where $\mathcal {C}$ denotes the complex conjugation operator and $|v|:=\sum _{i=1}^k v(i)$ .

The dual of the $U^k$ -norm, defined by $\|f\|_{U^k}^*:=\sup _{g\in \mathbb {C}^{\operatorname {\mathrm {Z}}}, \|g\|_{U^k}\leq 1} \mathbb {E}_{x\in \operatorname {\mathrm {Z}}} f(x)\overline {g(x)}$ , is a useful tool to measure the extent to which a function is structured in the sense of $(k-1)$ -th order Fourier analysis, as we will explain in more detail later (see Subsection 2.3). These dual norms have been analyzed previously, notably in the work of Host and Kra [Reference Host and Kra42]. Their earlier work [Reference Host and Kra45] already studied analogous dual objects for uniformity seminorms, and the work of Tao [Reference Tao64] introduced closely related uniform almost-periodicity norms, also mentioned later below. In this paper, we shall use the dual norms especially to define the quantitative and relatively elementary concept of structured function of order $k-1$ (see Definition 2.23).

The next theorem is one of the main results in this paper. It provides a regularity (or decomposition) result for functions in quadratic Fourier analysis, expressing the quadratically structured part of the function in terms of the dominant eigenvalues and corresponding eigenvectors of the appropriate $\operatorname {\mathrm {Z}}$ -matrix. Thus it connects regularity with spectral methods in the quadratic setting. This provides a theoretical basis for Algorithm 1 below.

The conclusion involving the approximating function h, of bounded $U^3$ -dual-norm, can be summarized as saying that $f_{\text {reg}}$ is a structured function of order 2 (see Definition 2.23). Note also that, as mentioned earlier, eigenvalues here are taken in a normalized form (see Definition 2.3).

The calculation of $f_{\text {reg}}$ in Theorem 1.1 is efficient in practice. It involves $|{\operatorname {\mathrm {Z}}}|$ independent applications of the operator $K_\varepsilon $ (which can be executed in parallel) and the computation of dominant eigenvectors and eigenvalues of a single ${\operatorname {\mathrm {Z}}}\times {\operatorname {\mathrm {Z}}}$ matrix. Each application of $K_\varepsilon $ can be done using a fast Fourier transform, a simple coefficient truncation, and an inverse fast Fourier transform. The whole process depends on the original function f defined on ${\operatorname {\mathrm {Z}}}$ , as well as on two positive parameters $\varepsilon $ and $\rho $ . The details of this calculation are outlined in the following pseudocode.

Algorithm 1U³-regularization algorithm

Algorithm 1 Long description

The algorithm is titled Algorithm 1 U super 3 regularization algorithm.

Input line: f maps Z to C, rho and epsilon are elements of R sub greater than 0.

Line 1: M is assigned f tensor product f-bar, which is an element of C to the power of Z times Z.

Line 2: for t element of Z do.

Line 3: M open parenthesis dot plus t, dot close parenthesis is assigned K sub epsilon open parenthesis M open parenthesis dot plus t, dot close parenthesis close parenthesis.

Line 4: end.

Line 5: The sequence of pairs open parenthesis mu sub 1, v sub 1 close parenthesis through open parenthesis mu sub absolute value Z, v sub absolute value Z close parenthesis is assigned Eigendecomposition of M.

Output line: f sub reg is assigned the summation over mu sub i greater than or equal to rho of the inner product of f and v sub i times v sub i, and the set of pairs open parenthesis mu sub 1, v sub 1 close parenthesis through open parenthesis mu sub absolute value Z, v sub absolute value Z close parenthesis cubed.

Remark 1.5 (Continuous versions).

Note that the output $H_{\varepsilon ,\rho }(f)$ of Algorithm 1 is not continuous in its input $f\in \mathbb {C}^{\operatorname {\mathrm {Z}}}$ for fixed positive $\rho ,\varepsilon $ . To see this, fix some $\rho ,\varepsilon $ and let $\operatorname {\mathrm {Z}}= \mathbb {Z}_p$ for some prime p, $g(x):=e^{2\pi i x^2/p}$ , and $c\in \mathbb {R}_{\ge 0}$ . It is easy to see that $\mathcal {K}_\varepsilon ((cg)\otimes \overline {(cg)})=[\sqrt {c^2-\varepsilon })g]\otimes \overline {[\sqrt {c^2-\varepsilon })g]}$ for $c^2>\varepsilon $ , and $0$ otherwise, so $H_{\varepsilon ,\rho }(cg(x))=(\sqrt {c^2-\varepsilon })g$ if $\rho <c^2-\varepsilon $ and $0$ otherwise. There is therefore a discontinuity in the output $H_{\varepsilon ,\rho }(cg)$ as the parameter c in the input $cg$ increases past $\sqrt {\rho +\varepsilon }$ .

There exist continuous variants of $H_{\varepsilon ,\rho }$ , which are worth mentioning since these variants can be convenient for instance in computer experiments. An example of a continuous version is $H^{\prime }_{\varepsilon ,\rho }(f):=\mathbb {E}_{\rho '\in [\rho /2,\rho ]}H_{\varepsilon ,\rho '}(f)$ . It is easy to see that $H^{\prime }_{\varepsilon ,\rho }$ can also be computed efficiently, with complexity similar to $H_{\varepsilon ,\rho }$ . To achieve this, the formula for the projection in $H_{\varepsilon ,\rho }$ must be augmented with certain weight terms that depend on the eigenvalues. Recall that the Fourier cut-off operator mentioned earlier suffered from a similar continuity problem, and was replaced by the continuous denoising operator $K_\varepsilon $ to obtain our proofs in the quadratic setting. Besides producing more stable and precise outcomes, continuous versions also appear to be more suitable when moving to higher orders via the order increment principle. These and other refinements of the algorithm lie outside the scope of this paper and will be explored in future work.

Figure 2

Removing added random noise from a quadratic structured function by a version of Algorithm 1 on the cyclic group $\mathbb {Z}_{500}$ . A window of length $50$ is plotted for illustration. In this example, the function $f(i):=\sin (8i^2+3i+1)$ , $i\in [500]$ (green graph) is perturbed by random noise, resulting in the function $g=f+r$ (red graph). The spectral algorithm is applied to g and the reconstruction $f_2$ of f (blue graph) is obtained by the projection of g to the space spanned by the $6$ leading eigenvectors of the operator constructed from g. The plot highlights the reconstruction error $|f(i)-f_2(i)|$ .

Figure 2 Long description

A line graph plotted on a vertical axis ranging from negative 1 to 1. At the top, a legend identifies three series. Green circles represent function f. Red circles represent f plus noise. Blue circles represent the reconstruction of f. The horizontal axis represents a window of 50 points from a cyclic group of 500. The red line shows high-frequency random fluctuations. The green line shows the underlying smooth quadratic structure. The blue line closely tracks the green line, demonstrating the effectiveness of the spectral algorithm. Black vertical bars between the green and blue points indicate the reconstruction error. Purple boxes highlight specific data points where the error is less than 0.05 (light purple) or less than 0.01 (dark purple). The reconstruction error is most visible at the peaks and troughs of the noisy red signal but remains small relative to the noise amplitude.

A deeper and more sophisticated version of our spectral algorithm focuses on the meaning of the individual eigenvectors of the operator $\mathcal {K}_\varepsilon (f\otimes \overline {f})$ . It turns out that these eigenvectors can give further information on the quadratic structured part of f. In particular, if the corresponding eigenvalue is large enough and well-separated from other eigenvalues, such an eigenvector behaves as a quadratic generalization of a Fourier character, and has large inner product with f. This generalization is an interesting notion in itself and can be defined in a basic way. The initial observation for this is that a classical Fourier character $\chi $ (or a constant multiple of it) is defined by the property that for every t the multiplicative derivatives $\Delta _t\chi $ is a constant function, and that constant functions are the structured functions in $0$ -th order Fourier analysis (this is proved, in greater generality, in Lemma 2.26). Going one degree higher, we obtain the following basic notion: a function $f\in \mathbb {C}^{\operatorname {\mathrm {Z}}}$ is a quadratic character if for every $t\in \operatorname {\mathrm {Z}}$ the multiplicative derivative $\Delta _t f$ is Fourier structured (of first order) in the sense of Definition 2.23. This notion of quadratic character depends on two parameters that measure the extent to which the multiplicative derivatives are Fourier structured; more precisely f is a quadratic character of complexity R and precision $\delta $ if for every t there is $g_t\in \mathbb {C}^{\operatorname {\mathrm {Z}}}$ such that $\|\Delta _t f -g_t\|_{L^2}\leq \delta $ and $\|g_t\|_{U^2}^*\leq R$ . Let us mention straightaway that a central result in this paper connects this basic notion of a quadratic character with the 2-step case of the deeper notion of nilspace characters mentioned earlier (we discuss this central result in the next subsection). We also introduce basic notions of characters of order k for each $k\geq 2$ , studied in Subsection 2.4 (see Definition 2.29).

It turns out that the quadratically structured part of f given by Theorem 1.1 can be decomposed in terms of quadratic characters of bounded complexity, and that our second algorithm is able to find (good approximations of) those characters. To state our main result validating this second algorithm, we use the following terms. For a self-adjoint $\operatorname {\mathrm {Z}}$ -matrix A, we denote by $\mathrm{Spec}_{\rho }(A)$ the set of eigenvalues of A larger than $\rho $ , and we denote by $\text {Eigen}_\rho (A)\subset \mathbb {C}^{\operatorname {\mathrm {Z}}}$ the subspace spanned by eigenvectors of A corresponding to eigenvalues in $\mathrm{Spec}_{\rho }(A)$ . We also say that a set $X\subset \mathbb {C}$ is $\delta $ -separated if for every $x,y\in X$ with $x\not =y$ we have $|x-y|\ge \delta $ . We can now state the main result in question.

A typical situation enabling a simple application of Theorem 1.6 is when the 1-bounded function $f:\operatorname {\mathrm {Z}}\to \mathbb {C}$ has the largest eigenvalues of $\mathcal {K}_\varepsilon (f\otimes \overline {f})$ sufficiently separated. Then, taking h to be the structured part $f_{\textrm {reg}}$ given byFootnote ⁴ Theorem 1.1, the spectral assumptions in Theorem 1.6 are satisfied, and we can then directly obtain the dominant quadratic characters of f from the eigenvectors associated with those large eigenvalues (see Lemma 7.18). This situation is actually generic, in a sense that we can use in order to handle the cases where those spectral assumptions might fail. Indeed, in such cases, we show that one can use a randomized procedure, consisting essentially in calculating the dominant eigenvectors and eigenvalues of $\mathcal {K}_\varepsilon (h\otimes \overline {h})$ where h is a random combination of the dominant eigenvectors of $\mathcal {K}_\varepsilon (f\otimes \overline {f})$ . We show that, with high probability, this procedure yields a function h to which Theorem 1.6 applies, thus resulting in a decomposition of the structured part of f into quadratic characters (see Subsection 7.2.1). The formalization of this probabilistic procedure is given in Theorem 7.24 and Remark 7.25. Theorems 1.6 and 7.24 together validate Algorithm 2 below.

Algorithm 2Quadratic character decomposition

Algorithm 2 Long description

The pseudocode begins with Input: f maps Z to C, and rho, epsilon, delta are elements of R greater than 0.

Line 1: f sub reg, open parenthesis mu sub 1, v sub 1 close parenthesis, ..., open parenthesis mu sub absolute value of Z, v sub absolute value of Z close parenthesis is assigned from Algorithm 1 with parameters f, rho, epsilon.

Line 2: S is assigned the maximum i such that mu sub i is greater than or equal to rho.

Line 3: If IsSeparated with parameters delta and the set mu sub 1 greater than or equal to ... greater than or equal to mu sub S then

Line 4: h is assigned f sub reg.

Line 5: else

Line 6: h is assigned RandomUnitVector with parameters v sub 1, ..., v sub S.

Line 7: end.

Line 8: f prime sub reg, open parenthesis mu prime sub 1, v prime sub 1 close parenthesis, ..., open parenthesis mu prime sub absolute value of Z, v prime sub absolute value of Z close parenthesis is assigned from Algorithm 1 with parameters h, delta, epsilon.

Line 9: S prime is assigned the maximum i such that mu prime sub i is greater than or equal to delta.

Output: The set of v prime sub i such that i is an element of S, and the boolean result of IsSeparated with parameters delta and the set mu prime sub 1 greater than or equal to ... greater than or equal to mu prime sub S prime, and absolute value of S equals absolute value of S prime.

For a 1-bounded function $f:\operatorname {\mathrm {Z}}\to \mathbb {C}$ , Algorithm 2 finds a decomposition of f into quadratic characters (the $ v_i'$ in the first output) if the second output equals True (which happens with high probability under appropriate choices of parameters, e.g., $\delta =\rho ^7$ as in Theorem 1.6). The following auxiliary routines are used: $\mathbf {IsSeparated}(\delta ,\{\mu _1\ge \cdots \ge \mu _S\})$ , which returns true only if $|\mu _i-\mu _{i+1}|\ge \delta $ for all i, and $\mathbf {RandomUnitVector}(v_1,\ldots ,v_S)$ , which returns a uniformly chosen random unit vector in the subspace $\langle v_1,\ldots ,v_S\rangle $ with respect to the $L^2$ norm.

1.3 Nilspace theoretic foundations for the spectral method

From Section 4 onwards, our main proofs rely strongly on the nilspace approach to higher-order Fourier analysis, which is summarized in Figure 3. Nilspaces are algebraic structures that emerge naturally from the large scale behavior of functions on abelian groups relative to the Gowers norms. Despite having been introduced relatively recently [Reference Camarena and Szegedy5], nilspaces are treated in various works (see in particular [Reference Candela6, Reference Candela7, Reference Candela and Szegedy12, Reference Gutman, Manners and Varjú37, Reference Gutman, Manners and Varjú38, Reference Gutman, Manners and Varjú39]). We shall recall some basic background on nilspace theory in Section 4.

Figure 3

The nilspace approach to k-th order Fourier analysis.

The nilspace approach yields a regularity theorem for the Gowers norms which holds in particular on any finite abelian group, and which is a key ingredient in this paper [Reference Candela and Szegedy12, Theorem 1.5]. According to this theorem, a 1-bounded function f on a finite abelian group $\operatorname {\mathrm {Z}}$ has a structured part of order k that is a so-called k-step nilspace polynomial, that is, a function of the form $F\operatorname {\mathrm {\circ }}\phi $ , where $\phi :\operatorname {\mathrm {Z}}\to \operatorname {\mathrm {X}}$ is a nilspace morphism from $\operatorname {\mathrm {Z}}$ (viewed as a 1-step nilspace) into a k-step nilspace $\operatorname {\mathrm {X}}$ which is compact and of finite rank (in particular $\operatorname {\mathrm {X}}$ is a finite-dimensional manifold), and F is a 1-bounded Lipschitz complex-valued function on $\operatorname {\mathrm {X}}$ (see Definition 4.1). The pair $(\operatorname {\mathrm {X}},F)$ is in some sense a compressed version of $(\operatorname {\mathrm {Z}},f)$ which contains information on the k-th order structure of f at a given “resolution”.

In this paper we refine this regularization technique by combining it with a generalized Fourier decomposition of F. To this end, we use the fact that the compact and finite-rank (cfr) nilspace $\operatorname {\mathrm {X}}$ has the structure of an iterated principal abelian bundle, with structure groups $\operatorname {\mathrm {Z}}_1(\operatorname {\mathrm {X}}),\operatorname {\mathrm {Z}}_2(\operatorname {\mathrm {X}}),\dots ,\operatorname {\mathrm {Z}}_k(\operatorname {\mathrm {X}})$ which are compact abelian Lie groups (see [Reference Candela7, Proposition 2.1.9]). The last structure group $\operatorname {\mathrm {Z}}_k(\operatorname {\mathrm {X}})$ acts on $\operatorname {\mathrm {X}}$ freely by nilspace automorphisms. This can be used to decompose a Lipschitz function $F:\operatorname {\mathrm {X}}\to \mathbb {C}$ uniquely as a series $F=\sum _{\chi \in \widehat {\operatorname {\mathrm {Z}}_k(\operatorname {\mathrm {X}})}}F_\chi $ converging uniformly. This yields the decomposition of the structured part of f as

(1.4) $$ \begin{align} F\operatorname{\mathrm{\circ}}\phi = \sum_{\chi\in\widehat{\operatorname{\mathrm{Z}}_k(\operatorname{\mathrm{X}})}}F_\chi\operatorname{\mathrm{\circ}}\phi. \end{align} $$

Our main theoretical results describe how this additional decomposition of the structured part further enhances the framework provided by the nilspace regularity theorem [Reference Candela and Szegedy12, Theorem 1.5], especially when this decomposition is combined with the strong equidistribution property of $\phi $ (known as balance) given by this theorem. Below, we summarize the main results that we obtain in this direction, and their connection with our spectral approach.

Nilspace characters. In Section 4 we introduce the generalizations of Fourier characters mentioned previously in this introduction, namely nilspace characters, and we describe their fundamental properties. Nilspace characters are basically the functions of the form $F_\chi \operatorname {\mathrm {\circ }}\phi $ obtained in the decomposition (1.4) (for their formal description see Definition 4.5). These functions provide a nilspace theoretic generalization of Fourier characters, valid on any finite abelian group. In the specific setting of cyclic groups, they are closely related (though not identical) to the notion of nilcharacter from [Reference Green, Tao and Ziegler35]. A central task in our study of nilspace characters is to connect this notion with the more elementary notion of a character of order k mentioned in the previous subsection (and formalized in Definition 2.29). Note that both notions are parametric (i.e., they depend on additional parameters, such as a complexity parameter). This is analogous to other concepts, such as being a “structured” or “quasirandom” function, which are also parametric, and become exact (i.e., parameter-free) only in a limit setting (see for instance [Reference Szegedy58, Reference Szegedy59, Reference Szegedy60]). A central result in Section 4 is Theorem 4.19, which focuses on the quadratic case and establishes, informally speaking, that 2-step nilspace characters are quadratic characters under an appropriate transformation of their parameters.

Decompositions with nilspace characters. In Section 5 we prove a refined version of the nilspace regularity theorem which implements the additional decomposition (1.4) of the structured part as a sum of nilspace characters; see Theorem 5.1. Moreover, using the balance property of the underlying morphism $\phi $ , this theorem guarantees that the nilspace characters satisfy approximate orthogonality properties which are crucial for our purposes. This new formulation of the nilspace regularity theorem also brings higher-order Fourier analysis more in line with classical Fourier analysis, in particular by yielding a new result constituting a generalization of Parseval’s identity (see Theorem 5.7). A result of this kind has been called for in several works as a desirable tool for higher-order Fourier analysis (see in particular [Reference Gowers25, Section 16], and [Reference Host and Kra42, end of Section 1]). Another new result of this type is an approximate diagonalization formula for the Gowers norms in terms of nilspace characters (see Theorem 5.4).

The bridge between spectral decompositions and higher-order characters. In Section 6, we prove Theorem 6.1, which links the spectral properties of the operators $\mathcal {K}_\varepsilon (f\otimes \overline {f})$ with the quadratic nilspace character components of f given by Theorem 5.1. An informal version of this theorem asserts that

$$ \begin{align*}\mathcal{K}_\varepsilon(f\otimes\overline{f})=\sum_{\chi\in S}g_\chi\otimes\overline{g_\chi}+E\end{align*} $$

where $\varepsilon $ is a small, appropriately chosen number, the functions $g_\chi :=F_\chi \operatorname {\mathrm {\circ }}\phi $ are the dominant 2-step nilspace character components of f for some finite set S, and the matrix E is a small error. In particular, by the orthogonality properties of the functions $g_\chi $ , this theorem implies that each $g_\chi $ is a so-called pseudoeigenvector of $\mathcal {K}_\varepsilon (f\otimes \overline {f})$ (see Definition 7.1). As a consequence, one obtains that the corresponding pseudoeigenvalue is close to some proper eigenvalue $\lambda $ . Furthermore, if $\lambda $ is well separated from other eigenvalues, then the corresponding eigenvector v is close to $g_\chi $ . This phenomenon yields a direct connection between the eigenvectors of $\mathcal {K}_\varepsilon (f\otimes \overline {f})$ and the nilspace characters $g_\chi $ .

1.4 Connection with the ultralimit setting

To close this introduction, let us mention that the spectral approach to higher-order Fourier analysis is deeply motivated by an analogous construction in the ultralimit setting [Reference Szegedy58, Reference Szegedy59, Reference Szegedy60], which, on one hand, is more exact (not depending on various parameters), and on the other hand is intrinsically qualitative (in particular not yielding effective bounds). This construction involves measurable functions on ultraproducts of abelian groups. In particular, the Gowers norms become seminorms on such groups. In general, a major benefit of this ultralimit approach is its simpler and more algebraic language. This is illustrated by the following facts that hold in this setting: the notion of noise of order $k-1$ becomes exact, namely, it becomes the property of having $U^k$ -seminorm equal to 0; then, every bounded measurable function f can be uniquely decomposed as $f = f_s + f_r$ , where $f_r$ is noise of order $k-1$ and $f_s$ is a structured function of order $k-1$ , that is, is orthogonal to any noise function of order $k-1$ . Moreover, it can be proved that there is a sub- $\sigma $ -algebra $\mathcal {F}_{k-1}$ on any such group such that a function g is $\mathcal {F}_{k-1}$ -measurable if and only if g is structured of order $k-1$ . Hence, the structured part $f_s$ can be obtained as the conditional expectation $\mathbb {E}(f|\mathcal {F}_{k-1})$ relative to the $\sigma $ -algebra $\mathcal {F}_{k-1}$ . In this setting, the spectral approach simplifies to applying the map $g \mapsto \mathbb {E}(g|\mathcal {F}_{k-1})$ to each $\operatorname {\mathrm {Z}}$ -diagonal function of $f \otimes \overline {f}$ , yielding the unique self-adjoint operator $M(f,k)$ .

Quite surprisingly, and nontrivially, the eigenvectors of $M(f,k)$ are structured functions of order k and their multiplicative derivatives are structured of order $k-1$ (see [Reference Szegedy58, Theorem 2] and [Reference Szegedy59]). This constitutes a limit-setting analogue of the notion of k-th order character introduced in this paper (Definition 2.29). Another useful property of $M(f,k)$ is that the k-th order structured part of f is equal to the projection of f onto the space spanned by the eigenvectors of $M(f,k)$ with nonzero eigenvalues. The resulting decomposition of the structured part of f into these eigenvectors is a higher-order generalization of the ordinary Fourier transform.

The ultralimit setting serves as a valuable framework for envisioning further results in higher-order Fourier analysis and deriving them at a qualitative level. This will be explored in future work in connection with the spectral approach, especially in the general k-th order case. On the other hand, for the aim of making higher-order Fourier analysis practical, it becomes vital to introduce useful notions of a finite (usually parametric) type, and in this effort one is also aided by the deeper insights provided by the structures underlying higher-order decompositions, such as nilmanifolds and, more broadly, nilspaces. Hence this paper’s focus on developing new connections between these structures, spectral properties, and higher-order decompositions.