2 Extension of representations

Let G be a locally compact group which we assume to be second countable. We will need the notion of a projective representation. Recall that a mapping $\pi : G \to U(\mathcal H)$ from G to the unitary group of the Hilbert space $\mathcal H$ is a projective representation of G if the following assertions hold.

• • $\pi (e)=I$ .

• • For all $g_1,g_2\in G,$ there exists $c(g_1 , g_2 )\in \mathbf C $ such that

  $$ \begin{align*} \pi(g_1 g_2 ) = c(g_1 , g_2 )\pi(g_1 )\pi(g_2). \end{align*} $$

• • The function $g\mapsto \langle \pi (g)\xi ,\eta \rangle $ is measurable for all $\xi ,\eta \in \mathcal H.$

The mapping $c:G \times G \to {\mathbf S}^1$ is a $2$ -cocycle with values in the unit circle ${\mathbf S}^1.$ Every projective unitary representation of G can be lifted to an ordinary unitary representation of a central extension of $G $ (for all this, see [Reference MackeyMack76] or [Reference MackeyMack58]).

Let N be a closed normal subgroup of G. Let $\pi $ be an irreducible unitary representation of N on a Hilbert space $\mathcal H.$ Consider the stabilizer

$$ \begin{align*} G_{\pi}=\{ g\in G\mid \pi^g \text{ is equivalent to } \pi\} \end{align*} $$

of $\pi $ in G for the natural action of G on the unitary dual $\widehat {N}$ given by $\pi ^g(n)= \pi (g^{-1}n g).$ Then $G_\pi $ is a closed subgroup of G containing $N.$ The following lemma is a well-known part of Mackey’s theory of unitary representations of group extensions.

Proof. For every $g\in {G}_\pi $ , there exists a unitary operator $\widetilde \pi (g)$ on $\mathcal H$ such that

$$ \begin{align*} \pi (g(n))= \widetilde\pi(g) \pi(n) \widetilde\pi(g)^{-1} \quad\text{for all } n\in N. \end{align*} $$

One can choose $\widetilde \pi (g)$ such that $g\mapsto \widetilde \pi (g)$ is a projective unitary representation of ${ G}_\pi $ which extends $\pi $ (see [Reference MackeyMack58, Theorem 8.2]). The uniqueness of $\pi $ follows from the irreducibility of $\pi $ and Schur’s lemma.

3 A weak containment result for induced representations

Let G be a locally compact group with Haar measure $\mu _G.$ Recall that a unitary representation $(\rho , \mathcal K)$ of G is weakly contained in another unitary representation $(\pi , \mathcal H)$ of $G,$ if every matrix coefficient

$$ \begin{align*}g\mapsto \langle \rho(g)\eta\mid \eta\rangle \quad \text{for }\eta \in \mathcal K\end{align*} $$

of $\rho $ is the limit, uniformly over compact subsets of $G,$ of a finite sum of matrix coefficients of $\pi $ ; equivalently, if $\Vert \rho (f)\Vert \leq \Vert \pi (f)\Vert $ for every $f\in C_c(G),$ where $C_c(G)$ is the space of continuous functions with compact support on G and where the operator $\pi (f)\in \mathcal B(\mathcal H)$ is defined by the integral

$$ \begin{align*} \pi(f) \xi= \int_G f(g)\pi(g)\xi \,d\mu_G(g) \quad\text{for all } \xi\in \mathcal H. \end{align*} $$

The trivial representation $1_G$ is weakly contained in $\pi $ if and only if there exists, for every compact subset Q of G and every $\varepsilon>0,$ a unit vector $\xi \in \mathcal H$ which is $(Q,\varepsilon )$ -invariant, that is, such that

$$ \begin{align*}\sup_{g\in Q} \Vert \pi(g)\xi- \xi\Vert\leq \varepsilon.\end{align*} $$

Let H be a closed subgroup of $G.$ We will always assume that the coset space $H\backslash G$ admits a non-zero G-invariant (possibly infinite) measure on its Borel subsets. Let $(\sigma ,\mathcal K)$ be a unitary representation of $H.$ We will use the following model for the induced representation $\pi :=\mathrm {Ind}_H^G \sigma $ . Choose a Borel fundamental domain $X\subset G$ for the action of G on $H\backslash G$ . For $x \in X$ and $g\in G,$ let $x\cdot g\in X$ and $c(x,g)\in H$ be defined by

$$ \begin{align*}xg= c(x,g) (x\cdot g).\end{align*} $$

There exists a non-zero G-invariant measure on X for the action $(x,g)\mapsto x\cdot g$ of G on $X.$ The Hilbert space of $\pi $ is the space $L^2(X, \mathcal K, \mu )$ of all square-integrable measurable mappings $\xi : X\to \mathcal K$ and the action of G on $L^2(X, \mathcal K, \mu )$ is given by

$$ \begin{align*} (\pi(g) \xi)(x)= \sigma (c(x,g))(\xi(x\cdot g)),\quad g\in G,\ \xi\in L^2(X, \mathcal K, \mu),\ x\in X. \end{align*} $$

Observe that, in the case where $\sigma $ is the trivial representation $1_H,$ the induced representation $\mathrm {Ind}_H^G1_H$ is equivalent to quasi-regular representation $\unicode{x3bb} _{H\backslash G}$ , that is, the natural representation of G on $L^2(H\backslash G,\mu )$ given by right translations.

We will use several times the following elementary but crucial lemma, which can be viewed as a generalization of Herz’s majoration principle (see [Reference Bekka and HeuBeGu15, Proposition 17]).

Proof. Let Q be a compact subset of G and $\varepsilon>0.$ For every $i\in I,$ let $X_i\subset G$ be a Borel fundamental domain for the action of G on $H_i\backslash G$ and $\mu _i$ a non-zero G-invariant measure on $X_i.$ There exists a family of vectors $\xi _i\in L^2(X_i, \mathcal K_i, \mu _i)$ such that $ \sum _{i}\Vert \xi _i\Vert ^2=1$ and

$$ \begin{align*}\sup_{g\in Q} \sum_{i}\Vert \mathrm{Ind}_{H_i}^G\sigma_i(g)\xi_i- \xi_i\Vert^2\leq \varepsilon.\end{align*} $$

Define $\varphi _i$ in $L^2(X_i, \mu _i)$ by $\varphi _i(x)=\Vert \xi _i(x)\Vert $ . Then $ \sum _{i}\Vert \varphi _i\Vert ^2=1$ and, denoting by $(x,g)\mapsto x\cdot _i g$ the action of G on $X_i$ and by $c_i:X_i\times G\to H_i$ the associated map as above, we have

for every $g\in G,$ and the claim follows.

4 Decay of matrix coefficients of unitary representations

We recall a few general facts about the decay of matrix coefficients of unitary representations, Recall that the projective kernel of a (genuine or projective) representation $\pi $ of the locally compact group G is the closed normal subgroup $P_\pi $ of G consisting of the elements $g\in G$ such that $\pi (g)$ is a scalar multiple of the identity operator, that is, such that $\pi (g)=\unicode{x3bb} _\pi (g) I$ for some $\unicode{x3bb} _\pi (g)\in {\mathbf S}^1.$

Observe also that, for $\xi , \eta \in \mathcal H,$ the absolute value of the matrix coefficient

$$ \begin{align*}C^{\pi}_{\xi,\eta}: g\mapsto \langle \pi(g)\xi,\eta\rangle\end{align*} $$

is constant on cosets modulo $P_\pi .$ For a real number p with $1\leq p <+\infty ,$ the representation $\pi $ is said to be strongly $L^p$ modulo $P_\pi $ , if there is a dense subspace $D\subset \mathcal H$ such that $|C^{\pi }_{\xi ,\eta }|\in L^p(G/P_\pi )$ for all $\xi ,\eta \in D.$

5 The Koopman representation of the automorphism group of a homogeneous space

We establish a decomposition result for the Koopman representation of a group of automorphisms of an S-adic compact nilmanifold. We will state the result in the general context of a compact homogeneous space.

Let G be a locally compact group and $\Lambda $ a lattice in $G.$ We assume that $\Lambda $ is cocompact in $G.$ The homogeneous space $X:= G/\Lambda $ carries a probability measure $\mu $ on the Borel subsets of X which is invariant by translations with elements from $G.$ Every element from

$$ \begin{align*}\mathrm{Aut} (X):= \{\gamma\in \mathrm{Aut}(G)\mid \gamma(\Lambda) \subset \Lambda\} \end{align*} $$

induces a Borel isomorphism of X, which leaves $\mu $ invariant, as follows from the uniqueness of $\mu $ .

Given a subgroup $\Gamma $ of $\mathrm {Aut}(X),$ the following crucial proposition gives a decomposition of the associated Koopman $\Gamma $ on $L^2(X,\mu )$ as direct sum of certain induced representations of $\Gamma $ .

Proposition 9. Let G be a locally compact group and $\Lambda $ a cocompact lattice in $G,$ and let $\Gamma $ be a countable subgroup of $\mathrm {Aut}(X)$ for $X:= G/\Lambda $ . Let $\kappa $ denote the Koopman representation of $\Gamma $ associated to the action $\Gamma \curvearrowright X.$ There exists a family $(\pi _i)_{i\in I}$ of irreducible unitary representations of G such that $\kappa $ is equivalent to a direct sum

$$ \begin{align*}\bigoplus_{i\in I} \mathrm{ Ind}_{\Gamma_i}^{\Gamma}(\widetilde{\pi_i}|_{\Gamma_i}\otimes W_i),\end{align*} $$

where $ \widetilde {\pi }_i$ is an irreducible projective representation of the stabilizer $G_i$ of $\pi _i$ in $\mathrm {Aut}(G)\ltimes G$ extending $\pi _i$ , and where $W_i$ is a finite-dimensional projective unitary representation of $\Gamma _i :=\Gamma \cap G_i$ .

Proof. We extend $\kappa $ to a unitary representation, again denoted by $\kappa ,$ of $\Gamma \ltimes G$ on $L^2(X,\mu )$ given by

$$ \begin{align*} \kappa(\gamma, g) \xi(x)=\xi(\gamma^{-1}(g x)) \quad\text{for all }\gamma\in\Gamma, g\in G, \xi\in L^2(X,\mu), \ x\in X. \end{align*} $$

Identifying $\Gamma $ and G with subgroups of $\Gamma \ltimes G$ , we have

(*) $$ \begin{align} \kappa({\gamma^{-1}}) \kappa(g) \kappa({\gamma})= \kappa({\gamma^{-1}(g)}) \quad\text{for all } \gamma\in \Gamma,\ g\in G. \end{align} $$

Since $\Lambda $ is cocompact in $G,$ we can consider the decomposition of $L^2(X,\mu )$ into G-isotypical components: we have (see [Reference Gelfand, Graev and Pyatetskii-ShapiroGGPS69, Ch. I, §3, Theorem])

$$ \begin{align*} L^2(X,\mu)=\bigoplus_{\pi\in\Sigma} \mathcal H_{\pi}, \end{align*} $$

where $\Sigma $ is a certain set of pairwise non-equivalent irreducible unitary representations of G; for every $\pi \in \Sigma $ , the space $\mathcal H_{\pi }$ is the union of the closed $\kappa (G)$ -invariant subspaces $\mathcal K$ of $\mathcal H:=L^2(X,\mu )$ for which the corresponding representation of G in ${\mathcal K}$ is equivalent to $\pi $ ; moreover, the multiplicity of every $\pi $ is finite, that is, every $\mathcal H_{\pi }$ is a direct sum of finitely many irreducible unitary representations of G.

Let $\gamma $ be a fixed automorphism in $\Gamma .$ Let $\kappa ^\gamma $ be the conjugate representation of $\kappa $ by $\gamma ,$ that is, $ \kappa ^\gamma (g)=\kappa (\gamma g\gamma ^{-1})$ for all $g\in \Gamma \ltimes G$ . On the one hand, for every $\pi \in \Sigma ,$ the isotypical component of $\kappa ^\gamma \vert _{G}$ corresponding to $\pi $ is $\mathcal H_{\pi ^{\gamma ^{-1}}}$ . On the other hand, relation $(*)$ shows that $\kappa (\gamma )$ is a unitary equivalence between $\kappa \vert _{G} $ and $\kappa ^\gamma \vert _{G}.$ It follows that

$$ \begin{align*} \kappa({\gamma}) (\mathcal H_{\pi}) = \mathcal H_{\pi^{\gamma}}\quad\text{for all } \gamma\in \Gamma; \end{align*} $$

so, $\Gamma $ permutes the $\mathcal H_\pi $ s among themselves according to its action on $\widehat {G}.$

Write $\Sigma =\bigcup _{i\in I} \Sigma _i,$ where the $\Sigma _i$ are the $\Gamma $ -orbits in $\Sigma ,$ and set

$$ \begin{align*} \mathcal H_{\Sigma_i}=\bigoplus_{\pi\in\Sigma_i} \mathcal H_{\pi}. \end{align*} $$

Every $\mathcal H_{\Sigma _i}$ is invariant under $\Gamma \ltimes G$ and we have an orthogonal decomposition

$$ \begin{align*} \mathcal H= \bigoplus_{i} \mathcal H_{\Sigma_i}. \end{align*} $$

Fix $i\in I.$ Choose a representation ${\pi }_i $ in $\Sigma _i$ and set $\mathcal H_i= \mathcal H_{\pi _i}.$ Let $\Gamma _i$ denote the stabilizer of $\pi _i$ in $\Gamma .$ The space $\mathcal H_i$ is invariant under $\Gamma _i.$ Let $V_i$ be the corresponding representation of $ \Gamma _i\ltimes G$ on $\mathcal H_i.$

Choose a set $S_i$ of representatives for the cosets in

$$ \begin{align*}\Gamma/\Gamma_i= (\Gamma\ltimes G)/ (\Gamma_i\ltimes G)\end{align*} $$

with $e\in S_i.$ Then $\Sigma _i=\{ \pi _i^s : s\in S_i\}$ and the Hilbert space $\mathcal H_{\Sigma _i}$ is the sum of mutually orthogonal spaces:

$$ \begin{align*} \mathcal H_{\Sigma_i}= \bigoplus_{s\in S_i}\mathcal H_i^s. \end{align*} $$

Moreover, $\mathcal H_i^s$ is the image under $\kappa (s)$ of $\mathcal H_i$ for every $s\in S_i.$ This means that the restriction $\kappa _i$ of $\kappa $ to $\mathcal H_{\Sigma _i}$ of the Koopman representation $\kappa $ of $\Gamma $ is equivalent to the induced representation $\mathrm {Ind}_{\Gamma _i}^{\Gamma } {V_i}.$

Since every $\mathcal H_{i}$ is a direct sum of finitely many irreducible unitary representations of $G,$ we can assume that $\mathcal H_i$ is the tensor product

$$ \begin{align*} \mathcal H_i =\mathcal K_i\otimes \mathcal L_i \end{align*} $$

of the Hilbert space $\mathcal K_i$ of $\pi _i$ with a finite-dimensional Hilbert space $\mathcal L_i,$ in such a way that

(**) $$\begin{align} V_i(g)= \pi_{i}(g) \otimes I_{\mathcal L_i} \quad\text{for all } g\in G.\end{align}$$

Let $\gamma \in \Gamma _i.$ By $(*)$ and $(**)$ above, we have

(***) $$\begin{align} V_i(\gamma) (\pi_{i}(g) \otimes I_{\mathcal L_i})V_i(\gamma) ^{-1} = \pi_{i}(\gamma g\gamma^{-1}) \otimes I_{\mathcal L_i} \end{align} $$

for all $g\in G.$ On the other hand, let $G_i$ be the stabilizer of $\pi _i$ in $\mathrm {Aut}(G)\ltimes G$ ; then $\pi _i$ extends to an irreducible projective representation $ \widetilde {\pi }_i$ of $G_i$ (see §2). Since

$$ \begin{align*} \widetilde{\pi_i}(\gamma) \pi_{i}(g)\ \widetilde{\pi_i}(\gamma^{-1})= \pi_{i}(\gamma g\gamma^{-1}) \quad\text{for all } g\in G, \end{align*} $$

it follows from $(***)$ that $(\widetilde {\pi _i}(\gamma ^{-1})\otimes I_{\mathcal L_i})V_i(\gamma )$ commutes with $\pi _i(g)\otimes I_{\mathcal L_i}$ for all $g\in G.$ As $\pi _i$ is irreducible, there exists a unitary operator $W_i(\gamma )$ on $\mathcal L_i$ such that

$$ \begin{align*} V_i(\gamma)= \widetilde{\pi_i}(\gamma)\otimes W_i(\gamma). \end{align*} $$

It is clear that $W_i$ is a projective unitary representation of $\Gamma _i\ltimes G$ , since $V_i$ is a unitary representation of $\Gamma _i\ltimes G$ .

6 Unitary dual of solenoids

Let p be either a prime integer or $p=\infty .$ Define an element $e_p$ in the unitary dual group $\widehat {\mathbf Q_p}$ of the additive group of $\mathbf Q_p$ (recall that $\mathbf Q_\infty = {\mathbf R}$ ) by $e_p(x)= e^{2\pi i x}$ if $p=\infty $ and $e_p(x)= \exp (2\pi i \{x\})$ otherwise, where $\{x\}= \sum _{j=m}^{-1} a_j p^j $ denotes the ‘fractional part’ of a p-adic number $x= \sum _{j=m}^\infty a_j p^j$ for integers $m\in {\mathbf Z}$ and $a_j \in \{0, \ldots , p-1\}$ . Observe that $\mathrm {Ker} (e_p)={\mathbf Z}$ if $p=\infty $ and that $\mathrm {Ker} (e_p)={\mathbf Z}_p$ if p is a prime integer, where ${\mathbf Z}_p$ is the ring of p-adic integers. The map

$$ \begin{align*} \mathbf Q_p \to \widehat{\mathbf Q_p}, \quad y\mapsto (x\mapsto e_p(xy))\end{align*} $$

is an isomorphism of topological groups (see [Reference Bekka, de la Harpe and ValetteBeHV08, §D.4]).

Fix an integer $d\geq 1.$ Then $\widehat {\mathbf Q_p^d}$ will be identified with $\mathbf Q_p^d$ by means of the map

$$ \begin{align*}\mathbf Q_p^d \to \widehat{\mathbf Q_p^d}, \quad y\mapsto x\mapsto e_p( x\cdot y),\end{align*} $$

where $x\cdot y= \sum _{i=1}^d x_i y_i$ for $x=(x_1, \ldots , x_d), y=(y_1, \ldots , y_d)\in \mathbf Q_p^d.$

Let $S= \{p_1, \ldots , p_r, \infty \}$ , where $p_1, \ldots , p_r$ are integer primes. For an integer $d\geq 1$ , consider the S-adic solenoid

$$ \begin{align*}\mathbf{Sol}_S=\mathbf Q_S^d/{\mathbf Z}[1/S]^d,\end{align*} $$

where ${\mathbf Z}[1/S]^d={\mathbf Z}[1/p_1, \ldots , 1/p_r]^d$ is embedded diagonally in $\mathbf Q_S= \prod _{p\in S} \mathbf Q_p.$ Then $\widehat {\mathbf {Sol}_S}$ is identified with the annihilator of ${\mathbf Z}[1/S]^d$ in $\mathbf Q_S^d,$ that is, with ${\mathbf Z}[1/S]^d$ embedded in $\mathbf Q_S^d$ via the map

$$ \begin{align*} {\mathbf Z}[1/S]^d\to \mathbf Q_S^d, \quad b\mapsto (b, -b, \cdots,- b). \end{align*} $$

Under this identification, the dual action of the automorphism group

$$ \begin{align*} \mathrm{Aut}(\mathbf Q_S^d)\cong GL_d({\mathbf R})\times GL_d(\mathbf Q_{p_1})\times\cdots\times GL_d(\mathbf Q_{p_r}). \end{align*} $$

on $\widehat {\mathbf Q_S^d}$ corresponds to the right action on ${\mathbf R}^d\times \mathbf Q_{p_1}^d\times \cdots \times \mathbf Q_{p_r}^d$ given by

$$ \begin{align*} ((g_\infty, g_1, \ldots, g_r), (a_\infty, a_1, \ldots, a_r))\mapsto (g_\infty^t a_\infty , g_1^t a_1, \ldots, g_r^t a_r), \end{align*} $$

where $(g,a)\mapsto ga$ is the usual (left) linear action of $GL_d(\mathbf {k})$ on $\mathbf {k}^d$ for a field $\mathbf {k}.$

7 Unitary representations of unipotent groups

Let $\mathbf {U}$ be a linear algebraic unipotent group defined over $\mathbf Q$ . The Lie algebra $\mathfrak {u}$ is defined over $\mathbf Q$ and the exponential map $\exp : \mathfrak {u}\to U$ is a bijective morphism of algebraic varieties.

Let p be either a prime integer or $p=\infty .$ The irreducible unitary representations of $U_p:=\mathbf {U}(\mathbf Q_p)$ are parametrized by Kirillov’s theory as follows.

The Lie algebra of $U_p$ is $\mathfrak {u}_p=\mathfrak {u}(\mathbf Q)\otimes _{\mathbf Q} \mathbf {Q}_p,$ where $\mathfrak {u}(\mathbf Q)$ is the Lie algebra over $\mathbf Q$ consisting of the $\mathbf Q$ -points in $\mathfrak u.$

Fix an element f in the dual space ${\mathfrak u}_p^*= {\mathcal Hom}_{\mathbf Q_p}({\mathfrak u}_p, \mathbf Q_p)$ of $\mathfrak {u}_p.$ There exists a polarization $\mathfrak m$ for $f,$ that is, a Lie subalgebra $\mathfrak m$ of ${\mathfrak u}_p$ such that $f([{\mathfrak m},{\mathfrak m}])=0$ and which is of maximal dimension. The induced representation $\mathrm {Ind}_M^{U_p} \chi _f$ is irreducible, where $M=\exp (\mathfrak m)$ and $\chi _f$ is the unitary character of M defined by

$$ \begin{align*}\chi_f(\exp X)=e_p(f(X)) \quad\text{for all } X \in {\mathfrak m},\end{align*} $$

where $e_p\in \widehat {\mathbf Q_p}$ is as in §6. The unitary equivalence class of $\mathrm {Ind}_M^{U_p} \chi _f$ only depends on the co-adjoint orbit $\mathrm {Ad}^* (U_p) f$ of f. The resulting map

$$ \begin{align*} {\mathfrak u}_p^*/\mathrm{Ad}^* (U_p)\to \widehat{U_p},\quad {\mathcal O}\mapsto \pi_{\mathcal O}, \end{align*} $$

called the Kirillov map, from the orbit space ${\mathfrak u}_p^*/\mathrm {Ad}^*(U_p)$ of the co-adjoint representation to the unitary dual $\widehat {U}_p$ of $U_p$ , is a bijection. In particular, $U_p$ is a so-called type I locally compact group. For all of this, see [Reference KirillovKiri62] or [Reference Corwin and GreenleafCoGr89] in the case of $p=\infty $ and [Reference MooreMoor65] in the case of a prime integer p.

The group $\mathrm {Aut} (U_p)$ of continuous automorphisms of $U_p$ can be identified with the group of $\mathbf Q_p$ -points of the algebraic group $\mathrm {Aut} (\mathfrak u)$ of automorphisms of the Lie algebra $\mathfrak u$ of $\mathbf {U}.$ Notice also that the natural action of $\mathrm {Aut}(U_p)$ on ${\mathfrak u}_p$ as well as its dual action on ${\mathfrak u}_p^*$ are algebraic.

Let $\pi \in \widehat {U_p}$ with corresponding Kirillov orbit $\mathcal {O}_\pi $ and $g\in \mathrm {Aut}(U_p).$ Then $g(\mathcal {O}_\pi )$ is the Kirillov orbit associated to the conjugate representation $\pi ^g.$

8 Decay of matrix coefficients of unitary representations of S-adic groups

Let p be an integer prime or $p=\infty $ and let $\mathbf {U}$ be a linear algebraic unipotent group defined over $\mathbf Q_p$ . Set $U_p:=\mathbf {U}(\mathbf Q_p).$

Let $\pi $ be an irreducible unitary representation of $U_p$ . Recall (see Lemma 10) that the stabilizer $G_\pi $ of $\pi $ in $\mathrm {Aut}(U_p)$ is an algebraic subgroup of $\mathrm {Aut}(U_p).$ Recall also (see Lemma 6) that $\pi $ extends to a projective representation of $G_\pi $ . The following result was proved in [Reference Bekka and HeuBeGu15, Proposition 22] in the case where $p=\infty ,$ using arguments from [Reference Howe and TanHoMo79]. The proof in the case where p is a prime integer is along similar lines and will be omitted.

We will need later a precise description of the projective kernel of a representation $\widetilde {\pi }$ as above.

Proof. We can assume that $\pi = \mathrm {Ind}_M^{U_p} \chi _{f_0}$ , for $f_0\in {\mathcal O}_\pi ,$ and $M= \exp \mathfrak m$ for a polarization $\mathfrak m$ of $f_0$ .

Let $g\in \mathrm {Aut}(U_p).$ If g is in the stabilizer $G_\pi $ of $\pi $ in $\mathrm {Aut}(U_p)$ , recall (see Proof of Lemma 6) that

$$ \begin{align*} \pi (g(u))=\widetilde{\pi} (g) \pi (u) \widetilde{\pi} (g^{-1}) \quad\text{for all } u\in U_p.\end{align*} $$

Since $\pi $ is irreducible, it follows from Schur’s lemma that $g\in P_{\widetilde {\pi }}$ if and only if

$$ \begin{align*} \pi(g(u))= \pi(u) \quad\text{for all } u\in U_p \end{align*} $$

that is,

$$ \begin{align*} g(u)u^{-1} \in \mathrm{Ker} (\pi) \quad\text{for all } u\in U_p. \end{align*} $$

Now we have (see [Reference Bekka and HeuBeGu15, Lemma 18])

$$ \begin{align*}\mathrm{Ker} (\pi) =\bigcap_{f\in {\mathcal O}_\pi} \mathrm{Ker} (\chi_{f}\!),\end{align*} $$

and so $g\in P_{\widetilde {\pi }}$ if and only if

$$ \begin{align*}g(u)u^{-1}\in \bigcap_{f\in {\mathcal O}_\pi} \mathrm{Ker} (\chi_{f}\!) \quad\text{for all } u\in U_p.\end{align*} $$

Let $g\in P_{\widetilde {\pi }}.$ Denote by $X\mapsto g(X)$ the automorphism of $\mathfrak {u}_p$ corresponding to $g.$ Let $u=\exp (X)$ for $X\in \mathfrak {u}_p$ and $f\in {\mathcal O}_\pi $ . Set $u_t= \exp (tX).$ By the Campbell Hausdorff formula, there exist $Y_1, \ldots Y_r\in \mathfrak {u}_p$ such that

$$ \begin{align*} g(u_t)(u_t)^{-1}= \exp (t Y_1 + t^2Y_2+\cdots+ t^r Y_r), \end{align*} $$

for every $t\in \mathbf Q_p$ . Since

(*) $$ \begin{align}1=\chi_{f} (g(u_t)(u_t)^{-1})=e_p(f( t Y_1+ t^2Y_2+\cdots+ t^r Y_2)), \end{align} $$

it follows that the polynomial

$$ \begin{align*}t\mapsto Q(t)= t f(Y_1) + t^2f(Y_2)+\cdots+ t^r f(Y_r)\end{align*} $$

takes its values in ${\mathbf Z}$ if $p=\infty $ , and in ${\mathbf Z}_p$ (and so Q has bounded image) otherwise. This clearly implies that $Q(t)=0$ for all $t\in \mathbf Q_p$ ; in particular, we have

$$ \begin{align*}\log (g(u)u^{-1})= Y_1+Y_2+\cdots +Y_r \in \mathrm{Ker} (f).\end{align*} $$

This shows that (i) implies (ii).

Conversely, assume that (ii) holds. Then clearly

$$ \begin{align*}g(u)u^{-1}\in \bigcap_{f\in {\mathcal O}_\pi} \mathrm{Ker} (\chi_{f}\!) \quad\text{for all } u\in U_p\end{align*} $$

and so $g\in P_{\widetilde {\pi }}$ .

9 Decomposition of the Koopman representation for a nilmanifold

Let $\mathbf {U}$ be a linear algebraic unipotent group defined over $\mathbf Q$ . Let $S= \{p_1, \ldots , p_r, \infty \}$ , where $p_1, \ldots , p_r$ are integer primes. Set

$$ \begin{align*}U:=\mathbf{U}(\mathbf Q_S)=\prod_{p\in S} U_p.\end{align*} $$

Since U is a type I group, the unitary dual $\widehat U$ of U can be identified with the cartesian product $\prod _{p\in S} \widehat {U_p}$ via the map

$$ \begin{align*} \prod_{p\in S} \widehat{U_p}\to \widehat U, \quad (\pi_p)_{p\in S} \mapsto\bigotimes_{p\in S} \pi_p, \end{align*} $$

where $\bigotimes _{p\in S} \pi _p= \pi _{\infty }\otimes \pi _{p_1} \otimes \cdots \otimes \pi_{p_r}$ is the tensor product of the $\pi _p$ .

Let $\Lambda :=\mathbf {U}({\mathbf Z}[1/S])$ and consider the corresponding S-adic compact nilmanifold

$$ \begin{align*}\mathbf{Nil}_S:= U/\Lambda,\end{align*} $$

equipped with the unique U-invariant probability measure $\mu $ on its Borel subsets.

The associated S-adic solenoid is

$$ \begin{align*} \mathbf{Sol}_S= \overline{U}/\overline{\Lambda}, \end{align*} $$

where $\overline {U}:=U/[U,U]$ is the quotient of U by its closed commutator subgroup $[U,U]$ and where $\overline {\Lambda }$ is the image of $\mathbf {U}({\mathbf Z}[1/S])$ in $\overline {U}.$

Set

$$ \begin{align*}\mathrm{Aut}(U):=\prod_{p\in S} \mathrm{Aut}(\mathbf{U}( \mathbf Q_p))\end{align*} $$

and denote by $\mathrm {Aut} (\mathbf {Nil}_S)$ the subgroup of all $g\in \mathrm {Aut}(U)$ with $g(\Lambda ) =\Lambda .$ Observe that $\mathrm {Aut}(\mathbf {Nil}_S)$ is a discrete subgroup of $\mathrm { Aut}(U)$ , where every $\mathrm {Aut}(U_p)$ is endowed with its natural (locally compact) topology and $\mathrm {Aut}(U)$ with the product topology.

Let $\Gamma $ be a subgroup of $\mathrm {Aut} (\mathbf {Nil}_S)$ . Let $\kappa $ be the Koopman representation of $\Gamma \ltimes U$ on $L^2(\mathbf {Nil}_S)$ associated to the action $\Gamma \ltimes U\curvearrowright \mathbf {Nil}_S.$ By Proposition 9, there exists a family $(\pi _i)_{i\in I}$ of irreducible representations of $U,$ such that $\kappa $ is equivalent to

$$ \begin{align*}\bigoplus_{i\in I} \mathrm{Ind}_{\Gamma_i\ltimes U}^{\Gamma\ltimes U}(\widetilde{\pi_i}\otimes W_i),\end{align*} $$

where $ \widetilde {\pi }_i$ is an irreducible projective representation $ \widetilde {\pi }_i$ of the stabilizer $G_i$ of $\pi _i$ in $\mathrm {Aut}(U)\ltimes U$ extending $\pi _i$ , and where $W_i$ is a projective unitary representation of $G_i \cap (\Gamma \ltimes U)$ .

Fix $i\in I.$ We have $\pi _i=\bigotimes _{p\in S}\pi _{i,p}$ for irreducible representations $\pi _{i, p}$ of $U_p.$

We will need the following more precise description of $\pi _i.$ Recall that $\mathfrak u$ is the Lie algebra of $\mathbf U$ and that $\mathfrak {u}(\mathbf Q)$ denotes the Lie algebra over $\mathbf Q$ consisting of the $\mathbf Q$ -points in $\mathfrak u.$

Let $\mathfrak {u}^*(\mathbf Q)$ be the set of $\mathbf Q$ -rational points in the dual space $\mathfrak u^*$ ; so, $\mathfrak {u}^*(\mathbf Q)$ is the subspace of $f\in \mathfrak u^*$ with $f(X)\in \mathbf Q$ for all $X\in \mathfrak {u}(\mathbf Q).$ Observe that, for $f\in \mathfrak {u}^*(\mathbf Q),$ we have $f(X)\in \mathbf Q_p$ for all $X\in \mathfrak {u}_p=\mathfrak {u}(\mathbf Q_p)$ .

A polarization for $f\in \mathfrak {u}^*(\mathbf Q)$ is a Lie subalgebra $\mathfrak m$ of ${\mathfrak u}(\mathbf Q)$ such that $f([{\mathfrak m},{\mathfrak m}])=0$ and which is of maximal dimension with this property.

Proof. The same result is proved in [Reference MooreMoor65, Theorem 11] (see also [Reference FoxFox89, Theorem 1.2]) for the Koopman representation of $\mathbf {U}(\mathbf {A})$ in $L^2(\mathbf {U}(\mathbf {A})/ \mathbf {U}(\mathbf {Q})),$ where $\mathbf {A}$ is the ring of adeles of $\mathbf Q.$ We could check that the proof, which proceeds by induction of the dimension of $\mathbf {U}$ , carries over to the Koopman representation on $L^2(\mathbf {U}(\mathbf Q_S)/ \mathbf {U}(\mathbf {{\mathbf Z}}[1/S]))$ , with the appropriate changes. We prefer to deduce our claim from the result for $\mathbf {U}(\mathbf {A})$ , as follows.

It is well known (see [Reference WeilWeil74]) that

$$ \begin{align*}\mathbf A= \bigg(\mathbf Q_S \times \prod_{p\notin S} {\mathbf Z}_p\bigg) +\mathbf Q\end{align*} $$

and that

$$ \begin{align*}\bigg(\mathbf Q_S \times \prod_{p\notin S} {\mathbf Z}_p\bigg)\cap \mathbf Q= {\mathbf Z}[1/S]. \end{align*} $$

This gives rise to a well-defined projection $\varphi :\mathbf {A}/ \mathbf {Q} \to \mathbf Q_S/{\mathbf Z}[1/S]$ given by

$$ \begin{align*}\varphi((a_S, (a_p)_{p\notin S}) +\mathbf Q)= a_S+{\mathbf Z}[1/S] \quad\text{for all } a_S\in \mathbf Q_S, (a_p)_{p\notin S}\in \prod_{p\notin S} {\mathbf Z}_p;\end{align*} $$

so the fiber over a point $a_S+{\mathbf Z}[1/S]\in \mathbf Q_S/{\mathbf Z}[1/S]$ is

$$ \begin{align*}\varphi^{-1}(a_S+{\mathbf Z}[1/S])= \{(a_S, (a_p)_{p\notin S}) +\mathbf Q\mid a_p\in {\mathbf Z}_p \text{ for all } p\}.\end{align*} $$

This induces an identification of $\mathbf {U}(\mathbf {\mathbf Q_S})/ \mathbf {U}(\mathbf {{\mathbf Z}}[1/S])=\mathbf {Nil}_S$ with the double coset space $K_S\backslash \mathbf {U}(\mathbf {A})/ \mathbf {U}(\mathbf {Q}),$ where $K_S$ is the compact subgroup

$$ \begin{align*}K_S=\prod_{p\notin S}\mathbf{U}({\mathbf Z}_p)\end{align*} $$

of $\mathbf {U}(\mathbf {A}).$ Observe that this identification is equivariant under translation by elements from $\mathbf {U}(\mathbf Q_S).$ In this way, we can view $L^2(\mathbf {Nil}_S)$ as the $\mathbf {U}(\mathbf Q_S)$ -invariant subspace $L^2(K_S\backslash \mathbf {U}(\mathbf {A})/ \mathbf {U}(\mathbf {Q}))$ of $L^2(\mathbf {U}(\mathbf {A})/ \mathbf {U}(\mathbf {Q})).$

Choose a system T of representatives for the $\mathrm {Ad}^*(\mathbf {U}(\mathbf Q))$ -orbits in $ \mathfrak {u}^*(\mathbf Q)$ . By [Reference MooreMoor65, Theorem 11], for every $f\in T,$ we can find a polarization $\mathfrak {m}_{f}\subset \mathfrak {u}(\mathbf Q)$ for f with the following property: setting

$$ \begin{align*}\mathfrak{m}_{f}(\mathbf{A})=\mathfrak{\mathfrak m}_{f}\otimes_{\mathbf Q} \mathbf{A},\end{align*} $$

we have a decomposition

$$ \begin{align*} L^2(\mathbf{U}(\mathbf{A})/ \mathbf{U}(\mathbf{Q}))= \bigoplus_{f\in T} \mathcal H_f \end{align*} $$

into irreducible $\mathbf {U}(\mathbf {A})$ -invariant subspaces $\mathcal H_f$ such that the representation $\pi _f$ of $\mathbf {U}(\mathbf {A})$ in $\mathcal H_f$ is equivalent to $\mathrm {Ind}_{ M_{f}(\mathbf {A})}^{\mathbf {U}(\mathbf {A}) }\chi _{f},$ where

$$ \begin{align*}M_{f}(\mathbf{A})=\exp(\mathfrak{m}_{f}(\mathbf{A}))\end{align*} $$

and $\chi _{f, \mathbf {A}}$ is the unitary character of $M_{f}(\mathbf {A})$ given by

$$ \begin{align*}\chi_{f, \mathbf{A}}(\exp X)=e( f(X))\quad\text{for all } X \in {\mathfrak m}_{f}(\mathbf{A});\end{align*} $$

here, e is the unitary character of $\mathbf {A}$ defined by

$$ \begin{align*} e((a_p)_p) = \prod_{p\in \mathcal P \cup \{\infty\}} e_p(a_p) \quad\text{for all } (a_p)_p\in \mathbf{A}, \end{align*} $$

where $\mathcal P$ is the set of integer primes.

We have

$$ \begin{align*}L^2(K_S\backslash \mathbf{U}(\mathbf{A})/ \mathbf{U}(\mathbf{Q}))= \bigoplus_{f\in T} \mathcal H_f^{K_S},\end{align*} $$

where $ \mathcal H_f^{K_S}$ is the space of $K_S$ -fixed vectors in $\mathcal H_f.$ It is clear that the representation of $\mathbf {U}(\mathbf Q_S)$ in $\mathcal H_f^{K_S}$ is equivalent to

$$ \begin{align*}\mathrm{Ind}_{ M_{f}(\mathbf{Q}_S)}^{\mathbf{U}(\mathbf Q_S) }\bigg(\bigotimes_{p\in S}\chi_{f, p}\bigg)= \bigotimes_{p\in S}(\mathrm{Ind}_{ M_{f}({\mathbf Q_p})}^{\mathbf{U}({\mathbf Q_p}) }\chi_{f,p}),\end{align*} $$

where $\chi _{f, p}$ is the unitary character of $M_{f}(\mathbf Q_p)$ given by

$$ \begin{align*}\chi_{f, p}(\exp X)=e_p( f(X))\quad\text{for all } X \in {\mathfrak m}_{f}(\mathbf Q_p).\end{align*} $$

Since $ M_{f}({\mathbf Q_p})$ is a polarization for f, each of the $\mathbf {U}(\mathbf Q_p)$ -representations $\mathrm {Ind}_{ M_{f}(\mathbf {\mathbf Q_p})}^{\mathbf {U}(\mathbf {\mathbf Q_p}) }\chi _{f,p}$ and, hence, each of the $\mathbf {U}(\mathbf Q_S)$ -representations

$$ \begin{align*}\mathrm{Ind}_{ M_{f}({\mathbf Q_S})}^{\mathbf{U}(\mathbf Q_S) }\bigg(\bigotimes_{p\in S}\chi_{f, p}\bigg)\end{align*} $$

is irreducible. This proves the claim.

We establish another crucial fact about the representations $\pi _i$ in the following proposition.

Proposition 14. With the notation of Proposition 13, let ${\mathcal O}_{\mathbf Q}(f_i)$ be the co-adjoint orbit of $f_i$ under $\mathbf {U}(\mathbf Q)$ and set

$$ \begin{align*} \mathfrak{k}_{i,p}=\bigcap_{f\in {\mathcal O}_{\mathbf Q}(f_i)} \mathfrak{k}_p(f), \end{align*} $$

where $\mathfrak {k}_{p}(f)$ is the kernel of f in $\mathfrak {u}_{p}.$ Let $K_{i,p}= \exp ( \mathfrak {k}_{i,p})$ and $K_i= \prod _{p\in S} K_{i,p}.$

1. (i) $K_i$ is a closed normal subgroup of U and $K_{i} \cap \Lambda = K_i\cap \mathbf {U}({\mathbf Z}[1/S])$ is a lattice in $K_i.$

2. (ii) Let $P_{\widetilde \pi _i}$ be the projective kernel of the extension $\widetilde \pi _i$ of $\pi _i$ to the stabilizer $G_i$ of $\pi _i$ in $\mathrm {Aut}(U)\ltimes U$ . For $g\in G_i$ , we have $g\in P_{\widetilde \pi _i}$ if and only if $g(u)\in u K_i$ for every $u\in U.$

Proof. (i) Let

$$ \begin{align*} \mathfrak{k}_{i,\mathbf Q}=\bigcap_{f\in {\mathcal O}_{\mathbf Q}(f_i)} \mathfrak{k}_{\mathbf Q(f)}, \end{align*} $$

where $\mathfrak {k}_{\mathbf Q}(f)$ is the kernel of f in $\mathfrak {u}(\mathbf Q).$ Observe that $ \mathfrak {k}_{i,\mathbf Q}$ is an ideal in $\mathfrak {u}(\mathbf Q),$ since it is $\mathrm {Ad}(\mathbf {U}(\mathbf Q))$ -invariant. So, we have

$$ \begin{align*}\mathfrak{k}_{i, \mathbf Q}= \mathfrak{k}_{i}(\mathbf Q)\end{align*} $$

for an ideal $\mathfrak {k}_{i}$ in $ \mathfrak {u}.$ Since $f\in \mathfrak {u}^*(\mathbf Q)$ for $f\in {\mathcal O}_{\mathbf Q}(f_i),$ we have

$$ \begin{align*} \mathfrak{k}_{i,p}(f)=\mathfrak{k}_{i,\mathbf Q}(f)\otimes_{\mathbf Q} \mathbf Q_p \end{align*} $$

and hence

$$ \begin{align*} \mathfrak{k}_{i,p}= \mathfrak{k}_{i}(\mathbf Q_p). \end{align*} $$

Let $\mathbf {K}_i=\exp ( \mathfrak {k}_{i}).$ Then $\mathbf {K}_i$ is a normal algebraic $\mathbf Q$ -subgroup of $\mathbf {U}$ and we have $K_{i,p}= \mathbf {K}_i(\mathbf Q_p)$ for every p; so,

$$ \begin{align*}K_i= \prod_{s\in S} \mathbf{K}_i(\mathbf Q_p)= \mathbf{K}_i(\mathbf Q_S)\end{align*} $$

and $K_i \cap \Lambda = \mathbf {K}_i({\mathbf Z}[1/S])$ is a lattice in $K_i.$ This proves (i).

To prove (ii), observe that

$$ \begin{align*}P_{\widetilde\pi_i}= \prod_{p\in S} P_{i,p},\end{align*} $$

where $P_{i,p}$ is the projective kernel of $\widetilde {\pi _{i,p}}$ .

Fix $p\in S$ and let $g\in G_i$ . By Lemma 12, $g\in P_{i,p}$ if and only if $g(u)\in u K_{i,p}$ for every $u\in U_p=\mathbf {U}(\mathbf Q_p). $ This finishes the proof.

10 Proof of Theorem 1

Let $\mathbf {U}$ be a linear algebraic unipotent group defined over $\mathbf Q$ and $S= \{p_1, \ldots , p_r, \infty \}$ , where $p_1, \ldots , p_r$ are integer primes. Set $U:=\mathbf {U}(\mathbf Q_S)$ and $\Lambda :=\mathbf {U}({\mathbf Z}[1/S])$ . Let $\mathbf {Nil}_S= U/\Lambda $ and $\mathbf {Sol}_S$ be the S-adic nilmanifold and the associated S-adic solenoid as in §9. Denote by $\mu $ the translation-invariant probability measure on $\mathbf {Nil}_S$ and let $\nu $ be the image of $\mu $ under the canonical projection $\varphi : \mathbf {Nil}_S\to \mathbf {Sol}_S.$ We identify $L^2(\mathbf {Sol}_S)=L^2(\mathbf {Sol}_S, \nu )$ with the closed $\mathrm {Aut}(\mathbf {Nil}_S)$ -invariant subspace

$$ \begin{align*}\{f\circ \varphi\mid f\in L^2(\mathbf{Sol}_S)\}\end{align*} $$

of $L^2(\mathbf {Nil}_S)=L^2(\mathbf {Nil}_S,\mu ).$ We have an orthogonal decomposition into $\mathrm {Aut}(\mathbf {Nil}_S)$ -invariant subspaces

$$ \begin{align*} L^2(\mathbf{Nil}_S)= {\mathbf C} \mathbf{1}_{\mathbf{Nil}_S}\oplus L_0^2(\mathbf{Sol}_S) \oplus \mathcal H, \end{align*} $$

where

$$ \begin{align*}L_0^2(\mathbf{Sol}_S)=\bigg\{f\in L^2(\mathbf{Sol}_S)\bigg| \int_{\mathbf{Nil}_S} f \,d\mu=0\bigg\}\end{align*} $$

and where $\mathcal H$ is the orthogonal complement of $L^2(\mathbf {Sol}_S)$ in $L^2(\mathbf {Nil}_S).$

Let $\Gamma $ be a subgroup of $\mathrm {Aut}(\mathbf {Nil}_S).$ Let $\kappa $ be the Koopman representation of $\Gamma $ on $L^2(\mathbf {Nil}_S)$ and denote by $\kappa _1$ and $\kappa _2$ the restrictions of $\kappa $ to $L_0^2(\mathbf {Sol}_S)$ and $ \mathcal H$ , respectively.

Let $\Sigma _1$ be a set of representatives for the $\Gamma $ -orbits in $\widehat {\mathbf {Sol}_S}\setminus \{ \mathbf {1}_{\mathbf {Sol}_S}\}$ . We have

where $\Gamma _\chi $ is the stabilizer of $\chi $ in $\Gamma $ and $\unicode{x3bb} _{\Gamma /\Gamma _\chi }$ is the quasi-regular representation of $\Gamma $ on $\ell ^2(\Gamma /\Gamma _\chi ).$

By Proposition 9, there exists a family $(\pi _i)_{i\in I}$ of irreducible representations of $U,$ such that $\kappa _2$ is equivalent to a direct sum

$$ \begin{align*}\bigoplus_{i\in I}\mathrm{ Ind}_{\Gamma_i}^{\Gamma}(\widetilde{\pi_i}|_{\Gamma_i}\otimes W_i),\end{align*} $$

where $\widetilde {\pi }_i$ is an irreducible projective representation of the stabilizer $G_i$ of $\pi _i$ in $\mathrm {Aut}(U)$ and where $W_i$ is a projective unitary representation of $\Gamma _i := \Gamma \cap G_i.$

Proof. By Proposition 11, there exists a real number $r\geq 1$ such that the representation $\widetilde {\pi }_i$ of the algebraic group $G_i$ is strongly $L^r$ modulo $P_{\widetilde \pi _i}$ . In order to show that $\widetilde {\pi _i}|_{\Gamma _i}$ is strongly $L^r$ modulo $ P_{\widetilde \pi _i}\cap \Gamma _i$ , it suffices to show that $\Gamma _i P_{\widetilde \pi _i}$ is closed in $G_i$ (compare with the proof of [Reference Howe and TanHoMo79, Proposition 6.2]).

Let $K_i$ be the closed $G_i$ -invariant normal subgroup $K_i$ of U as described in Proposition 14. Then $\overline {\Lambda }=K_i\Lambda /K_i$ is a lattice in the unipotent group $\overline {U}= U/K_i.$ By Proposition 14(ii), $P_{\widetilde \pi _i}$ coincides with the kernel of the natural homomorphism $\varphi : \mathrm {Aut}(U)\to \mathrm {Aut}(\overline {U})$ . Hence, we have

$$ \begin{align*}\Gamma_i P_{\widetilde\pi_i}= \varphi^{-1}(\varphi (\Gamma_i)).\end{align*} $$

Now, $\varphi (\Gamma _i)$ is a discrete (and hence closed) subgroup of $\mathrm {Aut}(\overline {U})$ , since $\varphi (\Gamma _i)$ preserves $\overline {\Lambda }$ (and so $\varphi ( \Gamma _i) \subset \mathrm {Aut}(\overline {U}/ \overline {\Lambda })).$ It follows from the continuity of $\varphi $ that $\varphi ^{-1}(\varphi (\Gamma _i))$ is closed in $\mathrm {Aut}(U)$ .

Proof of Theorem 1

We have to show that, if $1_\Gamma $ is weakly contained in $\kappa _2,$ then $1_\Gamma $ is weakly contained in $\kappa _1.$ It suffices to show that, if $1_\Gamma $ is weakly contained in $\kappa _2,$ then there exists a finite-index subgroup H of $\Gamma $ such that $1_H$ is weakly contained in $\kappa _1|_{H}$ (see [Reference Bekka and FranciniBeFr20, Theorem 2]).

We proceed by induction on the integer

$$ \begin{align*}n(\Gamma):=\sum_{p\in S} \dim \mathrm{Zc}_p(\Gamma),\end{align*} $$

where $\mathrm {Zc}_p (\Gamma )$ is the Zariski closure of the projection of $\Gamma $ in $GL_n(\mathbf Q_p)$ .

If $n(\Gamma )=0,$ then $\Gamma $ is finite and there is nothing to prove.

Assume that $n(\Gamma )\geq 1$ and that the claim above is proved for every countable subgroup H of $\mathrm {Aut}(\mathbf {Nil}_S)$ with $n(H) <n(\Gamma ).$

Let $I_{\mathrm {fin}} \subset I$ be the set of all $i\in I$ such that $\Gamma _i=G_i\cap \Gamma $ has finite index in $\Gamma $ and set $I_{\infty }=I \setminus I_{\mathrm {fin}}.$ With $V_i=\widetilde {\pi _i}|_{\Gamma _i}\otimes W_i$ , set

$$ \begin{align*} \kappa_2^{\mathrm{fin}} = \bigoplus_{i\in I_{\mathrm{fin}} } \mathrm{ Ind}_{\Gamma_i}^{\Gamma} V_i \quad\text{and}\quad \kappa_2^\infty= \bigoplus_{i\in I_{\infty} } \mathrm{Ind}_{\Gamma_i}^{\Gamma} V_i. \end{align*} $$

Two cases can occur.

First case: $1_\Gamma $ is weakly contained in $\kappa _2^\infty .$ Observe that $n(\Gamma _i)<n(\Gamma )$ for $i\in I_{\infty }.$ Indeed, otherwise $\mathrm {Zc}_p (\Gamma _i)$ and $\mathrm {Zc}_p (\Gamma )$ would have the same connected component $C^0_p$ for every $p\in S,$ since $\Gamma _i \subset \Gamma .$ Then

$$ \begin{align*}C^0:=\prod_{p\in S} C^0_p \end{align*} $$

would stabilize $\pi _i$ and $\Gamma \cap C^0$ would therefore be contained in $\Gamma _i.$ Since $\Gamma \cap C^0$ has finite index in $\Gamma ,$ this would contradict the fact that $\Gamma _i$ has infinite index in $\Gamma .$

By restriction, $1_{\Gamma _i}$ is weakly contained in $\kappa _2|_{\Gamma _i}$ for every $i\in I.$ Hence, by the induction hypothesis, $1_{\Gamma _i}$ is weakly contained in $\kappa _1|_{\Gamma _i}$ for every $i\in I_\infty .$ Now, on the one hand, we have

for a subset $T_i$ of $\widehat {\mathbf {Sol}_S}\setminus \{ \mathbf {1}_{\mathbf {Sol}_S}\}$ . It follows that $\mathrm {Ind}_{\Gamma _i}^\Gamma 1_{\Gamma _i}=\unicode{x3bb} _{\Gamma /\Gamma _i}$ is weakly contained in

for every $i\in I_\infty .$ On the other hand, since $1_\Gamma $ is weakly contained in

$$ \begin{align*}\kappa_2^\infty \cong \bigoplus_{i\in I_\infty}\mathrm{ Ind}_{\Gamma_i}^{\Gamma}(\widetilde{\pi_i}|_{\Gamma_i}\otimes W_i),\end{align*} $$

Lemma 7 shows that $1_\Gamma $ is weakly contained in $\bigoplus _{i\in I_\infty }\unicode{x3bb} _{\Gamma /\Gamma _i}.$ It follows that $1_\Gamma $ is weakly contained in

Hence, by Lemma 7 again, $1_\Gamma $ is weakly contained in

This shows that $1_\Gamma $ is weakly contained in $\kappa _1.$

Second case: $1_\Gamma $ is weakly contained in $\kappa _2^{\mathrm {fin}}.$ By the Noetherian property of the Zariski topology, we can find finitely many indices $i_1, \ldots , i_r$ in $I_{\mathrm {fin}}$ such that, for every $p\in S,$ we have

$$ \begin{align*} \mathrm{Zc}_p(\Gamma_{i_1})\cap\cdots \cap \mathrm{Zc}_p(\Gamma_{i_r}) =\bigcap_{i\in I_{\mathrm{fin}}} \mathrm{Zc}_p(\Gamma_{i}), \end{align*} $$

Set $H:=\Gamma _{i_1}\cap \cdots \cap \Gamma _{i_r}.$ Observe that H has finite index in $\Gamma .$ Moreover, it follows from Lemma 10 that $\mathrm {Zc}_p(\Gamma _{i_1})\cap \cdots \cap \mathrm {Zc}_p(\Gamma _{i_r}) $ stabilizes $\pi _{i,p}$ for every $i\in I_{\mathrm {fin}}$ and $p\in S.$ Hence, H is contained in $\Gamma _i$ for every $i\in I_{\mathrm {fin}}$ .

By Proposition 9, we have a decomposition of $\kappa _2^{\mathrm { fin}}|_H$ into the direct sum

$$ \begin{align*}\bigoplus_{i\in I_{\mathrm{fin}}} (\widetilde{\pi_i}\otimes W_i)|_H.\end{align*} $$

By Propositions 11 and 15, there exists a real number $r\ge 1 ,$ which is independent of $i,$ such that $(\widetilde {\pi _i}\otimes W_i)|_H$ is a strongly $L^r$ representation of H modulo its projective kernel $P_i$ . Observe that $P_i$ is contained in the projective kernel $ P_{\widetilde \pi _i}$ of $ \widetilde \pi _i,$ since $P_i= P_{\widetilde \pi _i}\cap H.$ Hence (see Proposition 8), there exists an integer $k\geq 1$ such that $(\kappa _2^{\mathrm {fin}}|_H)^{\otimes k}$ is contained in a multiple of the direct sum

$$ \begin{align*}\bigoplus_{i\in I_{\mathrm{fin}}} \mathrm{Ind}_{ P_i}^H \rho_i,\end{align*} $$

for representations $\rho _i$ of $ P_i.$ Since $1_H$ is weakly contained in $\kappa _2^{\mathrm {fin}}|H$ and hence in $(\kappa _2^{\mathrm {fin}}|_H)^{\otimes k},$ using Lemma 7, it follows that $1_H$ is weakly contained in

Let $i\in I.$ We claim that $P_i$ is contained in $\Gamma _\chi $ for some character $\chi $ from $\widehat {\mathbf {Sol}_S}\setminus \{ \mathbf {1}_{\mathbf {Sol}_S}\}$ . Once proved, this will imply, again by Lemma 7, $1_H$ is weakly contained in $\kappa _1|_H.$ Since H has finite index in $\Gamma ,$ this will show that $1_\Gamma $ is weakly contained in $\kappa _1$ and conclude the proof.

To prove the claim, recall from Proposition 14 that there exists a closed normal subgroup $K_i$ of U with the following properties: $K_i\Lambda /K_i$ is a lattice in the unipotent algebraic group $U/K_i$ , $K_i$ is invariant under $ P_{\widetilde \pi _i}$ and $ P_{\widetilde \pi _i}$ acts as the identity on $U/K_i.$ Observe that $K_i\neq U$ , since $\pi _i$ is not trivial on U. We can find a non-trivial unitary character $\chi $ of $U/K_i$ which is trivial on $K_i\Lambda /K_i$ . Then $\chi $ lifts to a non-trivial unitary character of U which is fixed by $ P_{\widetilde \pi _i}$ and hence by $P_i.$ Observe that $\chi \in \widehat {\mathbf {Sol}_S},$ since $\chi $ is trivial on $\Lambda $ .

11 An example: the S-adic Heisenberg nilmanifold

As an example, we study the spectral gap property for groups of automorphisms of the S-adic Heisenberg nilmanifold, proving Corollary 5. We will give a quantitative estimate for the norm of associated convolution operators, as we did in [Reference BekkaBeHe11] in the case of real Heisenberg nilmanifolds (that is, in the case $S=\{\infty \}$ ).

Let $\mathbf K$ be an algebraically closed field containing $\mathbf Q_p$ for $p=\infty $ and for all prime integers p. For an integer $n\geq 1$ , consider the symplectic form $\beta $ on $\mathbf K^{2n}$ given by

$$ \begin{align*}\beta((x,y),(x',y'))= (x,y)^t J (x',y')\quad\text{for all } (x,y),(x',y')\in \mathbf K^{2n},\end{align*} $$

where J is the $(2n\times 2n)$ -matrix

$$ \begin{align*} J=\bigg( \begin{array}{@{}cc@{}} 0&I\\ -I_{n}& 0 \end{array}\ \bigg). \end{align*} $$

The symplectic group

$$ \begin{align*} {Sp}_{2n}= \{g\in GL_{2n}(\mathbf K)\mid {^{t}g}Jg=J\} \end{align*} $$

is an algebraic group defined over $\mathbf Q.$

The $(2n+1)$ -dimensional Heisenberg group is the unipotent algebraic group $\mathbf {H}$ defined over $\mathbf Q,$ with underlying set $\mathbf K^{2n}\times \mathbf K$ and product

$$ \begin{align*}((x,y),s)((x',y'),t)=((x+x',y+y'),s + t +\beta((x,y),(x',y'))), \end{align*} $$

for $(x,y), (x',y')\in \mathbf K^{2n}, s,t\in \mathbf K.$

The group $Sp_{2n}$ acts by rational automorphisms of $\mathbf {H},$ given by

$$ \begin{align*} g((x,y),t)= (g(x,y),t) \quad\text{for all } g\in Sp_{2n}, (x,y)\in \mathbf K^{2n}, t\in \mathbf K. \end{align*} $$

Let p be either an integer prime or $p=\infty .$ Set $H_p= \mathbf {H}(\mathbf Q_p).$ The center Z of $H_p$ is $\{(0,0, t)\mid t\in \mathbf Q_p\}.$ The unitary dual $\widehat {H_p}$ of $H_p$ consists of the equivalence classes of the following representations:

• • the unitary characters of the abelianized group $H_p/Z$ ;

• • for every $t\in \mathbf Q_p\setminus \{0\},$ the infinite-dimensional representation $\pi _t$ defined on $L^2(\mathbf Q_p^n)$ by the formula

  $$ \begin{align*} \pi_t((a,b),s)\xi(x)= e_p({ts})e_p(\langle a, x-b \rangle) \xi(x-b) \end{align*} $$
  for $((a,b),s)\in H_p, \xi \in L^2(\mathbf Q_p^n),$ and $x\in \mathbf Q_p^n,$ where $e_p \in \widehat {\mathbf Q_p}$ is as in §6.

For $t\neq 0,$ the representation $\pi _t$ is, up to unitary equivalence, the unique irreducible unitary representation of $H_p$ whose restriction to the center Z is a multiple of the unitary character $s\mapsto e_p{(ts)}.$

For $g\in Sp_{2n}(\mathbf Q_p)$ and $t\in \mathbf Q_p\setminus \{0\},$ the representation $\pi _t^g$ is unitary equivalent to $\pi _t,$ since both representations have the same restriction to $Z.$ This shows that $Sp_{2n}(\mathbf Q_p)$ stabilizes $\pi _t$ . We denote the corresponding projective representation of $Sp_{2n}(\mathbf Q_p)$ by $\omega _t^{(p)}$ . The representation $\omega _t^{(p)}$ has different names: it is called the metaplectic representation, Weil’s representation or the oscillator representation. The projective kernel of $\omega _t^{(p)}$ coincides with the (finite) center of $Sp_{2n}(\mathbf Q_p)$ and $\omega _t^{(p)}$ is strongly $L^{4n+2+ \varepsilon }$ on $Sp_{2n}(\mathbf Q_p)$ for every $\varepsilon>0$ (see [Reference Howe and TanHoMo79, Proposition 6.4] or [Reference Howe and MooreHowe82, Proposition 8.1]).

Let $S= \{p_1, \ldots , p_r, \infty \}$ , where $p_1, \ldots , p_r$ are integer primes. Set $H:=\mathbf {H}(\mathbf Q_S)$ and

$$ \begin{align*}\Lambda:=\mathbf{H}({\mathbf Z}[1/S])=\{((x,y),s): x,y\in {\mathbf Z}^n[1/S], s\in {\mathbf Z}[1/S]\}.\end{align*} $$

Let $\mathbf {Nil}_S= H/\Lambda $ ; the associated S-adic solenoid is $\mathbf {Sol}_S= \mathbf Q_S^{2n}/{\mathbf Z}[1/S]^{2n}.$ The group $Sp_{2n}({\mathbf Z}[1/S])$ is a subgroup of $\mathrm { Aut}(\mathbf {Nil}_S)$ . The action of $Sp_{2n}({\mathbf Z}[1/S])$ on $\mathbf {Sol}_S$ is induced by its representation by linear bijections on $\mathbf Q_S^{2n}$ .

Let $\Gamma $ be a subgroup of $Sp_{2n}({\mathbf Z}[1/S])$ . The Koopman representation $\kappa $ of $\Gamma $ on $L^2(\mathbf {Nil}_S)$ decomposes as

$$ \begin{align*}\kappa=\mathbf{1}_{\mathbf{Nil}_S}\oplus \kappa_1\oplus \kappa_2, \end{align*} $$

where $\kappa _1$ is the restriction of $\kappa $ to $L_0^2(\mathbf {Sol}_S)$ and $\kappa _2$ the restriction of $\kappa $ to the orthogonal complement of $L^2(\mathbf {Sol}_S)$ in $L^2(\mathbf {Nil}_S).$ Since $Sp_{2n}(\mathbf Q_p)$ stabilizes every infinite-dimensional representation of $H_p,$ it follows from Proposition 13 that there exists a subset $I\subset \mathbf Q$ such that $\kappa _2$ is equivalent to a direct sum

$$ \begin{align*}\bigoplus_{t\in I}\Big(\bigotimes_{p\in S}(\omega_t^{(p)}|_\Gamma\otimes W_i)\Big),\end{align*} $$

where $ W_i$ is an projective representation of $\Gamma .$

Let $\nu $ be a probability measure on $\Gamma .$ We can give an estimate of the norm of $\kappa _2(\nu )$ as in [Reference BekkaBeHe11] in the case of $S=\{\infty \}.$ Indeed, by a general inequality (see [Reference Bekka and HeuBeGu15, Proposition 30]), we have

$$ \begin{align*} \Vert \kappa_2(\nu)\Vert \leq \Vert (\kappa_2\otimes\overline{\kappa_2})^{\otimes k}(\nu)\Vert^{1/2k}, \end{align*} $$

for every integer $k\geq 1,$ where $\overline {\kappa _2}$ denotes the representation conjugate to $\kappa _2$ . Since $\omega _t^{(p)}$ is strongly $L^{4n+2+ \varepsilon }$ on $Sp_{2n}(\mathbf Q_p)$ for any $t\in I$ and $p\in S,$ Proposition 8 implies that $(\kappa _2\otimes \overline {\kappa _2})^{\otimes (n+1)}$ is contained in an infinite multiple of the regular representation $\unicode{x3bb} _\Gamma $ of $\Gamma .$ Hence,

and so,

where $\kappa _0$ is the restriction of $\kappa $ to $L^2_0(\mathbf {Nil}_S).$

Assume that $\nu $ is aperiodic. If $\Gamma $ is not amenable then $\Vert \unicode{x3bb} _\Gamma (\nu )\Vert <1$ by Kesten’s theorem (see [Reference Bekka, de la Harpe and ValetteBeHV08, Appendix G]); so, in this case, the action of $\Gamma $ on $\mathbf {Nil}_S$ has a spectral gap if and only if $\Vert \kappa _1(\nu )\Vert <1,$ as stated in Theorem 1.

Observe that, if $\Gamma $ is amenable, then the action of $\Gamma $ on $\mathbf {Nil}_S$ or $\mathbf {Sol}_S$ does not have a spectral gap; indeed, by a general result (see [Reference del Junco and RosenblattJuRo79, Theorem 2.4]), no action of a countable amenable group by measure-preserving transformations on a non-atomic probability space has a spectral gap.

Let us look more closely to the case $n=1.$ We have $Sp_{2}({\mathbf Z}[1/S])=SL_2({\mathbf Z}[1/S])$ and the stabilizer of every element in $\widehat {\mathbf {Sol}_S}\setminus \{ \mathbf {1}_{\mathbf {Sol}_S}\}$ is conjugate to the group of unipotent matrices in $SL_2({\mathbf Z}[1/S])$ and hence amenable. This implies that $\kappa _1$ is weakly contained in $\unicode{x3bb} _\Gamma $ (see the decomposition of $\kappa _1$ appearing before Proposition 15); so, we have

$$ \begin{align*}\Vert \kappa_1(\nu)\Vert<1 \Longleftrightarrow\, \Gamma \text{ is not amenable.} \end{align*} $$

As a consequence, we see that the action of $\Gamma $ on $\mathbf {Nil}_S$ has a spectral gap if and only if $\Gamma $ is not amenable.