2.1 Algebraic groups, real Lie groups and the 5-dimensional hyperbolic space

Let $B$ be the definite quaternion algebra over $\mathbb{Q}$ with discriminant $d_{B}=2$. The algebra $B$ is given by $B=\mathbb{Q}+\mathbb{Q}i+\mathbb{Q}j+\mathbb{Q}k$ with a basis $\{1,~i,~j,~k\}$ characterized by the conditions

$$\begin{eqnarray}i^{2}=j^{2}=k^{2}=-1,\quad ij=-ji=k.\end{eqnarray}$$

Let ${\mathcal{G}}$ be the $\mathbb{Q}$-algebraic group defined by its group of $\mathbb{Q}$-rational points

$$\begin{eqnarray}{\mathcal{G}}(\mathbb{Q})=\text{GL}_{2}(B).\end{eqnarray}$$

Here, $\text{GL}_{2}(B)$ is the general linear group over $B$, which consists of elements in $M_{2}(B)$ whose reduced norms are nonzero. Let $\mathbb{H}=B\otimes _{\mathbb{Q}}\mathbb{R}$, which is nothing but the Hamilton quaternion algebra $\mathbb{R}+\mathbb{R}i+\mathbb{R}j+\mathbb{R}k$. Let $\mathbb{H}\ni x\mapsto \bar{x}\in \mathbb{H}$ denote the main involution of $\mathbb{H}$, and let $\operatorname{tr}(x)=x+\bar{x}$ and ${\it\nu}(x):=x\bar{x}$ be the reduced trace and the reduced norm of $x\in \mathbb{H}$ respectively. In what follows, we often use the notation $|{\it\beta}|:=\sqrt{{\it\nu}({\it\beta})}$ for ${\it\beta}\in \mathbb{H}$. We put $\mathbb{H}^{-}:=\{x\in \mathbb{H}\mid \operatorname{tr}(x)=0\}$ to be the set of pure quaternions, and $\mathbb{H}^{1}:=\{x\in \mathbb{H}\mid {\it\nu}(x)=1\}$.

Denote by $G:=\text{GL}_{2}(\mathbb{H})$ the general linear group of degree two with coefficients in the Hamilton quaternion algebra $\mathbb{H}$. The Lie group $G$ admits an Iwasawa decomposition

$$\begin{eqnarray}G=Z^{+}NAK,\end{eqnarray}$$

where

(2.1)$$\begin{eqnarray}\displaystyle \quad Z^{+} & := & \displaystyle \displaystyle \left\{\left.\left[\begin{array}{@{}cc@{}}c & 0\\ 0 & c\end{array}\right]\right|c\in \mathbb{R}_{+}^{\times }\right\},\quad N:=\left\{\left.n(x)=\left[\begin{array}{@{}cc@{}}1 & x\\ 0 & 1\end{array}\right]\right|x\in \mathbb{H}\right\},\nonumber\\ \displaystyle A & := & \displaystyle \displaystyle \left\{\left.a_{y}:=\left[\begin{array}{@{}cc@{}}\sqrt{y} & 0\\ 0 & \sqrt{y}^{-1}\end{array}\right]\right|y\in \mathbb{R}_{+}^{\times }\right\},\quad K:=\{k\in G\mid \text{}^{t}\bar{k}k=1_{2}\}.\end{eqnarray}$$

The subgroup $Z^{+}$ is contained in the center of $G$, and $K$ is a maximal compact subgroup of $G$, which is isomorphic to the definite symplectic group of degree two.

Let us consider the quotient $G/Z^{+}K$, which is realized as

$$\begin{eqnarray}\left\{\left.\!\left[\begin{array}{@{}cc@{}}y & x\\ 0 & 1\end{array}\right]\right|y\in \mathbb{R}_{+}^{\times },~x\in \mathbb{H}\right\}.\end{eqnarray}$$

This gives a realization of the 5-dimensional real hyperbolic space.

2.2 Lie algebras

The Lie algebra $\mathfrak{g}$ of $G$ is nothing but $M_{2}(\mathbb{H})$, and has an Iwasawa decomposition

$$\begin{eqnarray}\mathfrak{g}=\mathfrak{z}\oplus \mathfrak{n}\oplus \mathfrak{a}\oplus \mathfrak{k}.\end{eqnarray}$$

Here,

(2.2)$$\begin{eqnarray}\displaystyle \mathfrak{z} & := & \displaystyle \left\{\left.\left[\begin{array}{@{}cc@{}}c & 0\\ 0 & c\end{array}\right]\right|c\in \mathbb{R}\right\},\quad \mathfrak{n}:=\left\{\left.\left[\begin{array}{@{}cc@{}}0 & x\\ 0 & 0\end{array}\right]\right|x\in \mathbb{H}\right\},\nonumber\\ \displaystyle \mathfrak{a} & := & \displaystyle \left\{\left.\left[\begin{array}{@{}cc@{}}t & 0\\ 0 & -t\end{array}\right]\right|t\in \mathbb{R}\right\},\quad \mathfrak{k}:=\{X\in M_{2}(\mathbb{H})\mid \text{}^{t}\bar{X}+X=0_{2}\},\end{eqnarray}$$

where $\mathfrak{z},~\mathfrak{n},~\mathfrak{a}$ and $\mathfrak{k}$ are the Lie algebras of $Z^{+},~N,~A$ and $K$ respectively.

We next consider the root space decomposition of $\mathfrak{g}$ with respect to $\mathfrak{a}$. Let $H:=\left[\!\begin{smallmatrix}1 & 0\phantom{}\\ 0 & -1\end{smallmatrix}\!\right]$, and let ${\it\alpha}$ be the linear form of $\mathfrak{a}$ such that ${\it\alpha}(H)=1$. Then $\{\pm 2{\it\alpha}\}$ is the set of roots for $(\mathfrak{g},\mathfrak{a})$. For $z\in \mathbb{H}$ we put

$$\begin{eqnarray}E_{2{\it\alpha}}^{(z)}:=\left[\begin{array}{@{}cc@{}}0 & z\\ 0 & 0\end{array}\right],\quad E_{-2{\it\alpha}}^{(z)}:=\left[\begin{array}{@{}cc@{}}0 & 0\\ z & 0\end{array}\right].\end{eqnarray}$$

The set $\{E_{2{\it\alpha}}^{(1)},~E_{2{\it\alpha}}^{(i)},~E_{2{\it\alpha}}^{(j)},~E_{2{\it\alpha}}^{(k)}\}$  (respectively $\{E_{-2{\it\alpha}}^{(1)},~E_{-2{\it\alpha}}^{(i)},~E_{-2{\it\alpha}}^{(j)},~E_{-2{\it\alpha}}^{(k)}\}$) forms a basis of $\mathfrak{n}$ (respectively a basis of $\bar{\mathfrak{n}}:=\left\{\left.\!\left[\!\begin{smallmatrix}0 & 0\phantom{}\\ x & 0\end{smallmatrix}\!\right]\right|x\in \mathbb{H}\right\}\!$). Let $\mathfrak{z}_{\mathfrak{a}}(\mathfrak{k}):=\{X\in \mathfrak{k}\mid [X,A]=0~\forall A\in \mathfrak{a}\}$, which coincides with

$$\begin{eqnarray}\left\{\left.\left[\begin{array}{@{}cc@{}}a & 0\\ 0 & d\end{array}\right]\right|a,~d\in \mathbb{H}^{-}\right\}.\end{eqnarray}$$

Then $\mathfrak{z}\oplus \mathfrak{z}_{\mathfrak{a}}(\mathfrak{k})\oplus \mathfrak{a}$ is the eigen space with the eigenvalue zero. We then see from the root space decomposition of $\mathfrak{g}$ with respect to $\mathfrak{a}$ that $\mathfrak{g}$ decomposes into

$$\begin{eqnarray}\mathfrak{g}=(\mathfrak{z}\oplus \mathfrak{z}_{\mathfrak{a}}(\mathfrak{k})\oplus \mathfrak{a})\oplus \mathfrak{n}\oplus \bar{\mathfrak{n}}.\end{eqnarray}$$

We also introduce the simple Lie group $\text{SL}_{2}(\mathbb{H})$ consisting of elements in $\text{GL}_{2}(\mathbb{H})$ with their reduced norms $1$. The Lie algebra $\mathfrak{g}_{0}=\mathfrak{s}l_{2}(\mathbb{H})$ of $\text{SL}_{2}(\mathbb{H})$ is the Lie algebra consisting of elements in $M_{2}(\mathbb{H})$ with their reduced traces zero. For this we note that

$$\begin{eqnarray}\text{GL}_{2}(\mathbb{H})/Z^{+}\simeq \text{SL}_{2}(\mathbb{H}),\quad \mathfrak{g}/\mathfrak{z}\simeq \mathfrak{g}_{0}.\end{eqnarray}$$

We introduce the differential operator ${\rm\Omega}$ defined by the infinitesimal action of

(2.3)$$\begin{eqnarray}{\rm\Omega}:=\frac{1}{32}H^{2}-\frac{1}{4}H+\frac{1}{8}\mathop{\sum }_{z\in \{1,i,j,k\}}{E_{2{\it\alpha}}^{(z)}}^{2}.\end{eqnarray}$$

This differential operator ${\rm\Omega}$ coincides with the infinitesimal action of the Casimir element of $\mathfrak{g}_{0}$ (see [Reference Knapp15, p. 293, Proposition 5.28]) on the space of right $K$-invariant smooth functions of $G/Z^{+}$. To check this we note that $[E_{2{\it\alpha}}^{(z)},E_{-2{\it\alpha}}^{(\bar{z})}]=H$ for $z\in \mathbb{H}^{1}$ and Iwasawa decompositions $E_{-2{\it\alpha}}^{(z)}=E_{2{\it\alpha}}^{(\bar{z})}+\left(\!\begin{smallmatrix}0 & -\bar{z}\phantom{}\\ z & 0\end{smallmatrix}\!\right)$ for $z\in \mathbb{H}$. In what follows, we call ${\rm\Omega}$ the Casimir operator.

2.3 Automorphic forms

For ${\it\lambda}\in \mathbb{C}$ and a discrete subgroup ${\rm\Gamma}\subset \text{SL}_{2}(\mathbb{R})$ we denote by $S({\rm\Gamma},{\it\lambda})$ the space of Maass cusp forms of weight $0$ on the complex upper half plane $\mathfrak{h}$ whose eigenvalue with respect to the hyperbolic Laplacian is $-{\it\lambda}$.

For a discrete subgroup ${\rm\Gamma}\subset \text{GL}_{2}(\mathbb{H})$ and $r\in \mathbb{C}$ we denote by ${\mathcal{M}}({\rm\Gamma},r)$ the space of smooth functions $F$ on $\text{GL}_{2}(\mathbb{H})$ satisfying the following conditions:

1. (1) ${\rm\Omega}\cdot F=-\frac{1}{2}(r^{2}/4+1)F$, where ${\rm\Omega}$ is the Casimir operator defined in (2.3);

2. (2) for any $(z,{\it\gamma},g,k)\in Z^{+}\times {\rm\Gamma}\times G\times K$, we have $F(z{\it\gamma}gk)=F(g)$;

3. (3) $F$ is of moderate growth.

Let $K_{{\it\alpha}}$, with ${\it\alpha}\in \mathbb{C}$, denote the modified Bessel function (see [Reference Andrews, Askey and Roy1, §4.12]), which satisfies the differential equation

$$\begin{eqnarray}y^{2}\frac{d^{2}K_{{\it\alpha}}}{dy^{2}}+y\frac{dK_{{\it\alpha}}}{dy}-(y^{2}+{\it\alpha}^{2})K_{{\it\alpha}}=0.\end{eqnarray}$$

Proposition 2.1. Let ${\rm\Gamma}$ be an arithmetic subgroup of $\text{GL}_{2}(\mathbb{H})$, and let $L_{{\rm\Gamma}}:=\{x\in \mathbb{H}\mid n(x)\in N\cap {\rm\Gamma}\}$ and $\hat{L_{{\rm\Gamma}}}$ be the dual lattice of $L_{{\rm\Gamma}}$ with respect to $\operatorname{tr}$. Then $F\in {\mathcal{M}}({\rm\Gamma},r)$ admits a Fourier expansion

$$\begin{eqnarray}F(n(x)a_{y})=u(y)+\mathop{\sum }_{{\it\beta}\in \hat{L_{{\rm\Gamma}}}\setminus \{0\}}C({\it\beta})y^{2}K_{\sqrt{-1}r}(4{\it\pi}|{\it\beta}|y)e^{2{\it\pi}\sqrt{-1}\operatorname{tr}({\it\beta}x)},\end{eqnarray}$$

with a smooth function $u$ on $\mathbb{R}_{{>}0}$.

Proof. A Maass form $F\in {\mathcal{M}}({\rm\Gamma};r)$ is left-invariant with respect to $\{n({\it\beta})\mid {\it\beta}\in L_{{\rm\Gamma}}\}$. This implies that $F(n(x+{\it\alpha})g)=F(n(x)g)$ holds for ${\it\alpha}\in L_{{\rm\Gamma}}$ and $g\in G$. Therefore, $F$ has an expansion

$$\begin{eqnarray}F(n(x)a_{y})=\mathop{\sum }_{{\it\beta}\in \hat{L_{{\rm\Gamma}}}}W_{{\it\beta}}(y)e^{2{\it\pi}\sqrt{-1}\operatorname{tr}({\it\beta}x)},\end{eqnarray}$$

with a smooth function $W_{{\it\beta}}$ on $\mathbb{R}_{+}^{\times }$. For ${\it\xi}\in \mathbb{H}\setminus \{0\}$ we put ${\hat{W}}_{{\it\xi}}(y):=y^{-3/2}W_{{\it\xi}}(y)$. From the condition ${\rm\Omega}\cdot F=-\frac{1}{2}(r^{2}/4+1)F$ we deduce that ${\hat{W}}_{{\it\beta}}$ satisfies the differential equation

$$\begin{eqnarray}\left(\frac{d^{2}}{dY^{2}}+\left(-\frac{1}{4}+\frac{\frac{1}{4}+r^{2}}{Y^{2}}\right)\right){\hat{W}}_{{\it\beta}}\left(\frac{Y}{8{\it\pi}|{\it\beta}|}\right)=0\end{eqnarray}$$

for ${\it\beta}\in \hat{L_{{\rm\Gamma}}}\setminus \{0\}$, where $Y:=8{\it\pi}|{\it\beta}|y$. This is precisely the differential equation for the Whittaker function (see [Reference Andrews, Askey and Roy1, §4.3]). With the Whittaker function $W_{0,\sqrt{-1}r}$ parametrized by $(0,\sqrt{-1}r)$ we thereby see that

$$\begin{eqnarray}F(n(x)a_{y})=u(y)+\mathop{\sum }_{{\it\beta}\in \hat{L_{{\rm\Gamma}}}\setminus \{0\}}C^{\prime }({\it\beta})y^{3/2}W_{0,\sqrt{-1}r}(8{\it\pi}|{\it\beta}|y)e^{2{\it\pi}\sqrt{-1}\operatorname{tr}({\it\beta}x)},\end{eqnarray}$$

with constants $C^{\prime }({\it\beta})$ depending only on ${\it\beta}$. We now note the relation

$$\begin{eqnarray}W_{0,\sqrt{-1}r}(2y)=\sqrt{\frac{2y}{{\it\pi}}}K_{\sqrt{-1}r}(y)\end{eqnarray}$$

(see [Reference Oliver, Lozier, Boisvert and Clark21, §13, 13.18(iii), 13.18.9]). This means that $F$ has the Fourier expansion as in the statement of the proposition.◻

We consider the automorphic forms above with specified discrete subgroups of $\text{SL}_{2}(\mathbb{R})$ and $\text{GL}_{2}(\mathbb{H})$. As a discrete subgroup of $\text{SL}_{2}(\mathbb{R})$ we take the congruence subgroup ${\rm\Gamma}_{0}(2)$ of level $2$. For a choice of a discrete subgroup of $\text{GL}_{2}(\mathbb{H})$ we recall that $B$ denotes the definite quaternion algebra over $\mathbb{Q}$ with discriminant $d_{B}=2$. Up to $B^{\times }$-conjugation this has a unique maximal order. In fact, we have the maximal order ${\mathcal{O}}$ given by

$$\begin{eqnarray}{\mathcal{O}}=\mathbb{Z}+\mathbb{Z}i+\mathbb{Z}j+\mathbb{Z}\frac{1+i+j+ij}{2},\end{eqnarray}$$

which is called the Hurwitz order. As a discrete subgroup of $\text{GL}_{2}(\mathbb{H})$ we take $\text{GL}_{2}({\mathcal{O}})$.

Proposition 2.2. The group $\text{GL}_{2}({\mathcal{O}})$ is generated by

$$\begin{eqnarray}\left\{\left.\left[\begin{array}{@{}cc@{}}0 & 1\\ -1 & 0\end{array}\right],\left[\begin{array}{@{}cc@{}}u & 0\\ 0 & 1\end{array}\right],\left[\begin{array}{@{}cc@{}}1 & v\\ 0 & 1\end{array}\right]\right|u\in {\mathcal{O}}^{\times },v\in {\mathcal{O}}\right\}.\end{eqnarray}$$

Proof. Any element of the form $\left[\!\begin{smallmatrix}{\it\alpha} & {\it\beta}\phantom{}\\ 0 & {\it\delta}\end{smallmatrix}\!\right]\in \text{GL}_{2}({\mathcal{O}})$ can be expressed as

$$\begin{eqnarray}\left[\begin{array}{@{}cc@{}}{\it\alpha} & 0\\ 0 & 1\end{array}\right]\left[\begin{array}{@{}cc@{}}1 & 0\\ 0 & {\it\delta}\end{array}\right]\left[\begin{array}{@{}cc@{}}1 & {\it\alpha}^{-1}{\it\beta}\\ 0 & 1\end{array}\right].\end{eqnarray}$$

We see that ${\it\alpha},~{\it\delta}\in {\mathcal{O}}^{\times }$, and thus ${\it\alpha}^{-1}{\it\beta}\in {\mathcal{O}}$. We note that

$$\begin{eqnarray}\left[\begin{array}{@{}cc@{}}0 & 1\\ -1 & 0\end{array}\right]\left[\begin{array}{@{}cc@{}}u & 0\\ 0 & 1\end{array}\right]\left[\begin{array}{@{}cc@{}}0 & -1\\ 1 & 0\end{array}\right]=\left[\begin{array}{@{}cc@{}}1 & 0\\ 0 & u\end{array}\right],\quad \left[\begin{array}{@{}cc@{}}0 & 1\\ -1 & 0\end{array}\right]^{3}=\left[\begin{array}{@{}cc@{}}0 & -1\\ 1 & 0\end{array}\right].\end{eqnarray}$$

These imply the assertion for $\left[\!\begin{smallmatrix}{\it\alpha} & {\it\beta}\phantom{}\\ 0 & {\it\delta}\end{smallmatrix}\!\right]\in \text{GL}_{2}({\mathcal{O}})$. Next, we have the following claim.

For $a,~b\in {\mathcal{O}}$with $b\not =0$there exist $c,~d\in {\mathcal{O}}$such that $a=cb+d,\quad {\it\nu}(d)<{\it\nu}(b)$. This follows from [Reference Krieg16, Chap. I, Section 1, Corollary 1.8]. This reduces the general case of $\left[\!\begin{smallmatrix}{\it\alpha} & {\it\beta}\phantom{}\\ {\it\gamma} & {\it\delta}\end{smallmatrix}\!\right]\in \text{GL}_{2}({\mathcal{O}})$, ${\it\gamma}\not =0$, to the previous case. This completes the proof of the proposition.◻

Let ${\mathcal{G}}(\mathbb{A})=\text{GL}_{2}(B_{\mathbb{A}})$, where $B_{\mathbb{A}}$ denotes the adelization of $B$, and let $U$ be the compact subgroup of ${\mathcal{G}}(\mathbb{A})$ given by $\prod _{p<\infty }\text{GL}_{2}({\mathcal{O}}_{p})$, where ${\mathcal{O}}_{p}$ denotes the $p$-adic completion of ${\mathcal{O}}$ at a finite prime $p$. Then, the class number of ${\mathcal{G}}$ with respect to $U$ is defined as the number of cosets in $U{\mathcal{G}}(\mathbb{R})\setminus {\mathcal{G}}(\mathbb{A})/{\mathcal{G}}(\mathbb{Q})$.

We next put ${\mathcal{P}}$ to be a standard $\mathbb{Q}$-parabolic subgroup of ${\mathcal{G}}$ whose group of $\mathbb{Q}$-rational points is ${\mathcal{P}}(\mathbb{Q})=\left\{\left[\!\begin{smallmatrix}{\it\alpha} & {\it\beta}\phantom{}\\ 0 & {\it\delta}\end{smallmatrix}\!\right]\in {\mathcal{G}}(\mathbb{Q})\right\}$. We now recall that, for an arithmetic subgroup ${\rm\Gamma}\subset {\mathcal{G}}(\mathbb{Q})$, the cosets ${\rm\Gamma}\setminus {\mathcal{G}}(\mathbb{Q})/{\mathcal{P}}(\mathbb{Q})$ are called the set of ${\rm\Gamma}$-cusps.

Proof. According to [Reference Platonov and Rapinchuk24, Theorem 8.11] the class number of a reductive group over a number field $F$ is not greater than that of a parabolic $F$-subgroup of it. Furthermore, the class number of a parabolic $F$-subgroup is not greater than that of its Levi subgroup (see [Reference Platonov and Rapinchuk24, Proposition 5.4]). Hence, the class number of ${\mathcal{G}}$ with respect to $U$ turns out to be not greater than that of the Levi subgroup ${\mathcal{L}}$ defined by the $\mathbb{Q}$-rational points $B^{\times }\times B^{\times }$. Since the class number of $B^{\times }$ with respect to $\prod _{p<\infty }{\mathcal{O}}_{p}^{\times }$ is one, the class number of ${\mathcal{G}}$ is also one, which means that ${\mathcal{G}}(\mathbb{A})={\mathcal{G}}(\mathbb{Q}){\mathcal{G}}(\mathbb{R})U=U{\mathcal{G}}(\mathbb{R}){\mathcal{G}}(\mathbb{Q})$. This completes the proof of (1).

The first assertion implies that there is a bijection

$$\begin{eqnarray}{\rm\Gamma}\setminus {\mathcal{G}}(\mathbb{Q})/{\mathcal{P}}(\mathbb{Q})\simeq {\mathcal{G}}(\mathbb{R})U\setminus {\mathcal{G}}(\mathbb{A})/{\mathcal{P}}(\mathbb{Q}).\end{eqnarray}$$

Furthermore, note an Iwasawa decomposition

$$\begin{eqnarray}{\mathcal{G}}(\mathbb{Q}_{v})=\left\{\begin{array}{@{}ll@{}}\text{GL}_{2}({\mathcal{O}}_{p})\cdot P(\mathbb{Q}_{p})\quad & (v=p<\infty ),\\ K\cdot P(\mathbb{R})\quad & (v=\infty )\end{array}\right.\end{eqnarray}$$

at every place $v\leqslant \infty$. We can then reduce the counting of the number of ${\rm\Gamma}$-cusps to that of the class number of the Levi subgroup ${\mathcal{L}}$. This implies that the number of cusps with respect to ${\rm\Gamma}$ is one, and completes the proof of the lemma.◻

We define ${\rm\Gamma}_{T}$ as a subgroup $\text{GL}_{2}({\mathcal{O}})$ generated by

(2.4)$$\begin{eqnarray}\left[\begin{array}{@{}cc@{}}0 & -1\\ 1 & 0\end{array}\right],\quad \left[\begin{array}{@{}cc@{}}1 & {\it\beta}\\ 0 & 1\end{array}\right]\quad ({\it\beta}\in {\mathcal{O}}).\end{eqnarray}$$

In what follows, we deal mainly with $S({\rm\Gamma}_{0}(2);-(\frac{1}{4}+(\frac{r}{2})^{2}))$, ${\mathcal{M}}({\rm\Gamma}_{T};r)$ and ${\mathcal{M}}(\text{GL}_{2}({\mathcal{O}}),r)$. For this we should note that the Selberg conjecture for ${\rm\Gamma}_{0}(2)$ is verified (cf. [Reference Iwaniec12, Corollary 11.5]). This means the following.

By Proposition 2.1, the Fourier expansion of $F\in {\mathcal{M}}(\text{GL}_{2}({\mathcal{O}}),r)$ is then written as

$$\begin{eqnarray}\displaystyle F(n(x)a_{y}) & = & \displaystyle u(y)+\mathop{\sum }_{{\it\beta}\in \frac{1}{2}S\setminus \{0\}}C({\it\beta})y^{2}K_{\sqrt{-1}r}(4{\it\pi}{\it\nu}({\it\beta})y)e^{2{\it\pi}\sqrt{-1}\operatorname{tr}({\it\beta}x)}\nonumber\\ \displaystyle & = & \displaystyle u(y)+\mathop{\sum }_{{\it\beta}\in S\setminus \{0\}}A({\it\beta})y^{2}K_{\sqrt{-1}r}(2{\it\pi}|{\it\beta}|y)e^{2{\it\pi}\sqrt{-1}\operatorname{Re}({\it\beta}x)},\nonumber\end{eqnarray}$$

with a smooth function $u$ on $\mathbb{R}_{{>}0}$. Here,

(2.5)$$\begin{eqnarray}S:=\mathbb{Z}\cdot (1-ij)+\mathbb{Z}\cdot (-i-ij)+\mathbb{Z}\cdot (-j-ij)+\mathbb{Z}\cdot 2ij\end{eqnarray}$$

is the dual lattice of ${\mathcal{O}}$ with respect to the bilinear form on $\mathbb{H}\times \mathbb{H}$ defined by $\operatorname{Re}=\frac{1}{2}\operatorname{tr}$.

Now we introduce ${\it\varpi}_{2}=1+i$, which is a uniformizer of $B\otimes _{\mathbb{Q}}\mathbb{Q}_{2}$. We can verify the following lemma by a direct computation.

Theta functions

Consider the space of harmonic polynomials of degree $l$ on $\mathbb{H}$. These are homogeneous polynomials of degree $l$ and are annihilated by the Laplace operator in $4$ variables. We can act on this space by the cyclic group of order $8$ generated by $\frac{1+i}{\sqrt{2}}\in \mathbb{H}$. Let $\{P_{l,{\it\nu}}\}_{{\it\nu}}$ denote a basis for this space consisting of eigenvectors under the above action. Hence,

(3.1)$$\begin{eqnarray}P_{l,{\it\nu}}\left(\frac{1+i}{\sqrt{2}}x\right)={\it\epsilon}_{l,{\it\nu}}P_{l,{\it\nu}}(x),\end{eqnarray}$$

for some 8th root of unity ${\it\epsilon}_{l,v}$. Define the following theta function:

(3.2)$$\begin{eqnarray}{\rm\Theta}_{l,{\it\nu}}(z):=\mathop{\sum }_{{\it\beta}\in S}P_{l,{\it\nu}}({\it\beta})e^{2{\it\pi}\sqrt{-1}\frac{|{\it\beta}|^{2}}{2}z}=\mathop{\sum }_{m=0}^{\infty }b(2m)e^{2{\it\pi}\sqrt{-1}mz}\end{eqnarray}$$

on the complex upper half plane $\mathfrak{h}$, where $b(m):=\sum _{{\it\beta}\in S,~|{\it\beta}|^{2}=m}P_{l,{\it\nu}}({\it\beta})$. Since $S$ is invariant under ${\it\beta}\mapsto -{\it\beta}$ and $P_{l,{\it\nu}}(-x)=(-1)^{l}P_{l,{\it\nu}}(x)$, we see that ${\rm\Theta}_{l,{\it\nu}}(z)$ is the zero function if $l$ is odd.

Lemma 3.1. Let $l$ be an even nonnegative integer. Let ${\rm\Theta}_{l,{\it\nu}}$ be as defined in (3.2), with $P_{l,{\it\nu}}$ satisfying (3.1). Then ${\rm\Theta}_{l,{\it\nu}}$ is a holomorphic modular form of weight $l+2$ with respect to ${\rm\Gamma}_{0}(2)$, and is a cusp form if $l\geqslant 2$. Moreover, we have the following transformation formula:

(3.3)$$\begin{eqnarray}{\rm\Theta}_{l,{\it\nu}}\left(\frac{-1}{2z}\right)=-{\it\epsilon}_{l,{\it\nu}}^{-1}2^{l/2+1}z^{l+2}{\rm\Theta}_{l,{\it\nu}}(z).\end{eqnarray}$$

Proof. Set

$$\begin{eqnarray}B=\left[\begin{array}{@{}cccc@{}}1\\ & 1\\ & & 1\\ -1 & -1 & -1 & 2\end{array}\right]\quad \text{and}\quad A=\text{}^{t}BB.\end{eqnarray}$$

By (2.5), we see that the map $x\mapsto Bx$ is a bijection from $\mathbb{Z}^{4}\simeq \mathbb{Z}+\mathbb{Z}i+\mathbb{Z}j+\mathbb{Z}k$ to $B(\mathbb{Z}^{4})\simeq S$. Here, we consider $x$ as a column vector. For $P_{l,{\it\nu}}$ in the statement of the lemma, set $P(x):=P_{l,{\it\nu}}(Bx),x\in \mathbb{Z}^{4}$. Then $P$ is a homogeneous polynomial in $4$ variables of degree $l$ annihilated by the operator

$$\begin{eqnarray}{\rm\Delta}_{A}=\mathop{\sum }_{i,j=1}^{4}b_{i,j}\frac{\partial ^{2}}{\partial \,x_{i}\partial \,x_{j}},\quad \text{where }A^{-1}=(b_{i,j}).\end{eqnarray}$$

One can then see that ${\rm\Theta}_{l,{\it\nu}}(z)={\rm\Theta}(z;A,P)$, where

$$\begin{eqnarray}{\rm\Theta}(z;A,P)=\mathop{\sum }_{m\in \mathbb{Z}^{4}}P(m)e^{2{\it\pi}\sqrt{-1}\frac{\text{}^{t}mAm}{2}z}\end{eqnarray}$$

is as defined in [Reference Miyake20, Corollary 4.9.5]. Since all diagonal entries of $A$ and $2A^{-1}$ are even, [Reference Miyake20, part (3) of Corollary 4.9.5] implies that ${\rm\Theta}_{l,{\it\nu}}$ is a holomorphic modular form of weight $l+2$ with respect to ${\rm\Gamma}_{0}(2)$, and is a cusp form if $l\geqslant 2$. Once again, by [Reference Miyake20, part (3) of Corollary 4.9.5], we have

$$\begin{eqnarray}{\rm\Theta}\left(\frac{-1}{2z};A,P\right)=-2^{l+1}z^{l+2}{\rm\Theta}(z;A^{\ast },P^{\ast }),\end{eqnarray}$$

where $A^{\ast }=2A^{-1}$ and $P^{\ast }(x)=P(A^{-1}x)=P_{l,{\it\nu}}(\text{}^{t}B^{-1}x)$. Note that the map $x\mapsto \text{}^{t}B^{-1}x$ can be regarded as a bijection between $\mathbb{Z}^{4}$ and ${\mathcal{O}}$. Hence, we see that

$$\begin{eqnarray}\displaystyle {\rm\Theta}(z;A^{\ast },P^{\ast }) & = & \displaystyle \mathop{\sum }_{{\it\beta}\in {\mathcal{O}}}P_{l,{\it\nu}}({\it\beta})e^{2{\it\pi}\sqrt{-1}|{\it\beta}|^{2}z}\nonumber\\ \displaystyle & = & \displaystyle {\it\epsilon}_{l,{\it\nu}}^{-1}2^{-l/2}\mathop{\sum }_{m=0}^{\infty }b(2m)e^{2{\it\pi}\sqrt{-1}mz}={\it\epsilon}_{l,{\it\nu}}^{-1}2^{-l/2}{\rm\Theta}_{l,{\it\nu}}(z).\nonumber\end{eqnarray}$$

Here, we have used Lemma 2.5 and (3.1). This completes the proof of the lemma.◻

Eisenstein series with respect to ${\rm\Gamma}_{0}(2)$

We introduce an Eisenstein series

(3.4)$$\begin{eqnarray}\displaystyle \tilde{E}_{\infty }(z,s) & := & \displaystyle (4{\it\pi})^{l/2}\frac{{\rm\Gamma}(s+\frac{l}{2}+1)}{{\rm\Gamma}(s)}({\it\pi}^{-s}{\rm\Gamma}(s){\it\zeta}(2s))\frac{1}{2}\nonumber\\ \displaystyle & & \displaystyle \times \,\mathop{\sum }_{{\it\gamma}\in {\rm\Gamma}_{\infty }\setminus {\rm\Gamma}_{0}(2)}\left(\frac{cz+d}{|cz+d|}\right)^{l+2}\left(\frac{\operatorname{Im}(z)}{|cz+d|^{2}}\right)^{s}\end{eqnarray}$$

on $\mathfrak{h}$ with a complex parameter $s$, where ${\rm\Gamma}_{\infty }:=\left\{\left.\left[\!\begin{smallmatrix}1 & m\phantom{}\\ 0 & 1\end{smallmatrix}\!\right]\right|m\in \mathbb{Z}\right\}$. The Eisenstein series satisfies the following functional equation.

Lemma 3.2. Let $\tilde{E}_{0}(z,s):=(z/|z|)^{l+2}\tilde{E}_{\infty }(-1/2z,s)$. Then the functional equation

$$\begin{eqnarray}\tilde{E}_{\infty }(z,1-s)=\frac{2^{2s-2}}{1-2^{2s-2}}\tilde{E}_{\infty }(z,s)+\frac{2^{s-1}(1-2^{2s-1})}{1-2^{2s-2}}\tilde{E}_{0}(z,s)\end{eqnarray}$$

holds.

Let $f\in S({\rm\Gamma}_{0}(2);-(\frac{1}{4}+(r/2)^{2}))$ be an eigenfunction with respect to the Atkin–Lehner involution; that is, $f(-1/2z)={\it\epsilon}f(z)$ with some ${\it\epsilon}\in \{\pm 1\}$. Suppose the form $f$ has the Fourier expansion

$$\begin{eqnarray}f(x+\sqrt{-1}y)=\mathop{\sum }_{n\in \mathbb{Z}\setminus \{0\}}c(n)W_{0,\frac{\sqrt{-1}r}{2}}(4{\it\pi}|n|y)e^{2{\it\pi}\sqrt{-1}nx},\end{eqnarray}$$

where $W_{0,\sqrt{-1}r/2}$ denotes the Whittaker function with the parameter $(0,\sqrt{-1}r/2)$. Let $l$ be an even nonnegative integer. Let ${\rm\Theta}_{l,{\it\nu}}$ be as defined in (3.2), with $P_{l,{\it\nu}}$ satisfying (3.1). Let $\tilde{E}_{\infty }(z,s)$ be the Eisenstein series defined in (3.4). Let the zeta integral $I(s)$ be defined by

(3.5)$$\begin{eqnarray}I(s):=\int _{{\rm\Gamma}_{0}(2)\setminus \mathfrak{h}}f(z){\rm\Theta}_{l,{\it\nu}}(z)\tilde{E}_{\infty }(z,s)y^{l+2/2}\frac{dxdy}{y^{2}}.\end{eqnarray}$$

A similar Rankin–Selberg integral was considered in [Reference Duke and Imamoḡlu7, Theorem 6] (see also [Reference Pitale22, (3.16)]). This integral is the main tool to prove the analytic properties of certain Dirichlet series associated with the lifting defined in Section 4. We would like to remark that the integral $I(s)$ is, in general, not Eulerian and does not represent an $L$-function of $f$.

By Lemma 3.1, the above integral is well defined. Let us now state the theorem of this section.

Theorem 3.3. The zeta integral $I(s)$ is entire and is bounded on vertical strips. When ${\it\epsilon}{\it\epsilon}_{l,{\it\nu}}=1$, we have

$$\begin{eqnarray}(2^{s}-1)I(s)=(2^{1-s}-1)I(1-s).\end{eqnarray}$$

When ${\it\epsilon}{\it\epsilon}_{l,{\it\nu}}=-1$, we have

$$\begin{eqnarray}(2^{s}+1)I(s)=(2^{1-s}+1)I(1-s).\end{eqnarray}$$

Proof. The entireness and boundedness on vertical strips of $I(s)$ are verified by the same argument as [Reference Pitale22, §3.2]. We put

$$\begin{eqnarray}I_{0}(s):=\int _{{\rm\Gamma}_{0}(2)\setminus \mathfrak{h}}f(z){\rm\Theta}_{l,{\it\nu}}(z)\tilde{E}_{0}(z,s)y^{l+2/2}\frac{dxdy}{y^{2}}.\end{eqnarray}$$

Since ${\rm\Gamma}_{0}(2)$ is stable under conjugation by $\left[\begin{smallmatrix} & 1/\sqrt{2}\\ -\sqrt{2} & \end{smallmatrix}\right]\in \text{SL}_{2}(\mathbb{R})$, we can make a change of variable $z\mapsto -1/(2z)$. Now, using the assumption $f(-1/2z)={\it\epsilon}f(z)$, (3.3) and the definition of $\tilde{E}_{0}(z,s)$, we have

$$\begin{eqnarray}\displaystyle I_{0}(s) & = & \displaystyle \int _{{\rm\Gamma}_{0}(2)\setminus \mathfrak{h}}f\left(-\frac{1}{2z}\right){\rm\Theta}_{l,{\it\nu}}\left(-\frac{1}{2z}\right)\tilde{E}_{0}\left(-\frac{1}{2z},s\right)\text{Im}\left(\frac{-1}{2z}\right)^{l+2/2}\frac{dxdy}{y^{2}}\nonumber\\ \displaystyle & = & \displaystyle \int _{{\rm\Gamma}_{0}(2)\setminus \mathfrak{h}}\Big[{\it\epsilon}f(z)\Big]\left[-{\it\epsilon}_{l,{\it\nu}}^{-1}2^{l/2+1}z^{l+2}{\rm\Theta}_{l,{\it\nu}}(z)\right]\nonumber\\ \displaystyle & & \displaystyle \times \left[\left(\frac{-|z|}{z}\right)^{l+2}\tilde{E}_{\infty }(z,s)\right]\left[\frac{y}{2|z|^{2}}\right]^{l+2/2}\frac{dxdy}{y^{2}}\nonumber\\ \displaystyle & = & \displaystyle -{\it\epsilon}_{l,v}^{-1}{\it\epsilon}I(s).\nonumber\end{eqnarray}$$

From Lemma 3.2, we deduce

$$\begin{eqnarray}I(1-s)=\frac{2^{2s-2}}{1-2^{2s-2}}I(s)+\frac{2^{s-1}(1-2^{2s-1})}{1-2^{2s-2}}I_{0}(s).\end{eqnarray}$$

Observe that

$$\begin{eqnarray}\frac{2^{2s-2}}{1-2^{2s-2}}-{\it\epsilon}_{l,{\it\nu}}^{-1}{\it\epsilon}\frac{2^{s-1}(1-2^{2s-1})}{1-2^{2s-2}}=\left\{\begin{array}{@{}ll@{}}\displaystyle \frac{2^{s}-1}{2^{1-s}-1}\quad & \text{if }{\it\epsilon}_{l,{\it\nu}}{\it\epsilon}=1,\\ \displaystyle \frac{2^{s}+1}{2^{1-s}+1}\quad & \text{ if }{\it\epsilon}_{l,{\it\nu}}{\it\epsilon}=-1.\end{array}\right.\end{eqnarray}$$

We have therefore proved the theorem. ◻

5.1 Adelization of automorphic forms

To study the actions of Hecke operators on our cusp forms constructed by the lifting we need both adelic and nonadelic treatments of automorphic forms.

For a complex number $r\in \mathbb{C}$ we introduce another space $M({\mathcal{G}}(\mathbb{A}),r)$ of automorphic forms for ${\mathcal{G}}$.

According to part (1) of Lemma 2.3, the class number of ${\mathcal{G}}$ with respect to $U$ is one, which means that ${\mathcal{G}}(\mathbb{A})={\mathcal{G}}(\mathbb{Q}){\mathcal{G}}(\mathbb{R})U$. We can thus view $F\in M(\text{GL}_{2}({\mathcal{O}}),r)$ as a smooth function ${\rm\Phi}_{F}$ on ${\mathcal{G}}(\mathbb{A})$ by

$$\begin{eqnarray}{\rm\Phi}_{F}({\it\gamma}g_{\infty }u_{f})=F(g_{\infty })\quad \forall ({\it\gamma},g_{\infty },u_{f})\in {\mathcal{G}}(\mathbb{Q})\times {\mathcal{G}}(\mathbb{R})\times U.\end{eqnarray}$$

We therefore see the following.

5.2 Hecke operators

For each place $p\leqslant \infty$, let ${\mathcal{G}}_{p}:=\text{GL}_{2}(B_{p})$, with $B_{p}=B\otimes _{\mathbb{Q}}\mathbb{Q}_{p}$. For a finite prime $p\not =2$, we have $\text{GL}_{2}(B_{p})\simeq \text{GL}_{4}(\mathbb{Q}_{p})$. Let ${\mathcal{O}}_{p}$ be the $p$-adic completion of ${\mathcal{O}}$ for $p<\infty$. For a finite prime $p\not =2$, ${\mathcal{O}}_{p}\simeq M_{2}(\mathbb{Z}_{p})$ and $\text{GL}_{2}({\mathcal{O}}_{p})\simeq \text{GL}_{4}(\mathbb{Z}_{p})$. Set $K_{p}=\text{GL}_{2}({\mathcal{O}}_{p})$ for $p<\infty$.

We denote by ${\mathcal{H}}_{p}={\mathcal{H}}({\mathcal{G}}_{p},K_{p})$ the Hecke algebra for $\text{GL}_{2}(B_{p})$ with respect to $\text{GL}_{2}({\mathcal{O}}_{p})$ for $p<\infty$. According to [Reference Satake26, §8, Theorem 6], ${\mathcal{H}}_{p}$ has the following generators:

$$\begin{eqnarray}\left\{\begin{array}{@{}ll@{}}\{{\it\varphi}_{1}^{\pm 1},~{\it\varphi}_{2}\}\quad & \text{if }p=2,\\ \{{\it\phi}_{1}^{\pm 1},~{\it\phi}_{2},~{\it\phi}_{3},~{\it\phi}_{4}\}\quad & \text{if }p\not =2.\end{array}\right.\end{eqnarray}$$

Here, ${\it\varphi}_{1},~{\it\varphi}_{2}$ denote the characteristic functions for

(5.1)$$\begin{eqnarray}K_{2}\left[\begin{array}{@{}cc@{}}{\it\varpi}_{2} & 0\\ 0 & {\it\varpi}_{2}\end{array}\right]K_{2},\quad K_{2}\left[\begin{array}{@{}cc@{}}{\it\varpi}_{2} & 0\\ 0 & 1\end{array}\right]K_{2}\end{eqnarray}$$

respectively, and ${\it\phi}_{1},~{\it\phi}_{2},~{\it\phi}_{3},~{\it\phi}_{4}$ denote the characteristic functions for

(5.2)$$\begin{eqnarray}\displaystyle K_{p}\left[\begin{array}{@{}cccc@{}}p & & & \\ & p & & \\ & & p & \\ & & & p\end{array}\right]K_{p},~K_{p}\left[\begin{array}{@{}cccc@{}}p & & & \\ & p & & \\ & & p & \\ & & & 1\end{array}\right]K_{p}, & & \displaystyle \nonumber\\ \displaystyle ~K_{p}\left[\begin{array}{@{}cccc@{}}p & & & \\ & p & & \\ & & 1 & \\ & & & 1\end{array}\right]K_{p},~K_{p}\left[\begin{array}{@{}cccc@{}}p & & & \\ & 1 & & \\ & & 1 & \\ & & & 1\end{array}\right]K_{p} & & \displaystyle\end{eqnarray}$$

respectively when $p\not =2$. Recall that ${\it\varpi}_{2}$ denotes a prime element of $B_{2}$. We want to obtain the single coset decomposition for the above double cosets. For that, we next review the Bruhat decomposition of $K_{p}$ given by

$$\begin{eqnarray}K_{p}=\mathop{\sqcup }_{w\in W_{p}}T_{p}wT_{p},\end{eqnarray}$$

where $W_{p}$ denotes the Weyl group of $\text{GL}_{2}(B_{p})$, and $T_{p}$ is the subgroup of elements in $K_{p}$ which are upper triangular modulo $p$ for an odd $p$ (respectively, upper triangular modulo ${\it\varpi}_{2}{\mathcal{O}}_{2}$ for $p=2$).

Let $N_{p}$ be the standard maximal unipotent subgroup of $\text{GL}_{2}(B_{p})$ defined over $\mathbb{Q}_{p}$. We put $N_{p}^{0}(\mathbb{Z}_{p}):=T_{p}\cap \text{}^{t}N_{p}(\mathbb{Q}_{p})$. We furthermore introduce $N(\mathbb{Z}_{p})=N_{p}(\mathbb{Q}_{p})\cap K_{p}$ and $D(\mathbb{Z}_{p})=D_{p}(\mathbb{Q}_{p})\cap K_{p}$, where $D_{p}$ denotes the subgroup of diagonal matrices in $\text{GL}_{2}(B_{p})$. Then we have the Iwahori decomposition

$$\begin{eqnarray}T_{p}=N(\mathbb{Z}_{p})D(\mathbb{Z}_{p})N_{p}^{0}(\mathbb{Z}_{p})\end{eqnarray}$$

(see [Reference Iwahori and Matsumoto13, Theorem 2.5]). Let $h$ be one of

$$\begin{eqnarray}\displaystyle 1_{4}~(\text{or }1_{2}),\quad \left[\begin{array}{@{}cccc@{}}p & & & \\ & p & & \\ & & p & \\ & & & p\end{array}\right],\quad \left[\begin{array}{@{}cccc@{}}p & & & \\ & p & & \\ & & p & \\ & & & 1\end{array}\right],\quad & & \displaystyle \nonumber\\ \displaystyle \left[\begin{array}{@{}cccc@{}}p & & & \\ & p & & \\ & & 1 & \\ & & & 1\end{array}\right],\quad \left[\begin{array}{@{}cccc@{}}p & & & \\ & 1 & & \\ & & 1 & \\ & & & 1\end{array}\right]~\text{or}~\left[\begin{array}{@{}cc@{}}{\it\varpi}_{2} & \\ & 1\end{array}\right]. & & \displaystyle \nonumber\end{eqnarray}$$

Lemma 5.3.

$$\begin{eqnarray}K_{p}hK_{p}=\mathop{\sqcup }_{w\in W_{p}/W_{p}(h)}N(\mathbb{Z}_{p})whK_{p},\end{eqnarray}$$

where $W_{p}(h):=\{w\in W_{p}\mid whw^{-1}=h\}$.

To describe this coset decomposition of $K_{p}hK_{p}$ explicitly, we need a set of representatives for $W_{p}/W_{p}(h)$.

By Lemmas 5.3 and 5.4, we are now able to write down the coset decomposition of $K_{p}hK_{p}$ explicitly.

We can now describe the actions of the Hecke operators defined by the $K_{p}hK_{p}$s above. With the invariant measure $dx$ of ${\mathcal{G}}_{p}$ normalized so that $\int _{K_{p}}dx=1$, we define $K_{p}hK_{p}\cdot {\rm\Phi}$ by

$$\begin{eqnarray}(K_{p}hK_{p}\cdot {\rm\Phi})(g):=\displaystyle \int _{{\mathcal{G}}_{p}}\operatorname{char}_{K_{p}hK_{p}}(x){\rm\Phi}(gx)dx\end{eqnarray}$$

for ${\rm\Phi}\in M({\mathcal{G}}(\mathbb{A}),r)$, where $\operatorname{char}_{K_{p}hK_{p}}$ denotes the characteristic function for $K_{p}hK_{p}$.

We provide the nonadelic description of $K_{p}hK_{p}\cdot {\rm\Phi}$, which enables us to describe explicitly the influence of the $K_{p}hK_{p}$-action on Fourier coefficients of the lifting $F_{f}$.

To this end we now introduce the sets

(5.3)$$\begin{eqnarray}\displaystyle C_{p}:=\{{\it\alpha}\in {\mathcal{O}}\mid {\it\nu}({\it\alpha})=p\}/{\mathcal{O}}^{\times },\quad C_{p}^{\prime }:=\{x\in M_{2}(\mathbb{Z}_{p})\mid \det (x)=p\}/GL_{2}(\mathbb{Z}_{p}), & & \displaystyle \nonumber\\ \displaystyle & & \displaystyle\end{eqnarray}$$

and state the following lemma.

Let $F\in M(\text{GL}_{2}({\mathcal{O}}),r)$ correspond to ${\rm\Phi}$. By $K_{p}hK_{p}\cdot F$ we denote the cusp form in $M(\text{GL}_{2}({\mathcal{O}}),r)$ corresponding to $K_{p}hK_{p}\cdot {\rm\Phi}$. Due to Lemma 2.3, ${\mathcal{G}}(\mathbb{Q})\setminus {\mathcal{G}}(\mathbb{A})/U$ has a complete set of representatives in ${\mathcal{G}}(\mathbb{R})$, where ${\mathcal{G}}(\mathbb{R})$ is viewed as a subgroup of ${\mathcal{G}}(\mathbb{A})$ in the usual manner. The cusp form $K_{p}hK_{p}\cdot {\rm\Phi}$ is thereby determined by its restriction to ${\mathcal{G}}(\mathbb{R})$, which is nothing but $K_{p}hK_{p}\cdot F$. Moreover, we remark that an element in $M(\text{GL}_{2}({\mathcal{O}}),r)$ is determined by its restriction to $NA=\{n(x)a_{y}\mid x\in \mathbb{H},~y\in \mathbb{R}_{+}^{\times }\}$ (see (2.1)).

Proof. We prove only (2c). The other cases are settled similarly. We make use of the isomorphism ${\it\psi}:{\mathcal{O}}_{p}\rightarrow M_{2}(\mathbb{Z}_{p})$, given explicitly as follows. Since $p$ is odd, we can find $r,s\in \mathbb{Z}_{p}$ such that $1+r^{2}+s^{2}=0$. Then, we have

(5.4)$$\begin{eqnarray}{\it\psi}(a_{0}+a_{1}i+a_{2}j+a_{3}ij)=\left[\begin{array}{@{}ccc@{}}a_{0}-a_{1}r-a_{2}s & & -a_{3}-a_{1}s+a_{2}r\\ a_{3}-a_{1}s+a_{2}r & & a_{0}+a_{1}r+a_{2}s\end{array}\right].\end{eqnarray}$$

Let $x_{1},x_{2},x_{3},x_{4}\in {\mathcal{O}}$ such that

$$\begin{eqnarray}\displaystyle {\it\psi}(x_{1})\in \left[\begin{array}{@{}cc@{}}1 & 0\\ 0 & 0\end{array}\right]+M_{2}(p\mathbb{Z}_{p}),\qquad {\it\psi}(x_{2})\in \left[\begin{array}{@{}cc@{}}0 & 1\\ 0 & 0\end{array}\right]+M_{2}(p\mathbb{Z}_{p}), & & \displaystyle \nonumber\\ \displaystyle {\it\psi}(x_{3})\in \left[\begin{array}{@{}cc@{}}0 & 0\\ 1 & 0\end{array}\right]+M_{2}(p\mathbb{Z}_{p}),\qquad {\it\psi}(x_{4})\in \left[\begin{array}{@{}cc@{}}0 & 0\\ 0 & 1\end{array}\right]+M_{2}(p\mathbb{Z}_{p}). & & \displaystyle \nonumber\end{eqnarray}$$

For example, let $r_{0},s_{0}\in \{0,1,\ldots ,p-1\}$ such that $r-r_{0},s-s_{0}\in p\mathbb{Z}_{p}$. Then, we can choose $x_{4}=\frac{p+1}{2}(1-r_{0}i-s_{0}j)$, where note that $\frac{p+1}{2}\in \mathbb{Z}$ since $p$ is odd. Denote by ${\it\alpha}_{0},{\it\alpha}_{1},{\it\alpha}_{2},\ldots ,{\it\alpha}_{p}$ the representatives of $C_{p}$ such that

$$\begin{eqnarray}{\it\psi}({\it\alpha}_{0})\in \left[\begin{array}{@{}cc@{}}1 & \\ & p\end{array}\right]\text{GL}_{2}(\mathbb{Z}_{p})\quad \text{and}\quad {\it\psi}({\it\alpha}_{i})\in \left[\begin{array}{@{}cc@{}}p & i\\ & 1\end{array}\right]\text{GL}_{2}(\mathbb{Z}_{p})\quad \text{for }i\in \{1,\ldots ,p\}.\end{eqnarray}$$

The left coset decomposition of $K_{p}hK_{p}$ in part (2c) of Lemma 5.5 can be rewritten as

$$\begin{eqnarray}\displaystyle K_{p}hK_{p} & = & \displaystyle \mathop{\sqcup }_{x\in {\mathcal{O}}/p{\mathcal{O}}}\left[\begin{array}{@{}cc@{}}{\it\psi}(1) & {\it\psi}(x)\\ & {\it\psi}(1)\end{array}\right]\left[\begin{array}{@{}cc@{}}{\it\psi}(p) & \\ & {\it\psi}(1)\end{array}\right]K_{p}\nonumber\\ \displaystyle & \sqcup & \displaystyle \mathop{\sqcup }_{i,t\in \{1,\ldots ,p\}}\left[\begin{array}{@{}cc@{}}{\it\psi}(1) & t{\it\psi}(x_{4})\\ & {\it\psi}(1)\end{array}\right]\left[\begin{array}{@{}cc@{}}{\it\psi}({\it\alpha}_{0}) & \\ & {\it\psi}({\it\alpha}_{i})\end{array}\right]K_{p}\nonumber\\ \displaystyle & \sqcup & \displaystyle \mathop{\sqcup }_{t\in \{1,\ldots ,p\}}\left[\begin{array}{@{}cc@{}}{\it\psi}(1) & t{\it\psi}(x_{3})\\ & {\it\psi}(1)\end{array}\right]\left[\begin{array}{@{}cc@{}}{\it\psi}({\it\alpha}_{0}) & \\ & {\it\psi}({\it\alpha}_{0})\end{array}\right]K_{p}\nonumber\\ \displaystyle & \sqcup & \displaystyle \mathop{\sqcup }_{i,j,t\in \{1,\ldots ,p\}}\left[\begin{array}{@{}cc@{}}{\it\psi}(1) & t{\it\psi}(x_{2})\\ & {\it\psi}(1)\end{array}\right]\left[\begin{array}{@{}cc@{}}{\it\psi}({\it\alpha}_{i}) & \\ & {\it\psi}({\it\alpha}_{j})\end{array}\right]K_{p}\nonumber\\ \displaystyle & \sqcup & \displaystyle \mathop{\sqcup }_{i,t\in \{1,\ldots ,p\}}\left[\begin{array}{@{}cc@{}}{\it\psi}(1) & t{\it\psi}(x_{1})\\ & {\it\psi}(1)\end{array}\right]\left[\begin{array}{@{}cc@{}}{\it\psi}({\it\alpha}_{i}) & \\ & {\it\psi}({\it\alpha}_{0})\end{array}\right]K_{p}\nonumber\\ \displaystyle & \sqcup & \displaystyle \left[\begin{array}{@{}cc@{}}{\it\psi}(1) & \\ & {\it\psi}(p)\end{array}\right]K_{p}.\nonumber\end{eqnarray}$$

We note that $(K_{p}hK_{p}{\rm\Phi})((n(x)a_{y})_{\infty }\prod _{v<\infty }1_{v})$ is the sum of terms of the form ${\rm\Phi}((n(x)a_{y})_{\infty }\cdot {\it\gamma}_{p}\cdot \prod _{v\neq p,\infty }1_{v})$, where ${\it\gamma}_{p}$ runs over the single coset representatives above. Since we have chosen the coset representatives ${\it\gamma}_{p}$ above in $M_{2}({\mathcal{O}})$, ${\rm\Phi}((n(x)a_{y})_{\infty }\cdot {\it\gamma}_{p}\cdot \prod _{v\neq p,\infty }1_{v})={\rm\Phi}(({\it\gamma}_{p}^{-1}n(x)a_{y})_{\infty }\prod _{v<\infty }1_{v})=F({\it\gamma}_{p}^{-1}n(x)a_{y})$. Here, we have used the left ${\mathcal{G}}(\mathbb{Q})$-invariance and right $K_{v}$-invariance of ${\rm\Phi}$.

Next, we substitute the Fourier expansion of $F$ and perform an appropriate change of variable for ${\it\beta}$ to get the proposition. Note that, for the first set of left cosets, we have to use that ${\mathcal{O}}/p{\mathcal{O}}\ni x\mapsto \exp (2{\it\pi}i\operatorname{Re}({\it\beta}x/p))$ is a nontrivial character of $(\mathbb{Z}/p\mathbb{Z})^{4}$ if and only if $p$ does not divide ${\it\beta}$.◻

For this proposition we remark that the formulas above do not depend on the choices of representatives of $C_{p}$, since $F$ is left- and right-invariant with respect to $\{\left[\!\begin{smallmatrix}u_{1} & \phantom{}\\ & u_{2}\end{smallmatrix}\!\right]\mid u_{1},~u_{2}\in {\mathcal{O}}^{\times }\}$. Let the Fourier decomposition of $(K_{p}hK_{p}\cdot F)$ be given by

$$\begin{eqnarray}(K_{p}hK_{p}\cdot F)(n(x)a_{y})=\mathop{\sum }_{{\it\beta}\in S\setminus \{0\}}(K_{p}hK_{p}\cdot F)_{{\it\beta}}y^{2}K_{\sqrt{-1}r}(2{\it\pi}|{\it\beta}|y)e^{2{\it\pi}\sqrt{-1}\operatorname{Re}({\it\beta}x)}.\end{eqnarray}$$

The next proposition provides a formula for $(K_{p}hK_{p}\cdot F)_{{\it\beta}}$ in terms of the Fourier coefficients $A({\it\beta})$ of $F$.

For this proposition we note that the automorphy of $F$ with respect to $\{\left[\!\begin{smallmatrix}u_{1} & \phantom{}\\ & u_{2}\end{smallmatrix}\!\right]\mid u_{1},~u_{2}\in {\mathcal{O}}^{\times }\}$ implies $A(u_{1}{\it\beta}u_{2})=A({\it\beta})$ for ${\it\beta}\in S\setminus \{0\}$ and$u_{1},u_{2}\in {\mathcal{O}}^{\times }$. From this we see that the formulas in (2b) and (2c) do not depend on the choices of representatives for $C_{p}$. For (2a) we furthermore see that, given any complete set $\{{\it\alpha}_{i}\mid 0\leqslant i\leqslant p\}$ of representatives for $C_{p}$, $\{\bar{{\it\alpha}_{i}}\mid 0\leqslant i\leqslant p\}$ also forms such a set. As a result, we see that the formula in (2a) is also not dependent on the choices of representatives for $C_{p}$.

5.3 Hecke equivariance for $p=2$

Let $f\in S({\rm\Gamma}_{0}(2);-(\frac{1}{4}+r^{2}/4))$ be a new form (for the definition see [Reference Iwaniec12, §8.5]) with Hecke eigenvalue ${\it\lambda}_{p}$ for $p=2$. By the Hecke eigenvalue ${\it\lambda}_{2}$ we mean the eigenvalue of $f$ for the $U(2)$ operator defined by the action of the double coset ${\rm\Gamma}_{0}(2)\left[\begin{smallmatrix}1 & \\ & 2\end{smallmatrix}\right]\,{\rm\Gamma}_{0}(2)$. Let us also assume that $f$ is an eigenfunction of the Atkin–Lehner involution with eigenvalue ${\it\epsilon}$. It can be checked that ${\it\lambda}_{2}$ and ${\it\epsilon}$ are related by

(5.5)$$\begin{eqnarray}{\it\lambda}_{2}=-{\it\epsilon}.\end{eqnarray}$$

Using the single coset decomposition

$$\begin{eqnarray}{\rm\Gamma}_{0}(2)\left[\begin{array}{@{}cc@{}}1 & \\ & 2\end{array}\right]{\rm\Gamma}_{0}(2)={\rm\Gamma}_{0}(2)\left[\begin{array}{@{}cc@{}}1 & 1\\ & 2\end{array}\right]\sqcup {\rm\Gamma}_{0}(2)\left[\begin{array}{@{}cc@{}}1 & \\ & 2\end{array}\right],\end{eqnarray}$$

we get

$$\begin{eqnarray}f\left(\frac{z+1}{2}\right)+f\left(\frac{z}{2}\right)={\it\lambda}_{2}f(z).\end{eqnarray}$$

In terms of Fourier coefficients of $f$, using (5.5), we get

(5.6)$$\begin{eqnarray}c(2m)=\frac{{\it\lambda}_{2}}{2}c(m)=-\frac{{\it\epsilon}}{2}c(m),\quad \text{for all }m\in \mathbb{Z}.\end{eqnarray}$$

Proposition 5.9. Let $f\in S({\rm\Gamma}_{0}(2);-(\frac{1}{4}+r^{2}/4))$ be a new form and an eigenfunction of the Atkin–Lehner involution with eigenvalue ${\it\epsilon}$. Let $F=F_{f}$ be as defined in Theorem 4.4. Then,

(5.7)$$\begin{eqnarray}\left(K_{2}\left[\begin{array}{@{}cc@{}}{\it\varpi}_{2} & \\ & 1\end{array}\right]K_{2}\right)F=-3\sqrt{2}{\it\epsilon}F.\end{eqnarray}$$

Proof. Let ${\it\beta}={\it\varpi}_{2}^{u}d{\it\beta}_{0}$ be a decomposition according to Proposition 4.2. Hence, $u\geqslant 0,d$ is odd and ${\it\beta}_{0}\in S^{\text{prim}}$. Using (5.5) and (5.6), we see that

$$\begin{eqnarray}\displaystyle A({\it\beta}) & = & \displaystyle (2^{u+1}-1)|{\it\beta}|\mathop{\sum }_{n|d}c\Big(\frac{-|{\it\beta}|^{2}}{2n^{2}}\Big),\nonumber\\ \displaystyle A({\it\beta}{\it\varpi}_{2}) & = & \displaystyle (2^{u+2}-1)\frac{-{\it\epsilon}}{\sqrt{2}}|{\it\beta}|\mathop{\sum }_{n|d}c\Big(\frac{-|{\it\beta}|^{2}}{2n^{2}}\Big),\nonumber\\ \displaystyle A({\it\beta}{\it\varpi}_{2}^{-1}) & = & \displaystyle (2^{u}-1)(-{\it\epsilon}\sqrt{2})|{\it\beta}|\mathop{\sum }_{n|d}c\Big(\frac{-|{\it\beta}|^{2}}{2n^{2}}\Big).\nonumber\end{eqnarray}$$

Note that, if $u=0$, then $A({\it\beta}{\it\varpi}_{2}^{-1})=0$, and so is the right-hand side of the third equality above. We have

$$\begin{eqnarray}\frac{2^{u+2}-1}{\sqrt{2}}+(2^{u}-1)\sqrt{2}=\frac{3}{\sqrt{2}}(2^{u+1}-1).\end{eqnarray}$$

Hence, we have

$$\begin{eqnarray}2(A({\it\beta}{\it\varpi}_{2})+A({\it\beta}{\it\varpi}_{2}^{-1}))=-3\sqrt{2}{\it\epsilon}A({\it\beta}).\end{eqnarray}$$

The proposition now follows from part (1) of Proposition 5.8. ◻

5.4 Hecke equivariance for odd primes

We assume that $f\in S({\rm\Gamma}_{0}(2);-(\frac{1}{4}+r^{2}/4))$ is a Hecke eigenform with Hecke eigenvalue ${\it\lambda}_{p}$ for every odd prime $p$, but do not assume that $f$ is a new form. In terms of Fourier coefficients of $f$, the Hecke relation is given by

(5.8)$$\begin{eqnarray}p^{1/2}c(pn)+p^{-1/2}c(n/p)={\it\lambda}_{p}c(n),\end{eqnarray}$$

where $c(n/p)$ is assumed to be zero if $p$ does not divide $n$. The following lemma will play a key role in the computation of the Hecke operator.

Lemma 5.10. Let ${\it\beta}\in S^{\text{prim}}$. Then,

(5.9)$$\begin{eqnarray}\#\{{\it\alpha}\in C_{p}:p\mid {\it\beta}{\it\alpha}\}=\#\{{\it\alpha}\in C_{p}:p|{\it\alpha}{\it\beta}\}=\left\{\begin{array}{@{}l@{}}1\quad \text{if }p\,|\,|{\it\beta}|^{2},\quad \\ 0\quad \text{if }p\not |\,|{\it\beta}|^{2}.\quad \end{array}\right.\end{eqnarray}$$

In addition, $p^{2}$ does not divide ${\it\alpha}{\it\beta}$ or ${\it\beta}{\it\alpha}$ for any ${\it\alpha}\in C_{p}$.

Proof. Note that, by taking conjugates, it is enough to prove the statement of the lemma for $\{{\it\alpha}\in C_{p}:p\mid {\it\beta}{\it\alpha}\}$ for all ${\it\beta}$. Taking norms, it is clear that $p$ does not divide ${\it\beta}{\it\alpha}$ if $p$ does not divide $|{\it\beta}|^{2}$. Hence, assume that $p$ divides $|{\it\beta}|^{2}$. Let ${\it\beta}={\it\beta}_{1}+{\it\beta}_{2}i+{\it\beta}_{3}j+{\it\beta}_{4}ij$. The conditions $p||{\it\beta}|^{2}$ and $p\not |\,{\it\beta}$, imply that there is a pair amongst the set $\{{\it\beta}_{1},{\it\beta}_{2},{\it\beta}_{3},{\it\beta}_{4}\}$ which does not satisfy $x^{2}+y^{2}\equiv 0~(\text{mod}~p)$. From the proof it will be clear that we can take, without loss of generality, ${\it\beta}_{3}^{2}+{\it\beta}_{4}^{2}\not \equiv 0~(\text{mod}~p)$. Let ${\it\alpha}={\it\alpha}_{1}+{\it\alpha}_{2}i+{\it\alpha}_{3}j+{\it\alpha}_{4}ij$. The condition $p|{\it\beta}{\it\alpha}$ is equivalent to the following matrix equation modulo $p$:

$$\begin{eqnarray}\underbrace{\left[\begin{array}{@{}cccc@{}}{\it\beta}_{1} & -{\it\beta}_{2} & -{\it\beta}_{3} & -{\it\beta}_{4}\\ {\it\beta}_{2} & {\it\beta}_{1} & -{\it\beta}_{4} & {\it\beta}_{3}\\ {\it\beta}_{3} & {\it\beta}_{4} & {\it\beta}_{1} & -{\it\beta}_{2}\\ {\it\beta}_{4} & -{\it\beta}_{3} & {\it\beta}_{2} & {\it\beta}_{1}\end{array}\right]}_{P_{{\it\beta}}}\left[\begin{array}{@{}c@{}}{\it\alpha}_{1}\\ {\it\alpha}_{2}\\ {\it\alpha}_{3}\\ {\it\alpha}_{4}\end{array}\right]=\left[\begin{array}{@{}c@{}}0\\ 0\\ 0\\ 0\end{array}\right].\end{eqnarray}$$

The matrix $P_{{\it\beta}}$ considered over $\mathbb{Z}_{p}$ has rank $2$. By our assumption ${\it\beta}_{3}^{2}+{\it\beta}_{4}^{2}\not \equiv 0~(\text{mod}~p)$, we see that the kernel of $P_{{\it\beta}}$ is spanned by the first two rows of $P_{{\it\beta}}$. Hence, ${\it\alpha}$ is given by

$$\begin{eqnarray}{\it\alpha}_{1}=a{\it\beta}_{1}+b{\it\beta}_{2},\quad {\it\alpha}_{2}=-a{\it\beta}_{2}+b{\it\beta}_{1},\quad {\it\alpha}_{3}=-a{\it\beta}_{3}-b{\it\beta}_{4},\quad {\it\alpha}_{4}=-a{\it\beta}_{4}+b{\it\beta}_{3},\end{eqnarray}$$

for some $a,b\in \mathbb{Z}_{p}$. This gives us ${\it\alpha}_{3}^{2}+{\it\alpha}_{4}^{2}=(a^{2}+b^{2})({\it\beta}_{3}^{2}+{\it\beta}_{4}^{2})\neq 0$. This is because, by assumption, ${\it\beta}_{3}^{2}+{\it\beta}_{4}^{2}\neq 0$, and $|{\it\alpha}|^{2}=(a^{2}+b^{2})|{\it\beta}|^{2}$ and $p^{2}$ does not divide $|{\it\alpha}|^{2}$.

In [Reference Pitale23, page 69], it is shown that the set $S_{1}=\{{\it\alpha}\in C_{p}:{\it\alpha}_{3}^{2}+{\it\alpha}_{4}^{2}\not \equiv 0~(\text{mod}~p)\}$ is in bijection with the set $S_{2}=\{(x,y)\in \mathbb{Z}_{p}\times \mathbb{Z}_{p}:x^{2}+y^{2}+1\equiv 0~(\text{mod}~p)\}$. The map from $S_{1}$ to $S_{2}$ is given as follows. For ${\it\alpha}\in S_{1}$, we obtain $(x_{{\it\alpha}},y_{{\it\alpha}})$ as the solution to the matrix equation modulo $p$ given by

$$\begin{eqnarray}\left[\begin{array}{@{}cc@{}}{\it\alpha}_{3} & {\it\alpha}_{4}\\ -{\it\alpha}_{4} & {\it\alpha}_{3}\end{array}\right]\left[\begin{array}{@{}c@{}}x_{{\it\alpha}}\\ y_{{\it\alpha}}\end{array}\right]=\left[\begin{array}{@{}c@{}}{\it\alpha}_{1}\\ {\it\alpha}_{2}\end{array}\right].\end{eqnarray}$$

One can check that $(x,y)\in S_{2}$ for the following choice of $x$ and $y$:

(5.10)$$\begin{eqnarray}x=\frac{-{\it\beta}_{1}{\it\beta}_{3}-{\it\beta}_{2}{\it\beta}_{4}}{{\it\beta}_{3}^{2}+{\it\beta}_{4}^{2}},\qquad y=\frac{{\it\beta}_{2}{\it\beta}_{3}-{\it\beta}_{4}{\it\beta}_{1}}{{\it\beta}_{3}^{2}+{\it\beta}_{4}^{2}}.\end{eqnarray}$$

If we take ${\it\alpha}\in S_{1}$ to be the preimage of the above $(x,y)$, then we can check that ${\it\alpha}\in \text{Ker}(P_{{\it\beta}})$. On the other hand, if ${\it\alpha}\in C_{p}$ belongs to $\text{Ker}(P_{{\it\beta}})$, then it can also be checked that the corresponding $(x_{{\it\alpha}},y_{{\it\alpha}})$ are equivalent modulo $p$ to those in (5.10). This completes the proof of (5.9).

If $p^{2}$ divides ${\it\beta}{\it\alpha}$ or ${\it\alpha}{\it\beta}$ for some ${\it\alpha}\in C_{p}$, then it is clear that ${\it\beta}$ cannot be primitive. This completes the proof of the lemma.◻

Proposition 5.11. Let $f\in S({\rm\Gamma}_{0}(2);-(\frac{1}{4}+r^{2}/4))$ be a Hecke eigenform with Hecke eigenvalue ${\it\lambda}_{p}$ for every odd prime $p$. Let $F=F_{f}$ be as defined in Theorem 4.4. For an odd prime $p$ we then have

(5.11)$$\begin{eqnarray}\displaystyle \left(K_{p}\left[\begin{array}{@{}cccc@{}}p\\ & p\\ & & p\\ & & & 1\end{array}\right]K_{p}\right)F=\left(K_{p}\left[\begin{array}{@{}cccc@{}}p\\ & 1\\ & & 1\\ & & & 1\end{array}\right]K_{p}\right)F=p(p+1){\it\lambda}_{p}F. & & \displaystyle \nonumber\\ \displaystyle & & \displaystyle\end{eqnarray}$$

Proof. Using Proposition 5.8 we can show that, if the Fourier coefficients satisfy $A({\it\beta})=A(\bar{{\it\beta}})$ for all ${\it\beta}\in S$ and the second equality in (5.11) holds, then so does the first equality. Since the Fourier coefficients of $F=F_{f}$ satisfy the above condition, we are reduced to showing the second equality in (5.11).

We will compute the action of the Hecke operator on the Fourier coefficients $A({\it\beta})$ of $F$. Since all the computations only involve the prime $p$, it will be enough to consider the case ${\it\beta}=p^{s}{\it\beta}_{0}$ with $s\geqslant 0$ and ${\it\beta}_{0}\in S^{\text{prim}}$. For such a ${\it\beta}$, we have

$$\begin{eqnarray}A({\it\beta})=p^{s}|{\it\beta}_{0}|\mathop{\sum }_{k=0}^{s}c\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2s-2k}}{2}\Big).\end{eqnarray}$$

Note that ${\it\alpha}^{-1}{\it\beta}_{0}=(1/p)\bar{{\it\alpha}}{\it\beta}_{0}$ for ${\it\alpha}\in B$ with ${\it\nu}({\it\alpha})=p$. Hence, for such ${\it\alpha}$, ${\it\alpha}^{-1}{\it\beta}_{0}\in S$ if and only if $p$ divides $\bar{{\it\alpha}}{\it\beta}_{0}$.

Let us first consider the case where $p$ does not divide $|{\it\beta}_{0}|^{2}$. Hence, by Lemma 5.10, we see that ${\it\alpha}^{-1}{\it\beta}_{0}\not \in S$ and $p$ does not divide ${\it\beta}_{0}{\it\alpha}$ for any ${\it\alpha}\in C_{p}$. Hence, for any ${\it\alpha}\in C_{p}$, we have

(5.12)$$\begin{eqnarray}A({\it\beta}{\it\alpha})=\sqrt{p}p^{s}|{\it\beta}_{0}|\mathop{\sum }_{k=0}^{s}c\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2s+1-2k}}{2}\Big)\end{eqnarray}$$

and

(5.13)$$\begin{eqnarray}A({\it\alpha}^{-1}{\it\beta})=\frac{1}{\sqrt{p}}p^{s}|{\it\beta}_{0}|\mathop{\sum }_{k=0}^{s-1}c\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2s-1-2k}}{2}\Big).\end{eqnarray}$$

Note that, if $s=0$, both the left- and right-hand sides of the last equation are zero. Now, using (5.8), we get for any ${\it\alpha}\in C_{p}$

$$\begin{eqnarray}\displaystyle A({\it\alpha}^{-1}{\it\beta})+A({\it\beta}{\it\alpha}) & = & \displaystyle p^{s}|{\it\beta}_{0}|\Big(\mathop{\sum }_{k=0}^{s-1}\Big(p^{-1/2}c\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2s-1-2k}}{2}\Big)\nonumber\\ \displaystyle & & \displaystyle +\,p^{1/2}c\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2s+1-2k}}{2}\Big)\Big)+p^{1/2}c\Big(\frac{-|{\it\beta}_{0}|^{2}p}{2}\Big)\Big)\nonumber\\ \displaystyle & = & \displaystyle p^{s}|{\it\beta}_{0}|\Big(\mathop{\sum }_{k=0}^{s-1}{\it\lambda}_{p}c\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2s-2k}}{2}\Big)+{\it\lambda}_{p}c\Big(\frac{-|{\it\beta}_{0}|^{2}}{2}\Big)\Big)\nonumber\\ \displaystyle & = & \displaystyle {\it\lambda}_{p}A({\it\beta}).\nonumber\end{eqnarray}$$

Now, using the fact that the number of elements in $C_{p}$ is $p+1$ and part (2b) of Proposition 5.8, we get the result.

Next, let us assume that $p$ divides $|{\it\beta}_{0}|^{2}$. By Lemma 5.10, there is a unique ${\it\alpha}_{1}\in C_{p}$ such that $p$ divides ${\it\beta}_{0}{\it\alpha}_{1}$, and a unique ${\it\alpha}_{2}\in C_{p}$ such that ${\it\alpha}_{2}^{-1}{\it\beta}_{0}\in S$. If ${\it\alpha}\in C_{p}$ but ${\it\alpha}\neq {\it\alpha}_{1}$, then the formula for $A({\it\beta}{\it\alpha})$ is the same as in (5.12). For ${\it\alpha}={\it\alpha}_{1}$, we have

$$\begin{eqnarray}A({\it\beta}{\it\alpha}_{1})=\sqrt{p}p^{s}|{\it\beta}_{0}|\mathop{\sum }_{k=0}^{s+1}c\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2s+1-2k}}{2}\Big).\end{eqnarray}$$

For ${\it\alpha}\in C_{p}$ but ${\it\alpha}\neq {\it\alpha}_{2}$, the formula for $A({\it\alpha}^{-1}{\it\beta})$ is the same as in (5.13). For ${\it\alpha}={\it\alpha}_{2}$, we have

$$\begin{eqnarray}A({\it\alpha}_{2}^{-1}{\it\beta})=\frac{1}{\sqrt{p}}p^{s}|{\it\beta}_{0}|\mathop{\sum }_{k=0}^{s}c\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2s-1-2k}}{2}\Big).\end{eqnarray}$$

Hence, we get the following:

$$\begin{eqnarray}\displaystyle & & \displaystyle \mathop{\sum }_{{\it\alpha}\in C_{p}}(A({\it\alpha}^{-1}{\it\beta})+A({\it\beta}{\it\alpha}))\nonumber\\ \displaystyle & & \displaystyle \quad =pp^{s}|{\it\beta}_{0}|\Big[\mathop{\sum }_{k=0}^{s}p^{1/2}c\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2s+1-2k}}{2}\Big)+\mathop{\sum }_{k=0}^{s-1}p^{-1/2}c\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2s-1-2k}}{2}\Big)\Big]\nonumber\\ \displaystyle & & \displaystyle \quad \quad +\,p^{s}|{\it\beta}_{0}|\Big[\mathop{\sum }_{k=0}^{s+1}p^{1/2}c\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2s+1-2k}}{2}\Big)+\mathop{\sum }_{k=0}^{s}p^{-1/2}c\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2s-1-2k}}{2}\Big)\Big]\nonumber\\ \displaystyle & & \displaystyle \quad =pp^{s}|{\it\beta}_{0}|\Big[\mathop{\sum }_{k=0}^{s-1}{\it\lambda}_{p}c\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2s-2k}}{2}\Big)+p^{1/2}c\Big(\frac{-|{\it\beta}_{0}|^{2}p}{2}\Big)\Big]\nonumber\\ \displaystyle & & \displaystyle \quad \quad +\,p^{s}|{\it\beta}_{0}|\Big[\mathop{\sum }_{k=0}^{s}{\it\lambda}_{p}c\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2s-2k}}{2}\Big)+p^{1/2}c\Big(\frac{-|{\it\beta}_{0}|^{2}}{2p}\Big)\Big]\nonumber\\ \displaystyle & & \displaystyle \quad ={\it\lambda}_{p}A({\it\beta})+pp^{s}|{\it\beta}_{0}|\Big[\mathop{\sum }_{k=0}^{s-1}{\it\lambda}_{p}c\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2s-2k}}{2}\Big)\nonumber\\ \displaystyle & & \displaystyle \qquad +\,p^{1/2}c\Big(\frac{-|{\it\beta}_{0}|^{2}p}{2}\Big)+p^{-1/2}c\Big(\frac{-|{\it\beta}_{0}|^{2}}{2p}\Big)\Big]\nonumber\\ \displaystyle & & \displaystyle \quad ={\it\lambda}_{p}A({\it\beta})+p{\it\lambda}_{p}A({\it\beta})=(p+1){\it\lambda}_{p}A({\it\beta}).\nonumber\end{eqnarray}$$

This completes the proof of the proposition. ◻

Proposition 5.12. Let $f\in S({\rm\Gamma}_{0}(2);-(\frac{1}{4}+r^{2}/4))$ be a Hecke eigenform with Hecke eigenvalue ${\it\lambda}_{p}$ for every odd prime $p$. Let $F=F_{f}$ be as defined in Theorem 4.4. For an odd prime $p$ we then have

(5.14)$$\begin{eqnarray}\left(K_{p}\left[\begin{array}{@{}cccc@{}}p\\ & p\\ & & 1\\ & & & 1\end{array}\right]K_{p}\right)F=\big(p^{2}{\it\lambda}_{p}^{2}+p^{3}+p\big)F.\end{eqnarray}$$

Proof. First observe that, using (5.8), one can show that, for all $n$,

(5.15)$$\begin{eqnarray}pc(np^{2})=({\it\lambda}_{p}^{2}-1)c(n)-p^{-1/2}{\it\lambda}_{p}c(n/p).\end{eqnarray}$$

If we assume that $p|n$, then we can get another identity given by

(5.16)$$\begin{eqnarray}pc(np^{2})+p^{-1}c(n/p^{2})=({\it\lambda}_{p}^{2}-2)c(n).\end{eqnarray}$$

As in the proof of Proposition 5.11, we can assume that ${\it\beta}=p^{s}{\it\beta}_{0}$, where $s\geqslant 0$ and ${\it\beta}_{0}\in S^{\text{prim}}$. Let us abbreviate ${\it\nu}_{p}({\it\beta})=s$. For such a ${\it\beta}$ we have

$$\begin{eqnarray}A({\it\beta})=p^{s}|{\it\beta}_{0}|\mathop{\sum }_{k=0}^{s}c\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2s-2k}}{2}\Big).\end{eqnarray}$$

Hence,

$$\begin{eqnarray}A(p{\it\beta})=p^{s+1}|{\it\beta}_{0}|\mathop{\sum }_{k=0}^{s+1}c\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2s-2k+2}}{2}\Big)\end{eqnarray}$$

and

$$\begin{eqnarray}A(p^{-1}{\it\beta})=p^{s-1}|{\it\beta}_{0}|\mathop{\sum }_{k=0}^{s-1}c\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2s-2k-2}}{2}\Big).\end{eqnarray}$$

Next, we need to compute $\sum A({\it\alpha}_{1}^{-1}{\it\beta}{\it\alpha}_{2})$ where the sum is over all ${\it\alpha}_{1},{\it\alpha}_{2}$ in $C_{p}$. We consider three cases depending on whether $p\not |\,|{\it\beta}_{0}|^{2}$, or $p|\,|{\it\beta}_{0}|^{2}$ but $p^{2}\not |\,|{\it\beta}_{0}|^{2}$ or $p^{2}|\,|{\it\beta}_{0}|^{2}$.

Case 1: Let us assume that $p\not |\,|{\it\beta}_{0}|^{2}$. Applying Lemma 5.10 to ${\it\beta}_{0}$, we see that ${\it\beta}_{0}{\it\alpha}_{2}\in S^{\text{prim}}$ for all ${\it\alpha}_{2}\in C_{p}$. Again applying Lemma 5.10 to ${\it\beta}_{0}{\it\alpha}_{2}$ for a fixed ${\it\alpha}_{2}$, we see there is a unique ${\it\alpha}_{1,2}\in C_{p}$ such that ${\it\nu}_{p}({\it\alpha}_{1,2}^{-1}{\it\beta}{\it\alpha}_{2})=s$, and, for all ${\it\alpha}_{1}\neq {\it\alpha}_{1,2}$, we have ${\it\nu}_{p}({\it\alpha}_{1}^{-1}{\it\beta}{\it\alpha}_{2})=s-1$. Hence,

$$\begin{eqnarray}\displaystyle \mathop{\sum }_{{\it\alpha}_{1},{\it\alpha}_{2}\in C_{p}}A({\it\alpha}_{1}^{-1}{\it\beta}{\it\alpha}_{2}) & = & \displaystyle \mathop{\sum }_{{\it\alpha}_{2}\in C_{p}}\left(A({\it\alpha}_{1,2}^{-1}{\it\beta}{\it\alpha}_{2})+\mathop{\sum }_{\substack{ {\it\alpha}_{1}\in C_{p} \\ {\it\alpha}_{1}\neq {\it\alpha}_{1,2}}}A({\it\alpha}_{1}^{-1}{\it\beta}{\it\alpha}_{2})\right)\nonumber\\ \displaystyle & = & \displaystyle (p+1)A({\it\beta})+(p+1)pp^{s}|{\it\beta}_{0}|\mathop{\sum }_{k=0}^{s-1}c\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2s-2k}}{2}\Big).\nonumber\end{eqnarray}$$

Putting this all together, we see that $p^{2}(A(p{\it\beta})+A(p^{-1}{\it\beta}))+p\sum _{{\it\alpha}_{1},{\it\alpha}_{2}}A({\it\alpha}_{1}^{-1}{\it\beta}{\it\alpha}_{2})$ is equal to

$$\begin{eqnarray}\displaystyle & & \displaystyle \hspace{-5.0pt}p^{2}\Big(p^{s+1}|{\it\beta}_{0}|\mathop{\sum }_{k=0}^{s+1}c\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2s-2k+2}}{2}\Big)+p^{s-1}|{\it\beta}_{0}|\mathop{\sum }_{k=0}^{s-1}c\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2s-2k-2}}{2}\Big)\Big)\nonumber\\ \displaystyle & & \displaystyle \hspace{-6.00006pt}\qquad +\,p(p+1)A({\it\beta})+(p+1)p^{2}p^{s}|{\it\beta}_{0}|\mathop{\sum }_{k=0}^{s-1}c\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2s-2k}}{2}\Big)\nonumber\\ \displaystyle & & \displaystyle \hspace{-6.00006pt}\quad =p^{2}p^{s}|{\it\beta}_{0}|\Big(({\it\lambda}_{p}^{2}-2)\mathop{\sum }_{k=0}^{s-1}c\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2s-2k}}{2}\Big)+pc\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2}}{2}\Big)+pc\Big(\frac{-|{\it\beta}_{0}|^{2}}{2}\Big)\Big)\nonumber\\ \displaystyle & & \displaystyle \hspace{-6.00006pt}\qquad +\,p(p+1)A({\it\beta})+(p+1)p^{2}p^{s}|{\it\beta}_{0}|\mathop{\sum }_{k=0}^{s-1}c\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2s-2k}}{2}\Big)\nonumber\\ \displaystyle & & \displaystyle \hspace{-6.00006pt}\quad =p^{2}p^{s}|{\it\beta}_{0}|\Big(({\it\lambda}_{p}^{2}-2)\mathop{\sum }_{k=0}^{s}c\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2s-2k}}{2}\Big)\nonumber\\ \displaystyle & & \displaystyle \hspace{-6.00006pt}\qquad -\,({\it\lambda}_{p}^{2}-2)c\Big(\frac{-|{\it\beta}_{0}|^{2}}{2}\Big)+({\it\lambda}_{p}^{2}-1)c\Big(\frac{-|{\it\beta}_{0}|^{2}}{2}\Big)\nonumber\\ \displaystyle & & \displaystyle \hspace{-6.00006pt}\qquad +\,pc\Big(\frac{-|{\it\beta}_{0}|^{2}}{2}\Big)\Big)+p(p+1)A({\it\beta})+(p+1)p^{2}A({\it\beta})\nonumber\\ \displaystyle & & \displaystyle \hspace{-6.00006pt}\qquad -\,(p+1)p^{2}p^{s}|{\it\beta}_{0}|c\Big(\frac{-|{\it\beta}_{0}|^{2}}{2}\Big)\nonumber\\ \displaystyle & & \displaystyle \hspace{-6.00006pt}\quad =\big(p^{2}({\it\lambda}_{p}^{2}-2)+p(p+1)+p^{2}(p+1)\big)A({\it\beta})\nonumber\\ \displaystyle & & \displaystyle \hspace{-6.00006pt}\quad =\big(p^{2}{\it\lambda}_{p}^{2}+p^{3}+p\big)A({\it\beta}).\nonumber\end{eqnarray}$$

Here, we have used both (5.15) and (5.16).

Case 2: Let $p|\,|{\it\beta}_{0}|^{2}$ but $p^{2}\not |\,|{\it\beta}_{0}|^{2}$. Applying Lemma 5.10 to ${\it\beta}_{0}$, we see that there is a unique $\hat{{\it\alpha}}_{2}\in C_{p}$ such that $p|{\it\beta}_{0}\hat{{\it\alpha}}_{2}$. For ${\it\alpha}_{2}\neq \hat{{\it\alpha}}_{2}$, we have ${\it\beta}_{0}{\it\alpha}_{2}\in S^{\text{prim}}$. Let ${\it\beta}_{0}\hat{{\it\alpha}}_{2}=p{\it\beta}_{0}^{\prime }$. Then ${\it\beta}_{0}^{\prime }\in S^{\text{prim}}$ and $p\not |\,|{\it\beta}_{0}^{\prime }|^{2}$ (since we have assumed that $p^{2}\not |\,|{\it\beta}_{0}|^{2}$). Hence, by Lemma 5.10, we see that, for all ${\it\alpha}_{1}\in C_{p}$, we have ${\it\alpha}_{1}^{-1}{\it\beta}_{0}\hat{{\it\alpha}}_{2}=\bar{{\it\alpha}}_{1}{\it\beta}_{0}^{\prime }\in S^{\text{prim}}$. This implies that ${\it\nu}_{p}({\it\alpha}_{1}^{-1}{\it\beta}\hat{{\it\alpha}}_{2})=s$ for all ${\it\alpha}_{1}\in C_{p}$. If ${\it\alpha}_{2}\neq \hat{{\it\alpha}}_{2}$, then Lemma 5.10 implies that there is a unique ${\it\alpha}_{1,2}\in C_{p}$ such that ${\it\nu}_{p}({\it\alpha}_{1,2}^{-1}{\it\beta}{\it\alpha}_{2})=s$. For all ${\it\alpha}_{1}\neq {\it\alpha}_{1,2}$, we have ${\it\nu}_{p}({\it\alpha}_{1}^{-1}{\it\beta}{\it\alpha}_{2})=s-1$. This gives us

$$\begin{eqnarray}\displaystyle & & \displaystyle \mathop{\sum }_{{\it\alpha}_{1},{\it\alpha}_{2}\in C_{p}}A({\it\alpha}_{1}^{-1}{\it\beta}{\it\alpha}_{2})\nonumber\\ \displaystyle & & \displaystyle \quad =\mathop{\sum }_{{\it\alpha}_{1}\in C_{p}}A({\it\alpha}_{1}^{-1}{\it\beta}\hat{{\it\alpha}}_{2})+\mathop{\sum }_{\substack{ {\it\alpha}_{2}\in C_{p} \\ {\it\alpha}_{2}\neq \hat{{\it\alpha}}_{2}}}\Big(A({\it\alpha}_{1,2}^{-1}{\it\beta}{\it\alpha}_{2})+\mathop{\sum }_{\substack{ {\it\alpha}_{1}\in C_{p} \\ {\it\alpha}_{1}\neq {\it\alpha}_{1,2}}}A({\it\alpha}_{1}^{-1}{\it\beta}{\it\alpha}_{2})\Big)\nonumber\\ \displaystyle & & \displaystyle \quad =(p+1)A({\it\beta})+pA({\it\beta})+p^{2}p^{s}|{\it\beta}_{0}|\mathop{\sum }_{k=0}^{s-1}c\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2s-2k}}{2}\Big).\nonumber\end{eqnarray}$$

Putting this all together, we see that $p^{2}(A(p{\it\beta})+A(p^{-1}{\it\beta}))+p\sum _{{\it\alpha}_{1},{\it\alpha}_{2}}A({\it\alpha}_{1}^{-1}{\it\beta}{\it\alpha}_{2})$ is equal to

$$\begin{eqnarray}\displaystyle & & \displaystyle p^{2}p^{s}|{\it\beta}_{0}|\Big(({\it\lambda}_{p}^{2}-2)\mathop{\sum }_{k=0}^{s-1}c\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2s-2k}}{2}\Big)+pc\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2}}{2}\Big)+pc\Big(\frac{-|{\it\beta}_{0}|^{2}}{2}\Big)\Big)\nonumber\\ \displaystyle & & \displaystyle \qquad +\,p(p+1)A({\it\beta})+p^{2}A({\it\beta})+p^{3}p^{s}|{\it\beta}_{0}|\mathop{\sum }_{k=0}^{s-1}c\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2s-2k}}{2}\Big)\nonumber\\ \displaystyle & & \displaystyle \quad =p^{2}({\it\lambda}_{p}^{2}-2)A({\it\beta})+p^{2}p^{s}|{\it\beta}_{0}|\Big(-({\it\lambda}_{p}^{2}-2)c\Big(\frac{-|{\it\beta}_{0}|^{2}}{2}\Big)\nonumber\\ \displaystyle & & \displaystyle \qquad +\,({\it\lambda}_{p}^{2}-2)c\Big(\frac{-|{\it\beta}_{0}|^{2}}{2}\Big)+pc\Big(\frac{-|{\it\beta}_{0}|^{2}}{2}\Big)\Big)\nonumber\\ \displaystyle & & \displaystyle \qquad +\,p(p+1)A({\it\beta})+p^{2}A({\it\beta})+p^{3}A({\it\beta})-p^{3}p^{s}|{\it\beta}_{0}|c\Big(\frac{-|{\it\beta}_{0}|^{2}}{2}\Big)\nonumber\\ \displaystyle & & \displaystyle \quad =\big(p^{2}({\it\lambda}_{p}^{2}-2)+p(p+1)+p^{2}+p^{3}\big)A({\it\beta})\nonumber\\ \displaystyle & & \displaystyle \quad =\big(p^{2}{\it\lambda}_{p}^{2}+p^{3}+p\big)A({\it\beta}).\nonumber\end{eqnarray}$$

Here, we have used (5.16) and $p|\,|{\it\beta}_{0}|^{2}$.

Case 3: Let $p^{2}|\,|{\it\beta}_{0}|^{2}$. As in Case 2 above, Lemma 5.10 applied to ${\it\beta}_{0}$ implies that there is a unique $\hat{{\it\alpha}}_{2}\in C_{p}$ such that $p|{\it\beta}_{0}\hat{{\it\alpha}}_{2}$. For ${\it\alpha}_{2}\neq \hat{{\it\alpha}}_{2}$, we have ${\it\beta}_{0}{\it\alpha}_{2}\in S^{\text{prim}}$. Let ${\it\beta}_{0}\hat{{\it\alpha}}_{2}=p{\it\beta}_{0}^{\prime }$. Then ${\it\beta}_{0}^{\prime }\in S^{\text{prim}}$ and $p|\,|{\it\beta}_{0}^{\prime }|^{2}$ (since we have assumed $p^{2}|\,|{\it\beta}_{0}|^{2}$). Hence, by Lemma 5.10, there is a unique $\hat{{\it\alpha}}_{1,2}\in C_{p}$ such that ${\it\nu}_{p}(\hat{{\it\alpha}}_{1,2}^{-1}{\it\beta}\hat{{\it\alpha}}_{2})=s+1$, and for all ${\it\alpha}_{1}\neq \hat{{\it\alpha}}_{1,2}$, we have ${\it\nu}_{p}({\it\alpha}_{1}^{-1}{\it\beta}\hat{{\it\alpha}}_{2})=s$. If ${\it\alpha}_{2}\neq \hat{{\it\alpha}}_{2}$, then Lemma 5.10 implies that there is a unique ${\it\alpha}_{1,2}\in C_{p}$ such that ${\it\nu}_{p}({\it\alpha}_{1,2}^{-1}{\it\beta}{\it\alpha}_{2})=s$. For all ${\it\alpha}_{1}\neq {\it\alpha}_{1,2}$, we have ${\it\nu}_{p}({\it\alpha}_{1}^{-1}{\it\beta}{\it\alpha}_{2})=s-1$. This gives us that $\sum A({\it\alpha}_{1}^{-1}{\it\beta}{\it\alpha}_{2})$ is equal to

$$\begin{eqnarray}\displaystyle & & \displaystyle A(\hat{{\it\alpha}}_{1,2}^{-1}{\it\beta}\hat{{\it\alpha}}_{2})+\mathop{\sum }_{{\it\alpha}_{1}\neq \hat{{\it\alpha}}_{1,2}}A({\it\alpha}_{1}^{-1}{\it\beta}\hat{{\it\alpha}}_{2})\nonumber\\ \displaystyle & & \displaystyle \qquad +\,\mathop{\sum }_{{\it\alpha}_{2}\neq \hat{{\it\alpha}}_{2}}\Big(A({\it\alpha}_{1,2}{\it\beta}{\it\alpha}_{2})+\mathop{\sum }_{{\it\alpha}_{1}\neq {\it\alpha}_{1,2}}A({\it\alpha}_{1}^{-1}{\it\beta}{\it\alpha}_{2})\Big)\nonumber\\ \displaystyle & & \displaystyle \quad =p^{s}|{\it\beta}_{0}|\mathop{\sum }_{k=0}^{s+1}c\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2s-2k}}{2}\Big)+pA({\it\beta})\nonumber\\ \displaystyle & & \displaystyle \qquad +\,pA({\it\beta})+p^{2}p^{s}|{\it\beta}_{0}|\mathop{\sum }_{k=0}^{s-1}c\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2s-2k}}{2}\Big)\nonumber\\ \displaystyle & & \displaystyle \quad =A({\it\beta})+p^{s}|{\it\beta}_{0}|c\Big(\frac{-|{\it\beta}_{0}|^{2}}{2p^{2}}\Big)+pA({\it\beta})\nonumber\\ \displaystyle & & \displaystyle \qquad +\,pA({\it\beta})+p^{2}A({\it\beta})-p^{2}p^{s}|{\it\beta}_{0}|c\Big(\frac{-|{\it\beta}_{0}|^{2}}{2}\Big).\nonumber\end{eqnarray}$$

Putting this all together, we see that $p^{2}(A(p{\it\beta})+A(p^{-1}{\it\beta}))+p\sum _{{\it\alpha}_{1},{\it\alpha}_{2}}A({\it\alpha}_{1}^{-1}{\it\beta}{\it\alpha}_{2})$ is equal to

$$\begin{eqnarray}\displaystyle & & \displaystyle p^{2}({\it\lambda}_{p}^{2}-2)A({\it\beta})+p^{2}p^{s}|{\it\beta}_{0}|\Big(-({\it\lambda}_{p}^{2}-2)c\Big(\frac{-|{\it\beta}_{0}|^{2}}{2}\Big)\nonumber\\ \displaystyle & & \displaystyle \qquad +\,pc\Big(\frac{-|{\it\beta}_{0}|^{2}p^{2}}{2}\Big)+pc\Big(\frac{-|{\it\beta}_{0}|^{2}}{2}\Big)\Big)\nonumber\\ \displaystyle & & \displaystyle \qquad +\,p(1+2p+p^{2})A({\it\beta})+p^{2}p^{s}|{\it\beta}_{0}|\Big(p^{-1}c\Big(\frac{-|{\it\beta}_{0}|^{2}}{2p^{2}}\Big)-pc\Big(\frac{-|{\it\beta}_{0}|^{2}}{2}\Big)\Big)\nonumber\\ \displaystyle & & \displaystyle \quad =\big(p^{2}({\it\lambda}_{p}^{2}-2)+p(1+2p+p^{2})\big)A({\it\beta})\nonumber\\ \displaystyle & & \displaystyle \quad =\big(p^{2}{\it\lambda}_{p}^{2}+p^{3}+p\big)A({\it\beta}).\nonumber\end{eqnarray}$$

Here, we have used (5.16) and $p^{2}|\,|{\it\beta}_{0}|^{2}$. This completes the proof of the proposition.◻

6.1 The local components of the automorphic representation

Let $f\in S({\rm\Gamma}_{0}(2);-(\frac{1}{4}+r^{2}/4))$ be a Hecke eigenform with Hecke eigenvalue ${\it\lambda}_{p}$ for every odd prime $p$. Let $F=F_{f}$ be as defined in Theorem 4.4. Let ${\rm\Phi}_{F}:{\mathcal{G}}(\mathbb{A})\rightarrow \mathbb{C}$ be defined by

$$\begin{eqnarray}{\rm\Phi}_{F}({\it\gamma}g_{\infty }u_{f})=F(g_{\infty })\quad \forall ({\it\gamma},g_{\infty },u_{f})\in {\mathcal{G}}(\mathbb{Q})\times {\mathcal{G}}(\mathbb{R})\times U.\end{eqnarray}$$

See Section 5.1 for details. Let ${\it\pi}_{F}$ be the irreducible cuspidal automorphic representation of ${\mathcal{G}}(\mathbb{A})$ generated by the right translates of ${\rm\Phi}_{F}$. Note that the irreducibility follows from the strong multiplicity-one result for ${\mathcal{G}}(\mathbb{A})$ (see [Reference Badulescu2], [Reference Badulescu and Renard3]). The representation ${\it\pi}_{F}$ is cuspidal since $F$ is a cusp form. Let ${\it\pi}_{F}=\otimes _{p}^{\prime }{\it\pi}_{p}$, where ${\it\pi}_{p}$ is an irreducible admissible representation of ${\mathcal{G}}_{p}:={\mathcal{G}}(\mathbb{Q}_{p})$ for $p<\infty$, and ${\it\pi}_{\infty }$ is an irreducible admissible representation of ${\mathcal{G}}(\mathbb{R})$. Recall that $U=\prod _{p<\infty }K_{p}$, where $K_{p}$ is the maximal compact subgroup of ${\mathcal{G}}(\mathbb{Q}_{p})$ (cf. Section 2.3). Hence, for $p<\infty$, the representation ${\it\pi}_{p}$ is a spherical representation and can be realized as a subrepresentation of an unramified principal series representation, that is a representation induced from an unramified character of the Borel subgroup. The representation ${\it\pi}_{p}$ is completely determined by the action of the Hecke algebra ${\mathcal{H}}({\mathcal{G}}_{p},K_{p})$ on the spherical vector in ${\it\pi}_{p}$, which, in turn, is completely determined by the Hecke eigenvalues of $F$ obtained in the previous section. See [Reference Cartier4] for details. For $p=2$ we need to assume that $f$ is a new form for the determination of the Hecke eigenvalue of $F_{f}$ (cf. Section 5.3).

Description of ${\it\pi}_{p}$for $p$odd

If $p$ is an odd prime, then we have ${\mathcal{G}}_{p}=\text{GL}_{4}(\mathbb{Q}_{p})$ and $K_{p}=\text{GL}_{4}(\mathbb{Z}_{p})$. Given $4$ unramified characters ${\it\chi}_{1},{\it\chi}_{2},{\it\chi}_{3},{\it\chi}_{4}$ of $\mathbb{Q}_{p}^{\times }$, we obtain a character ${\it\chi}$ of the Borel subgroup $P$ of upper triangular matrices in ${\mathcal{G}}$, by

(6.1)$$\begin{eqnarray}{\it\chi}\left(\left[\begin{array}{@{}cccc@{}}a_{1} & \ast & \ast & \ast \\ & a_{2} & \ast & \ast \\ & & a_{3} & \ast \\ & & & a_{4}\end{array}\right]\right)={\it\chi}_{1}(a_{1}){\it\chi}_{2}(a_{2}){\it\chi}_{3}(a_{3}){\it\chi}_{4}(a_{4}).\end{eqnarray}$$

The modulus character ${\it\delta}_{P}$ is given by

(6.2)$$\begin{eqnarray}{\it\delta}_{P}\left(\left[\begin{array}{@{}cccc@{}}a_{1} & \ast & \ast & \ast \\ & a_{2} & \ast & \ast \\ & & a_{3} & \ast \\ & & & a_{4}\end{array}\right]\right)=|a_{1}^{3}a_{2}a_{3}^{-1}a_{4}^{-3}|_{p},\end{eqnarray}$$

where $|\ast |_{p}$ denotes the $p$-adic absolute value. The unramified principal representation corresponding to ${\it\chi}$ is given by $I({\it\chi})$, which consists of locally constant functions $f:\text{GL}_{4}(\mathbb{Q}_{p})\rightarrow \mathbb{C}$, satisfying

$$\begin{eqnarray}f(bg)={\it\delta}_{P}(b)^{1/2}{\it\chi}(b)f(g),\quad \text{for all }b\in P,g\in \text{GL}_{4}(\mathbb{Z}_{p}).\end{eqnarray}$$

The action of the Hecke algebra is as follows. If ${\it\phi}\in {\mathcal{H}}(\text{GL}_{4}(\mathbb{Q}_{p}),\text{GL}_{4}(\mathbb{Z}_{p}))$ and $f\in I({\it\chi})$, define

(6.3)$$\begin{eqnarray}\big({\it\phi}\ast f\big)(g)=\int _{\text{GL}_{4}(\mathbb{Q}_{p})}{\it\phi}(h)f(gh)dh.\end{eqnarray}$$

Recall that we have normalized the measure $dh$ on $\text{GL}_{4}(\mathbb{Q}_{p})$ so that the volume of $\text{GL}_{4}(\mathbb{Z}_{p})$ is $1$. Let $f=f_{0}$, the unique vector in $I({\it\chi})$ that is right-invariant under $\text{GL}_{4}(\mathbb{Z}_{p})$ and $f_{0}(1)=1$, and let ${\it\phi}={\it\phi}_{h}$, a characteristic function of $\text{GL}_{4}(\mathbb{Z}_{p})h\text{GL}_{4}(\mathbb{Z}_{p})=\sqcup _{i}h_{i}\text{GL}_{4}(\mathbb{Z}_{p})$. It follows from (6.3) that

(6.4)$$\begin{eqnarray}\big({\it\phi}_{h}\ast f_{0}\big)(1)=\mathop{\sum }_{i}f_{0}(h_{i})={\it\mu}_{h},\end{eqnarray}$$

where ${\it\mu}_{h}$ is determined by the representation ${\it\pi}_{p}$. The Hecke algebra ${\mathcal{H}}(\text{GL}_{4}(\mathbb{Q}_{p}),\text{GL}_{4}(\mathbb{Z}_{p}))$ is generated by $\{{\it\phi}_{1}^{\pm 1},{\it\phi}_{2},{\it\phi}_{3},{\it\phi}_{4}\}$, defined in (5.2).

Recall that Proposition 5.5 gives the double coset decompositions which can be used to determine the action of ${\it\phi}$ on $f_{0}$. Let us abbreviate ${\it\alpha}_{i}={\it\chi}_{i}(p)$ for $i=1,2,3,4$. Working in the induced model $I({\it\chi})$ of ${\it\pi}_{p}$, we see that

(6.5)$$\begin{eqnarray}\displaystyle ({\it\phi}_{1}\ast f_{0})(1) & = & \displaystyle {\it\alpha}_{1}{\it\alpha}_{2}{\it\alpha}_{3}{\it\alpha}_{4},\nonumber\\ \displaystyle ({\it\phi}_{2}\ast f_{0})(1) & = & \displaystyle p^{3}p^{-3/2}{\it\alpha}_{1}{\it\alpha}_{2}{\it\alpha}_{3}+pp^{1/2}{\it\alpha}_{1}{\it\alpha}_{3}{\it\alpha}_{4}\nonumber\\ \displaystyle & & \displaystyle +\,p^{2}p^{-1/2}{\it\alpha}_{1}{\it\alpha}_{2}{\it\alpha}_{4}+p^{3/2}{\it\alpha}_{2}{\it\alpha}_{3}{\it\alpha}_{4}\nonumber\\ \displaystyle & = & \displaystyle p^{3/2}{\it\alpha}_{1}{\it\alpha}_{2}{\it\alpha}_{3}{\it\alpha}_{4}({\it\alpha}_{1}^{-1}+{\it\alpha}_{2}^{-1}+{\it\alpha}_{3}^{-1}+{\it\alpha}_{4}^{-1}),\nonumber\\ \displaystyle ({\it\phi}_{4}\ast f_{0})(1) & = & \displaystyle p^{3}p^{-3/2}{\it\alpha}_{1}+pp^{1/2}{\it\alpha}_{3}+p^{2}p^{-1/2}{\it\alpha}_{2}+p^{3/2}{\it\alpha}_{4}\nonumber\\ \displaystyle & = & \displaystyle p^{3/2}({\it\alpha}_{1}+{\it\alpha}_{2}+{\it\alpha}_{3}+{\it\alpha}_{4}),\nonumber\\ \displaystyle ({\it\phi}_{3}\ast f_{0})(1) & = & \displaystyle p^{4}p^{-2}{\it\alpha}_{1}{\it\alpha}_{2}+p^{2}{\it\alpha}_{2}{\it\alpha}_{3}+pp{\it\alpha}_{2}{\it\alpha}_{4}\nonumber\\ \displaystyle & & \displaystyle +\,p^{3}p^{-1}{\it\alpha}_{1}{\it\alpha}_{3}+p^{2}{\it\alpha}_{1}{\it\alpha}_{4}+p^{2}{\it\alpha}_{3}{\it\alpha}_{4}\nonumber\\ \displaystyle & = & \displaystyle p^{2}({\it\alpha}_{1}{\it\alpha}_{2}+{\it\alpha}_{2}{\it\alpha}_{3}+{\it\alpha}_{2}{\it\alpha}_{4}+{\it\alpha}_{1}{\it\alpha}_{3}+{\it\alpha}_{1}{\it\alpha}_{4}+{\it\alpha}_{3}{\it\alpha}_{4}).\end{eqnarray}$$

Proposition 6.2. Let $f\in S({\rm\Gamma}_{0}(2);-(\frac{1}{4}+r^{2}/4))$ be a Hecke eigenform with Hecke eigenvalue ${\it\lambda}_{p}$ for every odd prime $p$. Let $F=F_{f}$ be as defined in Theorem 4.4. Let ${\it\pi}_{F}=\otimes _{p}^{\prime }{\it\pi}_{p}$ be the corresponding irreducible cuspidal automorphic representation of ${\mathcal{G}}(\mathbb{A})$. For an odd prime $p$, the representation ${\it\pi}_{p}$ is the unique spherical constituent of the unramified principal series representation $I({\it\chi})$, where, up to the action of the Weyl group of $\text{GL}_{4}$, the character ${\it\chi}$ is given by

(6.6)$$\begin{eqnarray}\displaystyle {\it\chi}_{1}(p)=p^{1/2}\frac{{\it\lambda}_{p}+\sqrt{{\it\lambda}_{p}^{2}-4}}{2},\qquad {\it\chi}_{2}(p)=p^{1/2}\frac{{\it\lambda}_{p}-\sqrt{{\it\lambda}_{p}^{2}-4}}{2}, & & \displaystyle \nonumber\\ \displaystyle {\it\chi}_{3}(p)=p^{-1/2}\frac{{\it\lambda}_{p}+\sqrt{{\it\lambda}_{p}^{2}-4}}{2},\qquad {\it\chi}_{4}(p)=p^{-1/2}\frac{{\it\lambda}_{p}-\sqrt{{\it\lambda}_{p}^{2}-4}}{2}. & & \displaystyle\end{eqnarray}$$

Let us remark here that we can use Lemma 6.1 to directly solve for ${\it\alpha}_{i}$ from (6.5). It is a tedious computation but results in the same answer as in the statement of the above proposition.

Description of ${\it\pi}_{2}$

Recall that $B_{2}=B\otimes _{\mathbb{Q}}\mathbb{Q}_{2}$, where $B$ is a definite quaternion algebra over $\mathbb{Q}$ with discriminant $2$, and ${\mathcal{O}}_{2}$ is the completion of the Hurwitz order ${\mathcal{O}}$ at $2$. In this case, ${\mathcal{G}}_{2}=\text{GL}_{2}(B_{2})$ and $K_{2}=\text{GL}_{2}({\mathcal{O}}_{2})$. Given two unramified characters ${\it\chi}_{1},{\it\chi}_{2}$ of $B_{2}^{\times }$, we obtain a character ${\it\chi}$ of the Borel subgroup of upper triangular matrices on ${\mathcal{G}}$ by

$$\begin{eqnarray}{\it\chi}\left(\left[\begin{array}{@{}cc@{}}{\it\alpha} & \ast \\ 0 & {\it\beta}\end{array}\right]\right)={\it\chi}_{1}({\it\alpha}){\it\chi}_{2}({\it\beta}).\end{eqnarray}$$

The modulus character is given by

$$\begin{eqnarray}{\it\delta}\left(\left[\begin{array}{@{}cc@{}}{\it\alpha} & \ast \\ 0 & {\it\beta}\end{array}\right]\right)=|{\it\alpha}/{\it\beta}|_{2}^{2}.\end{eqnarray}$$

Here, $|\,|_{2}$ is the $2$-adic absolute value of the reduced norm of $B_{2}$. The unramified principal series representation corresponding to ${\it\chi}$ is given by $I({\it\chi})$, which consists of locally constant functions $f:{\mathcal{G}}_{2}\rightarrow \mathbb{C}$, satisfying

$$\begin{eqnarray}f(bg)={\it\delta}(b)^{1/2}{\it\chi}(b)f(g),\quad \text{for all }b\in \text{Borel subgroup, }g\in {\mathcal{G}}_{2}.\end{eqnarray}$$

The action of the Hecke algebra is as follows. If ${\it\phi}\in {\mathcal{H}}({\mathcal{G}}_{2},K_{2})$ and $f\in I({\it\chi})$, define

(6.7)$$\begin{eqnarray}\big({\it\phi}\ast f\big)(g)=\int _{{\mathcal{G}}_{2}}{\it\phi}(h)f(gh)dh.\end{eqnarray}$$

Recall that we have normalized the measure $dh$ on ${\mathcal{G}}_{2}$ so that the volume of $K_{2}$ is $1$. Let $f=f_{0}$, the unique vector in $I({\it\chi})$ that is right-invariant under $K_{2}$ and $f_{0}(1)=1$, and let ${\it\phi}={\it\phi}_{h}$, a characteristic function of $K_{2}hK_{2}=\sqcup _{i}h_{i}K_{2}$. It follows from (6.3) that

(6.8)$$\begin{eqnarray}\big({\it\phi}_{h}\ast f_{0}\big)(1)=\mathop{\sum }_{i}f_{0}(h_{i})={\it\mu}_{h},\end{eqnarray}$$

where ${\it\mu}_{h}$ is determined by the representation ${\it\pi}_{2}$. The Hecke algebra ${\mathcal{H}}({\mathcal{G}}_{2},K_{2})$ is generated by $\{{\it\varphi}_{1}^{\pm 1},{\it\varphi}_{2}\}$, where ${\it\varphi}_{1},~{\it\varphi}_{2}$ denote the characteristic functions for

$$\begin{eqnarray}K_{2}\left[\begin{array}{@{}cc@{}}{\it\varpi}_{2} & 0\\ 0 & {\it\varpi}_{2}\end{array}\right]K_{2},\qquad K_{2}\left[\begin{array}{@{}cc@{}}{\it\varpi}_{2} & 0\\ 0 & 1\end{array}\right]K_{2}.\end{eqnarray}$$

Here, ${\it\varpi}_{2}$ is a uniformizer for $B_{2}$.

Recall that Proposition 5.5 gives the double coset decompositions which can be used to determine the action of ${\it\phi}$ on $f_{0}$. Let us abbreviate ${\it\alpha}_{i}={\it\chi}_{i}({\it\varpi}_{2})$ for $i=1,2$. Working in the induced model $I({\it\chi})$ of ${\it\pi}_{2}$, we see that

(6.9)$$\begin{eqnarray}\displaystyle ({\it\varphi}_{1}\ast f_{0})(1) & = & \displaystyle {\it\alpha}_{1}{\it\alpha}_{2},\nonumber\\ \displaystyle ({\it\varphi}_{2}\ast f_{0})(1) & = & \displaystyle 2({\it\alpha}_{1}+{\it\alpha}_{2}).\end{eqnarray}$$

Proposition 6.4. Let $f\in S({\rm\Gamma}_{0}(2);-(\frac{1}{4}+r^{2}/4))$ be a new form with Hecke eigenvalue ${\it\lambda}_{p}$ for $p=2$ and Atkin–Lehner eigenvalue ${\it\epsilon}$, for which ${\it\lambda}_{2}=-{\it\epsilon}$ holds (cf. (5.5)). Let $F=F_{f}$ be as defined in Theorem 4.4. Let ${\it\pi}_{F}=\otimes _{p}^{\prime }{\it\pi}_{p}$ be the corresponding irreducible cuspidal automorphic representation of ${\mathcal{G}}(\mathbb{A})$. The representation ${\it\pi}_{2}$ is the unique spherical constituent of the unramified principal series representation $I({\it\chi})$, where, up to the action of the Weyl group, the character ${\it\chi}$ is given by

(6.10)$$\begin{eqnarray}{\it\chi}_{1}({\it\varpi}_{2})=-\sqrt{2}{\it\epsilon},\qquad {\it\chi}_{2}({\it\varpi}_{2})=-1/\sqrt{2}{\it\epsilon}.\end{eqnarray}$$

Description of ${\it\pi}_{\infty }$

We now determine ${\it\pi}_{\infty }$. We note that $F=F_{f}\in {\mathcal{M}}(\text{GL}_{2}({\mathcal{O}});r)$ implies that the archimedean component ${\it\pi}_{\infty }$ of ${\it\pi}_{F}$ is spherical. Namely, ${\it\pi}_{\infty }$ has a $K_{\infty }$-invariant vector, where we put $K_{\infty }:=K$ with $K$ as in (2.1). In fact, up to constant multiples, such a vector is unique for ${\it\pi}_{\infty }$, as we soon see.

We now introduce $M_{\infty }:=\left\{\!\left.\left(\!\begin{smallmatrix}u_{1} & 0\phantom{}\\ 0 & u_{2}\end{smallmatrix}\!\right)\right|u_{1},u_{2}\in \mathbb{H}^{1}\right\}$ (see Section 2.1 for $\mathbb{H}^{1}$). Let $P_{\infty }$ be the standard proper parabolic subgroup ${\mathcal{G}}_{\infty }=\text{GL}_{2}(\mathbb{H})$ given by

$$\begin{eqnarray}\left\{\left(\begin{array}{@{}cc@{}}a & \ast \\ 0 & d\end{array}\right)\in {\mathcal{G}}_{\infty }\right\}.\end{eqnarray}$$

We have $P_{\infty }:=Z^{+}NAM_{\infty }$, where $Z^{+},~N$ and $A$ are as in (2.1). The group $Z^{+}AM_{\infty }$ is nothing but the Levi subgroup of $P_{\infty }$. We now note that the Langlands classification of real reductive groups (cf. [Reference Langlands18]) implies that ${\it\pi}_{\infty }$ has to be embedded into some principal series representation $I_{P_{\infty }}$ of ${\mathcal{G}}_{\infty }$ induced from a quasicharacter of $P_{\infty }$. Since ${\it\pi}_{\infty }$ is spherical, $I_{P_{\infty }}$ is also spherical. As ${\it\pi}_{\infty }$ has the trivial central character, so does $I_{P_{\infty }}$. Combining these with the Frobenius reciprocity for compact Lie groups (cf. [Reference Knapp15, Theorem 9.9]), one can verify that the quasicharacter of $P_{\infty }$ inducing $I_{P_{\infty }}$ has to be trivial on $Z^{+}M_{\infty }$, and that $I_{P_{\infty }}$ and ${\it\pi}_{\infty }$ have a unique $K_{\infty }$-invariant vector, up to constant multiples. For $s\in \mathbb{C}$ we introduce the quasicharacter ${\it\chi}_{s}$ of $P_{\infty }$ defined by

$$\begin{eqnarray}{\it\chi}_{s}\left(\left(\begin{array}{@{}cc@{}}a & \ast \\ 0 & d\end{array}\right)\right)={\it\nu}(ad^{-1})^{s},\end{eqnarray}$$

where we recall that ${\it\nu}$ denotes the reduced norm of $\mathbb{H}$ (cf. Section 2.1). We note that the quasicharacters of $P_{\infty }$ trivial on $Z^{+}M_{\infty }$ should be of this form. We furthermore introduce the modulus character ${\it\delta}_{\infty }$ of $P_{\infty }$. The principal series representation $I_{P_{\infty }}$ is then expressed as

$$\begin{eqnarray}I_{P_{\infty }}=\text{Ind}_{P_{\infty }}^{{\mathcal{G}}_{\infty }}({\it\delta}_{\infty }{\it\chi}_{s}).\end{eqnarray}$$

Proposition 6.5. We have an isomorphism

$$\begin{eqnarray}{\it\pi}_{\infty }\simeq \text{Ind}_{P_{\infty }}^{{\mathcal{G}}_{\infty }}({\it\delta}_{\infty }{\it\chi}_{\pm \sqrt{-1}r})\end{eqnarray}$$

as $(\mathfrak{g},K_{\infty })$-modules, where we recall that $\mathfrak{g}$ denotes the Lie algebra of ${\mathcal{G}}_{\infty }$ (cf. Section2.2).

Proof. Let $v$ be a unique $K_{\infty }$-invariant vector in the representation space of $\text{Ind}_{P_{\infty }}^{{\mathcal{G}}_{\infty }}({\it\delta}_{\infty }{\it\chi}_{s})$, which ${\it\pi}_{\infty }$ can be embedded into. Then, $v$ can also be regarded as a vector of ${\it\pi}_{\infty }$. We remark that ${\it\pi}_{\infty }$ can be viewed as a representation of $SL_{2}(\mathbb{H})\simeq \text{GL}_{2}(\mathbb{H})/Z^{+}$ (cf. Section 2.2) since it has the trivial central character. Consider the infinitesimal action of the Casimir operator ${\rm\Omega}$ (cf. (2.3)) on $v$. We then have

$$\begin{eqnarray}{\rm\Omega}\cdot v=\displaystyle \frac{1}{2}\left(\frac{s^{2}}{4}-1\right)v=\displaystyle \frac{1}{2}\left(-\frac{r^{2}}{4}-1\right)v,\end{eqnarray}$$

which leads to $s=\pm \sqrt{-1}r$. Now recall that we can assume $r\in \mathbb{R}$ (cf. Proposition 2.4). We thus know that the quasi-character ${\it\chi}_{s}$ is parametrized by a purely imaginary number $\pm \sqrt{-1}r$. By Harish-Chandra [Reference Chandra10, §41, Theorem 1], the spherical principal series representation $\text{Ind}_{P_{\infty }}^{{\mathcal{G}}_{\infty }}({\it\delta}_{\infty }{\it\chi}_{\pm \sqrt{-1}r})$ is an irreducible unitary representation. For this see also [Reference Collingwood6, Remark (2.1.13)] and note the accidental isomorphism $\text{Spin}(5,1)\simeq \text{SL}_{2}(\mathbb{H})$ as real Lie groups. Consequently we have the isomorphism in the assertion.◻

6.2 CAP representations

Let us first give the definition of CAP representations.

See [Reference Gan8] and [Reference Pitale22] for details on CAP representations defined for two groups instead of just one. Take $G_{1}={\mathcal{G}}=\text{GL}_{2}(B)$ and $G_{2}=\text{GL}_{4}$. Here, $B$ is a definite quaternion algebra with discriminant $2$. Since these groups are inner forms of each other, we have $G_{1,p}\simeq G_{2,p}$ for all odd primes $p$. Let $P_{2}$ be the standard parabolic of $\text{GL}_{4}$ with Levi subgroup $M_{2}=\text{GL}_{2}\times \text{GL}_{2}$. Let $f\in S({\rm\Gamma}_{0}(2);-(\frac{1}{4}+r^{2}/4))$ be a Hecke eigenform with Hecke eigenvalue ${\it\lambda}_{p}$ for every odd prime $p$ and Atkin–Lehner eigenvalue ${\it\epsilon}$. Let ${\it\sigma}=\otimes _{p}^{\prime }{\it\sigma}_{p}$ be the irreducible cuspidal automorphic representation of $\text{GL}_{2}$ corresponding to $f$. For an odd prime $p$, the representation ${\it\sigma}_{p}$ is the unramified principal series representation $I({\it\eta})$, where ${\it\eta}$ is given by

$$\begin{eqnarray}{\it\eta}\left(\left[\begin{array}{@{}cc@{}}a & b\\ & d\end{array}\right]\right)={\it\eta}_{0}(a){\it\eta}_{0}^{-1}(d).\end{eqnarray}$$

Here, ${\it\eta}_{0}$ is an unramified character of $\mathbb{Q}_{p}^{\times }$ such that ${\it\eta}_{0}(p)+{\it\eta}_{0}^{-1}(p)={\it\lambda}_{p}$. For $p=2$ assume that $f$ is a new form. Then, the representation ${\it\sigma}_{2}$ is the twist of the Steinberg representation of $\text{GL}_{2}(\mathbb{Q}_{2})$ by an unramified character ${\it\eta}^{\prime }$, with ${\it\eta}^{\prime }(2)=-{\it\epsilon}$. The representation ${\it\sigma}$ gives a representation $|\text{det}|_{\mathbb{A}}^{-1/2}{\it\sigma}\times |\text{det}|_{\mathbb{A}}^{1/2}{\it\sigma}$ of $M_{2}$, where $|~|_{\mathbb{A}}$ denotes the idele norm of $\mathbb{A}^{\times }$. We have the following theorem.

We can furthermore show that our cuspidal representations ${\it\pi}_{F}$ provide counterexamples to the Ramanujan conjecture.

Proof. The temperedness of ${\it\pi}_{\infty }$ is due to Proposition 6.5 and [Reference Collingwood6, Remark 2.1.13]. For an odd prime $p$, the unramified characters ${\it\chi}_{i}$ with $1\leqslant i\leqslant 4$ are not unitary (cf. (6.6)). This means that ${\it\pi}_{p}$ is nontempered (cf. [Reference Sarnak27, (9)]).

Let $p=2$, and suppose that $f$ is a new form. We recall that $f_{0}$ denotes the spherical vector in ${\it\pi}_{2}$, and introduce its dual vector $f_{0}^{\prime }$ in the contragredient representation of ${\it\pi}_{2}$. With the invariant measure $dg$ of ${\mathcal{G}}_{p}/Z_{p}$ normalized so that $\int _{K_{2}/Z_{p}}dg=1$, for any ${\it\delta}>0$, we consider the following integral of the matrix coefficient:

$$\begin{eqnarray}\displaystyle \int _{{\mathcal{G}}_{2}/Z_{2}}|\langle {\it\pi}_{2}(g)f_{0},f_{0}^{\prime }\rangle |^{2+{\it\delta}}dg\end{eqnarray}$$

over ${\mathcal{G}}_{2}$ modulo center $Z_{2}$, where $\langle \ast ,\ast \rangle$ denotes the canonical pairing of ${\it\pi}_{2}$ and its contragredient. If ${\it\pi}_{2}$ is tempered, this integral should be convergent. Now we note that the set $\left(\sqcup _{n\geqslant 0}K_{2}\left(\!\begin{smallmatrix}{\it\varpi}_{2}^{n} & 0\phantom{}\\ 0 & 1\end{smallmatrix}\!\right)K_{2}\right)/Z_{2}$ can be regarded as a subdomain of ${\mathcal{G}}_{2}/Z_{2}$ and that there is a decomposition

$$\begin{eqnarray}K_{2}\left(\begin{array}{@{}cc@{}}{\it\varpi}_{2}^{n} & 0\\ 0 & 1\end{array}\right)K_{2}=\underset{x\in {\mathcal{O}}_{2}/{\it\varpi}_{2}^{n}{\mathcal{O}}_{2}}{\sqcup }\left(\begin{array}{@{}cc@{}}\mathop{{\it\varpi}}_{2}^{n} & 0\\ 0 & 1\end{array}\right)\left(\begin{array}{@{}cc@{}}1 & \mathop{{\it\varpi}}_{2}^{-n}x\\ 0 & 1\end{array}\right)K_{p}\sqcup \left(\begin{array}{@{}cc@{}}1 & 0\\ 0 & {\it\varpi}_{2}^{n}\end{array}\right)K_{p}.\end{eqnarray}$$

It is verified that the Hecke operator defined by $K_{2}\left(\!\begin{smallmatrix}{\it\varpi}_{2}^{n} & 0\phantom{}\\ 0 & 1\end{smallmatrix}\!\right)K_{2}$ acts on $f_{0}$ as follows:

$$\begin{eqnarray}\left(K_{2}\left(\begin{array}{@{}cc@{}}{\it\varpi}_{2}^{n} & 0\\ 0 & 1\end{array}\right)K_{2}\right)\cdot f_{0}=(-{\it\epsilon})^{n}(2^{3n/2}+2^{n/2})f_{0}.\end{eqnarray}$$

We thereby have a divergent integral

$$\begin{eqnarray}\displaystyle & & \displaystyle \displaystyle \int _{\left(\mathop{\sqcup }_{n\geqslant 0}K_{2}\left(\begin{array}{@{}cc@{}}{\it\varpi}_{2}^{n} & 0\\ 0 & 1\end{array}\right)K_{2}\right)/Z_{p}}|\langle {\it\pi}_{2}(g)f_{0},f_{0}^{\prime }\rangle |^{2+{\it\delta}}dg\nonumber\\ \displaystyle & & \displaystyle \quad =\left(\displaystyle \mathop{\sum }_{n\geqslant 0}(2^{3n/2}+2^{n/2})^{2+{\it\delta}}\right)|\langle f_{0},f_{0}^{\prime }\rangle |^{2+{\it\delta}}=\infty ,\nonumber\end{eqnarray}$$

which leads to a contradiction. We therefore see that ${\it\pi}_{2}$ is nontempered. As a result we are done.◻