3.1 Galois representations

It has been explained by Deligne [Reference Deligne10, 2.1-2.9] how to pass to the automorphic side, and the reader is invited there for more details. This section aims to give precise information on the ramifications of automorphic representations determined by the Frobenius action on $\mathfrak {R}$ . The data on the local monodromies are carried over to the automorphic side, described by local Langlands correspondence.

We continue to use notation in the introduction. Let $v\in S$ and $x\in \overline {S}$ that lies over v.

We fix an algebraic closure $\overline {F}$ of F. Then $\overline {\eta }:=\operatorname {\mathrm {Spec}}(\overline {F})$ is a geometric point lying over the generic point of X. Let ${\overline {X}}_{(x)}$ be the Henselization of $\overline {X}$ in x and ${\overline {X}}_{(x)}^{\ast }={\overline {X}}_{(x)}-\{x\}$ . If we choose an embedding of $\overline {\mathbb {F}}_q({\overline {X}}_{(x)}^{\ast })$ in $\overline {F}$ , then the étale fundamental group $\pi _1({\overline {X}}_{(x)}^{*}, \overline {\eta })$ is canonically isomorphic to the inertial group $I_x=\mathrm {Gal}(\overline {F}/\overline {\mathbb {F}}_q({\overline {X}}_{(x)}^{\ast }) )$ . An $\ell $ -adic local system over $\overline {X}-\overline {S}$ (resp. ${\overline {X}}_{(x)}^{\ast }$ ) is equivalent to an $\ell $ -adic representation of $\pi _1(\overline {X}-\overline {S}, \overline {\eta })$ (resp. $I_x$ ). Let $I^{t}_x=\pi _1({\overline {X}}_{(x)}^{\ast }, \overline {\eta })^{t}$ be the tame fundamental group of ${\overline {X}}_{(x)}^{\ast }$ (i.e., the maximal prime-to-p quotient of $\pi _1({\overline {X}}_{(x)}^{\ast }, \overline {\eta })$ ). A tame $\ell $ -adic local system of ${\overline {X}}_{(x)}^{\ast }$ is equivalent to an $\ell $ -adic representation of $I_x^t$ .

For an algebraic closed field k, let

$$\begin{align*}\widehat{\mathbb{Z}}^{p'}(1)(k):= \lim \mu_n(k),\end{align*}$$

the projective limit where the transition map is the norm map. We denote $\widehat {\mathbb {Z}}^{p'}(1)$ for $\widehat {\mathbb {Z}}^{p'}(1)(\overline {\mathbb {F}}_q)$ . Let $\kappa _x\cong \overline {\mathbb {F}}_q$ be the residue field of the point x. We have a canonical isomorphism

$$\begin{align*}I_x^{t}\cong \widehat{\mathbb{Z}}^{p'}(1)(\kappa_x). \end{align*}$$

Choose an embedding $\kappa _v\hookrightarrow \overline {\mathbb {F}}_q$ . We deduce from

$$\begin{align*}\kappa_v\otimes_{{\mathbb{F}}_q} \overline{\mathbb{F}}_q \cong \prod_{x\mapsto v} \overline{\mathbb{F}}_q, \end{align*}$$

isomorphisms $\kappa _x\cong \overline {\mathbb {F}}_q$ . This gives us an isomorphism

$$\begin{align*}I_x^t\cong \widehat{\mathbb{Z}}^{p'}(1). \end{align*}$$

The morphism $\mathrm {Frob}: {\overline {X}}_x^{*}\longrightarrow {\overline {X}}_{\mathrm {Frob}(x)}^{*}$ induces an isomorphism between tame fundamental groups. It is the multiplication by q map on $\widehat {\mathbb {Z}}^{p'}(1)$ via the above isomorphisms.

To make the set $E_2(\mathfrak {R}) {\mathrm {Frob}}^\ast $ -stable, the $\ell $ -adic local systems $({\mathfrak {R}}_x)_{x\mapsto v}$ have to satisfy the compatibility condition

(3.1.1) $$ \begin{align}{\mathrm{Frob}}^{*}({\mathfrak{R}}_{\mathrm{Frob}(x)})\cong {\mathfrak{R}}_x. \end{align} $$

Let $I_v$ and $D_v$ be, respectively, the inertial subgroup and decomposition group of F at v. The condition (3.1.1) implies that $({\mathfrak {R}}_x)_{x\mapsto v}$ come from a representation $\rho _v$ of $D_v$ . By Grothendick’s local monodromy theorem, $\rho _v|_{I_v}$ is quasi-unipotent in the sense that it becomes unipotent on an open subgroup. We will use ${\mathfrak {R}}_v$ to denote $\rho _v|_{I_v}$ .

Let $W_F$ be the Weil group of F. Then $\pi _1(\overline {X}-\overline {S}, \overline {\eta })$ is a quotient of the degree $0$ part of $W_F$ . Let $\mathcal {G}_2(F)$ be the set of isomorphism classes of $\ell $ -adic representation of $W_F$ . We call two $\ell $ -adic representations $\sigma _1$ and $\sigma _2$ in $\mathcal {G}_2(F)$ inertially equivalent: $\sigma _1\sim \sigma _2$ if there is a character $\lambda : W_F\xrightarrow {\deg } \mathbb {Z}\longrightarrow {\overline {\mathbb {Q}}}_{\ell }^{\times }$ such that $\sigma _1\cong \sigma _2\otimes \lambda $ .

Proposition 3.1.1. The set

$$\begin{align*}E_2(\mathfrak{R})^{{\mathrm{Frob}}^{*k}}\end{align*}$$

is in bijection with the subset of inertially equivalent classes in $\mathcal {G}_2(F\otimes {\mathbb {F}}_{q^k})/\sim $ consisting of $\sigma $ such that

(3.1.2) $$ \begin{align}\sigma\otimes \lambda\cong \sigma \implies \lambda=1, \end{align} $$

$\sigma \vert _{I_v}$ is trivial for $v\notin S$ and

$$\begin{align*}\sigma\vert _{I_v}\cong {\mathfrak{R}}_v \end{align*}$$

for $v\in S$ .

Let $W_v$ be the local Weil group at v. We choose an isomorphism $\iota : {\overline {\mathbb {Q}}}_{\ell }\xrightarrow {\sim }\mathbb {C}$ . Recall that the local Langlands correspondence is a canonical bijection between the set of smooth irreducible $\mathbb {C}$ -representations of $G_v$ and the set of rank $2$ , Frobenius semisimple $\ell $ -adic (continuous) representations of $W_v$ .

For any $v\in S$ , let ${\mathrm {Irr}}^{\mathfrak {R}}(G_v)$ be the set of irreducible representations of $G_v$ whose associated $\ell $ -adic representation of the local Weil group $W_v$ under local Langlands correspondence extends ${\mathfrak {R}}_v$ . For a place $v\notin S$ , we define ${\mathrm {Irr}}^{\mathfrak {R}}(G_v)$ to be the set of unramified representations of $G_v$ (i.e., those representations whose associated $\ell $ -adic representation of $W_v$ under local Langlands correspondence is trivial when restricting to $I_v$ ).

We have the following theorem that characterizes the set ${\mathrm {Irr}}^{\mathfrak {R}}(G_v)$ purely by their representation theoretic structures.

Let $\pi =\otimes ' \pi _v$ be a cuspidal automorphic representation of $G(\mathbb {A})$ . We will say that $\pi _v$ has the correct ramification type (for our counting problem) if $\pi _v\in {\mathrm {Irr}}^{\mathfrak {R}}(G_v)$ .

3.2 Automorphic representations

Let $C_{cusp}(G(\mathbb {A}))$ be the space of cuspidal automorphic forms. Recall that a cuspidal automorphic form is a complex-valued function $\varphi $ over $G(F)\backslash G(\mathbb {A})$ that generates a finite-dimensional vector space under $G(\mathcal {O})$ -right translation and $Z_{G}(\mathbb {A})$ translation, such that the following cuspidality condition is satisfied:

$$\begin{align*}\int_{N(F)\backslash N(\mathbb{A})}\varphi(nx)\mathrm{d} n, \quad \forall x\in G(\mathbb{A}). \end{align*}$$

Note that the above integration is a finite sum because of $G(\mathcal {O})$ -finiteness. The right translation by $G(\mathbb {A})$ makes $C_{cusp}(G(\mathbb {A}))$ into a $G(\mathbb {A})$ -representation. It is known to be semisimple, and the multiplicity one theorem of Jacquet and Langlands and Piatetski-Shapiro implies that $C_{cusp}(G(\mathbb {A}))$ is multiplicity free. Its irreducible summands are called cuspidal automorphic representations. A cuspidal automorphic representation $\pi $ can be decomposed as a restricted tensor product $\pi =\otimes ' \pi _v$ for representations $\pi _v$ of $G_v$ which are called local components of $\pi $ .

Let $\mathcal {A}_2(F)$ be the set of isomorphic classes of cuspidal automorphic representations of $G(\mathbb {A})$ . We call two cuspidal automorphic representations $\pi _1$ and $\pi _2$ are inertially equivalent $\pi _1\sim \pi _2$ if there are is a character $\lambda : G(\mathbb {A})\xrightarrow {\deg \circ \det }\mathbb {Z}\longrightarrow \mathbb {C}^\times $ such that $\pi _1\cong \pi _2\otimes \lambda $ .

Theorem 3.2.1. The set $E_2(\mathfrak {R})^{{\mathrm {Frob}}^{*}}$ is in bijection with the subset of $\mathcal {A}_2(F)/\sim $ consisting of inertial equivalent classes of cuspidal automorphic representations $\pi $ such that for any character $\lambda : G(\mathbb {A})\xrightarrow {\deg \circ \det }\mathbb {Z}\longrightarrow \mathbb {C}^\times $ ,

$$\begin{align*}\pi\otimes \lambda \cong \pi \implies \lambda=1 ,\end{align*}$$

and $\pi _v\in {\mathrm {Irr}}^{\mathfrak {R}}(G_v)$ for all $v\in S$ .

4.1 Explicit spectral decomposition of $J^{1}_{spec}(f)$

Let M be either T or G. Let $X_M$ be the group of characters of $M(\mathbb {A})$ to $\mathbb {C}^{\times }$ that are trivial on $M(\mathbb {A})^{0}$ . Let $X_{M}^{G}\subseteq X_M$ be the subgroup consisting of those characters that are trivial on $Z_{G}(\mathbb {A})$ ; that is,

$$\begin{align*}X_M^G=\mathrm{Hom}(M(\mathbb{A})^{0}\backslash M(\mathbb{A})/Z_{G}(\mathbb{A}), \mathbb{C}^{\times}). \end{align*}$$

We have

$$\begin{align*}X_G^G= \{1, \epsilon\}, \end{align*}$$

where $\epsilon (x)=(-1)^{\deg (\det x) }$ is the sign character of $G(\mathbb {A})$ . We identify $X_G^G$ with $\{\pm 1\}\subseteq \mathbb {C}^{\times }$ . We also have an identification

$$\begin{align*}X_T^G\cong \mathbb{C}^{\times}, \end{align*}$$

where for any $\lambda \in \mathbb {C}^{\times }$ , we associate a character

$$\begin{align*}\lambda\left(\begin{pmatrix} a&0\\0& b \end{pmatrix} \right) = \lambda^{\deg(a)-\deg(b)}. \end{align*}$$

We will use this isomorphism in our calculations. Let $\mathrm {Im} X_T^G$ be the subgroup of $X_T^G$ consisting of unitary characters. Therefore, it is formed by elements $\lambda \in \mathbb {C}^{\times }$ of absolute value $1$ . We endow a Haar measure on $X_G^G$ and $\mathrm {Im} X_T^G$ so that the total volume is $1$ .

Following Jacquet-Langlands, we have the spectral decomposition

$$\begin{align*}L^2(G(F)\backslash G(\mathbb{A}))\cong L^2_{cusp}\oplus L^{2}_{res} \oplus L^{2}_{cont}, \end{align*}$$

where $L^2_{cusp}\oplus L^{2}_{res}$ is the largest semisimple subspace, $L^{2}_{cont}$ is its orthogonal complement, and $L^2_{cusp}$ is the completion of the space of cuspidal automorphic forms. The residual spectrum is decomposed as

$$\begin{align*}L^{2}_{res}\cong \widehat{\bigoplus_{\chi}} \chi, \end{align*}$$

where $\chi $ are compositions of Hecke characters with the determinant morphism. The sum here is the Hilbert direct sum. For continuous spectrum $L^{2}_{cont}$ , we have a decomposition

$$\begin{align*}L^{2}_{cont}=\widehat{\bigoplus_{\psi}} L^{2}_{[B, \psi]}, \end{align*}$$

where the sum is taken over the set of inertial equivalent classes of pairs $(B, \psi )$ with $\psi $ being a Hecke character of $T(\mathbb {A})\cong (\mathbb {A}^{\times })^2$ . The explicit construction of $L^{2}_{[B, \psi ]}$ is given by the theory of Eisenstein series.

We fix an idèle $a\in \mathbb {A}^\times $ of degree $1$ , viewed as a scalar matrix. Let $f\in C_c^\infty (G(\mathbb {A}))$ . It acts via the regular representation on $L^2(G(F)\backslash G(\mathbb {A})/a^{\mathbb {Z}})$ , equivalently by convolution from right by $\breve {f}:=(x\mapsto f(x^{-1}))$ , which is an integral operator. We denote this action by the function $R(f)$ . Therefore, its trace, if it exists, will be the integration of the kernel function on the diagonal. However, the trace of f on the whole space $L^2(G(F)\backslash G(\mathbb {A})/a^{\mathbb {Z}})$ does not exist in general. Arthur defines a truncated kernel function so that its integration on the diagonal will contain the information $\mathrm {Tr}(f|L^2_{cusp})$ and expresses this integration in two ways: a geometric expansion and a spectral expansion so that we have an identity. The spectral expansion contains a piece that gives the most interesting part $\mathrm {Tr}(f|L^2_{cusp})$ , and we usually hope to obtain information from the geometric expansion and an understanding of the error terms in the spectral expansion.

Over a function field, we have a decomposition

$$\begin{align*}G(\mathbb{A})=\coprod G(\mathbb{A})^{e}, \end{align*}$$

and $G(F)\backslash G(\mathbb {A})^{e}$ has finite volume. The two different ways to express Arthur’s truncated integral over the diagonal in $G(F)\backslash G(\mathbb {A})^{e}\times G(F)\backslash G(\mathbb {A})^{e}$ give an identity

$$\begin{align*}J^{e}_{geom}(f)=J^{e}_{spec}(f). \end{align*}$$

It is slightly simpler to consider an odd integer e or simply that $e=1$ . The following result is a special case of the formula obtained by L. Lafforgue.

Let $\mathcal {A}_1$ be the set of Hecke characters of $F^\times \backslash \mathbb {A}^{\times }/a^{\mathbb {Z}} $ . Let

$$\begin{align*}\mathcal{A}_{cont}:= \mathcal{A}_1\times\mathcal{A}_1/\mathfrak{S}_2, \end{align*}$$

where $\mathfrak {S}_2$ acts by permutation. An element $[(\psi _1, \psi _2)] \in \mathcal {A}_{cont}$ is called regular if $\psi _1\neq \psi _2$ and is called non-regular otherwise. Let

$$\begin{align*}\mathcal{A}_{res}\end{align*}$$

be the inertial equivalent classes of $1$ -dimensional representations of $G(\mathbb {A})$ trivial on $a^{\mathbb {Z}}G(F)$ . Let

$$\begin{align*}\mathcal{A}_0\end{align*}$$

be the set inertial equivalent classes of cuspidal automorphic representations of $G(\mathbb {A})$ whose central characters are trivial on $a^{\mathbb {Z}}$ .

Let $\mathcal {A}_{B,\psi }$ be the space of complex valued functions $\varphi $ over $G(\mathbb {A})$ satisfying that for any $k\in G(\mathcal {O})$ , there is a constant $c_k\in \mathbb {C}$ so that for any $n\in N(\mathbb {A})$ , $t\in T(\mathbb {A})$ , we have

$$\begin{align*}\varphi(ntk)=c_k\rho_B(t)\psi(t), \end{align*}$$

where $\rho _B(\begin {pmatrix} a& 0\\ 0& c \end {pmatrix})=\frac {\vert a\vert ^{\frac {1}{2}}}{\vert c\vert ^{\frac {1}{2}}}$ . Equivalently, it is the space of those $\varphi $ such that for any $x\in G(\mathbb {A})$ , $n\in N(\mathbb {A})$ and $t\in T(\mathbb {A})$ , we have $\varphi (ntx)= \rho _B(t)\psi (t)\varphi (x)$ . Let w be the non-trivial element in the Weyl group of $(G,T)$ and $\lambda \in X_T^G$ . We have the intertwining operator $\mathcal {A}_{B,\psi }\longrightarrow \mathcal {A}_{B,w(\psi )}$ defined by analytic continuation of the integral below which converges when $|Re \lambda |>>0$ ,

(4.1.1) $$ \begin{align}({\mathbf{M}(w,\lambda)}\varphi)(x):=\lambda(x)\int_{ {N}(\mathbb{A})}\varphi(w^{-1}nx) \lambda(w^{-1}nx) \mathrm{d} n, \quad \end{align} $$

where we view $\lambda \in X_T^G$ as a function over $G(\mathbb {A})$ using Iwasawa decomposition (i.e., if $x=ntk$ with $n\in N(\mathbb {A})$ , $t\in T(\mathbb {A})$ and $k\in \mathcal {K}$ , we define $\lambda (x):=\lambda (t)$ ).

Theorem 4.1.1 (Arthur, Lafforgue).

The spectral expansion is the identity

$$\begin{align*}J_{spec}^{1}(f) =\sum_{[\pi]\in \mathcal{A}_0} J_\pi(f) + \sum_{[\chi]\in\mathcal{A}_{res}} J_\chi(f)+\sum_{[\psi]\in \mathcal{A}_{cont}}J_{\psi}(f), \end{align*}$$

where each sum is taken over a set of representatives, and the terms are defined as follows. We denote the three sums by, respectively, $J^{1}_{cusp}(f)$ , $J_{res}^{1}(f)$ and $J_{cont}^{1}(f)$ .

For $\pi \in \mathcal {A}_0$ , if $\pi \otimes \epsilon \cong \pi $ , then

$$\begin{align*}J_{\pi}(f)= \frac{1}{2}(\mathrm{Tr}(R(f)\vert \pi) - \mathrm{Tr}(R(f)\circ \epsilon \vert \pi) ), \end{align*}$$

and if $\pi \otimes \epsilon \not \cong \pi $ , then

$$\begin{align*}J_{\pi}(f) = \mathrm{Tr}(R(f)\vert \pi). \end{align*}$$

For $\chi \in \mathcal {A}_{res}$ , we have

$$\begin{align*}J_{\chi}(f)=\mathrm{Tr}(R(f)\vert \chi). \end{align*}$$

For any $\lambda \in X_T^G$ , let $R(f,\lambda )$ be the twisted action on $\mathcal {A}_{B,\psi }$ :

$$\begin{align*}R(f,\lambda)\varphi = (R(f)(\varphi \lambda))\lambda^{-1}, \end{align*}$$

where we view $\lambda \in X_T^G$ as a function over $G(\mathbb {A})$ by Iwasawa decomposition: $\lambda (x)=\lambda (t)$ if $x=ntk$ for $n\in N(\mathbb {A})$ , $t\in T(\mathbb {A})$ and $k\in G(\mathcal {O})$ . For $\psi \in \mathcal {A}_{cont}$ , if $\psi $ is regular, then

$$\begin{align*}J_{\psi}(f)= \int_{\mathrm{Im} X_{T}^{{G}}} \lim_{\mu\longrightarrow 1} \mathbf{Tr}_{\mathcal{A}_{B,{\psi}}} \!\Big(\Big( - \frac{1}{\mu^{-1} - \mu} \mathbf{M}(w, \lambda)^{-1}\circ \mathbf{M}(w,\lambda/\mu )+ \frac{1}{\mu^{-1} - \mu} \Big) \circ R(f, \lambda)\Big) \mathrm{d}\lambda , \end{align*}$$

and if $\psi $ is not-regular, then

(4.1.2) $$ \begin{align} J_{\psi}(f) &=\frac{1}{2} \int_{\mathrm{Im} X_{T}^{{G}}} \lim_{\mu\longrightarrow 1} \mathbf{Tr}_{\mathcal{A}_{B,{\psi}}} \Big(\! \Big(\! - \frac{1}{\mu^{-1} - \mu} \mathbf{M}(w,\lambda)^{-1}\circ \mathbf{M}(w,\lambda/\mu )+ \frac{1}{\mu^{-1} - \mu} \Big) \circ R(f, \lambda) \!\Big) \mathrm{d}\lambda \nonumber \\ &\quad +\frac{1}{8} \sum_{ \lambda_{G}\in \{ \pm 1 \} } \sum_{\substack{ \lambda_{w}\in \mathrm{Im} X_{T}^{G} \\ \lambda_{w}^2=\lambda_{G}^{-1} } } {\lambda_{G}} \mathbf{Tr}_{\mathcal{A}_{B,{\psi}}}( \mathbf{M}(w, w^{-1}({\lambda_{w}})) \circ R(f, \lambda) ). \end{align} $$

Note that if f is supported in $G(\mathcal {O})$ , then $R(f, \lambda )=R(f)$ .

Proof. The theorem is established by L. Lafforgue in [Reference Lafforgue21], and we refer the reader to [Reference Yu37, Th. 5.2.2, Co. 5.2.3]. For the group $G=GL_2$ , we can make the result more explicit. In [Reference Yu37, 5.2.1], we have calculated explicitly the functions , and on $X_T^G$ , which appears in the spectral expansion of the trace formula. They are given by the following formula:

that is,

and

4.2 Summary of main results of this section

The primary purpose of this section is to construct a specific function $f\in C_c^{\infty }(G(\mathbb {A}))$ (Proposition 4.2.1), and calculate explicitly the spectral side of the trace formula $J^{1}_{spec}(f)$ (Theorem 4.1.1).

Proposition 4.2.1. We use notation of Theorem 3.1.2. For each $v\in S$ , we define the following functions following the ramification type of ${\mathfrak {R}}_v$ .

1. (r) Let $v\in S_r$ . The function $f_v\in C_c^{\infty }(G_v)$ is defined by

   $$\begin{align*}f_v(x)=\begin{cases} 0,\quad x\notin \mathcal{K}_v ;\\ \mathrm{Tr}(\rho_v(\overline{x}^{-1})), \quad x\in \mathcal{K}_v , \end{cases} \end{align*}$$
   where $\overline {x}$ is the image of x in $G(\kappa _v)$ under the projection $\mathcal {K}_v\longrightarrow G(\kappa _v)$ .

2. (c) Let $v\in S_c$ . The function $f_v\in C_c^{\infty }(G_v)$ is defined by

   $$\begin{align*}f_v(x)=\begin{cases} 0,\quad x\notin \mathcal{K}_v ;\\ \mathrm{Tr}(\rho_v(\overline{x}^{-1})), \quad x\in \mathcal{K}_v. \end{cases} \end{align*}$$

3. (s) Let $v\in S_s$ . The function $f_v\in C_c^{\infty }(G_v)$ is supported in $\mathcal {K}_v $ such that for any $x\in \mathcal {K}_v $ , we have

   $$\begin{align*}f_v(x)=\theta_v(\det \overline{x}^{-1}); \end{align*}$$

4. (u) Let $v\in S_u$ . The function $f_v\in C_c^{\infty }(G_v)$ is defined by

   

Let

$$\begin{align*}f = \otimes f_v\in C_c^{\infty}(G(\mathbb{A})), \end{align*}$$

where $f_v$ is the function defined above if $v\in S_{(?)}$ for $?=r, c, s, u$ and if $v\notin S$ . Then for any cuspidal automorphic representation $\pi $ of $G(\mathbb {A})$ , the condition

$$\begin{align*}\mathrm{Tr}(f\vert \pi)\neq 0 \end{align*}$$

holds if and only if $\pi $ has correct ramification type. If it is the case, we have

$$\begin{align*}\mathrm{Tr}(f\vert \pi)=1.\end{align*}$$

Proof. Suppose $\pi =\otimes '\pi _v$ is a cuspidal automorphic representation. We have

$$\begin{align*}\mathrm{Tr}(f\vert \pi)= \prod_{v}\mathrm{Tr}(f_v\vert \pi_v).\end{align*}$$

The statement is, therefore, of local nature.

For $v\notin S$ , the function acts as a projection to $\mathcal {K}_v$ -fixed part of $\pi _v$ . It is non-zero if and only if $\pi _v$ is unramified by definition. If this is the case, $\pi _v^{\mathcal {K}_v}$ is a $1$ -dimensional vector space, and the trace of is one. For $v\in S_s$ , it is similar. In fact, suppose that $\pi _v=\pi _v^{\prime }\otimes \eta $ with $\pi _v^{\prime }$ being unramified. Then

The trace is $1$ or $0$ depending on whether or not $\pi _v$ has the correct ramification.

For $v\in S_{c}$ and $v\in S_{r}$ , this result has been proved in [Reference Yu38, Theorem 2.4.4].

Finally, we consider the case $v\in S_{u}$ . As above, up to a twist, we are reduced to the case that $\chi '$ is trivial. Note that if

$$\begin{align*}\mathrm{Tr}(f_v\vert \pi_v)\neq 0, \end{align*}$$

then $\pi _v$ contains a non-zero fixed vector under $\mathcal {I}_v$ . By a result of Casselman [Reference Laumon20, Th. 7.4.4], $\pi _v$ must be an irreducible subrepresentation of a parabolic induction $\mathrm {Ind}_{B_v}^{G_v}\theta _v$ for a character $\theta _v$ of $T_v$ that is trivial on $T({\mathcal {O}}_v)$ . If the representation $\mathrm {Ind}_{B_v}^{G_v}\theta _v$ is irreducible, then it is unramified. Hence, its $\mathcal {I}_v$ -fixed subspace has dimension $2$ . We deduce that

$$\begin{align*}\mathrm{Tr}(f_v\vert \pi_v)= 0. \end{align*}$$

If $\mathrm {Ind}_{B_v}^{G_v}\theta _v$ is not irreducible, then it is of length $2$ whose irreducible quotients are a $1$ -dimensional representation and an unramified twist of the Steinberg representation. Since $\pi $ is cuspidal, $\pi _v$ cannot be $1$ -dimensional; therefore, $\pi _v$ is an unramified twist of the Steinberg representation whose $\mathcal {I}_v$ -fixed subspace is $1$ -dimensional and $\mathcal {K}_v$ -fixed subspace is $0$ . This completes the proof.

The following result is deduced from Lemma 4.5.1, Corollary 4.5.3, Proposition 4.6.1 and Proposition 4.7.1. Notations are those of the introduction.

4.3 Counting $\ell $ -adic local systems in rank 1

We need to discuss the cases in rank $1$ first, not only for completeness but also because these results will be needed when calculating the cases in rank $2$ . It has been dealt with in [Reference Deligne10, Section 6]; thus, our discussion will be concise.

A difference between a number field and a function field is that the function field F, and hence all its local fields, is an ${\mathbb {F}}_q$ -algebra. Given a character $\chi _v: F_v^\times \longrightarrow {\overline {\mathbb {Q}}}_{\ell }^{\times }$ , we can consider its restriction to ${\mathbb {F}}_q^{\times }$ . Let $\chi $ be an $\ell $ -adic Hecke character

$$\begin{align*}\chi=\prod_{v}\chi_v: F^{\times}\backslash \mathbb{A}^{\times} \longrightarrow {\overline{\mathbb{Q}}}_{\ell}^{\times} .\end{align*}$$

It is a character of $\mathbb {A}^{\times }$ trivial on $F^{\times }$ , in particular on ${\mathbb {F}}_q^{\times }$ ; therefore, necessarily, we have

(4.3.1) $$ \begin{align} \prod_{v}\chi_v\vert _{{\mathbb{F}}_q^{\times}}=1.\end{align} $$

Suppose that ${\mathfrak {R}}_1=({\mathfrak {R}}_{1,x})_{x\in \overline {S}}$ is a family of rank $1 \ell $ -adic local systems over $(\overline {X}^{\ast }_{(x)})_{x\in \overline {S}}$ fixed by ${\mathrm {Frob}}^{\ast }$ . Suppose that they are tame and $\varepsilon _x$ are eigenvalues of the local monodromy actions. By similar discussion as in 3.1, but using local class field theory, we obtain for each $v\in S$ a character $\theta _v$ of ${\mathcal {O}}_v^{\times }$ trivial on $1+\wp _v$ by tameness. The condition

(4.3.2) $$ \begin{align}\prod_{x\in \overline{S}}\varepsilon_x=1 \end{align} $$

is equivalent to

(4.3.3) $$ \begin{align} \prod_{v}\theta_v\vert _{{\mathbb{F}}_q^{\times}}=1. \end{align} $$

Let $A_1({\mathfrak {R}}_1)$ be the set of inertial equivalent classes of Hecke characters $\chi $ of $F^\times \backslash \mathbb {A}^\times $ such that $\chi _v$ extends $\theta _v$ . The set $A_1({\mathfrak {R}}_1)$ is in bijection with $E_1({\mathfrak {R}}_1)^{{\mathrm {Frob}}^{\ast }}$ .

If the condition (4.3.2) is satisfied, then we have

(4.3.4) $$ \begin{align}\vert E_1(\mathfrak{R})^{{\mathrm{Frob}}^*}\vert =\vert \mathrm{Pic}^{0}(X)({\mathbb{F}}_q)^{\vee}\vert =\mathrm{Pic}(1) ,\end{align} $$

otherwise

$$\begin{align*}E_1(\mathfrak{R})^{{\mathrm{Frob}}^*}=\emptyset. \end{align*}$$

In fact, as long as $ E_1(\mathfrak {R})^{{\mathrm {Frob}}^*}$ is non-empty, it is a principal homogenous space under $ E_1(\emptyset )^{{\mathrm {Frob}}^*}$ . In this case, it has cardinality $\mathrm {Pic}(1)=\vert \mathrm {Pic}^{0}(X)({\mathbb {F}}_q)\vert $ .

4.4 Eulerian decomposition and calculations on Whittaker functions

Let $f=\otimes f_v\in {C}_c^\infty (G(\mathbb {A}))$ be the function defined in Proposition 4.2.1. Let $\pi \cong \otimes ' \pi _v$ be a cuspidal automorphic representation of $G(\mathbb {A})$ .

Suppose that $\pi \cong \pi \otimes \epsilon $ as representations of $G(\mathbb {A})$ . We need to consider

$$\begin{align*}\mathrm{Tr}( \epsilon\circ R(f)\vert \pi).\end{align*}$$

Note that this trace is not well-defined from a pure representation theoretical point of view because the action of $\epsilon $ on $\pi $ relies on the isomorphism $\pi \cong \pi \otimes \epsilon $ . For $G=GL_2$ , a canonical isomorphism is furnished by the multiplicity one theorem, which says that $\pi $ and $\pi \otimes \epsilon $ have the same underlying space of cusp forms.

A similar problem arises if we want to get a Eulerian decomposition of the trace $\mathrm {Tr}( \epsilon \circ R(f)\vert \pi )$ . We need to choose isomorphisms $\pi _v\cong \pi _v\otimes \epsilon $ so that their tensor product is compatible with the global isomorphism (we also need to assume that the isomorphisms can be glued together to a restricted tensor product isomorphism; that is, they fix the implicit chosen $\mathcal {K}_v$ -invariant vector for almost all places v). A natural way to do so is by using Whittaker models.

Let $\psi =\otimes \psi _v: \mathbb {A}\longrightarrow \mathbb {C}^\times $ be an additive character of $\mathbb {A}$ , viewed as a character of $N(\mathbb {A})$ , where $\psi _v$ are additive characters of $F_v$ . Suppose that $\psi _v$ has conductor $\wp _v^{-n_v}$ (i.e., is trivial on $\wp _v^{-n_v}$ but not on $\wp _v^{-n_v-1}$ ). We have

$$\begin{align*}\sum_v n_v\deg v=2g-2. \end{align*}$$

Recall that g is the genus of the curve X. Let $\mathcal {W}(\pi )$ be the global Whittaker model of $\pi $ with respect to $\psi $ , that is, the space of smooth functions $\varphi $ over $G(\mathbb {A})$ such that

$$\begin{align*}\varphi(ux)=\psi(u)\varphi(x), \quad \forall u\in N(\mathbb{A}) , \forall x\in G(\mathbb{A}), \end{align*}$$

and $\mathcal {W}(\pi _v)$ the local Whittaker model of $\pi _v$ with respect to $\psi _v$ (space of functions over $G_v$ similarly defined). Then we have a natural decomposition $\mathcal {W}(\pi )=\otimes _v' \mathcal {W}(\pi _v)$ and $\mathcal {W}(\pi _v)=\mathcal {W}(\pi _v\otimes \epsilon )$ . Therefore, we have

(4.4.1) $$ \begin{align} \mathrm{Tr}( \epsilon\circ R(f)\vert \pi)=\prod_{v\in |X|} \mathrm{Tr}( \epsilon_v\circ R(f_v)\vert \mathcal{W}({\pi_v})), \end{align} $$

where $\epsilon _v(x_v)=(-1)^{\deg v \deg (\det (x_v))}$ for $v\in G_v$ .

Proof. This is a corollary of Paskunas and Stevens’ result [Reference Paskunas and Stevens26, Theorem 5.8]. Let us explain their notation (see p. 1233 of op. cit.) from type theory that we need to apply to our specific case. The group J is $\mathcal {K}_v$ , the group $\mathbf {J}$ is $Z_v\mathcal {K}_v$ , the group U is $N_v$ , the representation $\Lambda $ of $\mathbf {J}$ is one that extends $\rho $ , and the character $\Psi _\alpha $ is obtained by restriction of $\psi ^{\prime }_v$ to $N({\mathcal {O}}_v)$ , then extends to $N({\mathcal {O}}_v)(1+\wp _vM_2({\mathcal {O}}_v))$ .

Let $X\in \pi _v$ and $Y\in \pi _v^\vee $ , where $\pi _v^\vee $ is the contragredient representation of $\pi _v$ . Let $\Phi _{X,Y}$ be the matrix coefficient of $\pi _v$ defined by $\Phi _{X,Y}(x)=\langle \pi _v(x)X, Y\rangle $ , for any $x\in G_v$ . If there are $X, Y$ such that $\Phi _{X,Y}\neq 0$ , then the map $X\mapsto \Phi _{X,Y}$ embeds $\pi _v$ into $\mathcal {W}\subseteq C_c^\infty (G_v)$ .

The result [Reference Paskunas and Stevens26, Th. 5.8] shows that $\pi _v^\vee $ contains a special vector $Y_\alpha ^\vee $ so that the linear map from $\pi _v$ to the space of Whittaker functions

$$\begin{align*}X\mapsto \left( x\mapsto \int_{N_v}\psi^{\prime}_v(u) \Phi_{X,Y_\alpha^\vee}(u^{-1}x) \mathrm{d} u \right) \end{align*}$$

is non-zero. Moreover, Paskunas and Stevens show that $\pi _v$ contains a vector $Y_\alpha $ contained in $\rho $ -isotypic part $(\pi _v)_{\rho }$ of $\pi _v$ , so that $\Phi _{Y_\alpha , Y_\alpha ^\vee }$ has support in $Z_v\mathcal {K}_v$ ([Reference Paskunas and Stevens26, Prop. 5.7]) and its associated Whittaker function

$$\begin{align*}x\mapsto \int_{N_v}\psi^{\prime}_v(u) \Phi_{Y_\alpha,Y_\alpha^\vee}(u^{-1}x)\mathrm{d} u\end{align*}$$

is supported in $Z_vN_v\mathcal {K}_v$ and extends the function $\Phi _{Y_\alpha , Y_\alpha ^\vee }$ supported on $Z_v\mathcal {K}_v$ . Note that since the irreducible representation $\rho $ is contained with multiplicity one in $\pi _v$ ([Reference Yu38, Theorem 2.4.4]), every $X\in (\pi _v)_{\rho }$ is generated by $\langle \pi _v(k)Y_\alpha : k\in \mathcal {K}_v\rangle $ . Therefore, every Whittaker function associated to $\Phi _{X,Y_\alpha ^\vee }$ for $X\in (\pi _v)_{\rho }$ has support in $Z_vN_v\mathcal {K}_v$ .

Corollary 4.4.2. Suppose we are in the situation of Theorem 4.4.1. Let $\mathcal {W}(\pi _v)$ be the Whittaker model for the character $\psi _v$ of conductor $\wp _{v}^{-n_v}$ . Then every function in $\mathcal {W}(\pi _v)_\rho $ is supported in

$$\begin{align*}\{x\in G_v\vert v(\det(x))\in -n_v-1+2\mathbb{Z} \}, \end{align*}$$

where v is the valuation of $F_v$ normalized to be surjective to $\mathbb {Z}$ .

Proof. Let $t_v$ be a uniformizer of $\wp _v$ . Let $\psi ^{\prime }_v:=(y\mapsto \psi _v(t_v^{-n_v-1} y))$ . Then $\psi ^{\prime }_v$ has conductor $\wp _v$ . Let $b=\begin {pmatrix} t_v^{-n_v-1}&0\\0&1 \end {pmatrix} $ . We have an $G_v$ -equivariant isomorphism:

$$ \begin{align*} \mathcal{W}(\pi_v, \psi_v)&\longrightarrow \mathcal{W}(\pi_v, \psi_v') , \\ \varphi &\mapsto (x\mapsto \varphi(bx)). \end{align*} $$

By Theorem 4.4.1, we deduce that functions in $\mathcal {W}(\pi _v, \psi _v)_\rho $ are supported in $bZ_vN_v\mathcal {K}_v$ . This implies the desired result.

4.5 Cuspidal terms

We apply the previous preparation works to compute the cuspidal terms $J_{\pi }(f)$ in the spectral expansion. In fact, it is the case that $\pi \otimes \epsilon \cong \pi $ that is non-trivial.

Recall that we have defined in the Introduction two commutative actions $\sigma $ and ${\mathrm {Frob}}^\ast $ on $P_{\mathfrak {R}}$ (see (1.1.2)) so that $ b_{\mathfrak {R}}(k)$ is the cardinality of the set of fixed points of $\sigma \circ {\mathrm {Frob}}^{\ast k}$ and $ c_{\mathfrak {R}}(k)$ is that of ${\mathrm {Frob}}^{\ast k}$ .

Proof. The first statement is trivial because, in this case, $P_{\mathfrak {R}}$ is at most a singleton.

Suppose that $S_r$ contains a point of odd degree, meaning that ${\mathrm {Frob}}^\ast $ has an orbit of odd length on ${\overline {S}}_{cr}$ . Suppose that $a\in 2\mathbb {Z}+1$ is the length of such an orbit and $x_0\in {S}_r\otimes \overline {\mathbb {F}}_q$ is a point in it. We have $\varepsilon _{x_0}(1)\neq \varepsilon _{x_0}(2)$ . Suppose that ${\mathrm {Frob}}^{\ast a}((\varepsilon _{x}(i_x))_{x})=(\varepsilon _{x}')_x$ . Here, $i_x\in \{1,2\}$ for each $x\in \overline {S}$ . By the choice of $x_0$ and a, we have $\varepsilon ^{\prime }_{x_0}= \varepsilon _{x_0}(i_{x_0})$ . In particular,

$$\begin{align*}{\mathrm{Frob}}^{\ast a}((\varepsilon_{x}(i_x))_{x})\neq \sigma((\varepsilon_{x}(i_x))_{x}), \end{align*}$$

for any $(i_x)_x\in \{1,2\}^{\overline {S}}$ . This implies that $b_{\mathfrak {R}}(a)=0$ . Since a is odd, $\sigma =\sigma ^a$ , we have $b_{\mathfrak {R}}(1)=0$ .

Similarly, if $S_c$ contains a point of degree a, we have $c_{\mathfrak {R}}(a)=0$ . It implies that $c_{\mathfrak {R}}(1)=0$ . If a is even, then $\sigma ^{a}=\operatorname {\mathrm {Id}}$ . Hence, $b_{\mathfrak {R}}(1)=0$ .

Theorem 4.5.2. Let $\pi $ be a cuspidal automorphic representation of $G(\mathbb {A})$ . Recall that $\epsilon $ is the sign character of $G(\mathbb {A})$ that factors through $\deg \circ \det $ . Suppose that $\pi \otimes \epsilon \cong \pi $ . Let f be the function introduced in Proposition 4.2.1. Then

$$\begin{align*}\mathrm{Tr}(\epsilon \circ R(f)\vert \pi ) =0 , \end{align*}$$

unless all the following conditions are met: 1. $b_{\mathfrak {R}}(1)\neq 0$ ; 2. every place in $S_u$ has odd degree; 3. $\pi _v\in {\mathrm {Irr}}^{\mathfrak {R}}(G_v)$ for $v\in |X|-S_u$ and $\pi _v$ has scalar ramification determined by semisimplification of ${\mathfrak {R}}_v$ for $v\in S_u$ . If this is the case, we have

$$\begin{align*}\mathrm{Tr}(\epsilon \circ R(f)\vert \pi ) =(-1)^{|S_u|}2^{|S_u|} (-1)^{\deg S_c }. \end{align*}$$

Proof. By Langlands correspondence, if $\pi \cong \pi \otimes \epsilon $ , no local component of $\pi $ can be a twisted Steinberg representation. In fact, since a Hecke character of $F^\times \backslash \mathbb {A}^\times $ is of finite order if and only if it sends an element of degree $1$ to a root of unity, if necessary, by replacing $\pi $ by an inertially equivalent, we may assume that the central character of $\pi $ is of finite order. Suppose that $\mathfrak {L}$ is the $\ell $ -adic local system over ${X}-{S}$ that corresponds to $\pi $ . If $\pi \otimes \epsilon \cong \pi $ , then

$$\begin{align*}\mathfrak{L}|_{\overline{X}-\overline{S}}\cong \mathfrak{L}_1\oplus \mathfrak{L}_2, \end{align*}$$

and ${\mathrm {Frob}}^\ast $ permutes $\mathfrak {L}_1$ and $\mathfrak {L}_2$ ([Reference Yu37, Prop. 2.1.3]). In particular, the ramification of $\mathfrak {L}$ at every $x\in \overline {S}$ is semisimple. Therefore, $\pi $ does not have a twisted Steinberg component by Theorem 3.1.2.

It is clear that if

$$\begin{align*}\mathrm{Tr}( \epsilon\circ R(f)\vert \pi)\neq 0, \end{align*}$$

then $\pi _v$ has the desired ramification type for $v\in |X|-S_u$ , and the $(\mathcal {I}_v,\iota \circ \chi )$ -isotypic subspace $(\pi _v)_{(\mathcal {I}_v,\iota \circ \chi )}$ is non-trivial (notation as in Theorem 3.1.2) for $v\in S_u$ . In particular, for $v\in S_u$ , $\pi _v$ is either a twisted Steinberg representation in ${\mathrm {Irr}}^{\mathfrak {R}}(G_v)$ or a twisted unramified principal series in ${\mathrm {Irr}}^{\mathfrak {R}^{ss}}(G_v)$ , where $\mathfrak {R}^{ss}$ is the semisimplification of $\mathfrak {R}$ . We have seen that $\pi _v$ cannot be a twisted Steinberg representation. Moreover, the product of all eigenvalues of ramifications of $\mathfrak {L}_1$ and $\mathfrak {L}_2$ at all points $x\in \overline {S}$ should be $1$ , and they are permuted by Frobenius action. This implies that $b_{\mathfrak {R}}(1)\neq 0$ .

Next, we need to calculate

$$\begin{align*}\mathrm{Tr}(\epsilon \circ R(f)\vert \pi )\end{align*}$$

when $\pi \cong \pi \otimes \epsilon $ and $\pi _v$ has the above described property. The equation (4.4.1) allows us to calculate it locally.

We note that Theorem 3.1.2 says if

$$\begin{align*}\mathrm{Tr}(R(f_v)\vert \pi_v)\neq 0,\end{align*}$$

then $\pi _v\in {\mathrm {Irr}}^{\mathfrak {R}}(G_v)$ . Let $v\in S_u$ . If $\deg (v)$ is even, then $\epsilon _v$ equals the trivial character of $G_v$ , and we have

$$\begin{align*}\mathrm{Tr}(\epsilon_v\circ R(f_v)|\mathcal{W}(\pi_v))=\mathrm{Tr}(R(f_v)\vert \pi_v). \end{align*}$$

It must be zero as $\pi _v$ is not a twisted Steinberg representation.

Suppose now that $v\in S_u$ and $\deg (v)$ is odd. Up to a twist by a character, we may assume that $\pi _v$ is unramified. We take a Whittaker function $\varphi _v\in \mathcal {W}(\pi _v)^{\mathcal {K}_v}$ . Note that since both $\epsilon _v\varphi _v$ and $\varphi _v$ are contained in $\mathcal {W}(\pi _v)^{\mathcal {K}_v}=\mathcal {W}(\pi _v\otimes \epsilon _v)^{\mathcal {K}_v}$ , they are differed by a scalar:

$$\begin{align*}\epsilon_v\varphi_v=c\varphi_v. \end{align*}$$

Let $x_v$ be any element in the support of $\varphi _v$ . We deduce that $c=\epsilon _v(x_v)$ . Let $\wp _v^{c(\psi _v)}$ be the conductor of $\psi _v$ . We have shown in [Reference Yu37, Lemme 5.5.3] that the support of $\varphi _v$ contains an element of valuation $ c(\psi _v) $ . Therefore, $c=(-1)^{\deg (v) c(\psi _v)}$ . Let $\varphi _v'\in \mathcal {W}(\pi _v)$ be the function defined by

$$\begin{align*}x\mapsto \varphi_v\left(x{\begin{pmatrix} \varpi_v& 0\\ 0& 1 \end{pmatrix}} \right) .\end{align*}$$

Then $\{\varphi _v, \varphi _v'\}$ is a basis of $ \mathcal {W}(\pi _v)^{\mathcal {I}_v}$ . In fact, by Bruhat-Iwahori decomposition, we know that $\dim _{\mathbb {C}}\mathcal {W}(\pi _v)^{\mathcal {I}_v}\leq 2$ and it is clear that $\varphi _v, \varphi _v'$ are linearly independent. The endomorphisms on $\mathcal {W}(\pi _v)$

are composition of a projection onto $\pi _{v}^{\mathcal {I}_v}$ together with a linear map represented respectively by the matrix

$$\begin{align*}\begin{pmatrix} (-1)^{\deg(v) c(\psi_v)} & 0\\ 0& (-1)^{\deg(v)(c(\psi_v)+1) } \end{pmatrix}, \begin{pmatrix} (-1)^{\deg(v) c(\psi_v)} & 0\\ 0& 0 \end{pmatrix}. \end{align*}$$

We deduce that

$$\begin{align*}\mathrm{Tr}(\epsilon_v \circ R(f_v)\vert \mathcal{W}(\pi_v)) = (-1)^{\deg(v)(c(\psi_v)+1)} - (-1)^{\deg(v) c(\psi_v)}. \end{align*}$$

Since $\deg (v)$ is odd, it equals

$$\begin{align*}\mathrm{Tr}(\epsilon_v \circ R(f_v)\vert \mathcal{W}(\pi_v))=-2(-1)^{c(\psi_v)}. \end{align*}$$

Similarly, for $v\in |X|-S_{cr}-S_u$ , we have

$$\begin{align*}\mathrm{Tr}(\epsilon_v \circ R(f_v)\vert \mathcal{W}(\pi_v))=(-1)^{c(\psi_v)\deg(v)}. \end{align*}$$

For $v\in S_c$ , we deduce from Corollary 4.4.2 that

$$\begin{align*}\mathrm{Tr}(\epsilon_v \circ R(f_v)\vert \mathcal{W}(\pi_v) )= (-1)^{\deg(v)(c(\psi_v)+1)}\mathrm{Tr}(R(f_v)\vert \mathcal{W}(\pi_v))=(-1)^{\deg(v)(c(\psi_v)+1)}. \end{align*}$$

For $v\in S_r$ , as we have seen in Lemma 4.5.1 that v must have even degree, otherwise $b_{\mathfrak {R}}(1)=0$ . We have

$$\begin{align*}\mathrm{Tr}(\epsilon_v \circ R(f_v)\vert \mathcal{W}(\pi_v) )=\mathrm{Tr}(R(f_v)\vert \mathcal{W}(\pi_v))=1. \end{align*}$$

In conclusion, by (4.4.1) we have

$$\begin{align*}\mathrm{Tr}(\epsilon \circ R(f)\vert \pi )= (-1)^{|S_u|}2^{|S_u|} (-1)^{\sum_{v\in |X|-S_c}\deg(v) c(\psi_v)+\sum_{v\in S_c} \deg(v)(c(\psi_v)+1) }. \end{align*}$$

We have

$$\begin{align*}\sum_{v} c(\psi_v)\deg(v) = - (2g-2). \end{align*}$$

Therefore,

$$\begin{align*}\mathrm{Tr}(\epsilon \circ R(f)\vert \pi ) = (-1)^{|S_u|} 2^{|S_u|} (-1)^{\deg S_c }. \end{align*}$$

Corollary 4.5.3. If $S_u\neq \emptyset $ and at least one place in it is of even degree, then

$$\begin{align*}J_{cusp}^1(f)=\vert E_2(\mathfrak{R})^{{\mathrm{Frob}}^*}\vert. \end{align*}$$

If $S_u\neq \emptyset $ and every place in it has an odd degree, then

$$\begin{align*}J_{cusp}^1(f)=\vert E_2(\mathfrak{R})^{{\mathrm{Frob}}^*}\vert + \begin{cases}(-1)^{|{S}_u|+1}{b_{\mathfrak{R}}(1)} 2^{|S_u|-2} (\mathrm{Pic}(2)-\mathrm{Pic}(1)) , \quad S_{cr}= \emptyset; \\ (-1)^{|{S}_u|+1} {b_{\mathfrak{R}}(1)}{2}^{|S_u|-2}(-1)^{\deg S_c}\mathrm{Pic}(2) , \quad S_{cr}\neq \emptyset. \end{cases} \end{align*}$$

If $S_u=\emptyset $ , then

$$\begin{align*}J_{cusp}^1(f)=\vert E_2(\mathfrak{R})^{{\mathrm{Frob}}^*}\vert + \begin{cases}0 , \quad S_{cr}= \emptyset; \\ \frac{b_{\mathfrak{R}}(1)}{4}(1-(-1)^{\deg S_c})\mathrm{Pic}(2) , \quad S_{cr}\neq \emptyset. \end{cases} \end{align*}$$

Proof. In fact, the sum of $J_\pi (f)$ over inertial equivalent classes of cuspidal automorphic representations $\pi $ such that $\pi \otimes \epsilon \not \cong \pi $ gives $|E_2(\mathfrak {R})^{{\mathrm {Frob}}^\ast }|$ after Theorem 3.2.1.

We need to consider the sum

$$\begin{align*}\frac{1}{2}\sum_{\pi} \mathrm{Tr}(R(f)|\pi) - \frac{1}{2}\sum_{\pi}\mathrm{Tr}(\epsilon\circ R(f)| \pi) , \end{align*}$$

where the sums over $\pi $ are taken over inertial equivalent classes of cuspidal automorphic representations $\pi $ such that $\pi \otimes \epsilon \cong \pi $ . The Langlands correspondence (see the proof of Theorem 4.5.2) shows that no such cuspidal automorphic representation $\pi $ can have a twisted Steinberg component. Therefore, if $S_u\neq \emptyset $ , by Proposition 4.2.1,

$$\begin{align*}\frac{1}{2}\sum_{\pi} \mathrm{Tr}(R(f)|\pi)=0. \end{align*}$$

If $S_u=\emptyset $ , we need to know the number of equivalence classes of such $\pi $ . By [Reference Yu37, 2.1.3] and the first paragraph of the proof of Theorem 4.5.2, the set of such $\pi $ is in bijection with the set of non-ordered pairs $(\mathfrak {L}_1, \mathfrak {L}_2)$ of rank $1 \ell $ -adic systems over $\overline {X}-\overline {S}$ such that ${\mathrm {Frob}}^{\ast }{\mathfrak {L}_i}\cong \mathfrak {L}_{3-i}$ , $\mathfrak {L}_1\not \cong \mathfrak {L}_2$ , and that the local monodromies of the direct sum $\mathfrak {L}_1\oplus \mathfrak {L}_2$ are given by $\mathfrak {R}$ . If $S_{cr}=\emptyset $ , then $b_{\mathfrak {R}}(1)$ equals $0$ or $1$ and there are ${b_{\mathfrak {R}(1)}}\frac {\mathrm {Pic}(2)-\mathrm {Pic}(1)}{2}$ such pairs. If $S_{cr}\neq \emptyset $ , then there are $\frac {b_{\mathfrak {R}(1)}}{2}\mathrm {Pic}(2)$ such pairs. We have

$$\begin{align*}\frac{1}{2}\sum_{\pi} \mathrm{Tr}(R(f)|\pi)=\begin{cases}\frac{b_{\mathfrak{R}}(1)}{4}(\mathrm{Pic}(2)-\mathrm{Pic}(1)) , \quad S_{cr}= \emptyset; \\ \frac{b_{\mathfrak{R}}(1)}{4}\mathrm{Pic}(2) , \quad S_{cr}\neq \emptyset. \end{cases}\end{align*}$$

By Theorem 4.5.2, if $b_{\mathfrak {R}}(1)=0$ or if $S_u\neq \emptyset $ and contains a place of even degree, then the sum

$$\begin{align*}\frac{1}{2}\sum_{\pi}\mathrm{Tr}(\epsilon\circ R(f)| \pi) \end{align*}$$

is $0$ . Otherwise if $S_u\neq \emptyset $ and does not contain any place of even degree or $S_u=\emptyset $ , then by Theorem 4.5.2, we have

$$\begin{align*}\frac{1}{2}\sum_{\pi} \mathrm{Tr}(\epsilon\circ R(f)|\pi)= (-1)^{|S_u|}\begin{cases}{b_{\mathfrak{R}}(1)}2^{|S_u|-2}(\mathrm{Pic}(2)-\mathrm{Pic}(1)) , \quad S_{cr}= \emptyset; \\ {b_{\mathfrak{R}}(1)}{2}^{|S_u|-2}(-1)^{\deg S_c}\mathrm{Pic}(2) , \quad S_{cr}\neq \emptyset. \end{cases}\end{align*}$$

4.6 Residual terms

The proposition below describes the contributions to the trace formula from the residual spectrum.

Proposition 4.6.1. If $S_{cr}$ is non-empty, then

$$\begin{align*}J_{res}^{1}(f)=0. \end{align*}$$

If $S_{cr}=\emptyset $ , then

$$\begin{align*}J_{res}^{1}(f)=c_{\mathfrak{R}}(1)(-1)^{\vert S_u\vert}\mathrm{Pic}(1). \end{align*}$$

Proof. For any $v\in S_{cr}$ and any character $\mu _v$ of $G_v$ , we have

$$\begin{align*}\mathrm{Tr}(R(f_v)\vert \mu_v)=0. \end{align*}$$

Therefore, the first statement holds, and it suffices to consider the case that $S_{cr}=\emptyset $ .

It is clear that if $v\notin S$ , then for a character $\mu _v$ of $G_v$ , we have

Let $v\in S_s$ . We have

$$\begin{align*}\mathrm{Tr}(R(f_v)\vert \mu_v)=\begin{cases} 1,\quad \mu_v\vert_{\mathcal{K}_v}=\theta_v;\\ 0, \quad \mu_v\vert_{\mathcal{K}_v}\neq \theta_v; \end{cases} \end{align*}$$

where $\theta _v$ is the character defined in Theorem 3.1.2. Similarly, let $v\in S_u$ . We have

$$\begin{align*}\mathrm{Tr}(R(f_v)\vert \mu_v)=\begin{cases}-1,\quad \mu_v\vert_{\mathcal{K}_v}=\theta_v;\\ 0, \quad \mu_v\vert_{\mathcal{K}_v}\neq \theta_v. \end{cases} \end{align*}$$

Following our discussions in rank $1$ in 4.3, we deduce that if

(4.6.1) $$ \begin{align} \prod_{v}\theta_v|_{{\mathbb{F}}_q^\times}\neq 1, \end{align} $$

then

$$\begin{align*}J^{1}_{res}(f)=0. \end{align*}$$

Otherwise, there are $\mathrm {Pic}(1)$ such equivalent classes of $\mu $ . Therefore, we have

$$\begin{align*}J^{1}_{res}(f)=(-1)^{\vert S_u\vert}\mathrm{Pic}(1). \end{align*}$$

The result is thus proved since (4.6.1) is equivalent to $c_{\mathfrak {R}}(1)\neq 0$ (cf. [Reference Deligne10, 3.2] or [Reference Yu38, Proposition 2.4.2]).

4.7 Continuous terms

The proposition below describes the contributions to the trace formula from the continuous spectrum.

Proposition 4.7.1. If $S_c\neq \emptyset $ , then

$$\begin{align*}J^{1}_{cont}(f)=0. \end{align*}$$

If $S_c=\emptyset $ , then we have the following results.

1. (1) $S_{r}\neq \emptyset $ .

   $$\begin{align*}J_{cont}^1(f)=\begin{cases} \frac{1}{2} c_{\mathfrak{R}}(1)\mathrm{Pic}(1)^2(2g-2+\deg S_{r}), \quad S_u=\emptyset; \\ \frac{1}{2}c_{\mathfrak{R}}(1)\mathrm{Pic}(1)^2 \deg v , \quad S_u=\{v\}; \\ 0, \quad |S_u|\geq 2. \end{cases} \end{align*}$$

2. (2) $S_{r}=\emptyset $ .

   $$\begin{align*}J_{cont}^1(f)=\begin{cases} c_{\mathfrak{R}}(1)\mathrm{Pic}(1)^2(g-1), \quad S_u=\emptyset; \\ c_{\mathfrak{R}}(1)\mathrm{Pic}(1)^2 \frac{\deg v}{2}+ \frac{1}{2}c_{\mathfrak{R}}(1)\mathrm{Pic}(1) , \quad S_u=\{v\} \text{ and } 2\nmid \deg v; \\c_{\mathfrak{R}}(1)\mathrm{Pic}(1)^2 \frac{\deg v}{2} , \quad S_u=\{v\} \text{ and } 2\mid \deg v; \\ c_{\mathfrak{R}}(1)\mathrm{Pic}(1) (-1)^{|S_u|+1}2^{|S_u|-2}, \quad |S_u|\geq 2, \text{ and } S_{u, even}= \emptyset; \\ 0, \quad |S_u|\geq 2, \text{ and } S_{u, even}\neq \emptyset. \end{cases} \end{align*}$$

The proof is given in 4.7.6. We need first to do some calculations about L-functions and intertwining operators.

The first thing to remark is that if $S_c\neq \emptyset $ , then for any $\psi \in \mathcal {A}_{cont}$ , the action of $R(f)$ on $\mathcal {A}_{B,\psi }$ must be the $0$ map. In fact, at a place $v\in S_c$ , we have $\mathcal {A}_{B,\psi }\cong I_B(\psi _v)$ , the induced representation of $\psi _v$ . By Theorem 3.1.2, it does not contain $\rho _v$ , as $\rho _v$ is cuspidal. Therefore, we assume that $S_c=\emptyset $ in the following.

4.7.1 L-functions of Hecke characters

We will need information on L-functions of Hecke characters when dealing with the continuous part of the trace formula. Over a function field, we have a complete understanding of them.

Let $\chi : \mathbb {A}^{\times }/F^{\times }\longrightarrow \mathbb {C}^{\times }$ be a Hecke character of finite order. Let $\chi =\otimes _{v\in \vert F\vert }\chi _v$ be its factorisation. Recall that the L-function $L(\chi , z)$ can be defined by the formal power series

$$\begin{align*}L(\chi, z):= \prod_{v\in \vert X\vert } L_v(\chi, z), \end{align*}$$

where

$$\begin{align*}L_v(\chi, z) = \begin{cases} \frac{1}{1-\chi_v(\varpi_v)z^{\deg v}}, \quad \text{ if } \chi_v \text{ is unramified}; \\ 1\quad otherwise. \end{cases} \end{align*}$$

The infinite product is absolutely convergent and is holomorphic if $\vert z\vert <q^{-1}$ . It admits a meromorphic continuation to the whole $\mathbb {C}$ . Moreover, it is a rational function in z. Let R be the set of places of ramifications of $\chi $ , identified with a subset of closed points of X. We have fixed an isomorphism $\iota :{\overline {\mathbb {Q}}}_{\ell }\xrightarrow {\sim } \mathbb {C}$ . Let $\mathfrak {L}_\chi $ be an $\ell $ -adic local system over ${X} - {R}$ corresponding to $\chi $ obtained by the global class field theory. The L-function of $\mathfrak {L}_\chi $ equals that of $\chi $ :

$$\begin{align*}\iota( L(\mathfrak{L}_\chi, z)) =L(\chi, z) .\end{align*}$$

Moreover, we know from Grothendieck’s cohomological interpretation (cf., for example, [Reference Lafforgue22, Théorème VI.1]) that

$$\begin{align*}L(\mathfrak{L}_\chi, z) = \frac{\det(1- z\mathrm{F}_q \vert H^{1}_{c}(\overline{X}-\overline{R}, \mathfrak{L}_\chi ))}{ \det(1- z\mathrm{F}_q\vert H^{2}_{c}(\overline{X}-\overline{R}, \mathfrak{L}_\chi )) \det(1- z\mathrm{F}_q \vert H^{1}_{c}(\overline{X}-\overline{R}, \mathfrak{L}_\chi )) } , \end{align*}$$

where $\mathrm {F}_q$ is a geometric Frobenius element acting on cohomologies with compact support.

Proposition 4.7.2. (Riemann hypothesis) Let $\chi $ be a Hecke character on $F^{\times }\backslash \mathbb {A}^{\times }$ of finite order so that the set of ramified places is R. If $\chi $ is inertially equivalent to the trivial character (in particular $R=\emptyset $ ), then

$$\begin{align*}L(\chi, z)=\frac{P(z)}{(1-z)(1-qz)},\end{align*}$$

where $P(z)$ is a polynomial of $2g$ .

If $\chi $ is not inertially equivalent to the trivial character, then its L-function $ L(\chi , z) $ is a polynomial in z of degree $2g-2+\deg R$ .

In any case, all of the zeros of $ L(\chi , z) $ satisfy $\vert z\vert =q^{-\frac {1}{2}}. $

Proof. If R is empty, then there are two cases. If $\mathfrak {L}_{\chi }|_{\overline {X}}$ is trivial, up to a twist of a rank $1$ sheaf over $\operatorname {\mathrm {Spec}}({\mathbb {F}}_q)$ , we may assume that $\mathfrak {L}_\chi \cong {\overline {\mathbb {Q}}}_\ell $ is the constant sheaf. We have

$$\begin{align*}L(\mathfrak{L}_\chi, z)=\zeta_{X}(z)=\frac{P(z)}{(1-z)(1-qz)}, \end{align*}$$

$\deg P(z)=2g$ and all of the zeros of $ P(z) $ satisfy $\vert z\vert =q^{-\frac {1}{2}}$ . If $\mathfrak {L}_{\chi }|_{\overline {X}}$ is non-trivial, then $L(\mathfrak {L}_\chi , z)$ is a polynomial of degree $2g-2$ . For reference, see [Reference Yu37, Prop. 6.1.1].

We consider the case that R is non-empty. We know that

(4.7.1) $$ \begin{align} H^{0}_c(\overline{X}-\overline{R}, \mathfrak{L}_\chi)=0 \end{align} $$

as $ \mathfrak {L}_\chi $ is non-trivial. By Poincaré duality,

(4.7.2) $$ \begin{align}H^{2}_c(\overline{X}-\overline{R}, \mathfrak{L}_\chi)^{\vee} \cong H^{0}(\overline{X}-\overline{R}, \mathfrak{L}_\chi^{\vee})(1)\cong Hom_{ \overline{X}-\overline{R}}(\mathfrak{L}_\chi^{\vee}, {\overline{\mathbb{Q}}}_{\ell} )(1). \end{align} $$

Therefore, we deduce that $H^{2}_c(\overline {X}-\overline {R}, \mathfrak {L}_\chi )$ is $0$ as well. The dimension of $H^{1}_c(\overline {X}-\overline {R}, \mathfrak {L}_\chi )$ can then be derived from the Euler-Poincaré characteristic:

$$ \begin{align*}\dim H^{1}_c(\overline{X}-\overline{R}, \mathfrak{L}_\chi) = -\chi_c(\overline{X}-\overline{R}, \mathfrak{L}_\chi),\end{align*} $$

which can be calculated by the Grothendieck-Ogg-Shafarevich formula; see [Reference Raynaud27, Théorème 1, p.133]. In fact, since every local monodromy of $\mathfrak {L}_\chi $ is tamely ramified, the Swan conductor is zero. We deduce from loc. cit. that $\chi _c(\overline {X}-\overline {R}, \mathfrak {L}_\chi )= \chi _c(\overline {X}-\overline {R})=2-2g-\deg R$ .

The assertion about the positions of zeros is the Riemann hypothesis for rank $1$ local systems (see [Reference Lafforgue22, Théorème VI.10] for a general statement).

4.7.2 Eulerian expansions of intertwining operators

Let $\psi \in \mathcal {A}_{cont}$ . Let

$$\begin{align*}{\mathbf{M}}(w,\lambda): \mathcal{A}_{B,\psi} \longrightarrow \mathcal{A}_{B,w(\psi)}\end{align*}$$

be an intertwining operator. It is a $G(\mathbb {A})$ -morphism.

Let $I_B(\psi _v)$ be the space of functions $\varphi $ over $G_v$ satisfying $\varphi (ntx)=\rho _B(t)\psi _v(t)\varphi (x)$ for any $n\in N_v$ , $t\in T_v$ and $x\in G_v$ . By definition of intertwining operator, we have an Eulerian expansion. Indeed, let $\mathbf {M}_{v}(w, \lambda ): I_B(\psi _v)\longrightarrow I_{B}(w(\psi _v))$ be an operator defined by analytic continuation of the integral which converges when $|\mathrm {Re} \lambda |>>0$ ,

(4.7.3) $$ \begin{align} (\mathbf{M}_{v}(w, \lambda) \varphi )(x) =\lambda(x) \int_{ {N}_v} \varphi(w^{-1}nx)\lambda(w^{-1}nx) \mathrm{d} n. \end{align} $$

Choosing isomorphisms $c_\psi : \mathcal {A}_{B,\psi }\longrightarrow \otimes _{v} I_B(\psi _v)$ and $c_{w(\psi )}: \mathcal {A}_{B,w(\psi )}\longrightarrow \otimes _{v} I_B(w(\psi _v))$ , there is a constant c depending only on these isomorphisms such that the following diagram is commutative:

In the special case that $\psi $ is non-regular (i.e., $w(\psi )=\psi $ ), we have

$$\begin{align*}c=q^{1-g}.\end{align*}$$

This number is derived from the difference in the normalizations of the Haar measures on $N_v$ and $N(\mathbb {A})$ .