1 Introduction
To an elliptic curve E over a number field K, one can associate representations
of the absolute Galois group
. If E does not have complex multiplication (CM), then the images of these representations are open [Reference SerreSer72], and thus of finite index. It is conjectured ([Reference ZywinaZyw15a, Conjecture 1.12], [Reference SutherlandSut16, Conjecture 1.1]) that $\rho _{E, \ell }$ is surjective for nonCM $E/\mathbb {Q}$ for all $\ell> 37$ and that there is an absolute bound on the index of $\rho _E(G_{\mathbb {Q}})$ .
Concordant with his watershed papers [Reference MazurMaz77b, Reference MazurMaz78] classifying rational points on the modular curves $X_1(N)$ and $X_0(N)$ of prime level, Mazur proposed the following program (labeled ‘Program B’) in his article [Reference MazurMaz77a]:
‘Given a number field K and a subgroup H of $\operatorname {GL}_2({\widehat {\mathbb {Z}}}) \simeq \prod _p \operatorname {GL}_2 (\mathbb {Z}_p)$ , classify all elliptic curves $E/K$ whose associated Galois representation on torsion points maps $\operatorname {\mathrm {Gal}}({\overline {K}}/K)$ into $H \leq \operatorname {GL}_2 ({\widehat {\mathbb {Z}}})$ .’
Mazur’s Program B has seen substantial progress over the past decade: for prime level, see [Reference Bilu and ParentBP11, Reference Bilu, Parent and RebolledoBPR13, Reference ZywinaZyw15a, Reference SutherlandSut16, Reference Balakrishnan, Dogra, Müller, Tuitman and VonkBDM+19, Reference Balakrishnan, Dogra, Müller, Tuitman and VonkBDM+21, Reference Le Fourn and LemosLFL21]; for prime power level, see [Reference Rouse and ZureickBrownRZB15, Reference Sutherland and ZywinaSZ17]; for multiprime level, see [Reference ZywinaZyw15b, Reference Daniels and GonzálezJiménezDGJ, Reference GonzálezJiménez and LozanoRobledoGJLR16, Reference Brau and JonesBJ16, Reference MorrowMor19, Reference Jones and McMurdyJM22, Reference Daniels and LozanoRobledoDLR, Reference Daniels and MorrowDM, Reference Daniels, LozanoRobledo and MorrowDLRM21, Rak21, Reference Barbulescu and ShindeBS22]; and for CM curves, see [Reference Bourdon and ClarkBC20, Reference LozanoRobledoLR, Reference Campagna and PengoCP, Reference LombardoLom17].
1.1 Main theorem
Our main theorem addresses almost all of the remaining groups of prime power level. For each open subgroup H of $\operatorname {GL}_2({\widehat {\mathbb {Z}}})$ , we use $X_H$ to denote the modular curve whose noncuspidal Krational points classify elliptic curves $E/K$ for which $\rho _E(G_K)$ is contained in H (see Section 2.3 for a precise definition).
Remark 1.1. We write $\rho _{E,\ell ^{\infty }}(G_K){\,=\,} H$ to indicate that H is conjugate to the inverse image of $\rho _{E,\ell ^{\infty }}(G_K)$ under the projection $\operatorname {GL}_2({\widehat {\mathbb {Z}}})\to \operatorname {GL}_2(\mathbb {Z}_{\ell })$ . In general, we are only interested in the image of $\rho _E$ up to conjugacy (the exact image depends on a choice of basis).
Definition 1.2. Let $\ell $ be a prime, let H be an open subgroup of $\operatorname {GL}_2({\widehat {\mathbb {Z}}})$ of $\ell $ power level, and let K be a number field. We say that a point $P \in X_H(K)$ is exceptional if $X_H(K)$ is finite and P corresponds to a nonCM elliptic curve $E/K$ . We call an element of K exceptional if it is the jinvariant of an exceptional point. If there is a nonCM elliptic curve $E/K$ such that $j(E)$ is exceptional and $\rho _{E,\ell ^{\infty }}(G_K)=H$ (as opposed to a subgroup of H), then we refer to H as Kexceptional; we may omit K when $K = \mathbb {Q}$ .
For $K=\mathbb {Q}$ , the known exceptional jinvariants of prime power level are listed in Table 1 along with the known exceptional points on $X_H$ for exceptional subgroups H of prime power level that do not contain $I$ ; the groups of prime level appear in both [Reference SutherlandSut16] and [Reference ZywinaZyw15a].
Remark 1.3. For the H and j listed in Table 1, if $I \in H$ , then all but finitely many elliptic curves with $j(E) = j$ have $\ell $ adic image equal to H (rather than a proper subgroup). If $H' \subsetneq H$ is a proper subgroup such that $H = \langle H',I\rangle $ and $E/\mathbb {Q}$ is an elliptic curve with $\ell $ adic image H and $j(E) \ne 1728$ , then there are $[N_{\operatorname {GL}_{2}(\mathbb {Z}_{\ell })}(H) : N_{\operatorname {GL}_{2}(\mathbb {Z}_{\ell })}(H')]$ quadratic twists $E'$ of E whose $\ell $ adic image is $H'$ ; see Section 10.
Remark 1.4. If $P \in X_H(K)$ corresponds to an elliptic curve E, then $\rho _E(G_K)$ may be a proper subgroup of H. For example, any elliptic curve $E/\mathbb {Q}$ with jinvariant $11^2$ admits a rational 11isogeny, which forces $\rho _{E,11^{\infty }}(G_{\mathbb {Q}})$ to lie in the Borel group $B(11)$ , but $\rho _{E,11^{\infty }}(G_{\mathbb {Q}})$ is generically the index5 subgroup $\langle \left [\begin {smallmatrix} 10 & 1 \\ 0 & 10 \end {smallmatrix}\right ], \left [\begin {smallmatrix} 3 & 0 \\ 0 & 2 \end {smallmatrix}\right ] \rangle $ with label 11.60.1.4. In this example, $11^2$ is an exceptional jinvariant and 11.60.1.4 is an exceptional group, but $B(11)$ is not an exceptional group, because no nonCM elliptic curve over $\mathbb {Q}$ has $11$ adic image $B(11)$ . The group $B(11)={\texttt {11.12.1.1}}$ appears in Table 11 along with 13 other groups H of prime power level for which a similar phenomenon occurs.
Conjecture 1.5. Table 1 lists all rational exceptional jinvariants of prime power level.
Theorem 1.6. Let $\ell $ be a prime, let $E/\mathbb {Q}$ be an elliptic curve without complex multiplication, and let $H = \rho _{E,{\ell }^{\infty }}(G_{\mathbb {Q}})$ . Exactly one of the following is true:

(i) the modular curve $X_H$ is isomorphic to $\mathbb {P}^1$ or an elliptic curve of rank one, in which case H is identified in [Reference Sutherland and ZywinaSZ17, Corollary 1.6] and $\langle H,I\rangle $ is listed in [Reference Sutherland and ZywinaSZ17, Tables 1–4];

(ii) the modular curve $X_H$ has an exceptional rational point, and H is listed in Table 1;

(iii) $H \leq N_{\operatorname {\mathrm {ns}}}(3^3), N_{\operatorname {\mathrm {ns}}}(5^2), N_{\operatorname {\mathrm {ns}}}(7^2), N_{\operatorname {\mathrm {ns}}}(11^2)$ or $N_{\operatorname {\mathrm {ns}}}(\ell )$ for some $\ell \ge 19$ ;

(iv) H is a subgroup of one of the groups labeled $\mathrm {{\texttt {49.147.9.1}}}$ or $\mathrm {{\texttt {49.196.9.1}}}$ .
See Subsection 2.4 for a discussion of the group labels, and see Tables 2 and 3 for a summary of the groups H that appear in Theorem 1.6 for $\ell \leq 37$ .
We conjecture that in cases (iii) and (iv), the curve $X_H$ has no exceptional rational points; see Section 9 for a discussion of evidence for this conjecture. For the six groups in Table 2, the genus of $X_H$ is 9, 12, 14, 69 or 511, and for each isogeny factor A of $J_H$ , the analytic rank of A is equal to the dimension of A. We were not able to provably determine the rational points on these curves; see Section 9 for a more detailed discussion of these cases.
Remark 1.7. The recent groundbreaking work of [Reference Balakrishnan, Dogra, Müller, Tuitman and VonkBDM+19, Reference Balakrishnan, Dogra, Müller, Tuitman and VonkBDM+21] yields a complete classification of the possible $13$ adic and $17$ adic images for elliptic curves over $\mathbb {Q}$ ; these are the only primes other than $\ell = 2$ for which this is currently known.
1.2 Outline of proof
The proof of Theorem 1.6 breaks down into the following steps:

○ We begin by computing the collection ${\mathcal S}$ of arithmetically maximal subgroups of prime power level; see Section 3 and the tables in Appendix B.

○ For $H \in {\mathcal S}$ , we compute the isogeny decomposition of $J_H$ and the analytic ranks of its simple factors; see Sections 5 and 6.

○ For many $H \in {\mathcal S}$ , we compute equations for $X_H$ and $j_H\colon X_H \to X(1)$ ; see Section 7.

○ For most $H \in {\mathcal S}$ with $I \in H$ , we determine $X_H(\mathbb {Q})$ ; see Sections 4, 5, 8.

○ For $H \in {\mathcal S}$ with $I \not \in H$ , we compute equations for the universal curve $E \to U$ , where $U \subset X_H$ is the locus of points with $j \ne 0, 1728,\infty $ ; see Section 10.
Analyses of many of the relevant $X_H$ already exist in the literature, which is summarised in Section 4; the remaining cases are listed in Table 5. Some are routine; for example, some of the genus 2 curves in Section 8 have rank zero Jacobians and are handled with no additional work by Magma’s Chabauty0 command, but many require novel arguments.

○ We use the moduli enumeration of Section 5 to show that $X_H({\mathbb F}_p)$ and therefore $X_H(\mathbb {Q})$ is empty in several cases without needing equations for $X_H$ ; see Table 7.

○ In Section 8, a few curves admit an involution $\iota $ such that the map $P \mapsto P  \iota (P)$ projects to the torsion subgroup of the Jacobian, and we ‘sieve’ with this map. In one case, we are only able to compute an upper bound on $J_H(\mathbb {Q})_{{\operatorname {tors}}}$ but are then able to circumvent this via sieving.

○ In Section 7, we compute canonical models of two high genus curves (genus 36 and 43); and in Section 8.6, we use these models to set up a nontrivial local computation to show that $X_H(\mathbb {Q}_p)$ is empty, where p divides the level of H and is thus the only possible prime of bad reduction.

○ The curve $X_H$ from Section 8.7 has genus 41, and we were not able to compute a model; $X_H$ maps to $X_{\mathrm {ns}}^{+}(11)$ , which is a rank 1 elliptic curve, and we perform an ‘equationless’ Mordell–Weil sieve (via moduli) to show that $X_H(\mathbb {Q})$ is empty.

○ Throughout, our approaches are heavily informed by the isogeny decompositions and analytic ranks computed in Section 6. In the end, for a rigorous proof, we often do not need to know that the analytic and algebraic ranks agree, but some cases in Section 8 rely on this and thus make crucial use of Appendix A.

○ Appendix A contains an additional novelty: we make use of the Jacobian of a connected but geometrically disconnected curve.
1.3 Applications
In [Reference GreenbergGre12], Greenberg shows that if $\ell $ is a prime number and $E/\mathbb {Q}$ is an elliptic with a cyclic $\ell $ isogeny, then $\rho _{E,\ell ^{\infty }}(G_{\mathbb {Q}})$ is as large as possible given the existence of the isogeny if $\ell> 7$ , or the existence of $\ell $ power isogenies if $\ell = 5$ . The $\ell = 7$ case is handled in [Reference Greenberg, Rubin, Silverberg and StollGRSS14]. As a consequence of our work, we obtain a classification of the $3$ adic image for a nonCM elliptic curve $E/\mathbb {Q}$ with a rational $3$ isogeny.
Corollary 1.1. Suppose that $E/\mathbb {Q}$ is a nonCM elliptic curve with a rational $3$ isogeny, and let $H = \langle \rho _{E,3^{\infty }}(G_{\mathbb {Q}}), I \rangle $ . Then exactly one of the following is true:

(i) $[\operatorname {GL}_{2}(\mathbb {Z}_{3}) : H] = 4$ , and H has label $\mathrm {\texttt {3.4.0.1}}$ ;

(ii) $[\operatorname {GL}_{2}(\mathbb {Z}_{3}) : H] = 12$ , and H has label $\mathrm {\texttt {3.12.0.1}}$ , $\mathrm {\texttt {9.12.0.1}}$ , or $\mathrm {\texttt {9.12.0.2}}$ ;

(iii) $[\operatorname {GL}_{2}(\mathbb {Z}_{3}) : H] = 36$ , and H has label $\mathrm {\texttt {9.36.0.1}}$ , $\mathrm {\texttt {9.36.0.2}}, \mathrm {\texttt {9.36.0.3}}, \mathrm {\texttt {9.36.0.4}}, \mathrm {\texttt {9.36.0.5}}$ , $\mathrm {\texttt {9.36.0.6}}, \mathrm {\texttt {9.36.0.7}}, \mathrm {\texttt {9.36.0.8}}, \mathrm {\texttt {9.36.0.9}}$ , or $\mathrm {\texttt {27.36.0.1}}$ .
In each case, the relevant modular curve has genus zero and infinitely many rational points. Moreover, $\rho _{E,3^{\infty }}(G_{\mathbb {Q}})$ (which may have index two in H) contains all matrices congruent to the identity matrix modulo $27$ and all matrices $\left [\begin {smallmatrix} \lambda & 0 \\ 0 & \lambda \end {smallmatrix}\right ]$ with $\lambda \in \mathbb {Z}_{3}$ and $\lambda \equiv 1 \pmod {9}$ .
This result plays a role in the classification of odddegree isolated points on $X_{1}(N)$ given in [Reference Bourdon, Gill, Rouse and WatsonBGRW20]. In [Reference Lombardo and TrontoLT, Reference Lombardo and TrontoLT21], Lombardo and Tronto study the degree of the extension $\mathbb {Q}(E[N],N^{1} \alpha )/\mathbb {Q}(E[N])$ , where $\alpha \in E(\mathbb {Q})$ is a point of infinite order and the presence of scalars in $\rho _{E,\ell ^{\infty }}(G_{\mathbb {Q}})$ is used to bound certain Galois cohomology groups. Corollary 1.1 gives precise information about the scalars present in the $3$ adic image.
An additional application of Theorem 1.6 is a very fast ‘algebraic’ algorithm to compute all of the $\ell $ adic Galois images of a given nonCM elliptic curve $E/\mathbb {Q}$ ; this algorithm is described in Section 11. By exploiting the classification of $\ell $ adic Galois images of CM elliptic curves obtained by LozanoRobledo [Reference LozanoRobledoLR], we can extend this algorithm to also handle CM elliptic curves over $\mathbb {Q}$ (and, more generally, any CM elliptic curve over its minimal field of definition), as described in Section 12. An implementation of this algorithm is available in the GitHub repository associated to this paper [Reference Rouse, Sutherland and ZureickBrownRSZB21], and we have applied it to the elliptic curves $E/\mathbb {Q}$ in the Lfunctions and Modular Forms Database (LMFDB) [LMF] as well as two other large databases of elliptic curves over $\mathbb {Q}$ [Reference Stein and WatkinsSW02, Reference Balakrishnan, Ho, Kaplan, Spicer, Stein and WeigandtBHK+16]. This data has now been added to the LMFDB, where it extends existing $2$ adic and mod $\ell $ Galois image data that was previously available. As a result, the $\ell $ adic images $\rho _{E,\ell ^{\infty }}(G_{\mathbb {Q}})$ are now known for all elliptic curves $E/\mathbb {Q}$ of conductor up to $500000$ and every prime $\ell $ .
Other applications abound; examples include [Reference Jones and RouseJR10, Reference Cerchia and RouseCR21, Reference Lombardo and TrontoLT, Reference GužvićGuž21, Reference Bourdon, Gill, Rouse and WatsonBGRW20, Reference GonzálezJiménez and NajmanGJN20, Reference Bourdon, Ejder, Liu, Odumodu and VirayBEL+19, Reference GonzálezJiménez and LozanoRobledoGJLR18, Reference GonzálezJiménez and LozanoRobledoGJLR17, Reference Daniels, LozanoRobledo, Najman and SutherlandDLRNS18, Reference ReiterRei21, Reference Chiloyan and LozanoRobledoCLR21, Reference Barbulescu and ShindeBS22].
1.4 Code
We make extensive use of the computer algebra system Magma [Reference Bosma, Cannon and PlayoustBCP97]. Code verifying the computational claims made in this paper is available at the GitHub repository https://github.com/AndrewVSutherland/elladicgaloisimages, which also includes related algorithms and data.
2 The Modular curves $X_H$
2.1 Notation
In this article, $\ell $ and p denote primes, and N is a positive integer. We define
with the understanding that $\mathbb {Z}(1)$ is the zero ring and $\operatorname {GL}_2(1)$ and $\operatorname {SL}_2(1)$ are trivial groups. We then have ${\widehat {\mathbb {Z}}}=\varprojlim \mathbb {Z}(N)$ and $\operatorname {GL}_2({\widehat {\mathbb {Z}}})=\varprojlim \operatorname {GL}_2(N)$ and natural projection maps
We view $\mathbb {Z}(N)^2$ as a $\mathbb {Z}(N)$ module of column vectors equipped with a left action of $\operatorname {GL}_2(N)$ via matrixvector multiplication.
For $N>1$ , we use $B(N)$ to denote the Borel subgroup of $\operatorname {GL}_2(N)$ consisting of upper triangular matrices, $C_{\operatorname {\mathrm {sp}}}(N)$ to denote the split Cartan subgroup of diagonal matrices in $\operatorname {GL}_2(N)$ and $C_{\operatorname {\mathrm {ns}}}(N)$ to denote the nonsplit Cartan subgroup of $\operatorname {GL}_2(N)$ , which can be constructed by picking an imaginary quadratic order $\mathcal O$ in which every prime $\ell N$ is inert and embedding $(\mathcal O/N\mathcal O)^{\times }$ in $\operatorname {GL}_2(N)$ via its action on a $\mathbb {Z}(N)$ basis for $\mathcal O/N\mathcal O$ (see Section 12), which is unique up to conjugacy. We use $N_{\operatorname {\mathrm {sp}}}(N)$ to denote the subgroup of the normaliser of $C_{\operatorname {\mathrm {sp}}}(N)$ in $\operatorname {GL}_2(N)$ whose projection to $\operatorname {GL}_2(M)$ contains $C_{\operatorname {\mathrm {sp}}}(M)$ with index 2 for all $M>2$ dividing N, which is unique up to conjugacy and equal to the normaliser when $N>2$ is a prime power; we similarly define $N_{\operatorname {\mathrm {ns}}}(N)$ . If G is a subgroup of the projectivisation $\operatorname {PGL}_2(N)$ of $\operatorname {GL}_2(N)$ , then we denote by $G(N)$ the preimage of G in $\operatorname {GL}_2(N)$ .
2.2 Group theory and Galois representations
Let $N \ge 1$ be an integer, let k be a perfect field of characteristic coprime to N, and let
. A basis $(P_1,P_2)$ of $E[N]({\overline {k}})$ determines an isomorphism $E[N]({\overline {k}}) \overset \sim \rightarrow \mathbb {Z}(N)^2$ via the map $P_i \mapsto e_i$ . This gives rise to an isomorphism $\iota \colon \operatorname {\mathrm {Aut}} \left ( E[N]({\overline {k}})\right ) \simeq \operatorname {GL}_2(N)$ as follows: for $\phi \in \operatorname {\mathrm {Aut}} \left (E[N]({\overline {k}})\right )$ such that
we define
which yields a Galois representation $G_k \to \operatorname {GL}_2(N)$ given by $\sigma \mapsto \iota (\sigma _N)$ , where $\sigma _N$ is the automorphism of $E[N]({\overline {k}})$ induced by $\sigma $ .
Remark 2.1. Our choice of $\iota $ corresponds to the left action of $\operatorname {GL}_2(N)$ on $\mathbb {Z}(N)^2$ . Many sources are ambiguous about the choice of left vs. right action; this ambiguity often makes no difference (see [Reference Rouse and ZureickBrownRZB15, Remark 2.2]), but it does when $H\le \operatorname {GL}_2(N)$ is not conjugate to its transpose. Our choice here is consistent with [Reference SutherlandSut16, Reference Sutherland and ZywinaSZ17], but differs from [Reference Rouse and ZureickBrownRZB15], which uses right actions.
For each elliptic curve $E/k$ , we fix a system of compatible bases for $E[N]({\overline {k}})$ for all $N\ge 1$ coprime to the characteristic of k, meaning if $N=N_1N_2$ , then the bases for $E[N_1]({\overline {k}})$ and $E[N_2]({\overline {k}})$ are the images of the basis for $E[N]({\overline {k}})$ under multiplication by $N_2$ and $N_1$ , respectively. Let $\iota _E\colon \! \operatorname {\mathrm {Aut}}(E[N]({\overline {k}})) \overset \sim \longrightarrow \operatorname {GL}_2(N)$ denote the isomorphism determined by the basis for $E[N]({\overline {k}})$ , and define the Galois representation $\rho _{E,N}\colon G_k\to \operatorname {GL}_2(N)$ via . When k has characteristic zero, we shall also use $\iota _E$ to denote the isomorphism $\iota _E\colon \operatorname {\mathrm {Aut}}(E_{\mathrm {tor}}({\overline {k}})) \overset \sim \longrightarrow \operatorname {GL}_2({\widehat {\mathbb {Z}}})$ determined by this system of bases and use it to define the Galois representation $\rho _E\colon G_k\to \operatorname {GL}_2({\widehat {\mathbb {Z}}})$ via . If k is a number field and ${\mathfrak p}$ is a prime of good reduction for E, for each $N\ge 1$ coprime to $N({\mathfrak p})$ , we shall use the reduction of our chosen basis for $E[N]({\overline {k}})$ as a basis for $E_{\mathfrak p}[N](\overline {\mathbb F}_{\mathfrak p})$ .
For a subgroup $H \leq \operatorname {GL}_2({\widehat {\mathbb {Z}}})$ , we use $H(N)$ to denote the image $\pi _N(H)$ of H under $\pi _N$ . An open subgroup $H \leq \operatorname {GL}_2({\widehat {\mathbb {Z}}})$ contains the kernel of $\pi _N\colon \operatorname {GL}_2({\widehat {\mathbb {Z}}})\to \operatorname {GL}_2(N)$ for some $N\ge 1$ ; the least such N is the level of H, in which case $H = \pi _N^{1}(H(N))$ is completely determined by $\pi _N(H)$ . We will typically work directly with subgroups of $\operatorname {GL}_2(N)$ instead of $\operatorname {GL}_2({\widehat {\mathbb {Z}}})$ , with the understanding that every subgroup of $\operatorname {GL}_2(N)$ uniquely determines an open subgroup of $\operatorname {GL}_2({\widehat {\mathbb {Z}}})$ as its inverse image under $\pi _N$ ; we may use the symbol H to denote both H and $\pi _N(H)$ , which we note have the same index $[\operatorname {GL}_2({\widehat {\mathbb {Z}}}):H]=[\operatorname {GL}_2(N):\pi _N(H)]$ . We define the level of a subgroup $H \leq \operatorname {GL}_2(N)$ to be the level of $\pi _N^{1}(H)$ .
We may also identify open subgroups of $\operatorname {GL}_2(\mathbb {Z}_{\ell })$ with their preimages in $\operatorname {GL}_2({\widehat {\mathbb {Z}}})$ under the projection map $\operatorname {GL}_2({\widehat {\mathbb {Z}}})\to \operatorname {GL}_2(\mathbb {Z}_{\ell })$ . For any $e\ge 0$ , we may identify $H\le \operatorname {GL}_2(\ell ^e)$ with the corresponding open subgroups of $\operatorname {GL}_2(\mathbb {Z}_{\ell })$ or $\operatorname {GL}_2({\widehat {\mathbb {Z}}})$ of the same index. In what follows, we shall only be interested in subgroups of $\operatorname {GL}_2({\widehat {\mathbb {Z}}})$ up to conjugacy and view each subgroup H as a representative of its conjugacy class. All inclusions $H_1\le H_2$ of subgroups of $\operatorname {GL}_2({\widehat {\mathbb {Z}}})$ should be understood to indicate that $H_1$ is conjugate to a subgroup of $H_2$ .
2.3 Modular curves
Let H be a subgroup of $\operatorname {GL}_2(N)$ of level N. We define the modular curve $Y_H$ (respectively, $X_H$ ) to be the coarse moduli space of the stack $\mathcal {M}^0_H$ (respectively, $\mathcal {M}_H$ ), over $\operatorname {\mathrm {Spec}} \mathbb {Z}[1/N]$ , which parametrises elliptic curves (respectively, generalised elliptic curves) with Hlevel structure. Equivalently, by [Reference Deligne and RapoportDR73, IV3.1] (see also [Reference Rouse and ZureickBrownRZB15, Lemma 2.1]), $X_H$ is isomorphic to the coarse space of the stack quotient $X(N)/H$ , where $X(N)$ is the classical modular curve parametrising full level structures.
More precisely, an Hlevel structure on an elliptic curve $E/{\overline {k}}$ is an equivalence class $[\iota ]_H$ of isomorphisms $\iota \colon E[N]({\overline {k}})\to \mathbb {Z}(N)^2$ , where $\iota \sim \iota '$ if $\iota = h\circ \iota '$ for some $h\in H$ . The set $Y_H({\overline {k}})$ consists of equivalence classes of pairs $(E,[\iota ]_H)$ , where $(E,[\iota ]_H)\sim (E',[\iota ']_H)$ if there is an isomorphism $\phi \colon E\to E'$ for which the induced isomorphism $\phi _N\colon E[N]\to E'[N]$ satisfies $\iota \sim \iota '\circ \phi _N$ . Equivalently, $Y_H({\overline {k}})$ consists of pairs $(j(E),\alpha )$ , where $\alpha =HgA_E$ is a double coset in $H\backslash \operatorname {GL}_2(N)/A_E$ , where with .
Each $\sigma \in G_k$ induces an isomorphism $E^{\sigma }[N]\to E[N]$ defined by $P \mapsto \sigma ^{1}(P)$ , which we denote $\sigma ^{1}$ . We define a right $G_k$ action on $Y_H({\overline {k}})$ via $(E,[\iota ]_H)\mapsto (E^{\sigma },[\iota \circ \sigma ^{1}]_H)$ . The set of krational points $Y_H(k)$ consists of the elements in $Y_H({\overline {k}})$ that are fixed by this $G_k$ action. Each $P\in Y_H(k)$ is represented by a pair $(E,[\iota ]_H)\in Y_H(k)$ such that E is defined over k and for all $\sigma \in G_k$ there exist $\varphi \in \operatorname {\mathrm {Aut}}(E_{{\overline {k}}})$ and $h\in H$ such that
as shown in [Reference ZywinaZyw15b, Lemma 3.1].Footnote ^{1} Equivalently, a pair $(j(E),\alpha )$ with $\alpha =HgA_E$ lies in the set $Y_H(k)$ if and only if we have $j(E)\in k$ and $Hg\rho _{E,N}(\sigma ) A_E=HgA_E$ for all $\sigma \in G_k$ .
Remark 2.1. For an elliptic curve $E/k$ , if $\rho _{E,N}(G_k)\le H$ , then there exists an isomorphism $\iota \colon E[N]\overset \sim \rightarrow \mathbb {Z}(N)^2$ for which $(E, [\iota ]_H)\in Y_H(k)$ . Conversely, assuming $\textrm {char}(k)\ne 2,3$ , if $(E, [\iota ]_H)\in Y_H(k)$ , then for every twist $E'$ of E, there is an isomorphism $\iota '\colon E'[N]\overset \sim \rightarrow \mathbb {Z}(N)^2$ with $(E', [\iota ']_H)\in Y_H(k)$ , and for at least one $E'$ , we have $\rho _{E',N}(G_k)\le H$ , as we now show.
If $E'$ is a twist of E, then there is an isomorphism $\phi \colon E^{\prime }_{\bar k}\overset {\sim }{\to }E_{\bar k}$ that induces an isomorphism $\phi _N\colon E'[N]({\overline {k}})\overset {\sim }{\to }E[N]({\overline {k}})$ , and for the pairs $(E,[\iota ]_H)$ and $(E',[\iota ']_H)$ represent the same point in $Y_H({\overline {k}})$ ; thus if $(E,[\iota ]_H)$ lies in $Y_H(k)$ , then so does $(E',[\iota ']_H)$ .
If the point $(E,[\iota ]_H)=(j(E),\alpha )$ lies in $Y_H(k)$ , say for $\alpha =HgA_E$ , then $\rho _{E,N}(G_k)$ must lie in $g^{1}HgA_E$ . In other words, up to conjugacy in $\operatorname {GL}_2(N)$ , the subgroups $\rho _{E,N}(G_k)$ and H can differ only by elements of $A_E$ , which for $\textrm {char}(k)\ne 2,3$ is cyclic of order 2, 4, 6, the last two occurring only for $j(E)=0,1728$ . When $\#A_E=2$ , every twist of E is a quadratic twist, and [Reference SutherlandSut16, Lemma 5.24] implies that there is a quadratic twist $E'$ of E for which $\rho _{E',N}(G_k)\le H$ . Otherwise E has potential CM, and we may apply Proposition 12.1, which also addresses quartic and sextic twists. These results assume k is a number field, but the arguments apply whenever $A_E$ is cyclic (including $\textrm {char}(k)= 2,3$ provided $j(E)\ne 0,1728$ ).
Remark 2.2. The elliptic curves
with $j(E)=j(E') = 0$ are cubic twists that correspond to the same point in $X_0(2)(\mathbb {Q})$ , but E has a rational point of order 2 and $\rho _{E,2}(G_{\mathbb {Q}}) = B(2)$ , while $E'$ has no rational points of order 2 and $\rho _{E',2}(G_{\mathbb {Q}}) = \operatorname {GL}_2(2)$ . In particular, [Reference Rouse and ZureickBrownRZB15, Lemma 2.1] is incorrectly stated: it holds only up to twist.
The complement is the set of cusps of $X_H$ and corresponds to generalised elliptic curves with Hlevel structure. The set of krational cusps $X_H^{\infty }(k)$ can be alternatively described as follows. Let . We define a right $G_k$ action on $H\backslash \operatorname {GL}_2(N)/U(N)$ via $hgu\mapsto hg\chi _N(\sigma )u$ , where is defined by $\sigma (\zeta _N)=\zeta _N^e$ . By [Reference Deligne and RapoportDR73, IV5.3], $X_H^{\infty }(k)$ is in bijection with the subset of $H\backslash \operatorname {GL}_2(N)/U(N)$ fixed by $\chi _N(G_k)$ .
We define the genus $g(H)$ of H and $\pi _N^{1}(H)$ to be the genus of each of the geometric connected components of the modular curve $X_H$ , which can be directly computed from H via the usual formula
if we take , let $i(\Gamma _H)=[\operatorname {SL}_2(N):\Gamma _H]$ , let $\nu _2$ (respectively, $\nu _3$ ) count the right cosets of $\Gamma _H$ in $\operatorname {SL}_2(N)$ that contain a conjugate of $\left [\begin {smallmatrix} 0 & 1 \\ 1 & 0 \end {smallmatrix}\right ]$ (respectively, $\left [\begin {smallmatrix} 0 & 1 \\ 1 & 1 \end {smallmatrix}\right ]$ ) and let $\nu _{\infty }(\Gamma _H)$ count the orbits of $\Gamma _H\backslash \operatorname {SL}_2(N)$ under the right action of $\left [\begin {smallmatrix} 1 & 1 \\ 0 & 1 \end {smallmatrix}\right ]$ . The function GL2Genus in the file gl2.m in [Reference Rouse, Sutherland and ZureickBrownRSZB21] implements this computation.
We define the index $i(H)$ of H to be the integer $[\operatorname {GL}_2({\widehat {\mathbb {Z}}})\!:\!H]=[\operatorname {GL}_2(N)\!:\!H(N)]$ .
Remark 2.3. For $H\le \operatorname {GL}_2({\widehat {\mathbb {Z}}})$ of level N, the inverse image $\Gamma $ of $\Gamma _H\leq \operatorname {SL}_2(N)$ in $\operatorname {SL}_2(\mathbb {Z})$ is a congruence subgroup of some level $MN$ , but M may be strictly smaller than N; see [Reference Sutherland and ZywinaSZ17] for examples. In any case, the base change of $X_H$ to $\mathbb {Q}(\zeta _N)$ breaks into connected components that are geometrically connected and are each isomorphic over $\mathbb {C}$ to the modular curve $X_{\Gamma }$ obtained by taking the quotient of the extended upper half plane by $\Gamma $ .
Remark 2.4. For $H\le \operatorname {GL}_2(N)$ , if $\det H = \mathbb {Z}(N)^{\times }$ , then $X_H$ is geometrically connected. We mostly consider the case where $\det H = \mathbb {Z}(N)^{\times }$ in this paper, although in Appendix A, we make use of the case $H = \{I\}$ , where $X_H = X(N)$ . When $\det H < \mathbb {Z}(N)^{\times }$ is a proper subgroup, $X_H$ is still a curve defined over $\mathbb {Q}$ , but it is not geometrically connected; its connected components are defined over $\mathbb {Q}(\zeta _N)^H$ , indexed (via the Weil pairing on the level structures) by cosets of $\det H$ and isomorphic (over $\mathbb {Q}(\zeta _N)^H$ ) to $X_{\Gamma }$ . See Appendix A where this is worked out in more detail for $X(N)$ , and Example 6.3, where we discuss the isogeny decomposition of the ‘Jacobian’ of such an $X_H$ .
2.4 Subgroup labels
To help organise our work, we introduce the following labeling convention for uniquely identifying (conjugacy classes of) open subgroups H of $\operatorname {GL}_2({\widehat {\mathbb {Z}}})$ , which will also serve as labels of the corresponding modular curves $X_H$ . All the subgroups we shall consider satisfy $\det (H)={\widehat {\mathbb {Z}}}^{\times }$ and have labels of the form
where N, i, g, n are the decimal representations of integers $N,i,g,n$ defined as follows:

○ $N=N(H)$ is the level of H;

○ $i=i(H)$ is the index of H;

○ $g=g(H)$ is the genus of H;

○ $n\ge 1$ is an ordinal indicating the position of H among all subgroups of the same level, index and genus with $\det (H)={\widehat {\mathbb {Z}}}^{\times }$ under the ordering defined below.
To define the ordinal n, we must totally order the set $S(N,i,g)$ of open subgroups of $\operatorname {GL}_2({\widehat {\mathbb {Z}}})$ of level N, index i and genus g with $\det (H)={\widehat {\mathbb {Z}}}^{\times }$ , and we want this ordering to be efficiently computable. With this in mind, we define the following invariants for each $H\in S(N,i,g)$ :

○ the parent list of H is the lexicographically ordered list of labels of the subgroups $G\leq \operatorname {GL}_2({\widehat {\mathbb {Z}}})$ of which H is a maximal subgroup;

○ the orbit signature of H as the ordered list of triples $(e,s,m)$ , where m counts orbits of $\mathbb {Z}(N)^2$ under the leftaction of H of size s and exponent e (the least integer that kills every element of the orbit);Footnote ^{2}

○ the class signature of H is the ordered set of tuples $(o,d,t,s,m)$ , where m counts the conjugacy classes of elements of $H\le \operatorname {GL}_2(N)$ of size s whose elements have order o, determinant d and trace t;

○ the minimal conjugate of H is the least subgroup of $\operatorname {GL}_2(N)$ conjugate to H, where subgroups of $\operatorname {GL}_2(N)$ are ordered lexicographically as ordered lists of tuples of integers $(a,b,c,d)$ with $a,b,c,d\in [0,N1]$ corresponding to the matrix $\left [\begin {smallmatrix} a & b \\ c & d \end {smallmatrix}\right ]\in H$ .
We now totally order the set $S(N,i,g,d)$ lexicographically by parent list, orbit signature, class signature and minimal conjugate; these invariants are intentionally ordered according to the difficulty of computing them, and in most cases the first two or three suffice to distinguish every element of $S(N,i,g,d)$ , meaning we rarely need to compute minimal conjugates, which is by far the most expensive invariant to compute.
Example 2.1. Let $H=N_{\operatorname {\mathrm {ns}}}(11^2)$ be the normaliser of the nonsplit Cartan subgroup of $\mathbb {Z}(11^2)$ , which has level $11^2=121$ and index $5\cdot 11^3 = 6655$ . There are just two subgroups $H\le \operatorname {GL}_2(11^2)$ of index 6655 (up to conjugacy), only one of which has $\det (H)={\widehat {\mathbb {Z}}}^{\times }$ , so computing $g(H)=511$ suffices to determine the label 121.6655.511.1 of $N_{\operatorname {\mathrm {ns}}}(11^2)$ .
If we instead take $H=N_{\operatorname {\mathrm {ns}}}(11)$ with level 11 and index 55, we find there are two subgroups of $\operatorname {GL}_2(11)$ of index 55; they have genus 1 and determinant index 1 and thus belong to the set $S(11,55,1,1)$ . These subgroups and have parent list $(\texttt {1.1.0.1})$ and orbit signature $(1,1,1), (11,120,1)$ . Their class signatures begin $(1,1,2,1,1), (2,1,9,1,1), (2,10,0,12,1)$ but differ in the fourth tuple, which is $(3,1,10,2,1)$ for $H_1$ but $(3,1,10,8,1)$ for $H_2$ , meaning $H_1$ contains a conjugacy class of size 2 whose elements have order 3, determinant 1, and trace 10, but $H_2$ does not. It follows that the labels of $H_1$ and $H_2$ are 11.55.1.1 and 11.55.1.2, and that $N_{\operatorname {\mathrm {ns}}}(11)=H_1$ .
Remark 2.2. This labeling system can be extended to arbitrary open $H\le \operatorname {GL}_2({\widehat {\mathbb {Z}}})$ using labels of the form M.d.mN.i.g.n, where M.d.m uniquely identifies $\det (H)\le {\widehat {\mathbb {Z}}}^{\times }$ according to its level $MN$ , index $d:=[{\widehat {\mathbb {Z}}}^{\times }\!\!:\!\det (H)]$ and an ordinal m that distinguishes $\det (H)$ among the open subgroups $D\le {\widehat {\mathbb {Z}}}^{\times }$ of level M and index d.Footnote ^{3} To define m, we order the index d subgroups $D\le (\mathbb {Z}/M\mathbb {Z})^{\times }$ lexicographically according to the lists of labels $\texttt {M.a}$ of Conrey characters $\chi _M(a,\cdot )$ of modulus M whose kernels contain D; see [Reference Best, Bober, Booker, Costa, Cremona, Maarten, Lee, LowryDuda, Roe, Sutherland and VoightBBB+21, Section 3.2] for the definition of $\chi _M(a,\cdot )$ . We then redefine N and i to be the relative level and index of H as a subgroup of the preimage G of $\det (H)$ in $\operatorname {GL}_2({\widehat {\mathbb {Z}}})$ : this means N is the least positive integer for which $H=\pi _N^{1}(\pi _N(H))$ , where $\pi _N\colon G\to G(N)$ is the reduction map and $i=[G:H]$ . This convention ensures that if $E/K$ has Galois image $\rho _E(G_K)=H$ , the values of N, i, g in the label of $\rho _E(G_L)$ will be the same for every finite extension $L/K$ .
3 Arithmetically maximal subgroups
Definition 3.1. We define an open subgroup $H \le \operatorname {GL}_2({\widehat {\mathbb {Z}}})$ to be arithmetically maximal if

(i) $\det (H)={\widehat {\mathbb {Z}}}^{\times }$ ,

(ii) H contains an element conjugate to $\left [\begin {smallmatrix} 1 & 0 \\ 0 & 1 \end {smallmatrix}\right ]$ or $\left [\begin {smallmatrix} 1 & 1 \\ 0 & 1 \end {smallmatrix}\right ]$ ,

(iii) $j(X_H(\mathbb {Q}))$ is finite and $j(X_K(\mathbb {Q}))$ is infinite for all $H< K\le \operatorname {GL}_2({\widehat {\mathbb {Z}}})$ .
Properties of the Weil pairing imply that if $H = \rho _E(G_{\mathbb {Q}})$ , then $\det (H)= \mathbb {Z}^{\times }$ , and more generally, that if E is an elliptic curve over a number field K and $H=\rho _E(G_K)$ , then $[\mathbb {Z}^{\times }:\det (H)]=[K:K\cap \mathbb {Q}^{\mathrm {cyc}}]$ . In particular, condition (i) holds for every H that arises as $\rho _E(G_{\mathbb {Q}})$ for an elliptic curve $E/\mathbb {Q}$ . Condition (ii) is a necessary and sufficient condition for $X_{H}$ to have noncuspidal real points, by Proposition 3.5 of [Reference ZywinaZyw15b], and it follows from Proposition 3.1 of [Reference Sutherland and ZywinaSZ17] that every genus zero H of prime power level $N=\ell ^e$ that satisfies equations (i) and (ii) has infinitely many $\mathbb {Q}$ points (the assumption $I\in H$ is not used in the proof of Proposition 3.1 in [Reference Sutherland and ZywinaSZ17]). For odd $\ell $ , the matrices in condition (ii) are conjugate in $\operatorname {GL}_2(N)$ , and only the first is needed. This definition differs slightly from that used in [Reference Rouse and ZureickBrownRZB15, Definition 3.1] for $\ell =2$ .
Let ${\mathcal S}_{\ell }(\mathbb {Q})$ be the set of arithmetically maximal subgroups of $\ell $ power level, and let ${\mathcal S}_{\ell }^{\infty }(\mathbb {Q})$ be the set of open subgroups of $\operatorname {GL}_2({\widehat {\mathbb {Z}}})$ of $\ell $ power level for which $j(X_H(\mathbb {Q}))$ is infinite.
The sets ${\mathcal S}_{\ell }^{\infty }(\mathbb {Q})$ are explicitly determined in [Reference Sutherland and ZywinaSZ17] (and in [Reference Rouse and ZureickBrownRZB15] for $\ell =2$ ) and have cardinalities $1208,47,23,15,2,11$ for $\ell =2,3,5,7,11,13$ and cardinality $1$ for all $\ell>13$ .Footnote ^{4} To enumerate the set ${\mathcal S}_{\ell }(\mathbb {Q})$ , it would suffice to enumerate the maximal subgroups of all the groups in ${\mathcal S}_{\ell }^{\infty }(\mathbb {Q})$ , but we take a slightly different approach and use our knowledge of ${\mathcal S}_{\ell }^{\infty }(\mathbb {Q})$ to derive an upper bound on the level of the groups in ${\mathcal S}_{\ell }(\mathbb {Q})$ .
Lemma 3.2. Let for $\ell =2,3,5,7,11,13$ , respectively, and let for all primes $\ell>13$ . Then $\ell ^{e_{\ell }}$ is an upper bound on the level of every $H\in {\mathcal S}_{\ell }^{\infty }(\mathbb {Q})$ and $\ell ^{e_{\ell }+1}$ is an upper bound on the level of every $H\in {\mathcal S}_{\ell }(\mathbb {Q})$ .
Proof. The bound for $H\in {\mathcal S}_{\ell }^{\infty }(\mathbb {Q})$ follows from [Reference Sutherland and ZywinaSZ17, Corollary 1.6] and its associated data. The bound for $H\in {\mathcal S}_{\ell }(\mathbb {Q})$ follows from [Reference Sutherland and ZywinaSZ17, Lemma 3.7] for $\ell \le 13$ (note $e_2\ge 2$ ); and for $\ell>13$ , we can apply [Reference SerreSer98, Lemma 3, pages IV23], since $e_{\ell }=0$ .
Lemma 3.2 reduces the problem of enumerating the sets ${\mathcal S}_{\ell }(\mathbb {Q})$ to a finite computation in $\operatorname {GL}_2(\ell ^{e_{\ell }+1})$ . This computation can be further constrained by computing the maximal index $i_{\ell }$ of any maximal subgroup of $H_0\in {\mathcal S}_{\ell }^{\infty }(\mathbb {Q})$ and then enumerating all (conjugacy classes of) subgroups of $\operatorname {GL}_2(\ell ^{e_{\ell }+1})$ of index at most $i_{\ell }$ , which can be efficiently accomplished using algorithms for subgroup enumeration that are implemented in Magma and other computer algebra systems. The advantages of this approach are that it enumerates subgroups up to conjugacy in $\operatorname {GL}_2({\widehat {\mathbb {Z}}})$ , rather than up to conjugacy in $H_0$ , and it includes all subgroups that have the same level and index of any $H\in {\mathcal S}_{\ell }(\mathbb {Q})$ , as well as all of the groups that contain them, which may be needed to compute the group labels defined in the previous section.
Table 4 summarises this computation. The ‘subgroups’ row counts the number of subgroups $H\le \operatorname {GL}_2({\widehat {\mathbb {Z}}})$ of $\ell $ power level with $\det (H)={\widehat {\mathbb {Z}}}^{\times }$ that satisfy the level and index bounds in the first two rows, and the last three rows list the maximum level, index and genus that arise among the groups $H\in {\mathcal S}_{\ell }(\mathbb {Q})$ .
4 Bookkeeping
By [Reference MazurMaz78], [Reference SerreSer81, Section 8.4] and [Reference Bilu and ParentBP11], if $\ell \geq 17$ and $\rho _{E,\ell }$ is not surjective, then either $\ell = 17$ or $37$ and $j(E)$ is listed in Table 1, or the image $\rho _{E,\ell }(G_{\mathbb {Q}})$ is a subgroup of $N_{\operatorname {\mathrm {ns}}}(\ell )$ . In the latter case, for $\ell \equiv 1\pmod {3}$ , the image must be equal to $N_{\operatorname {\mathrm {ns}}}(\ell )$ (not a proper subgroup); and for $\ell \equiv 2\pmod {3}$ , the image is either $N_{\operatorname {\mathrm {ns}}}(\ell )$ or the index3 subgroup
by [Reference ZywinaZyw15a, Proposition 1.13]. Recent work by le Fourn and Lemos [Reference Le Fourn and LemosLFL21, Theorem 1.2] shows that if $\ell> 1.4\times 10^7$ , then $\rho _{E,\ell }(G_{\mathbb {Q}})$ cannot be a proper subgroup of $N_{\operatorname {\mathrm {ns}}}(\ell )$ .
Numerous individual cases remain. See Tables 12–17 in Appendix B for a list of all arithmetically maximal $H\le \operatorname {GL}_2({\widehat {\mathbb {Z}}})$ of $\ell $ power level for $\ell \leq 37$ , and see Table 4 for a numerical summary of this data. Many of the corresponding modular curves $X_H$ are already handled in the literature, and these account for most of the exceptional points listed in Table 1. Below is a summary of prior results.

1. Ligozat determined the rational points on $X_0(N)$ for $N = 27,49,11,17,19$ with labels 27.36.1.1, 49.56.1.1, 11.12.1.1, 17.18.1.1, 19.20.1.1 in [Reference LigozatLig75, 5.2.3.1]; there are exceptional points for $N = 11,17$ .

2. Ligozat addressed $X_{S_4}(11)$ with label 11.55.1.2 in [Reference LigozatLig77, Proposition 4.4.8.1].

3. Kenku determined the rational points on $X_0(13^2)$ (labeled 169.182.8.1) [Reference KenkuKen80] and on $X_0(125)$ (labeled 125.150.8.1) [Reference KenkuKen81]; the modular curve $X_0(5,25)$ with label 25.150.8.1 is isomorphic to $X_0(125)$ and thus also handled by Kenku.

4. The curves $X_{\operatorname {\mathrm {sp}}}^+(25)$ and $X_{\operatorname {\mathrm {sp}}}^+(49)$ have labels 25.375.22.1 and 49.1372.94.1; they are isomorphic to $X_0^+(25^2)$ and $X_0^+(49^2)$ and are thus handled by Momose and Shimura [Reference Momose and ShimuraMS02, Theorems 0.1 and 3.14] and by Momose [Reference MomoseMom86, Theorem 3.6].

5. Momose also handled $X_{\operatorname {\mathrm {sp}}}^+(11)$ with label 11.66.2.1 in [Reference MomoseMom84, Theorem 0.1].

6. The curves $X_{\operatorname {\mathrm {ns}}}^+(13)$ and $X_{\operatorname {\mathrm {sp}}}^+(13)$ labeled 13.78.3.1 and 13.91.3.1 are handled via nonabelian Chabauty by Balakrishnan et al. in [Reference Balakrishnan, Dogra, Müller, Tuitman and VonkBDM+19].

7. The curves $X_{S_4}(13)$ and $X_{\operatorname {\mathrm {ns}}}^+(17)$ labeled 13.91.3.2 and 17.136.6.1 are handled via nonabelian Chabauty by Balakrishnan et al. in [Reference Balakrishnan, Dogra, Müller, Tuitman and VonkBDM+21].

8. Rouse and ZureickBrown addressed all $X_H$ of $2$ power level in [Reference Rouse and ZureickBrownRZB15].

9. The group with label 7.42.1.1 is the index 2 subgroup of $N_{\operatorname {\mathrm {sp}}}(7)$ containing the elements of $C_{\operatorname {\mathrm {sp}}}(7)$ with square determinant and the elements of $N_{\operatorname {\mathrm {sp}}}(7)  C_{\operatorname {\mathrm {sp}}}(7)$ with nonsquare determinant, handled by [Reference SutherlandSut12a, Section 3] and [Reference ZywinaZyw15a, Remark 4.3]. There is one exceptional jinvariant, which is interesting because elliptic curves with this mod 7 image admit a rational 7isogeny locally everywhere but not globally; this is the unique example of this phenomenon over $\mathbb {Q}$ for any prime $\ell $ .

10. Zywina addressed all remaining curves of prime level $\ell \leq 37$ except $N_{\operatorname {\mathrm {ns}}}(\ell )$ in [Reference ZywinaZyw15a].

11. Elkies showed that the curve 9.81.1.1 has no rational points in [Reference ElkiesElk06, p. 5].

12. Noncuspidal points on the modular curves labeled 49.168.12.1 and 49.168.12.2 are ‘ $7$ exceptional’ in the sense of [Reference Greenberg, Rubin, Silverberg and StollGRSS14, Definition 1.1].Footnote ^{5} By [Reference Greenberg, Rubin, Silverberg and StollGRSS14, page 3], the only $7$ exceptional elliptic curves are CM curves with $j = 15^3$ and $j = 255^3$ , which give rise to two points on the curve with label 49.168.12.2.

13. The groups with labels 25.150.4.1, 25.150.4.2, 25.150.4.5 and 25.150.4.6 correspond to elliptic curves $E/\mathbb {Q}$ that admit a rational $5$ isogeny with $5$ adic image of index divisible by $25$ ; their existence is ruled out by [Reference GreenbergGre12, Theorem 2].

14. The genus 2 rank 2 curves 25.50.2.1 and 25.75.2.1 are handled by Balakrishnan et al. in [Reference Balakrishnan, Dogra, Müller, Tuitman and VonkBDM+21, 4.3]. Each has a single exceptional jinvariant, and the minimal conductors for the two jinvariants are 396900 and 21175, respectively.
Of the 210 arithmetically maximal subgroups in Table 3, only 32 are not addressed above. Five are the groups $N_{\mathrm {ns}}(\ell )$ for $19\le \ell \le 37$ ; the remaining 27 are listed in Table 5.
5 Counting ${\mathbb F}_q$ points on $X_H$
For any prime power $q=p^e$ coprime to N and $H\le \operatorname {GL}_2(N)$ , we may consider the modular curve $X_H$ over the finite field ${\mathbb F}_q$ . We can count ${\mathbb F}_q$ rational points on $X_H$ via
by counting fixed points of a right $G_{{\mathbb F}_q}$ action on certain double coset spaces of $\operatorname {GL}_2(N)$ defined in Subsection 2.3. In this section, we explain how to do this explicitly and efficiently via a refinement of the method proposed in [Reference ZywinaZyw15b, Section 3]. Applications include:

○ checking for padic obstructions to rational points (by checking if $\#X_H({\mathbb F}_p)=0$ );

○ computing the zeta function of $X_H/{\mathbb F}_p$ (by computing $\#X_H({\mathbb F}_{p^r})$ for $1\le r\le g(H)$ );

○ determining the isogeny decomposition of $\operatorname {\mathrm {Jac}}(X_H)$ ;

○ determining the analytic rank of $\operatorname {\mathrm {Jac}}(X_H)$ .
The last two are described in Section 6. None require an explicit model for $X_H$ .
5.1 Counting points
If we put , we can use the permutation representation provided by the right $G_{{\mathbb F}_q}$ action on the coset space $[\overline {H}\backslash \operatorname {GL}_2(N)]$ to compute
as the number of $\left [\begin {smallmatrix} 1 & 1 \\ 0 & 1 \end {smallmatrix}\right ]$ orbits of $[\overline {H}\backslash \operatorname {GL}_2(N)]$ stable under the action of $\left [\begin {smallmatrix} q & 0 \\ 0 & 1 \end {smallmatrix}\right ]$ , which we note depends only on $q\bmod N$ ; if one expects to compute $\#X_H^{\infty }({\mathbb F}_q)$ for many finite fields ${\mathbb F}_q$ (as when computing the Lfunction of $X_H/\mathbb {Q}$ , for example), one can simply precompute a table of rational cusp counts indexed by $(\mathbb {Z}/N\mathbb {Z})^{\times }$ .
To compute
we first note that , so we may replace H with $\overline {H}$ and work with the quotient $A_E/\pm I$ , which is trivial for $j(E)\ne 0,1728$ . It then suffices to count elements of $[\overline {H}\backslash \operatorname {GL}_2(N)]$ fixed by $G_{{\mathbb F}_q}$ using the action of the Frobenius endomorphism $\pi $ on $E[N]$ with respect to some basis; we are free to choose any basis we like, since the number of elements of $[\overline {H}\backslash \operatorname {GL}_2(N)]$ fixed by a matrix in $\operatorname {GL}_2(N)$ is invariant under conjugation. By [Reference Duke and TóthDT02, Theorem 2.1], we can use the reduction modulo N of the integer matrix
where $a,b,\Delta ,d\in \mathbb {Z}$ are defined as follows. Let , let , let , and let if $\mathbb {Z}[\pi ]\ne \mathbb {Z}$ , and let otherwise, so that we always have
and let . We then have $\operatorname {\mathrm {tr}} A_{\pi }\equiv \operatorname {\mathrm {tr}} \pi \bmod N$ and $\det A_{\pi }\equiv q\bmod N$ . We note that $A_{\pi }$ is uniquely determined by $\operatorname {\mathrm {tr}}\pi $ and $\Delta $ and represents an element of $\operatorname {GL}_2(N)$ for every integer N coprime to q.
Let $\chi _{\overline {H}}\colon \operatorname {GL}_2(N)\to \mathbb {Z}_{\ge 0}$ denote the character of the permutation representation given by the right $\operatorname {GL}_2(N)$ set $[\overline {H}\backslash \operatorname {GL}_2(N)]$ . Each term in equation (5.1) for $j(E)\ne 0,1728$ can be computed as $\chi _{\overline {H}}(A_{\pi })$ . It follows from the theory of complex multiplication that for each imaginary quadratic discriminant $\Delta $ for which the norm equation (5.3) has a solution with $a>0$ coprime to q, there are exactly $h(\Delta )$ ordinary jinvariants of elliptic curves over ${\mathbb F}_q$ with $\operatorname {\mathrm {disc}}(R_{\pi })=\Delta $ , where $h(\Delta )$ is the class number; see [Reference SutherlandSut11, Proposition 1], for example. We thus compute the number of ${\mathbb F}_q$ points on $X_H$ corresponding to ordinary $j(E)\ne 0,1728$ via
To count supersingular points in $X_H({\mathbb F}_q)$ with $j\ne 0,1728$ , we rely on the following lemma.
Lemma 5.1. Let $q=p^e$ be a prime power, let $H\le \operatorname {GL}_2(N)$ with $\gcd (q,N)=1$ , let $h'=\lfloor h(4p)/2\rfloor $ , and let $s_0=1$ for $p\equiv 2\bmod 3$ and $s_0=0$ otherwise. The number $\#X_H^{\mathrm {ss'}}({\mathbb F}_q)$ of ${\mathbb F}_q$ points on $X_H$ corresponding to supersingular $j(E)\ne 0,1728$ can be computed as follows.

○ If $p\le 3$ , then $\#X_H^{\mathrm {ss'}}({\mathbb F}_q)=0$ .

○ If e is odd and $p\equiv 1\bmod 4$ , then
$$\begin{align*}\#X_H^{\mathrm{ss'}}({\mathbb F}_q) = (h's_0)\chi_{\overline{H}}\bigl(A(0,p^{(e1)/2},4p)\bigr). \end{align*}$$ 
○ If e is odd and $p\equiv 3\bmod 4$ , then
$$ \begin{align*} \#X_H^{\mathrm{ss'}}({\mathbb F}_q) = h'\chi_{\overline{H}}\bigl(A(0,2p^{(e1)/2},p)\bigr) + (h's_0)\chi_{\overline{H}}\bigl(A(0,p^{(e1)/2},4p)\bigr). \end{align*} $$ 
○ If e is even then
$$\begin{align*}\#X_H^{\mathrm{ss'}}({\mathbb F}_q) = \Biggl(\frac{p6+2\left(\frac{3}{p}\right)+3\left(\frac{4}{p}\right)}{12}\Biggr) \chi_{\overline{H}}\bigl(A(2p^{e/2},0,1)\bigr). \end{align*}$$
Proof. These formulas are derived from [Reference SchoofSch87, Theorem 4.6] by discarding ${\mathbb F}_q$ isomorphism classes with $j(E)=0,1728$ and dividing by 2 to count jinvariants rather than counting ${\mathbb F}_q$ isomorphism classes. For $p=2,3$ , there are no supersingular $j(E)\ne 0,1728$ , and for $p>3$ and $j(E)\ne 0,1728$ , we only need to consider supersingular ${\mathbb F}_q$ isomorphism classes with trace $a=0$ and $\Delta =0,4p$ when e is odd, and with trace $a=2p^{e/2}$ and $\Delta =1$ when e is even. In each case, the number of supersingular ${\mathbb F}_q$ isomorphism classes with jinvariant $0,1728$ that we need to discard can be determined using Table 6 below.
For $j(E)=0,1728$ , rather than computing double cosets fixed by $G_k$ , we instead compute a weighted sum of $\chi _{\overline {H}}(A_{\pi })$ over kisomorphism classes of elliptic curves with $j(E)=0,1728$ via the table below. If we extend the permutation character $\chi _{\overline {H}}$ to the group ring $\mathbb {Q}[\operatorname {GL}_2(N)]$ , we can compute the number of krational points of $X_H$ above $j=0,1728\in X(1)$ via
where the sums are over ${\mathbb F}_q$ isomorphism classes of elliptic curves $E/{\mathbb F}_q$ , and the values of $a,b,\Delta $ are listed in Table 6. These values are derived from [Reference WaterhouseWat69] and [Reference SchoofSch87], and we note that for each triple $(a,b,\Delta )$ with $a>0$ listed in the table with multiplicity m, the triples $(a,b,\Delta )$ and $(a,b,\Delta )$ each occur with multiplicity $m/2$ .
To sum up, for any $H\le \operatorname {GL}_2(N)$ , we can compute $\#X_H({\mathbb F}_q)$ as
where each term in the sum is computed using the permutation representation $[\overline {H}\backslash \operatorname {GL}_2(N)]$ : the term $\#X_H^{\infty }({\mathbb F}_q)$ is computed by counting $\left [\begin {smallmatrix} q & 0 \\ 0 & q \end {smallmatrix}\right ]$ stable $\left [\begin {smallmatrix} 1 & 1 \\ 0 & 1 \end {smallmatrix}\right ]$ orbits, and the remaining terms are weighted sums of values of the permutation character $\chi _{\overline {H}}$ on the matrices $A(a,b,\Delta )$ arising for elliptic curves $E/{\mathbb F}_q$ . A Magma implementation of the formula in equation (5.5) is provided by the function GL2PointCount in the file gl2.m in [Reference Rouse, Sutherland and ZureickBrownRSZB21].
Example 5.2. Consider $H=\{\left [\begin {smallmatrix} 1 & * \\ 0 & * \end {smallmatrix}\right ]\}\le \operatorname {GL}_2(13)$ with label 13.168.2.1. The modular curve $X_H=X_1(13)$ parametrises elliptic curves with a rational point of order 13. Over ${\mathbb F}_{37}$ , there are four such elliptic curves, up to ${\mathbb F}_{37}$ isomorphism, each with 12 points of order 13 with distinct jinvariants $0$ , $16$ , $26=1728$ , $35$ (none are supersingular) and automorphism groups of order 6, 2, 4, 2, respectively. We thus expect $\#Y_H({\mathbb F}_{37}) = 12/6 + 12/2 + 12/4 + 12/2 = 17$ , and one can check that 6 of the 12 cusps of $X_1(13)$ are ${\mathbb F}_{37}$ rational (those of width of 13). This agrees with the fact that there are 23 rational points on the mod37 reduction of the genus 2 curve $y^2+(x^3+x+1)y=x^5+x^4$ , which is a $\mathbb {Z}[1/13]$ model for $X_H=X_1(13)$ .
To compute $\#X_H({\mathbb F}_{37})$ using the algorithm proposed above, we apply equation (5.5), which does not require us to enumerate elliptic curves with a point of order 13. We compute $\#X_H^{\infty }({\mathbb F}_{37})=6$ by noting that the coset space $[\overline {H}\backslash \operatorname {GL}_2(13)]$ has twelve $\left [\begin {smallmatrix} 1 & 1 \\ 0 & 1 \end {smallmatrix}\right ]$ orbits, six of size 1 and six of size 13, the latter of which are stable under the action of $\left [\begin {smallmatrix} 37 & 0 \\ 0 & 1 \end {smallmatrix}\right ]\equiv \left [\begin {smallmatrix} 11 & 0 \\ 0 & 1 \end {smallmatrix}\right ]$ . Enumerating integers $a\in [1,\lfloor 2\sqrt {37}\rfloor ]$ yields 17 triples $(a,b,\Delta )$ with $\Delta < 4$ that satisfy the norm equation (5.3). Among these, only $A(1,1,147)$ fixes an element of $[\overline {H}\backslash \operatorname {GL}_2(13)]$ , and from equation (5.4), we obtain
To compute $\#X_H^{\mathrm {ss'}}({\mathbb F}_{37})$ , we apply Lemma 5.1 and find that $\chi _{\overline {H}}(A(0,0,4\cdot 37))=0$ , so $\#X_H^{\mathrm {ss'}}({\mathbb F}_q) =0$ . Using Table 6, we compute
and equation (5.5) then yields
In contrast to the method described in [Reference ZywinaZyw15b, Section 3], in our computation above, we did not enumerate elliptic curves or jinvariants over ${\mathbb F}_q$ , nor did we compute any Hilbert class polynomials (a commonly used but inefficient way to compute $A_{\pi }$ ). Instead, we enumerated possible Frobenius traces $a>0$ , solved norm equations, computed class numbers and computed values of the permutation character $\chi _{\overline {H}}$ . This can be viewed as an algorithmic generalisation of the Eichler–Selberg trace formula that works for any open $H\le \operatorname {GL}_2({\widehat {\mathbb {Z}}})$ . It has roughly the same asymptotic complexity as the trace formula (the dependence on q is $\tilde O(q^{1/2})$ if we ignore the cost of computing class numbers, which in practice we simply look up in a table such as those available for $D\le 2^{40}$ in the LMFDB). But the constant factors are better, and even for groups that do not contain the $\operatorname {GL}_2$ analog of $\Gamma _1(N)$ , we are always able to work at level N rather than level $N^2$ , as typically required by generalisations of the trace formula to handle congruence subgroups that do not contain $\Gamma _1(N)$ .
5.2 Local obstructions
An easy application of this point counting algorithm is checking for local obstructions to rational points on the arithmetically maximal modular curves $X_H$ . For $X_H$ of genus g, it suffices to compute $\#X({\mathbb F}_p)$ for $p < \lfloor 2g\sqrt {p}\rfloor $ , since the Weil bounds force $\#X({\mathbb F}_p)>0$ for all larger primes p. Doing this for the 210 arithmetically maximal groups summarised in Table 4 yields Theorem 5.1, whose results are summarised in Table 7.
Theorem 5.1. For $\ell \le 37$ , the arithmetically maximal $H\le \operatorname {GL}_2({\widehat {\mathbb {Z}}})$ of $\ell $ power level for which $X_H$ has no ${\mathbb F}_{p}$ points for some prime $p\ne \ell $ are precisely those listed in Table 7.