Sketch of the proof of the tail estimate for odd n

Our proof of Theorem 5(b) makes use of the representation of $G=\mathrm {{SL}}_n$ on the space $W=2\otimes \mathrm {{Sym}}_2(n)$ of pairs $(A,B)$ of symmetric $n\times n$ matrices, studied in detail in [Reference Wood39, Reference Bhargava5, Reference Bhargava, Gross and Wang7, Reference Bhargava, Gross and Wang8]. The group G acts on W via $\gamma \cdot (A,B)=(\gamma A\gamma ^t,\gamma B\gamma ^t$ ) for $\gamma \in G$ and $(A,B)\in W$ . We define the invariant binary form of an element $(A,B)\in W$ by

$$ \begin{align*}\smash{f_{A,B}(x,y) = (-1)^{n(n-1)/2}\det(Ax - By).}\end{align*} $$

Then $f_{A,B}$ is a binary n-ic form satisfying $f_{\gamma (A,B)}=f_{A,B}$ . Moreover, the ring of polynomial invariants for the action of G on W is freely generated by the coefficients of the invariant binary form. Define the discriminant $\Delta (A,B)$ and height $H(A,B)$ of an element $(A,B)\in W$ by $\Delta (A,B)=\Delta (f_{A,B})$ and $H(A,B)=H(f_{A,B})$ .

The first step of our proof is the construction, for every squarefree integer $m>0$ , of a map

$$ \begin{align*} \sigma_m:{\mathcal{W}}_m^{\mathrm{{{(2)}}}}\to W({\mathbb{Z}}), \end{align*} $$

such that $f_{\sigma _m(f)}(x,y)=f(x,y)$ for every $f\in {\mathcal {W}}_m^{\mathrm {{{(2)}}}}$ . In our construction, the image of $\sigma _m$ , in fact, lies in $W_0({\mathbb {Z}})$ , where $W_0$ is the subspace of W consisting of pairs of matrices whose top left $g\times g$ blocks are $0$ , where $n=2g+1$ . The action of the group G does not preserve $W_0$ , and we take $G_0$ to be the maximal parabolic subgroup of G that does preserve $W_0$ . When the discriminant polynomial $\Delta \in {\mathbb {Z}}[W]$ is restricted to $W_0$ , it is no longer irreducible but rather is divisible by the square of a polynomial $Q\in {\mathbb {Z}}[W_0]$ . This polynomial Q is a relative invariant for the action of $G_0$ on $W_0$ . Its significance is that, by construction of $\sigma _m$ , every element in the image of $\sigma _m$ has Q-invariant equal to m. To prove Part (b) of Theorem 5, it therefore suffices to estimate the number of $G_0({\mathbb {Z}})$ -orbits on $W_0({\mathbb {Z}})$ having height less than X and Q-invariant greater than M.

Bounding the number of these orbits is complicated by the fact that $G_0$ is not reductive. We are rescued by using the full action of $G({\mathbb {Z}})$ on $W({\mathbb {Z}})$ . This necessitates expanding the definition of the Q-invariant from $W_0({\mathbb {Z}})$ to all ‘distinguished’ elements of $W({\mathbb {Z}})$ . An element $(A,B)\in W({\mathbb {Z}})$ is distinguished if A and B have a common isotropic g-dimensional subspace defined over ${\mathbb {Q}}$ . Thus, every element in $W_0({\mathbb {Z}})$ (and thus every element in the image of $\sigma _m$ ) is distinguished. The Q-invariant, though defined initially on $W_0$ , can be extended as a function on the set of all triples $(A,B,\Lambda )$ , where $(A,B)\in W({\mathbb {Z}})$ is distinguished, and $\Lambda $ is a common isotropic subspace of A and B. For all but a negligible number of distinguished elements $(A,B)\in W({\mathbb {Z}})$ , A and B have exactly one common isotropic subspace $\Lambda $ defined over ${\mathbb {Q}}$ . Thus, we may define a $G({\mathbb {Z}})$ -invariant function Q on the set of distinguished pairs $(A,B)\in W({\mathbb {Z}})$ outside a negligible number of them. It then suffices to bound the number of $G({\mathbb {Z}})$ -orbits on distinguished elements in $W({\mathbb {Z}})$ having bounded height and large Q-invariant.

To obtain such a bound, we construct fundamental domains for the action of $G({\mathbb {Z}})$ on elements in $W({\mathbb {R}})$ with height less than X. Such a fundamental domain has a natural partition into three parts that we term the main body, the shallow cusp and the deep cusp. We have little control over the Q-invariants of elements in the main body and the shallow cusp. However, it is known [Reference Ho, Shankar and Varma20, Proposition 4.3] that there are a negligible number of integral elements in the shallow cusp. Meanwhile, distinguished elements occur rarely in the main body, a fact we prove via the large sieve.

Finally, the deep cusp lies in $W_0$ , where an upper bound for the Q-invariant can be obtained. Imposing the condition that this upper bound is greater than M, and counting the number of such points in the deep cusp using the averaging method of [Reference Bhargava3], gives the desired saving for the number of elements in the deep cusp having Q-invariant larger than M. Combining the estimates for the main body, the shallow cusp and the deep cusp yields Part (b) of Theorem 5.

Sketch of the proof of the tail estimate for even n

With W again denoting the space of pairs of symmetric $n\times n$ matrices, we may attempt to proceed in the same manner as in the case of odd n, by constructing a map

$$ \begin{align*} \smash{\sigma_m:{\mathcal{W}}_m^{\mathrm{{{(2)}}}}\to W({\mathbb{Z}})} \end{align*} $$

such that $f_{\sigma _m(f)}(x,y)=f(x,y)$ for every $f\in {\mathcal {W}}_m^{\mathrm {{{(2)}}}}$ . However, such a map does not exist in the case that n is even! Indeed, there exist integral binary n-ic forms $f(x,y)$ that cannot be expressed as $\det (Ax - By)$ – even up to sign – for any integral $n\times n$ symmetric matrices A and B. This phenomenon was extensively studied in [Reference Bhargava5, Reference Bhargava, Gross and Wang7, Reference Bhargava, Gross and Wang8]. It is in this sense that the strategy to prove Theorem 5(b) for odd n fails spectacularly for even n – and at the very first step.

We address this issue by replacing $f(x,y)\in {\mathcal {W}}_m^{(2)}$ by $xf(x,y)$ , which is a reducible binary $(n+1)$ -ic form whose discriminant, at least generically, remains weakly divisible by $m^2$ . For these forms $xf(x,y)$ , we can use the lift $\sigma _m$ constructed in the odd case. However, since $xf(x,y)$ has vanishing $y^{n+1}$ term, the image of $\sigma _m$ lies within the set of pairs $(A,B)$ where B is singular.

The singularity of B introduces additional difficulties with respect to both the algebraic and the analytic aspects of the proof. On the algebraic side, the main new problem is that distinguished elements $(A,B)$ with B singular have at least two values for the Q-invariant, since they share at least two different common isotropic $(g+1)$ -dimensional subspaces, where $n=2g+2$ . So it is no longer well defined to impose the condition that Q is large. Imposing the condition that the maximum value of Q is large does not yield sufficient savings to prove an analogue of Theorem 5(b). We thus instead construct a new invariant, termed q, such that for all but a negligible number of elements $(A,B)$ in the image of our map $\sigma _m$ , the invariant q is the minimum value taken by Q, and it satisfies $q(\sigma _m(xf(x,y)))=\pm m$ .

As in the odd degree case, we once again construct fundamental domains ${\mathcal {F}}_X$ for the action of $G({\mathbb {Z}})$ on $W({\mathbb {R}})$ with height less than X, and partition such a domain into three parts: the main body, the shallow cusp and the deep cusp. However, we must now only count integer elements $(A,B)$ where B is singular. The beautiful work of Eskin and Katznelson [Reference Eskin and Katznelson17] provides asymptotics for the number of singular symmetric matrices in homogenously expanding domains, but this work is not directly applicable to our case since we need to estimate the number of singular symmetric matrices B in skewed domains. To achieve this, we provide a simplification of the proof of the upper bounds in [Reference Eskin and Katznelson17], at the cost of some extra $\log $ factors, which gives us a flexible method by which to obtain upper bounds on the number of singular symmetric matrices in arbitrarily skewed domains.

Accounting for the singularity of the B’s introduces complications in each region of the fundamental domain. In the main body, the fact that the singular matrices B lie on the subvariety cut out the determinant means that we cannot directly apply the large sieve, and the lack of an exact count with a power-saving error term means we also cannot directly apply a Selberg sieve to bound the number of distinguished elements. Instead, we fiber over the singular matrices B and apply the large sieve to bound the number of possible A’s. This requires us to prove new density estimates on the number of distinguished elements $(A,B)$ over ${\mathbb {F}}_p$ , when B is fixed.

Furthermore, unlike in the odd degree case, we no longer have an automatic power-saving on the number of pairs $(A,B)\in W({\mathbb {Z}})$ lying in the shallow cusp of the fundamental domain and where B is singular. As we go closer to the deep cusp, there are regions in which imposing the condition that B is singular yields no saving whatsoever. To obtain the required bounds, we isolate this region of the shallow cusp and prove that integral elements $(A,B)$ in them either satisfy $\Delta (A,B)=0$ or $|q(A,B)|$ is small.

Finally, for the deep cusp of ${\mathcal {F}}$ , we once again use the condition that the q-invariant is large to obtain a power saving. Unlike the situation with the Q-invariant in the odd-degree case, the invariant q in the even degree case behaves more wildly and is much harder to control. This is because q is not a polynomial in the coefficients of $W_0$ but rather is a minimum of the different possible values of Q. In fact, there are regions within the deep cusp where the q-invariant of elements $(A,B)$ are not small. However, we show that these regions correspond to an archimedean condition on the invariant binary form f of $(A,B)$ – namely, that the discriminant of f is much smaller than is typical for the height bound on f. Separately bounding the number of such binary forms yields the desired result.

Organization of the paper

This paper is organized as follows. We begin in §3 by recalling the arithmetic invariant theory for the representations $W_n:=2\times \mathrm {{Sym}}_2(n)$ of $\mathrm {{SL}}_n$ and $2\otimes g\otimes (g+1)$ of $\mathrm {{SL}}_2\times \mathrm {{SL}}_g\times \mathrm {{SL}}_{g+1}$ . In particular, we define the fundamental invariants Q and q. We then construct our maps from ${\mathcal {W}}_{m}^{(2)}$ into $W_n({\mathbb {Z}})$ when n is odd and into $W_{n+1}({\mathbb {Z}})$ when n is even.

The analytic parts of the paper are carried out in §4–6. In §4, we prove the tail estimates of Theorem 5 for odd degrees n using geometry-of-numbers techniques. In §5, we carry out the necessary groundwork to count the number of singular symmetric matrices that lie in skewed domains. Using these results, we prove the tail estimates for even degrees n in §6, completing the proof of Theorem 5. In §7, we deduce the main results, Theorems 1–4, from the tail estimates using a squarefree sieve, although the exact constants occurring in Theorems 1 and 2 remain conditional upon certain local density computations. Finally, in the Appendix, we compute the local densities of integral binary n-ic forms whose discriminants are indivisible by $p^2$ (resp., whose associated rings are maximal at p), thereby completing the proofs of Theorems 1 and 2.

3.1 Arithmetic invariant theory for the representation $2\otimes \mathrm {{Sym}}_2(n)$ of $\mathrm {{SL}}_n$

First, let $n=2g+1$ be an odd integer with $g\geq 1$ . We recall some of the arithmetic invariant theory of the representation $W:=W_n$ of $\mathrm {{SL}}_n$ and its map (1) to $V:=V_n;$ see [Reference Bhargava, Gross and Wang7] for more details.

Let k be a field of characteristic not $2$ . For a binary n-ic form $f(x,y)=a_0x^n + \cdots + a_ny^n\in V(k)$ with $\Delta (f)\neq 0$ and $a_0\neq 0$ , let $C_f$ denote the smooth hyperelliptic curve $z^2=f(x,y)y$ of genus g viewed as a curve in the weighted projective space ${\mathbb {P}}(1,1,g+1)$ . Let $J_f$ denote the Jacobian of $C_f$ . Then the stabilizer of an element $(A,B)\in W(k)$ with invariant binary form $f(x,y)$ is isomorphic to $J_f[2](k)$ . The set of $\mathrm {{SL}}_n(k)$ -orbits on $W(k)$ with invariant binary form $f(x,y)$ maps injectively into $H^1(k,J_f[2])$ . An element $(A,B)$ (or an $\mathrm {{SL}}_n(k)$ -orbit) is distinguished if $\Delta (A,B)\neq 0$ and there exists a g-dimensional subspace defined over k that is isotropic with respect to both A and B. If $(A,B)$ is distinguished, then its $\mathrm {{SL}}_n(k)$ -orbit corresponds to the identity element of $H^1(k,J_f[2])$ , and the set of these g-dimensional subspaces is in bijection with $J_f[2](k)$ .

Let $W_{0}\subset W$ be the subspace of pairs of matrices whose top left $g\times g$ blocks are zero. Then elements $(A,B)$ in $W_{0}(k)$ with nonzero discriminant are all distinguished since the g-dimensional subspace $Y_g$ spanned by the first g basis vectors is isotropic with respect to both A and B. Moreover, every distinguished element of $W(k)$ is $\mathrm {{SL}}_n(k)$ -equivalent to some element in $W_{0}(k)$ since $\mathrm {{SL}}_n(k)$ acts transitively on the set of g-dimensional subspaces of ${\mathbb {P}}^{n-1}(k)$ . Let $G_0$ be the maximal parabolic subgroup of $\mathrm {{SL}}_n$ consisting of elements $\gamma $ that preserve $Y_g$ . Elements of $W_{0}$ have block matrix form

(2) $$ \begin{align} (A,B)=\left(\Big(\begin{array}{cc}0 & A^{\mathrm{{top}}}\\ (A^{\mathrm{{top}}})^t & A_1\end{array}\Big), \Big(\begin{array}{cc}0 & B^{\mathrm{{top}}}\\ (B^{\mathrm{{top}}})^t & B_1\end{array}\Big)\right), \end{align} $$

where $A^{\mathrm {{top}}}$ , $B^{\mathrm {{top}}}$ are $g\times (g+1)$ matrices and $A_1$ , $B_1$ are $(g+1)\times (g+1)$ -symmetric matrices. Meanwhile, elements of $G_0$ have the block matrix form

(3) $$ \begin{align} \gamma=\Big(\begin{array}{cc}\gamma_1 & 0\\ n & \gamma_2 \end{array}\Big)\in\Big(\begin{array}{cc}\mathrm{{GL}}_g & 0\\ M_{(g+1)\times g} & \mathrm{{GL}}_{g+1} \end{array}\Big). \end{align} $$

An element $\gamma \in G_0$ acts on the top right $g\times (g+1)$ block of elements of $W_{0}$ by

$$ \begin{align*}\gamma(A^{\mathrm{{top}}},B^{\mathrm{{top}}}) = (\gamma_1A^{\mathrm{{top}}}\gamma_2^t,\gamma_1B^{\mathrm{{top}}}\gamma_2^t), \end{align*} $$

where we use the superscript ‘top’ to denote the top right $g\times (g+1)$ block of an $n\times n$ symmetric matrix. The action of $G_0$ on $W_{0}$ restricts to an action on the space $U_g:=2\otimes g\otimes (g+1)$ of pairs of $g\times (g+1)$ -matrices, Moreover, the unipotent radical $M_{(g+1)\times g}$ of $G_0$ acts trivially on $U_g$ . We study the invariant theory for this action more closely in the next subsection.

We will also need some results in the case when $n=2g+2$ is even in Section 6 (specifically in the proof of Lemma 6.7). Let $f(x,y) = a_0x^n + \cdots + a_ny^n\in V(k)$ with $\Delta (f)\neq 0$ and $a_0\neq 0$ . Let $L = k[x]/(f(x,1))$ . Let $V_f(k)$ denote the set of $(A,B)\in W_n(k)$ with $f_{A,B} = f(x,y)$ . Then $V_f(k)$ is nonempty if and only if $f_0\in k^{\times 2}N_{L/k}(L^\times )$ (see also [Reference Bhargava, Gross and Wang8, Theorem 7]). Note in particular that if $f(x,y)\in V({\mathbb {R}})$ is negative definite, so that $L = {\mathbb {R}}[x]/(f(x))\simeq {\mathbb {C}}^{n/2}$ and $a_0<0$ , then $V_f({\mathbb {R}})$ is empty. However, if k is a finite field of characteristic not $2$ , then $V_f(k)$ is always nonempty and the number of $\mathrm {{SL}}_n(k)$ -orbits equals the number of even degree factorizations of $f(x,y)$ over k.

3.2 The representation $2\otimes g\otimes (g+1)$ of $\mathrm {{SL}}_2\times \mathrm {{SL}}_g\times \mathrm {{SL}}_{g+1}$ and the Q-invariant

In this section, we collect some algebraic facts about the representation $U_g;=2\otimes g\otimes (g+1)$ of the group $H_g:=\mathrm {{SL}}_2\times \mathrm {{SL}}_g\times \mathrm {{SL}}_{g+1}$ . We start with the following proposition.

Since castling transforms also preserve polynomial invariants and their irreducibility [Reference Sato and Kimura30, Proposition 18], it follows that the ring of polynomial invariants for this action of $H_g$ on $U_g$ is generated by an irreducible polynomial. We now give an explicit description of this invariant.

Write an element in $U_g=2\times g\times (g+1)$ as a pair $(A^{\mathrm {{top}}},B^{\mathrm {{top}}})$ of $g\times (g+1)$ matrices. For $1\leq i\leq g+1$ , let $A_i$ and $B_i$ denote the $g\times g$ -matrices obtained from $A^{\mathrm {{top}}}$ and $B^{\mathrm {{top}}}$ , respectively, by deleting the ith column. Define the binary g-ic form $f_i(x,y)$ to be $(-1)^{i+1}\det (A_ix-B_iy)$ . Consider the $(g+1)\times (g+1)$ matrix C whose $(i,j)$ -entry is the jth-coefficient of $f_i(x,y)$ . Taking the determinant of C yields a polynomial $Q=Q(A^{\mathrm {{top}}},B^{\mathrm {{top}}})$ in the coordinates of $U_g$ . The polynomial Q is the hyperdeterminant of the $2\times g \times (g+1)$ matrix $(A^{\mathrm {{top}}},B^{\mathrm {{top}}})$ (cf. [Reference Gelfond, Kapranov and Zelevinsky18, Chapter 14, Theorem 3.18] with $m = g$ , $n = g+1$ , $p = 2$ ). As a consequence, it is irreducible and invariant under the action of $H_g$ on $U_g$ and thus generates the ring of polynomials for the action of $H_g$ on $U_g$ .

Let $n=2g+1$ again be an odd integer. We return to the representation $W_{0}$ of $G_0$ . Given an element $(A,B)\in W_{0}$ , recall that we obtain an element $(A^{\mathrm {{top}}},B^{\mathrm {{top}}})\in U_g$ by taking the top right $g\times (g+1)$ blocks of A and B. We define the Q-invariant of $(A,B)\in W_{0}$ as the Q-invariant of $(A^{\mathrm {{top}}},B^{\mathrm {{top}}})$ :

(4) $$ \begin{align} Q(A,B):=Q(A^{\mathrm{{top}}},B^{\mathrm{{top}}}). \end{align} $$

Then the Q-invariant is a relative invariant for $G_0$ . More precisely, for any $\gamma \in G_0$ in the block matrix form (3), we have

(5) $$ \begin{align} Q(\gamma\cdot (A,B)) = \det(\gamma_1)^{g+1}\det(\gamma_2)^gQ(A,B)=\det(\gamma_1)Q(A,B), \end{align} $$

since $\det (\gamma _1)\det (\gamma _2)=1$ . If $\gamma \in G_0({\mathbb {Z}})$ , then we have $\det (\gamma _1)=\det (\gamma _2)=\pm 1$ . Hence, the absolute value $|Q|$ of Q is an invariant for the action of $G_0({\mathbb {Z}})$ on $W_{0}({\mathbb {Z}})$ .

3.3 Divisibility properties of $\Delta $ when restricted to $W_{0}$

Let $n=2g+1$ be an odd integer. Write the coordinates on $W_{0}$ as $a_{ij},b_{ij}$ with $i,j$ in the appropriate ranges. Let R denote the ring of regular functions of $W_0$ over ${\mathbb {Z}}$ (i.e., $R={\mathbb {Z}}[W_0]={\mathbb {Z}}[a_{ij},b_{ij}]$ ). Consider the discriminant polynomial $\Delta \in R$ given by $\Delta (A,B):=\Delta (f_{A,B})$ . In this section, we prove that $Q^2\mid \Delta $ as polynomials in R, along with another useful divisibility result.

Let Z be the closed subvariety of $W_{0}$ consisting of elements $(A,B)$ with $\Delta (A,B)=0$ , and let $Y\subset Z$ denote the closed subvariety of $W_{0}$ consisting of elements $(A,B)$ such that $f_{A,B}$ is either divisible by the cube of a binary form with degree $\geq 1$ or the square of a binary form with degree $\geq 2$ . Both of these varieties Y and Z are defined over ${\mathbb {Z}}$ and are clearly $\mathrm {{SL}}_2\times G_0$ -invariant.

Our first result states that the variety in $W_{0}$ cut out by $Q=0$ does not lie in Y.

Proposition 3.2. Let $(A,B) = ((a_{ij})_{ij},(b_{ij})_{ij}) \in W_0(R)$ be the generic element. Then

$$ \begin{align*} (A,B)\mathrm{{ \;mod\; }}Q\not\in Y(R/(Q)). \end{align*} $$

Proof. Fix an odd prime p. Let $f(x,y)$ be an element of $V({\mathbb {Z}})$ , such that the reduction of $f(x,y)$ modulo p factors as $x^2h(x,y)$ , where h is irreducible. In particular, $f(x,y)$ mod p is not divisible by either the cube of a binary form with degree $\geq 1$ , or the square of a binary form with degree $\geq 2$ . Let $(A_f,B_f)\in W_{0}({\mathbb {Z}})$ be an element with invariant binary n-ic form equal to f and $Q(A_f,B_f) = p$ . Such an element $(A_f,B_f)$ is constructed in the next subsection (see (9) with $m = p$ ).

Let $\pi :R\rightarrow {\mathbb {Z}}$ denote the specialization map assigning integer values to $a_{ij},b_{ij}$ such that

$$ \begin{align*} \pi(A,B) = (A_f,B_f). \end{align*} $$

Then $\pi (Q) = p$ and so $\pi $ induces a map $R/(Q) \rightarrow {\mathbb {F}}_p$ . Since $(A_f,B_f)\text { mod }p\notin Y({\mathbb {F}}_p)$ , we see that $(A,B)\text { mod }Q\notin Y(R/(Q))$ .

The next lemma, which follows from a direct computation, gives the Q-invariant for elements in $W_{0}$ having a specific form.

Lemma 3.3. Let k be a field and let $(A,B)\in W_{0}(k)$ be an element such that the top right $g\times (g+1)$ blocks of $(A,B)$ are of the following form:

(6)

Then

$$ \begin{align*} Q(A,B)=\pm a_1^ga_2^{g-1}\cdots a_gb_1b_2^2\cdots b_g^g. \end{align*} $$

Next, we have the following proposition that gives a normal form for elements $(A,B)\not \in Y$ whose Q-invariant is $0$ .

Proposition 3.4. Let k be a field. Let $(A,B)$ be an element of $W_{0}(k)\backslash Y(k)$ such that $Q(A,B)=0$ . Then $(A,B)$ is $\mathrm {{SL}}_2(k)\times G_0(k)$ -equivalent to an element of the form $(A',B')$ where the top right $g\times (g+1)$ blocks of $A'$ and $B'$ are given by

(7)

where $a_1,\ldots ,a_g,b_2,\ldots ,b_g\in k^\times .$ In the displayed matrices above, any empty entry is $0$ .

We are now ready to prove that $Q^2\mid \Delta $ :

Proof. Let $(A,B)\in W_{0}(R)$ be the generic element. We begin by proving that $(A,B)\in Z(R/(Q))$ , or equivalently that $Q\mid \Delta $ in R. Let $(\bar {A},\bar {B})\in W_{0}(R/(Q))$ denote the reduction of $(A,B)$ mod Q, and let F denote the field of fractions of $R/(Q)$ . By Proposition 3.2, we know $(\bar {A},\bar {B})\notin Y(F)$ . Since $Q(\bar {A},\bar {B}) = 0$ , by Proposition 3.4, there exists $\gamma \in \mathrm {{SL}}_2(F)\times G_0(F)$ such that $\gamma (\bar {A},\bar {B})=(A',B')$ , where $(A^{\prime \,\mathrm {{top}}},B^{\prime \,\mathrm {{top}}})$ is of the form (7). The invariant binary form of $(A',B')$ has a factor of $x^2$ , and so $(A',B')\in Z(F)$ . Since Z is $\mathrm {{SL}}_2\times G_0$ -invariant, we see that $(\bar {A},\bar {B})\in Z(F)$ .

Since $Q\mid \Delta $ in R, there exists an element $\delta \in R$ such that $\Delta =Q\delta $ . Let $Z_1$ denote the closed subvariety of $W_{0}$ cut out by $\delta $ . It now suffices to prove that $Q\mid \delta $ or, equivalently, that the generic element $(A,B)$ belongs to $Z_1(R/Q)$ . We claim that for any field k, and every element $(A,B)\in W_{0}(k)$ such that $(A^{\mathrm {{top}}},B^{\mathrm {{top}}})$ has the form (7), we have $\delta (A,B)=0$ . Indeed, let $(A,B)$ be such an element. Let $(A^{(\epsilon )},B^{(\epsilon )})\in W_{0}(k[\epsilon ])$ be such that $A^{(\epsilon )}=A$ , the $(1,n-1)$ -entry and the $(n-1,1)$ -entry of $B^{(\epsilon )}$ equal $\epsilon $ , and the other coefficients of $B^{(\epsilon )}$ are the same as those of B. By Lemma 3.3, we have

$$ \begin{align*} Q(A^{(\epsilon)},B^{(\epsilon)})=\pm \epsilon\, a_1^ga_2^{g-1}\cdots a_gb_2^2\cdots b_g^g. \end{align*} $$

Moreover, $\epsilon ^2$ divides the $y^n$ -coefficient of $f_{A^{(\epsilon )},B^{(\epsilon )}}$ and $\epsilon $ divides the $xy^{n-1}$ -coefficient of $f_{A^{(\epsilon )},B^{(\epsilon )}}$ . Hence, $\epsilon ^2\mid \Delta (A^{(\epsilon )},B^{(\epsilon )})$ , which implies (since $\epsilon ^2\nmid Q(A^{(\epsilon )},B^{(\epsilon )})$ ) that $\epsilon \mid \delta (A^{(\epsilon )},B^{(\epsilon )})$ . Since $(A,B)$ is obtained from $(A^{(\epsilon )},B^{(\epsilon )})$ by setting $\epsilon =0$ , we have $\delta (A,B)=0$ . We have proven that the generic element $(A,B)\in W_{0}(R)$ belongs to $Z_1(R/(Q))$ . Therefore, $Q\mid \delta $ .

We end this section with another divisibility result for $\Delta $ , which will be used in §6.

Proof. It suffices to prove that $\det (B^{\mathrm {{top}}} (B^{\mathrm {{top}}})^t)$ divides $\Delta $ in ${\mathbb {Z}}[W_{0}]$ . Suppose $(A,B)\in W_{0}({\mathbb {C}})$ with $\det (B^{\mathrm {{top}}} (B^{\mathrm {{top}}})^t) = 0$ . Then $B^{\mathrm {{top}}}$ does not have full rank. Hence, there exists some nonzero $v\in \text {Span}_{\mathbb {C}}\{e_1,\ldots ,e_g\}$ such that $Bv = 0$ . However, any such v is isotropic with respect to A. As a result, $\Delta (A,B) = 0$ . Thus, by the Nullstellensatz, $\det (B^{\mathrm {{top}}} (B^{\mathrm {{top}}})^t) \mid c \Delta ^{d}$ in ${\mathbb {Z}}[W_{0}]$ for some nonzero integer c and positive integer d.

Define $P_g\in {\mathbb {Z}}[M_{g\times (g+1)}]$ by $P_g(M) = \det (MM^t)$ . For the purpose of proving Proposition 3.6, it suffices to prove that $P_g$ is squarefree in ${\mathbb {Z}}[M_{g\times (g+1)}]$ . We proceed by induction on g. Denote the $(i,j)$ -entry of any $M\in M_{g\times (g+1)}$ by $u_{ij}$ . When $g = 1$ , we have $P_1 = u_{11}^2 + u_{12}^2$ , which is squarefree in ${\mathbb {Z}}[u_{11},u_{12}]$ . For general $g\geq 2$ , consider

$$ \begin{align*}M = \begin{pmatrix} u_{11} & \cdots & u_{1\,g-1} & u_{1\,g} & 0\\ \vdots & \ddots & \vdots & \vdots & \vdots \\ u_{g-1\, 1} & \cdots & u_{g-1\,g-1} & u_{g-1\,g} & 0\\ 0&\cdots&0&\alpha&\beta \end{pmatrix}.\end{align*} $$

Then

$$ \begin{align*}\det(MM^t) = \beta^2 P_{g-1} + \alpha^2 D_{g-1}^2,\end{align*} $$

where $D_{g-1}$ is the determinant of the top left $(g-1)\times (g-1)$ block of M. Any square factor of $\det (MM^t)$ must be a common square factor of $P_{g-1}$ and $D_{g-1}^2$ , which can only be $\pm 1$ since $P_{g-1}$ is squarefree by induction. We have shown that $P_g$ is squarefree even after setting certain variables to $0$ . Therefore, $P_g$ is squarefree in ${\mathbb {Z}}[M_{g\times (g+1)}]$ .

3.4 Embedding ${\mathcal {W}}_{m,n}^{\mathrm {{{(2)}}}}$ into $W_n({\mathbb {Z}})$ , for n odd

Let $n=2g+1$ be an odd integer, and set $W=W_n$ . For an odd squarefree integer $m>0$ , let ${\mathcal {W}}_m^{\mathrm {{{(2)}}}}={\mathcal {W}}_{m,n}^{\mathrm {{{(2)}}}}$ denote the set of integer n-ic binary forms whose discriminants are weakly divisible by $p^2$ for every prime factor p of m. Fix an element $f(x,y)\in {\mathcal {W}}_m^{\mathrm {{{(2)}}}}$ . Then just as shown in [Reference Bhargava, Shankar and Wang11, §3.2], there exists an $\mathrm {{SL}}_2({\mathbb {Z}})$ -change of variable such that $f((x,y)\gamma )$ has the form

(8) $$ \begin{align} f((x,y)\gamma) = m^2b_0x^n + mb_1x^{n-1}y + \cdots + b_ny^n \end{align} $$

for some integers $b_0,\ldots ,b_n$ and where m and $b_0$ are coprime.

Consider the following pair of matrices:

(9) $$ \begin{align} A = \left(\begin{array}{ccccccc}&&&&&&1\\&&&&&\unicode{x22F0}&\\ &&&&m&&\\ &&&c_0&&& \\ &&m&&c_{2}&&\\ &\unicode{x22F0}&&&&\ddots&\\ 1&&&&&&c_{n-1} \end{array}\right),\;\; B = \left(\begin{array}{ccccccc}&&&&&1&0\\&&&&\unicode{x22F0}&\unicode{x22F0}&\\ &&&1&r&&\\ &&1&c_1&&& \\ &\unicode{x22F0}&r&&c_3&&\\ 1&\unicode{x22F0}&&&&\ddots&\\ 0&&&&&&c_n \end{array}\right). \end{align} $$

Here, the dots on the antidiagonal of A are all $1$ and the dots on the antidiagonal of B are all $0$ . We claim that $c_i,r$ can be chosen to be integers so that $(-1)^g\det (xA - yB) = f((x,y)\gamma )$ . It is clear that $c_0 = b_0$ and $2mrc_0 + m^2c_1 = -mb_1$ . Choose $r\in {\mathbb {Z}}$ such that $m\mid 2rc_0+b_{1}$ ; this then determines $c_{1}$ . It is then not hard to check that the coefficient of $x^{n-i}y^i$ in $(-1)^g\det (xA - yB)$ is of the form $(-1)^ic_i + L(c_0,\ldots ,c_{i-1})$ where L is a linear form with coefficients in ${\mathbb {Z}}[r]$ . The existence of integers $c_2,\ldots ,c_n$ now follows by induction.

Set $\sigma _m(f)=\sigma _{m,n}(f)$ to be the element $(A_f,B_f)$ such that

$$ \begin{align*}\begin{pmatrix} A_f \\ -B_f \end{pmatrix} = \gamma^{-1} \begin{pmatrix} A \\ -B \end{pmatrix}.\end{align*} $$

Then $f_{\sigma _{m}(f)} = f.$ Next, we note that $(A,B)$ , and thus, $(A_f,B_f)$ are in $W_{0}({\mathbb {Z}})$ , and from Lemma 3.3, we obtain that $|Q|(A,B) = m$ . Since Q is $\mathrm {{SL}}_2$ -invariant, we conclude that

$$ \begin{align*}|Q|({\sigma_{m}(f)})=m.\end{align*} $$

We have proven the following theorem.

Theorem 3.7. Let $m>0$ be a squarefree integer. There exists a map $\sigma _{m}:{\mathcal {W}}_{m}^{\mathrm {{{(2)}}}}\to W_{0}({\mathbb {Z}})$ such that

$$ \begin{align*}f_{\sigma_{m}(f)}=f,\qquad |Q|(\sigma_{m}(f)) = m\end{align*} $$

for every $f\in {\mathcal {W}}_{m}^{\mathrm {{{(2)}}}}$ .

We will later use the image of $\sigma _{1}$ as a fundamental set for the action of $\mathrm {{SL}}_n({\mathbb {R}})$ on the set of distinguished elements of $W({\mathbb {R}})$ . We now extend the function $|Q|$ to the set of distinguished elements of $W({\mathbb {Z}})$ having irreducible invariant binary form. Suppose that $(A,B)$ is a distinguished element of $W({\mathbb {Z}})$ . Then there is a g-dimensional subspace X isotropic with respect to A and B. Let $\Lambda = X\cap {\mathbb {Z}}^n$ be the primitive lattice in X. There exists an element $\gamma $ in $\mathrm {{SL}}_n({\mathbb {Z}})$ , unique up to left multiplication by an element in $G_0({\mathbb {Z}})$ , such that $\Lambda = \gamma ^t\cdot \,\text {Span}_{\mathbb {Z}}\{e_1,\ldots ,e_g\}$ , where $e_1,\ldots ,e_n$ is the standard basis of ${\mathbb {Z}}^n$ . Then $\gamma \cdot (A,B)\in W_{0}({\mathbb {Z}})$ , and we can thus define the $|Q|$ -invariant on the triple $(A,B,\Lambda )$ by

$$ \begin{align*}|Q|(A,B,\Lambda):=|Q|(\gamma\cdot(A,B)).\end{align*} $$

That is, we complete an integral basis of $\Lambda $ to an integral basis of ${\mathbb {Z}}^n$ with respect to which the pair $(A',B')$ of Gram matrices for the quadratic forms defined by A and B lies inside $W_0({\mathbb {Z}})$ , and we define $|Q|(A,B,\Lambda )$ to be $|Q|(A',B').$

We end with the following result that will be crucial in Section 4.

3.5 Embedding ${\mathcal {W}}_{m,n}^{(2),\,\mathrm {{gen}}}$ into $W_{n+1}({\mathbb {Z}})$ , for n even

Suppose now that $n = 2g+2$ is even with $g\geq 1$ . For an odd squarefree integer $m>0$ , let ${\mathcal {W}}_{m,n}^{\mathrm {{{(2)}}}}$ denote the set of integer binary forms having discriminant weakly divisible by $p^2$ for every prime factor p of m. Let ${\mathcal {W}}_{m,n}^{(2),\,\mathrm {{gen}}}\subset {\mathcal {W}}_{m,n}^{(2)}$ consist of those $f(x,y)$ with $f(0,1)$ coprime to m. Since $\Delta (xf(x,y)) = \Delta (f(x,y))f(0,1)^2$ , we see that if $f(x,y)\in {\mathcal {W}}_{m,n}^{(2),\,\mathrm {{gen}}}$ , then $xf(x,y)\in {\mathcal {W}}_{m,n+1}^{(2)}$ . We define $\sigma _{m,n}:{\mathcal {W}}_{m,n}^{(2),\,\mathrm {{gen}}}\rightarrow W_{n+1}({\mathbb {Z}})$ via $\sigma _{m,n}(f) = \sigma _{m,n+1}(xf)$ . For the rest of this subsection, we drop subscripts and denote $\sigma _{m,n}$ by $\sigma _m$ , ${\mathcal {W}}_{m,n}^{\mathrm {{{(2)}}},\,\mathrm {{gen}}}$ by ${\mathcal {W}}_m^{\mathrm {{{(2)}}}\,\mathrm {{gen}}}$ , and $W_{n+1}$ by W.

We now define the finer q-invariant. Let $f\in {\mathcal {W}}_{m}^{(2),\,\mathrm {{gen}}}$ and suppose $(A,B) = \sigma _{m}(f)$ . Then B is singular since $xf(x,y)$ has vanishing $y^n$ -term. Moreover, since $\Delta (xf)\neq 0$ , the kernel of B has dimension exactly $1$ and is not isotropic with respect to A (see Lemma 6.4). Fix an integral domain D. Let $W_{1}(D)$ be the subset of $W(D)$ consisting of pairs $(A,B)$ of symmetric $(n+1)\times (n+1)$ matrices satisfying the following conditions:

1. (a) The top left $(g+1)\times (g+1)$ block of A is $0$ .

2. (b) The top left $(g+2)\times (g+2)$ block of B is $0$ (implying that B is singular).

3. (c) The kernel of B has dimension exactly $1$ (over the fraction field of D) and is not isotropic with respect to A.

Take any $(A,B)\in W_{1}(D)$ . Conditions (a) and (c) imply that the first $g+1$ columns of B are linearly independent over the fraction field of D. Let $B'$ denote the top right $(g+1)\times (g+1)$ block of B. Since B is symmetric, we see that $B'$ is nonsingular. It is now easy to see from the definition of the Q-invariant that as polynomials in the coordinates of $W_{1}(D)$ , we have

$$ \begin{align*}\det(B') \mid Q(A^{\mathrm{{top}}}, B^{\mathrm{{top}}}).\end{align*} $$

We define the quotient to be the q-invariant of $(A,B)$ :

(10) $$ \begin{align} q(A,B) := Q(A^{\mathrm{{top}}}, B^{\mathrm{{top}}})/\det(B'). \end{align} $$

Let $G_1(D)$ denote the subgroup of $\mathrm {{SL}}_{n+1}(D)$ preserving $W_{1}(D)$ . Then elements of $G_1(D)$ have the following block matrix form:

(11) $$ \begin{align} \gamma=\left(\begin{array}{ccc}\gamma_1 & 0 & 0\\ n_1 & \gamma_2 & 0\\ n_2 & n_3 & \gamma_3 \end{array}\right)\in\left(\begin{array}{ccc}\mathrm{{GL}}_{g+1} & &\\ M_{1\times (g+1)} & \mathrm{{GL}}_1 &\\ M_{(g+1)\times (g+1)} & M_{(g+1)\times 1} & \mathrm{{GL}}_{g+1} \end{array}\right). \end{align} $$

It is easy to check that for any $(A,B)\in W_{1}(D)$ ,

(12) $$ \begin{align} q(\gamma(A,B)) = \det(\gamma_1)(\det(\gamma_1)\det(\gamma_3))^{-1}q(A,B) = \det(\gamma_1)\gamma_2\,q(A,B). \end{align} $$

We now consider the situation over ${\mathbb {Z}}$ . Let $(A,B)\in W({\mathbb {Z}})$ be a distinguished element having nonzero discriminant such that B is singular. Let X denote a common isotropic $(g+1)$ -dimiensional subspace of A and B. We know that the kernel $\langle v\rangle $ of B has trivial intersection with X. Denote the span of X and v by $X'$ , which is a $(g+2)$ -dimensional subspace containing X that is isotropic with respect to B. Let $\Lambda = X\cap {\mathbb {Z}}^{n+1}$ and $\Lambda ' = X'\cap {\mathbb {Z}}^{n+1}$ be the primitive lattices in X and $X'$ , respectively. There exists an element $\gamma $ in $\mathrm {{SL}}_{n+1}({\mathbb {Z}})$ , unique up to left multiplication by an element in $G_1({\mathbb {Z}})$ , such that $\Lambda = \gamma ^t.\,\text {Span}_{\mathbb {Z}}\{e_1,\ldots ,e_{g+1}\}$ and $\Lambda ' = \gamma ^t.\,\text {Span}_{\mathbb {Z}}\{e_1,\ldots ,e_{g+2}\}$ . Then $\gamma (A,B)\in W_{1}({\mathbb {Z}})$ , and we can thus define the $|q|$ -invariant for the quadruple $(A,B,\Lambda ,\Lambda ')$ by

$$ \begin{align*}|q|(A,B,\Lambda,\Lambda'):= |q(\gamma(A,B))|.\end{align*} $$

In other words, we complete an integral basis $\{v_1,\ldots ,v_{g+1}\}$ of $\Lambda $ to an integral basis $\{v_1,\ldots ,v_{n+1}\}$ of ${\mathbb {Z}}^{n+1}$ such that $\{v_1,\ldots ,v_{g+2}\}$ forms an integral basis of $\Lambda '$ . When expressed in this basis, the pair $(A',B')$ of Gram matrices for the quadratic forms defined by A and B lies in $W_{1}({\mathbb {Z}})$ and we define $|q|(A,B,\Lambda ,\Lambda ') := |q|(A',B').$

Finally, we compute the $|Q|$ - and $|q|$ -invariants of $\sigma _{m}(f(x,y))$ , where $f(x,y)\in {\mathcal {W}}_{m}^{(2),\,\mathrm {{gen}}}$ is irreducible.

Proof. The size of $J_f[2]({\mathbb {Q}})$ is $2$ since $xf(x,y)$ has a unique even degree factor (namely, $f(x,y)$ ) over ${\mathbb {Q}}$ . Therefore, the pair $(A,B)$ has two $(g+1)$ -dimensional common isotropic subspaces $X_1$ and $X_2$ over ${\mathbb {Q}}$ . Let $\Lambda _1$ and $\Lambda _2$ denote the corresponding primitive lattices contained in $X_1$ and $X_2$ . The unique $(g+2)$ -dimensional subspace $X_1'$ (resp., $X_2'$ ) isotropic with respect to B and containing $X_1$ (resp., $X_2$ ) is the span of $X_1$ (resp., $X_2$ ) with the kernel of B. Let $\Lambda ^{\prime }_1$ and $\Lambda _2'$ denote the primitive lattices contained in $X_1'$ and $X_2'$ . We compute the $|Q|$ - and $|q|$ -invariants associated to these lattices.

We may assume that $(A,B)=\sigma _{m}(f)$ since the action of $\mathrm {{SL}}_{n+1}({\mathbb {Z}})$ does not change the $|Q|$ - or $|q|$ invariants. Since $|Q|$ is $\mathrm {{SL}}_2$ -invariant, and $|q|$ remains unchanged when we add a multiple of B to A, we may also assume that

$$ \begin{align*}xf(x,y) = m^2b_0x^{n+1} + mb_1x^{n}y + \cdots + b_nxy^n\end{align*} $$

and so

$$ \begin{align*} A = \left(\begin{array}{ccccccc}&&&&&&1\\&&&&&\unicode{x22F0}&\\ &&&&m&&\\ &&&c_0&&& \\ &&m&&c_{2}&&\\ &\unicode{x22F0}&&&&\ddots&\\ 1&&&&&&c_n \end{array}\right),\;\; B = \left(\begin{array}{ccccccc}&&&&&1&0\\&&&&\unicode{x22F0}&\unicode{x22F0}&\\ &&&1&r&&\\ &&1&c_1&&& \\ &\unicode{x22F0}&r&&c_3&&\\ 1&\unicode{x22F0}&&&&\ddots&\\ 0&&&&&&c_{n+1} \end{array}\right). \end{align*} $$

Comparing the $y^{n+1}$ - and the $xy^{n}$ -coefficients, we have $c_{n+1} = 0$ and $c_{n}=b_{n}$ .

Let $\langle \,,\rangle _A$ and $\langle \,,\rangle _B$ denote the quadratic forms corresponding to A and B. Let $e_1,\ldots ,e_{n+1}$ be the standard basis on ${\mathbb {Z}}^{n+1}$ . Since $c_{n+1}=0$ , the vector $e_{n+1}$ spans the kernel of B. We may take the subspace spanned by $e_1,\ldots ,e_{g+1}$ as $X_1$ . Then by construction, $|Q|(A,B,\Lambda _1) = m$ . When expressed in terms of the ordered integral basis $\{e_1,\ldots ,e_{g+1},e_n,e_{g+2},\ldots ,e_{n-1}\}$ , the top right $(g+1)\times (g+1)$ block of B has $1$ ’s on the antidiagonal and $0$ ’s above the antidiagonal, and so has determinant $\pm 1$ . Hence, $|q|(A,B,\Lambda _1,\Lambda _1') = m$ .

The second common isotropic $(g+1)$ -dimensional subspace $X_2$ is the reflection of $X_1$ in the hyperplane perpendicular to $e_{n+1}$ with respect to $\langle \,,\rangle _A$ . That is,

$$ \begin{align*} X_2 &= \mathrm{Span}_{\mathbb{Q}}\Big\{e_1 - \frac{2\langle e_1,e_{n+1}\rangle_A}{\langle e_{n+1},e_{n+1}\rangle_A}e_{n+1},\ldots,e_{g+1} - \frac{2\langle e_{g+1},e_{n+1}\rangle_A}{\langle e_{n+1},e_{n+1}\rangle_A}e_{n+1}\Big\}\\ &= \mathrm{Span}_{\mathbb{Q}}\Big\{e_1 - \frac{2}{b_n}e_{n+1},e_2,e_3,\ldots,e_{g+1}\Big\}. \end{align*} $$

Suppose first that $b_n$ is odd. Then we have the following integral basis for ${\mathbb {Z}}^{n+1}$ :

$$ \begin{align*} \Big\{b_{n}e_1-2e_{n+1},e_2,e_3,\ldots,e_{g+1},\frac{b_{n}+1}{2}e_1-e_{n+1},e_{g+2},\ldots,e_{n}\Big\}. \end{align*} $$

In terms of this basis, the top right $(g+1)\times (g+2)$ blocks of A and B have the following form:

(13) $$ \begin{align}A^{\mathrm{{top}}} = \left(\begin{array}{cccccc} -1&0&0&\cdots&\cdots&0\\0&0&0&&\unicode{x22F0}&1\\ \vdots&\vdots&\vdots&\unicode{x22F0}&\unicode{x22F0}&\vdots\\ \vdots&\vdots&0&1&&\vdots \\ 0&0&m&\cdots&\cdots&0\end{array}\right),\;\; B^{\mathrm{{top}}} = \left(\begin{array}{cccccc}0&&&&&b_{n}\\0&&&&1&\\ \vdots&&&\unicode{x22F0}&&\\ \vdots&&\unicode{x22F0}&&& \\ 0&1&&&&\end{array}\right). \end{align} $$

It is then easy to check that $|Q|(A,B,\Lambda _2) = |b_{n}|m$ and $|q|(A,B,\Lambda _2,\Lambda _2') = m$ .

When $b_{n}$ is even, we have the following integral basis:

$$ \begin{align*} \Big\{\frac{b_{n}}{2}e_1-e_n,e_2,e_3,\ldots,e_{g+1},(b_{n}+1)e_1-2e_{n+1},e_{g+2},\ldots,e_{n}\Big\}. \end{align*} $$

In terms of this basis, the top right $(g+1)\times (g+2)$ blocks of A and B have the same form as in (13). Hence, the $|Q|$ - and $|q|$ -invariants are as stated in the proposition.

4.1 Reduction theory and averaging over fundamental domains

Recall that the Iwasawa decomposition of $\mathrm {{SL}}_n({\mathbb {R}})$ is given by $ \mathrm {{SL}}_n({\mathbb {R}})=NTK, $ where N is the group of unipotent lower triangular matrices in $\mathrm {{SL}}_n({\mathbb {R}})$ , $K=\mathrm {{SO}}(n)$ is a maximal compact subgroup of $\mathrm {{SL}}_n({\mathbb {R}})$ , and T is the split torus of $\mathrm {{SL}}_n({\mathbb {R}})$ consisting of $n\times n$ diagonal matrices with positive diagonal entries and determinant $1$ . We denote elements in T by $s=\mathrm {{diag}}(t_1^{-1},t_2^{-1},\ldots ,t_n^{-1})$ , where $t_i>0$ for $1\leq 1\leq n$ and $t_1t_2\cdots t_n=1$ . It will be convenient to make the following change of variables. For $1\leq i\leq n-1$ , set $s_i$ to be

$$ \begin{align*} s_i=(t_i/t_{i+1})^{1/n},\text{ which implies }\; t_i=\prod_{k=1}^{i-1}s_k^{-k}\prod_{k=i}^{n-1}s_k^{n-k} \end{align*} $$

for $1\leq i<n$ . The Haar measure of $G({\mathbb {R}})$ in these coordinates is then given by

$$ \begin{align*}dg=dn\delta(s)d^\times s dk, \;\;\;\;\;\text{where}\;\;\;\;\; \delta(s)= \prod_{1\leq i < j\leq n}\frac{t_j}{t_i} = \prod_{k=1}^{n-1}s_k^{-nk(n-k)},\end{align*} $$

$dn$ and $dk$ are Haar measures on N and K, respectively, and $d^\times s=\prod _{i=1}^{n-1}s_i^{-1}ds_i$ .

We denote the coordinates on W by $a_{ij},b_{ij}$ for $1\leq i\leq j\leq n$ . These coordinates are eigenvectors for the action of T on the dual $W^*$ of W. Denote the T-weight of a coordinate $\alpha $ on W, or more generally a product $\alpha $ of powers of such coordinates, by $w(\alpha )$ . Then $w(a_{ij}) = w(b_{ij}) = t_i^{-1}t_j^{-1}.$ It will be useful in what follows to compute the weight of the Q-invariant, which is a homogeneous polynomial of degree $g(g+1)$ in the coordinates of $W_0$ . We view the torus T as sitting inside $G_0$ . Then by (5), we have

(15) $$ \begin{align} w(Q)=\prod_{k=1}^gt_k^{-1}. \end{align} $$

Let ${\mathcal {F}}$ be a fundamental set for the action of $\mathrm {{SL}}_n({\mathbb {Z}})$ on $\mathrm {{SL}}_n({\mathbb {R}})$ that is contained in a Siegel set (i.e., contained in $N'T'K$ , where $N'$ is a set consisting of elements in N whose coefficients are absolutely bounded and $T'\subset T$ consists of elements in $s\in T$ with $s_i\geq c$ for some positive constant c). Let ${\mathcal {W}}(1)$ denote the subset of real binary n-ic forms of height bounded by $1$ and let $R' = \sigma _1({\mathcal {W}}(1))$ , where $\sigma _1$ is as in §3.4. Set $R:={\mathbb {R}}_{>0}\cdot R'$ , and note that every distinguished element of $W({\mathbb {R}})$ is $\mathrm {{SL}}_n({\mathbb {R}})$ -equivalent to some element in R.

Let $H_0$ be a nonempty open bounded left K-invariant set in $\mathrm {{SL}}_n({\mathbb {R}})$ . Denote the set $H_0\cdot R'$ by ${\mathcal {B}}_1$ . Then ${\mathcal {B}}_1$ is an absolutely bounded set in $W({\mathbb {R}})$ . Let ${\mathcal {L}}$ be any $\mathrm {{SL}}_n({\mathbb {Z}})$ -invariant subset of $W({\mathbb {Z}})$ consisting of elements that are distinguished over ${\mathbb {R}}$ , and denote the set of elements in ${\mathcal {L}}$ with height less than X by ${\mathcal {L}}_X$ . Throughout this section, let $Y=X^{1/n}$ . Then the averaging method as described in [Reference Bhargava and Shankar10, §2.3] yields the bound

(16) $$ \begin{align} \#\big(\mathrm{{SL}}_n({\mathbb{Z}})\backslash{\mathcal{L}}_X\big) \;\ll\; \int_{\gamma\in{\mathcal{F}}} \#\big(\gamma (Y{\mathcal{B}}_1)\cap {\mathcal{L}}\big)d\gamma \;\ll\; \int_{\substack{s=(s_i)_i\\ s_i\geq c}} \#\big(s (Y{\mathcal{B}})\cap{\mathcal{L}}\big)\delta(s)d^\times s \end{align} $$

for some absolutely bounded open set ${\mathcal {B}}$ containing ${\mathcal {B}}_1$ .

We denote the second integral on the right-hand side of (16) by ${\mathcal {I}}_X({\mathcal {L}})$ , and break it up into an integral over the main body, the shallow cusp and the deep cusp. We define the main body to be the range of the integral where $|a_{11}|\geq 1$ for some element in $s(Y{\mathcal {B}})$ , and denote the main-body portion of ${\mathcal {I}}_X({\mathcal {L}})$ by ${\mathcal {I}}_X^{\mathrm {{main}}}({\mathcal {L}})$ . We define the shallow cusp to be the range of the integral where $|a_{11}|< 1$ for all elements in $s(Y{\mathcal {B}})$ but $|a_{ij}|\geq 1$ for some $i,j\leq g$ , and denote the shallow-cusp portion of ${\mathcal {I}}_X({\mathcal {L}})$ by ${\mathcal {I}}_X^{\mathrm {{scusp}}}({\mathcal {L}})$ . We define the deep cusp to be the range of the integral where $|a_{ij}|<1$ for all $i, j\leq g$ and all elements in $s(Y{\mathcal {B}})$ , and denote the deep-cusp portion of ${\mathcal {I}}_X({\mathcal {L}})$ by ${\mathcal {I}}_X^{\mathrm {{dcusp}}}({\mathcal {L}})$ . Then

(17) $$ \begin{align} {\mathcal{I}}_X({\mathcal{L}})={\mathcal{I}}_X^{\mathrm{{main}}}({\mathcal{L}})+{\mathcal{I}}_X^{\mathrm{{scusp}}}({\mathcal{L}})+{\mathcal{I}}_X^{\mathrm{{dcusp}}}({\mathcal{L}}). \end{align} $$

In the next subsection, we prove bounds for the main body, the shallow cusp and the deep cusp when ${\mathcal {L}}=W({\mathbb {Z}})_{|Q|>M}^{\mathrm {{dist}},\mathrm {{irr}}}$ .

We will need the following result of Davenport to estimate the number of lattice points in bounded regions.

Proposition 4.1 [Reference Davenport14].

Let ${\mathcal {R}}$ be a bounded, semi-algebraic multiset in ${\mathbb {R}}^n$ having maximum multiplicity m that is defined by at most k polynomial inequalities, each having degree at most $\ell $ . Let ${\mathcal {R}}'$ denote the image of ${\mathcal {R}}$ under any $($ upper or lower $)$ triangular, unipotent transformation of ${\mathbb {R}}^n$ . Then the number of lattice points $($ counted with multiplicity $)$ contained in the region ${\mathcal {R}}'$ is given by

$$ \begin{align*} \mathrm{{Vol}}({\mathcal{R}}) + O (\mathrm{{max}}\{\overline{\mathrm{{Vol}}}(\overline{{\mathcal{R}}}),1\}), \end{align*} $$

where $\overline {\mathrm {{Vol}}}(\overline {{\mathcal {R}}})$ denotes the greatest d-dimensional volume of any projection of ${\mathcal {R}}$ onto a coordinate subspace obtained by equating $n-d$ coordinates to zero, as d ranges over all values in $\{1, \dots , n-1\}$ . The implied constant in the second summand depends only on n, m, k and $\ell $ .

4.2 The number of orbits of distinguished elements with large Q-invariant

In this subsection, we obtain the following upper bound on ${\mathcal {I}}_X(W({\mathbb {Z}})^{\mathrm {{dist}},\mathrm {{irr}}}_{|Q|>M})$ , thus yielding the same bound on the quantity $\#\big (\mathrm {{SL}}_n({\mathbb {Z}})\backslash \{w\in W({\mathbb {Z}})^{\mathrm {{dist}},\mathrm {{irr}}}_{|Q|>M}:H(w)<X\}\big )$ by (16).

Note that (14), (16), and Theorem 4.2 immediately imply Part (b) of Theorem 5.

We bound ${\mathcal {I}}_X(W({\mathbb {Z}})^{\mathrm {{dist}},\mathrm {{irr}}}_{|Q|>M})$ by obtaining bounds for the main body, the shallow cusp and the deep cusp. We consider first the main body. In [Reference Ho, Shankar and Varma20, Proposition 4.6], an upper bound of $o(X^{n+1})$ is obtained on ${\mathcal {I}}_X^{\mathrm {{main}}}(W({\mathbb {Z}})^{\mathrm {{dist}}})$ . This is proved using the following two ingredients: estimates with a power saving error tern on ${\mathcal {I}}_X^{\mathrm {{main}}}({\mathcal {L}})$ for lattices ${\mathcal {L}}\subset W({\mathbb {Z}})$ , and a proof that the density of elements in $W({\mathbb {F}}_p)$ that are not ${\mathbb {F}}_p$ -distinguished is bounded below by some positive constant, independent of p. To obtain a power saving bound on ${\mathcal {I}}_X(W({\mathbb {Z}})^{\mathrm {{dist}},\mathrm {{irr}}}_{|Q|>M})$ , we use the large sieve.

Proposition 4.3. Let $V\cong {\mathbb {A}}^N$ be an affine space. For every prime p, let $\Omega _p\subset V({\mathbb {F}}_p)$ and let $\omega (p) = \#\Omega _p/\#V({\mathbb {F}}_p)$ . For a rectangular box ${\mathcal {B}}' = [M_1,M_1+X_1]\times \cdots \times [M_N,M_N+X_N]$ where $M_1,\ldots ,M_N,X_1,\ldots , X_N$ are real numbers with $X_1,\ldots , X_n$ positive. Let

$$ \begin{align*}S(V,\{\Omega_p\},{\mathcal{B}}') = \{v\in V({\mathbb{Z}})\cap{\mathcal{B}}'\colon v\text{ mod }p\notin\Omega_p\text{ for all }p\}.\end{align*} $$

Then for any $L> 0$ ,

(18) $$ \begin{align} |S(V,\{\Omega_p\},{\mathcal{B}}')| \leq \prod_{i=1}^N (\sqrt{X_i} + L)^2\cdot\left(\sum_{\substack{m < L\\ m\textrm{ }\mathrm{squarefree}}}\prod_{p\mid m}\frac{\omega(p)}{1 - \omega(p)}\right)^{-1}. \end{align} $$

In particular, if $\omega (p) \gg 1$ , that is, all $\omega (p)$ are bounded below by some positive constant for large enough p, then

$$ \begin{align*}|S(V,\{\Omega_p\},{\mathcal{B}}')| \ll_{n,\epsilon} \frac{X_1\cdots X_N}{\min\{X_1,\ldots,X_N\}^{1/2 - \epsilon}}.\end{align*} $$

Proof. The bound (18) follows from [Reference Huxley19, Theorem 1] and [Reference Kowalski23, Proposition 2.4]. For the second statement, we have $\omega (p)/(1 - \omega (p))\gg 1$ , and so

$$ \begin{align*}\sum_{\substack{m < L\\ m\textrm{ }\mathrm{squarefree}}}\prod_{p\mid m}\frac{\omega(p)}{1 - \omega(p)} \gg \sum_{\substack{p < L\\ p\textrm{ }\mathrm{prime}}}1 \gg_\epsilon L^{1 - \epsilon}.\end{align*} $$

We are then done by taking $L = \min \{X_1,\ldots ,X_N\}^{1/2}.$

In the situation when, for each prime p, a positive density subset of the lattice is being excluded by the sieve, the large sieve yields a better upper bound than the Selberg sieve. See, for example, [Reference Shankar and Tsimerman35], which gives a power-saving error term of $O_\epsilon (X^{399/400+\epsilon })$ on the count of quintic fields. Applying the large sieve above instead of the Selberg sieve, and following the argument of [Reference Shankar and Tsimerman35], would yield the better error term of $O_\epsilon (X^{159/160+\epsilon })$ .

We now apply the large sieve, as stated in Proposition 4.3, to bound the number of distinguished elements in the main ball:

Proof. We apply Proposition 4.3 with $\Omega _p$ being the set of non-distinguished elements of $w({\mathbb {F}}_p)$ and the rectangular box being $s(Y{\mathcal {B}})$ . The shortest side has length $Yw(a_{11})$ . Note that

$$ \begin{align*}w(a_{11})^{-1/2}\delta(s) = \prod_{k=1}^{n-1}s_k^{n-k}\prod_{k=1}^{n-1}s_k^{-nk(n-k)} = \prod_{k=1}^{n-1}s_k^{(1-nk)(n-k)}\ll 1.\end{align*} $$

Hence, we have

$$ \begin{align*}{\mathcal{I}}_X^{\mathrm{{main}}}(W({\mathbb{Z}})^{\mathrm{{dist}}}) \ll_\epsilon Y^{n(n+1) - 1/2+\epsilon}\int_{\substack{s\in T'\\ Yw(a_{11})\gg 1}}w(a_{11})^{-1/2}\delta(s)d^\times s \ll Y^{n(n+1) - 1/2+\epsilon}.\end{align*} $$

We are now done as $Y = X^{1/n}$ .

Next, a bound on the shallow cusp follows directly from the proof of [Reference Ho, Shankar and Varma20, Proposition 4.3]:

In [Reference Ho, Shankar and Varma20, Proposition 4.3], the shallow and deep cusps were treated simultaneously, but the points in the deep cusp were ruled out since only nondistinguished elements were counted there. Hence, the proof of [Reference Ho, Shankar and Varma20, Proposition 4.3] yields the claimed bound in Proposition 4.5.

Finally, to treat the deep cusp, let $U=\{a_{ij},b_{ij}:1\leq i\leq j\leq n\}$ denote the set of coordinates on W, and let $U_0=\{a_{ij},b_{ij}\mid i\leq j, \:j\geq g+1\}$ denote the set of coordinates on $W_0$ . We define a partial order $\lesssim $ on U by setting $\alpha \lesssim \beta $ if all the powers of $s_i$ in $w(\alpha )^{-1}w(\beta )$ are nonnegative. Explicitly, $a_{ij}\leq a_{i'j'}$ if and only if $i\leq i'$ and $j\leq j'$ (and similarly for $b_{ij}$ , as $a_{ij}$ and $b_{ij}$ have the same weight). A subset ${\mathcal {Z}}$ of $U_0$ is saturated if for any $\beta \in {\mathcal {Z}}$ and any $\alpha \in U_0$ with $\alpha \lesssim \beta $ , the coordinate $\alpha $ also lies in ${\mathcal {Z}}$ . We pick positive constants $c_{ij}$ for $1\leq i\leq j\leq n$ such that

1. (a) If $|Yw(a_{ij})|<c_{ij}$ , then $|a_{ij}|<1$ and $|b_{ij}|<1$ for every $(A,B)\in s(Y{\mathcal {B}})$ .

2. (b) For all $s\in T'$ and $a_{ij}\lesssim a_{i'j'}$ , we have $w(a_{ij})/c_{ij}\leq w(a_{i'j'})/c_{i'j'}$ .

More explicitly, we may choose $c_{nn}$ to be sufficiently small and take

$$ \begin{align*}c_{ij} = \Big(\sup_{s\in T'} \frac{w(a_{ij})}{w(a_{nn})}\Big) c_{nn},\quad\text{for }i\leq j\leq n. \end{align*} $$

The significance of these constants $c_{ij}$ is the following: for every $Y>1$ , first, if $Yw(a_{ij})<c_{ij}$ , then every integral element in $s(Y{\mathcal {B}})$ has $a_{ij}$ - and $b_{ij}$ -coordinates equal to $0$ ; and second, if $a_{ij}\lesssim a_{i'j'}$ , then $Yw(a_{i'j'})<c_{i'j'}$ implies $Yw(a_{ij})<c_{ij}$ .

The following lemma gives conditions that ensure an element in $W({\mathbb {R}})$ has discriminant $0$ .

The next lemma states that when ${\mathcal {L}}\subset W({\mathbb {Z}})$ consists of elements with nonzero discriminant, the integral defining ${\mathcal {I}}_X({\mathcal {L}})$ can be cut off by conditions of the form $s_i\ll X^{\Theta }$ for some absolute constant $\Theta $ depending only on n.

We now estimate the contribution to ${\mathcal {I}}_X(W({\mathbb {Z}})_{|Q|>M}^{\mathrm {{dist}},\mathrm {{irr}}})$ coming from the deep cusp.

Proof. For a subset ${\mathcal {Z}}$ of $U_0$ , let $T^{\prime }_{\mathcal {Z}}$ denote the subset of $s\in T'$ with $Y^{g(g+1)}w(Q)\gg M$ , and $|Yw(a_{ij})|< c_{ij}$ precisely for those $(i,j)$ where $a_{ij}\in {\mathcal {Z}}$ or $b_{ij}\in {\mathcal {Z}}$ . Note that $T^{\prime }_{\mathcal {Z}}$ is empty if ${\mathcal {Z}}$ is not saturated. Define

$$ \begin{align*} N({\mathcal{Z}},X) &:= \int_{s\in T^{\prime}_{\mathcal{Z}}}\#\big(s(Y{\mathcal{B}})\cap W_0({\mathbb{Z}})\big)\,\delta(s)d^\times s \\ & \ll \int_{s\in T^{\prime}_{\mathcal{Z}}} \Big(\prod_{\alpha\in U_0\backslash {\mathcal{Z}}}Yw(\alpha)\Big) \,\delta(s) d^\times s \\ &= \int_{s\in T^{\prime}_{\mathcal{Z}}} \Big(\prod_{\alpha\in U_0}Yw(\alpha)\Big) \Big(\prod_{\alpha\in {\mathcal{Z}}}Y^{-1}w(\alpha)^{-1}\Big) \,\delta(s) d^\times s, \end{align*} $$

where the bound on the second line follows from Proposition 4.1. Let $U' := \{a_{ij},b_{ij}\mid i+j<n\}$ . If ${\mathcal {Z}}$ is saturated and not contained in $U'$ , then ${\mathcal {Z}}$ contains $a_{k,n-k}$ for some $k=1,\ldots ,g$ . Hence, for any $s\in T^{\prime }_{\mathcal {Z}}$ , every integral element in $s(Y {\mathcal {B}})\cap W_0({\mathbb {Z}})$ satisfies $a_{ij}=b_{ij}=0$ , for $i\leq k$ and $j\leq n-k$ , and so has zero discriminant by Lemma 4.6. Therefore,

$$ \begin{align*}{\mathcal{I}}_X^{\mathrm{{dcusp}}}(W({\mathbb{Z}})^{\mathrm{{irr}}}_{|Q|>M})\ll \sum_{\mathcal{Z}} N({\mathcal{Z}},X),\end{align*} $$

where the sum is over saturated subsets ${\mathcal {Z}}$ of $U_0$ contained in $U'$ .

Now

(19) $$ \begin{align} \nonumber\prod_{\alpha\in U_0} Yw(\alpha) &= Y^{n(n+1)-g(g+1)} (t_1\cdots t_{g})^{2g+2}\\[-.1in] \nonumber&= \frac{Y^{n(n+1)}}{Y^{g(g+1)}w(Q)} (t_1\cdots t_{g})^{g+1}(t_{g+1}\cdots t_n)^{-g}\\ &= \frac{Y^{n(n+1)}}{Y^{g(g+1)}w(Q)} \prod_{i=1}^{g}\prod_{j=g+1}^n \frac{t_i}{t_j}. \end{align} $$

Fix a saturated subset ${\mathcal {Z}}$ of $U_0$ contained in $U'$ . We define a map $\pi :{\mathcal {Z}}\rightarrow U_0\backslash U'$ by

$$ \begin{align*} \pi(a_{ij}) = a_{i,n-i},\qquad \pi(b_{ij}) = b_{n-j,j}. \end{align*} $$

Note that for any $\alpha \in {\mathcal {Z}}$ , we have $\pi (\alpha )\notin U'$ and so $Yw(\pi (\alpha ))\gg 1$ . Furthermore, for every $\alpha \in U'$ , we have $\alpha \lesssim \pi (\alpha )$ and so $w(\pi (\alpha ))/w(\alpha )\gg 1$ . Hence, for any $s\in T^{\prime }_{\mathcal {Z}}$ ,

(20) $$ \begin{align} \prod_{\alpha\in {\mathcal{Z}}} (Yw(\alpha))^{-1} \ll \prod_{\alpha\in {\mathcal{Z}}}\frac{Yw(\pi(\alpha))}{Yw(\alpha)} \ll \prod_{\alpha\in U'}\frac{Yw(\pi(\alpha))}{Yw(\alpha)} = \Big(\prod_{g+1\leq i< j\leq n-1} \frac{t_i}{t_j}\Big) \Big(\prod_{1\leq i< j\leq g+1} \frac{t_i}{t_j}\Big). \end{align} $$

Here, the first product on the right-hand side is the contribution from all $a_{ij}\in U'$ , and the second product is the contribution from all $b_{ij}\in U'$ . Note that when multiplying the right-hand sides of (19) and (20), we get all of the $t_i/t_j$ for $1\leq i < j \leq n$ except for the ones with $i\geq g+1$ and $j = n$ . For any $s\in T^{\prime }_{\mathcal {Z}}$ , we have $t_i/t_j\gg 1$ for any $i < j$ , and so

$$ \begin{align*}\prod_{\alpha\in U_0\backslash {\mathcal{Z}}}Yw(\alpha) \ll \frac{Y^{n(n+1)}}{Y^{g(g+1)}w(Q)} \prod_{1\leq i < j\leq n} \frac{t_i}{t_j} = \frac{Y^{n(n+1)}}{Y^{g(g+1)}w(Q)}\,\delta(s)^{-1} \ll \frac{Y^{n(n+1)}}{M}\,\delta(s)^{-1}. \end{align*} $$

Since each $s_i$ is bounded below by an absolute constant and bounded above by a power of X by Lemma 4.7, we obtain

$$ \begin{align*}N({\mathcal{Z}},X) = O_\epsilon\Big(\frac{X^{n+1+\epsilon}}{M}\Big).\end{align*} $$

The proof is completed by summing over all saturated subsets ${\mathcal {Z}}$ contained in $U_1$ .

Theorem 4.2 now follows from Propositions 4.4, 4.5 and 4.8.