2. Trace formula set-up

2·1. Set-up

Fix a square-free positive integer N. Set $H^{\mathrm{new}}(N)$ to denote a basis of the space $S^{\mathrm{new}}(N)$ of weight 2 Hecke cusp newforms for $\Gamma_0(N)$ , and let $a_f(P) = \lambda_f(P)\sqrt{P}$ denote the eigenvalue under the P-th Hecke operator $T_p$ of $f \in H^{\mathrm{new}}(N)$ . Let $\epsilon(f)$ denote the root number of f and recall that $-\epsilon(f)$ is equal to the eigenvalue of f under the Atkin–Lehner involution $W_N$ .

In contrast to previous considerations of murmuration behaviour we are interested in the Fourier-coefficient $\lambda_f(P^2)$ . When $P\nmid N$ , we know that

\[ \lambda_f(P^2) = \lambda_f(P)^2 -1.\]

Consider the operator $(\!-\!1)T_{P^2}\circ W_N$ on $S^{\mathrm{new}}(N)$ ; its trace is

\[\sum_{f\in H^{\mathrm{new}}(N)} a_f(P^2)\epsilon(f) = \sum_{f\in H^{\mathrm{new}}(N)} P \lambda_f(P^2)\epsilon(f).\]

More generally, if we consider $P^k$ Fourier coefficients, we apply the operator $(\!-\!1)T_{P^k} \circ W_N$ on $S^{\mathrm{new}}(N)$ and its trace is

\[\sum_{f\in H^{\mathrm{new}}(N)} a_f(P^k)\epsilon(f) = \sum_{f\in H^{\mathrm{new}}(N)} P^{\frac{k}{2}} \lambda_f(P^k)\epsilon(f).\]

On the other hand, when $P\mid N$ , we have the equality

\[\lambda_f(P^\nu) = \left(\frac{a_f(P)}{\sqrt{P}}\right)^\nu \text{ for all } \nu\geq 0.\]

Recall that $a_f(P)=\pm1$ so, for $\nu$ even we have

\[\lambda_f(P^{2\nu'}) = P^{-\nu'} \text{ for all } \nu'\geq 0.\]

In particular, when $\nu'=1$ , we have $\lambda_f(P^2) = {1}/{P}$ .

2·2. Trace formula

We record a result of Skoruppa–Zagier [ Reference Skoruppa and ZagierSZ88 , section 2] that will be useful for us. The following result is a special case of the formula in equation (7) of op. cit. The result there is more general but since we restrict ourselves to N square-free, the expression simplifies in our setting to the following

Theorem 2·1. With notation introduced above, in the case that $P\nmid N$ for P-power coefficients

\begin{align*}\sum_{f\in H^{\mathrm{new}}(N)} a_f(P^k)\epsilon(f) &= \sum_{f\in H^{\mathrm{new}}(N)} P^{\frac{k}{2}} \lambda_f(P^k)\epsilon(f) \\[5pt] &= \frac{H_1(-4P^kN)} {2} + \sum_{0\lt r\leq 2 \frac{P^{\frac{k}{2}}}{\sqrt{N}}} H_1(r^2N^2 - 4P^{k}N) - \left(\sum_{i=0}^k P^i\right).\end{align*}

In particular, when $k=2$

\begin{align*}\sum_{f\in H^{\mathrm{new}}(N)} a_f(P^2)\epsilon(f) &= \sum_{f\in H^{\mathrm{new}}(N)} P \lambda_f(P^2)\epsilon(f)\\[5pt] & = \frac{H_1(-4P^2N)} {2} + \sum_{0\lt r\leq 2P/\sqrt{N}} H_1(r^2N^2 - 4P^2N) - (1+P+P^2).\end{align*}

Remark 2·2. In the notation of [ Reference Skoruppa and ZagierSZ88 ], the quantity $n_1$ takes the value N and $n_2 =1$ . A priori, the second term should have a coefficient $U_0(r\sqrt{N}/2P^{k/2})$ but we note that $U_0(x)=1$ . The final term in the expression is $\sigma_1(P^{k}) = \sum_{d\mid P^{k}} d = 1 + P + P^2 + \cdots + P^k$ .

In the above formula, the Hurwitz class number $H_1(\!-\!d)$ is the number of equivalence classes with respect to $\mathrm{SL}_2(\mathbb{Z})$ of positive-definite binary quadratic forms of discriminant $-d$ weighed by the number of automorphisms (i.e., with forms corresponding to multiples of $x^2 +y^2$ and $x^2 +xy+y^2$ counted with multiplicities ${1}/{2}$ and ${1}/{3}$ , accordingly). In other words, $H_1$ can be expressed in terms of the (Gauss) class number h via:

\[H_1(\!-\!d) = \sum_{f\in \mathbb{Z}^{\gt0}, \,f^2\mid d} h(\!-\!d/f^2) +O(1),\]

with the error term disappearing if d is not of the form 3 or 4 times a square.

Using the relationship between the Hurwitz class number $H_1(\,\!\cdot\!\,)$ and class number $h(\,\!\cdot\!\,)$ explained above, we see that

\begin{align*}&H_1(\!-\!4P^kN) \\[5pt] &\quad = \begin{cases}\displaystyle{\sum_{j=0}^{k/2} h(\!-\!4P^{2(\frac{k}{2} - j)}N) + \sum_{j=0}^{k/2} h(\!-\!P^{2(\frac{k}{2} - j)}N)} + O(1) & \textrm{ when }k=\textrm{ even}\\[5pt] \displaystyle{\sum_{j=0}^{(k-1)/2} h(\!-\!4P^{2(\frac{k-1}{2} - j)}PN) + \sum_{j=0}^{(k-1)/2} h(\!-\!P^{2(\frac{k-1}{2} - j)}PN)} + O(1) & \textrm{ when }k=\textrm{ odd}.\end{cases}\end{align*}

Throughout this paper we assume that $P\neq 2$ . Since we assume that $\gcd(P,N)=1$ and that N is square-free, the square factors of $4P^{k}N$ are $1, 4, P^2, 4P^2, \ldots, P^{\lfloor k/2 \rfloor}, 4P^{\lfloor k/2 \rfloor}$ . The above expression simplifies to the following formula when $k=2$ (which is what is required for our main result)

\[H_1(\!-\!4P^2N) = h(\!-\!4P^2N) + h(\!-\!P^2N) + h(\!-\!4N) + h(\!-\!N) + O(1).\]

If q is any prime and $r\geq 1$ , we note that $q^2\mid N(r^2 N -4P^{k})$ is satisfied if $q^2\mid (r^2 N -4P^{k})$ or if $q\mid N$ and $q\mid 4P^{k}$ . The latter condition is satisfied precisely when $q=2$ (i.e., N is even). Writing $N=2N'$ (where N ^′ must now be odd), if $4d^2 \mid (r^2 N^2 -4P^{k}N)$ , we can note that

\[\frac{r^2 N^2 -4P^{k}N}{4d^2} = \frac{r^2 N'^2 -2P^{k}N'}{d^2}\]

is not a square modulo 4 and the corresponding class number vanishes. Indeed, we remind the reader that Dirchlet’s class number formula $h(\!-\!d)$ is defined for fundamental discriminants; in particular, $h(\!-\!d)$ is defined to be 0 otherwise. It will therefore be enough to consider those square-divisors which satisfy $d^2\mid (r^2N - 4P^{k})$ . In the general case, the trace formula becomes

\begin{align*}\sum_{f\in H^{\mathrm{new}}(N)} P \lambda_f(P^k)\epsilon(f) & = \displaystyle{\sum_{j=0}^{\lfloor k/2 \rfloor} h(\!-\!4P^{2(\frac{k}{2} - j)}N) + \sum_{j=0}^{\lfloor k/2 \rfloor} h(\!-\!P^{2(\frac{k}{2} - j)}N)}{2} - \left(\sum_{i=1}^k P^i\right) \\[5pt] & \quad + \sum_{0\lt r\leq 2\sqrt{\frac{P^k}{N}}} \sum_{d^2\mid r^2N - 4P^{k}} h(N(r^2N - 4P^{k})/d^2) + O(1).\end{align*}

When $k=2$ the trace formula further simplifies to

\begin{align*}\sum_{f\in H^{\mathrm{new}}(N)} P \lambda_f(P^2)\epsilon(f) & = \frac{h(\!-\!4P^2N) + h(\!-\!P^2N) + h(\!-\!4N) + h(\!-\!N)} {2} - (P+P^2) \\[5pt] & \quad + \sum_{0\lt r\leq 2P/\sqrt{N}} \sum_{d^2\mid r^2N - 4P^{2}} h(N(r^2N - 4P^{2})/d^2) + O(1).\end{align*}

3. Main term calculation via averages of class numbers I

The aim of this section is to estimate the Hurwitz class number $H_1(\!-\!4P^2N)$ which (as we have noted above) can be written as a sum of four class numbers.

Recall that if $-d\equiv 0 \text{ or }1\pmod{4}$ then the class number can be written in terms of a special value of an L-function. More precisely,

\[h(\!-\!d) = \frac{\sqrt{d}}{\pi} L(1, \chi_d),\]

where $\chi_d$ is the quadratic Dirichlet character of modulus d or 4d which can be calculated explicitly via the Kronecker symbol and $L(1, \chi_d)$ is the value at 1 of the Dirichlet series for the Kronecker symbol $( {d}/{\cdot})$ . The calculation in this section will be separated into two cases, first when $-d\equiv 1\pmod{4}$ and second when $-d\equiv 0\pmod{4}$ . The first calculation will allow us to estimate averages of $h(\!-\!P^2N)$ and $h(\!-\!N)$ and the second will allow us to estimate averages of $h(\!-\!4P^2N)$ and $h(\!-\!4N)$ .

When $\chi$ is a non-principal Dirichlet character of modulus d, for a truncation parameter T we know

\[L(1,\chi) = \sum_{n=1}^T \frac{\chi(n)}{n} + O\left(\frac{\sqrt{d} \log( d)}{T}\right).\]

The main result we want to prove in this section is an analogue of [ Reference ZubrilinaZub23 , proposition 3·1].

Proposition 3·1. Let $P\neq 2$ be a prime and let $[X,X+Y]$ be an interval of length $Y=o(X)$ . Then as $X\rightarrow \infty$ ,

\begin{align*}\frac{\zeta(2) \pi}{XY}\sideset{}{'}\sum_{\substack{N\in [X,X+Y] \\ P\nmid N}}\frac{h(\!-\!P^2 N)}{2} &+ \frac{h(\!-\! N)}{2} + \frac{h(\!-\!4P^2 N)}{2} + \frac{h(\!-\! 4N)}{2} \\[5pt] & = \frac{AP}{\sqrt{X}}+ O_\epsilon\left(\left(\frac{1}{\sqrt{X}}+\frac{P^{\frac{7}{6}}X^{\frac{1}{12} }}{Y^{\frac{5}{6}}}+\frac{PY}{X^{\frac32}}\right) (PXY)^\epsilon\right).\end{align*}

Here the error term $\rightarrow 0$ as $P,X\rightarrow\infty$ and $Y=o({X}/{\log(P^2 X)})$ . The notation $\sideset{}{'}\sum$ means that we are summing over all square-free N.

Proof. We prove that for a cut-off parameter T

\begin{align*}&\frac{\zeta(2) \pi}{XY}\sideset{}{'}\sum_{\substack{N\in [X,X+Y] \\ P\nmid N}} \frac{h(\!-\!P^2 N)}{2} + \frac{h(\!-\! N)}{2} + \frac{h(\!-\!4P^2 N)}{2} + \frac{h(\!-\! 4N)}{2}\\[5pt] &\quad = (3{\cdot}1) + (3{\cdot}2) + (3{\cdot}3) + (3.4) \\[5pt] &\quad = \frac{AP}{\sqrt{X}} + O_{\epsilon}\left(\frac{1}{\sqrt{X}} + \frac{P}{\sqrt{TX}} + \frac{PT^{\frac{1}{5}+\epsilon}X^{\frac{1}{10}+\epsilon}}{Y} + \frac{P Y\log( T)}{X^{\frac{3}{2}}} + \frac{P^2\log(P^2 X)}{T}\right).\end{align*}

Choosing $T = {(PY)^{{5}/{6} - \epsilon}}/{X^{{1}/{12}}}$ , the right-hand side of the above expression becomes

\[\frac{AP}{\sqrt{X}} + O_\epsilon\left(\frac{1}{\sqrt{X}}+\frac{P^{\frac{7}{12}}}{X^{\frac{11}{24}}Y^{\frac{5}{12}}}+ \frac{(PY)^{1+\epsilon}}{X^{\frac32 -\epsilon }} + \frac{P^{\frac{7}{6}+\epsilon}X^{\frac{1}{12} + \epsilon}}{Y^{\frac{5}{6}-\epsilon}}\right)\]

and the result follows.

3·1. When $-d \equiv 1\pmod{4}$

Throughout this section $P\neq 2$ and $[X, X+Y]$ is an interval of length $Y=o(X)$ . We provide the following estimate for the first two terms in Proposition 3·1.

\begin{align*}&\frac{\zeta(2)\pi}{XY}\sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} \frac{h(\!-\!P^2N)}{2} + \frac{h(\!-\!N)}{2} \\[5pt] & \quad = \frac{2A}{11} \frac{P}{\sqrt{X}} + O_\epsilon\left(\frac{1}{\sqrt{X}}+\frac{P^{\frac{7}{12}}}{X^{\frac{11}{24}}Y^{\frac{5}{12}}}+\frac{P^{\frac{7}{6} + \epsilon}}{X^{\frac{1}{12}- \epsilon}Y^{\frac{5}{6}- \epsilon}}+\frac{P^{\frac{7}{6}+\epsilon}X^{\frac{1}{12} + \epsilon}}{Y^{\frac{5}{6}-\epsilon}}\right).\end{align*}

Here, the main term contribution is from the estimates in Lemma 3·2. The leading term of the expression in Lemma 3·3 contributes ${1}/{\sqrt{X}}$ to the error term of this summation and subsumes the error term ${1}/{P\sqrt{X}}$ arising in Lemma 3·2.

3·1·1. Averages of $h(\!-\!P^2N)$

Lemma 3·2. With notation as introduced above,

\begin{equation*}\frac{\zeta(2)\pi}{XY}\sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} \frac{h(\!-\!P^2N)}{2} = \frac{2AP}{11\sqrt{X}} + O_\epsilon\left(\frac{1}{P\sqrt{X}}+\frac{P^{\frac{7}{12}}}{X^{\frac{11}{24}}Y^{\frac{5}{12}}}+\frac{(PY)^{1+\epsilon}}{X^{\frac32 -\epsilon }} + \frac{P^{\frac{7}{6}+\epsilon}X^{\frac{1}{12} + \epsilon}}{Y^{\frac{5}{6}-\epsilon}}\right).\end{equation*}

The first step is to obtain an expression for $({\zeta(2)\pi}/{XY})\sum h(\!-\!P^2N)$ in terms of a cut-off parameter T where the sum runs over square-free $N \in [X, X+Y]$ such that $P\nmid N$ . To ensure that the error-term is smaller than the main term, we choose the cut-off parameter appropriately. The proof of the lemma occupies the remainder of this section.

Proof. We calculate the averages in intervals using the class number formula, i.e.,

\begin{align*}&\frac{1}{P\sqrt{X}} \sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} h(\!-\!P^2N) \\[5pt] & \quad = \frac{1}{\pi} \sideset{}{'}\sum_{\substack{N\in [X, X+Y]\\ P\nmid N\\[5pt] N \equiv 3\pmod{4}}} \sqrt{\frac{N}{X}} L(1, \chi_{-P^2N})\\[5pt] & \quad = \frac{1}{\pi}\sideset{}{'}\sum_{\substack{N\in [X, X+Y]\\ P\nmid N \\[5pt] N \equiv 3\pmod{4}}} \sqrt{\frac{N}{X}} \sum_{n=1}^T \frac{\left( \frac{-P^2N}{n}\right)}{n} + O\left(\frac{P\sqrt{X}Y\log(P^2 X)}{T}\right)\\[5pt] & \quad = \frac{1}{\pi} \sideset{}{'}\sum_{\substack{N\in [X, X+Y]\\ P\nmid N \\[5pt] N \equiv 3\pmod{4}}} \sum_{m=1}^{\sqrt{T}} \frac{\sqrt{\frac{N}{X}}\left( \frac{-P^2N}{m^2}\right)}{m^2} + \frac{1}{\pi} \sideset{}{'}\sum_{\substack{N\in [X, X+Y]\\ P\nmid N \\[5pt] N \equiv 3\pmod{4}}} \sum_{\substack{n=1 \\ n\neq \square}}^T \frac{\sqrt{\frac{N}{X}} \left( \frac{-P^2N}{n}\right)}{n} \\[5pt] & \quad + O\left(\frac{P\sqrt{X}Y\log(P^2 X)}{T}\right),\end{align*}

where to show the second equality we use the fact that $\chi_{-P^2N}$ is always a non-principal character when N is square-free and $P\nmid N$ .

Next, we calculate the two (double) sums appearing in the above expression separately. Note that the first sum contains principal characters.

\begin{align*}\mathrm{Sq} &= \frac{1}{\pi}\sideset{}{'}\sum_{\substack{N\in [X, X+Y]\\ P\nmid N \\[5pt] N \equiv 3\pmod{4}}} \sum_{m=1}^{\sqrt{T}} \frac{\sqrt{\frac{N}{X}}\left( \frac{-P^2N}{m^2}\right)}{m^2} \\[5pt] & =\frac{1}{\pi}\sum_{m=1}^{\sqrt{T}} \frac{1}{m^2}\sum_{\substack{N\in [X, X+Y]\\ P\nmid N \\[5pt] N \equiv 3\pmod{4}}} \mu^2(N)\left( \frac{-P^2N}{m^2}\right)\left(1 + \left(\sqrt{1 + \frac{N-X}{X}} - 1\right) \right) \\[5pt] &= \frac{1}{\pi} \sum_{\substack{m=1\\ \gcd(P,m)=1}}^{\sqrt{T}} \frac{1}{m^2} \left(\sum_{N\in [X, X+Y]} \mu^2(N)\left( \frac{N}{m^2}\right)\frac{\chi_1(N) - \chi_2(N)}{2}\right) + O\left(\frac{Y^2}{X}\right)\\[5pt] &= \frac{1}{\pi}\sum_{\substack{m=1\\ \gcd(P,m)=1}}^{\sqrt{T}} \frac{Y}{\zeta(2)} \frac{\eta(2m)}{2m^2} + O_{\epsilon}\left( m^{\frac{1}{5}+\epsilon}X^{\frac{3}{5}+\epsilon}\frac{1}{m^2}\right) + O\left(\frac{Y^2}{X}\right) \\[5pt] &= \frac{4YA}{11\pi\zeta(2)} + O_{\epsilon}\left( \frac{Y}{P^2} + \frac{Y}{\sqrt{T}} + X^{\frac{3}{5}+\epsilon} +\frac{Y^2}{X}\right),\end{align*}

by Lemma 7·1 where $A\;:\!=\; \prod_{p}(1 + {p}/{(p+1)^2 (p-1)} )$ . Here, $\chi_1$ and $\chi_2$ are characters modulo 4, and $\chi_1$ is principal. The character $({N}/{m^2})\chi_1(N)$ is principal modulo 2m whereas, $({N}/{m^2})\chi_2(N)$ is always non-principal modulo 4m. Note that for the second last equality we use [ Reference ZubrilinaZub23 , lemma 6·7]. Next we calculate the non-square term in a manner identical to the one above.

\begin{align*} \mathrm{NSq} &= \frac{1}{\pi} \sideset{}{'}\sum_{\substack{N\in [X, X+Y]\\ P\nmid N \\ N \equiv 3\pmod{4}}} \sum_{\substack{n=1 \\ n\neq \square}}^T \frac{\sqrt{\frac{N}{X}} \left( \frac{-P^2N}{n}\right)}{n}\\[5pt] & = \frac{1}{\pi} \sideset{}{'}\sum_{\substack{N\in [X, X+Y]\\ P\nmid N \\[5pt] N \equiv 3\pmod{4}}} \sum_{\substack{n=1 \\ n\neq \square}}^{T} \frac{1}{n} \left( \frac{-P^2N}{n}\right)\left(1 + \left(\sqrt{1 + \frac{N-X}{X}} - 1\right) \right) \\[5pt] & \quad \leq \frac{1}{\pi} \sum_{\substack{n=1 \\ n\neq \square}}^{T} \sideset{}{'}\sum_{\substack{N\in [X, X+Y]\\ N \equiv 3\pmod{4}}} \frac{\left( \frac{-N}{n}\right)}{n} + O\left(\frac{Y^2\log( T)}{X} \right) \\[5pt] &=\frac{1}{\pi} \sum_{\substack{n=1 \\ n\neq \square}}^{T}\frac{\left( \frac{-1}{n}\right)}{n} \left( \sideset{}{'}\sum_{\substack{N\in [X, X+Y]}} \left(\frac{N}{n}\right) \frac{\chi_{1}(N)-\chi_2(N)}{2}\right) + O\left(\frac{Y^2\log( T)}{X} \right) \\ &=\frac{1}{\pi} \sum_{\substack{n=1 \\ n\neq \square}}^{T} \frac{1}{n} O_{\epsilon}\left(n^{\frac{1}{5}+\epsilon}X^{\frac{3}{5}+\epsilon}\right) + O\left(\frac{Y^2\log( T)}{X} \right) \\ & \quad \ll_{\epsilon} T^{\frac{1}{5}+\epsilon}X^{\frac{3}{5}+\epsilon} + \frac{Y^2\log( T)}{X}.\end{align*}

Since N is not a square, note that $({N}/{n})$ is non-principal. Since $({N}/{2})$ is primitive modulo 8, the characters $({N}/{n})\chi_1(N)$ and $({N}/{n})\chi_2(N)$ are non-principal. For the last equality, we use [ Reference ZubrilinaZub23 , lemma 6·7].

In conclusion we obtain that for a truncation parameter T,

(3·1) \begin{equation}\begin{split}&\frac{\zeta(2)\pi}{XY}\sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} \frac{h(\!-\!P^2N)}{2} = \frac{2AP}{11\sqrt{X}} \\ & \quad + O_\epsilon\left( \frac{1}{P \sqrt{X}} + \frac{P}{\sqrt{TX}} + \frac{PT^{\frac{1}{5}+\epsilon}X^{\frac{1}{10}+\epsilon}}{Y} + \frac{P Y\log( T)}{X^\frac{3}{2}} + \frac{P^2\log(P^2 X)}{T}\right).\end{split}\end{equation}

Choosing $T = {(PY)^{{5}/{6}}}/{X^{{1}/{12}}}$ , completes the proof of the lemma.

3·1·2. Averages of $h(\!-\!N)$

Lemma 3·3. With notation as before

\begin{align*}&\frac{\zeta(2)\pi}{XY}\sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} \frac{h(\!-\!N)}{2} \\ & \quad = \frac{2A}{11\sqrt{X}} + O_\epsilon\left(\frac{1}{(PY)^{\frac{5}{12}}X^{\frac{11}{24}}} + \frac{P^{\frac{1}{6} + \epsilon}X^{\frac{1}{12} + \epsilon}}{Y^{\frac{5}{6} - \epsilon}}+\frac{P^\epsilon Y^{1+\epsilon}}{X^{\frac{3}{2}-\epsilon}}+\frac{X^{\frac{1}{12}+\epsilon}}{P^{\frac{5}{6}-\epsilon}Y^{\frac{5}{6}-\epsilon}}\right).\end{align*}

Proof. We begin by calculating the average

\begin{align*}\frac{1}{\sqrt{X}} \sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} h(\!-\!N) &= \frac{1}{\pi}\sideset{}{'}\sum_{\substack{N\in [X, X+Y]\\ P\nmid N \\ N \equiv 3\pmod{4}}}\sqrt{\frac{N}{X}} L(1, \chi_{-N})\\ &= \frac{1}{\pi}\sideset{}{'}\sum_{\substack{N\in [X, X+Y]\\ P\nmid N \\ N \equiv 3\pmod{4}}} \sum_{m=1}^{\sqrt{T}} \frac{\sqrt{\frac{N}{X}}\left( \frac{-N}{m^2}\right)}{m^2} + \frac{1}{\pi}\sideset{}{'}\sum_{\substack{N\in [X, X+Y]\\ P\nmid N \\ N \equiv 3\pmod{4}}}\sum_{\substack{n=1 \\ n\neq \square}}^T \frac{\sqrt{\frac{N}{X}} \left( \frac{-N}{n}\right)}{n} \\ & \quad + O\left(\frac{\sqrt{X}Y\log( X)}{T}\right),\end{align*}

where to show the second equality we use the fact that $\chi_{-N}$ is always a non-principal character when N is square-free. We proceed as before to obtain the estimates. First we work with the square terms

\begin{align*}\mathrm{Sq} &= \frac{1}{\pi}\sideset{}{'}\sum_{\substack{N\in [X, X+Y]\\ P\nmid N \\ N \equiv 3\pmod{4}}} \sum_{m=1}^{\sqrt{T}} \frac{\sqrt{\frac{N}{X}}\left( \frac{-N}{m^2}\right)}{m^2}\\[5pt] &= \frac{1}{\pi}\sum_{m=1}^{\sqrt{T}} \frac{1}{m^2} \left(\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} \mu^2(N)\left( \frac{N}{m^2}\right)\frac{\chi_1(N) - \chi_2(N)}{2}\right) + O\left(\frac{Y^2}{X}\right)\\[5pt] &= \frac{1}{\pi}\sum_{m=1}^{\sqrt{T}} \frac{Y}{\zeta(2)} \frac{\eta(2m)}{2m^2} + O_{\epsilon}\left( \frac{m^{\frac{1}{5}+\epsilon}X^{\frac{3}{5}+\epsilon}}{m^2}\right) + O\left(\frac{Y^2}{X}\right)\\[5pt] &= \frac{1}{\pi}\frac{4YA}{11\zeta(2)} + O_{\epsilon}\left(\frac{Y}{\sqrt{T}} + X^{\frac{3}{5}+\epsilon} + \frac{Y^2}{X}\right) \text{by Lemma 7${\cdot}$1}.\end{align*}

Here, $\chi_1$ and $\chi_2$ are characters modulo 4, and $\chi_1$ is principal. The character $({N}/{m^2})\chi_1(N)$ is principal modulo 2m whereas, $({N}/{m^2})\chi_2(N)$ is non-principal modulo 4m. For the third equality we use [ Reference ZubrilinaZub23 , lemma 6·7]. Next we calculate the non-square terms.

\begin{align*} \mathrm{NSq} &= \frac{1}{\pi}\sum_{\substack{N\in [X, X+Y]\\ P\nmid N \\ N \equiv 3\pmod{4}}}\sum_{\substack{n=1 \\ n\neq \square}}^T \frac{\sqrt{\frac{N}{X}} \left( \frac{-N}{n}\right)}{n}\\[5pt] &=\frac{1}{\pi} \sum_{\substack{n=1 \\ n\neq \square}}^{T}\frac{\left( \frac{-1}{n}\right)}{n} \left( \sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} \left(\frac{N}{n}\right) \frac{\chi_{1}(N)-\chi_2(N)}{2}\right) + O\left(\frac{Y^2\log( T)}{X} \right) \\[5pt] &=\frac{1}{\pi} \sum_{\substack{n=1 \\ n\neq \square}}^{T} \frac{1}{n} O_{\epsilon}\left(n^{\frac{1}{5}+\epsilon}X^{\frac{3}{5}+\epsilon}\right) + O\left(\frac{Y^2\log(T)}{X} \right) \ll_{\epsilon} T^{\frac{1}{5}+\epsilon}X^{\frac{3}{5}+\epsilon} + \frac{Y^2\log( T)}{X}.\end{align*}

We remind the reader that since n is not a square, we have that $({N}/{n})$ is non-principal. Also, we know that $({N}/{2})$ is primitive modulo 8. Therefore, the characters $({N}/{n})\chi_1(N)$ and $({N}/{n})\chi_2(N)$ are both non-principal. We apply [ Reference ZubrilinaZub23 , lemma 6·7] to obtain the last equality.

In conclusion we obtain that for a truncation parameter T, we have

(3·2) \begin{equation}\frac{\zeta(2)\pi}{XY}\sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} \frac{h(\!-\!N)}{2} =\frac{2A}{11\sqrt{X}} + O_{\epsilon}\left( \frac{1}{\sqrt{TX}} + \frac{T^{\frac{1}{5}+\epsilon}X^{\frac{1}{10}+\epsilon}}{Y} + \frac{Y\log( T)}{X^{\frac{3}{2}}} + \frac{\log(X)}{T} \right).\end{equation}

As in Section 3·1·1, we choose $T = {(PY)^{\frac{5}{6}}}/{X^{{1}/{12}}}$ to obtain the expression in the statement of the lemma.

3·2. When $d \equiv 0\pmod{4}$

As before, we assume that $P\neq 2$ and that $[X, X+Y]$ is an interval of length $Y=o(X)$ . The calculations in Lemmas 3·4 and 3·5 will account for the last two terms in Proposition 3·1. More precisely,

\begin{align*}&\frac{\zeta(2)\pi}{XY}\sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} \frac{h(\!-\!4P^2N)}{2} + \frac{h(\!-\!4N)}{2} \\[5pt] & \quad = \frac{9A}{11} \frac{P}{\sqrt{X}} + O_\epsilon\left(\frac{1}{\sqrt{X}}+\frac{P^{\frac{7}{12}}}{X^{\frac{11}{24}}Y^{\frac{5}{12}}}+ \frac{(PY)^{1+\epsilon}}{X^{\frac32 -\epsilon }} + \frac{P^{\frac{7}{6}+\epsilon}X^{\frac{1}{12} + \epsilon}}{Y^{\frac{5}{6}-\epsilon}}\right).\end{align*}

3·2·1. Averages of $h(\!-\!4P^2N)$

Lemma 3·4. With notation as above

\begin{align*}&\frac{\zeta(2)\pi}{XY} \sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} \frac{h(\!-\!4P^2N)}{2} \\[5pt] & \quad = \frac{9AP}{11\sqrt{X}} + O_\epsilon\left(\frac{1}{P\sqrt{X}}+\frac{P^{\frac{7}{12}}}{X^{\frac{11}{24}}Y^{\frac{5}{12}}}+ \frac{(PY)^{1+\epsilon}}{X^{\frac32 -\epsilon }} + \frac{P^{\frac{7}{6}+\epsilon}X^{\frac{1}{12} + \epsilon}}{Y^{\frac{5}{6}-\epsilon}}\right).\end{align*}

Proof. As we have done previously, consider

\begin{align*}&\frac{1}{P\sqrt{X}} \sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} h(\!-\!4P^2N) \\[5pt] & \quad = \frac{2}{\pi} \sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} \sqrt{\frac{N}{X}} L(1, \chi_{-4P^2N})\\[5pt] & \quad = \frac{2}{\pi} \sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} \sqrt{\frac{N}{X}} \sum_{n=1}^T \frac{\left( \frac{-4P^2N}{n}\right)}{n} + O\left(\frac{P\sqrt{X}Y\log(P^2 X)}{T}\right)\\[5pt] & \quad =\frac{2}{\pi} \sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} \sqrt{\frac{N}{X}} \sum_{m=1}^{\sqrt{T}} \frac{\left( \frac{-4P^2N}{m^2}\right)}{m^2} + \frac{2}{\pi} \sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}}\sqrt{\frac{N}{X}} \sum_{\substack {n=1 \\ n\neq \square}}^T \frac{\left( \frac{-4P^2N}{n}\right)}{n}\\[5pt] & \qquad +O\left(\frac{P\sqrt{X}Y\log(P^2 X)}{T}\right).\end{align*}

As before, we estimate the two double sums separately. First we consider the square terms

\begin{align*}\mathrm{Sq} &=\frac{2}{\pi} \sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} \sqrt{\frac{N}{X}} \sum_{m=1}^{\sqrt{T}} \frac{\left( \frac{-4P^2N}{m^2}\right)}{m^2}\\[5pt] &=\frac{2}{\pi} \sum_{m=1}^{\sqrt{T}}\frac{1}{m^2} \sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} \left( \frac{-4P^2N}{m^2}\right)\mu^2(N)+O\left(\frac{Y^2}{X}\frac{1}{m^2}\right)\\[5pt] &=\frac{2}{\pi}\sum_{\substack{m=1 \\ \gcd(m,2P)=1}}^{\sqrt{T}}\frac{1}{m^2}\frac{Y}{\zeta(2)}\eta(m)+O_{\epsilon}\left(X^{\frac{3}{5}+\epsilon}m^{\epsilon-2}+\frac{Y^2}{X}\frac{1}{m^2}\right)\\[5pt] &=\frac{1}{\pi} \frac{18YA}{\zeta(2)11}+O_{\epsilon}\left( \frac{Y}{P^2}+\frac{Y}{\sqrt{T}} + X^{\frac{3}{5}+\epsilon}+\frac{Y^2}{X}\right) \text{by Lemma 7${\cdot}$1},\end{align*}

The second last equality follows from [ Reference ZubrilinaZub23 , lemma 6·7]. Next we estimate the other double sum to obtain

\begin{align*}\mathrm{NSq} &= \frac{2}{\pi} \sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} \sqrt{\frac{N}{X}} \sum_{\substack {n=1 \\ n\neq \square}}^T \frac{\left( \frac{-4P^2N}{n}\right)}{n}\\[5pt] &=\frac{2}{\pi}\sum_{\substack {n=1 \\ n\neq \square}}^T\frac{1}{n}\left(\frac{-4P^2}{n}\right)\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}}\left( \frac{N}{n}\right)\mu^2(N)+O\left(\frac{Y^2\log(T)}{X}\right)\\[5pt] &=\frac{2}{\pi}\sum_{\substack {n=1 \\ n\neq \square}}^T\frac{1}{n}\left(\frac{-4P^2}{n}\right) O_{\epsilon}\left(X^{\frac{3}{5}+\epsilon}n^{\frac{1}{5}+\epsilon}\right)+O\left(\frac{Y^2\log(T)}{X}\right)\\[5pt] & \quad \ll_{\epsilon} T^{\frac{1}{5}+\epsilon}X^{\frac{3}{5} + \epsilon}+\frac{Y^2\log(T)}{X}.\end{align*}

For a truncation parameter T, we have

(3·3) \begin{equation}\begin{split}&\frac{\zeta(2)\pi}{XY} \sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} \frac{h(\!-\!4P^2N)}{2} = \frac{9AP}{11\sqrt{X}} \\[5pt] & \quad + O_\epsilon\left( \frac{1}{P\sqrt{X}} + \frac{P}{\sqrt{TX}} + \frac{PT^{\frac{1}{5}+\epsilon}X^{\frac{1}{10}+\epsilon}}{Y} + \frac{P Y\log( T)}{X^{\frac{3}{2}}} + \frac{P^2\log(P^2 X)}{T}\right).\end{split}\end{equation}

The lemma follows by choosing $T={(PY)^{{5}/{6}}}/{X^{{1}/{12}}}$ .

3·2·2. Averages of $h(\!-\!4N)$

Lemma 3·5. With notation as before,

\begin{align*}&\frac{\zeta(2)\pi}{XY} \sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} \frac{h(\!-\!4N)}{2} = \frac{9A}{22\sqrt{X}} \\[5pt] & \quad + O_\epsilon\left(\frac{1}{(PY)^{\frac{5}{12}}X^{\frac{11}{24}}} + \frac{P^{\frac{1}{6} + \epsilon}X^{\frac{1}{12} + \epsilon}}{Y^{\frac{5}{6} - \epsilon}}+\frac{Y^{1+\epsilon}P^\epsilon}{X^{\frac{3}{2}-\epsilon}}+\frac{X^{\frac{1}{12}+\epsilon}}{Y^{\frac{5}{6}-\epsilon}P^{\frac{5}{6}-\epsilon}}\right).\end{align*}

Proof. The idea of the proof is exactly the same as before.

\begin{align*} \frac{1}{\sqrt{X}} \sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} h(\!-\!4N) &= \frac{2}{\pi}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}}\sqrt{\frac{N}{X}} L(1, \chi_{-4N})\\[5pt] &= \frac{2}{\pi} \sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} \sqrt{\frac{N}{X}} \sum_{n=1}^T \frac{\left( \frac{-4N}{n}\right)}{n} + O\left(\frac{\sqrt{X}Y\log( X)}{T}\right)\\[5pt] &=\frac{2}{\pi} \sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} \sqrt{\frac{N}{X}} \sum_{m=1}^{\sqrt{T}} \frac{\left( \frac{-4N}{m^2}\right)}{m^2} + \frac{2}{\pi} \sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} \sqrt{\frac{N}{X}} \sum_{\substack {n=1 \\ n\neq \square}}^T \frac{\left( \frac{-4N}{n}\right)}{n}\\[5pt] & \quad + O\left(\frac{\sqrt{X}Y\log( X)}{T}\right).\end{align*}

For the first double sum which are the ‘square terms’, we obtain the following expression where we note that the sum is over all m that is odd

\begin{align*} \mathrm{Sq} &= \frac{2}{\pi} \sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} \sqrt{\frac{N}{X}} \sum_{m=1}^{\sqrt{T}} \frac{\left( \frac{-4N}{m^2}\right)}{m^2} \\[5pt] &=\frac{2}{\pi} \sum_{\substack{m=1\\ \gcd(m,2)=1}}^{\sqrt{T}}\frac{1}{m^2}\frac{Y}{\zeta(2)}\eta(m)+O_{\epsilon}\left(\frac{X^{\frac{3}{5} + \epsilon}}{m^{2-\epsilon}} + \frac{Y^2}{X}\frac{1}{m^2}\right)\\[5pt] & = \frac{9YA}{11\pi \zeta(2)}+O_{\epsilon}\left( \frac{Y}{\sqrt{T}} + X^{\frac{3}{5}+\epsilon} + \frac{Y^2}{X}\right) \text{by Lemma 7${\cdot}$1}.\end{align*}

The second double sum which are the ‘non-square’ terms can be estimated as before and we obtain

\begin{align*}\mathrm{NSq} & = \frac{2}{ \pi} \sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} \sqrt{\frac{N}{X}} \sum_{\substack {n=1 \\ n\neq \square}}^T \frac{\left( \frac{-4N}{n}\right)}{n}\ll_{\epsilon}\left( T^{\frac{1}{5}+\epsilon}X^{\frac{3}{5}+\epsilon} + \frac{Y^2\log(T)}{X}\right).\end{align*}

Therefore,

(3·4) \begin{equation}\frac{\zeta(2)\pi}{XY} \sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} \frac{h(\!-\!4N)}{2} = \frac{9A}{22\sqrt{X}} + O_{\epsilon}\left( \frac{1}{\sqrt{TX}} + \frac{T^{\frac{1}{5}+\epsilon}X^{\frac{1}{10}+\epsilon}}{Y} + \frac{Y\log( T)}{X^{\frac{3}{2}}} + \frac{\log(X)}{T} \right).\end{equation}

The result follows from choosing $T = (PY)^{{5}/{6}}/{X^{{1}/{12}}}$ .

Remark 3·6. The main term contribution obtained from calculations in this section would remain unchanged if we worked with P-power coefficients rather than $P^2$ -coefficients.

4. Remainder analysis

In this section, we work under the assumption that $P\neq 2$ is a prime, r is a positive integer, and $X\gt Y\gt0$ satisfies $r^2(X+Y)\lt 4P^2$ . Given positive integers r and d, define the set

\begin{align*} & \mathcal{A}_{r,d}=\\&\!\bigg\{\! N\in \mathbb{Z} \ \Big\vert \ {N \text{ square-free}, \ \gcd(P,N)=1, \ d^2\mid r^2N-4P^2, \ \frac{r^2N^2-4P^2N}{d^2} \equiv 0\ \text{or }1\!\! \pmod 4}\bigg\}.\end{align*}

First, we calculate the size of the above set. We do so by breaking the calculation into several cases:

1. (r odd) Assume that r is odd. Then $r^2N-4P^2$ is not divisible by 4 as N is square-free. Thus, any divisor d satisfying $d^2\mid (r^2N-4P^2)$ has to be odd as well.

   Observe that,

   \[ r^2 N^2 - 4P^2 N\equiv 0 \text{ or } 1\pmod 4. \]
   As d is odd, it suffices to determine whether we can solve the congruence
   \[ r^2N\equiv 4P^2\pmod{d^2}. \]
   There is a unique solution when $\gcd(d,r)=\gcd(d,2)=1$ and no solutions otherwise.

2. (r even) Assume that r is even.

   Claim. $\gcd(P,d)=\gcd(P,r)=1$ .

   Justification. By definition $r^2N\le 4P^2$ . As $N\ge X\gt4$ , we deduce that $r\lt P$ . If $P\mid d$ , we obtain that $P^2\mid r^2N$ . As N is square-free, this implies $P\mid r$ which is impossible. This proves the claim.

   Let us write $r=2\ell$ . The condition

   \[ \frac{r^2N^2-4P^2N}{d^2}\equiv 0\text{ or }1\pmod{4} \]
   is equivalent to the existence of an integer k such that $4\ell^2 N - 4P^2=kd^2$ and $kN\equiv 0 \text{ or } 1\pmod {4}$ . This calculation can be further divided into two cases.

3. (d odd) If d is odd, then $4\mid k$ . Let $k=4k'$ . Thus, we look for an N that satisfies

   \[ \ell^2 N - P^2=k'd^2 \]
   which is equivalent to $\ell^2N\equiv P^2 \pmod{d^2}$ . This congruence has exactly one solution if $\gcd(\ell,d)=1$ and zero solutions otherwise.

4. (d even) Assume that $d=2b$ is even. As N is square-free it is either odd or congruent to 2 modulo 4.

   If N is even, then $4\ell^2N$ and $4kb^2$ have to be divisible by 8; but $8 \nmid 4P^2$ .

   So N must be odd if d is even. As $kN\equiv 0 \text{ or }1\pmod{4}$ we will distinguish two more cases

1. (4 | k) In this situation we try to solve

   \[ \ell^2 N - P^2\equiv 0\mod {4b^2}. \]
   This congruence has a solution if and only if $\ell$ is coprime to 2b and no solutions otherwise.

2. (4 k) We now consider the case that $N\equiv k^{-1}\equiv k\pmod 4$ . Writing $k=N+4k'$ we obtain

   \[ \ell^2N-P^2=Nb^2 + 4k'b^2. \]
   This is equivalent to $N(\ell^2-b^2)\equiv P^2 \pmod{4b^2}$ . This congruence has a solution if $\ell$ and b are coprime and exactly one of them is even. In all other cases there are no solutions.

In summary we obtain that

\[\mathcal{A}_{r,d}=\left\{N\in \mathbb{Z} \ \Big\vert \ N \text{ square-free}, \ \gcd(P,N)=1, \ N\pmod{d^2}\in \mathcal{R}_{r,d}\right\},\]

where

\[|\mathcal{R}_{r,d}| =\begin{cases}1 & \text{ if } (r,d)=(d,2)=1 \\[5pt] 1 & \text{ if }r \text{ even}, \ (r,d)=1 \\[5pt] 1 & \text{ if } (r,d)=2, \ 4\nmid d\\[5pt] 2 & \text{ if } r \text{ even}, \ (r,d)=2, \ 4\mid d \\[5pt] 0 & \text{ otherwise.}\end{cases}\]

All pairs (r, d) for which $\mathcal{R}_{r,d} \neq \emptyset$ will henceforth be called an admissible pair. The two elements in the fourth point appear as we get one solution in case (r even)–(d even)–( $4 \mid k$ ) and one solution for the case (r even)–(d even)–( $4 \nmid k$ ).

5. Main term calculation via averages of class numbers II

Throughout this section we assume that $P^2\ll X^{1+\delta_2}$ .

For N a square-free positive integer and P a prime satisfying $\gcd(P,N)=1$ , the goal of this section is to calculate $\sum H_1(r^2N^2 - 4P^2N)$ with the sum of running over $0\lt r\leq 2P/\sqrt{X+Y}$ . As observed before, the summation can be expressed as a double sum of class numbers which in turn can be written explicitly in terms of special values of L-functions. More precisely, for a divisor $d^2\mid (r^2 N - 4P^2)$ satisfying $({r^2 N^2 - 4P^2 N})/{d^2} \equiv 0 \text{ or }1\pmod{4}$ , the class number formula asserts that

\[h\left( \frac{r^2 N^2 - 4P^2 N}{d^2} \right) = \frac{\sqrt{4P^2 N - r^2 N^2}}{\pi d} L(1, \chi_{\frac{r^2 N^2 - 4P^2 N}{d^2}}).\]

Therefore, as r varies in the range 1 to $ 2P/\sqrt{X+Y}$ , we have

\[\sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} H_1(r^2 N^2 - 4P^2 N) = \frac{1}{\pi}\sum_{d\lt 2P} \sum_{\substack{N\in [X,X+Y]\\ N\in \mathcal{A}_{r,d}}} \frac{L\left(1, \chi_{\frac{r^2 N^2 - 4P^2 N}{d^2}}\right)}{d} \sqrt{4P^2 N - r^2 N^2}.\]

Note that

\begin{align*}\sqrt{4P^2N-r^2N^2}-\sqrt{4P^2X-r^2X^2}\ll P\frac{Y}{\sqrt{X}}+r\sqrt{Y}\sqrt{X}.\end{align*}

For a non-trivial character $\chi$ of conductor q, we know by Siegel’s bound that $|{L(1, \chi)}| \ll \log(q)$ (see for example [ Reference Friedlander and IwaniecFI18 , p. 2]). Therefore,

(5·1) \begin{equation}\begin{split} \sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} H_1(r^2 N^2 - 4P^2 N) &= \frac{1}{\pi}\sum_{d\lt 2P} \sum_{\substack{N\in [X,X+Y]\\ N\in \mathcal{A}_{r,d}}} \frac{L\left(1, \chi_{\frac{r^2 N^2 - 4P^2 N}{d^2}}\right)}{d} \sqrt{4P^2 X - r^2 X^2}\\[5pt] & \quad + O\left(P^{1+\epsilon}X^{-\frac{1}{2}+\epsilon}Y^2 + rP^{\epsilon}X^{\frac{1}{2}+\epsilon}Y^{\frac{3}{2}}\right).\end{split}\end{equation}

We now truncate the main term and obtain

where

\[S_{n,d,r} \;:\!=\;\sum_{\substack{N\in [X,X+Y]\\ N\in \mathcal{A}_{r,d}}}\mu^2(N)\left(\frac{N}{n}\right)\left(\frac{(r^2N-4P^2)/d^2}{n}\right) = \sum_{\substack{N\in [X,X+Y]\\ N\in \mathcal{A}_{r,d}}}\left(\frac{N}{n}\right)\left(\frac{(r^2N-4P^2)/d^2}{n}\right).\]

The last equality follows from the fact that $\mu^2(N)=1$ . Indeed, $N\in \mathcal{A}_{r,d}$ forces N to be always square-free. We can now state the main result of this section; its proof will occupy the remainder of this section.

Proposition 5·1. Let $P\neq 2$ be a prime, let r be a positive integer, and let $X\gt Y\gt0$ be such that $4P^2 \gt r^2(X+Y)$ . Then

\begin{equation*}\begin{split} & \sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} H_1(r^2 N^2 - 4P^2 N) \\[5pt] & \quad = \frac{Y\sqrt{4P^2 X - r^2X^2}}{\zeta(2)\pi}\prod_{p}\frac{p^4 - 2p^2 - p +1}{(p^2 - 1)^2}\prod_{p\mid r}\left( 1+ \frac{p^2}{p^4 - 2p^2 - p + 1}\right) \\[5pt] & \qquad + O\left( \left(PXY\right)^\epsilon\left((P^2 XY)^{\frac{3}{5}} + \frac{P Y^2}{\sqrt{X}} + r\sqrt{X}Y^{\frac{3}{2}} + PXY^{\frac{5}{18}} + P\sqrt{X}Y^{\frac{8}{9}}\right)\right).\end{split}\end{equation*}

Proof. In Proposition 5·6 we show that

\begin{equation*}\begin{split}\sum_{d \leq 2P} \sum_{n\leq T} \frac{S_{n,d,r}}{nd} &= \frac{Y}{\zeta(2)}\prod_{p}\frac{p^4 - 2p^2 - p +1}{(p^2 - 1)^2}\prod_{p\mid r}\left( 1+ \frac{p^2}{p^4 - 2p^2 - p + 1}\right) \\[5pt] & + O\left(\sqrt{X}Y^{\frac{5}{18}} + T^{\frac{1}{4}+\epsilon}\sqrt{Y} + Y^{\frac{8}{9}}\log(T) \right).\end{split}\end{equation*}

The simplified expression of (5·1) combined with the aforementioned proposition implies that

\begin{equation*}\begin{split} &\sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} H_1(r^2 N^2 - 4P^2 N) \\[5pt] & \quad = \frac{Y\sqrt{4P^2 X - r^2X^2}}{\zeta(2)\pi}\prod_{p}\frac{p^4 - 2p^2 - p +1}{(p^2 - 1)^2}\prod_{p\mid r}\left( 1+ \frac{p^2}{p^4 - 2p^2 - p + 1}\right) \\[5pt] &\qquad + O\left(\frac{P^2XY\log(P^2 X)}{T}+\frac{P^{1+\epsilon}Y^2}{X^{\frac{1}{2}-\epsilon}}+rP^{\epsilon}X^{\frac{1}{2}+\epsilon}Y^{\frac{3}{2}}\right)\\[5pt] &\qquad + O\left(PXY^{\frac{5}{18}} + PT^{\frac{1}{4}+\epsilon}\sqrt{X}\sqrt{Y} + P^{1} X^{\frac{1}{2}} Y^{\frac{8}{9}}\log(T) \right).\end{split}\end{equation*}

Now, by choosing $T = (P^2XY)^{\frac25}$ gives an error term of

\begin{align*} \ O\left((PXY)^\epsilon\left((P^2 XY)^{\frac{3}{5}} +\frac{PY^2}{\sqrt{X}} + r\sqrt{X}Y^{\frac{3}{2}} + PXY^{\frac{5}{18}} + P X^{\frac{1}{2}} Y^{\frac{8}{9}}\right)\right).\end{align*}

Corollary 5·2. Let $P\neq 2$ be a prime, let r be a positive integer, and let $X\gt Y\gt0$ be such that $4P^2 \gt r^2(X+Y)$ for each $r\lt{2P}/{\sqrt{X}}$ . Then

\begin{align*}&\frac{\zeta(2) \pi}{XY} \sum_{r\lt \frac{2P}{\sqrt{X}}} \sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} H_1(r^2 N^2 - 4P^2 N) \\[5pt] & \quad = \sum_{r\le \frac{P}{\sqrt{X}}} \left(\sqrt{\frac{4P^2}{X} - r^2}\right)\prod_{p}\frac{p^4 - 2p^2 - p +1}{(p^2 - 1)^2}\prod_{p\mid r}\left( 1+ \frac{p^2}{p^4 - 2p^2 - p + 1}\right) \\[5pt] & \qquad + O\left((PX)^\epsilon\left(\frac{P^{\frac{11}{5}}}{X^{\frac{9}{10}}Y^{\frac25}}+\frac{P^2Y}{X^2}+\frac{P^2Y^{\frac12}}{X^{\frac32}} + \frac{P^2}{X^{\frac12}Y^{\frac{13}{18}}}+\frac{P^2}{XY^{\frac19}}\right)\right).\end{align*}

Comparing the conditions from each of the error terms, we see that the most restrictive one arises from the last term. It yields the relation ${\delta_2}/{2}+ {\delta}/{9} \lt {1}/9$ . So a strict condition would be to chooseFootnote ² $\delta_2 \lt \delta \lt {2}/{11}$ . A more relaxed bound on $\delta$ is mentioned in the statement of the main theorem.

The main task is now to estimate $S_{n,d,r}$ .

5·1. Some preliminary calculations

Let a be an integer such that $a\pmod{d^2}\in \mathcal{R}_{r,d}$ . The character $({(r^2x-4P^2)/d^2})/{n}$ is a character modulo $fnd^2$ where $f=4$ if n is even and $f=1$ otherwise. Thus, we have

\begin{align*}&\sum_{\substack{N\in [X,X+Y]\\ P\nmid N\\ N\equiv a\pmod {d^2}}}\mu^2(N)\left(\frac{N}{n}\right)\left(\frac{(r^2 N-4P^2)/d^2}{n}\right) \\[5pt] & \quad = \sum_{\substack{b \pmod {fd^2n}\\ a\equiv b \pmod{d^2}}}\left(\frac{b}{n}\right)\left(\frac{(r^2b-4P^2)/d^2}{n}\right)\sum_{\substack{N\in [X,X+Y]\\ \gcd(P,N)=1\\ N\equiv b\pmod{fnd^2}}}\mu^2(N).\end{align*}

Note that $P^2\gt{X}/{4}$ and throughout our calculations we also have that $P\gt({d}/{2})$ . If $\gcd(P,fnd^2)\gt 1$ , then $P\mid n$ and $({N}/{n})=0$ for all N satisfying $P\mid N$ . If $\gcd(P,fnd^2)=1$ , then the number of elements in $[X,X+Y]$ that are divisible by P and congruent to $b\pmod{fnd^2}$ are $O({X}/{fd^2nP})$ . Thus, deleting the condition $P\nmid N$ in the inner sum gives

\[\sum_{\substack{b \pmod {fd^2n}\\ a\equiv b \pmod{d^2}}}\left(\frac{b}{n}\right)\left(\frac{(r^2b-4P^2)/d^2}{n}\right) \left( \sum_{\substack{N\in [X,X+Y]\\ N\equiv b\pmod{fnd^2}}}\mu^2(N) + O\left(\frac{X}{fd^2nP}\right)\right).\]

Now using a result of C. Hooley (see also [ Reference ZubrilinaZub23 , lemma 6·4]) the above expression can be rewritten as

\[\frac{Y\eta(d^2n)}{\zeta(2)f\varphi(d^2n)} \sum_{\substack{b \pmod {fd^2n}\\ a\equiv b \pmod{d^2}}}\left(\frac{b}{n}\right)\left(\frac{(r^2b-4P^2)/d^2}{n}\right)+O\left(\sqrt{Xn}/d+d^{1+\epsilon}n^{\frac{3}{2}+\epsilon}+\frac{X}{Pd^2}\right).\]

Definition 5·3. Define the functions

\begin{align*} \theta_r(m) & =\sum_{a\pmod{m}}\left(\frac{a}{m}\right)\left(\frac{ar^2-4P^2}{m}\right)\\[5pt] \widetilde{\varphi}_{r,d}(g) & =\sum_{\substack{a \pmod{d^2g}\\ a\pmod{d^2}\in \mathcal{R}_{r,d}}}\left(\frac{a}{g}\right)\left(\frac{ar^2-4P^2}{g}\right).\end{align*}

Lemma 5·4. Let $g=(d^\infty, n)$ and set $n'=n/g$ . Then

\[\sum_{\substack{b \pmod {fd^2n}\\ b \pmod{d^2}\in \mathcal{R}_{r,d}}} \left( \frac{b}{n} \right) \left(\frac{(r^2b-4P^2)/d^2}{n} \right) = f\widetilde{\varphi}_{r,d}(g)\theta_r(n').\]

Proof. Assume first that $f\mid d^2$ (i.e either $f=1$ or d is even). Then $(d^\infty,fn)=fg$ and moreover, $2\mid g$ if $f=4$ . Here, we recall that $f=4$ precisely when n is even.

\begin{align*}& \sum_{\substack{b \pmod {fd^2n}\\ b \pmod{d^2}\in \mathcal{R}_{r,d}}}\left(\frac{b}{n}\right)\left(\frac{(r^2b-4P^2)/d^2}{n}\right)=\sum_{a\in \mathcal{R}_{r,d}}\hspace{1mm} \sum_{\substack{b \pmod {fd^2n}\\ a\equiv b \pmod{d^2}}}\left(\frac{b}{n}\right)\left(\frac{(r^2b-4P^2)/d^2}{n}\right)\\[5pt] &=\left(\sum_{b \pmod{n'}}\left(\frac{b}{n'}\right)\left(\frac{(r^2b-4P^2)/d^2}{n'}\right)\right)\left(\sum_{a\in \mathcal{R}_{r,d}}\sum_{\substack{b\pmod{d^2gf}\\ b\equiv a\pmod{d^2}}}\left(\frac{b}{gf}\right)\left(\frac{(r^2b-4P^2)/d^2}{gf}\right)\!\!\right)\\[5pt] &=\theta_r(n')\widetilde{\varphi}_{r,d}(fg).\end{align*}

If $f=1$ , this is the desired result. If $f=4$ , then g and d are even and $\widetilde{\varphi}_{r,d}(fg)=f\widetilde{\varphi}_{r,d}(g)$ .

It remains to consider the case that n is even and d is odd.

\begin{align*} & \sum_{\substack{b \pmod {fd^2n}\\ b \pmod{d^2}\in \mathcal{R}_{r,d}}}\left(\frac{b}{n}\right)\left(\frac{(r^2b-4P^2)/d^2}{n}\right)=\sum_{a\in \mathcal{R}_{r,d}}\hspace{1mm} \sum_{\substack{b \pmod {fd^2n}\\ a\equiv b \pmod{d^2}}}\left(\frac{b}{n}\right)\left(\frac{(r^2b-4P^2)/d^2}{n}\right)\\[5pt] &=\left(\sum_{b \pmod{fn'}}\left(\frac{b}{fn'}\right)\left(\frac{(r^2b-4P^2)/d^2}{fn'}\right)\right)\left(\sum_{a\in \mathcal{R}_{r,d}}\sum_{\substack{b\pmod{d^2g}\\ b\equiv a\pmod{d^2}}}\left(\frac{b}{g}\right)\left(\frac{(r^2b-4P^2)/d^2}{g}\right)\right)\\[5pt] &=\widetilde{\varphi}_{r,d}(g)\theta_r(fn').\end{align*}

As n ^′ and f are even in this case, we obtain $\theta_r(fn')=f\theta_r(n')$ finishing the proof.

Lemma 5·5. Let (r, d) be an admissible pair. Then

\[\widetilde{\varphi}_{r,d}(g)=\begin{cases} \varphi(g)\delta_{g=\square} \quad & 2\nmid g, \ 4\nmid d\\[5pt] 2\varphi(g)\delta_{g=\square} \quad &2\nmid g, \ 4\mid d\\[5pt] 2\varphi(g)\delta_{g=\square} \quad & 2\mid g, \ 4\mid r, \ \gcd(d,r)=2\\[5pt] 2\varphi(g)\delta_{g=\square} \quad & 2\mid g, \ 4\mid d, \ \gcd(d,r)=2\\[5pt] 0\quad & \text{else}. \end{cases}\]

Let p be a prime coprime to P. The function $\theta_r$ is a multiplicative function satisfying

\[\theta_r(p^\alpha)=\begin{cases} -p^{\alpha-1}\quad & 2\neq p, \ 2\nmid \alpha, \ p\nmid r\\[5pt] p^{\alpha-1}(p-2)\quad & 2\neq p, \ 2\mid \alpha, \ p\nmid r\\[5pt] 0\quad & p\mid r, \ 2\neq p, \ 2\nmid \alpha \text{ or } p=2, \ 2\mid r\\[5pt] p^{\alpha-1}(p-1)\quad & p\mid r, \ p\neq 2, \ 2\mid \alpha\\[5pt] (\!-\!1)^\alpha2^{\alpha-1} \quad & p=2, \ 2\nmid r.\end{cases}\]

Proof. The definition of the functions $\theta$ and $\varphi$ differ from the ones given in [ Reference ZubrilinaZub23 ]. Nevertheless, the proofs of Lemmas 6·2 and 6·3 in loc. cit. still apply with minimal changes (substitute P by $P^2$ ).

In nutshell, the calculation we have done so far yields

\begin{align*}S_{n,d,r} & \;:\!=\; \sum_{\substack{N\in [X,X+Y]\\ N\in \mathcal{A}_{r,d}}}\mu^2(N)\left(\frac{N}{n}\right)\left(\frac{(r^2N-4P^2)/d^2}{n}\right)\\[5pt] & = \frac{Y\eta(d^2n)}{\zeta(2)\varphi(d^2n)} \widetilde{\varphi}_{r,d}(g)\theta_r(n') + O\left(\sqrt{Xn}/d+d^{1+\epsilon}n^{\frac{3}{2}+\epsilon}+\frac{X}{Pd^2}\right).\end{align*}

This is an approximation of $S_{n,d,r}$ in terms of multiplicative functions which are easier to manipulate. In the remainder of this section we will prove the following result.

Proposition 5·6. Let d and r be positive integers and let $X\gt Y\gt0$ . Let P be a prime satisfying $4P^2 \gt r^2(X+Y)$ . Then for a cut-off parameter $T\gg Y$

\begin{equation*}\begin{split}\sum_{d \leq 2P} \sum_{n\leq T} \frac{S_{n,d,r}}{nd} &= \frac{Y}{\zeta(2)}\prod_{p}\frac{p^4 - 2p^2 - p +1}{(p^2 - 1)^2}\prod_{p\mid r}\left( 1+ \frac{p^2}{p^4 - 2p^2 - p + 1}\right) \\[5pt] & \quad + O\left(\sqrt{X}Y^{\frac{5}{18}} + \sqrt{Y}T^{\frac{1}{4}+\epsilon} + Y^{\frac{8}{9}}\log(T) \right).\end{split}\end{equation*}

Proof. The equality is obtained by combining the results obtained in Propositions 5·8 and 5·9, and the calculation in Section 5·4.

5·2. Small d and small n

In this section, we estimate the sum of ${S_{n,d,r}}/{nd}$ in the range $n\leq Y^\sigma$ and $d\leq Y^\tau$ (where $\sigma, \tau$ are parameters in the interval [0,1]) by explicitly examining equidistribution of square-free numbers in residue classes modulo n.

Lemma 5·7. Let $\sigma, \tau$ be parameters with $0\leq \sigma,\tau \leq 1$ . Let $d,n,r\geq 1$ be integers.

\begin{align*}\sum_{\substack{n\le Y^\sigma \\ d\le Y^\tau}}\frac{\eta(d^2n)\tilde{\varphi}_{n,d}(r)\theta_r(n')}{\varphi(d^2n)nd} &= \prod_{p}\frac{p^4 - 2p^2 - p +1}{(p^2 - 1)^2}\prod_{p\mid r}\left( 1+ \frac{p^2}{p^4 - 2p^2 - p + 1}\right)\\[5pt] & \quad + O\left(Y^{-2\tau}+Y^{-\frac{\sigma}{5}}\right).\end{align*}

Proof. The proof follows as in [ Reference ZubrilinaZub23 , lemma 6·6]. Note that we can apply the computations in loc. cit. because of Lemma 5·5.

For the ease of notation, set

\[Bc(r) \;:\!=\; \prod_{p}\frac{p^4 - 2p^2 - p +1}{(p^2 - 1)^2}\prod_{p\mid r}\left( 1+ \frac{p^2}{p^4 - 2p^2 - p + 1}\right).\]

Proposition 5·8. Let $\sigma, \tau$ be parameters with $0\leq \sigma,\tau \leq 1$ . Let $d,n, r\geq 1$ be integers such that $P\nmid n$ , $4P^2 \gt r^2(X+Y)$ , and (r,d) is an admissible pair. Then

\[\sum_{\substack{n\le Y^\sigma \\ d\le Y^\tau}}\frac{S_{n,d,r}}{nd}=\frac{YBc(r)}{\zeta(2)} + O\left(\sqrt{X}Y^{\frac{\sigma}{2}}+Y^{\tau+\epsilon+\frac{3}{2}\sigma}+Y^{1-2\tau}+Y^{1-\frac{\sigma}{5}}\right).\]

Proof. With notation introduced above, we have

\begin{align*}\sum_{\substack{n\le Y^\sigma \\ d\le Y^\tau}}\frac{S_{n,d,r}}{nd} & = \sum_{\substack{n\le Y^\sigma \\ d\le Y^\tau}} \left(\frac{Y\eta(d^2n)\tilde{\varphi}_{r,d}(g)\theta_r(n')}{\zeta(2)\varphi(d^2n)nd}+O\left(\frac{\sqrt{Xn}}{nd^2}+\frac{d^{1+\epsilon}n^{\frac{3}{2}+\epsilon}}{nd}+\frac{X}{Pd^3n}\right)\right)\\[5pt] & = \frac{Y}{\zeta(2)} \left( Bc(r)+O\left(Y^{-2\tau}+Y^{-\frac{\sigma}{5}}\right) \right) + \sum_{\substack{n\le Y^\sigma \\ d\le Y^\tau}} O\left(\frac{\sqrt{X}}{d^2\sqrt{n}} + d^{\epsilon}n^{\frac{1}{2}+\epsilon} + \frac{X}{Pd^3n}\right) \\[5pt] & \quad\text{ by Lemma 5${\cdot}$7} \\[5pt] & = \frac{YBc(r)}{\zeta(2)}+O\left(\sqrt{X}Y^{\frac{\sigma}{2}}+Y^{\tau+\epsilon+\frac{3}{2}\sigma}+Y^{1-2\tau}+Y^{1-\frac{\sigma}{5}}+\frac{X}{P}\log(Y^\sigma)\right).\end{align*}

Recall that $P^2 \gt {X}/{4}$ . Equivalently, we have that ${X}/{P} \lt 2\sqrt{X}$ . The claim follows immediately.

5·3. Small d and large n

The main result of this section is the following. The proof of [ Reference ZubrilinaZub23 , proposition 3·9] goes through almost verbatim even in our setting. We give a sketch of the proof to show that the shape of the error term is the same.

Proposition 5·9. Let P be an odd prime and d,r be positive integers such that (d,r) is an admissible pair. Let $X\gt Y\gt0$ satisfying $(X+Y)^2 \lt 4P^2$ . For parameters $\sigma, \tau$ between 0 and 1,

\[\sum_{\substack{n\in [Y^\sigma, T] \\ d\lt Y^\tau} }\frac{S_{n,d,r}}{nd} \ll \sqrt{X}\log( T) \log( Y) + Y^{1-\frac{\sigma}{2}+\epsilon} + T^{\frac{1}{4}+\epsilon}\sqrt{Y}\log( Y) \text{ as } X\rightarrow\infty.\]

Lemma 5·10 ([ Reference ZubrilinaZub23 , lemma 3·12]). Let $Y\leq X$ , let n,w, D be positive integers and q be any integer. Let a be an integer satisfying $wa \equiv q\pmod{D}$ and $\gcd(a,D)=1$ . Define the set

\[\mathcal{P} = \left\{ p\mid n \text{ odd } : \ 2\nmid \mathrm{val}_p(n), \ p\nmid \gcd(q,n), \text{ and } \gcd(D,w,p)=1 \right\}.\]

Then

\[\sum_{\substack{N\in[X,X+Y] \\ N\equiv{a\pmod{D}}}} \mu^2(N)\left( \frac{N}{n}\right)\left( \frac{\frac{wN-q}{D}}{n}\right) \ll \sqrt{X} + \frac{Y}{D\prod_{p\in \mathcal{P}} \frac{\sqrt{p}}{2}} + \frac{\log( Y)}{\sqrt{D}}\left( \sqrt{\frac{nY}{\prod_{p\in \mathcal{P}} \frac{\sqrt{p}}{2}}}\right).\]

Proof of Proposition 5·9. Choose $w=r^2$ , $D=d^2$ , $q=4P^2$ in the above lemma, and choose a to be an integer such that $a\pmod{d^2}\in \mathcal{R}_{r,d}$ . If $Y\leq X$ , (d, r) is an admissible pair of positive integers, n is any positive integer, and P is a prime satisfying $4P^2 \gt r^2(X+Y)$ then the set

\[\mathcal{P} = \left\{ p\mid n \ : \ 2\nmid \mathrm{val}_p(n), \ p\neq 2,P \right\}.\]

This is precisely the set $\mathcal{P}_n$ in [ Reference ZubrilinaZub23 , corollary 3·13]. Thus, the calculations in [ Reference ZubrilinaZub23 , proof of proposition 3·9] go through without change. We explain some of the calculations in more detail than in [ Reference ZubrilinaZub23 ]

\begin{align*}S_{n,d,r} & = \sum_{\substack{N\in [X,X+Y]\\ P\nmid N \\[5pt] N\equiv a\pmod{d^2}}}\mu^2(N)\left(\frac{N}{n}\right)\left(\frac{(r^2N-4P^2)/d^2}{n}\right)\\[5pt] & \quad \ll \sqrt{X} + \frac{Y}{d^2 \prod_{p\in \mathcal{P}} \frac{\sqrt{p}}{2}} + \frac{\log( Y)}{d}\left( \sqrt{\frac{nY}{\prod_{p\in \mathcal{P}} \frac{\sqrt{p}}{2}}}\right).\end{align*}

Hence,

\begin{align*}\sum_{\substack{n\in [Y^\sigma,T] \\ d\lt Y^\tau}} \frac{S_{n,d,r}}{nd} &\ll \sqrt{X}\log( T) \log(Y) + Y\sum_{n \geq Y^\sigma} \frac{1}{n \prod_{p\in \mathcal{P}} \frac{\sqrt{p}}{2}} + \sqrt{Y}{\log( Y)} \sum_{n\leq T} \frac{1}{\sqrt{n \prod_{p\in \mathcal{P}} \frac{\sqrt{p}}{2}}}.\end{align*}

We write $n=x^2 y 2^\alpha P^\beta$ , where y is square-free and $\gcd(x,2P)=\gcd(y,2P)=1$ . Therefore $\mathcal{P} = \{ p \ : \ p\mid y\}$ . We now focus on the second term of the expression

\begin{align*}\sum_{n\geq Y^\sigma} \frac{1}{n \prod_{p\in \mathcal{P}} \frac{\sqrt{p}}{2}} &\leq \sum_{n\geq Y^\sigma} \frac{x^2 y^{\frac{1}{2}+\epsilon} 2^\alpha P^\beta}{x^4 y^2 2^{2\alpha} P^{2\beta}}\\[5pt] &\ll \sum_{\alpha,\beta} \sum_{x\leq Y^{\frac{\sigma}{2}}} \frac{1}{2^{\alpha} P^\beta x^2}\sum_{y\gt Y^{\sigma/2}}y^{\frac{-3}{2} + \epsilon} + \sum_{\alpha,\beta}\sum_{x\gt Y^{\frac{\sigma}{2}}} \frac{1}{2^{\alpha} P^\beta x^2}\\[5pt] &\ll Y^{\frac{-\sigma}{2} + \epsilon}.\end{align*}

Next, we focus on the third term.

\begin{align*}\sum_{n\leq T} \frac{1}{\sqrt{n \prod_{p\in \mathcal{P}} \frac{\sqrt{p}}{2}}} & \ll \sum_{\substack{\alpha,\beta: \\ 2^\alpha P^\beta \leq T}} \sum_{x\leq \sqrt{\frac{T}{2^\alpha P^\beta}}} \sum_{y \leq \frac{T}{2^\alpha P^\beta x^2}} \frac{1}{{2^\alpha P^\beta x^2 y}} \times \frac{2^{\frac{\alpha}{2}} P^{\frac{\beta}{2}} x y^{\frac{1}{2}+\epsilon}}{y^{\frac{1}{4}}} \\[5pt] &\ll T^{\frac{1}{4} - \epsilon}.\end{align*}

Putting this together we get that as $X\rightarrow \infty$ ,

\[\sum_{\substack{n\in [Y^\sigma,T] \\ d\lt Y^\tau}} \frac{S_{n,d,r}}{nd} \ll \sqrt{X}\log( T) \log( Y) + Y^{1-\frac{\sigma}{2}+\epsilon} + T^{\frac{1}{4}+\epsilon}\sqrt{Y}\log( Y).\]

5·4. Large d and error terms

The first main task of this section is to estimate the sum ${S_{n,d,r}}/{nd}$ over all n when $d \gg Y^\tau$ . We will observe that this also contributes (only) to the error term.

For $Y^\tau \ll d \ll P$ and truncation parameter T, we have that

\begin{align*}\sum_{\substack{n\\ Y^\tau \ll d \ll P}} \frac{S_{n,d,r}}{nd} &= \sum_{Y^\tau \ll d \ll P}\frac{1}{d}\sum_n \frac{1}{n} \sum_{\substack{N\in [X,X+Y]\\ P\nmid N \\[5pt] N\equiv a\pmod{d^2}}}\mu^2(N)\left(\frac{N}{n}\right)\left(\frac{(r^2N-4P^2)/d^2}{n}\right)\\[5pt] & \quad \ll \sum_{Y^\tau \ll d \ll P} \frac{1}{d} \left(\frac{Y}{d^2} + 1 \right)\log(T) \\[5pt] & \quad \ll \log(T) \left( Y^{1-2\tau} + \log( P)\right).\end{align*}

We obtain the second line by the trivial inequality $\mu(N)^2({N}/{n})(({(r^2N-4P^2)/d^2})/{n})\le 1 $ . The term $(({Y}/{d^2})+1)$ comes from the sum over N.

We now collect all the error terms. The cumulative error terms from Propositions 5·8 and 5·9, and the above calculations is

\begin{equation*}\begin{split}&\left(\sqrt{X}Y^{\frac{\sigma}{2}}+Y^{\tau+\epsilon+\frac{3}{2}\sigma}+Y^{1-2\tau}+Y^{1-\frac{\sigma}{5}}\right)\\[5pt] & \quad + \left( \sqrt{X}\log( T) \log( Y) + Y^{1-\frac{\sigma}{2}+\epsilon} + T^{\frac{1}{4}+\epsilon}\sqrt{Y}\log( Y) \right)\\[5pt] & \quad + \log(T) \left( Y^{1-2\tau} + \log( P)\right)\end{split}\end{equation*}

Throughout this section we are assuming that $P^2 \ll X^{1+\delta_2}$ where $\delta_2$ is chosen as in the statement of the main theorem. In particular, we may always choose $\delta_2 \lt {2}/{11}$ . Therefore, we may assume that $P^\epsilon \ll \sqrt{X}$ and choose the cut-off parameter $T\gg Y$ , we can rewrite the cumulative error term as

\begin{align*}\left(\sqrt{X}Y^{\frac{\sigma}{2}}+Y^{\tau+\epsilon+\frac{3}{2}\sigma}+Y^{1-2\tau}+Y^{1-\frac{\sigma}{5}}\right) &+ \left(\sqrt{X} \log( T) \log( Y) + Y^{1-\frac{\sigma}{2}+\epsilon} + T^{\frac{1}{4}+\epsilon}\sqrt{Y} \right) \\[5pt] &+ \log(T) Y^{1-2\tau}.\end{align*}

Choosing $\tau = {1}/{18}$ , $\sigma={5}/{9}$ and assuming that $\log( T) \ll Y$ , the above expression is bounded by

\begin{align*}\left(\sqrt{X}Y^{\frac{5}{18}}+Y^{\frac{8}{9}+\epsilon}+2Y^{\frac{8}{9}}\right) + \left( \sqrt{X}\log( T) \log( Y) + Y^{\frac{13}{18}+\epsilon} + T^{\frac{1}{4}+\epsilon}\sqrt{Y} \right) + Y^{\frac{8}{9}}\log(T) &\\[5pt] \ll \sqrt{X}Y^{\frac{5}{18}} + T^{\frac{1}{4}+\epsilon}\sqrt{Y} + Y^{\frac{8}{9}}\log(T).\end{align*}

Remark 5·11. The calculations in this section rely crucially on the fact that we are working with $P^2$ -coefficients. If we were to work with higher P-power coefficients, there would be many more terms to analyse which would make the calculations more cumbersome. However, we expect that the shape of the contribution would remain unchanged.

6. The remaining terms

To complete the proof of the main theorem, we are still left to bound the following terms:

1. (1) those in the range of r not covered in Corollary 5·2;

2. (2) those involving levels N when $P\mid N$ ;

3. (3) those involving the P term in the trace formula.

First, we evaluate

\begin{align*}\frac{\zeta(2) \pi}{XY} \sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\nmid N}} \sum_{ \frac{2P}{\sqrt{X+Y}} \leq r\leq \frac{2P}{\sqrt{N}}} H_1(r^2 N^2 - 4P^2 N) & \ll \frac{Y}{XY} \left( P \left( \frac{1}{\sqrt{X}} - \frac{1}{\sqrt{X+Y}}\right)(P^2 X)^{\frac{1}{2} + \epsilon}\right) \\[5pt] & \ll X^{\delta_2 - \frac{\delta}{2} + \epsilon}.\end{align*}

This follows from the assumptions that $Y=(1+o(1))X^{1-\delta}$ , that $P^2\ll X^{1+\delta_2}$ , and that $P^\epsilon \ll \sqrt{X}$ . Since the main term is ${P}/{\sqrt{X}}$ which is of size $X^{\frac{\delta_2}{2}}$ , it follows that we require $\delta_2\lt\delta$ .

When $P\mid N$ , we have noted before that

\[ \lambda_f(P^2) = \frac{1}{P}.\]

In other words, at the ramified primes P, we notice that using the trivial bound

\begin{align*}\frac{\zeta(2) \pi}{XY}\sideset{}{'}\sum_{\substack{N\in [X, X+Y] \\ P\mid N}} \sum_{f\in H^{\text{new}(N,2)}}P\lambda_f(P^2)\epsilon(f) &\leq \frac{\zeta(2) \pi}{XY} X \sum_{\substack{N\in [X, X+Y] \\ P\mid N \\[5pt] }} |\epsilon(f)| \\[5pt] &\leq \zeta(2)\pi \frac{\#\{N \ : \ P\mid N \text{ and } N\in [X, X+Y]\}}{Y} \\[5pt] &=O\left(\frac{1}{P} + \frac{1}{Y}\right).\end{align*}

Finally, note that

\begin{align*} \frac{\zeta(2)\pi}{XY} \sideset{}{'}\sum_{\substack{N \in [X, X+Y]} }(P+P^2) & = \frac{\zeta(2)\pi }{XY} (P+P^2) \left(\sum_{N\leq X+Y} \mu^2 (N) - \sum_{N\leq X}\mu^2(N)\right)\\[5pt] &= \frac{\zeta(2)\pi}{XY} (P+P^2) \left( \frac{Y}{\zeta(2)} + O(\sqrt{X+Y} - \sqrt{X})\right)\\[5pt] &= \frac{\pi P^2}{X} + O\left( \frac{P}{X} + \frac{P^2}{XY^{\frac12}}\right).\end{align*}

7. Arithmetic functions

The purpose of this section is to record technical results required in the calculations performed above.

Lemma 7·1. Let P be an odd prime and K be a cut-off parameter. Define

\[A\;:\!=\;\prod_p\left(1+\frac{p}{(p+1)^2(p-1)}\right) \quad \textrm{ and } \quad \eta(m) \;:\!=\; \prod_{p\mid m} \frac{p}{p+1}.\]

Then:

(7·1) \begin{align}\sum_{\substack{m=1}}^K\frac{\eta(m)}{m^2} & = \prod_{p}\left(1+\frac{p}{(p+1)^2(p-1)}\right)+O\left(\frac{1}{K}\right)= A+O\left(\frac{1}{K}\right) \end{align}

(7·2) \begin{align} \sum_{\substack{m=1\\ \gcd(m,2)=1}}^K\frac{\eta(m)}{m^2} & = \prod_{p\neq 2}\left(1+\frac{p}{(p+1)^2(p-1)}\right)+O\left(\frac{1}{K}\right)=\frac{9}{11} A + O\left(\frac{1}{K}\right) \end{align}

(7·3) \begin{align} \sum_{\substack{m=1}}^K\frac{\eta(2m)}{m^2} & = \frac{8}{11}A+O\left(\frac{1}{K}\right)\qquad\qquad\qquad \end{align}

(7·4) \begin{align} \sum_{\substack{m=1\\ \gcd(m,2P)=1}}^K\frac{\eta(m)}{m^2} & = \prod_{p\neq 2,P}\left(1+\frac{p}{(p+1)^2(p-1)}\right)+O\left(\frac{1}{K}\right)=\frac{9}{11} A + O\left(\frac{1}{P^2} + \frac{1}{K}\right) \end{align}

(7·5) \begin{align} \sum_{\substack{m=1 \\ \gcd(m,P)=1}}^K\frac{\eta(2m)}{m^2} & = \frac{8}{11}A+O\left(\frac{1}{P^2} + \frac{1}{K}\right).\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\end{align}

Proof. We leave the proof as an exercise. A useful observation for these calculations is the following:

\[\sum_{m=1}^\infty \frac{\eta(m)}{m^2} = \prod_p \left(\sum_{k=0}^{\infty}\frac{\eta(p^k)}{p^{2k}} \right) = \prod_p\left( 1+\frac{1}{p(p+1)} + \frac{1}{p(p+1)^3} +\ldots\right).\]

Note that (7·4) and (7·5) are already stated in [ Reference ZubrilinaZub23 , lemma 6·5].

8. Main result and proof

The main result we prove in this paper is the following

Theorem 8·1. Let $H^{\mathrm{new}}(N)$ be a Hecke basis for trivial character for weight 2 cusp forms for $\Gamma_0(N)$ with $f\in H^{\mathrm{new}}(N)$ normalised to have leading coefficient 1. Let $\epsilon(f)$ be the root number of f and let $a_f(p)$ be the p-th Fourier coefficient of f, and set $\lambda_f(p) = {a_f(p)}/{\sqrt{p}}$ . Let P be a prime, and suppose that the parameters P, X, and Y go to infinity. Further suppose that $Y= (1+o(1))X^{1-\delta}$ and $P^2\ll X^{1+\delta_2}$ where $0\lt\delta_2 \lt \delta \lt {9}/{11}$ and ${\delta}/{9} + {\delta_2}/{2} \lt {1}/9$ . Set $\delta'=\delta_2 - {\delta}/{2}$ . Then writing $y={P^2}/{X}$ ,

\begin{align*}&\frac{\displaystyle\sideset{}{'}\sum_{\substack{N\in [X, X+Y]}}\sum_{f\in H^{\mathrm{new}}(N)} P \lambda_{f}(P^2)\epsilon(f)}{\displaystyle\sideset{}{'}\sum_{\substack{N\in [X, X+Y]}}\sum_{f\in H^{\mathrm{new}}(N)} 1} \\[5pt] & \quad = \frac{12}{\pi \prod_p\left( 1 - \frac{1}{p(p+1)}\right)}\left(A\sqrt{y} + B\sum_{r\leq \sqrt{y}} C(r)\left( \sqrt{4y-r^2}\right) - \pi y\right) \\[5pt] & \qquad + O_\epsilon\left( X^{\delta' + \epsilon} +\frac{1}{P}\right).\end{align*}

Proof. In [ Reference ZubrilinaZub23 , section 3·4], it is proven that for N square-free

\[\sideset{}{'}\sum_{\substack{N\in [X, X+Y]}} \sum_{f\in H^{\mathrm{new}}(N)} 1 = \frac{XY}{12 \zeta(2)}\prod_p \left(1- \frac{1}{p(p+1)}\right) + O\left(X^\epsilon Y + X^{\frac85 + \epsilon} + Y^2 \right).\]

Set $y={P^2}/{X}$ . It will follow from Proposition 3·1, Corollary 5·2 and calculations in Section 6 that

\begin{align*}\frac{\zeta(2)\pi}{XY} \sideset{}{'}\sum_{\substack{N\in [X, X+Y]}} \sum_{f\in H^{\mathrm{new}}(N)} P\lambda_f(P^2) \epsilon(f) & = A\sqrt{y} + B \sum_{r\le \sqrt{y}} C(r)\left(\sqrt{4y - r^2}\right)\\[5pt] & \quad - \pi y + O_\epsilon\left( X^{\delta'+\epsilon}+\frac{1}{P}\right).\end{align*}

The error terms in Proposition 3·1, Corollary 5·2, and the calculations in Section 6 gives that the cumulative error term is of size $O\left(X^{\delta'+\epsilon}+{1}/{P}\right)$ where $\delta'=\delta_2-{\delta}/{2}$ and we have used the fact that $\delta_2\lt\delta\lt{2}/{11}$ and $\delta_2+9\delta\lt2$ .