2.1 The MHS associated to a homologically trivial cycle

Suppose that X is a smooth projective variety and that $Z= \sum _j n_j Z_j$ is an algebraic d-cycle on X where the $Z_j$ are distinct reduced irreducible subschemes of X of dimension d. Denote the support of Z by $|Z|$ . When Z is homologically trivial, it determines an extension

of mixed Hodge structures. It is obtained from the long exact homology sequence of $(X,|Z|)$ by pulling back along the map

$$ \begin{align*}cl_Z : {\mathbb Z}(d) \to H_{2d}(|Z|) = \textstyle{\bigoplus_j} {\mathbb Z}[Z_j] \end{align*} $$

that takes $1$ to $[Z] = \sum n_j[Z_j]$ and then twisting by ${\mathbb Z}(-d)$ :

The extension depends only on the rational equivalence class of Z. The extension $(E_Z)_{\mathbb Z}$ is generated by $H_{2d+1}(X;{\mathbb Z})$ and $\Gamma $ , where $\partial \Gamma = Z$ .

2.2 Extensions of mixed Hodge structure and intermediate jacobians

Suppose that V is a Hodge structure of negative weight. The group of extensions of mixed Hodge structure (MHS) of the form

$$ \begin{align*}0 \to V \to E \to {\mathbb Z} \to 0 \end{align*} $$

forms a group $\operatorname {Ext}^1_{\mathsf {MHS}}({\mathbb Z},V)$ that is isomorphic to the complex torus

$$ \begin{align*}J(V) := V_{\mathbb C}/(V_{\mathbb Z} + F^0 V). \end{align*} $$

The extension determines, and is determined by, the image of

$$ \begin{align*}\mathbf{e}_{\mathbb Z} - \mathbf{e}_F \in V_{\mathbb C} \end{align*} $$

in $J(V)$ , where $\mathbf {e}_{\mathbb Z} \in E_{\mathbb Z}$ and $\mathbf {e}_F \in F^0 E$ both project to $1\in {\mathbb Z}$ .

The torus $J(V)$ is compact when V has weight $-1$ . This is because, in this case,

$$ \begin{align*}V_{\mathbb C} = F^0 V \oplus \overline{F^0 V}. \end{align*} $$

This implies that the composite

$$ \begin{align*}V_{\mathbb R} \hookrightarrow V_{\mathbb C} \to V_{\mathbb C}/F^0 V \end{align*} $$

is an isomorphism of real vector spaces which, in turn, induces an isomorphism of $J(V)$ with the compact real torus $V_{\mathbb R}/V_{\mathbb Z}$ .

So a homologically trivial d-cycle Z in a smooth projective variety X determines a point $\nu _Z$ in the Griffiths intermediate jacobian

$$ \begin{align*}J(H_{2d+1}(X)(-d)). \end{align*} $$

$$ \begin{align*}\textstyle{\int_\Gamma} \in \operatorname{Hom}_{\mathbb C}(F^{d+1} H^{2d+1}(X),\mathbb C) \cong H_{2d+1}(X)(-d)/F^0, \end{align*} $$

where, as above, $\partial \Gamma = Z$ .

2.3 Normal functions

Suppose $\mathbb V$ is a variation of Hodge structure of negative weight over a smooth variety S. Set

$$ \begin{align*}\mathcal V = \mathbb V_{\mathbb Z}\otimes_{\mathbb Z} {\mathcal O}_S. \end{align*} $$

It is a flat holomorphic vector bundle over S with connection

$$ \begin{align*}\nabla : \mathcal V \to \mathcal V \otimes \Omega^1_S. \end{align*} $$

Denote the pth term of its Hodge filtration by $F^p\mathcal V$ . The connection satisfies Griffiths transversality

$$ \begin{align*}\nabla : F^p \mathcal V \to F^{p-1}\mathcal V \otimes \Omega^1_S. \end{align*} $$

To such a variation, we can associate a family of intermediate jacobians $J(\mathbb V)$ over S. It is the quotient of the vector bundle $\mathcal V/F^0\mathcal V$ by the image of $\mathbb V_{\mathbb Z} \to \mathcal V/F^0\mathcal V$ . Its fibre over $s \in S$ is $J(V_s)$ , the intermediate jacobian associated to the fibre $V_s$ of $\mathbb V$ over s.

Each extension

(2) $$ \begin{align} 0 \to \mathbb V \to {\mathbb E} \to {\mathbb Z}_S \to 0 \end{align} $$

of variations of mixed Hodge structure gives rise to a holomorphic section $\nu _{{\mathbb E}}$ of the bundle

$$ \begin{align*}J(\mathbb V) \to S \end{align*} $$

of intermediate jacobians. Its value over $s\in S$ is the point of $J(V_s)$ that corresponds to the extension

$$ \begin{align*}0 \to V_s \to E_s \to {\mathbb Z} \to 0 \end{align*} $$

of MHS obtained by restricting (2) to s.

Every continuous section $\sigma $ of $J(\mathbb V)$ determines a class $c(\sigma )$ in $H^1(S,\mathbb V_{\mathbb Z})$ . A detailed description of $c(\sigma )$ can be found in [Reference Hain16, §4.1]. Since $H^1(S,\mathbb V_{\mathbb Z})$ is the group of congruence classes of extensions of ${\mathbb Z}_S$ by $\mathbb V_{\mathbb Z}$ in the category of local systems over S, each section $\sigma $ of $J(\mathbb V)$ determines an extension of local systems

$$ \begin{align*}0 \to \mathbb V \to {\mathbb E} \to {\mathbb Z}_S \to 0 \end{align*} $$

over S. The value of $\sigma $ at $s\in S$ determines a MHS on the fibre of ${\mathbb E}$ over s.

In other words, the group of normal function sections of $J(\mathbb V)$ is isomorphic to the group of extensions of ${\mathbb Z}_S$ by $\mathbb V$ in the category of variations of MHS over S. Normal functions have to be holomorphic. This corresponds to the condition that the Hodge filtration varies holomorphically. They also have to satisfy the Griffiths infinitesimal period relation, which is equivalent to the condition that the canonical flat connection on

$$ \begin{align*}\mathcal E := {\mathbb E}\otimes_{\mathbb Z} {\mathcal O}_S \end{align*} $$

satisfies Griffiths transversality, $\nabla : F^p \mathcal E \to F^{p-1} \mathcal E \otimes \Omega ^1_S$ .

2.4 Families of homologically trivial cycles

Suppose now that S is a smooth variety and that $f : X \to S$ is a family of smooth projective varieties. Suppose that Z is a relative d-cycle. That is, it is an algebraic cycle on X whose restriction $Z_s$ to each fibre $X_s := f^{-1}(s)$ over $s\in S$ is a d-cycle. Suppose that each $Z_s$ is homologically trivial in $X_s$ . Denote the local system over S whose fibre over $s\in S$ is $H_{2d+1}(X_s)(-d)$ by $\mathbb V$ . It underlies a variation of Hodge structure of weight $-1$ . The construction in Section 2.1 gives an extension of local systems

$$ \begin{align*}0 \to \mathbb V \to {\mathbb E}_Z \to {\mathbb Z}_S \to 0 \end{align*} $$

and a mixed Hodge structure on the fibre $E_s$ over $s\in S$ that is an extension of ${\mathbb Z}$ by $V_s$ . The work of Griffiths [Reference Griffiths9] implies that the corresponding section $\nu _Z : S \to J(\mathbb V)$ is holomorphic and satisfies his infinitesimal period relation. It is therefore an extension of variations of MHS. The corresponding section $\nu _Z$ of $J(\mathbb V)$ is thus a normal function. The main result of [Reference Steenbrink and Zucker28] implies that it is admissible.

2.5 The foliation of $J(\mathbb V)$

Suppose now that $\mathbb V$ has weight $-1$ . This implies that the map

$$ \begin{align*}J(\mathbb V_{\mathbb R}) := \mathbb V_{\mathbb R}/\mathbb V_{\mathbb Z} \to J(\mathbb V) \end{align*} $$

is an isomorphism of families of tori over S. The family $J(\mathbb V_{\mathbb R})$ is a locally constant family of tori and is thus foliated by its locally constant leaves. Consequently, the family $J(\mathbb V) \to S$ is a foliated family of complex tori. This foliation $\mathscr F$ is a complex foliation, but not a holomorphic foliation. This means that each leaf of $\mathscr F$ is a complex submanifold of $J(\mathbb V)$ but that $\mathscr F$ is not, in general, locally biholomorphic to a product foliation.

2.6 The Griffiths infinitesimal invariant

Griffiths’ infinitesimal invariant is an obstruction to a normal function being locally constant. Suppose that $\nu $ is a normal function section of $J(\mathbb V)$ . Let

$$ \begin{align*}0 \to \mathbb V \to {\mathbb E} \to {\mathbb Z} \to 0 \end{align*} $$

be the corresponding variation of MHS over S. Suppose that $s \in S$ and that U is an open neighbourhood of s that is biholomorphic to a complex ball. Since U is Stein, the restriction of each of the bundles $F^p \mathcal E$ to U is holomorphically trivial, and since U is contractible, the restriction of ${\mathbb E}_{\mathbb Z}$ to U is trivial. Let

$$ \begin{align*}\mathbf{e}_F \in H^0(U,F^0 \mathcal E) \text{ and } \mathbf{e}_{\mathbb Z} \in H^0(U,{\mathbb E}_{\mathbb Z}) \end{align*} $$

be lifts of $1 \in {\mathbb Z}$ . Then

$$ \begin{align*}v := \mathbf{e}_{\mathbb Z} - \mathbf{e}_F \in H^0(U,\mathcal V) \end{align*} $$

is a lift of the restriction of $\nu $ to U. Since $\mathbf {e}_F$ is a local section of $F^0\mathcal E$ , Griffiths transversality implies that

(3) $$ \begin{align} \nabla \nu \in H^0(U,F^{-1}\mathcal V\otimes\Omega^1_S). \end{align} $$

Since $\mathbf {e}_F$ is well defined mod $H^0(U,F^0\mathcal V)$ , this descends to a well defined element $\delta _U(\nu )$ of

$$ \begin{align*}\mathcal K(U) := \ker\{\nabla : H^0(U, F^{-1}\mathcal V\otimes\Omega^1_S)\to H^0(U, F^{-2}\mathcal V\otimes \Omega^2_S)\}/\nabla H^0(U,F^0\mathcal V). \end{align*} $$

The functor $U \mapsto \mathcal K(U)$ is a presheaf on S.

This obstruction group has a more intrinsic description in terms of the Hodge filtration of $\mathcal V\otimes \Omega ^{\bullet }_S$ , which is defined by

$$ \begin{align*}F^p(\mathcal V\otimes \Omega_S^j) := (F^{p-j}\mathcal V) \otimes \Omega_S^j. \end{align*} $$

Each $(F^p(\mathcal V\otimes \Omega _S^{\bullet }),\nabla )$ is a complex of locally free complex analytic sheaves on S. Denote their cohomology sheaves by $\mathcal H^{\bullet }(F^p(\mathcal V\otimes \Omega _S^{\bullet }))$ . Since $\mathcal H^1(F^0(\mathcal V\otimes \Omega _S^{\bullet }))$ is the sheafification of $\mathcal K$ , the germs of the $\delta _U(\nu )$ at each $s\in S$ define a section

$$ \begin{align*}\delta(\nu) \in H^0(S,\mathcal H^1(F^0(\mathcal V\otimes\Omega_S^{\bullet}))). \end{align*} $$

This is the Griffiths invariant of $\nu $ . It vanishes if and only if $\nu $ is locally constant on S.

2.7 Green’s refinement of the Griffiths invariant

Green’s refinement [Reference Green8, §1] of Griffiths construction is a useful tool for understanding the Griffiths invariant. A key point is that each graded quotient

(4) $$ \begin{align} \operatorname{Gr}_F^p (\mathcal V\otimes \Omega_S^{\bullet}) := [0 \to \operatorname{Gr}^p_F \mathcal V \to \operatorname{Gr}^{p-1}_F \mathcal V \otimes \Omega^1_S \to \operatorname{Gr}^{p-2}_F \mathcal V \otimes \Omega^2_S \to \cdots] \end{align} $$

of the Hodge filtration of $\mathcal V\otimes \Omega _S^{\bullet }$ is a complex of vector bundles as the differential $\overline {\nabla }$ induced by the connection $\nabla $ is ${\mathcal O}_S$ -linear.

Denote the homology sheaves of the complex (4) by $\mathcal H^{\bullet }(\operatorname {Gr}_F^p(\mathcal V\otimes \Omega _S^{\bullet }))$ . Suppose that $\nu $ is a normal function. Denote the image of $\delta (\nu )$ under the map

(5) $$ \begin{align} H^0(S,F^{-1}\mathcal V\otimes\Omega^1_S/\nabla (F^0\mathcal V)) \to H^0(S,\mathcal H^1(\operatorname{Gr}_F^0(\mathcal V\otimes \Omega_S^{\bullet}))) \end{align} $$

by $\overline {\delta }(\nu )$ . In favourable situations, Green’s invariant $\overline {\delta }(\nu )$ determines the Griffiths invariant.

Proof. Green proves this using a spectral sequence argument. Here we give an elementary direct proof. Since

$$ \begin{align*}0 \to \operatorname{Gr}_F^1(\mathcal V\otimes\Omega_S^{\bullet}) \to F^0(\mathcal V\otimes\Omega_S^{\bullet})/F^2 \to \operatorname{Gr}_F^0(\mathcal V\otimes\Omega_S^{\bullet}) \to 0 \end{align*} $$

is exact, so is

$$ \begin{align*}\mathcal H^1(\operatorname{Gr}_F^1(\mathcal V\otimes\Omega_S^{\bullet})) \to \mathcal H^1(F^0(\mathcal V\otimes\Omega_S^{\bullet})/F^2) \to \mathcal H^1(\operatorname{Gr}_F^0(\mathcal V\otimes\Omega_S^{\bullet})). \end{align*} $$

The vanishing of $\mathcal H^1(\operatorname {Gr}_F^1(\mathcal V\otimes \Omega _S^{\bullet }))$ implies the injectivity of

$$ \begin{align*}\mathcal H^1(F^0(\mathcal V\otimes\Omega_S^{\bullet})/F^2) \hookrightarrow \mathcal H^1(\operatorname{Gr}_F^0(\mathcal V\otimes\Omega_S^{\bullet})). \end{align*} $$

Similarly, the vanishing of $\mathcal H^1(\operatorname {Gr}_F^2(\mathcal V\otimes \Omega _S^{\bullet }))$ and the exactness of

$$ \begin{align*}0 \to \operatorname{Gr}_F^2(\mathcal V\otimes\Omega_S^{\bullet}) \to F^0(\mathcal V\otimes\Omega_S^{\bullet})/F^3 \to F^0(\mathcal V\otimes\Omega_S^{\bullet})/F^2 \to 0 \end{align*} $$

implies the injectivity of

$$ \begin{align*}\mathcal H^1(F^0(\mathcal V\otimes\Omega_S^{\bullet})/F^3) \hookrightarrow \mathcal H^1(F^0(\mathcal V\otimes\Omega_S^{\bullet})/F^2). \end{align*} $$

One continues in this manner to obtain injections

(6) $$ \begin{align} \mathcal H^1(F^0(\mathcal V\otimes\Omega_S^{\bullet})/F^{p+1}) \hookrightarrow \mathcal H^1(F^0(\mathcal V\otimes\Omega_S^{\bullet})/F^p) \end{align} $$

for all $p>0$ . Since

$$ \begin{align*}F^0(\mathcal V\otimes\Omega_S^{\bullet})/F^m = F^0(\mathcal V\otimes\Omega_S^{\bullet}) \end{align*} $$

for m sufficiently large, the composition of the injections (6)

$$ \begin{align*}\mathcal H^1(F^0(\mathcal V\otimes\Omega^{\bullet}_S)) \to \mathcal H^1(\operatorname{Gr}_F^0(\mathcal V\otimes\Omega_S^{\bullet})) \end{align*} $$

is injective. The result follows by taking global sections.

3.1 Definition of the rank

We can regard the foliation $\mathscr F$ of $J(\mathbb V)$ (Section 2.5) as a smooth sub-bundle of the tangent bundle of $J(\mathbb V)$ . Since the leaves of the foliation are complex submanifolds of $J(\mathbb V)$ , we can regard $\mathscr F$ as a complex sub-bundle of $TJ(\mathbb V)$ . The projection $J(\mathbb V) \to S$ induces an isomorphism $\mathscr F_v \to T_s S$ for all $v\in J(V_s)$ . This gives a splitting of the sequence

Let $\phi : TJ(\mathbb V) \to \mathcal V/F^0$ be the corresponding projection. It is $\mathbb C$ -linear. Composing it with $(d\nu )_s$ gives a $\mathbb C$ -linear map $\phi \circ (d\nu )_s : T_s S \to V_s/F^0 V_s$ :

Definition 3.1. The rank $\operatorname {rk}_s\nu $ of $\nu $ at $s\in S$ is the rank of $\phi \circ (d\nu )_s$ . Define

$$ \begin{align*}\operatorname{rk}_S \nu = \max_{s\in S} \operatorname{rk}_s \nu \end{align*} $$

The rank of $\nu $ vanishes if and only if its Griffiths invariant $\delta (\nu )$ vanishes. In general, the relation between the Griffiths invariant and the rank is more subtle.

3.2 The canonical derivative and the Green–Griffiths invariant

In order to relate the rank of a normal function to the rank of the associated invariants defined by Griffiths and Green, we need to consider real lifts of normal functions.

We will work with complexified tangent spaces as we need to compare invariants of real analytic and holomorphic lifts of a normal function. In this subsection, and only this subsection, we denote the real tangent space of a complex manifold Y at $y\in Y$ by $T_y Y$ . It has a canonical almost complex structure $I : T_y Y \to T_y Y$ . Its complexification decomposes

$$ \begin{align*}T_y Y \otimes \mathbb C = T_y' Y \oplus T_y"Y \end{align*} $$

into the $I=i$ and $I=-i$ eigenspaces. The $I=i$ eigenspace $T_y' Y$ is the holomorphic tangent space. An ${\mathbb R}$ -linear map $T_y Y \to V$ into a complex vector space V is complex linear if and only if the induced map $T_y Y\otimes \mathbb C \to V$ vanishes on $T_yY"$ . The inclusion of $T_y Y$ into $T_y Y\otimes \mathbb C$ induces a complex linear isomorphism

$$ \begin{align*}T_y Y \to (T_y Y\otimes \mathbb C)/T_y" Y \cong T_y' Y. \end{align*} $$

Definition 3.2. The canonical derivative $\overline {\nabla }_s \nu $ of $\nu $ at s is the composite

Griffiths transversality implies that its image lies in the subspace $\operatorname {Gr}^{-1}_F V_s$ of $V_s/F^0$ . The function $\overline {\nabla } : s \mapsto \overline {\nabla }_s\nu $ is a real analytic section of $\operatorname {Gr}^{-1}_F \mathcal V$ .

Denote the sheaf of real analytic functions on S by $\mathscr E_S$ . The normal function $\nu $ corresponds to a real analytic section $\nu _{\mathbb R}$ of $J(\mathbb V_{\mathbb R})$ under the isomorphism $J(\mathbb V_{\mathbb R}) \cong J(\mathbb V)$ . Locally it lifts to a section $\tilde {\nu }_{\mathbb R}$ of $\mathbb V_{\mathbb R}\otimes \mathscr E_S$ . Its derivative $(\nabla \tilde {\nu }_{\mathbb R})_s \in V_{s,{\mathbb R}}\otimes T_s^\vee S$ at $s\in S$ does not depend on the choice of $\tilde {\nu }_{\mathbb R}$ , so we will denote it by $(\nabla \nu _{\mathbb R})_s$ .

The goal of the next result is to clarify the relationship between the various derivatives of $\nu $ .

Proposition 3.3. The canonical derivative $\overline {\nabla }_s\nu $ is related to the derivatives of $\nu $ and $\nu _{\mathbb R}$ at $s\in S$ by the commutative diagram

(7)

In particular,

$$ \begin{align*}\operatorname{rk}_s \nu = \operatorname{rk} \overline{\nabla}_s \nu = \operatorname{rk}_{\mathbb R} (\nabla \nu_{\mathbb R})_s/2. \end{align*} $$

It is important to note that, in general, $\overline {\nabla } \nu $ is a real analytic, but not holomorphic, section of $\operatorname {Gr}_F^{-1}\mathcal V \otimes \Omega _S^1$ , even though $\overline {\delta }(\nu )$ , its reduction mod $\overline {\nabla } \operatorname {Gr}_F^0 \mathcal V \otimes \mathscr E_S$ , is a holomorphic section of $\operatorname {Gr}_F^{-1}\mathcal V \otimes \Omega _S^1/\overline {\nabla }\operatorname {Gr}_F^0\mathcal V$ . This will be important in Section 10 where the possibility that $\overline {\nabla } \nu $ is not holomorphic is the obstruction to obtaining an explicit formula for $\overline {\nabla }\nu $ in genus 3.

3.3 Symmetry

Suppose that S is a complex analytic variety and that $\mathbb V \to S$ is a variation of Hodge structure of weight $-1$ . Suppose that G is a finite group that acts holomorphically on S and that this action lifts to an action on $\mathbb V \to S$ as a variation of Hodge structure. If $s \in S$ is a fixed point of the action, then G acts on $T_s S$ and as an automorphism group of the Hodge structure $V_s$ and its intermediate jacobian $J(V_s)$ . If $\nu $ is a G-invariant normal function section of $J(\mathbb V)$ , then $\nu (s) \in J(V_s)$ will be a fixed point of the G action.

3.4 The rank filtration

We can filter S

$$ \begin{align*}S = S_d \supseteq S_{d-1} \supseteq \cdots \supseteq S_0 \supseteq S_{-1} := \emptyset \end{align*} $$

by the strata $S_r$ where $\nu $ has rank $\le r$ , where $d=\dim S$ . Since $\nu _{\mathbb R}$ is real analytic, each $S_r$ is a real analytic subvariety of S. Gao and Zhang [Reference Gao and Zhang6, Thm. 1.4] have proved a stronger result. They define the foliation filtration $S_{\mathscr F}$ of S to be the union of the leaves of the ‘pullback foliation’ $\nu ^\ast \mathscr F$ of codimension $\le r$ and show that each $S_{\mathscr F}(r)$ is a complex algebraic subvariety of S. The rank condition implies that $S_r$ contains $S_{\mathscr F}(r)$ for all r.

The following much weaker density result is sufficient for our purposes.

Proposition 3.6. If there is a point $x_0 \in S$ where $\operatorname {rk}_{x_0} \nu = \dim S$ , then

$$ \begin{align*}\Sigma := S_{-1+\dim S} = \{x \in S : \operatorname{rk}_x \nu < \dim S\} \end{align*} $$

is a real analytic subvariety of S of real codimension $\ge 1$ . In particular, $S-\Sigma $ is dense in S.

Part 2. The Ceresa cycle

In this part we recall the construction of the normal function of the Ceresa cycle and discuss generalities related to its Green–Griffiths invariant and canonical derivative.

8.1 Some representation theory

Denote the top exterior power of a representation V by $\det V$ . Since the pairing $B \otimes \Lambda ^2 B \to \det B$ is nonsingular in genus 3, we have a canonical isomorphism

$$ \begin{align*}\Lambda^2 B \cong A \otimes \det B \end{align*} $$

of $\mathrm {GL}(B)$ modules. Similarly, $\Lambda ^2 A \cong B \otimes \det A$ . We also have a canonical isomorphism

$$ \begin{align*}S^2 S^2 B \cong S^4 B \oplus S^2 \Lambda^2 B \end{align*} $$

which holds for all $g \ge 2$ . When $g=3$ , we thus have a canonical isomorphism

$$ \begin{align*}S^2 S^2 B \cong S^4 B \oplus (S^2 A \otimes (\det B)^2). \end{align*} $$

Both summands are irreducible.

Lemma 8.1. In genus 3, the are natural $\mathrm {GL}(B)$ module isomorphisms

$$ \begin{align*}\frac{\Lambda^2 A \otimes B}{\theta \cdot A} \cong S^2 B \otimes \det A \text{ and } \frac{A \otimes \Lambda^2 B}{\theta \cdot B} \cong S^2 A \otimes \det B. \end{align*} $$

Proof. The first isomorphism follows from the isomorphisms

$$ \begin{align*}\Lambda^2 A \otimes B \cong B^{\otimes 2} \otimes \det A \cong (S^2 B \oplus \Lambda^2 B) \otimes \det A \cong (S^2 B\otimes \det A) \oplus A. \end{align*} $$

The second is proved similarly, or by taking duals.

8.1.1 Green–Griffiths cohomology

The following result combines computations of Nori [Reference Nori23, pp. 371–372] and Collino–Pirola [Reference Collino and Pirola3, Lem. 4.2.3].

Proposition 8.2. If C is a nonhyperelliptic curve of genus 3, then

1. (1) If $p \ge 0$ , then $H^0(\operatorname {Gr}_F^p(V\otimes \Lambda ^{\bullet } S^2 B))$ vanishes.

2. (2) If $p> 0$ , then $H^1(\operatorname {Gr}_F^p(V\otimes \Lambda ^{\bullet } S^2 B))$ vanishes.

3. (3) The space of 1-cocycles in the complex $\operatorname {Gr}_F^0 (V\otimes \Lambda ^{\bullet } S^2 B)$ is naturally isomorphic to

   $$ \begin{align*}S^2 S^2 B \otimes \det A \cong (S^4 B \otimes \det A) \oplus (S^2 A \otimes \det B) \end{align*} $$
   and the space of 1-coboundaries to $S^2 A \otimes \det B$ . Consequently,
   $$ \begin{align*}H^1(\operatorname{Gr}_F^0 (V\otimes \Lambda^{\bullet} S^2 B)) \cong S^4 B \otimes \det A. \end{align*} $$

Proof. The complexes $\operatorname {Gr}_F^p (V \otimes \Lambda ^{\bullet } S^2 B)$ are:

Each is a complex of $\mathrm {GL}(B)$ modules. The vanishing of the homology of the first two rows in degrees 0 and 1 is sketched by Nori [Reference Nori23, §7]. Here we sketch an elementary proof. The exactness of the first row $\operatorname {Gr}_F^2$ follows from the fact that $S^2 B\otimes \det B$ is irreducible and the differential is nonzero and therefore injective. Exactness of the second row $\operatorname {Gr}_F^1$ follows (using Lemma 8.1) from the fact that

$$ \begin{align*}\displaystyle{\frac{ A \otimes \Lambda^2 B}{\theta\cdot B}}\otimes S^2 B \cong \operatorname{End} (S^2 B) \otimes \det B \end{align*} $$

which has three irreducible factors. The differential takes $\Lambda ^3 B$ to $\det B\otimes \operatorname {id}_{S^2B}$ . The other two factors map injectively into the degree 2 term. So we focus on the complex $\operatorname {Gr}_F^0$ .

Lemma 8.1 implies that the degree 0 term is $S^2 A \otimes \det B$ and that the degree 1 term is

$$ \begin{align*} S^2 B \otimes S^2 B \otimes \det A &\cong (S^2 S^2 B\otimes \det A) \oplus (\Lambda^2 S^2 B \otimes \det A) \cr &\cong (S^4 B \otimes \det A) \oplus (S^2 A \otimes \det B) \oplus (\Lambda^2 S^2 B \otimes \det A). \end{align*} $$

Since the degree 0 term is irreducible and the differential is nonzero, we see that the differential is injective and that the group of 1-coboundaries is $S^2 A\otimes \det B$ . Since the degree 2 term is irreducible and not isomorphic to either of the first two components of the degree 1 term, and since the second differential is nonzero, it follows that the group of 1-cocycles is $(S^4 B\otimes \det A) \oplus (S^2 A \otimes \det B)$ and that $H^1(\operatorname {Gr}_F^0)$ is $S^4 B \otimes \det A$ .

Since each $\operatorname {Gr}_F^p(\mathcal V \otimes \Omega _{{\mathcal M}_3}^{\bullet })$ is a complex of vector bundles over the complement of the hyperelliptic locus, the previous result implies:

Corollary 8.3. When $g=3$ , the differential $\nabla : F^0 \mathcal V \to \mathcal V \otimes \Omega ^1_{{\mathcal M}_3}$ is injective on the complement of the hyperelliptic locus. In addition, the homology sheaves $\mathcal H^1(\operatorname {Gr}^p_F(\mathcal V\otimes \Omega _{{\mathcal M}_3}^{\bullet }))$ vanish on the complement of the hyperelliptic locus when $p>0$ .

Since $\mathbb P(H^0(\Omega ^1_C))^\vee = \mathbb P(A)$ , the Green–Griffiths invariant $\overline {\delta }_C(\nu )$ , if it is nonzero, defines a plane quartic in $\mathbb P(A)$ .

Theorem 8.4 (Collino–Pirola [Reference Collino and Pirola3, Thm. 4.2.4]).

If C is a nonhyperelliptic curve of genus 3, then $\overline {\delta }_C(\nu )$ is a nonzero quartic polynomial that defines the canonical image of C in $\mathbb P(A)$ .

We will give a new proof of it in Section 10 using Teichmüller modular forms.

8.2 Linear algebra

Suppose that C is not hyperelliptic of genus 3. We will regard $S^2 S^2 B \otimes \det A$ as the space of symmetric bilinear forms $S^2 A \otimes S^2 A \to \det A$ . In view of Proposition 8.2(c), we can thus regard the derivative $\overline {\nabla }_C \nu $ as a symmetric bilinear form

$$ \begin{align*}D_C : S^2 A \otimes S^2 A \to \det A. \end{align*} $$

It decomposes as the sum

$$ \begin{align*}D_C = Q_C + R_C \end{align*} $$

of two symmetric forms, where $Q_C \in S^4 B \otimes \det A$ , a natural representative of $\overline {\delta }_C(\nu )$ , and $R_C \in S^2 A \otimes \det B$ , which is a coboundary.

Fix a volume form $\mathrm {vol}_A \in \det A$ and let $\mathrm {vol}_B \in \det B$ be the dual volume form. Identify $\det A$ and $\det B$ with $\mathbb C$ via the isomorphisms

$$ \begin{align*}\mathbb C \to \det A,\quad t \mapsto t\, \mathrm{vol}_A \text{ and } \mathbb C \to \det B, \quad t \mapsto t\, \mathrm{vol}_B. \end{align*} $$

The pairing $Q_C : S^2 A \otimes S^2 A \to \mathbb C$ is (up to a nonzero multiple) given by the formula

$$ \begin{align*}Q_C(u,v) = f(uv)/24, \end{align*} $$

where $f \in S^4 B$ is a generator of the ideal of functions that vanish on the canonical image of C in $\mathbb P(A)$ and where $uv\in S^4 A$ is the product of $u,v\in S^2 A$ .

The pairing $R_C$ is the composite

$$ \begin{align*}S^2 A \to S^2 \Lambda^2 B \to (S^2 B)^{\otimes 2} \to \operatorname{Hom}(S^2 A \otimes S^2 A,\mathbb C), \end{align*} $$

where the first map is induced by the isomorphism

and the second by

$$ \begin{align*}(b_1\wedge b_2) \cdot (b_1'\wedge b_2') \mapsto b_1b_1'\otimes b_2b_2' + b_2b_2'\otimes b_1b_1' - b_1b_2' \otimes b_2b_1' - b_2b_1' \otimes b_1b_2'. \end{align*} $$

We now give explicit formulas (up to multiplication by a nonzero scalar) of them. Fix a basis $\mathbf {e}_0,\mathbf {e}_1,\mathbf {e}_2$ of A such that $\mathrm {vol}_A = \mathbf {e}_0 \wedge \mathbf {e}_1 \wedge \mathbf {e}_2$ . Let $x_0,x_1,x_2$ be the dual basis of B.

Proposition 8.5. If the defining equation f of C in $\mathbb P(A)$ is

$$ \begin{align*}f(x_0,x_1,x_2) = \sum_j a_j x_j^4 + 4\sum_{j\neq k} b_{jk} x_j x_k^3 + 6\sum_{j<k} c_{jk} x_j^2 x_k^2 + 12\sum_j\sum_{k,\ell\neq j} d_j x_j^2x_k x_\ell, \end{align*} $$

then the matrix of $Q_C$ with respect to the basis

$$ \begin{align*}\mathbf{e}_0^2,\ \mathbf{e}_1^2,\ \mathbf{e}_2^2,\ \mathbf{e}_0\mathbf{e}_1,\ \mathbf{e}_0\mathbf{e}_2,\ \mathbf{e}_1 \mathbf{e}_2 \end{align*} $$

of $S^2 A$ is a nonzero multiple of

(12) $$ \begin{align} \begin{pmatrix} a_0 & c_{01} & c_{02} & b_{10} & b_{20} & d_0 \\ c_{01} & a_1 & c_{12} & b_{01} & d_1 & b_{21} \\ c_{02} & c_{12} & a_2 & d_2 & b_{02} & b_{12} \\ b_{10} & b_{01} & d_2 & c_{01} & d_0 & d_1 \\ b_{20} & d_1 & b_{02} & d_0 & c_{02} & d_2 \\ d_0 & b_{21} & b_{12} & d_1 & d_2 & c_{12} \end{pmatrix} \end{align} $$

Sketch of proof.

The $\mathrm {GL}(A)$ -invariant dual pairing $S^4 B \otimes S^4 A \to \mathbb C$ is unique up to a constant. We choose the normalization

$$ \begin{align*}\langle x_{j_1}x_{j_2}x_{j_3}x_{j_4}, \mathbf{e}_{k_1}\mathbf{e}_{k_2}\mathbf{e}_{k_3}\mathbf{e}_{k_4} \rangle = \sum_{\sigma \in \mathbb S_4} \prod_{i=1}^4 \langle x_{j_i}, \mathbf{e}_{k_{\sigma(i)}} \rangle, \end{align*} $$

where $\mathbb S_4$ is the symmetric group on four letters. So, for example,

$$ \begin{align*}f(\mathbf{e}_0^4) = 24\, a_0,\ f(\mathbf{e}_0^3 \mathbf{e}_1) = 4\cdot 6\, b_{10},\ f(\mathbf{e}_0^2\mathbf{e}_1^2) = 6 \cdot 4\, c_{01},\ f(\mathbf{e}_0^2\mathbf{e}_1\mathbf{e}_2) = 12\cdot 2\, d_0. \end{align*} $$

Thus

$$ \begin{align*} Q_C(\mathbf{e}_0^2,\mathbf{e}_0^2) &= f(\mathbf{e}_0^4)/24 = a_0,\ Q_C(\mathbf{e}_0^2,\mathbf{e}_0\mathbf{e}_1) = f(\mathbf{e}_0^3 \mathbf{e}_1)/24 = b_{10}, \\ Q_C(\mathbf{e}_0^2,\mathbf{e}_1^2) &= Q_C(\mathbf{e}_0\mathbf{e}_1,\mathbf{e}_0\mathbf{e}_1) = f(\mathbf{e}_0^2\mathbf{e}_1^2)/24 = c_{01}, \\ Q_C(\mathbf{e}_0^2,\mathbf{e}_1\mathbf{e}_2) &= Q_C(\mathbf{e}_0\mathbf{e}_1,\mathbf{e}_0\mathbf{e}_2) = f(\mathbf{e}_0^2\mathbf{e}_1\mathbf{e}_2)/24 = d_0. \end{align*} $$

The remaining entries are obtained by permuting the indices.

Remark 8.6. For the sake of completeness, we give a formula for the bilinear form $R_C$ . If

$$ \begin{align*}h = \sum_j p_j \mathbf{e}_j^2 + 2\sum_{j<k} q_{jk} \mathbf{e}_j \mathbf{e}_k \in S^2 A \end{align*} $$

is the projection of $\overline {\nabla }_C \nu $ onto $S^2 A\otimes \det B$ , then the matrix of $R_C$ with respect to the basis

$$ \begin{align*}\mathbf{e}_0^2,\ \mathbf{e}_1^2,\ \mathbf{e}_2^2,\ \mathbf{e}_0\mathbf{e}_1,\ \mathbf{e}_0\mathbf{e}_2,\ \mathbf{e}_1 \mathbf{e}_2 \end{align*} $$

of $S^2 A$ is a nonzero multiple of

$$ \begin{align*}\begin{pmatrix} 0 & p_2 & p_1 & 0 & 0 & -q_{12} \\ p_2 & 0 & p_0 & 0 & -q_{02} & 0 \\ p_1 & p_0 & 0 & -q_{01} & 0 & 0 \\ 0 & 0 & -q_{01} & -p_2 & q_{12} & q_{02} \\ 0 & -q_{02} & 0 & q_{12} & -p_1 & q_{01} \\ -q_{12} & 0 & 0 & q_{02} & q_{01} & -p_0 \end{pmatrix} \end{align*} $$

To prove this, one easily checks that the map $S^2 A \to (S^2B)^{\otimes 2}$ satisfies

$$ \begin{align*} \mathbf{e}_0^2 &\mapsto x_1^2 \otimes x_2^2 + x_2^2 \otimes x_1^2 - 2 x_1x_2 \otimes x_1 x_2 \cr \mathbf{e}_0\mathbf{e}_1 &\mapsto x_0 x_2 \otimes x_1 x_2 + x_1 x_2 \otimes x_0 x_2 - x_0 x_1 \otimes x_2^2 - x_2^2 \otimes x_0 x_1. \end{align*} $$

The formulas for the images of the other basis vectors is obtained by cyclically permuting the indices.

8.3 The Klein quartic

Here we prove Theorem 1 in genus 3 by checking that $\overline {\nabla }\nu $ has maximal rank at the moduli point of the Klein quartic C. We do this by using the symmetries of C to show that the tensor $R_C$ vanishes.

The Klein curve is the projective completion $X(7)$ of the modular curve $Y(7) = \Gamma (7)\backslash \mathfrak {h}$ , where $\Gamma (7)$ is the full level 7 subgroup of $\mathrm {SL}_2({\mathbb Z})$ . It has genus 3 and automorphism group $\mathrm {PSL}_2(\mathbb F_7)$ , a simple group of order 168. Its canonical image is the plane quartic

$$ \begin{align*}x^3y+y^3z+z^3x=0. \end{align*} $$

The corresponding matrix (12) has rank 6 and determinant $-729 = -3^6$ . To show that $\overline {\nabla }_C(\nu )$ has maximal rank it suffices to show that $R_C$ vanishes.

Since $\overline {\nabla } \nu $ is a holomorphic section of $\mathcal V\otimes \Omega _{{\mathcal M}_3}^1$ , we conclude that $\overline {\nabla } \nu $ has maximal rank on an open dense subset of ${\mathcal M}_3$ .

10.1 Proof of Corollary 3

Since the Ceresa normal function vanishes on the hyperelliptic locus, a smooth divisor, the rank of $\nu $ at each point of the hyperelliptic locus is at most 1. By Lemma 10.1(e), $\overline {\nabla } f$ vanishes on the hyperelliptic locus. So, to prove the result, we need to show that $\overline {\delta }(\nu )$ has no zeros on the hyperelliptic locus. By Theorem 10.2, the restriction of $\overline {\delta }(\nu )$ to the hyperelliptic locus can be identified with $\chi _{4,0,8}$ . So it suffices to show that the restriction of $\chi _{4,0,8}$ to the hyperelliptic locus has no zeros.

In genus 3, the restriction of $\mathscr B$ to the hyperelliptic locus (regarded as a stack) is $S^2\mathscr W$ , where $\mathscr W$ is the rank 2 vector bundle whose fibre over the moduli point of a hyperelliptic curve is the linear system associated to its hyperelliptic series. By the representation theory of $\mathrm {GL}_2$ , the restriction of $S^4\mathscr B$ to the hyperelliptic locus in genus 3 decomposes

$$ \begin{align*}S^8 \mathscr W \oplus S^4 \mathscr W \otimes (\det \mathscr W)^2 \oplus (\det \mathscr W)^4 \end{align*} $$

and $\det \mathscr B$ restricts to $(\det \mathscr W)^3$ . So the restriction of $S^4\mathscr B\otimes (\det \mathscr B)^8$ to the hyperelliptic locus projects to

$$ \begin{align*}S^8\mathscr W \otimes (\det\mathscr W)^{24} = [S^8\mathscr W \otimes (\det\mathscr W)^{-4}] \otimes (\det \mathscr W)^{28}. \end{align*} $$

By [Reference van der Geer, Kouvidakis and Alexis7, §6], the projection of the restriction of $\chi _{4,0,8}$ to the hyperelliptic locus onto this factor is the product $f_{8,-2}$ of the universal octic times the discriminant $\mathfrak {d}$ of the binary octic, which is a section of $(\det \mathscr W)^{28}$ . Since neither vanish at the moduli point of a smooth hyperelliptic curve, $\chi _{4,0,8}$ has no zeros there.

This strengthens the genus 3 case of the result [Reference Harris18, Thm. 6.5] of Bruno Harris. He proved that the derivative of the normal function of the Ceresa cycle has rank 1 at the general hyperelliptic curve of genus $g\ge 3$ .

Part 4. Admissible normal functions

The inductive proof of the general case requires technical Hodge theory. In particular, it requires an understanding of the asymptotic behaviour of the variations of mixed Hodge structure that correspond to normal functions.

After recalling the definition and basic properties of admissible variations of mixed Hodge structure, we apply them to describe the boundary behaviour of normal functions. We review the construction of the Néron model of a family of intermediate jacobians by Green, Griffiths and Kerr [Reference Green, Griffiths and Kerr10] in the case where the variation is a nilpotent orbit. This is applied to study the asymptotic behaviour of admissible normal functions in codimension 1. In particular, it is used in the definition of residual normal functions, which plays a key role in the inductive proof of Theorem 1.

11.1 Nilpotent orbits

The 1-variable nilpotent orbit above is the restriction of a several variable nilpotent orbit. Such nilpotent orbits of limit MHS approximate admissible variations of MHS near a point of the boundary divisor D and are useful for understanding the boundary behaviour of admissible variations.

Suppose that $U = \mathbb D^n$ is a polydisk neighbourhood in ${\overline {S}}$ of a point of D. Suppose that the intersection of D with U is defined by $t_1t_2\dots t_k = 0$ , where $t_j$ is the coordinate in the jth disk. Let $D_j$ be the component of $D\cap U$ defined by $t_j = 0$ . Denote the fibre of $\mathcal E$ over the origin of U by $E_0$ . Set

$$ \begin{align*}N_j = -\operatorname{Res}_{D_j,0} \nabla \in \operatorname{End} E_0 \end{align*} $$

Then $N_1,\dots ,N_k$ are commuting nilpotent endomorphisms of $E_0$ . There is a trivialization

(18) $$ \begin{align} \mathcal E|_U \cong U\times E_0 \end{align} $$

of the restriction of $\mathcal E$ to U in which

1. (1) $\nabla $ is given by

   $$ \begin{align*}\nabla = d - \sum_{j=1}^k N_j \frac{dt_j}{t_j} \end{align*} $$

2. (2) The isomorphism (18) restricts to an isomorphism of $W_m \mathcal E|_U$ with $U\times W_m E_0$ .

Define a Hodge filtration on $U\times E_0$ by $F^p(U\times E_0) = U \times F^p E_0$ . The fibre of $F^p \mathcal E$ over $t\in U$ corresponds to a subspace $F^p_t E_0$ of $E_0$ via the isomorphism (18). Since both Hodge filtrations of the bundle $U\times E_0$ are holomorphic and agree at the origin, we have:

Suppose that $x=(0,\dots ,0,t_{k+1},\dots ,t_n) \in \cap D_j$ . For each tangent vector

$$ \begin{align*}{\vec v} = \sum_{j=1}^n \lambda_j \frac{\partial}{\partial t_j} \in T_x U \end{align*} $$

that is not tangent to any component of D — equivalently, $\lambda _j \neq 0$ for $1\le j \le k$ — we can restrict ${\mathbb E}$ to the arc $t\mapsto (\lambda _1 t_1,\dots , \lambda _k t_k,t_{k+1},\dots ,t_n)$ to obtain a limit MHS $E_{\vec v}$ whose weight filtration is the relative weight filtration of

$$ \begin{align*}N = N_1 + N_1 + \dots + N_k. \end{align*} $$

The $E_{\vec v}$ form a nilpotent orbit of MHS over

$$ \begin{align*}T_x U - \bigcup_{j=1}^k T_x D_j \end{align*} $$

that is constant on the cosets of $T_x(\bigcap _{j=1}^k D_j)$ . It therefore descends to a nilpotent orbit of MHS on the normal bundle of the intersection of the $D_j$ with the normal bundles of the $D_j$ removed. We shall denote this nilpotent orbit by ${\mathbb E}^{\mathrm {nil}}$ . It is an admissible variation of MHS. The case $k=1$ plays an important role in the proof of Theorem 1.

Nilpotent orbits are themselves admissible VMHS. They behave well under subquotients of admissible variations of MHS. In particular, the nilpotent orbit associated to the subquotient $\operatorname {Gr}^W_r {\mathbb E}$ is $\operatorname {Gr}^W_r{\mathbb E}^{\mathrm {nil}}$ .

11.2 Admissible normal functions

Suppose that $\mathbb V$ is a polarizable variation of Hodge structure of negative weight over a smooth variety S. An admissible normal function is a section $\nu $ of $J(\mathbb V)$ that corresponds to an admissible variation of MHS ${\mathbb E}$ over S that is an extension of ${\mathbb Z}_S(0)$ by $\mathbb V$ .

12.1 Setup

Suppose that ${\overline {S}}$ is a smooth variety (not necessarily compact) and that $\Delta $ is a connected smooth divisor in ${\overline {S}}$ . Set $S={\overline {S}}-\Delta $ . Suppose that $\mathbb V$ is a polarizable variation of Hodge structure of weight $-1$ over S and that $\nu : S \to J(\mathbb V)$ is an admissible normal function. Denote the corresponding variation of MHS by ${\mathbb E}$ . We shall assume that $\mathbb V_{\mathbb Z}$ is torsion free, a condition that is satisfied by all variations in Part 5.

Denote the normal bundle of $\Delta $ in ${\overline {S}}$ by L and the complement of its 0-section, which we identify with $\Delta $ , by $L'$ . Let $\pi : L \to \Delta $ be the projection and $\pi '$ its restriction to $L'$ . By the discussion in Section 11.1, there are nilpotent orbits of MHS ${\mathbb E}^{\mathrm {nil}}$ and $\mathbb V^{\mathrm {nil}}$ over $L'$ and an extension

(19) $$ \begin{align} 0 \to \mathbb V^{\mathrm{nil}} \to {\mathbb E}^{\mathrm{nil}} \to {\mathbb Q}\mathbf{e} \to 0 \end{align} $$

of variations over $L'$ . Both $\mathbb V^{\mathrm {nil}}$ and ${\mathbb E}^{\mathrm {nil}}$ are filtered by subvariations $M_j\mathbb V^{\mathrm {nil}}$ and $M_j{\mathbb E}^{\mathrm {nil}}$ .

Even though $\mathbb V^{\mathrm {nil}}$ is not a variation of Hodge structure of pure weight $-1$ (except when $N=0$ ), we can still define $J(\mathbb V^{\mathrm {nil}})$ to be the (not necessarily topologically locally trivial) family of complex tori over $L'$ whose fibre over ${\vec v} \in L_x'$ is

$$ \begin{align*}J(V_{\vec v}) := V_{x,\mathbb C}/(V_{{\vec v},{\mathbb Z}} + F^0 V_x). \end{align*} $$

Schmid’s result [Reference Schmid26, Thm. 4.9] implies that there is a neighbourhood U of the zero section of L such that $\mathbb V^{\mathrm {nil}}$ is a polarizable variation of HS of weight $-1$ over $U' := U - \Delta $ . Consequently, $J(\mathbb V^{\mathrm {nil}})$ is a family of compact complex tori over $U'$ . Denote by $\nu ^{\mathrm {nil}} : U' \to J(\mathbb V^{\mathrm {nil}})$ the normal function section of it that corresponds to the restriction of (19) to $U'$ .

Since the monodromy logarithm $N : \mathbb V^{\mathrm {nil}} \to \mathbb V^{\mathrm {nil}}(-1)$ is a morphism, $\ker N$ is a subvariation of $M_{-1}\mathbb V^{\mathrm {nil}}$ . Since it has trivial monodromy, it is constant on each $L_x'$ and thus extends to a variation over L, which we denote by ${\mathbb K}$ . Since it is constant on each $L_x$ , it is the pullback along $\pi $ of its restriction ${\mathbb K}_0$ to $\Delta $ .

12.2 The Néron model associated to a nilpotent orbit

Here we give the construction of the Néron model for $\mathbb V^{\mathrm {nil}}$ on the punctured neighbourhood $U'$ of the zero section of L. It is more general than the construction given in [Reference Green, Griffiths and Kerr10] as we allow the base to have dimension $>1$ , but it is less general in that we construct Néron models only for nilpotent orbits. Working with nilpotent orbits is sufficient for our purposes and simplifies the construction as nilpotent orbits are filtered by their relative weight filtrations.

Regard $J(\mathbb V_{\mathbb R})$ as a local system of abelian groups over $L'$ .

Lemma 12.1. For each $x\in \Delta $ , there is an exact sequence

$$ \begin{align*}0 \to J({\mathbb K}_{\mathbb R})|_{L_x} \to H^0(L_x',J(\mathbb V_{\mathbb R})) \to G_x \to 0, \end{align*} $$

where $G_x$ is the finite abelian group

$$ \begin{align*}G_x := \text{torsion subgroup of }H_0(L_x',\mathbb V_{\mathbb Z}|_{L_x'}). \end{align*} $$

In more concrete terms

$$ \begin{align*}G_x = \big(V_{{\vec v},{\mathbb Z}} \cap (h-1)V_{{\vec v},{\mathbb Q}} \big)/(h - 1) V_{{\vec v},{\mathbb Z}}, \end{align*} $$

where ${\vec v} \in L_x'$ and $h : V_{{\vec v},{\mathbb Z}} \to V_{{\vec v},{\mathbb Z}}$ is the local monodromy operator.

Proof. Denote $\mathbb V\otimes _{\mathbb Z} {\Bbbk }$ by $\mathbb V_{\Bbbk }$ , where ${\Bbbk }$ is a subring of ${\mathbb R}$ . Since $V_{\mathbb Z}$ is torsion free, the long exact sequence of cohomology associated to the exact sequence

$$ \begin{align*}0 \to \mathbb V_{\mathbb Z} \to \mathcal V_{\mathbb R} \to J(\mathbb V_{\mathbb R}) \to 0 \end{align*} $$

of local systems over $L_x'$

$$ \begin{align*}0 \to H^0(L_x',\mathbb V_{\mathbb Z}) \to H^0(L_x',\mathcal V_{\mathbb R}) \to H^0(L_x',J(\mathbb V_{\mathbb R})) \to H^1(L_x',\mathbb V_{\mathbb Z}) \to H^1(L_x',\mathbb V_{\mathbb R}). \end{align*} $$

Since $H^1(L_x',\mathbb V_{\mathbb R})$ is torsion free and since $H^0(L_x',\mathbb V_{\Bbbk }) = K_{\Bbbk }$ , the sequence

$$ \begin{align*}0 \to K_{\mathbb Z} \to K_{\mathbb R} \to H^0(L_x',J(\mathbb V_{\mathbb R})) \to H^1(L_x',\mathbb V_{\mathbb Z})^{\mathrm{tor}} \to 0 \end{align*} $$

is exact, where $K_{\Bbbk }$ denotes the fibre of ${\mathbb K}_{\Bbbk }$ over $x\in \Delta $ . Since $L_x'$ has the homotopy type of a circle, we have the Poincaré duality isomorphism $H^1(L_x',\mathbb V_{\mathbb Z}) \cong H_0(L_x',\mathbb V_{\mathbb Z})$ . It restricts to an isomorphism on torsion subgroups. The results now follows as $H^0(L_x',J({\mathbb K})) = K_{\mathbb R}/K_{\mathbb Z}$ .

The groups $G_x$ form a local system over L. Denote it by $\mathbb G$ . Denote the local system over L whose restriction to $L_x$ is $H^0(L_x',\mathbb V)$ by $\mathbb T$ . We have a short exact sequence of local systems

$$ \begin{align*}0 \to {\mathbb K} \to \mathbb T \to \mathbb G \to 0. \end{align*} $$

All terms in this sequence are admissible variations of MHS over L. The quotient variation $\mathbb G$ is Tate of weight 0. Define $J(\mathbb T)$ by defining its fibre over $x\in \Delta $ to be

$$ \begin{align*}J(T_x) = T_{x,\mathbb C}/(K_{x,{\mathbb Z}} + F^0 K_x). \end{align*} $$

There is an exact sequence

$$ \begin{align*}0 \to J({\mathbb K}) \to J(\mathbb T) \to \mathbb G \to 0 \end{align*} $$

over L. The inclusion $\mathbb T \to \mathbb V^{\mathrm {nil}}$ induces a morphism

$$ \begin{align*}J(\mathbb T)|_{L'} \to J(\mathbb V^{\mathrm{nil}}) \end{align*} $$

over $L'$ . The Néron model $\widehat {J}(\mathbb V^{\mathrm {nil}})$ of $J(\mathbb V^{\mathrm {nil}})$ is the family of abelian complex Lie groups over U obtained by glueing $J(\mathbb V^{\mathrm {nil}})|_{U'}$ to $J(\mathbb T)|_U$ by identifying $J(\mathbb T)|_{U'}$ . In other words

is a pushout square. Its restriction to $U'$ is $J(\mathbb V^{\mathrm {nil}})|_{U'}$ and its fibre over the zero section is $J(\mathbb T)|_\Delta $ . It is a separated slit analytic space. (See [Reference Green, Griffiths and Kerr10, p. 308] for the definition.) If $N^2 \neq 0$ , $\widehat {J}(\mathbb V^{\mathrm {nil}})$ is not a manifold.

The following result is closely related to the main results of [Reference Green, Griffiths and Kerr10] and [Reference Schnell27].

Theorem 12.3. The normal function $\nu ^{\mathrm {nil}}$ lifts to a holomorphic section $\tilde {\nu }^{\mathrm {nil}}$ of $J(\mathbb T)$ over U and this section projects to a holomorphic section $\widehat {\nu }^{\mathrm {nil}}$ of $\widehat {J}(\mathbb V^{\mathrm {nil}})$ whose restriction to $U'$ is $\nu ^{\mathrm {nil}}$ .

Proof. We work locally in L. Let X be a contractible open Stein subset of $\Delta $ . Set

$$ \begin{align*}Z = U \cap \pi^{-1}(X), \end{align*} $$

where $\pi : L \to \Delta $ denotes the projection, and $Z' = Z\cap U'$ . Recall that the restriction of $\mathcal E$ (and thus $\mathcal V$ ) to each $L_x$ is trivial, so we can (and will) identify their fibres $E_z$ and $V_z$ over $z \in Z$ with $E_x$ and $V_x$ , respectively, where $z\in L_x'\cap U$ .

Exactness of $F^0$ and $M_0$ implies that there is a holomorphic section $\mathbf {e}_F$ of $F^0 M_0 \mathcal E$ over Z that is constant (with respect to the trivialization) on each $L_x'$ and projects to the section $1$ of $\operatorname {Gr}^W_0{\mathbb E}_{\mathbb Z} \cong {\mathbb Z}$ . There is also a multivalued section $\mathbf {e}_{\mathbb Z}$ of ${\mathbb E}_{\mathbb Z}$ over $Z'$ that projects to $1\in H^0(Z',\operatorname {Gr}^W_0 {\mathbb E}^{\mathrm {nil}}$ ). Then

$$ \begin{align*}\mathbf{v} := \mathbf{e}_{\mathbb Z} - \mathbf{e}_F \end{align*} $$

is a multivalued holomorphic section of $\mathcal V$ over $Z'$ which descends to $\nu ^{\mathrm {nil}} \in H^0(Z',J(\mathbb V^{\mathrm {nil}}))$ . To prove the theorem, we will show that it lifts to a section of $J(\mathbb T)$ over $Z'$ .

Denote the local monodromy operator by h and its logarithm by N. Since h is unipotent, the endomorphisms $(h-1)$ and N of $E_{z,{\Bbbk }}$ satisfy

$$ \begin{align*}\operatorname{im} N = \operatorname{im} (h-1) \text{ and } \ker N = \ker (h-1) \end{align*} $$

when ${\Bbbk }$ is a subfield of $\mathbb C$ . The property (16) of the relative weight filtration implies that

$$ \begin{align*}\ker\{N : V_{z,{\Bbbk}} \to V_{z,{\Bbbk}}\} \subseteq M_{-1} V_{z,{\Bbbk}},\quad \operatorname{im}\{ (h-1) : V_{z,{\Bbbk}} \to V_{z,{\Bbbk}}\} \supseteq M_{-2} V_{z,{\Bbbk}}. \end{align*} $$

Since N is a morphism of type $(-1,-1)$ ,

$$ \begin{align*}\operatorname{im} \{ N : F^0 M_0 E_z \to V_z\} \supseteq F^{-1} M_{-2} V_z. \end{align*} $$

So $N \mathbf {e}_F$ is a section of $F^{-1}M_{-2}\mathcal V$ . The properties of the relative weight filtration imply that there is a holomorphic section $\mathbf {e}'$ of $F^0 M_0 \mathcal V$ over Z such that $N\mathbf {e}_F = N\mathbf {e}'$ . It is constant (with respect to the trivialization) on each $L_x'\cap U$ . By replacing $\mathbf {e}_F$ by $\mathbf {e}_F-\mathbf {e}'$ , we may assume that $N\mathbf {e}_F = 0$ . That is, the restriction of $\mathbf {e}_F$ to each $L_x$ is also a horizontal section of $\mathcal K$ .

The class of $\nu ^{\mathrm {nil}}$ in $H^0(X,\mathbb G)$ is represented by $(h-1)\mathbf {e}_{\mathbb Z}(z)$ in

$$ \begin{align*}G_x = \big(V_{z,{\mathbb Z}} \cap (h-1)V_{z,{\mathbb Q}} \big)/(h - 1) V_{z,{\mathbb Z}} \quad \text{for all } z \in L_x'\cap U. \end{align*} $$

If it vanishes at one (and hence all) $x\in X$ , there is a multivalued section $\mathbf {v}_{\mathbb Z}$ of $\mathbb V_{\mathbb Z}$ over $Z'$ such that $(h-1)\mathbf {v}_{\mathbb Z} = (h-1)\mathbf {e}_{\mathbb Z}$ . By replacing $\mathbf {e}_{\mathbb Z}$ by $\mathbf {e}_{\mathbb Z} - \mathbf {v}_{\mathbb Z}$ , we see that if the class of $\nu ^{\mathrm {nil}}$ vanishes in $H^0(Z',\mathbb G)$ , then we may choose $\mathbf {e}_{\mathbb Z}$ so that $(h-1)\mathbf {e}_{\mathbb Z} = 0$ . That is, $\mathbf {e}_{\mathbb Z}$ extends to a section of ${\mathbb K}_{\mathbb Z}$ over Z. Since $\mathbf {e}_F$ extends to a holomorphic section of $\mathcal K$ over Z, $\mathbf {v} = \mathbf {e}_{\mathbb Z} - \mathbf {e}_F$ also extends to a holomorphic section of $\mathcal K$ over Z. It is constant on each slice $L_x'\cap U$ and descends to a holomorphic section of $J({\mathbb K})$ over Z whose restriction to $Z'$ is a lift of $\nu ^{\mathrm {nil}}$ . This completes the proof when the obstruction in $H^0(X,\mathbb G)$ vanishes. We now prove the general case.

Every element of $H^0(U',J(\mathbb V))$ extends to U. Since $J(\mathbb V) \cong J(\mathbb V_{\mathbb R})$ , every element of $G_x$ is the class of a constant section of $J(\mathbb V)$ over $U\cap L_x'$ . All such sections lift to sections of $\widehat {J}(\mathbb T)$ over $U\cap L_x$ . So, if the class of $\nu ^{\mathrm {nil}}$ in $H^0(X,\mathbb G)$ is nonzero, we can find a constant section $\overline {\nu }$ of $J(\mathbb V)$ with the same class in $G_x$ . Since the class of $\nu ^{\mathrm {nil}}-\overline {\nu }$ vanishes, by the previous paragraph, it lifts to a holomorphic section of $J({\mathbb K})$ over $U\cap L_x$ for all $x \in X$ . Since $\overline {\nu }$ lifts to a section of $\widehat {J}(\mathbb T)$ , it follows that $\nu ^{\mathrm {nil}}$ lifts to a section $\widehat {\nu }^{\mathrm {nil}}$ of $\widehat {J}(\mathbb T)$ over Z whose restriction to $Z'$ projects to $\nu ^{\mathrm {nil}}$ .

Since ${\overline {S}}-S$ is a smooth connected divisor $\Delta $ , the local system $\mathbb V$ extends to a local system on ${\overline {S}}$ if and only if $N=0$ . In this case, the variation $\mathbb V$ extends to a polarized variation of Hodge structure over ${\overline {S}}$ . The fibre over $x\in \Delta $ is the limit MHS associated to any tangent vector ${\vec v}$ of ${\overline {S}}$ at x that is not tangent to $\Delta $ . The limit does not depend on the choice of ${\vec v}$ .

The derivative of the lift $\tilde {\nu }^{\mathrm {nil}}$ of $\nu ^{\mathrm {nil}}$ constructed in Theorem 12.3 is a section of $F^{-1}\mathcal V\otimes \Omega ^1_{\overline {S}}(\log \Delta )$ . Combined with the Nilpotent Orbit theorem (Proposition 11.2), this implies that the Green–Griffiths invariant of an admissible normal function is logarithmic along the smooth boundary divisor $\Delta $ .

Corollary 12.6. The Green–Griffiths invariant $\overline {\delta }(\nu )$ of $\nu $ extends to a holomorphic section of

$$ \begin{align*}\operatorname{Gr}_F^{-1}\mathcal V \otimes \Omega^1_{\overline{S}}(\log\Delta)/\overline{\nabla} \operatorname{Gr}_F^0 \mathcal V. \end{align*} $$

over ${\overline {S}}$ .

13.1 The residual normal function $\nu _\Delta $

By Theorem 12.3, $\nu ^{\mathrm {nil}}$ has a natural lift $\tilde {\nu }^{\mathrm {nil}}$ to a section of $J(\mathbb T)$ over a neighbourhood U of $\Delta $ in L. The order of the class of $\nu ^{\mathrm {nil}}$ in $H^0(U,\mathbb G)$ is the least positive integer such that $k\tilde {\nu }^{\mathrm {nil}}$ is a section of $J({\mathbb K})$ . Since ${\mathbb K} \subseteq M_{-1}\mathbb V^{\mathrm {nil}}$ , $k\tilde {\nu }^{\mathrm {nil}}$ descends to a section of $J(\operatorname {Gr}^M_{-1}\mathbb V^{\mathrm {nil}})$ over U. Set

$$ \begin{align*}\mathbb V_\Delta := \operatorname{Gr}^M_{-1} \mathbb V^{\mathrm{nil}}. \end{align*} $$

This is a polarizable variation of Hodge structure over $\Delta $ of weight $-1$ .