Proof Our assumptions on the prime p guarantee that ${\mathsf {s}}_{ii}\neq {\mathsf {s}}_{jj}$ if and only if ${\bar {{\mathsf {s}}}_{ii}\neq \bar {{\mathsf {s}}}_{jj}}$ . Since ${\mathsf {n}}$ is a 0–1 matrix, we furthermore have that ${\mathsf {n}}$ and $\bar {{\mathsf {n}}}$ have the same pivots, and in particular, that ${\mathsf {n}}_{{\mathsf {s}}_{ii}}$ and $\bar {{\mathsf {n}}}_{{\mathsf {s}}_{ii}}$ have pivots that occur in the rightmost columns. Since $g\in \bar {U}$ , g is upper triangular with $1$ ’s on the diagonal. We have

$$\begin{align*}g^{-1}\bar{\mathsf{x}}g = g^{-1}(\bar{{\mathsf{s}}}+\bar{{\mathsf{n}}})g = \bar{{\mathsf{s}}} + {\mathsf{n}}', \end{align*}$$

where ${\mathsf {n}}':= g^{-1}\bar {{\mathsf {n}}}g + g^{-1}\bar {{\mathsf {s}}}g-\bar {{\mathsf {s}}} \in \mathfrak {gl}_n(\mathbb {F}_p)$ is a nilpotent matrix. Since $\bar {{\mathsf {s}}}$ is diagonal, $(g^{-1}\bar {{\mathsf {s}}}g-\bar {{\mathsf {s}}})_{ij}=0$ whenever ${\mathsf {s}}_{ii}={\mathsf {s}}_{jj}$ . In particular, we get that nilpotent matrices ${\mathsf {n}}^{\prime }_{{\mathsf {s}}_{ii}}$ and $\bar {{\mathsf {n}}}_{{\mathsf {s}}_{ii}}$ and ${\mathsf {n}}_{{\mathsf {s}}_{ii}}$ have pivots in exactly the same entries (cf. [Reference TymoczkoTym06b, Proposition 4.6]).

By definition,

$$\begin{align*}\bar{U}_{i}\cap \bar{U}_{w} = \left\{ I_n + \sum_{(i\neq\ell)} a_{i\ell}E_{i\ell} \mid a_{i,\ell} \in \mathbb{F}_p,\, a_{i,\ell}=0 \,\text{ unless }\, (i<\ell)\in \mathsf{Inv}(w^{-1}) \right\}. \end{align*}$$

To complete the proof, we argue that the system of equations defining the set (3.2) is consistent and has $d_i$ free variables. Let $u = I_n + \sum _{(i\neq \ell )} a_{i,\ell }E_{i,\ell }$ be a generic element of $\bar {U}_{i} \cap \bar {U}_{w}$ . Note that $u^{-1}= I_n - \sum _{(i\neq \ell )} a_{i,\ell }E_{i,\ell }$ . A straightforward computation now yields

$$ \begin{align*} (u^{-1}g^{-1}\bar{\mathsf{x}} g u)_{ij} &= (u^{-1}\bar{{\mathsf{s}}} u)_{ij} + (u^{-1}{\mathsf{n}}'u)_{ij}\\ &= (\bar{{\mathsf{s}}}_{ii}-\bar{{\mathsf{s}}}_{jj})a_{ij} + {\mathsf{n}}_{ij}' - \sum_{\substack{(i<\ell) \in \mathsf{Inv}(w^{-1})\\ \ell<j}} a_{i\ell}{\mathsf{n}}_{\ell j}'. \end{align*} $$

If ${\mathbf m}(w^{-1}(j)) < w^{-1}(i)$ , then $(i<j) \in \mathsf {Inv}(w^{-1})$ and using the formula above, we see that each of the linear equations defining the set (3.2) is of the form

(3.3) $$ \begin{align} 0= (\bar{{\mathsf{s}}}_{ii}-\bar{{\mathsf{s}}}_{jj})a_{ij} + {\mathsf{n}}_{ij}' - \sum_{\substack{(i<\ell) \in \mathsf{Inv}(w^{-1})\\ \ell<j}} a_{i\ell}{\mathsf{n}}_{\ell j}'. \end{align} $$

If ${\mathsf {s}}_{ii}\neq {\mathsf {s}}_{jj}$ , then (3.3) has solution

$$\begin{align*}a_{ij} = (\bar{{\mathsf{s}}}_{jj}-\bar{{\mathsf{s}}}_{ii})^{-1} \left( {\mathsf{n}}_{i,j}' - \sum_{\substack{(i<\ell) \in \mathsf{Inv}(w^{-1})\\ \ell<j}} a_{i\ell}{\mathsf{n}}_{\ell j}' \right). \end{align*}$$

Now suppose ${\mathsf {s}}_{ii}= {\mathsf {s}}_{jj}$ and (3.3) has at least one nonzero term, namely, there exists $\ell $ with $i\leq \ell <j$ such that ${\mathsf {n}}_{\ell j}'\neq 0$ . Since $\mathsf {x}$ is in HFPJF, all the pivots of ${\mathsf {n}}_{{\mathsf {s}}_{ii}}'$ occur in the rightmost columns, so ${\mathsf {n}}'$ and therefore ${\mathsf {n}}$ must have a pivot in column j and row k with $\ell \leq k<j$ . Equivalently, entry $(k,j)$ is a pivot of ${\mathsf {n}}_{{\mathsf {s}}_{ii}} = {\mathsf {n}}_{{\mathsf {s}}_{kk}}$ , and our assumption that ${\mathsf {n}} \in \dot {w}H({\mathbf m})\dot {w}^{-1}$ implies $w^{-1}(k)\leq {\mathbf m}(w^{-1}(j))$ so $w^{-1}(k)< w^{-1}(i)$ . Thus, $(i<k)\in \mathsf {Inv}(w^{-1})$ and ${\mathsf {n}}_{kj}'\neq 0$ so the term $a_{ik}{\mathsf {n}}_{kj}'$ appears in the sum on the RHS of (3.3). This shows that, in the case where ${\mathsf {s}}_{ii}={\mathsf {s}}_{jj}$ , equation (3.3) is not vacuous if and only if there exists $(i<k)\in \mathsf {Inv}(w^{-1})$ such that ${\mathsf {n}}_{{\mathsf {s}}_{ii}}$ has a pivot in entry $(k,j)$ . Furthermore, in this case, the equation has solution:

$$\begin{align*}a_{ik} = ({\mathsf{n}}_{kj}')^{-1} \left( {\mathsf{n}}_{i,j}' - \sum_{\substack{(i<\ell) \in \mathsf{Inv}(w^{-1})\\ \ell<j, \ell\neq k}} a_{i\ell}{\mathsf{n}}_{\ell j}' \right). \end{align*}$$

We have now shown that each equation in the system has a solution, and the system must in fact be consistent since each solution corresponds to either a unique pair $(i<j)\in \mathsf {Inv}(w^{-1})$ such that ${\mathsf {s}}_{ii}\neq {\mathsf {s}}_{jj}$ or a unique pair $(i<k)\in \mathsf {Inv}(w^{-1})$ such that ${\mathsf {s}}_{ii}={\mathsf {s}}_{kk}$ and entry $(k,j)$ is a pivot of ${\mathsf {n}}_{{\mathsf {s}}_{ii}}$ (all pivots must occur in distinct rows). The number of free variables in the system is equal to the number of inversions $(i<j)$ of $w^{-1}$ such that ${\mathsf {s}}_{ii}\neq {\mathsf {s}}_{jj}$ and $w^{-1}(i)\leq {\mathbf m}(w^{-1}(j))$ plus

$$\begin{align*}\left|\left\{ (i<k) \in \mathsf{Inv}(w^{-1}) \mid \begin{array}{@{}c@{}} {\mathsf{s}}_{ii}={\mathsf{s}}_{kk} \text{ and if } {\mathsf{n}}_{kj} \text{ is a pivot of} \\{\mathsf{n}}_{{\mathsf{s}}_{ii}}, \text{ then } w^{-1}(i)\leq {\mathbf m}(w^{-1}(j)) \end{array} \right\} \right|. \end{align*}$$

The proof of the lemma is complete.

Proof Let $w\in S_n$ . Since ${\mathsf {n}}$ is a 0–1 matrix, $\bar {{\mathsf {n}}}_{ij}\neq 0$ if and only if ${\mathsf {n}}_{ij}\neq 0$ . We therefore have ${\mathsf {n}} \in \dot {w} H({\mathbf m})\dot {w}^{-1}$ if and only if $\bar {{\mathsf {n}}}\in \dot {w}\bar {H}({\mathbf m})\dot {w}^{-1}$ . Furthermore, for every $g\in \bar {U}_{w}$ , we get $g^{-1}\bar {{\mathsf {n}}} g = \bar {{\mathsf {n}}} + \mathsf {y}$ , where

$$\begin{align*}\mathsf{y} \in \mathbb{F}_p\{ E_{ij} \mid \text{there exists } (k<\ell) \text{ such that } i<k, j>\ell,\; {\mathsf{n}}_{k\ell}\neq 0 \}. \end{align*}$$

Thus, we may conclude that $g^{-1}\bar {{\mathsf {n}}} g \in \dot {w}\bar {H}({\mathbf m})\dot {w}^{-1}$ implies $\bar {{\mathsf {n}}} \in \dot {w}\bar {H}({\mathbf m})\dot {w}^{-1}$ since $\bar {{\mathsf {n}}}$ and $\mathsf {y}$ have disjoint support in $\mathfrak {gl}_n({\mathbb F}_p)$ . This proves that

(3.4) $$ \begin{align} {C}_{w,p}\cap \mathcal{B}_p(\bar{\mathsf{x}},\bar{H}({\mathbf m}))\neq \emptyset\Leftrightarrow \bar{{\mathsf{n}}} \in \dot{w}\bar{H}({\mathbf m})\dot{w}^{-1} \Leftrightarrow {\mathsf{n}}\in \dot{w}H({\mathbf m})\dot{w}^{-1}. \end{align} $$

We argue now that any nonempty $C_{w,p}\cap \mathcal {B}_p(\bar {\mathsf {x}},\bar {H}({\mathbf m}))$ is isomorphic with the affine space ${\mathbb F}_p^{d_{w,{\mathsf x}}^{\mathbf m}}$ .

We adapt Tymoczko’s proof of Theorem 3.2 to the finite field setting (see [Reference TymoczkoTym06b, Theorem 6.1]). Define

$$ \begin{align*}Z_i:=\{u \in (\bar{U}_{n}\bar{U}_{n-1} \ldots \bar{U}_{i}) \cap \bar{U}_{w}\mid (u^{-1}\bar{\mathsf{x}}u)_{jk}=0\, \text{ for all }\, k>j\geq i,\; {\mathbf m}(w^{-1}(k))<w^{-1}(j) \}. \end{align*} $$

If $u\in (\bar {U}_{n}\bar {U}_{n-1} \ldots \bar {U}_{i})\cap \bar {U}_{w}$ , then

(3.5) $$ \begin{align} u = gu'\, \text{ for unique } \,u'\in \bar{U}_{i}\cap \bar{U}_{w}\, \text{ and } g\in (\bar{U}_{n}\bar{U}_{n-1}\dots \bar{U}_{i+1})\cap \bar{U}_{w}. \end{align} $$

Furthermore, since conjugation by an element in $\bar {U}_{i}$ affects only the first i rows of an upper triangular matrix, if $u=gu'\in Z_i$ , then $g\in Z_{i+1}$ . Thus, the factorization from (3.5) yields a well-defined projection $\pi _i: Z_i \to Z_{i+1}$ defined by $\pi _i(u) = g$ . We obtain a sequence of projections

$$ \begin{align*}Z_1 \xrightarrow{\; \pi_1\; }Z_2 \xrightarrow{\; \pi_2 \;} \dots Z_{n-1} \xrightarrow{\; \pi_{n-1}\;} Z_n = \{I_n\} \end{align*} $$

such that for each $g\in Z_{i+1}$ ,

$$ \begin{align*} \pi_i^{-1}(g) = \{ u\in \bar{U}_{i}\cap \bar{U}_{w} \mid (u^{-1}g^{-1}\bar{\mathsf{x}}gu)_{ik} = 0 \, \text{ for all }\, k>i, {\mathbf m}(w^{-1}(k))<w^{-1}(i) \}. \end{align*} $$

By Lemma 3.4, the fiber $\pi _i^{-1}(g)$ is isomorphic to affine space of dimension $d_i$ . Furthermore, the system of equations from (3.2) defining $\pi ^{-1}(g)$ varies continuously in g by conjugation, so $\pi _i$ is a fiber bundle. We have an isomorphism $Z_{i} \to Z_{i+1}\times \mathbb {F}_p^{d_i}$ defined by sending $gu'$ to $(g, v_{u'})$ , where $v_{u'}$ is the vector in $\mathbb {F}_p^{d_i}$ determined by the free entries in $u'$ (as defined by the consistent system of equations from (3.2)). Thus, $\pi _i$ is a trivial bundle.

To complete the proof, note that the isomorphism $\bar {U}_{w} \to C_{w,p}$ from (3.1) restricts to an isomorphism $Z_1\to C_{w,p}\cap \mathcal {B}_p(\bar {\mathsf {x}}, \bar {H}({\mathbf m}))$ . Since $Z_n = \{I_n\}$ and each $\pi _i$ is a trivial bundle, we conclude that $C_{w,p}\cap \mathcal {B}_p(\bar {\mathsf {x}},\bar {H}({\mathbf m}))$ is isomorphic to affine space of dimension $d_{w,\mathsf {x}}^{\mathbf m}=\sum _{i=1}^{n-1}d_i$ . This immediately implies $|C_{w,p}\cap \mathcal {B}_p(\bar {\mathsf {x}},\bar {H}({\mathbf m}))| = p^{d_{w,\mathsf {x}}^{{\mathbf m}}}$ , as desired.

Proof of Theorem 5

Fix $gB \in \mathcal {B}(\mathsf {x},H({\mathbf m}_{\mathbf {{max}}}))$ . By Lemma 5.1 and the Jacobian criterion for smoothness, $gB$ is a smooth point if and only if $\det (A_g)$ has a nonzero linear term. Write

$$ \begin{align*}g=[v_1|\dots |v_n]. \end{align*} $$

So, $v_i=ge_i$ for all $i \in [n]$ . Now, for each i, set

$$ \begin{align*}v_i^\ast:=\sum_{j=i+1}^nz_{ji}v_j \in M_{n,1}({\mathbb{C}}[{\mathbf z}]). \end{align*} $$

We observe that

$$ \begin{align*}A_g=[\mathsf{x}v_1+\mathsf{x}v_1^\ast | v_1+v_1^\ast | \dots | v_{n-1}+v_{n-1}^\ast]. \end{align*} $$

Using multilinearity, we write $\det (A_g)$ as the sum of $2^n$ terms of the form

(5.3) $$ \begin{align} \det([t_0 | t_1 | \dots | t_{n-1}]), \end{align} $$

where $t_0 \in \{\mathsf {x}v_1,\mathsf {x}v_1^\ast \}$ and $t_i \in \{v_i,v_i^\ast \}$ for each $i \in [n-1]$ . If there exist $i,j \in [n-1]$ such that $t_i=v_i^\ast $ and $t_j=v_j^\ast $ then the determinant in (5.3) has no linear term. The same holds if $t_0=\mathsf {x}v_1^\ast $ and there is some $i \in [n-1]$ with $t_i=v_i^\ast $ . (Indeed, in both these cases, the matrix $[t_0 | t_1 | \dots | t_{n-1}]$ contains two columns whose entries are polynomials in ${\mathbb {C}}[{\mathbf z}]$ with no constant term.) If $t_0=\mathsf {x}v_1$ and $t_i=v_i$ for all $i \in [n-1],$ we have that $\det ([t_0 | t_1 | \dots | t_{n-1}])\in {\mathbb {C}}$ . The remaining terms from (5.3) are

$$ \begin{align*}D_0:=\det([\mathsf{x}v_1^\ast | v_1 | v_2 | \dots | v_{n-1}]) \end{align*} $$

and, for each $i \in [n-1]$ ,

$$ \begin{align*}D_i:=\det([\mathsf{x}v_1 | v_1 | \dots | v_{i-1} | v_i^\ast | v_{i+1} | \dots | v_{n-1}]). \end{align*} $$

Using multilinearity again, we see that

(5.4) $$ \begin{align} D_0=\sum_{j=2}^n z_{j1} \det([\mathsf{x}v_j|v_1| \dots | v_{n-1}]), \end{align} $$

and

(5.5) $$ \begin{align} D_i=z_{ni}\det([\mathsf{x}v_1|v_1| \dots | v_{i-1} | v_n | v_{i+1} | \dots | v_{n-1}]) \end{align} $$

for each $i \in [n-1]$ .

Keeping in mind the bijection from cosets to flags mentioned in Section 2, we see that $gB \in \mathcal {B}(\mathsf {x},H({\mathbf m}_{\mathbf {{max}}}))$ implies $\mathsf {x}v_1 \in {\mathbb {C}}\{v_i \mid i \in [n-1]\}$ . Write

$$ \begin{align*}\mathsf{x}v_1=\sum_{k=1}^{n-1} \gamma_k v_k, \end{align*} $$

with each $\gamma _i \in {\mathbb {C}}$ . We observe that, for each $i \in [n-1]$ , $\gamma _i \neq 0$ if and only if $D_i \neq 0$ . Similarly, $D_0=0$ if and only if $\mathsf {x}v_j \in {\mathbb {C}}\{v_i\mid i \in [n-1]\}$ whenever $2 \leq j \leq n$ . The only variable that appears more than once in the $D_j$ ’s, $j\in \{0,1,\ldots , n-1\}$ , is $z_{n1}$ , which appears in both $D_0$ and $D_1$ . We conclude that $\det (A_g)$ contains a linear term whenever $D_i\neq 0$ for some $2\leq i\leq n-1$ or whenever $\det ([\mathsf {x}v_i|v_1| \dots | v_{n-1}])\neq 0$ for some $2\leq i\leq n-1$ .

To prove the first statement of the theorem, assume that $gB$ is singular. Since $\det (A_g)$ has no linear term, we have $D_i=0$ for all $2\le i \le n-1$ and it follows that

(5.6) $$ \begin{align} \mathsf{x}v_1\in {\mathbb{C}}\{v_1\}. \end{align} $$

We also have that $\det ([\mathsf {x}v_i|v_1| \dots | v_{n-1}])=0$ for all $2\le i \le n-1$ which, combined with (5.6), implies that for all $i\in [n-1]$ ,

(5.7) $$ \begin{align} \mathsf{x}({\mathbb{C}}\{v_j \mid j\in[i]\})\subseteq {\mathbb{C}}\{v_k \mid k \in [n-1]\}. \end{align} $$

The bijection from cosets to flags gives us that $gB\in \mathcal {B}(\mathsf {x},H((1,n-1,\ldots ,n-1,n)))$ if and only if (5.6) and (5.7) hold. We conclude that $gB\in \mathcal {B}(\mathsf {x},H((1,n-1,\ldots , n-1,n)))$ .

To prove the second statement of the theorem, assume that $\textsf {x}$ is nilpotent and $gB\in \mathcal {B}(\mathsf {x},H((1,n-1,\ldots ,n-1,n)))$ . Since (5.6) holds and the only eigenvalue of $\mathsf {x}$ is zero, $\mathsf {x}v_1 = 0$ . It follows that for each $i\in [n-1]$ , $\gamma _i=0$ and thus $D_i=0$ . Equation (5.7) must also hold, so the hyperplane ${\mathbb {C}}\{v_i \mid i \in [n-1]\}$ is $\mathsf {x}$ -stable. As $\mathsf {x}$ is nilpotent, we have $\mathsf {x}v_n \in {\mathbb {C}}\{v_i\mid i \in [n-1]\}$ by Remark 2.2. We conclude $D_0=0$ and $\det (A_g)$ has no linear terms. This concludes the proof of the second statement.

Proof It is shown in [Reference Escobar, Precup and ShareshianEPS21, Example 5.2] that

(7.1) $$ \begin{align} \mathsf{Poin}(X_{s_2w_0};q^{1/2})&=\mathsf{Poin}(X_{s_{n-2}w_0};q^{1/2})\\&=[n-2]_q!\left([n]_q[n-1]_q-q^{2n-3}-q^{2n-4}\right).\nonumber \end{align} $$

Combining (7.1) and Theorem 3, we see that if $\mathsf {x} \in \mathfrak {g}$ has eigenvalues $\lambda _1,\ldots ,\lambda _\ell $ with respective multiplicities $d_1,\ldots ,d_\ell $ and $i \in \{2,n-2\}$ with $\mathsf {Poin}(\mathcal {B}(\mathsf {x},H({\mathbf m}_{\mathbf {{max}}});q))=\mathsf {Poin}(X_{s_iw_0};q),$ then

$$ \begin{align*}[n]_q[n-2]_q+q^{n-2}\sum_{j=1}^\ell [d_j]_q = [n]_q[n-1]_q-q^{2n-3}-q^{2n-4}, \end{align*} $$

from which it follows that

$$ \begin{align*}q^{n-1}+q^{n-2}+\sum_{j=1}^\ell[d_j]_q = [n]_q. \end{align*} $$

We see now that if $\mathsf {Poin}(\mathcal {B}(\mathsf {x},H({\mathbf m}_{\mathbf {{max}}}));q)=\mathsf {Poin}(X_{s_iw_0};q)$ under the given conditions on $\mathsf {x}$ and i, then $\ell =1$ and $d_1=n-2$ .

Proof We begin with (1). Let $w=s_2w_0$ and write

$$ \begin{align*}w=[n,n-1,\ldots,5,4,2,3,1] \end{align*} $$

and

$$ \begin{align*}v(2)=[n,n-1,\ldots,5,2,1,4,3] \end{align*} $$

in one-line notation. We show first that $v(2) \in Z_w$ . If we set $(i,j,k,\ell )=(i^\prime ,j^\prime ,k^\prime ,\ell ^\prime )=(n-3,n-2,n-1,n),$ we see that Lemma 7.2(2) can be applied, with $v^\prime =v=v(2)$ and $w^\prime =w$ .

Now assume $v \in Z_w$ . We observe that $v,w$ must be as in Lemma 7.2(2), with $(j,k,\ell )=(n-2,n-1,n)$ . It follows that

$$ \begin{align*}(v^\prime_{n-2},v^\prime_{n-1},v^\prime_n)=(1,n+1-i,3) \end{align*} $$

for some $i \in [n-3]$ . By the tableau criterion, $v^\prime \leq v$ implies $1 \in \{v_{n-2},v_{n-1},v_n\}$ . As $v_{j^\prime }=w_\ell =1$ , we see that $j^\prime \in \{n-2,n-1,n\}$ . It follows that $(j^\prime ,k^\prime ,\ell ^\prime )=(n-2, n-1,n)$ . Therefore,

$$ \begin{align*}(v_{n-2},v_{n-1},v_n)=(w_\ell,w_i,w_k)=(1,n+1-i,3), \end{align*} $$

for some $i \in [n-3]$ . Applying the tableau criterion again, we get that $v \leq v(2)$ . Indeed, $I_{p,q}(v)\le I_{p,q}(w_0) = I_{p,q}(v(2))$ for $(p,q)\notin \{(1,n-3),(1,n-2),(2,n-2), (1,n-1), (2,n-1)\}$ and the remaining cases follow from our analysis. So, $Z_{w, \mathrm {max}}=\{v(2)\}$ and (1) follows from Lemma 7.2.

The proof of (2) is similar: If we set $w=s_{n-2}w_0,$ we conclude that $v(n-2) \in Z_w$ by applying Lemma 7.2(2) with $(i,j,k,\ell )=(1,2,3,4)$ . If $v \in Z_w$ , then $v,w$ must be as in Lemma 7.2(2) with $(i,j,k)=(1,2,3)$ . Now $(v^\prime _1,v^\prime _2,v^\prime _3)=(n-2,n+1-\ell ,n)$ for some $4 \leq \ell \leq n$ . By the tableau criterion, $n \in \{v_1,v_2,v_3\}$ , hence $(i^\prime ,j^\prime ,k^\prime )=(1,2,3)$ and $(v_1,v_2,v_3)=(n-2,n+1-\ell ,n)$ . It follows that $v \leq v(n-2)$ , and this implies (2).

Proof We obtain the Euler characteristic by counting the number of $u \leq v(i)$ in the Bruhat order. Fix $u=[u_1,\ldots , u_n] \in S_n$ . Using the tableau criterion, we see that $u \not \leq v(2)$ if and only if $\{u_{n-1},u_n\} \cap \{1,2\} \neq \emptyset $ and $u \not \leq v(n-2)$ if and only if $\{u_1,u_2\} \cap \{n-1,n\} \neq \emptyset $ . The corollary follows from a straightforward inclusion–exclusion argument.

Proof We use Theorem 1, assuming without loss of generality that $\mathsf {x}$ is in highest form and permuted Jordan form. So, we fix any prime $p>2$ and count flags $0=V_0\subset V_1\subset \dots \subset V_n={\mathbb F}_p^n$ satisfying

1. (a) $V_1 \subseteq \ker (\bar {\mathsf {x}}),$ and

2. (b) $\operatorname {\mathrm {Im}}(\bar {\mathsf {x}}) \subseteq V_{n-1}$ .

Note that Remark 2.2 allows us to replace $\bar {\mathsf {x}}V_{n-1} \subseteq V_{n-1}$ , implied by our choice of ${\mathbf m}$ , with (b) as stated.

We observe that $\bar {\mathsf {x}}^2=0$ if and only if $\mathsf {x}^2=0$ and consider first the case $\mathsf {x}^2=0$ . We fix one-dimensional $Z\subset {\mathbb F}_p^n$ and count flags satisfying (a) and (b) in which $V_1=Z$ . If $Z \subseteq \operatorname {\mathrm {Im}}(\bar {\mathsf {x}}),$ then we can choose $V_{n-1}$ to be any of the $[n-2]_p$ hyperplanes in ${\mathbb F}_p^n$ containing the two-dimensional subspace $\operatorname {\mathrm {Im}}(\bar {\mathsf {x}})$ and then choose $V_2,\ldots ,V_{n-2}$ in any of $[n-2]_p!$ ways. Since $\dim \operatorname {\mathrm {Im}}(\bar {\mathsf {x}})=2,$ there are $[2]_p$ possibilities for Z and it follows that there are

$$ \begin{align*}[2]_p[n-2]_p[n-2]_p! \end{align*} $$

flags satisfying (a) and (b) in which $V_1 \subset \operatorname {\mathrm {Im}}(\bar {\mathsf {x}})$ .

Now consider the case of one-dimensional $Z\subset {\mathbb F}_p^n$ such that $Z \subseteq \ker (\bar {\mathsf {x}})$ and ${Z \not \subseteq \operatorname {\mathrm {Im}}(\bar {\mathsf {x}})}$ . Since $\bar {\mathsf {x}}^2=0$ , we know that $\operatorname {\mathrm {Im}}(\bar {\mathsf {x}}) \subseteq \ker (\bar {\mathsf {x}})$ . Therefore, there are $[n-2]_p-[2]_p$ possibilities for Z. Having fixed one such Z, we can choose $V_{n-1}$ containing $Z+\operatorname {\mathrm {Im}}(\bar {\mathsf {x}})$ in $[n-3]_p$ ways, and then choose $V_2,\ldots ,V_{n-2}$ in $[n-2]_p!$ ways. We see there are

$$ \begin{align*}\left([n-2]_p-[2]_p\right)[n-3]_p[n-2]_p! \end{align*} $$

flags satisfying (a) and (b) in which $V_1 \not \subseteq \operatorname {\mathrm {Im}}(\bar {\mathsf {x}})$ . Direct computation, along with Theorem 1, gives

$$ \begin{align*}\mathsf{Poin}(\mathcal{B}(\mathsf{x},{\mathbf m});q)=[n-2]_{q^2}!\left([n-2]_{q^2}[n-3]_{q^2}+(1+q^2)q^{2n-6}\right), \end{align*} $$

and (1) follows.

The case $\mathsf {x}^2 \neq 0$ is handled similarly. We observe first that in this case

$$ \begin{align*}\dim(\ker(\bar{\mathsf{x}}) \cap \operatorname{\mathrm{Im}}(\bar{\mathsf{x}}))=1. \end{align*} $$

Arguing as we did above, consider first the case in which $V_1 = \ker (\bar {\mathsf {x}}) \cap \operatorname {\mathrm {Im}}(\bar {\mathsf {x}})$ . Then we calculate that there are

$$ \begin{align*}[n-2]_p[n-2]_p! \end{align*} $$

flags satisfying (a), (b), and $V_1 = \ker (\bar {\mathsf {x}}) \cap \operatorname {\mathrm {Im}}(\bar {\mathsf {x}})$ . On the other hand, if $V_1\not \subseteq \operatorname {\mathrm {Im}}(\bar {\mathsf {x}}),$ we get

$$ \begin{align*}p[n-3]_p[n-3]_p[n-2]_p! \end{align*} $$

flags satisfying (a) and (b). As above, Theorem 1 now yields

$$ \begin{align*}\mathsf{Poin}(\mathcal{B}(\mathsf{x},{\mathbf m});q)=[n-2]_{q^2}!\left([n-2]_{q^2}+q^2[n-3]_{q^2}^2\right) \end{align*} $$

and (2) follows.

Proof The backward direction of (1) is obvious. To prove the forward direction and (2), we proceed by induction on n.

Suppose $n=3$ . Now

(8.2) $$ \begin{align} \det(L) = \det \begin{bmatrix} \ell_1 & 1 & 0 \\ \ell_2 & z_{21} & 1\\ \ell_3 & z_{31} & z_{32} \end{bmatrix} = (z_{32}z_{21}-z_{31})\ell_1 -z_{32}\ell_2+\ell_3. \end{align} $$

Write $\ell _i = a_{i1}+\sum _{j=2}^m a_{ij}z_{j1}$ for $1\leq i \leq 3$ . Substituting these expressions into (8.2) and collecting like terms, we obtain the following formula for $\det (L)$ :

(8.3) $$ \begin{align} & a_{12}z_{32}z_{21}^2+a_{13}z_{32}z_{31}z_{21}+ (a_{11}-a_{22})z_{32}z_{21} -a_{12}z_{31}z_{21}-a_{13}z_{31}^2 \\ \nonumber& \quad\quad\quad\quad -a_{23}z_{32}z_{31} -a_{21}z_{32} +a_{32}z_{21} + (a_{33}-a_{11})z_{31}+a_{31} \\ \nonumber& \quad\quad\quad\quad + \left( \sum_{j=4}^m a_{1j}z_{32}z_{21}z_{j1} - a_{1j}z_{31}z_{j1} -a_{2j}z_{32}z_{j1} +a_{3j}z_{j1}\right). \end{align} $$

If (8.3) is the zero polynomial, then $a_{ij} = 0$ whenever $i\neq j$ and $a_{11}=a_{22}=a_{33}$ . The forward direction of (1) now follows. Notice that (8.3) has only two monomial terms containing squares: $z_{32}z_{21}^2$ with coefficient $a_{12}$ and $z_{31}^2$ with coefficient $a_{13}$ . If $a_{13}=0$ and $a_{12}=0,$ then (2) is immediate, as (8.3) is then a sum of square-free monomials. By inspection, if $a_{13}\neq 0,$ then the lexicographic term order $\preceq $ associated with a total ordering of ${\mathbf z}$ in which $z_{32}$ is the maximal element and $z_{31}$ is the maximal element of ${\mathbf z}\setminus \{z_{32}\}$ yields ${\text i}{\text n}_\preceq (\det (L)) = z_{32}z_{31}z_{21}$ . The final case is $a_{13}= 0$ and $a_{12}\neq 0$ . Taking the lexicographic term order $\preceq $ associated with a total ordering of ${\mathbf z}$ in which $z_{21}$ is the maximal element and $z_{31}$ is the maximal element of ${\mathbf z}\setminus \{z_{21}\}$ gives ${\text i}{\text n}_\preceq (\det (L)) = z_{31}z_{21}$ . This completes the proof of the base case.

Assume now that $n>3$ . For $i,j \in [n],$ write $L(i,j)$ for the matrix obtained from L by deleting the $i^{th}$ row and the $j^{th}$ column. Expanding along the last column, we see that

(8.4) $$ \begin{align} \det(L) = \pm z_{n,n-1}\det(L(n,n)) \mp \det(L(n-1,n)). \end{align} $$

Let ${\mathbf z}' = \{ z_{ji} \mid 1\leq i<j \leq n-1 \} \sqcup \{ z_{j1} \mid n\leq j \leq m \}$ and note that $L(n,n) \in M_{n-1,n-1} ({\mathbb {C}}[{\mathbf z}'])$ .

Suppose first that $\det (L(n,n))=0$ . By the induction hypothesis, there exists $c\in {\mathbb {C}}$ such that $\ell _1 = c $ and $\ell _i = cz_{i1}$ for $2\leq i \leq n-1$ . We observe that, in this case, $\det (L) = \pm \det (L(n-1,n))= \pm (\ell _n - cz_{n1})$ . If $\det (L)=0,$ then $\ell _n = cz_{n1}$ , proving (1). Claim (2) follows immediately from the fact that $\ell _n$ is linear.

Next suppose $\det (L(n,n))\neq 0$ . Since $z_{n,n-1}$ does not appear in any monomial term of $\det (L(n-1,n))$ , we have $\det (L)\neq 0$ and (1) is vacuously true. By the induction hypothesis, there exists a lexicographic term order $\preceq '$ on $\mathcal {M}({\mathbf z}')$ such that ${\text i}{\text n}_{\preceq '}(\det (L(n,n)))$ is square-free. Let $\preceq $ be the lexicographic term order associated with any total order which respects the total order on ${\mathbf z}'$ corresponding to $\preceq '$ and with maximal element $z_{n,n-1}$ . Equation (8.4) now implies ${\text i}{\text n}_\preceq (\det (L)) = z_{n,n-1}{\text i}{\text n}_{\preceq '}(\det (L(n,n)))$ . The proof is now complete.

Proof of Proposition 8.1

Set $\mathsf {y}:=g^{-1}\mathsf {x}g$ and

$$ \begin{align*}A^{\prime}_g:=[\mathsf{y}u_1|u_1|\dots |u_{n-1}]. \end{align*} $$

Comparing $A_g'$ to the definition of $A_g$ in (5.2), we observe that $A_g=gA^{\prime }_g$ , hence $\det (A^{\prime }_g)$ and $\det (A_g)$ generate the same ideal in ${\mathbb {C}}[{\mathbf z}]$ . Now $A^{\prime }_g$ is as in Lemma 8.3, where

$$ \begin{align*} \ell_i = \mathsf{y}_{i1} + \sum_{j=2}^n \mathsf{y}_{ij} z_{j1} \; \text{ for } \; i=1, 2, \ldots, n. \end{align*} $$

Proposition 8.1 now follows from Lemma 8.2 and 8.3(2).