2.1.1 The Shimura variety and its integral model at a good prime

Let $E$ be a quadratic imaginary field, $0\leqslant m\leqslant n$ and $\unicode[STIX]{x1D6EC}={\mathcal{O}}_{E}^{n+m}$ , equipped with the hermitian pairing

$$\begin{eqnarray}(u,v)=\text{}^{t}\overline{u}\left(\begin{array}{@{}ccc@{}} & & 1_{m}\\ & 1_{n-m}\\ 1_{m}\end{array}\right)v.\end{eqnarray}$$

Here $1_{k}$ is the identity matrix of size $k$ . Let $\unicode[STIX]{x1D6FF}$ be a square root of the discriminant of $E$ , so that $\overline{\unicode[STIX]{x1D6FF}}=-\unicode[STIX]{x1D6FF}$ , and denote by $\operatorname{Im}_{\unicode[STIX]{x1D6FF}}(z)=(z-\overline{z})/\unicode[STIX]{x1D6FF}$ . Then

$$\begin{eqnarray}\langle u,v\rangle =\operatorname{Im}_{\unicode[STIX]{x1D6FF}}(u,v)\end{eqnarray}$$

is $\mathbb{Q}$ -bilinear, skew-symmetric, satisfies $\langle au,v\rangle =\langle u,\overline{a}v\rangle$ , and $\unicode[STIX]{x1D6EC}$ is self-dual, i.e. $\langle .\,,.\rangle$ induces $\unicode[STIX]{x1D6EC}\simeq \operatorname{Hom}(\unicode[STIX]{x1D6EC},\mathbb{Z})$ . Let $\boldsymbol{G}$ be the general unitary group of $(\unicode[STIX]{x1D6EC},(.\,,.))$ , viewed as a group scheme over  $\mathbb{Z}$ . For every commutative ring $R$

$$\begin{eqnarray}\boldsymbol{G}(R)=\{g\in \text{GL}_{n+m}({\mathcal{O}}_{E}\otimes R)\mid \exists \unicode[STIX]{x1D707}(g)\in R^{\times },(gu,gv)=\unicode[STIX]{x1D707}(g)(u,v)\}.\end{eqnarray}$$

Fix an odd prime $p$ which is unramified in $E$ , and an integer $N\geqslant 3$ relatively prime to $p$ . Let $\mathbb{A}=\mathbb{R}\times \mathbb{A}_{f}$ be the adèle ring of $\mathbb{Q}$ , where $\mathbb{A}_{f}=\mathbb{Q}\cdot \widehat{\mathbb{Z}}$ are the finite adèles. Let $K_{f}\subset \boldsymbol{G}(\widehat{\mathbb{Z}})$ be an open subgroup of the form $K_{f}=K^{p}K_{p}$ , where $K^{p}\subset \boldsymbol{G}(\mathbb{A}^{p})$ is the principal congruence subgroup of level $N$ , and

$$\begin{eqnarray}K_{p}=\boldsymbol{G}(\mathbb{Z}_{p})\subset \boldsymbol{G}(\mathbb{Q}_{p})\end{eqnarray}$$

the hyperspecial maximal compact subgroup at $p$ . Let $K_{\infty }\subset \boldsymbol{G}(\mathbb{R})$ be the stabilizer of the negative definite subspace spanned by $\{-e_{i}+e_{n+i};\,1\leqslant i\leqslant m\}$ in $\unicode[STIX]{x1D6EC}_{\mathbb{R}}=\mathbb{C}^{n+m}$ , where $\{e_{i}\}$ stands for the standard basis. This $K_{\infty }$ is a maximal compact-modulo-center subgroup, isomorphic to $G(U(m)\times U(n))$ . By $G(U(m)\times U(n))$ we mean the pairs of matrices $(g_{1},g_{2})\in GU(m)\times GU(n)$ having the same similitude factor. Let $K=K_{\infty }K_{f}\subset \boldsymbol{G}(\mathbb{A})$ and $X=\boldsymbol{G}(\mathbb{R})/K_{\infty }$ .

To the Shimura datum $(\boldsymbol{G},X)$ there is associated a Shimura variety $\mathit{Sh}_{K}$ . It is a quasi-projective smooth variety of dimension $nm$ over $E$ . If $m=n$ our Shimura variety is even defined over $\mathbb{Q}$ , but we still denote by $\mathit{Sh}_{K}$ its base-change to $E$ . The complex points of $\mathit{Sh}_{K}$ are identified, as a complex manifold, with

$$\begin{eqnarray}\mathit{Sh}_{K}(\mathbb{C})=\boldsymbol{G}(\mathbb{Q})\backslash \boldsymbol{G}(\mathbb{A})/K.\end{eqnarray}$$

Fix an embedding $\overline{\mathbb{Q}}\subset \overline{\mathbb{Q}}_{p}$ and let $v|p$ be the prime of $E$ induced by it. Following Kottwitz [Reference KottwitzKot92], but using a somewhat more restrictive set-up suitable for principally polarized abelian varieties, we define a scheme ${\mathcal{S}}$ over the localization ${\mathcal{O}}_{E,v}$ of ${\mathcal{O}}_{E}$ at  $v$ . This ${\mathcal{S}}$ is a fine moduli space whose $R$ -points, for every ${\mathcal{O}}_{E,v}$ -algebra $R$ , classify isomorphism types of tuples $\text{}\underline{A}=(A,\unicode[STIX]{x1D704},\unicode[STIX]{x1D719},\unicode[STIX]{x1D702})$ where

• ∙ $A$ is an abelian scheme of dimension $n+m$ over $R$ ,

• ∙ $\unicode[STIX]{x1D704}:{\mathcal{O}}_{E}{\hookrightarrow}\operatorname{End}(A)$ has signature $(n,m)$ on the Lie algebra of $A$ ,

• ∙ $\unicode[STIX]{x1D719}:A\overset{{\sim}}{\rightarrow }A^{t}$ is a principal polarization whose Rosati involution induces $\unicode[STIX]{x1D704}(a)\mapsto \unicode[STIX]{x1D704}(\overline{a})$ on the image of $\unicode[STIX]{x1D704}$ ,

• ∙ $\unicode[STIX]{x1D702}$ is an ${\mathcal{O}}_{E}$ -linear full level- $N$ structure on $A$ compatible with $(\unicode[STIX]{x1D6EC},\langle .\,,.\rangle )$ and $\unicode[STIX]{x1D719}$  [Reference LanLan13, §1.3.6].

We shall summarize the above requirements by saying that $\text{}\underline{A}$ is a structure of type  $\mathscr{D}$ over $R$ . See also [Reference LanLan13] for the comparison of the various languages used to define the moduli problem.

The generic fiber $S_{K}$ of ${\mathcal{S}}$ is, in general, a union of several Shimura varieties of the type $\mathit{Sh}_{K}$ . This is due to the failure of the Hasse principle, which can happen when $m+n$ is odd. We also remark that the assumption $N\geqslant 3$ could be avoided if we were willing to use the language of stacks. As this is not essential to the present paper, we keep the scope slightly limited for the sake of clarity.

As shown by Kottwitz, ${\mathcal{S}}$ is smooth of relative dimension $nm$ over ${\mathcal{O}}_{E,v}$ . It even admits smooth (toroidal) compactifications at the cusps, cf. [Reference LanLan13].

2.1.2 The universal abelian variety and the Kodaira–Spencer isomorphism

The moduli space ${\mathcal{S}}$ carries a universal abelian scheme ${\mathcal{A}}$ of dimension $n+m$ , equipped with $\unicode[STIX]{x1D704},\unicode[STIX]{x1D719},\unicode[STIX]{x1D702}$ as above. Let $\unicode[STIX]{x1D70B}:{\mathcal{A}}\rightarrow {\mathcal{S}}$ be the structure morphism. We denote by ${\mathcal{A}}^{t}$ the dual abelian scheme.

We let $\unicode[STIX]{x1D6F4}$ denote the identity embedding of ${\mathcal{O}}_{E}$ in ${\mathcal{O}}_{E,v}$ and $\overline{\unicode[STIX]{x1D6F4}}$ its complex conjugate. Since $p$ is unramified in $E$ , the locally free sheaves $H_{\text{dR}}^{1}({\mathcal{A}}/{\mathcal{S}}),\unicode[STIX]{x1D714}_{{\mathcal{A}}/{\mathcal{S}}}=R^{0}\unicode[STIX]{x1D70B}_{\ast }\unicode[STIX]{x1D6FA}_{{\mathcal{A}}/{\mathcal{S}}}^{1}$ and $\unicode[STIX]{x1D714}_{{\mathcal{A}}^{t}/{\mathcal{S}}}^{\vee }=R^{1}\unicode[STIX]{x1D70B}_{\ast }{\mathcal{O}}_{{\mathcal{A}}}$ decompose as direct sums of their $\unicode[STIX]{x1D6F4}$ and $\overline{\unicode[STIX]{x1D6F4}}$ -parts under the action of $\unicode[STIX]{x1D704}({\mathcal{O}}_{E})$ . We write

$$\begin{eqnarray}{\mathcal{P}}=\unicode[STIX]{x1D714}_{{\mathcal{A}}/{\mathcal{S}}}(\unicode[STIX]{x1D6F4}),\quad {\mathcal{Q}}=\unicode[STIX]{x1D714}_{{\mathcal{A}}/{\mathcal{S}}}(\overline{\unicode[STIX]{x1D6F4}}).\end{eqnarray}$$

These are locally free sheaves of ranks $n$ and $m$ respectively on ${\mathcal{S}}$ .

The Kodaira–Spencer map is the sheaf homomorphism,

$$\begin{eqnarray}\text{KS}:{\mathcal{P}}=\unicode[STIX]{x1D714}_{{\mathcal{A}}/{\mathcal{S}}}(\unicode[STIX]{x1D6F4})\rightarrow \unicode[STIX]{x1D6FA}_{{\mathcal{S}}}^{1}\otimes \unicode[STIX]{x1D714}_{{\mathcal{A}}^{t}/{\mathcal{S}}}^{\vee }(\unicode[STIX]{x1D6F4}),\end{eqnarray}$$

obtained by embedding $\unicode[STIX]{x1D714}_{{\mathcal{A}}/{\mathcal{S}}}$ in $H_{\text{dR}}^{1}({\mathcal{A}}/{\mathcal{S}})$ , applying the Gauss–Manin connection

$$\begin{eqnarray}\unicode[STIX]{x1D6FB}:H_{\text{dR}}^{1}({\mathcal{A}}/{\mathcal{S}})\rightarrow \unicode[STIX]{x1D6FA}_{{\mathcal{S}}}^{1}\otimes H_{\text{dR}}^{1}({\mathcal{A}}/{\mathcal{S}}),\end{eqnarray}$$

and finally projecting $H_{\text{dR}}^{1}({\mathcal{A}}/{\mathcal{S}})$ to $\unicode[STIX]{x1D714}_{{\mathcal{A}}^{t}/{\mathcal{S}}}^{\vee }$ . Since the polarization $\unicode[STIX]{x1D719}$ induces an identification

$$\begin{eqnarray}\unicode[STIX]{x1D719}^{\ast \vee }:\unicode[STIX]{x1D714}_{{\mathcal{A}}^{t}/{\mathcal{S}}}^{\vee }(\unicode[STIX]{x1D6F4})\overset{{\sim}}{\leftarrow }\unicode[STIX]{x1D714}_{{\mathcal{A}}/{\mathcal{S}}}^{\vee }(\overline{\unicode[STIX]{x1D6F4}})={\mathcal{Q}}^{\vee }\end{eqnarray}$$

the Kodaira–Spencer map yields a homomorphism, which we denote by the same symbol

$$\begin{eqnarray}\text{KS}:{\mathcal{P}}\otimes {\mathcal{Q}}\rightarrow \unicode[STIX]{x1D6FA}_{{\mathcal{S}}}^{1}.\end{eqnarray}$$

This map turns out to be an isomorphism. See [Reference LanLan13, Proposition 2.3.5.2].

2.1.3 The NP and EO stratifications of the special fiber of ${\mathcal{S}}$

We briefly review some facts about these two stratifications, as the EO stratification is going to play a central role later in the paper.

Let $S$ be the special fiber of ${\mathcal{S}}$ . It is a smooth variety over $\unicode[STIX]{x1D705}=\unicode[STIX]{x1D705}_{v}$ , the residue field of $v$ . Let $k$ be an algebraically closed field containing $\unicode[STIX]{x1D705}$ , and $x\in S(k)$ . Let $NP_{x}$ be the Newton polygon of the $p$ -divisible group ${\mathcal{A}}_{x}[p^{\infty }]$ . It is lower convex, starts at $(0,0)$ , ends at $(2(n+m),n+m)$ , and has integral break-points. Then $NP_{x}$ classifies the $k$ -isogeny class of ${\mathcal{A}}_{x}[p^{\infty }]$ . The set of Newton polygons is partially ordered, where $P^{\prime }\geqslant P$ if $P^{\prime }$ lies on or above $P$ . For every Newton polygon $P$ there is a locally closed stratum $S_{P}$ in $S$ , defined over $\unicode[STIX]{x1D705}$ , whose geometric points are precisely those with $NP_{x}=P$ . The closure of a non-empty $S_{P}$ is the union of the $S_{P^{\prime }}$ for all $P^{\prime }$ satisfying $P^{\prime }\geqslant P$  [Reference Viehmann and WedhornVW13, §11]. This gives the Newton polygon (NP) stratification of $S$ .

The Ekedahl–Oort (EO) stratification of $S$ is another stratification, by the isomorphism type of ${\mathcal{A}}_{x}[p]$ . In addition to the references already cited in the introduction, see also [Reference WoodingWoo16] for a thorough discussion of the case at hand. The EO strata $S_{w}$ are locally closed subsets labeled by certain elements $w$ in the Weyl group $W$ of $\boldsymbol{G}$ . More precisely, the $w$ are distinguished representatives for the cosets $W_{J}\backslash W$ , where the subgroup $W_{J}$ is determined by the signature condition. The $S_{w}$ are equidimensional, smooth and quasi-affine. The dimension of $S_{w}$ is $l(w)$ , the length of $w$ relative to the Bruhat order on $W$ . The closure of $S_{w}$ is the union of $S_{w^{\prime }}$ for $w^{\prime }\preccurlyeq w$ under a certain rather complicated order (related to, but different from the usual Bruhat order; see [Reference Viehmann and WedhornVW13, Theorems 2 and 3]). We call it the EO order on the Weyl group elements indexing the strata. See below for a full description when $p$ is inert in $E$ and $m<n$ .

Wedhorn and Moonen have proved the following [Reference Wedhorn, Faber, van der Geer and OortWed01, Reference MoonenMoo04].

If $p$ is split in $E$ or $n=m$ then ${\mathcal{A}}_{x}[p^{\infty }]\simeq (\mathbb{Q}_{p}/\mathbb{Z}_{p})^{n+m}\times \unicode[STIX]{x1D707}_{p^{\infty }}^{n+m}$ for all $x\in S^{\text{ord}}(k)$ , so ${\mathcal{A}}_{x}$ is ordinary. If $p$ is inert in $E$ and $m<n$ this is not the case, and for $x\in S^{\text{ord}}$

(2.1) $$\begin{eqnarray}{\mathcal{A}}_{x}[p^{\infty }]\simeq ({\mathcal{O}}_{E}\otimes \mathbb{Q}_{p}/\mathbb{Z}_{p})^{m}\times \mathscr{G}_{\unicode[STIX]{x1D6F4}}^{n-m}\times ({\mathcal{O}}_{E}\otimes \unicode[STIX]{x1D707}_{p^{\infty }})^{m},\end{eqnarray}$$

where $\mathscr{G}_{\unicode[STIX]{x1D6F4}}$ is the unique 1-dimensional, height 2, slope 1/2, self-dual $p$ -divisible group over $k$ . The subscript $\unicode[STIX]{x1D6F4}$ means that the embedding of ${\mathcal{O}}_{E}$ in $\operatorname{End}_{k}(\mathscr{G})$ via $\unicode[STIX]{x1D704}$ induces on $\operatorname{Lie}(\mathscr{G})$ the type $\unicode[STIX]{x1D6F4}$ , rather than $\overline{\unicode[STIX]{x1D6F4}}$ . In this case it is customary to call ${\mathcal{A}}_{x}$ , for $x$ in the ordinary locus, $\unicode[STIX]{x1D707}$ -ordinary.

Under this assumption $v=(p)$ , so we write ${\mathcal{O}}_{E,(p)}$ instead of ${\mathcal{O}}_{E,v}$ , and $\unicode[STIX]{x1D705}=\mathbb{F}_{p^{2}}$ . The lowest EO strata $S_{\text{id}}$ (labeled by $w=\operatorname{id}$ ) is $0$ -dimensional and when $p$ is inert it classifies superspecial abelian varieties, i.e. those for which ${\mathcal{A}}_{x}[p^{\infty }]\simeq \mathscr{G}_{\unicode[STIX]{x1D6F4}}^{n}\times \mathscr{G}_{\overline{\unicode[STIX]{x1D6F4}}}^{m}$ . We call this stratum the core points.

If $m=1$ the EO stratification has been worked out completely by Bültel and Wedhorn [Reference Bültel and WedhornBW06]. The strata are linearly ordered, their dimensions dropping by 1 each time. Thus there are $n+1$ EO strata altogether. As long as the dimension of the stratum is strictly larger than $\lfloor n/2\rfloor$ , the EO strata are also NP strata. In fact, the isomorphism type of the whole $p$ -divisible group (with its endomorphisms and polarization) is constant along these strata, as it was on $S^{\text{ord}}$ . Half the way through, in dimension $\lfloor n/2\rfloor$ , one reaches the supersingular NP stratum, which is stratified further by EO strata. We remark also that from dimension $\lfloor n/2\rfloor$ down, the isomorphism type of ${\mathcal{A}}_{x}[p^{\infty }]$ is no longer constant along the EO strata, only that of  ${\mathcal{A}}_{x}[p]$ .

In general, we may identify the Weyl group of $\boldsymbol{G}$ with $\mathfrak{S}_{n+m}$ , the group of permutations of $\{1,\ldots ,n+m\}$ . Let $W_{J}=\mathfrak{S}_{n}\times \mathfrak{S}_{m}$ . The elements $w$ indexing the EO strata belong then to the set $\unicode[STIX]{x1D6F1}(n,m)$ of $(n,m)$ -shuffles in $\mathfrak{S}_{n+m}$ . A permutation $w$ is called an $(n,m)$ -shuffle if

$$\begin{eqnarray}w^{-1}(1)<\cdots <w^{-1}(n),\quad w^{-1}(n+1)<\cdots <w^{-1}(n+m).\end{eqnarray}$$

The set $\unicode[STIX]{x1D6F1}(n,m)$ is clearly a set of representatives for $W_{J}\backslash W$ .

The dimension of $S_{w}$ is given by the formula

(2.2) $$\begin{eqnarray}l(w)=\mathop{\sum }_{i=1}^{n}(w^{-1}(i)-i).\end{eqnarray}$$

We shall also need to know a formula for

$$\begin{eqnarray}a_{\unicode[STIX]{x1D6F4}}(w)=\dim {\mathcal{P}}_{x}[V]\end{eqnarray}$$

$(x\in S_{w}(k))$ . Here $V$ is the map induced on cohomology by Verschiebung, see below. This number is the $\unicode[STIX]{x1D6F4}$ -part of Oort’s $a$ -number of ${\mathcal{A}}_{x}$ . It turns out that it is given by

(2.3) $$\begin{eqnarray}a_{\unicode[STIX]{x1D6F4}}(w)=|\{i\mid 1\leqslant i\leqslant n,1\leqslant w^{-1}(i)\leqslant n\}|.\end{eqnarray}$$

For the formulae (2.2), (2.3) see [Reference WoodingWoo16, §§3.4 and 3.5]. For example, if $m=1$ , $a_{\unicode[STIX]{x1D6F4}}(w)=n-1$ except if $w=1$ (corresponding to the core points), where it becomes $n$ . In general, for $w$ the longest $(n,m)$ -shuffle (of length $nm$ ), where $S_{w}=S^{\text{ord}}$ , $a_{\unicode[STIX]{x1D6F4}}(w)=n\,-\,m$ . For $w=\operatorname{id}$ , corresponding to the core points, $a_{\unicode[STIX]{x1D6F4}}(\operatorname{id})=n$ .

Finally, we make explicit the EO order relation $w^{\prime }\preccurlyeq w$ on the set $\unicode[STIX]{x1D6F1}(n,m)$ , following [Reference WoodingWoo16, Example 3.1.3]. Recall that $w^{\prime }\preccurlyeq w$ if and only if $S_{w^{\prime }}\subset \overline{S_{w}}$ . Let

$$\begin{eqnarray}w_{0,J}=\left(\begin{array}{@{}cccccc@{}}1 & \cdots \, & n & n+1 & \cdots \, & n+m\\ n & \cdots \, & 1 & n+m & \cdots \, & n+1\end{array}\right).\end{eqnarray}$$

Let ${\leqslant}$ be the usual Bruhat order on $\mathfrak{S}_{n+m}$ with respect to the standard set of reflections $s_{i}=(i,i+1)$ ( $1\leqslant i<n+m$ ). Note that $w_{0,J}$ is the element of maximal length in $W_{J}$ . Then $w^{\prime }\preccurlyeq w$ if and only if there exists a $y\in W_{J}$ such that

$$\begin{eqnarray}yw^{\prime }w_{0,J}y^{-1}w_{0,J}\leqslant w.\end{eqnarray}$$

Taking $y=1$ we see that if $w^{\prime }\leqslant w$ then also $w^{\prime }\preccurlyeq w$ . This is the only property of the EO order relation that will be used in the paper.

2.1.4 Frobenius, Verschiebung and the Hasse invariant

For any scheme $X$ in characteristic $p$ we denote by $\unicode[STIX]{x1D6F7}_{X}$ the absolute Frobenius morphism of degree $p$ of $X$ . Let

$$\begin{eqnarray}{\mathcal{A}}^{(p)}=S\times _{\unicode[STIX]{x1D6F7}_{S},S}{\mathcal{A}}\end{eqnarray}$$

be the base-change of the universal abelian variety. Let

$$\begin{eqnarray}\text{Fr}=\mathit{Fr}_{{\mathcal{A}}/S}:{\mathcal{A}}\rightarrow {\mathcal{A}}^{(p)}\end{eqnarray}$$

be the relative Frobenius morphism. It is an isogeny of abelian schemes over $S$ , of degree $p^{n+m}$ . The isogeny dual to $\mathit{Fr}_{{\mathcal{A}}^{t}/S}$ is called the Verschiebung of ${\mathcal{A}}$ and is denoted

$$\begin{eqnarray}\text{Ver}=\mathit{Ver}_{{\mathcal{A}}/S}:{\mathcal{A}}^{(p)}\rightarrow {\mathcal{A}}.\end{eqnarray}$$

It too is of degree $p^{n+m}$ and $\mathit{Ver}_{{\mathcal{A}}/S}\circ \mathit{Fr}_{{\mathcal{A}}/S}$ is multiplication by $p$ on ${\mathcal{A}}$ .

The maps induced by $\mathit{Fr}_{{\mathcal{A}}/S}$ and $\mathit{Ver}_{{\mathcal{A}}/S}$ on cohomology will be denoted $F$ and  $V$ . It is well-known that

$$\begin{eqnarray}F:H_{\text{dR}}^{1}({\mathcal{A}}^{(p)}/S)\rightarrow H_{\text{dR}}^{1}({\mathcal{A}}/S)\end{eqnarray}$$

is a homomorphism of vector bundles of constant rank $\text{rk}(\text{Im}(F))=n+m$ . Its image is a sub-bundle (i.e. a locally free sub-sheaf, the quotient of $H_{\text{dR}}^{1}({\mathcal{A}}/S)$ by which is also locally free) and coincides with $H_{\text{dR}}^{1}({\mathcal{A}}/S)[V]$ . Similarly, the image of

$$\begin{eqnarray}V:H_{\text{dR}}^{1}({\mathcal{A}}/S)\rightarrow H_{\text{dR}}^{1}({\mathcal{A}}^{(p)}/S)\end{eqnarray}$$

is a sub-bundle, equal to $H_{\text{dR}}^{1}({\mathcal{A}}^{(p)}/S)[F]$ .

The same cannot be said about the restriction of $V$ to $\unicode[STIX]{x1D714}_{{\mathcal{A}}/S}$ . While $\unicode[STIX]{x1D714}_{{\mathcal{A}}/S}[V]$ is clearly a saturated sub-sheaf of $\unicode[STIX]{x1D714}_{{\mathcal{A}}/S}$ , its rank may increase when we move from one EO stratum to a smaller one, contained in its closure. Hence, $\unicode[STIX]{x1D714}_{{\mathcal{A}}/S}[V]$ is in general not a sub-bundle. It is, however, a sub-bundle of rank $n-m$ , if we restrict it to the ordinary locus.

Since $F$ and $V$ commute with the endomorphisms, they induce maps between the $\unicode[STIX]{x1D6F4}$ and the $\overline{\unicode[STIX]{x1D6F4}}$ -parts. Note however that

$$\begin{eqnarray}H_{\text{dR}}^{1}({\mathcal{A}}^{(p)}/S)(\unicode[STIX]{x1D6F4})=H_{\text{dR}}^{1}({\mathcal{A}}/S)^{(p)}(\unicode[STIX]{x1D6F4})=H_{\text{dR}}^{1}({\mathcal{A}}/S)(\overline{\unicode[STIX]{x1D6F4}})^{(p)}.\end{eqnarray}$$

In particular we get maps

$$\begin{eqnarray}V_{{\mathcal{P}}}:{\mathcal{P}}\rightarrow {\mathcal{Q}}^{(p)},\quad V_{{\mathcal{Q}}}:{\mathcal{Q}}\rightarrow {\mathcal{P}}^{(p)}.\end{eqnarray}$$

The sheaf homomorphism

$$\begin{eqnarray}H_{{\mathcal{A}}/S}=V_{{\mathcal{P}}}^{(p)}\circ V_{{\mathcal{Q}}}:{\mathcal{Q}}\rightarrow {\mathcal{Q}}^{(p^{2})}\end{eqnarray}$$

is called the Hasse map. Let ${\mathcal{L}}=\det ({\mathcal{Q}})$ , a line bundle on $S$ . Note that for every line bundle ${\mathcal{L}}$ there is a canonical isomorphism ${\mathcal{L}}^{(p)}\simeq {\mathcal{L}}^{p}$ , sending $1\otimes s$ to $s\otimes \cdots \otimes s$ (here $s$ is a section of  ${\mathcal{L}}$ ). Thus $h_{{\mathcal{A}}/S}=\det (H_{{\mathcal{A}}/S})$ is a homomorphism from ${\mathcal{L}}$ to ${\mathcal{L}}^{p^{2}}$ , which is the same as a global section

$$\begin{eqnarray}h_{{\mathcal{A}}/S}\in H^{0}(S,{\mathcal{L}}^{p^{2}-1}),\end{eqnarray}$$

i.e. a modular form ‘of weight ${\mathcal{L}}^{p^{2}-1}$ ’ called the Hasse invariant. This construction of the Hasse invariant, which plays an important role in the study of $p$ -adic modular forms, is due to Goldring and Nicole. See [Reference Goldring and NicoleGN17, Appendix B], and the main body of their paper for further generalizations. The relation with the stratifications of $S$ is the following.

Fact.

[Reference WoodingWoo16] Let $S^{\text{no}}$ be the complement of $S^{\text{ord}}$ in $S$ , endowed with its reduced subscheme structure. Then $S^{\text{no}}$ is a Cartier divisor and

$$\begin{eqnarray}S^{\text{no}}=\operatorname{div}(h_{{\mathcal{A}}/S}).\end{eqnarray}$$

2.1.5 Pairings in de Rham cohomology of abelian varieties

We review some general facts on de Rham cohomology of abelian varieties. If $A/k$ is an abelian variety over a field $k$ (or more generally, an abelian scheme over a ring) we let $A^{t}$ denote the dual abelian variety. There is then a canonical perfect bilinear pairing

$$\begin{eqnarray}\{.\,,\,.\}=\{.\,,\,.\}_{A}:H_{\text{dR}}^{1}(A/k)\times H_{\text{dR}}^{1}(A^{t}/k)\rightarrow k.\end{eqnarray}$$

When we use the canonical identification of $A$ with $(A^{t})^{t}$ we have

$$\begin{eqnarray}\{u,v\}_{A}=-\{v,u\}_{A^{t}}.\end{eqnarray}$$

Indeed, one may identify $H_{\text{dR}}^{1}(A^{t}/k)$ with $H_{\text{dR}}^{2g-1}(A/k)$ ( $g=\dim A$ ), and then the pairing is given by cup product, followed by the trace. If $\unicode[STIX]{x1D6FC}:A\rightarrow B$ is an isogeny, we let $\unicode[STIX]{x1D6FC}^{t}:B^{t}\rightarrow A^{t}$ be the dual isogeny, and then

$$\begin{eqnarray}\{\unicode[STIX]{x1D6FC}^{\ast }u,v\}_{A}=\{u,(\unicode[STIX]{x1D6FC}^{t})^{\ast }v\}_{B}.\end{eqnarray}$$

If $\unicode[STIX]{x1D719}:A\overset{{\sim}}{\rightarrow }A^{t}$ is a principal polarization then $\unicode[STIX]{x1D719}=\unicode[STIX]{x1D719}^{t}$ (using the identification of $(A^{t})^{t}$ with $A$ ). The polarization pairing

$$\begin{eqnarray}\{u,v\}_{\unicode[STIX]{x1D719}}=\{u,(\unicode[STIX]{x1D719}^{-1})^{\ast }v\}_{A}:H_{\text{dR}}^{1}(A)\times H_{\text{dR}}^{1}(A)\rightarrow k\end{eqnarray}$$

is skew-symmetric, as follows from the preceding two properties.

If $\unicode[STIX]{x1D6FC}\in \operatorname{End}(A)$ then $\text{Ros}_{\unicode[STIX]{x1D719}}(\unicode[STIX]{x1D6FC})\in \operatorname{End}(A)$ is defined by

$$\begin{eqnarray}\text{Ros}_{\unicode[STIX]{x1D719}}(\unicode[STIX]{x1D6FC})=\unicode[STIX]{x1D719}^{-1}\circ \unicode[STIX]{x1D6FC}^{t}\circ \unicode[STIX]{x1D719}.\end{eqnarray}$$

The previous properties imply then

$$\begin{eqnarray}\{\unicode[STIX]{x1D6FC}^{\ast }u,v\}_{\unicode[STIX]{x1D719}}=\{u,\text{Ros}_{\unicode[STIX]{x1D719}}(\unicode[STIX]{x1D6FC})^{\ast }v\}_{\unicode[STIX]{x1D719}}.\end{eqnarray}$$

If we apply the above for our values of $A$ , figuring in a tuple $\text{}\underline{A}\in S(k)$ , we get that $\{.\,,\,.\}$ pairs $H_{\text{dR}}^{1}(A/k)(\unicode[STIX]{x1D6F4})$ non-trivially with $H_{\text{dR}}^{1}(A^{t}/k)(\unicode[STIX]{x1D6F4})$ , while $\{.\,,\,.\}_{\unicode[STIX]{x1D719}}$ pairs $H_{\text{dR}}^{1}(A/k)(\unicode[STIX]{x1D6F4})$ non-trivially with $H_{\text{dR}}^{1}(A/k)(\overline{\unicode[STIX]{x1D6F4}})$ .

Finally, we recall that $\unicode[STIX]{x1D714}_{A/k}$ and $\unicode[STIX]{x1D714}_{A^{t}/k}$ are mutual annihilators of each other under $\{.\,,\,.\}$ . This induces a perfect pairing between $\unicode[STIX]{x1D714}_{A/k}$ with $H^{1}(A^{t},{\mathcal{O}})$ , hence the identification of $H^{1}(A^{t},{\mathcal{O}})$ with the Lie algebra of $A$ .

2.2.1 Foliations of height 1

In this section we review some general facts from algebraic geometry in characteristic $p$ , due to Rudakov and Shafarevich [Reference Rudakov and ShafarevichRS76], Ekedahl [Reference EkedahlEke87] and Miyaoka [Reference MiyaokaMiy87]. At the birational level they should be traced back, as mentioned in the introduction, to Jacobson’s theorem which establishes a ‘Galois theory’ for finite purely inseparable field extensions using derivations [Reference JacobsonJac75].

Let $k$ be an algebraically closed field of characteristic $p$ , and $X$ a non-singular $n$ -dimensional variety over $k$ . Let ${\mathcal{T}}X$ be the tangent sheaf of $X$ , a locally free sheaf of rank $n$ . Recall that ${\mathcal{T}}X$ becomes a $p$ -Lie algebra over $k$ if for any two vector fields $\unicode[STIX]{x1D709},\unicode[STIX]{x1D702}$ defined in some open set $U$ and regarded as operators on ${\mathcal{O}}_{X}(U)$ , we let

$$\begin{eqnarray}[\unicode[STIX]{x1D709},\unicode[STIX]{x1D702}]=\unicode[STIX]{x1D709}\circ \unicode[STIX]{x1D702}-\unicode[STIX]{x1D702}\circ \unicode[STIX]{x1D709},\quad \unicode[STIX]{x1D709}^{(p)}=\unicode[STIX]{x1D709}\circ \unicode[STIX]{x1D709}\circ \cdots \circ \unicode[STIX]{x1D709}\end{eqnarray}$$

(composition $p$ times).

Foliations of higher height (as in [Reference EkedahlEke87]) will not show up in this paper, until we discuss integral subvarieties at the end. We shall therefore refer to height 1 foliations simply as ‘foliations’. If ${\mathcal{E}}$ is a line sub-bundle then ${\mathcal{E}}$ is automatically involutive, as any two sections of ${\mathcal{E}}$ are proportional. But even in rank 1 the condition of being $p$ -closed is non-void, as the following example shows. Let $X=\mathbb{A}_{k}^{2}$ and let ${\mathcal{E}}={\mathcal{O}}_{X}\cdot \unicode[STIX]{x1D709}$ where

$$\begin{eqnarray}\unicode[STIX]{x1D709}=x\frac{\unicode[STIX]{x2202}}{\unicode[STIX]{x2202}x}+\frac{\unicode[STIX]{x2202}}{\unicode[STIX]{x2202}y}.\end{eqnarray}$$

It is easily checked that $\unicode[STIX]{x1D709}^{(p)}=x(\unicode[STIX]{x2202}/\unicode[STIX]{x2202}x)$ , but this is not a section of ${\mathcal{E}}$ .

If $Y\subset X$ is a non-singular subvariety then ${\mathcal{T}}Y\subset {\mathcal{T}}X|_{Y}$ may be considered ‘a foliation of height 1 along $Y$ ’. We call $Y$ an integral subvariety for the foliation ${\mathcal{E}}$ if ${\mathcal{E}}|_{Y}={\mathcal{T}}Y$ . Integral subvarieties always exist in a formal neighborhood of a point [Reference EkedahlEke87, Proposition 3.2], but they need not be unique, even if they are global. The foliation generated by the vector field $\unicode[STIX]{x2202}/\unicode[STIX]{x2202}y$ in $\mathbb{A}_{k}^{2}$ admits all the curves $x=a\,+\,by^{p}$ as integral curves, and infinitely many such curves pass through any given point. As another example, if $X$ is a simple abelian surface, then any non-zero tangent vector at the origin generates a unique translation-invariant line sub-bundle of ${\mathcal{T}}_{X}$ . Often it can be shown to be $p$ -closed, yielding a rank 1 foliation. This foliation has no integral curves at all, because an integral curve would necessarily be an elliptic curve, contradicting the fact that $X$ was assumed to be simple.

2.2.2 The relation between foliations and inseparable morphisms of height 1

Let $X$ be a non-singular $k$ -variety. A finite $k$ -morphism $X\overset{f}{\rightarrow }Y$ from $X$ to a $k$ -variety $Y$ is called of height 1 if there is a $k$ -morphism $Y\overset{g}{\rightarrow }X^{(p)}$ such that the composition

$$\begin{eqnarray}X\overset{f}{\rightarrow }Y\overset{g}{\rightarrow }X^{(p)}\end{eqnarray}$$

is $\mathit{Fr}_{X/k}$ . Here $X^{(p)}=\operatorname{Spec}(k)\times _{\unicode[STIX]{x1D6F7}_{k},\operatorname{Spec}(k)}X$ . If $f$ is also flat, then $Y$ is non-singular, since the property of being regular descends under flat morphisms between locally Noetherian schemes. In this case $f$ is faithfully flat, since it is surjective on $k$ -points, hence surjective. Therefore $g$ is also finite and faithfully flat. Since $f^{(p)}\circ g\circ f=f^{(p)}\circ \mathit{Fr}_{X/k}=\mathit{Fr}_{Y/k}\circ f$ we get also $f^{(p)}\circ g=\mathit{Fr}_{Y/k}$ . The link between foliations and height 1 morphisms is given by the following proposition.

Proposition 1 [Reference EkedahlEke87, Proposition 2.4].

Let $X$ be a non-singular $k$ -variety. There is a natural one-to-one correspondence between finite flat height 1 morphisms $f:X\rightarrow Y$ and height 1 foliations ${\mathcal{E}}\subset {\mathcal{T}}X$ . One has

$$\begin{eqnarray}\deg (f)=p^{\text{rk}({\mathcal{E}})}.\end{eqnarray}$$

Given $f$ , $\unicode[STIX]{x1D6FA}_{X/Y}$ is a locally free sheaf, hence the short exact sequence

$$\begin{eqnarray}f^{\ast }\unicode[STIX]{x1D6FA}_{Y}\rightarrow \unicode[STIX]{x1D6FA}_{X}\rightarrow \unicode[STIX]{x1D6FA}_{X/Y}\rightarrow 0\end{eqnarray}$$

splits and we let ${\mathcal{E}}=\unicode[STIX]{x1D6FA}_{X/Y}^{\vee }\subset \unicode[STIX]{x1D6FA}_{X}^{\vee }={\mathcal{T}}X$ , a height 1 foliation. Conversely, given ${\mathcal{E}}$ , we let ${\mathcal{O}}_{Y}={\mathcal{O}}_{X}^{{\mathcal{E}}=0}$ , the sheaf of functions annihilated by the derivations in ${\mathcal{E}}$ .

The fact that ${\mathcal{E}}$ is a sub-bundle, and not only a saturated subsheaf, is essential. Consider, for example, the subsheaf ${\mathcal{E}}$ of ${\mathcal{T}}\mathbb{A}^{2}$ generated by $\unicode[STIX]{x1D709}=x\cdot \unicode[STIX]{x2202}/\unicode[STIX]{x2202}x+y\cdot \unicode[STIX]{x2202}/\unicode[STIX]{x2202}y$ , which is saturated, but fails to be a sub-bundle at the origin. The quotient of $\mathbb{A}^{2}$ by ${\mathcal{E}}$ is the scheme $Y$ for which ${\mathcal{O}}_{Y}$ is the sheaf of functions $h$ on $\mathbb{A}^{2}$ satisfying $\unicode[STIX]{x1D709}(h)=0$ . This $Y$ is singular at the origin, and the quotient map is not flat. Note that $h=x^{i}y^{p-i}$ is such a function for $0\leqslant i\leqslant p$ .

Let $f:X\rightarrow Y$ be as in the proposition, $x\in X$ and $y=f(x)$ . Then one has the following local description of the completed local rings.

3.2.1 The Shimura variety $S_{K_{0}(p)}$

In addition to the lattice $\unicode[STIX]{x1D6EC}={\mathcal{O}}_{E}^{n+m}$ considered in § 2.1, consider also the ${\mathcal{O}}_{E}$ -lattices

$$\begin{eqnarray}\unicode[STIX]{x1D6EC}\supset \unicode[STIX]{x1D6EC}^{\prime }\supset \unicode[STIX]{x1D6EC}^{\prime \prime }\supset p\unicode[STIX]{x1D6EC},\end{eqnarray}$$

where

$$\begin{eqnarray}\unicode[STIX]{x1D6EC}^{\prime }=\langle pe_{1},\ldots ,pe_{m},e_{m+1},\ldots ,e_{n+m}\rangle ,\quad \unicode[STIX]{x1D6EC}^{\prime \prime }=\langle pe_{1},\ldots ,pe_{n},e_{n+1},\ldots ,e_{n+m}\rangle .\end{eqnarray}$$

Note that the dual of $\unicode[STIX]{x1D6EC}^{\prime }$ is $p^{-1}\unicode[STIX]{x1D6EC}^{\prime \prime }$ . Let $\mathscr{L}$ be the lattice chain in $K_{p}^{n+m}$

$$\begin{eqnarray}\cdots \supset \unicode[STIX]{x1D6EC}_{p}\supset \unicode[STIX]{x1D6EC}_{p}^{\prime }\supset \unicode[STIX]{x1D6EC}_{p}^{\prime \prime }\supset p\unicode[STIX]{x1D6EC}_{p}\supset \cdots\end{eqnarray}$$

obtained by tensoring with $\mathbb{Z}_{p}$ and extending by periodicity, and let $K_{0}(p)_{p}$ be its stabilizer in $K_{p}=\boldsymbol{G}(\mathbb{Z}_{p})$ . This is a parahoric subgroup, and if we let $K_{0}(p)$ be the adèlic level subgroup corresponding to it (and to full level $N$ as usual), we get the Shimura variety $S_{K_{0}(p)}$ , which is again defined over $E$ , and is an étale cover of $S_{K}$ .

3.2.2 The moduli problem ${\mathcal{S}}_{0}(p)$

Let ${\mathcal{S}}_{0}(p)$ be the integral model of $S_{K_{0}(p)}$ over ${\mathcal{O}}_{E,(p)}$ which was constructed by Rapoport and Zink in [Reference Rapoport and ZinkRZ96, §6.9]. We want to give a more concrete description of the moduli problem parametrized by ${\mathcal{S}}_{0}(p)$ . Let $R$ be an ${\mathcal{O}}_{E,(p)}$ -algebra and $\text{}\underline{A}\in {\mathcal{S}}(R)$ as in § 2.1. A finite flat ${\mathcal{O}}_{E}$ -subgroup scheme $H\subset A[p]$ is called Raynaud if for every characteristic $p$ geometric point $x:R\rightarrow k$ of $\operatorname{Spec}(R)$ , the Dieudonné module $M(H_{x})$ is balanced, in the sense that

$$\begin{eqnarray}\dim _{k}M(H_{x})(\unicode[STIX]{x1D6F4})=\dim _{k}M(H_{x})(\overline{\unicode[STIX]{x1D6F4}}).\end{eqnarray}$$

By $M(H_{x})$ we denote the covariant Dieudonné module of $H_{x}$ . It coincides with the contravariant Dieudonné module of the Cartier dual $H_{x}^{D}$ of $H_{x}$ . See [Reference de Shalit, Goren, Akbary and GundeSG18, §1.2.1] for a discussion of the Raynaud condition when $m=1$ , and its relation to the original condition imposed by Raynaud in [Reference RaynaudRay74].

We denoteFootnote ³ by $\unicode[STIX]{x1D70B}:{\mathcal{S}}_{0}(p)\rightarrow {\mathcal{S}}$ the morphism which on the moduli problem is ‘forget $H$ ’. The scheme ${\mathcal{S}}_{0}(p)$ is not smooth over ${\mathcal{O}}_{E,(p)}$ but Görtz [Reference GörtzGör01], and later Pappas and Zhu [Reference Pappas and ZhuPZ13], proved the following.

3.2.3 The ordinary-multiplicative and ordinary-étale loci

Let $S_{0}(p)$ be the special fiber of ${\mathcal{S}}_{0}(p)$ . Let $S_{0}(p)^{\text{ord}}=\unicode[STIX]{x1D70B}^{-1}(S^{\text{ord}})$ . If $x\in S^{\text{ord}}(k)$ is a geometric point, then ${\mathcal{A}}_{x}[p^{\infty }]$ is given by (2.1), and any isotropic Raynaud ${\mathcal{O}}_{E}$ -subgroup scheme of ${\mathcal{A}}_{x}[p]$ of rank $p^{2m}$ is of the form

$$\begin{eqnarray}H\simeq ({\mathcal{O}}_{E}\otimes \unicode[STIX]{x1D707}_{p})^{r}\times \mathscr{G}_{\unicode[STIX]{x1D6F4}}[p]^{s}\times ({\mathcal{O}}_{E}\otimes \mathbb{Z}/p\mathbb{Z})^{t}\end{eqnarray}$$

for an ‘ordinary type’ $(r,s,t)$ , $r+s+t=m$ . The type $(r,s,t)$ is locally constant on $S_{0}(p)^{\text{ord}}$ in the Zariski topology. Indeed, $p^{2r+2s}$ is the rank of the connected part $H^{\text{conn}}$ , which can only go up under specialization, by duality the same is true of $p^{2s+2t}$ , but $r+s+t$ is constant. Thus under specialization the only possibility is for $r$ and $t$ to go down, and for $s$ to go up. But the same must be true of $A[p]/H$ . If we specialize to a $\unicode[STIX]{x1D707}$ -ordinary point, the type of $A[p]/H$ is $(m-r,n-m-s,m-t)$ , hence $m-r$ and $m-t$ must also go down, forcing $r,s$ and $t$ to remain constant.

It follows that the discrete invariants $(r,s,t)$ decompose $S_{0}(p)^{\text{ord}}$ into disjoint open sets. We denote by $S_{0}(p)_{m}^{\text{ord}}$ the locus where $H$ is of multiplicative type ( $s=t=0$ ) and by $S_{0}(p)_{\acute{\text{e}}\text{t}}^{\text{ord}}$ the locus where $H$ is étale ( $r=s=0$ ). The other loci are denoted by $S_{0}(p)_{r,s,t}^{\text{ord}}$ , and the projection from them to $S^{\text{ord}}$ will be denoted $\unicode[STIX]{x1D70B}_{r,s,t}$ .

Denote by $\unicode[STIX]{x1D70B}_{m}$ and $\unicode[STIX]{x1D70B}_{\acute{\text{e}}\text{t}}$ the restriction of the map $\unicode[STIX]{x1D70B}$ to $S_{m}^{\text{ord}}$ and $S_{\acute{\text{e}}\text{t}}^{\text{ord}}$ (i.e. the maps $\unicode[STIX]{x1D70B}_{m,0,0}$ and $\unicode[STIX]{x1D70B}_{0,0,m}$ ). The map $\unicode[STIX]{x1D70B}_{m}$ is an isomorphism. In fact, it has the section associating to every $\text{}\underline{A}\in S^{\text{ord}}(R)$ the $R$ -point $(\text{}\underline{A},A[p]^{\text{mult}})\in S_{0}(p)_{m}^{\text{ord}}(R)$ where $A[p]^{\text{mult}}$ is the maximal finite flat subgroup scheme of $A[p]$ which is of multiplicative type (connected with étale dual). This subgroup is automatically isotropic and Raynaud. Denote this section by

$$\begin{eqnarray}\unicode[STIX]{x1D70E}_{m}:S^{\text{ord}}\rightarrow S_{0}(p)_{m}^{\text{ord}}.\end{eqnarray}$$

3.2.4 Morphisms between ordinary-étale components

We shall define morphisms $\unicode[STIX]{x1D703},\unicode[STIX]{x1D703}^{\prime }$ , $\unicode[STIX]{x1D70C}$ and $\unicode[STIX]{x1D70C}^{\prime }$ that fit into the commutative diagram below. The scheme $(S_{0}(p)_{\acute{\text{e}}\text{t}}^{\text{ord}})^{(p)}$ appearing in the diagram classifies the same objects as $S_{0}(p)_{\acute{\text{e}}\text{t}}^{\text{ord}}$ , except that the signature of the ${\mathcal{O}}_{E}$ action is $(m,n)$ instead of $(n,m)$ .

Before we embark on the definition, we want to make a notational remark.

Remark about Frobenii. We denote by $\mathit{Fr}_{p}$ the Frobenius of the scheme $S$ (or $S_{0}(p)$ ) relative to  $\unicode[STIX]{x1D705}$ . Thus, $\mathit{Fr}_{p}$ is a morphism of $\unicode[STIX]{x1D705}$ -schemes

$$\begin{eqnarray}\mathit{Fr}_{p}:S\rightarrow S^{(p)},\end{eqnarray}$$

where $S^{(p)}=\unicode[STIX]{x1D6F7}_{\unicode[STIX]{x1D705}}^{\ast }S$ is the base change of $S$ with respect to $\unicode[STIX]{x1D6F7}_{\unicode[STIX]{x1D705}}$ . If $\unicode[STIX]{x1D709}\in S(R)$ for some $\unicode[STIX]{x1D705}$ -algebra $R$ and $\unicode[STIX]{x1D709}$ corresponds to the tuple $\text{}\underline{A}=\unicode[STIX]{x1D709}^{\ast }\text{}\underline{{\mathcal{A}}}$ , then $\mathit{Fr}_{p}(\unicode[STIX]{x1D709})\in S^{(p)}(R)$ corresponds to $\text{}\underline{A}^{(p)}=\unicode[STIX]{x1D6F7}_{R}^{\ast }\text{}\underline{A}$ , the base change of $A$ (with the associated PEL structure) with respect to  $\unicode[STIX]{x1D6F7}_{R}$ . Note that $A^{(p)}$ has type $(m,n)$ . We write in short

$$\begin{eqnarray}\mathit{Fr}_{p}(\text{}\underline{A})=\text{}\underline{A}^{(p)}.\end{eqnarray}$$

This should not be confused with the isogeny $\text{Fr}:A\rightarrow A^{(p)}$ , which is a morphism of abelian schemes over $\operatorname{Spec}(R)$ . A more appropriate convention would have been to denote $\mathit{Fr}_{p}$ by $\mathit{Fr}_{S/\unicode[STIX]{x1D705}}$ and the isogeny $\text{Fr}$ by $\mathit{Fr}_{A/R}$ or $\mathit{Fr}_{{\mathcal{A}}/S}$ , but this would result in a pretty heavy notation.

To define the map $\unicode[STIX]{x1D70C}$ we consider the map

$$\begin{eqnarray}\unicode[STIX]{x1D703}:S_{0}(p)_{m}^{\text{ord}}\rightarrow S_{0}(p)_{\acute{\text{e}}\text{t}}^{\text{ord}}\end{eqnarray}$$

defined on the moduli problems as

$$\begin{eqnarray}\unicode[STIX]{x1D703}(\text{}\underline{A},H)=(\text{}\underline{A}^{(p^{2})},\text{Fr}(\text{Ver}^{-1}(H))).\end{eqnarray}$$

Then we let

$$\begin{eqnarray}\unicode[STIX]{x1D70C}=\unicode[STIX]{x1D703}\circ \unicode[STIX]{x1D70E}_{m}.\end{eqnarray}$$

Some words of explanation are in order. Here $\text{Ver}^{-1}(H)$ is the kernel of the isogeny $A^{(p)}\overset{\text{Ver}}{\rightarrow }A\rightarrow A/H$ , hence is finite flat of rank $p^{3m+n}$ . For $H=A[p]^{\text{mult}}$ , where $\text{Ver}$ is an isomorphism from $H^{(p)}$ to $H$ , $\text{Ver}^{-1}(H)$ coincides, as a group functor, with $H^{(p)}+A^{(p)}[\text{Ver}]$ , so is seen to be contained in $A^{(p)}[p]$ . Also in this case, the kernel of $\text{Fr}:A^{(p)}[p]\rightarrow A^{(p^{2})}[p]$ is contained in $\text{Ver}^{-1}(H)$ , hence the image $\text{Fr}(\text{Ver}^{-1}(H))$ is a finite flat subgroup scheme of rank $p^{2m}$ . It is easily seen that this subgroup scheme is ${\mathcal{O}}_{E}$ -stable, Raynaud, isotropic and étale, since these properties can be checked on the geometric fibers. We also remark that for $H=A[p]^{\text{mult}}$

$$\begin{eqnarray}\text{Fr}(\text{Ver}^{-1}(H))=\text{Fr}(A^{(p)}[\text{Ver}])=\text{Fr}^{2}(A[p]).\end{eqnarray}$$

The reason we chose to define $\unicode[STIX]{x1D703}$ the way we did is that this is the definition that will generalize later on, in § 4.2, when $H$ is no longer multiplicative, to a map between the Zariski closures $S_{0}(p)_{m}$ and $S_{0}(p)_{\acute{\text{e}}\text{t}}$ in $S_{0}(p)$ .

There is a similar map

$$\begin{eqnarray}\unicode[STIX]{x1D703}^{\prime }:S_{0}(p)_{\acute{\text{e}}\text{t}}^{\text{ord}}\rightarrow S_{0}(p)_{m}^{\text{ord}}\end{eqnarray}$$

defined on the moduli problems as

$$\begin{eqnarray}\unicode[STIX]{x1D703}^{\prime }(\text{}\underline{A},H)=(\text{}\underline{A},\text{Ver}(\text{Fr}^{-1}(H^{(p^{2})}))).\end{eqnarray}$$

The proof that it is well-defined is similar to the one for $\unicode[STIX]{x1D703}$ . Note that

$$\begin{eqnarray}\unicode[STIX]{x1D70B}_{\acute{\text{e}}\text{t}}\circ \unicode[STIX]{x1D703}=\mathit{Fr}_{p}^{2}\circ \unicode[STIX]{x1D70B}_{m},\quad \unicode[STIX]{x1D70B}_{m}\circ \unicode[STIX]{x1D703}^{\prime }=\unicode[STIX]{x1D70B}_{\acute{\text{e}}\text{t}},\quad \unicode[STIX]{x1D703}\circ \unicode[STIX]{x1D703}^{\prime }=\mathit{Fr}_{p}^{2},\quad \unicode[STIX]{x1D703}^{\prime }\circ \unicode[STIX]{x1D703}=\mathit{Fr}_{p}^{2}.\end{eqnarray}$$

The definition of $\unicode[STIX]{x1D70C}^{\prime }$ is a little more subtle.Footnote ⁴ Let $(\text{}\underline{A}_{1},H_{1})\in (S_{0}(p)_{\acute{\text{e}}\text{t}}^{\text{ord}})^{(p)}(R)$ . Thanks to the polarization, the subgroup scheme $H_{1}^{\bot }$ which is the annihilator of $H_{1}$ relative to the Weil pairing on $A_{1}[p]$ , is well-defined, and is finite flat of rank $p^{2n}$ over $R$ . We claim that the closed subgroup scheme $H_{1}^{\bot }[\text{Fr}]$ is finite flat of rank $p^{n-m}$ . Indeed, it is enough to check it for the universal $(\text{}\underline{A}_{1},H_{1})$ over $(S_{0}(p)_{\acute{\text{e}}\text{t}}^{\text{ord}})^{(p)}$ . But now the base is reduced (by [Reference GörtzGör01]) so it is enough to check that all the geometric fibers of $H_{1}^{\bot }[\text{Fr}]$ are of the same rank, and that this rank is $p^{n-m}$ . This is straightforward, given that over an algebraically closed field we have the description (2.1). For any geometric point $x:\operatorname{Spec}(k)\rightarrow \operatorname{Spec}(R)$ , $H_{1}^{\bot }[\text{Fr}]$ becomes the $\unicode[STIX]{x1D6FC}_{p}$ -subgroup which is the kernel of Frobenius in the local-local part of $x^{\ast }(A_{1}[p])$ . Seen in another light, giving $H_{1}$ not only splits the connected-étale exact sequence over $\operatorname{Spec}(R)$ , but allows us to split off the local-local part from the multiplicative part in $A_{1}[\text{Fr}]$ . In particular, $H_{1}^{\bot }[\text{Fr}]$ does not intersect $H_{1}$ , so

$$\begin{eqnarray}K_{1}:=H_{1}^{\bot }[\text{Fr}]+H_{1}\simeq H_{1}^{\bot }[\text{Fr}]\times H_{1}\end{eqnarray}$$

is finite flat of rank $p^{n+m}$ . This $K_{1}$ is a maximal isotropic subgroup of $A_{1}[p]$ , stable under ${\mathcal{O}}_{E}$ , whose tangent space is $n-m$ dimensional, of type $\overline{\unicode[STIX]{x1D6F4}}$ (sic!). Descending the polarization and the endomorphisms to $A=A_{1}/K_{1}$ we get a principally polarized abelian scheme over $R$ , of type $(n,m)$ . We let

(3.1) $$\begin{eqnarray}\text{}\underline{A}=\unicode[STIX]{x1D70C}^{\prime }(\text{}\underline{A}_{1},H_{1}):=\langle p\rangle _{N}^{-1}\text{}\underline{A}_{1}/K_{1}\in S^{\text{ord}}(R).\end{eqnarray}$$

The underlying principally polarized abelian scheme with endomorphisms by ${\mathcal{O}}_{E}$ is $A$ . The level- $N$ structure differs from the one descended from $\text{}\underline{A}_{1}$ by the diamond operator $\langle p\rangle _{N}^{-1}$ . Recall that the diamond operator  $\langle a\rangle _{N}$ , for $a\in ({\mathcal{O}}_{E}/N{\mathcal{O}}_{E})^{\times }$ takes an ${\mathcal{O}}_{E}$ -level- $N$ structure $\unicode[STIX]{x1D702}:({\mathcal{O}}_{E}/N{\mathcal{O}}_{E})^{n+m}\simeq A[N]$ to $\unicode[STIX]{x1D702}\circ [a]$ , where $[a]$ is multiplication by $a$ .

Having defined the maps in the diagram, we now check its commutativity. We only have to check the commutativity of the top two triangles, the bottom two being obvious. Let $\text{}\underline{A}_{1}$ and $\text{}\underline{A}$ be related by (3.1). Consider the morphism $\text{Fr}:\text{}\underline{A}\rightarrow \text{}\underline{A}^{(p)}$ obtained by dividing $A$ by $A[\text{Fr}]$ and descending the polarization, the endomorphisms, and the level structure. Since $A[\text{Fr}]=A_{1}[p]/K_{1}$ we get the string of isomorphisms

(3.2) $$\begin{eqnarray}\text{}\underline{A}^{(p)}\simeq \text{}\underline{A}/A[\text{Fr}]\simeq (\langle p\rangle _{N}^{-1}\text{}\underline{A}_{1}/K_{1})/(A_{1}[p]/K_{1})\simeq \langle p\rangle _{N}^{-1}\text{}\underline{A}_{1}/A_{1}[p]\simeq \text{}\underline{A}_{1}.\end{eqnarray}$$

(The last isomorphism is multiplication by $p$ , and it is the reason for introducing $\langle p\rangle _{N}^{-1}$ in the definition of $\unicode[STIX]{x1D70C}^{\prime }$ .) We conclude that to accommodate an étale subgroup scheme like $H_{1}$ , $\text{}\underline{A}_{1}$ must be of the form $\text{}\underline{A}^{(p)}$ and $K_{1}=A^{(p)}[\text{Ver}]$ . This may not be said of $H_{1}$ itself, in general. However, if this is the case and $(\text{}\underline{A}_{1},H_{1})=(\text{}\underline{A}^{(p)},H^{(p)})=\mathit{Fr}_{p}((\text{}\underline{A},H))$ then the above discussion shows that $\unicode[STIX]{x1D70C}^{\prime }(\text{}\underline{A}_{1},H_{1})=\text{}\underline{A}$ , proving the commutativity of the first triangle in the diagram:

$$\begin{eqnarray}\unicode[STIX]{x1D70C}^{\prime }\circ \mathit{Fr}_{p}=\unicode[STIX]{x1D70B}_{\acute{\text{e}}\text{t}}.\end{eqnarray}$$

For the second triangle consider

$$\begin{eqnarray}\displaystyle \unicode[STIX]{x1D70C}\circ \unicode[STIX]{x1D70C}^{\prime }(\text{}\underline{A}_{1},H_{1}) & = & \displaystyle \unicode[STIX]{x1D70C}(\text{}\underline{A})=(\text{}\underline{A}^{(p^{2})},\text{Fr}(\text{Ver}^{-1}(A[p]^{\text{mult}})))=(\text{}\underline{A}^{(p^{2})},\text{Fr}^{2}(A[p]))\nonumber\\ \displaystyle & = & \displaystyle (\text{}\underline{A}_{1}^{(p)},H_{1}^{(p)})=\mathit{Fr}_{p}((\text{}\underline{A}_{1},H_{1})).\nonumber\end{eqnarray}$$

To justify the transition from the first to the second line, note that $H_{1}^{(p)}=\text{Fr}(H_{1})\subset \text{Fr}^{2}(A[p])$ as $H_{1}\subset \text{Fr}(A[p])$ , but $H_{1}^{(p)}$ and $\text{Fr}^{2}(A[p])$ are both finite flat of rank $p^{2m}$ , so they coincide.

Lemma 10. The degrees of $\unicode[STIX]{x1D70C}$ and $\unicode[STIX]{x1D70C}^{\prime }$ are given by

$$\begin{eqnarray}\deg (\unicode[STIX]{x1D70C})=p^{m^{2}},\quad \deg (\unicode[STIX]{x1D70C}^{\prime })=p^{(n-m)m}.\end{eqnarray}$$

Furthermore, $\deg (\unicode[STIX]{x1D703})=\deg (\unicode[STIX]{x1D70C})$ and $\deg (\unicode[STIX]{x1D703}^{\prime })=\deg (\unicode[STIX]{x1D70B}_{\acute{\text{e}}\text{t}})=p^{(2n-m)m}$ .

Proof. Since $\unicode[STIX]{x1D70C}\circ \unicode[STIX]{x1D70C}^{\prime }=\mathit{Fr}_{p}$ is of degree $p^{nm}$ , it is enough to prove the formula for $\deg (\unicode[STIX]{x1D70C}^{\prime })$ . Since $\unicode[STIX]{x1D70C}^{\prime }\circ \mathit{Fr}_{p}=\unicode[STIX]{x1D70B}_{\acute{\text{e}}\text{t}}$ it is enough to prove that $\deg (\unicode[STIX]{x1D70B}_{\acute{\text{e}}\text{t}})=p^{2nm-m^{2}}$ . We use a method of degeneration from characteristic 0, based on the flatness of $\unicode[STIX]{x1D70B}:S_{0}(p)^{\text{ord}}\rightarrow S^{\text{ord}}$ . This map is flat because it is finite, $S^{\text{ord}}$ is regular and $S_{0}(p)^{\text{ord}}$ is Cohen–Macaulay. (In fact, the arguments of Proposition 8 prove that $S_{0}(p)^{\text{ord}}$ is non-singular.)

Denote by ${\mathcal{S}}^{\text{ord}}$ the complement in ${\mathcal{S}}$ of the non-ordinary locus in the special fiber, and similarly ${\mathcal{S}}_{0}(p)^{\text{ord}}$ . Since ${\mathcal{S}}_{0}(p)^{\text{ord}}$ and ${\mathcal{S}}^{\text{ord}}$ are flat over ${\mathcal{O}}_{E,(p)}$ and both the generic and special fibers of $\unicode[STIX]{x1D70B}:{\mathcal{S}}_{0}(p)^{\text{ord}}\rightarrow {\mathcal{S}}^{\text{ord}}$ are flat, then by the criterion for flatness fiber-by-fiber $\unicode[STIX]{x1D70B}$ is flat also on the ordinary parts of the arithmetic schemes.

Fix a $W(k)$ -valued point

$$\begin{eqnarray}\unicode[STIX]{x1D709}:\operatorname{Spec}(W(k))\rightarrow {\mathcal{S}}^{\text{ord}},\end{eqnarray}$$

and denote its specialization by $\unicode[STIX]{x1D709}_{0}:\operatorname{Spec}(k)\rightarrow S^{\text{ord}}$ . Consider the pull-back

$$\begin{eqnarray}\unicode[STIX]{x1D709}^{\ast }{\mathcal{S}}_{0}(p)\rightarrow \operatorname{Spec}(W(k)).\end{eqnarray}$$

As the base change of the finite flat morphism ${\mathcal{S}}_{0}(p)^{\text{ord}}\rightarrow {\mathcal{S}}^{\text{ord}}$ , this map is also finite flat. We denote by $\unicode[STIX]{x1D702}:\unicode[STIX]{x1D709}^{\ast }{\mathcal{S}}_{0}(p)\rightarrow {\mathcal{S}}_{0}(p)$ the base change of $\unicode[STIX]{x1D709}$ and by $\unicode[STIX]{x1D702}_{0}$ that of $\unicode[STIX]{x1D709}_{0}$ .

Let $\unicode[STIX]{x1D709}^{\ast }{\mathcal{S}}_{0}(p)_{\acute{\text{e}}\text{t}}$ be the connected component of $\unicode[STIX]{x1D709}^{\ast }{\mathcal{S}}_{0}(p)$ containing, in the special fiber, $\unicode[STIX]{x1D702}_{0}^{\ast }(S_{0}(p)_{\acute{\text{e}}\text{t}}^{\text{ord}})$ . It is finite flat over $W(k)$ , and the degree of its special fiber is $\deg (\unicode[STIX]{x1D70B}_{\acute{\text{e}}\text{t}})$ . We compute this degree in the generic fiber. Let $A=\unicode[STIX]{x1D709}^{\ast }{\mathcal{A}}$ be the pull-back of the universal abelian scheme to $W(k)$ , and let $A_{0}$ be its special fiber. Consider $A(\overline{E}_{p})[p]$ , the $p$ -torsion in the group of points of $A$ in a fixed algebraic closure $\overline{E}_{p}$ of the local field $E_{p}$ . The geometric points in the generic fiber of $\unicode[STIX]{x1D709}^{\ast }{\mathcal{S}}_{0}(p)_{\acute{\text{e}}\text{t}}$ are in one-to-one correspondence with the isotropic, ${\mathcal{O}}_{E}$ -stable subgroups $H\subset A(\overline{E}_{p})[p]$ of rank $p^{2m}$ specializing to $A_{0}[p]^{\acute{\text{e}}\text{t}}\subset A_{0}[p]$ .

Denote by $A(\overline{E}_{p})^{0}$ the kernel of the reduction map $A(\overline{E}_{p})\rightarrow A_{0}(k)$ . We have to count isotropic, ${\mathcal{O}}_{E}$ -stable subgroups $H\subset A(\overline{E}_{p})[p]$ of rank $p^{2m}$ satisfying

$$\begin{eqnarray}H+A(\overline{E}_{p})^{0}[p]=A(\overline{E}_{p})[p].\end{eqnarray}$$

We are now reduced to linear algebra. The ${\mathcal{O}}_{E}$ -module $A(\overline{E}_{p})[p]$ with the hermitian pairing derived from the polarization is isomorphic to $\unicode[STIX]{x1D705}^{n+m}$ with the $\unicode[STIX]{x1D705}$ -hermitian form

$$\begin{eqnarray}(u,v)=\,^{t}u^{(p)}\left(\begin{array}{@{}ccc@{}} & & 1_{m}\\ & 1_{n-m}\\ 1_{m}\end{array}\right)v\end{eqnarray}$$

and we may choose the isomorphism so that $A(\overline{E}_{p})^{0}[p]$ is the subspace with the last $m$ entries $0$ . We thus have to count equivalence classes of $\unicode[STIX]{x1D705}$ -linear maps $\unicode[STIX]{x1D706}:\unicode[STIX]{x1D705}^{m}{\hookrightarrow}\unicode[STIX]{x1D705}^{n+m}$ satisfying (1) the image of $\unicode[STIX]{x1D706}$ is isotropic, and (2) the projection of the image of $\unicode[STIX]{x1D706}$ on the last $m$ coordinates is an isomorphism. Two such maps $\unicode[STIX]{x1D706}_{1}$ and $\unicode[STIX]{x1D706}_{2}$ are equivalent if $\unicode[STIX]{x1D706}_{2}=\unicode[STIX]{x1D706}_{1}\circ \unicode[STIX]{x1D6FC}$ for $\unicode[STIX]{x1D6FC}\in \text{GL}_{m}(\unicode[STIX]{x1D705})$ . This is the same as counting matrices $\unicode[STIX]{x1D6E4}\in M_{n\times m}(\unicode[STIX]{x1D705})$ satisfying

$$\begin{eqnarray}(\text{}^{t}\unicode[STIX]{x1D6E4}^{(p)},1_{m})\left(\begin{array}{@{}ccc@{}} & & 1_{m}\\ & 1_{n-m}\\ 1_{m}\end{array}\right)\left(\begin{array}{@{}c@{}}\unicode[STIX]{x1D6E4}\\ 1_{m}\end{array}\right)=0,\end{eqnarray}$$

or equivalently, counting pairs $(\unicode[STIX]{x1D6E4}_{1},\unicode[STIX]{x1D6E4}_{2})\in M_{m\times m}(\unicode[STIX]{x1D705})\times M_{(n-m)\times m}(\unicode[STIX]{x1D705})$ satisfying

$$\begin{eqnarray}\unicode[STIX]{x1D6E4}_{1}+\,^{t}\unicode[STIX]{x1D6E4}_{1}^{(p)}+\,^{t}\unicode[STIX]{x1D6E4}_{2}^{(p)}\unicode[STIX]{x1D6E4}_{2}=0.\end{eqnarray}$$

This number is easily seen to be $p^{2nm-m^{2}}$ . Indeed, there are $p^{2(n-m)m}$ choices for $\unicode[STIX]{x1D6E4}_{2}$ . For each choice of $\unicode[STIX]{x1D6E4}_{2}$ there are $p^{2(m(m-1)/2)}$ choices for the entries of $\unicode[STIX]{x1D6E4}_{1}$ above the diagonal, which are arbitrary and determine the entries below the diagonal uniquely, and $p^{m}$ choices for the entries on the diagonal. This concludes the proof of the lemma.◻

3.2.5 The relation between the foliation and $\unicode[STIX]{x1D70C}$

In § 3.1 we have constructed the foliation ${\mathcal{T}}S^{+}$ in the tangent bundle of $S^{\text{ord}}$ , while in § 3.2.4 we have constructed a flat height 1 morphism $\unicode[STIX]{x1D70C}:S^{\text{ord}}\rightarrow S_{0}(p)_{\acute{\text{e}}\text{t}}^{\text{ord}}$ . We shall now prove that the two correspond to each other under the dictionary between height 1 morphisms and height 1 foliations discussed in Proposition 1.

Proof. We have to show that the image of $\unicode[STIX]{x1D70C}^{\ast }(\unicode[STIX]{x1D6FA}_{S_{0}(p)_{\acute{\text{e}}\text{t}}^{\text{ord}}})$ in $\unicode[STIX]{x1D6FA}_{S^{\text{ord}}}$ is $\text{KS}({\mathcal{P}}_{0}\otimes {\mathcal{Q}})$ . Since $\unicode[STIX]{x1D70C}$ is a finite flat height 1 morphism of degree $p^{m^{2}}$ we know, from the general theory explained in § 2.2, that the image of $\unicode[STIX]{x1D70C}^{\ast }(\unicode[STIX]{x1D6FA}_{S_{0}(p)_{\acute{\text{e}}\text{t}}^{\text{ord}}})$ in $\unicode[STIX]{x1D6FA}_{S}$ is a sub-bundle of $\unicode[STIX]{x1D6FA}_{S^{\text{ord}}}$ of rank $(n-m)m$ . Since the same is true of $\text{KS}({\mathcal{P}}_{0}\otimes {\mathcal{Q}})$ , it is enough to prove the inclusion

$$\begin{eqnarray}\text{KS}({\mathcal{P}}_{0}\otimes {\mathcal{Q}})\subset \text{Im}(\unicode[STIX]{x1D70C}^{\ast }(\unicode[STIX]{x1D6FA}_{S_{0}(p)_{\acute{\text{e}}\text{t}}^{\text{ord}}})\rightarrow \unicode[STIX]{x1D6FA}_{S^{\text{ord}}}).\end{eqnarray}$$

As the right-hand side is equal to $\ker ({\unicode[STIX]{x1D70C}^{\prime }}^{\ast }:\unicode[STIX]{x1D6FA}_{S^{\text{ord}}}\rightarrow \unicode[STIX]{x1D6FA}_{(S_{0}(p)_{\acute{\text{e}}\text{t}}^{\text{ord}})^{(p)}})$ (they are both sub-bundles of rank $(n-m)m$ and the image of $\unicode[STIX]{x1D70C}^{\ast }$ is contained in the kernel of ${\unicode[STIX]{x1D70C}^{\prime }}^{\ast }$ because $\unicode[STIX]{x1D70C}\circ \unicode[STIX]{x1D70C}^{\prime }=\mathit{Fr}_{p}$ ), it will be enough to prove that $\text{KS}({\mathcal{P}}_{0}\otimes {\mathcal{Q}})$ is contained in the latter. More precisely, we have to show that

$$\begin{eqnarray}{\unicode[STIX]{x1D70C}^{\prime }}^{\ast }\text{KS}({\mathcal{P}}_{0}\otimes {\mathcal{Q}})\subset \ker ({\unicode[STIX]{x1D70C}^{\prime }}^{\ast }:{\unicode[STIX]{x1D70C}^{\prime }}^{\ast }\unicode[STIX]{x1D6FA}_{S^{\text{ord}}}\rightarrow \unicode[STIX]{x1D6FA}_{(S_{0}(p)_{\acute{\text{e}}\text{t}}^{\text{ord}})^{(p)}}).\end{eqnarray}$$

For this purpose consider the universal pair $({\mathcal{A}}_{1},{\mathcal{H}}_{1})$ over $(S_{0}(p)_{\acute{\text{e}}\text{t}}^{\text{ord}})^{(p)}$ . In (3.2) we have constructed ${\mathcal{A}}$ such that, at the level of points, $\unicode[STIX]{x1D70C}^{\prime }(({\mathcal{A}}_{1},{\mathcal{H}}_{1}))={\mathcal{A}}$ (where to simplify typesetting, we omit the underline symbol). Note that ${\mathcal{A}}$ is a scheme over $S^{\text{ord}}$ and, letting ${\mathcal{B}}=\unicode[STIX]{x1D70C}^{\prime \ast }{\mathcal{A}}$ , the relations obtained in (3.2) imply a canonical isomorphism ${\mathcal{A}}_{1}={\mathcal{B}}^{(p)}$ , as abelian schemes over $(S_{0}(p)_{\acute{\text{e}}\text{t}}^{\text{ord}})^{(p)}$ . The construction also provides a canonical isogeny ${\mathcal{A}}_{1}\rightarrow {\mathcal{B}}$ , which is nothing else than $\text{Ver}$ .

The kernel of $\text{Ver}$ contains the finite flat group scheme ${\mathcal{H}}_{1}$ . Thus, letting ${\mathcal{C}}={\mathcal{A}}_{1}/{\mathcal{H}}_{1}$ , we get a decomposition of $\text{Ver}$ as the composition of two isogenies between abelian schemes over $(S_{0}(p)_{\acute{\text{e}}\text{t}}^{\text{ord}})^{(p)}$ as follows.

Here $\unicode[STIX]{x1D713}$ is the isogeny with kernel ${\mathcal{H}}_{1}$ , and $\unicode[STIX]{x1D711}$ is the isogeny with kernel ${\mathcal{A}}_{1}[\text{Ver}]/{\mathcal{H}}_{1}$ . Note that although $\text{Ver}:{\mathcal{A}}_{1}\rightarrow {\mathcal{B}}$ is a pull-back by $\unicode[STIX]{x1D70C}^{\prime }$ of a similar isogeny over $S^{\text{ord}}$ , only over $(S_{0}(p)_{\acute{\text{e}}\text{t}}^{\text{ord}})^{(p)}$ does it factor through ${\mathcal{C}}$ , because ${\mathcal{H}}_{1}$ is not the pull-back of a group scheme on $S^{\text{ord}}$ .

Now consider the commutative diagram

(3.3)

resulting from the functoriality of the Gauss–Manin connection with respect to the isogeny $\unicode[STIX]{x1D711}$ . Here $\text{KS}_{{\mathcal{B}}}$ is the Kodaira–Spencer map for the family ${\mathcal{B}}\rightarrow (S_{0}(p)_{\acute{\text{e}}\text{t}}^{\text{ord}})^{(p)}$ and likewise for ${\mathcal{C}}$ .

The kernel of the left vertical arrow $\unicode[STIX]{x1D711}^{\ast }$ is precisely ${\unicode[STIX]{x1D70C}^{\prime }}^{\ast }({\mathcal{P}}_{0})$ . This is because $\unicode[STIX]{x1D713}$ is étale, so $\unicode[STIX]{x1D713}^{\ast }$ is an isomorphism on cotangent spaces, hence

$$\begin{eqnarray}\ker (\unicode[STIX]{x1D711}^{\ast }|_{{\unicode[STIX]{x1D70C}^{\prime }}^{\ast }{\mathcal{P}}})=\ker (\text{Ver}^{\ast }|_{{\unicode[STIX]{x1D70C}^{\prime }}^{\ast }{\mathcal{P}}})={\unicode[STIX]{x1D70C}^{\prime }}^{\ast }({\mathcal{P}}_{0}).\end{eqnarray}$$

On the right side of (3.3), we claim that $1\otimes \unicode[STIX]{x1D711}^{\ast }$ is injective. To verify this, consider the following commutative diagram.

(3.4)

The right vertical arrow may be identified with the $\unicode[STIX]{x1D6F4}$ -component of the map $\unicode[STIX]{x1D711}_{\ast }^{t}:\operatorname{Lie}({\mathcal{B}}^{t})\rightarrow \operatorname{Lie}({\mathcal{C}}^{t})$ . The signature of ${\mathcal{B}}^{t}$ is $(m,n)$ , and at every geometric point $x$

$$\begin{eqnarray}{\mathcal{B}}_{x}^{t}[p]\simeq ({\mathcal{O}}_{E}\otimes \unicode[STIX]{x1D707}_{p})^{m}\oplus (\mathscr{G}_{\overline{\unicode[STIX]{x1D6F4}}}[p])^{n-m}\oplus ({\mathcal{O}}_{E}\otimes \mathbb{Z}/p\mathbb{Z})^{m}.\end{eqnarray}$$

As $\ker (\unicode[STIX]{x1D711})={\mathcal{A}}_{1}[\text{Ver}]/{\mathcal{H}}_{1}$ is local-local, so is its dual $\ker (\unicode[STIX]{x1D711}^{t})$ ; in fact, its geometric fibers are all isomorphic to $\unicode[STIX]{x1D6FC}_{p}^{n-m}\subset (\mathscr{G}_{\overline{\unicode[STIX]{x1D6F4}}}[p])^{n-m}$ . The Lie algebra of $\ker (\unicode[STIX]{x1D711}^{t})$ , i.e. $\ker (\unicode[STIX]{x1D711}_{\ast }^{t})$ , is therefore of type $\overline{\unicode[STIX]{x1D6F4}}$ . We conclude that the right vertical arrow of (3.4), and with it the right vertical arrow of (3.3), are injective, as claimed.

As ${\mathcal{B}}={\unicode[STIX]{x1D70C}^{\prime }}^{\ast }{\mathcal{A}}$ , the morphism $\text{KS}_{{\mathcal{B}}}$ is the composition of the map

$$\begin{eqnarray}{\unicode[STIX]{x1D70C}^{\prime }}^{\ast }(\text{KS}_{{\mathcal{A}}}):{\unicode[STIX]{x1D70C}^{\prime }}^{\ast }({\mathcal{P}})\rightarrow {\unicode[STIX]{x1D70C}^{\prime }}^{\ast }(\unicode[STIX]{x1D6FA}_{S^{\text{ord}}})\otimes {\unicode[STIX]{x1D70C}^{\prime }}^{\ast }({\mathcal{Q}}^{\vee })\end{eqnarray}$$

(we identify ${\mathcal{Q}}(\overline{\unicode[STIX]{x1D6F4}})$ with $\unicode[STIX]{x1D714}_{{\mathcal{A}}^{t}}(\unicode[STIX]{x1D6F4})$ via the polarization as usual, hence also $R^{1}\unicode[STIX]{x1D70B}_{\ast }{\mathcal{O}}_{{\mathcal{A}}}(\unicode[STIX]{x1D6F4})=\operatorname{Lie}({\mathcal{A}}^{t})(\unicode[STIX]{x1D6F4})$ with ${\mathcal{Q}}^{\vee }$ ) and the map induced by

$$\begin{eqnarray}{\unicode[STIX]{x1D70C}^{\prime }}^{\ast }:{\unicode[STIX]{x1D70C}^{\prime }}^{\ast }(\unicode[STIX]{x1D6FA}_{S^{\text{ord}}})\rightarrow \unicode[STIX]{x1D6FA}_{(S_{0}(p)_{\acute{\text{e}}\text{t}}^{\text{ord}})^{(p)}}.\end{eqnarray}$$

From the commutativity of (3.3) we conclude that $\text{KS}_{{\mathcal{B}}}({\unicode[STIX]{x1D70C}^{\prime }}^{\ast }({\mathcal{P}}_{0}))=0$ , hence the desired inclusion

$$\begin{eqnarray}{\unicode[STIX]{x1D70C}^{\prime }}^{\ast }(\text{KS}({\mathcal{P}}_{0}\otimes {\mathcal{Q}}))\subset \ker ({\unicode[STIX]{x1D70C}^{\prime }}^{\ast }:{\unicode[STIX]{x1D70C}^{\prime }}^{\ast }\unicode[STIX]{x1D6FA}_{S^{\text{ord}}}\rightarrow \unicode[STIX]{x1D6FA}_{(S_{0}(p)_{\acute{\text{e}}\text{t}}^{\text{ord}})^{(p)}}).\Box\end{eqnarray}$$

4.1.1 Definition and general properties

Recall (§2.1.4) that the map $V_{{\mathcal{P}}}$ induced by Verschiebung maps ${\mathcal{P}}$ to ${\mathcal{Q}}^{(p)}$ , hence its kernel at any point of $S$ is at least $(n-m)$ -dimensional. Over $S^{\text{ord}}$ , but not only there, $V_{{\mathcal{P}}}$ maps ${\mathcal{P}}$ onto ${\mathcal{Q}}^{(p)}$ , so the kernel ${\mathcal{P}}[V]$ is precisely of dimension $n-m$ .

Define a moduli problem $S^{\sharp }$ on $\unicode[STIX]{x1D705}$ -algebras $R$ by setting

$$\begin{eqnarray}S^{\sharp }(R)=\{(\text{}\underline{A},{\mathcal{P}}_{0})\mid \text{}\underline{A}\in S(R),{\mathcal{P}}_{0}\subset {\mathcal{P}}[V]\text{ a subbundle of rank }n-m\}.\end{eqnarray}$$

There is a forgetful map $f:S^{\sharp }\rightarrow S$ , which is bijective over $S^{\text{ord}}$ . Let $\text{Gr}(n-m,{\mathcal{P}})$ be the relative Grassmanian over $S$ classifying sub-bundles ${\mathcal{N}}$ of rank $n-m$ in ${\mathcal{P}}$ . It is a smooth scheme over $S$ , of relative dimension $(n-m)m$ . As the condition $V({\mathcal{N}})=0$ is closed, the moduli problem $S^{\sharp }$ is representable by a closed subscheme of $\text{Gr}(n-m,{\mathcal{P}})$ . The fiber $S_{x}^{\sharp }=f^{-1}(x)$ is the Grassmanian of $(n-m)$ -dimensional subspaces in ${\mathcal{P}}_{x}[V]$ , and if $x\in S_{w}$ its dimension is, in the notation of § 2.1.3,

$$\begin{eqnarray}\dim S_{x}^{\sharp }=(n-m)(a_{\unicode[STIX]{x1D6F4}}(w)-n+m).\end{eqnarray}$$

Denote by $S_{\sharp }$ the open subset of $S$ where $f$ is an isomorphism, i.e. where $a_{\unicode[STIX]{x1D6F4}}(w)=n-m$ . It is a union of EO strata, containing $S^{\text{ord}}$ .

For any $(n,m)$ -shuffle $w$ denote the pre-image of the EO stratum $S_{w}$ by

$$\begin{eqnarray}S_{w}^{\sharp }=f^{-1}(S_{w}).\end{eqnarray}$$

Proof. Using the labeling of the EO strata by the set $\unicode[STIX]{x1D6F1}(n,m)$ of $(n,m)$ -shuffles in $\mathfrak{S}_{n+m}$ , and formula (2.3), we see that $a_{\unicode[STIX]{x1D6F4}}(w)=n-m$ if and only if

$$\begin{eqnarray}w^{-1}(n-m+j)=n+j\end{eqnarray}$$

for all $1\leqslant j\leqslant m$ . Thus, the set of $w$ satisfying $a_{\unicode[STIX]{x1D6F4}}(w)=n-m$ is in bijection with $\unicode[STIX]{x1D6F1}(n-m,m)$ , the set of $(n-m,m)$ -shuffles in $\mathfrak{S}_{n}$ . More precisely, we have to arrange the numbers

$$\begin{eqnarray}\{1,\ldots ,n-m;n+1,\ldots ,n+m\}\end{eqnarray}$$

in the interval $[1,n]$ , preserving the order within each block. There are $\binom{n}{m}$ such shuffles.

Let $\unicode[STIX]{x1D704}:\unicode[STIX]{x1D6F1}(n-m,m){\hookrightarrow}\unicode[STIX]{x1D6F1}(n,m)$ be the inclusion described above. The element

$$\begin{eqnarray}w_{\text{fol}}=\left(\begin{array}{@{}ccccccccc@{}}1 & \cdots \, & n-m & n-m+1 & \cdots \, & n & n+1 & \cdots \, & n+m\\ 1 & \cdots \, & n-m & n+1 & \cdots \, & n+m & n-m+1 & \cdots \, & n\end{array}\right)\end{eqnarray}$$

belongs to $\unicode[STIX]{x1D704}(\unicode[STIX]{x1D6F1}(n-m,m))$ and is the unique minimal element there in the usual Bruhat order. From the remark at the end of § 2.1.3 we deduce that it is also the unique minimal element among $\unicode[STIX]{x1D704}(\unicode[STIX]{x1D6F1}(n-m,m))$ in the EO order $\preccurlyeq$ . This $w_{\text{fol}}$ must represent a stratum $S_{\text{fol}}=S_{w_{\text{fol}}}$ of minimal dimension among the EO strata in $S_{\sharp }$ . Since $\dim S_{w}=l(w)$ we conclude that its dimension is $l(w_{\text{fol}})=m^{2}$ .◻

In Figure 4.1, taken from [Reference WoodingWoo16], we illustrate the EO stratification when $(n,m)=(4,2)$ . There are 15 EO strata altogether, labeled by $(4,2)$ -shuffles $w$ . We write the $(4,2)$ -shuffle $w$ as $w(1)\cdots w(6)$ . The strata are arranged from top to bottom in rows, according to their dimension (equal to the length of $w$ ). The top row contains only $S^{\text{ord}}$ , whose dimension is 8, and the bottom row contains only the core stratum in dimension 0. The EO order relation is represented by downward lines. The 6 strata in $S_{\sharp }$ are those in which $w$ ends with $(\ldots 34)$ . The lowest one, $S_{\text{fol}}$ , has dimension 4. Note that $S_{\sharp }$ contains two 6-dimensional strata.

By construction, $S^{\sharp }$ carries a tautological sub-bundle

$$\begin{eqnarray}{\mathcal{P}}_{0}\subset f^{\ast }{\mathcal{P}}\end{eqnarray}$$

of rank $n-m$ , which extends the sub-bundle ${\mathcal{P}}_{0}$ defined on $S^{\text{ord}}$ . As long as we are above $S_{\sharp }$ it can be viewed as a bundle on $S$ .

4.1.2 Example: the case of $U(n,1)$

This case is particularly simple. There are $n+1$ EO strata, and all of them, except for the core points, lie in $S_{\sharp }$ . The fibers of $S^{\sharp }$ at the core points are projective spaces of dimension $n-1$ . In fact, at such a point $x\in S$ we have ${\mathcal{P}}_{x}[V]={\mathcal{P}}_{x}$ , because ${\mathcal{A}}_{x}$ is superspecial, so there is a canonical identification

$$\begin{eqnarray}S_{x}^{\sharp }=\text{Gr}(n-1,{\mathcal{P}}_{x})=\text{Gr}(n-1,{\mathcal{P}}_{x}\otimes {\mathcal{Q}}_{x})\end{eqnarray}$$

because ${\mathcal{Q}}$ is a line bundle. But under the Kodaira–Spencer map ${\mathcal{P}}_{x}\otimes {\mathcal{Q}}_{x}$ is identified with the cotangent space of $S$ at $x$ , so by duality we have a canonical identification of $S_{x}^{\sharp }$ with $\text{Gr}(1,{\mathcal{T}}S_{x})$ . In fact, $S^{\sharp }$ is the blow-up of $S$ at the core points.

4.1.3 Smoothness and irreducibility

Proof. We work over an algebraically closed field $k$ containing $\unicode[STIX]{x1D705}$ . Let $y\in S^{\sharp }(k)$ and $x=f(y)$ . Let $k[\unicode[STIX]{x1D716}]$ be the ring of dual numbers. Denote by $S^{\sharp }(k[\unicode[STIX]{x1D716}])_{y}$ the tangent space at $y$ to the moduli problem $S^{\sharp }$ . This is the set of elements in $S^{\sharp }(k[\unicode[STIX]{x1D716}])$ mapping to $y$ modulo $\unicode[STIX]{x1D716}$ , equipped with the natural structure of a $k$ -vector space. We shall show that

(4.1) $$\begin{eqnarray}\dim S^{\sharp }(k[\unicode[STIX]{x1D716}])_{y}=nm.\end{eqnarray}$$

Let us first see how this implies the theorem. Let $Y$ be the Zariski closure of an irreducible component of $S^{\text{ord}}$ in $S^{\sharp }$ . It is $nm$ dimensional, hence (4.1), applied to $y\in Y$ , shows that $Y$ is non-singular, and any other irreducible component of $S^{\sharp }$ is disjoint from $Y$ . Since the fibers of $f$ are connected, there do not exist any other irreducible components in $f^{-1}(f(Y))$ . It remains to prove (4.1).

Figure 4.1. The EO strata for $(n,m)=(4,2)$ .

Standard techniques in deformation theory show that we have to compute the tangent space to a certain incidence variety between Grassmanians (see [Reference HarrisHar92, Example 16.2]). We introduce the following notation:

$$\begin{eqnarray}W=\unicode[STIX]{x1D714}_{{\mathcal{A}}_{x}/k}\subset H=H_{\text{dR}}^{1}({\mathcal{A}}_{x}),\quad H_{0}=H_{\text{dR}}^{1}({\mathcal{A}}_{x})[V]\end{eqnarray}$$

and

$$\begin{eqnarray}P_{0}={\mathcal{P}}_{0,y}\subset H_{0}\cap W.\end{eqnarray}$$

These are $k$ -vector spaces with $\unicode[STIX]{x1D705}$ action. The polarization pairing $\{\,.\,,\,.\,\}_{\unicode[STIX]{x1D719}}:H\times H\rightarrow k$ induces a perfect pairing

$$\begin{eqnarray}\{\,.\,,\,.\,\}_{\unicode[STIX]{x1D719}}:W\times H/W\rightarrow k,\end{eqnarray}$$

satisfying $\{\unicode[STIX]{x1D704}(a)u,v\}_{\unicode[STIX]{x1D719}}=\{u,\unicode[STIX]{x1D704}(\overline{a})v\}_{\unicode[STIX]{x1D719}}$ . We claim that $S^{\sharp }(k[\unicode[STIX]{x1D716}])_{y}$ is identified with

$$\begin{eqnarray}\{(\unicode[STIX]{x1D711},\unicode[STIX]{x1D713})\mid \unicode[STIX]{x1D711}\in \operatorname{Hom}_{\unicode[STIX]{x1D705}}(W,H/W)^{\text{sym}},\unicode[STIX]{x1D713}\in \operatorname{Hom}_{\unicode[STIX]{x1D705}}(P_{0},H_{0}/P_{0}),\unicode[STIX]{x1D711}|_{P_{0}}=\unicode[STIX]{x1D713}\;\text{mod}\;W\}.\end{eqnarray}$$

Indeed, by Grothendieck’s crystalline deformation theory [Reference GrothendieckGro74], $S(k[\unicode[STIX]{x1D716}])_{x}$ is identified with $\operatorname{Hom}_{\unicode[STIX]{x1D705}}(W,H/W)^{\text{sym}}$ . The superscript $sym$ refers to homomorphisms symmetric with respect to $\{.\,,\,.\}_{\unicode[STIX]{x1D719}}$ , i.e. satisfying $\{w,\unicode[STIX]{x1D711}(w^{\prime })\}_{\unicode[STIX]{x1D719}}=\{w^{\prime },\unicode[STIX]{x1D711}(w)\}_{\unicode[STIX]{x1D719}}$ for all $w,w^{\prime }\in W$ .

The space $\operatorname{Hom}_{\unicode[STIX]{x1D705}}(P_{0},H_{0}/P_{0})$ classifies infinitesimal deformations of $P_{0}$ preserving the type $\unicode[STIX]{x1D6F4}$ and the property of being killed by $V$ . This is because under the canonical identification

$$\begin{eqnarray}H_{\text{cris}}^{1}({\mathcal{A}}_{x})_{\operatorname{Spec}(k){\hookrightarrow}\operatorname{Spec}(k[\unicode[STIX]{x1D716}])}=H_{\text{dR}}^{1}({\mathcal{A}}_{x})\otimes _{k}k[\unicode[STIX]{x1D716}]\end{eqnarray}$$

the map induced on the left-hand side by functoriality from $\text{Ver}:{\mathcal{A}}_{x}^{(p)}\rightarrow {\mathcal{A}}_{x}$ is $V_{\text{cris}}=V_{\text{dR}}\otimes 1$ .

Finally, the condition $\unicode[STIX]{x1D711}|_{P_{0}}=\unicode[STIX]{x1D713}\;\text{mod}\;W$ means that the infinitesimal deformation of $P_{0}$ stays in the Hodge filtration.

Our problem is now reduced to linear algebra. Note first that

$$\begin{eqnarray}\unicode[STIX]{x1D711}\in \operatorname{Hom}_{\unicode[STIX]{x1D705}}(W,H/W)^{\text{sym}}=\operatorname{Hom}(P,H(\unicode[STIX]{x1D6F4})/P),\end{eqnarray}$$

where we have written $P=W(\unicode[STIX]{x1D6F4})$ , the symmetry condition with respect to the pairing $\{.\,,\,.\}_{\unicode[STIX]{x1D719}}$ then determining uniquely the component in $\operatorname{Hom}(Q,H(\overline{\unicode[STIX]{x1D6F4}})/Q)$ , where $Q=W(\overline{\unicode[STIX]{x1D6F4}})$ . Likewise,

$$\begin{eqnarray}\unicode[STIX]{x1D713}\in \operatorname{Hom}_{\unicode[STIX]{x1D705}}(P_{0},H_{0}/P_{0})=\operatorname{Hom}(P_{0},H_{0}(\unicode[STIX]{x1D6F4})/P_{0}).\end{eqnarray}$$

The dimension of $H_{0}(\unicode[STIX]{x1D6F4})$ is $n$ . Indeed, $H_{0}=H[V]$ is the image of the map $F:H^{(p)}\rightarrow H$ , whose kernel is $W^{(p)}$ . As $H$ itself is balanced (of type $(m+n,m+n)$ ), and $H^{(p)}[F]=W^{(p)}$ is of type $(m,n)$ ,

$$\begin{eqnarray}H_{0}=H[V]\simeq H^{(p)}/W^{(p)}\end{eqnarray}$$

is of type $(n,m)$ . Thus, $\unicode[STIX]{x1D713}$ varies in a space of dimension $(n-m)m$ .

Given $\unicode[STIX]{x1D713}$ , $\unicode[STIX]{x1D711}|_{P_{0}}$ is determined, and by this we take care of the constraint $\unicode[STIX]{x1D711}|_{P_{0}}=\unicode[STIX]{x1D713}\;\text{mod}\;W$ . It remains to extend $\unicode[STIX]{x1D711}$ from $P_{0}$ to $P$ . As the codimension of $P_{0}$ in $P$ is $m$ and the dimension of $H(\unicode[STIX]{x1D6F4})/P$ is $(n+m)-n=m$ , this adds $m^{2}$ dimensions to the tangent space. Altogether

$$\begin{eqnarray}\dim S^{\sharp }(k[\unicode[STIX]{x1D716}])_{y}=(n-m)m+m^{2}=nm\end{eqnarray}$$

as desired. ◻