Introduction

A fundamental problem asks for a formula for the cohomology (or Chow) class of a degeneracy locus, as a polynomial in the Chern classes of the vector bundles involved. In its simplest form, the answer is given by the Giambelli–Thom–Porteous formula: the locus is where two subbundles of a given vector bundle meet in at least a given dimension, and the formula is a determinant.

The aim of this article is to prove formulas for certain degeneracy loci in classical types. One has maps of vector bundles, or flags of subbundles of a given bundle; degeneracy loci come from imposing conditions on the ranks of maps, or dimensions of intersections. The particular loci we consider are indexed by triples of $s$ -tuples of integers, $\boldsymbol{\unicode[STIX]{x1D70F}}=(\mathbf{r},\mathbf{p},\mathbf{q})$ (in type A) or $\boldsymbol{\unicode[STIX]{x1D70F}}=(\mathbf{k},\mathbf{p},\mathbf{q})$ (in types B, C, and D). In type A, each $(r_{i},p_{i},q_{i})$ specifies a rank condition on maps of vector bundles $\operatorname{rk}(E_{p_{i}}\rightarrow F_{q_{i}})\leqslant r_{i}$ ; in other types, $E_{\bullet }$ and $F_{\bullet }$ are flags of isotropic or coisotropic bundles inside some vector bundle with bilinear form, and each $(k_{i},p_{i},q_{i})$ specifies $\dim (E_{p_{i}}\cap F_{q_{i}})\geqslant k_{i}$ .

In each case, we will write $\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}\subseteq X$ for the degeneracy locus. Its expected codimension depends on the type. In fact, to each triple we associate a partition $\unicode[STIX]{x1D706}(\boldsymbol{\unicode[STIX]{x1D70F}})$ (again depending on type), whose size is equal to the codimension of $\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}$ .

The resulting degeneracy loci of type A have a determinantal formula which generalizes that of Giambelli–Thom–Porteous. The loci corresponding to triples are exactly those defined by vexillary permutations according to the recipe of [Reference FultonFul92]; building on work of Kempf and Laksov, Lascoux and Schützenberger, and others, it was shown in [Reference FultonFul92] that

$$\begin{eqnarray}[\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}]=\unicode[STIX]{x1D6E5}_{\unicode[STIX]{x1D706}(\boldsymbol{\unicode[STIX]{x1D70F}})}(c(1),\ldots ,c(\ell )):=\det (c(i)_{\unicode[STIX]{x1D706}_{i}+j-i})_{1\leqslant i,j\leqslant \ell }.\end{eqnarray}$$

Here each $c(k)$ is a (total) Chern class $c(F_{q_{i}}-E_{p_{i}})=c(F_{q_{i}})/c(E_{p_{i}})$ , and $\unicode[STIX]{x1D6E5}_{\unicode[STIX]{x1D706}}$ is a Schur determinant; more details will be given in §1.

In other classical types, work of Pragacz and his collaborators showed that Pfaffians should play a role analogous to determinants in type A, at least for cases where there is a single bundle $E$ and all $F_{\bullet }$ are isotropic [Reference PragaczPra88, Reference PragaczPra91, Reference Pragacz and RatajskiPR97, Reference Lascoux and PragaczLP00]. More recent work of Buch, Kresch, and Tamvakis exploits a crucial insight: both determinants and Pfaffians can be defined via raising operators, and by adopting the raising operator point of view, one can define theta- and eta-polynomials, which interpolate between determinants and Pfaffians. These provide representatives for Schubert classes in nonmaximal isotropic Grassmannians; here one has a single isotropic $E$ , and a flag of trivial bundles $F_{\bullet }$ , some of which may be coisotropic [Reference Buch, Kresch and TamvakisBKT17, Reference Buch, Kresch and TamvakisBKT15, Reference TamvakisTam14]. Wilson extended this idea to define double theta-polynomials, and conjectured that they represent equivariant Schubert classes in nonmaximal isotropic Grassmannians [Reference WilsonWil10]. This was proved in [Reference Ikeda and MatsumuraIM15], and, via a different method, in [Reference Tamvakis and WilsonTW16].

We will introduce triples $\boldsymbol{\unicode[STIX]{x1D70F}}$ for each classical type, and study degeneracy loci defined by $\dim (E_{p_{i}}\cap F_{q_{i}})\geqslant k_{i}$ , with all $E_{\bullet }$ isotropic, and $F_{\bullet }$ either isotropic or coisotropic. When the $F_{\bullet }$ are all isotropic, the formulas are (multi-)Pfaffians (as in the preprints [Reference KazarianKaz00] and [Reference Anderson and FultonAF12]); allowing coisotropic conditions presents some subtleties, but leads directly to the definitions of multi-theta polynomials $\unicode[STIX]{x1D6E9}_{\unicode[STIX]{x1D706}}$ and multi-eta-polynomials $\text{H}_{\unicode[STIX]{x1D706}}$ . Our main theorem is stated in terms of these polynomials.

The entries $c(i)$ vary by type, and along with the definitions of $\unicode[STIX]{x1D6E9}_{\unicode[STIX]{x1D706}}$ and $\text{H}_{\unicode[STIX]{x1D706}}$ , these are specified in Theorems 2, 3, and 4. For now, let us mention some special cases. When the triple has all $q_{i}\geqslant 0$ , the loci are defined by conditions on isotropic bundles, and each formula is a Pfaffian. If the triple has all $p_{i}=p$ , the loci come from Grassmannians; in the type C case, we recover the formulas of [Reference Ikeda and MatsumuraIM15] and [Reference Tamvakis and WilsonTW16], and in case the $F$ are trivial, we recover the formulas of [Reference Buch, Kresch and TamvakisBKT17, Reference Buch, Kresch and TamvakisBKT15]. (The general type D formula includes a definition of double eta-polynomial, which is new even in the Grassmannian case.Footnote ¹ )

The structure of the argument in each type is essentially the same. First, one has a basic formula for the case where the only condition is $E_{p_{1}}\subseteq F_{q_{1}}$ , and furthermore $E_{p_{1}}$ is a line bundle. Next, there is the case where $E_{p_{i}}$ has rank $i$ , and the conditions are $E_{p_{i}}\subseteq F_{q_{i}}$ ; the formula here is easily seen to be a product, and one uses some elementary algebra to convert the product into a raising operator formula. (In type A, these loci correspond to dominant permutations.) The ‘main case’ is where the conditions are $\dim (E_{p_{i}}\cap F_{q_{i}})\geqslant i$ ; such loci are resolved by a birational map from a dominant locus, and the pushforward can be decomposed into a series of projective bundles. Finally, a little more elementary algebra reduces the general case to the main case.

Proving the theorem requires only a few general facts. Some of these are treated in the appendices, but we collect four basic formulas here for reference. Let $E$ be a vector bundle of rank $e$ on a variety $X$ .

1. (a) If $L$ is a line bundle on $X$ , then

   $$\begin{eqnarray}c_{e}(E-L)=c_{e}(E\otimes L^{\ast }).\end{eqnarray}$$

2. (b) If $L$ is a line bundle on $X$ , for any $b\geqslant 0$ and $j\geqslant e$ we have

   $$\begin{eqnarray}(-c_{1}(L))^{b}\,c_{j}(E-L)=c_{b+j}(E-L).\end{eqnarray}$$

3. (c) If $F^{\prime }$ is a subbundle of a vector bundle $F$ , then

   $$\begin{eqnarray}c(E-F/F^{\prime })=c(E-F)\,c(F^{\prime }).\end{eqnarray}$$

4. (d) Let $\unicode[STIX]{x1D70B}:\mathbb{P}(E)\rightarrow X$ be the projective bundle, and $Q=\unicode[STIX]{x1D70B}^{\ast }E/{\mathcal{O}}(-1)$ the universal quotient bundle. For any $\unicode[STIX]{x1D70E}\in A^{\ast }X$ , we have

   $$\begin{eqnarray}\unicode[STIX]{x1D70B}_{\ast }(\unicode[STIX]{x1D70B}^{\ast }\unicode[STIX]{x1D70E}\cdot c_{j}(Q))=\left\{\begin{array}{@{}ll@{}}\unicode[STIX]{x1D70E}\quad & \text{if }j=e-1,\\ 0\quad & \text{otherwise.}\end{array}\right.\end{eqnarray}$$

(Identities (a)–(c) are easy to deduce from the Whitney sum formula, and (d) follows from the formula for $A^{\ast }\mathbb{P}(E)$ as an algebra over $A^{\ast }X$ .)

We conclude this introduction with some remarks on the development and context of our results. The double Schubert polynomials of Lascoux and Schützenberger, which represent type A degeneracy loci, have many wonderful combinatorial properties. A problem that received a great deal of attention in the 1990s was to find similar polynomials representing loci of other classical types. First steps in this direction were taken by Billey and Haiman, who defined (single) Schubert polynomials for types B, C, and D [Reference Billey and HaimanBH95]; double versions were obtained by Ikeda et al. [Reference Ikeda, Mihalcea and NaruseIMN11] and studied further by Tamvakis [Reference TamvakisTam14]. A simplified development of these double Schubert polynomials follows from the degeneracy locus formulas proved here. We should point out that in types B, C, and D, any theory of Schubert polynomials involves working not in a polynomial ring, but in a ring with relations; modulo these relations, however, stable formulas are essentially unique. See [Reference Anderson and FultonAF12], or the survey [Reference TamvakisTam16a], for more perspective on this history.

The Schubert varieties and degeneracy loci in types B, C, and D are indexed by signed permutations, and it is natural to ask whether certain signed permutations correspond to Pfaffians, by analogy with the determinantal formulas for vexillary permutations in type A. In the preprint [Reference Anderson and FultonAF12], we identified such a class of vexillary signed permutations,Footnote ² defined via triples $\boldsymbol{\unicode[STIX]{x1D70F}}$ such that all $q_{i}\geqslant 0$ . Following ideas of Kazarian [Reference KazarianKaz00], we proved Pfaffian formulas for vexillary loci, and also studied the relationship between these Pfaffians and the double Schubert polynomials of [Reference Ikeda, Mihalcea and NaruseIMN11]. We plan to revisit the combinatorics and algebra of Schubert polynomials in separate work.

In the present work, we focus on the Pfaffian formulas and their generalizations. The setup is heavily influenced by Kazarian’s preprint [Reference KazarianKaz00]. We wish to emphasize that our key contribution is the argument itself: by including the more general vexillary loci, we separate algebra (showing a product equals a determinant or Pfaffian, in the ‘dominant’ case of the proof) from geometry (constructing a resolution of singularities and pushing forward the formula, in the ‘main’ case).

In revisiting our earlier approach, we found the situation was clarified by making explicit the role of raising operators. For this, we owe a great debt to the work of Buch, Kresch and Tamvakis, whose remarkable theta- and eta-polynomial formulas have convincingly demonstrated the utility of raising operators in geometry. This inspired us to apply our geometric method to more general loci, simultaneously generalizing their formulas and yielding shorter and more uniform proofs.

Finally, although the recent prominence of raising operators is due to Buch, Kresch and Tamvakis, it was Pragacz who first brought them to geometry. We take both combinatorial and geometric inspiration from his work, and dedicate this article to him on the occasion of his sixtieth birthday.

1 Type A revisited

The determinantal formula describing degeneracy loci in Grassmann bundles, or (more generally) vexillary loci in flag bundles, is, by now, quite well known; see [Reference Kempf and LaksovKL74, Reference FultonFul92] for recent versions. However, our reformulation of its setup and proof will provide a model for the (new) formulas in other types, so we will go through it in detail.

A triple of type A is the data $\boldsymbol{\unicode[STIX]{x1D70F}}=(\mathbf{r},\mathbf{p},\mathbf{q})$ , where each of $\mathbf{r}$ , $\mathbf{p}$ , and $\mathbf{q}$ is an $s$ -tuple of nonnegative integers, with

$$\begin{eqnarray}\displaystyle 0 & {<} & \displaystyle p_{1}\leqslant p_{2}\leqslant \cdots \leqslant p_{s}\quad \text{and}\nonumber\\ \displaystyle & & \displaystyle q_{1}\geqslant q_{2}\geqslant \cdots \geqslant q_{s}>0.\nonumber\end{eqnarray}$$

Setting $k_{i}=p_{i}-r_{i}$ and $l_{i}=q_{i}-r_{i}$ , we further require that

$$\begin{eqnarray}\displaystyle 0 & {<} & \displaystyle k_{1}<k_{2}<\cdots <k_{s}\quad \text{and}\nonumber\\ \displaystyle & & \displaystyle l_{1}\geqslant l_{2}\geqslant \cdots \geqslant l_{s}\geqslant 0.\nonumber\end{eqnarray}$$

The last condition is equivalent to requiring

(a) $$\begin{eqnarray}k_{i}-k_{i-1}\leqslant (q_{i}-q_{i-1})-(p_{i}-p_{i-1})\end{eqnarray}$$

for all $i$ .

Associated to a triple there is a partition $\unicode[STIX]{x1D706}=\unicode[STIX]{x1D706}(\boldsymbol{\unicode[STIX]{x1D70F}})$ , defined by setting $\unicode[STIX]{x1D706}_{k_{i}}=l_{i}$ , and filling in the remaining parts minimally so that $\unicode[STIX]{x1D706}_{1}\geqslant \unicode[STIX]{x1D706}_{2}\geqslant \cdots \geqslant \unicode[STIX]{x1D706}_{k_{s}}\geqslant 0$ . (An essential triple specifies only the ‘corners’ of the Young diagram for $\unicode[STIX]{x1D706}$ , so in a sense it is a minimal way of packaging this information.)

Given a partition $\unicode[STIX]{x1D706}=(\unicode[STIX]{x1D706}_{1}\geqslant \cdots \geqslant \unicode[STIX]{x1D706}_{\ell }\geqslant 0)$ and symbols $c(1),\ldots ,c(\ell )$ , the associated Schur determinant is

$$\begin{eqnarray}\unicode[STIX]{x1D6E5}_{\unicode[STIX]{x1D706}}(c(1),\ldots ,c(\ell )):=\det (c(i)_{\unicode[STIX]{x1D706}_{i}+j-i})_{1\leqslant i,j\leqslant \ell }.\end{eqnarray}$$

For a positive integer $\ell$ , let $R^{(\ell )}$ be the raising operator

(1) $$\begin{eqnarray}R^{(\ell )}=\mathop{\prod }_{1\leqslant i<j\leqslant \ell }(1-R_{ij}),\end{eqnarray}$$

where $R_{ij}=T_{i}/T_{j}$ is the operator defined in Appendix A.2. A simple application of the Vandermonde identity shows that

(2) $$\begin{eqnarray}\unicode[STIX]{x1D6E5}_{\unicode[STIX]{x1D706}}(c(1),\ldots ,c(\ell ))=R^{(\ell )}\cdot (c(1)_{\unicode[STIX]{x1D706}_{1}}\cdots c(r)_{\unicode[STIX]{x1D706}_{\ell }}),\end{eqnarray}$$

and we will use this observation crucially in proving the degeneracy locus formula.

Here is the geometric setup. On a variety $X$ we have a sequence of vector bundles

$$\begin{eqnarray}E_{p_{1}}{\hookrightarrow}E_{p_{2}}{\hookrightarrow}\cdots {\hookrightarrow}E_{p_{s}}\xrightarrow[{}]{\unicode[STIX]{x1D711}}F_{q_{1}}{\twoheadrightarrow}F_{q_{2}}{\twoheadrightarrow}\cdots {\twoheadrightarrow}F_{q_{s}},\end{eqnarray}$$

where subscripts indicate ranks; for each $i$ , there is an induced map $E_{p_{i}}\rightarrow F_{q_{i}}$ . The degeneracy locus corresponding to the triple $\boldsymbol{\unicode[STIX]{x1D70F}}$ is

$$\begin{eqnarray}\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}:=\{x\in X\,|\,\operatorname{rk}(E_{p_{i}}\rightarrow F_{q_{i}})\leqslant r_{i}\text{ for }1\leqslant i\leqslant s\}.\end{eqnarray}$$

This comes equipped with a natural subscheme structure defined locally by the vanishing of certain determinants.

Let $c(k_{i})=c(F_{q_{i}}-E_{p_{i}})$ , and set $c(k)=c(k_{i})$ whenever $k_{i-1}<k\leqslant k_{i}$ . We also set $\ell =k_{s}$ , and by convention we always take $k_{0}=0$ . With this notation, the degeneracy locus formula can be stated as follows.

The case where there is only one $F$ , so $q_{1}=\cdots =q_{s}$ , was proved by Kempf and Laksov, starting the modern search for such formulas. The general case was proved in [Reference FultonFul92].

These formulas are to be interpreted as usual: when the bundles and the map $\unicode[STIX]{x1D711}$ are sufficiently generic, then $\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}$ has codimension equal to $|\unicode[STIX]{x1D706}|$ and the formula is an identity relating the fundamental class of $\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}$ with the Chern classes of $E$ and $F$ . In general, the left-hand side should be regarded as a refined class of codimension $|\unicode[STIX]{x1D706}|$ , supported on $\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}$ ; see [Reference FultonFul98].

The proof proceeds in four steps.

1.1 Basic case

Assume $s=1$ , $p_{1}=1$ , $r_{1}=0$ , so $\boldsymbol{\unicode[STIX]{x1D70F}}=(0,1,q_{1})$ . The locus is where $E_{1}\rightarrow F_{q_{1}}$ vanishes, so it is the zeroes of a section of $E_{1}^{\ast }\otimes F_{q_{1}}$ , a vector bundle of rank $q_{1}$ . Therefore

$$\begin{eqnarray}[\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}]=c_{q_{1}}(E_{1}^{\ast }\otimes F_{q_{1}})=c_{q_{1}}(F_{q_{1}}-E_{1}),\end{eqnarray}$$

using identity (a) to obtain the second equality.

1.2 Dominant case

Assume $p_{i}=i$ and $r_{i}=0$ for $1\leqslant i\leqslant s$ , so $k_{i}=i$ and $l_{i}=q_{i}$ . By imposing one condition at a time, we obtain a sequence

$$\begin{eqnarray}X=Z_{0}\supseteq Z_{1}\supseteq Z_{2}\supseteq \cdots \supseteq Z_{s}=\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}};\end{eqnarray}$$

$Z_{1}$ is the locus where $E_{1}\rightarrow F_{q_{1}}$ is zero; $Z_{2}$ is where also $E_{2}/E_{1}\rightarrow F_{q_{2}}$ is zero; and generally $Z_{j}$ is defined on $Z_{j-1}$ by the condition that $E_{j}/E_{j-1}\rightarrow F_{q_{j}}$ be zero. This is an instance of the basic case, and using the projection formula, it follows that

(3) $$\begin{eqnarray}[\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}]=\mathop{\prod }_{j=1}^{s}c_{q_{j}}(F_{q_{j}}-E_{j}/E_{j-1}).\end{eqnarray}$$

Writing $c(j)=c(F_{q_{j}}-E_{j})$ and $t_{i}=-c_{1}(E_{i}/E_{i-1})$ , an application of identity (c) transforms (3) into

(4) $$\begin{eqnarray}[\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}]=\mathop{\prod }_{j=1}^{s}\mathop{\biggl[c(j)\cdot \mathop{\prod }_{i=1}^{j-1}(1-t_{i})\biggr]}\nolimits_{q_{j}}.\end{eqnarray}$$

Using identity (b), this becomes

(5) $$\begin{eqnarray}\displaystyle [\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}] & = & \displaystyle \biggl(\mathop{\prod }_{1\leqslant i<j\leqslant s}(1-R_{ij})\biggr)\cdot (c(1)_{q_{1}}c(2)_{q_{2}}\cdots c(s)_{q_{s}})\nonumber\\ \displaystyle & = & \displaystyle R^{(s)}\cdot (c(1)_{q_{1}}c(2)_{q_{2}}\cdots c(s)_{q_{s}}).\end{eqnarray}$$

In other words, the product (3) is equal to the determinant $\unicode[STIX]{x1D6E5}_{\unicode[STIX]{x1D706}}(c(1),\ldots ,c(s))$ , where $\unicode[STIX]{x1D706}=(q_{1}\geqslant q_{2}\geqslant \cdots \geqslant q_{s})$ . (To deduce (5) from (4), use identity (b) and descending induction on $k$ to show (4) equals

$$\begin{eqnarray}\displaystyle \biggl(\mathop{\prod }_{\substack{ 1\leqslant i<j\leqslant s \\ k<j}}(1-R_{ij})\biggr)\cdot c^{\prime }(1)_{q_{1}}\cdots c^{\prime }(k)_{q_{k}}\cdot c(k+1)_{q_{k+1}}\cdots c(s)_{q_{s}}, & & \displaystyle \nonumber\end{eqnarray}$$

where $c^{\prime }(k)=c(F_{q_{k}}-E_{k}/E_{k-1})=c(k)\cdot \prod _{i=1}^{k-1}(1-t_{i})$ . The case $k=s$ is (4), and the case $k=1$ is (5).)

1.3 Main case

Assume $k_{i}=p_{i}-r_{i}=i$ for $1\leqslant i\leqslant s$ . There is a sequence of projective bundles

$$\begin{eqnarray}X=X_{0}\leftarrow X_{1}=\mathbb{P}(E_{p_{1}})\leftarrow X_{2}=\mathbb{P}(E_{p_{2}}/D_{1})\leftarrow \cdots \leftarrow X_{s}=\mathbb{P}(E_{p_{s}}/D_{s-1}),\end{eqnarray}$$

where, suppressing notation for pullbacks of bundles, $D_{j}/D_{j-1}\subseteq E_{p_{j}}/D_{j-1}$ is the tautological line bundle on $X_{j}$ . Let us write $\unicode[STIX]{x1D70B}^{(j)}:X_{j}\rightarrow X_{j-1}$ for the projection, and $\unicode[STIX]{x1D70B}:X_{s}\rightarrow X$ for the composition of all the projections $\unicode[STIX]{x1D70B}^{(j)}$ .

On $X_{s}$ , there is the locus $\widetilde{\unicode[STIX]{x1D6FA}}$ where $D_{i}/D_{i-1}\rightarrow F_{q_{i}}$ is zero for $1\leqslant i\leqslant s$ . This is an instance of the dominant case, so in $A^{\ast }X_{s}$ we have

(6) $$\begin{eqnarray}[\widetilde{\unicode[STIX]{x1D6FA}}]=R^{(s)}\cdot \widetilde{c}(1)_{\widetilde{\unicode[STIX]{x1D706}}_{1}}\cdots \widetilde{c}(s)_{\widetilde{\unicode[STIX]{x1D706}}_{s}},\end{eqnarray}$$

where $\widetilde{\unicode[STIX]{x1D706}}_{j}=q_{j}$ and $\widetilde{c}(j)=c(F_{q_{j}}-D_{j})$ . Furthermore, $\unicode[STIX]{x1D70B}$ maps $\widetilde{\unicode[STIX]{x1D6FA}}$ birationally onto $\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}$ ; it is an isomorphism over the dense open set where $\operatorname{rk}(E_{p_{i}}\rightarrow F_{q_{i}})=r_{i}$ . (Take $D_{i}$ to be the kernel.) So $[\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}]=\unicode[STIX]{x1D70B}_{\ast }[\widetilde{\unicode[STIX]{x1D6FA}}]$ .

To compute this pushforward, use identity (c) to write $\widetilde{c}(j)=c(j)\cdot c(E_{p_{j}}/D_{j})$ , recalling that $c(j)=c(F_{q_{j}}-E_{p_{j}})$ . Note that $E_{p_{j}}/D_{j}$ is the tautological quotient bundle for $\unicode[STIX]{x1D70B}^{(j)}:X_{j}\rightarrow X_{j-1}$ . By identity (d), we have

$$\begin{eqnarray}\unicode[STIX]{x1D70B}_{\ast }^{(j)}(c(j)_{a}\cdot c_{b}(E_{p_{j}}/D_{j}))=c(j)_{a}\end{eqnarray}$$

when $b=p_{j}-j$ , and this pushforward equals $0$ otherwise. Therefore

$$\begin{eqnarray}\unicode[STIX]{x1D70B}_{\ast }^{(j)}(\widetilde{c}(j)_{k})=\unicode[STIX]{x1D70B}_{\ast }^{(j)}\biggl(\mathop{\sum }_{a+b=k}c(j)_{a}\cdot c_{b}(E_{p_{j}}/D_{j})\biggr)=c(j)_{k-p_{j}+j}.\end{eqnarray}$$

Applying $\unicode[STIX]{x1D70B}_{\ast }$ to (6) and using linearity of the raising operator yields

$$\begin{eqnarray}\displaystyle [\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}] & = & \displaystyle \unicode[STIX]{x1D70B}_{\ast }(R^{(s)}\cdot \widetilde{c}(1)_{\widetilde{\unicode[STIX]{x1D706}}_{1}}\cdots \widetilde{c}(s)_{\widetilde{\unicode[STIX]{x1D706}}_{s}})\nonumber\\ \displaystyle & = & \displaystyle R^{(s)}\unicode[STIX]{x1D70B}_{\ast }(\widetilde{c}(1)_{\widetilde{\unicode[STIX]{x1D706}}_{1}}\cdots \widetilde{c}(s)_{\widetilde{\unicode[STIX]{x1D706}}_{s}})\nonumber\\ \displaystyle & = & \displaystyle R^{(s)}c(1)_{\unicode[STIX]{x1D706}_{1}}\cdots c(s)_{\unicode[STIX]{x1D706}_{s}}\nonumber\\ \displaystyle & = & \displaystyle \unicode[STIX]{x1D6E5}_{\unicode[STIX]{x1D706}}(c(1),\ldots ,c(s)).\nonumber\end{eqnarray}$$

2 Type C: symplectic bundles

A triple of type C is $\boldsymbol{\unicode[STIX]{x1D70F}}=(\mathbf{k},\mathbf{p},\mathbf{q})$ , with

$$\begin{eqnarray}\displaystyle 0 & {<} & \displaystyle k_{1}<k_{2}<\cdots <k_{s},\nonumber\\ \displaystyle & & \displaystyle p_{1}\geqslant p_{2}\geqslant \cdots \geqslant p_{s}>0,\nonumber\\ \displaystyle & & \displaystyle q_{1}\geqslant q_{2}\geqslant \cdots \geqslant q_{s}.\nonumber\end{eqnarray}$$

The $q_{i}$ are allowed to be negative, but not zero, and if $p_{s}=1$ then all $q_{i}$ must be positive. Since the difference between positive and negative $q$ plays a major role, let $a=a(\boldsymbol{\unicode[STIX]{x1D70F}})$ be the index such that $q_{a}>0>q_{a+1}$ (allowing $a=0$ and $a=s$ for the cases where all $q_{i}$ are negative or all $q_{i}$ are positive, respectively). We will also require five further conditions, listed as (c1)–(c5) below, which arise naturally from the geometric setup. Consider an even-rank vector bundle $V$ , equipped with a symplectic form and two flags of subbundles

$$\begin{eqnarray}\displaystyle E_{p_{1}} & \subset & \displaystyle E_{p_{2}}\subset \cdots \subset E_{p_{s}}\subset V,\nonumber\\ \displaystyle F_{q_{1}} & \subset & \displaystyle F_{q_{2}}\subset \cdots \subset F_{q_{s}}\subset V.\nonumber\end{eqnarray}$$

When $q>0$ , the subbundles $F_{q}$ are isotropic; when $q<0$ , $F_{q}$ is coisotropic; and all the bundles $E_{p}$ are isotropic. If the rank of $V$ is $2n$ , the isotropic bundles $E_{p}$ and $F_{q}$ (for $q>0$ ) have rank $n+1-p$ and $n+1-q$ , respectively; and for $q<0$ , the coisotropic bundles $F_{q}$ have rank $n-q$ . (So the order on the $p_{i}$ and $q_{i}$ is compatible with the inclusion of bundles of corresponding ranks.)

The degeneracy locus is

$$\begin{eqnarray}\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}=\{x\in X\,|\,\dim (E_{p_{i}}\cap F_{q_{i}})\geqslant k_{i}\text{ for }1\leqslant i\leqslant s\}.\end{eqnarray}$$

Note that $E_{1}^{\bot }=E_{1}$ , so a condition on its intersection with a coisotropic space is equivalent to one for the intersection with an isotropic space; we shall prefer the latter, and this explains why negative $q_{i}$ are prohibited when $p=1$ . Demanding that the rank conditions be feasible, and generically attained with equality, leads to the further requirements on the triple $\boldsymbol{\unicode[STIX]{x1D70F}}$ .

The conditions on $\boldsymbol{\unicode[STIX]{x1D70F}}$ are likely to appear somewhat technical at first. The reader may find it helpful to first assume all $q_{i}$ are positive, which is the simplest case. The next easiest case is when $k_{i}=i$ for all $i$ , which corresponds to the ‘main case’ in the proof. Linear-algebraic reasons for the conditions, as well as combinatorial explanations and the relationship with [Reference Buch, Kresch and TamvakisBKT17], can be found in the remarks at the end of this section.

In what follows, when indices fall out of the range $[1,s]$ , we use conventions so that the inequalities become trivial, e.g., $k_{0}=0$ , $q_{0}=+\infty$ , and $q_{s+1}=-\infty$ .

First, for $i\leqslant a$ , we require

(c1) $$\begin{eqnarray}\displaystyle k_{i}-k_{i-1}\leqslant (p_{i-1}-p_{i})+(q_{i-1}-q_{i}). & & \displaystyle\end{eqnarray}$$

When $k_{i}=i$ for all $i$ , this says that either $p_{i-1}>p_{i}$ or $q_{i-1}>q_{i}$ . When all $q_{i}$ are positive, this is the only condition required of a triple $\boldsymbol{\unicode[STIX]{x1D70F}}$ .

The other requirements on $\boldsymbol{\unicode[STIX]{x1D70F}}$ concern negative $q$ , so they describe intersections of an isotropic $E_{p}$ with a coisotropic $F_{q}$ .

For each $j\leqslant a$ , let $m(j)=\min \{m\,|\,q_{j}+(k_{j}-k_{j-1}-1)\geqslant q_{m}\}$ . The second condition is as follows.

(c2) $$\begin{eqnarray}\displaystyle & & \displaystyle \text{The negative values }\nonumber\\ \displaystyle & & \displaystyle \quad -q_{j},\,-q_{j}-1,\,\ldots ,\,-q_{j}-(k_{j}-k_{m(j)-1}-1)\nonumber\\ \displaystyle & & \displaystyle \text{are all prohibited as values of }q_{i}\text{ for }i>a.\end{eqnarray}$$

(Here is an equivalent, algorithmic condition: let $N_{0}=\{1,2,3,\ldots \}$ ; form $N_{1}$ by removing the $k_{1}$ consecutive elements of $N_{0}$ starting at $q_{1}$ ; then form $N_{2}$ by removing $k_{2}-k_{1}$ consecutive elements of $N_{1}$ , starting at the $q_{2}$ th element; and so on up to $N_{a}$ . The absolute values of $q_{i}$ for $i\geqslant a$ are required to lie in $N_{a}$ .) When $k_{i}=i$ for all $i$ , this simply says that a negative value $q_{i}$ cannot appear if $|q_{i}|$ has already appeared as some $q_{j}$ for $j<i$ .

Next, for each $i>a$ , so $q_{i}<0$ , we define

$$\begin{eqnarray}\unicode[STIX]{x1D70C}_{k_{i}}:=k_{j},\end{eqnarray}$$

where $j$ is the index such that $q_{j}>-q_{i}>q_{j+1}$ , and require that

(c3) $$\begin{eqnarray}k_{i}-\unicode[STIX]{x1D70C}_{k_{i}}\leqslant -q_{i}.\end{eqnarray}$$

When $k_{i}=i$ for all $i$ , $\unicode[STIX]{x1D70C}$ has an easy characterization as $\unicode[STIX]{x1D70C}_{i}=\#\{j<i\,|\,q_{j}>-q_{i}\}$ .

The fourth condition is that

(c4) $$\begin{eqnarray}(k_{i}-k_{i-1})+(\unicode[STIX]{x1D70C}_{k_{i-1}}-\unicode[STIX]{x1D70C}_{k_{i}})\leqslant (p_{i-1}-p_{i})+(q_{i-1}-q_{i})\end{eqnarray}$$

for $i>a+1$ . This is a refinement of the first requirement (c1).

The fifth and final condition is that

(c5) $$\begin{eqnarray}k_{i}\geqslant 1-p_{i}-q_{i}+\unicode[STIX]{x1D70C}_{k_{i}}\end{eqnarray}$$

for $i>a$ . (In conjunction with (c4), it suffices to require (c5) only for $i=s$ .)

As in type A, a triple of type C has an associated partition $\unicode[STIX]{x1D706}=\unicode[STIX]{x1D706}(\boldsymbol{\unicode[STIX]{x1D70F}})$ , defined by

$$\begin{eqnarray}\displaystyle \unicode[STIX]{x1D706}_{k_{i}}=\left\{\begin{array}{@{}ll@{}}p_{i}+q_{i}-1\quad & \text{if }i\leqslant a,\\ p_{i}+q_{i}+k_{i}-1-\unicode[STIX]{x1D70C}_{k_{i}}\quad & \text{if }i>a.\end{array}\right. & & \displaystyle \nonumber\end{eqnarray}$$

The conditions on $\boldsymbol{\unicode[STIX]{x1D70F}}$ imply that

$$\begin{eqnarray}\unicode[STIX]{x1D706}_{k_{1}}>\unicode[STIX]{x1D706}_{k_{2}}>\cdots >\unicode[STIX]{x1D706}_{k_{a}}>\unicode[STIX]{x1D706}_{k_{a+1}}\geqslant \cdots \geqslant \unicode[STIX]{x1D706}_{k_{s}}\geqslant 0.\end{eqnarray}$$

The other parts of $\unicode[STIX]{x1D706}$ are defined by filling in $\unicode[STIX]{x1D706}_{k}$ minimally subject to these inequalities (strict if $k<k_{a}$ , weak if $k>k_{a}$ ).

Generally, given a sequence of nonnegative integers $\unicode[STIX]{x1D70C}=(\unicode[STIX]{x1D70C}_{1},\ldots ,\unicode[STIX]{x1D70C}_{\ell })$ , we will say that a partition $\unicode[STIX]{x1D706}=(\unicode[STIX]{x1D706}_{1}\geqslant \cdots \geqslant \unicode[STIX]{x1D706}_{\ell })$ is $\boldsymbol{\unicode[STIX]{x1D70C}}$ -strict if the sequence $\unicode[STIX]{x1D707}_{j}=\unicode[STIX]{x1D706}_{j}+\unicode[STIX]{x1D70C}_{j}$ is nonincreasing. (See Remark 3 for the relation with the corresponding notion from [Reference Buch, Kresch and TamvakisBKT17].)

From this point of view, it is useful to extend the definition of $\unicode[STIX]{x1D70C}$ given above to a unimodal sequence of integers $\unicode[STIX]{x1D70C}(\boldsymbol{\unicode[STIX]{x1D70F}})=(\unicode[STIX]{x1D70C}_{1},\unicode[STIX]{x1D70C}_{2},\ldots ,\unicode[STIX]{x1D70C}_{k_{s}})$ , as follows. As before, for $i>a$ , define $\unicode[STIX]{x1D70C}_{k_{i}}=k_{j}$ , where $q_{j}>-q_{i}>q_{j+1}$ . For $k\leqslant k_{a}$ , set $\unicode[STIX]{x1D70C}_{k}=k-1$ . Then fill in the remaining entries by setting $\unicode[STIX]{x1D70C}_{k}=\unicode[STIX]{x1D70C}_{k_{i}}$ for $i>a$ and $k_{i-1}<k\leqslant k_{i}$ . This means $\unicode[STIX]{x1D70C}_{k_{a}+1}\geqslant \cdots \geqslant \unicode[STIX]{x1D70C}_{k_{s}}$ .

The conditions (c1) and (c3)–(c5) are equivalent to requiring that

For example, suppose $\boldsymbol{\unicode[STIX]{x1D70F}}=(\,1\,3\,5\,6\,7\,9\,,\;9\,7\,6\,5\,2\,2\,,\;6\,3\,\overline{2}\,\overline{5}\,\overline{7}\,\overline{9}\,)$ , using a bar to indicate negative integers. Here $a=2$ , $\unicode[STIX]{x1D70C}=(0,1,2,3,3,1,0,0,0)$ , and $\unicode[STIX]{x1D706}(\boldsymbol{\unicode[STIX]{x1D70F}})=(14,10,9,5,5,4,1,1,1)$ . (See Figure 1.)

Figure 1. The shape for a $\unicode[STIX]{x1D70C}(\boldsymbol{\unicode[STIX]{x1D70F}})$ -strict partition $\unicode[STIX]{x1D706}(\boldsymbol{\unicode[STIX]{x1D70F}})$ . The boxes of $\unicode[STIX]{x1D70C}$ are shaded.

Given an integer $\ell >0$ , a sequence of nonnegative integers $\unicode[STIX]{x1D70C}=(\unicode[STIX]{x1D70C}_{1},\ldots ,\unicode[STIX]{x1D70C}_{\ell })$ with $\unicode[STIX]{x1D70C}_{j}<j$ , define the raising operator

$$\begin{eqnarray}\displaystyle R^{(\unicode[STIX]{x1D70C},\ell )} & =\biggl(\mathop{\prod }_{1\leqslant i<j\leqslant \ell }(1-R_{ij})\biggr)\biggl(\mathop{\prod }_{1\leqslant i\leqslant \unicode[STIX]{x1D70C}_{j}<j\leqslant \ell }(1+R_{ij})^{-1}\biggr). & \displaystyle \nonumber\end{eqnarray}$$

Inspired by [Reference Buch, Kresch and TamvakisBKT17], given symbols $c(1),\ldots ,c(\ell )$ , and a $\unicode[STIX]{x1D70C}$ -strict partition $\unicode[STIX]{x1D706}$ , we define the theta-polynomial to be

$$\begin{eqnarray}\unicode[STIX]{x1D6E9}_{\unicode[STIX]{x1D706}}^{(\unicode[STIX]{x1D70C})}(c(1),\ldots ,c(\ell ))=R^{(\unicode[STIX]{x1D70C},\ell )}\cdot (c(1)_{\unicode[STIX]{x1D706}_{1}}\cdots c(\ell )_{\unicode[STIX]{x1D706}_{\ell }}).\end{eqnarray}$$

When $\unicode[STIX]{x1D70C}=\emptyset$ , $\unicode[STIX]{x1D6E9}_{\unicode[STIX]{x1D706}}^{(\unicode[STIX]{x1D70C})}$ is a Schur determinant, and when $\unicode[STIX]{x1D70C}_{j}=j-1$ , $\unicode[STIX]{x1D6E9}_{\unicode[STIX]{x1D706}}^{(\unicode[STIX]{x1D70C})}$ is a Schur Pfaffian.

For a triple and the corresponding geometry described above, let $c(k_{i})=c(V-E_{p_{i}}-F_{q_{i}})$ , and for general $k$ , take $c(k)=c(k_{i})$ where $i$ is minimal such that $k_{i}\geqslant k$ . Set $\ell =k_{s}$ .

The proof follows the same pattern as the one we saw in type A. As before, there are four cases. The appearance of Pfaffians in the formulas can be traced to a basic fact about isotropic subbundles: if $D\subset V$ is isotropic, then the symplectic form identifies $D^{\bot }$ with $(V/D)^{\ast }$ . This is used in the second case.

2.1 Basic case

Take $s=1$ , $k_{1}=\ell =1$ , and $p_{1}=n$ , so $E_{n}$ is a line bundle and we are looking at $\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}=\{x\,|\,E_{n}\subseteq F_{q_{1}}\}$ . Equivalently, $E_{n}\rightarrow V/F_{q_{1}}$ is zero, so identity (a) lets us write

$$\begin{eqnarray}[\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}]=c_{\unicode[STIX]{x1D706}_{1}}(V/F_{q_{1}}\otimes E_{n}^{\ast })=c_{\unicode[STIX]{x1D706}_{1}}(V-F_{q_{1}}-E_{n}),\end{eqnarray}$$

where

$$\begin{eqnarray}\unicode[STIX]{x1D706}_{1}=\operatorname{rk}(V/F_{q_{1}})=\left\{\begin{array}{@{}ll@{}}n+q_{1}-1\quad & \text{ if }q_{1}>0,\\ n+q_{1}\quad & \text{ if }q_{1}<0.\end{array}\right.\end{eqnarray}$$

2.2 Dominant case

Now take $k_{i}=i$ and $p_{i}=n+1-i$ , for $1\leqslant i\leqslant s$ . Write $D_{i}=E_{p_{i}}$ , so this is a vector bundle of rank $i$ , and we have $D_{1}\subset D_{2}\subset \cdots \subset D_{s}\subset V$ . Letting $Z_{j}$ be the locus in $X$ where $D_{i}\subseteq F_{q_{i}}$ for all $i\leqslant j$ , we have

$$\begin{eqnarray}X\supseteq Z_{1}\supseteq \cdots \supseteq Z_{s-1}\supseteq Z_{s}=\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}.\end{eqnarray}$$

On $Z_{j-1}$ , we have $D_{j-1}\subseteq F_{q_{j-1}}\subseteq F_{q_{j}}\subseteq V$ . Since $D_{j}$ is isotropic, we automatically have $D_{j}\subseteq D_{j-1}^{\bot }$ , so $Z_{j}$ is defined by the condition

$$\begin{eqnarray}D_{j}/D_{j-1}\subseteq (F_{q_{j}}\cap D_{j-1}^{\bot })/D_{j-1}\subseteq D_{j-1}^{\bot }/D_{j-1},\end{eqnarray}$$

or equivalently, $D_{j}/D_{j-1}\rightarrow D_{j-1}^{\bot }/(F_{q_{j}}\cap D_{j-1}^{\bot })$ is zero.

When $j\leqslant a$ , the bundle $F_{q_{j}}$ is isotropic, and this implies $F_{q_{j}}\subseteq D_{j-1}^{\bot }$ . In this case, $Z_{j}$ is equivalently defined by $D_{j}/D_{j-1}\subseteq F_{q_{j}}/D_{j-1}\subseteq D_{j-1}^{\bot }/D_{j-1}$ , and the basic case says

$$\begin{eqnarray}\displaystyle [Z_{j}]=[Z_{j-1}]\cdot c_{\unicode[STIX]{x1D706}_{j}}(D_{j-1}^{\bot }/D_{j-1}-F_{q_{j}}/D_{j-1}-D_{j}/D_{j-1}), & & \displaystyle \nonumber\end{eqnarray}$$

with $\unicode[STIX]{x1D706}_{j}=q_{j}+p_{j}-1=q_{j}+n-j$ . (That is, $\unicode[STIX]{x1D706}_{j}=\operatorname{rk}(D_{j-1}^{\bot }/F_{q_{j}})$ .)

When $j>a$ (so $q_{j}<0$ ), we have

$$\begin{eqnarray}D_{j-1}^{\bot }/(F_{q_{j}}\cap D_{j-1}^{\bot })=(D_{j-1}\cap F_{q_{j}}^{\bot })^{\bot }/F_{q_{j}}=D_{\unicode[STIX]{x1D70C}_{j}}^{\bot }/F_{q_{j}}\end{eqnarray}$$

by property (∗) from Remark 1. (Recall that $\unicode[STIX]{x1D70C}_{j}=i$ , where $q_{i}>-q_{j}>q_{i+1}$ .) The basic case says

$$\begin{eqnarray}\displaystyle [Z_{j}]=[Z_{j-1}]\cdot c_{\unicode[STIX]{x1D706}_{j}}(D_{\unicode[STIX]{x1D70C}_{j}}^{\bot }/D_{j-1}-F_{q_{j}}/D_{j-1}-D_{j}/D_{j-1}), & & \displaystyle \nonumber\end{eqnarray}$$

with $\unicode[STIX]{x1D706}_{j}=n+q_{j}-i=p_{j}+q_{j}+j-1-\unicode[STIX]{x1D70C}_{j}$ . (That is, $\unicode[STIX]{x1D706}_{j}=\operatorname{rk}(D_{\unicode[STIX]{x1D70C}_{j}}^{\bot }/F_{q_{j}})$ .)

With $\unicode[STIX]{x1D70C}_{j}=j-1$ for $1\leqslant j\leqslant a$ , it follows that

$$\begin{eqnarray}\displaystyle [\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}]=\biggl(\mathop{\prod }_{j=1}^{s}c_{\unicode[STIX]{x1D706}_{j}}(D_{\unicode[STIX]{x1D70C}_{j}}^{\bot }/D_{j-1}-F_{q_{j}}/D_{j-1}-D_{j}/D_{j-1})\biggr). & & \displaystyle \nonumber\end{eqnarray}$$

Using the symplectic form to identify $D_{\unicode[STIX]{x1D70C}_{j}}^{\bot }$ with $(V/D_{\unicode[STIX]{x1D70C}_{j}})^{\ast }$ , we have

$$\begin{eqnarray}c(D_{\unicode[STIX]{x1D70C}_{j}}^{\bot }/D_{j-1}-F_{q_{j}}/D_{j-1}-D_{j}/D_{j-1})=c(V-D_{j}-F_{q_{j}})\cdot c(D_{j-1}-D_{\unicode[STIX]{x1D70C}_{j}}^{\ast });\end{eqnarray}$$

setting $t_{j}=-c_{1}(D_{j}/D_{j-1})$ and applying identity (c), this becomes

(7) $$\begin{eqnarray}\displaystyle [\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}]=\mathop{\prod }_{j=1}^{s}\mathop{\biggl[c(V-D_{j}-F_{q_{j}})\frac{\mathop{\prod }_{i=1}^{j-1}(1-t_{i})}{\mathop{\prod }_{i=1}^{\unicode[STIX]{x1D70C}_{j}}(1+t_{i})}\biggr]}\nolimits_{\unicode[STIX]{x1D706}_{j}}. & & \displaystyle\end{eqnarray}$$

Setting $c(j)=c(V-D_{j}-F_{q_{j}})=c(V-E_{p_{j}}-F_{q_{j}})$ and applying identity (b), this is

(8) $$\begin{eqnarray}[\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}]=R^{(\unicode[STIX]{x1D70C},s)}c(1)_{\unicode[STIX]{x1D706}_{1}}\cdots c(s)_{\unicode[STIX]{x1D706}_{s}}.\end{eqnarray}$$

2.3 Main case

Here we only assume $k_{i}=i$ , for $1\leqslant i\leqslant s$ . For $j\leqslant a$ , we set $\unicode[STIX]{x1D70C}_{j}=j-1$ , and for $j>a$ , we have $\unicode[STIX]{x1D70C}_{j}=i$ , where $q_{i}>-q_{j}>q_{i+1}$ . We have the same sequence of projective bundles as in type A,

$$\begin{eqnarray}X=X_{0}\leftarrow X_{1}=\mathbb{P}(E_{p_{1}})\leftarrow X_{2}=\mathbb{P}(E_{p_{2}}/D_{1})\leftarrow \cdots \leftarrow X_{s}=\mathbb{P}(E_{p_{s}}/D_{s-1});\end{eqnarray}$$

again, $D_{j}/D_{j-1}\subseteq E_{p_{j}}/D_{j-1}$ is the tautological subbundle on $X_{j}$ . Write $\unicode[STIX]{x1D70B}^{(j)}:X_{j}\rightarrow X_{j-1}$ for the projection, and $\unicode[STIX]{x1D70B}:X_{s}\rightarrow X$ for the composition.

On $X_{s}$ , we have the locus $\widetilde{\unicode[STIX]{x1D6FA}}=\{D_{i}\subseteq F_{q_{i}}\,|\,\text{for }1\leqslant i\leqslant s\}$ . By the previous case, as a class in $A^{\ast }X_{s}$ we have

$$\begin{eqnarray}[\widetilde{\unicode[STIX]{x1D6FA}}]=R^{(\unicode[STIX]{x1D70C},s)}\widetilde{c}(1)_{\widetilde{\unicode[STIX]{x1D706}}_{1}}\cdots \widetilde{c}(s)_{\widetilde{\unicode[STIX]{x1D706}}_{s}},\end{eqnarray}$$

where $\widetilde{c}(j)=c(V-D_{j}-F_{q_{j}})$ and

$$\begin{eqnarray}\widetilde{\unicode[STIX]{x1D706}}_{j}=\left\{\begin{array}{@{}ll@{}}q_{j}+n-j\quad & \text{ if }j\leqslant a,\\ n+q_{j}-\unicode[STIX]{x1D70C}_{j}\quad & \text{ if }j>a.\end{array}\right.\end{eqnarray}$$

Furthermore, $\unicode[STIX]{x1D70B}$ maps $\widetilde{\unicode[STIX]{x1D6FA}}$ birationally onto $\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}$ .

For each $i$ , using identity (d) we have

$$\begin{eqnarray}\unicode[STIX]{x1D70B}_{\ast }^{(i)}\widetilde{c}(i)_{\widetilde{\unicode[STIX]{x1D706}}_{i}}=c(i)_{\unicode[STIX]{x1D706}_{i}},\end{eqnarray}$$

where $\unicode[STIX]{x1D706}=\unicode[STIX]{x1D706}(\boldsymbol{\unicode[STIX]{x1D70F}})$ . (Note that $\unicode[STIX]{x1D706}_{i}=\widetilde{\unicode[STIX]{x1D706}}_{i}-\operatorname{rk}(E_{p_{i}}/D_{i})=\widetilde{\unicode[STIX]{x1D706}}_{i}-n-1+p_{i}+i$ .)

This case follows, since

$$\begin{eqnarray}\displaystyle \unicode[STIX]{x1D70B}_{\ast }[\widetilde{\unicode[STIX]{x1D6FA}}] & = & \displaystyle \unicode[STIX]{x1D70B}_{\ast }R^{(\unicode[STIX]{x1D70C},s)}\widetilde{c}(1)_{\widetilde{\unicode[STIX]{x1D706}}_{1}}\cdots \widetilde{c}(s)_{\widetilde{\unicode[STIX]{x1D706}}_{s}}\nonumber\\ \displaystyle & = & \displaystyle R^{(\unicode[STIX]{x1D70C},s)}\unicode[STIX]{x1D70B}_{\ast }\widetilde{c}(1)_{\widetilde{\unicode[STIX]{x1D706}}_{1}}\cdots \widetilde{c}(s)_{\widetilde{\unicode[STIX]{x1D706}}_{r}}\nonumber\\ \displaystyle & = & \displaystyle R^{(\unicode[STIX]{x1D70C},s)}c(1)_{\unicode[STIX]{x1D706}_{1}}\cdots c(s)_{\unicode[STIX]{x1D706}_{s}}\nonumber\\ \displaystyle & = & \displaystyle \unicode[STIX]{x1D6E9}_{\unicode[STIX]{x1D706}}^{(\unicode[STIX]{x1D70C})}(c(1),\ldots ,c(s)).\nonumber\end{eqnarray}$$

2.4 General case

As in type A, any triple $\boldsymbol{\unicode[STIX]{x1D70F}}=(\mathbf{k},\mathbf{p},\mathbf{q})$ can be inflated to a triple $\boldsymbol{\unicode[STIX]{x1D70F}}^{\prime }=(\mathbf{k}^{\prime },\mathbf{p}^{\prime },\mathbf{q}^{\prime })$ having $k_{i}^{\prime }=i$ , for $1\leqslant i\leqslant \ell =k_{s}$ , so that $\boldsymbol{\unicode[STIX]{x1D70F}}$ and $\boldsymbol{\unicode[STIX]{x1D70F}}^{\prime }$ define equivalent degeneracy loci. To do this, it suffices to insert $(k^{\prime },p^{\prime },q^{\prime })$ between $(k_{i-1},p_{i-1},q_{i-1})$ and $(k_{i},p_{i},q_{i})$ whenever $k_{i}-k_{i-1}>1$ . If $p_{i-1}>p_{i}$ , one can always insert $(k_{i}-1,p_{i}+1,q_{i})$ . If $q_{i-1}>q_{i}$ , when $i\leqslant a$ , one can insert $(k_{i}-1,p_{i},q_{i}+1)$ . (Condition (c2) ensures the result is still a triple.) When $i>a$ , one can insert $(k_{i}-1,p_{i},q^{\prime })$ , where $q^{\prime }$ is the smallest negative integer greater than $q_{i}$ which is allowed by condition (c2).

When all the additional bundles $E_{p_{i}^{\prime }}$ and $F_{q_{i}^{\prime }}$ are present on $X$ , one has $\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}^{\prime }}=\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}$ ; otherwise, they can be found by appropriate projective bundles, producing a birational map $\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}^{\prime }}\rightarrow \unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}$ .

The ‘main case’ provides a theta-polynomial formula using the triple $\boldsymbol{\unicode[STIX]{x1D70F}}^{\prime }$ and classes $c^{\prime }(k_{i}^{\prime })=c^{\prime }(i)=c(V-E_{p_{i}^{\prime }}-F_{q_{i}^{\prime }})$ . For $k\leqslant k_{a}$ and $m\geqslant \unicode[STIX]{x1D706}_{k}$ , we have relations

$$\begin{eqnarray}\displaystyle c(k)_{m}^{2}+2\mathop{\sum }_{j>0}(-1)^{j}\,c(k)_{m+j}\,c(k)_{m-j}=0. & & \displaystyle \nonumber\end{eqnarray}$$

Indeed, we may suppose $k=k_{i}$ for some $i$ , so the left-hand side can be written

$$\begin{eqnarray}\displaystyle \mathop{\sum }_{j=-\infty }^{\infty }(-1)^{j}c(k)_{m+j}\,c(k)_{m-j} & = & \displaystyle (-1)^{m}c_{2m}(V-E_{p_{i}}-F_{q_{i}}+V^{\ast }-E_{p_{i}}^{\ast }-F_{q_{i}}^{\ast })\nonumber\\ \displaystyle & = & \displaystyle (-1)^{m}c_{m}(E_{p_{i}}^{\bot }/E_{p_{i}}+(F_{q_{i}}^{\bot }/F_{q_{i}})^{\ast }),\nonumber\end{eqnarray}$$

which vanishes because the bundle in the last line has rank $2\unicode[STIX]{x1D706}_{k}-2$ . These are the relations required in the hypothesis of §A.3, Lemma A.1, which shows that $\unicode[STIX]{x1D6E9}_{\unicode[STIX]{x1D706}}^{(\unicode[STIX]{x1D70C})}(c^{\prime }(1),\ldots ,c^{\prime }(\ell ))=\unicode[STIX]{x1D6E9}_{\unicode[STIX]{x1D706}}^{(\unicode[STIX]{x1D70C})}(c(1),\ldots ,c(\ell ))$ . ◻

As mentioned above, in extreme cases the theta-polynomial is a determinant or Pfaffian. To include the case where $\ell$ is odd, we recall that Pfaffians can be defined for odd matrices $(m_{ij})$ by introducing a zeroth row, $m_{0j}$ (see §A.1).

Corollary. If all $q_{i}>0$ , then

$$\begin{eqnarray}[\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}]=\operatorname{Pf}_{\unicode[STIX]{x1D706}}(c(1),\ldots ,c(\ell )),\end{eqnarray}$$

where the right-hand side is defined to be the Pfaffian of the matrix $(m_{ij})$ , with

$$\begin{eqnarray}m_{ij}=c(i)_{\unicode[STIX]{x1D706}_{i}}\,c(j)_{\unicode[STIX]{x1D706}_{j}}+2\mathop{\sum }_{a>0}(-1)^{a}c(i)_{\unicode[STIX]{x1D706}_{i}+a}\,c(j)_{\unicode[STIX]{x1D706}_{j}-a},\end{eqnarray}$$

and when $\ell$ is odd, the matrix is augmented by $m_{0j}=c(j)_{\unicode[STIX]{x1D706}_{j}}$ for $0<j\leqslant \ell$ .

This follows from the proposition of §A.2. Our conventions for Pfaffians, as in Appendix A, are that one forms a skew-symmetric matrix by defining $m_{ji}=-m_{ij}$ for $i<j$ , and $m_{ii}=0$ . (In fact, following the proof of [Reference KazarianKaz00, Theorem 1.1], it is equivalent to define the $m_{ij}$ for all $i,j$ by the formula of the above corollary, but we do not need this.)

Remark 1. Here are geometric reasons for the conditions on a triple. Only elementary linear algebra and basic facts about nondegenerate bilinear forms are needed: an isotropic subspace has dimension at most half that of the ambient space; $(E\cap F)^{\bot }=E^{\bot }+F^{\bot }$ ; and $(V/E)^{\ast }\cong E^{\bot }$ .

Condition (c1) has a simple explanation:

$$\begin{eqnarray}(E_{p_{i}}\cap F_{q_{i}})/(E_{p_{i-1}}\cap F_{q_{i-1}})\subseteq E_{p_{i}}/E_{p_{i-1}}\oplus F_{q_{i}}/F_{q_{i-1}}.\end{eqnarray}$$

Condition (c3) arises from asking that the coisotropic condition imposed by $F_{q_{i}}$ be independent of any isotropic conditions, in the following sense. With $i$ and $j$ as in the definition of $\unicode[STIX]{x1D70C}$ , so $i>a$ and $\unicode[STIX]{x1D70C}_{k_{i}}=k_{j}$ , we have $F_{q_{j}}\subseteq F_{q_{i}}^{\bot }$ (and this is the largest among the $F_{q}$ which is contained in $F_{q_{i}}^{\bot }$ ). For generic $E_{p}\supseteq E_{p_{j}}$ , we require

$$\begin{eqnarray}\dim (E_{p}\cap F_{q_{i}}^{\bot })=\dim (E_{p_{j}}\cap F_{q_{i}}^{\bot })=\dim (E_{p_{j}}\cap F_{q_{j}})=k_{j},\end{eqnarray}$$

and in particular this holds for $p=p_{i}$ and $p=p_{i-1}$ . Thus

for generic spaces, a formulation which is used in proving the main theorem of this section. Now Condition (c3) is a consequence of (∗) and the fact that

$$\begin{eqnarray}(E_{p_{i}}\cap F_{q_{i}})/(E_{p_{i}}\cap F_{q_{i}}^{\bot })=((E_{p_{i}}\cap F_{q_{i}})+F_{q_{i}}^{\bot })/F_{q_{i}}^{\bot }\subseteq F_{q_{i}}/F_{q_{i}}^{\bot }.\end{eqnarray}$$

(Since $\dim (F_{q_{i}}/F_{q_{i}}^{\bot })=-2q_{i}$ , an isotropic subspace has dimension at most $-q_{i}$ .)

Condition (c4) is similar to (c1): one has

$$\begin{eqnarray}(E_{p_{i}}\cap F_{q_{i}})/(E_{p_{i-1}}\cap F_{q_{i-1}}){\hookrightarrow}E_{p_{i}}/E_{p_{i-1}}\oplus F_{q_{i}}/F_{q_{i-1}}{\twoheadrightarrow}(E_{p_{i}}+F_{q_{i}})/(E_{p_{i-1}}+F_{q_{i-1}}),\end{eqnarray}$$

and one sees $\dim ((E_{p_{i}}+F_{q_{i}})/(E_{p_{i-1}}+F_{q_{i-1}}))\geqslant \unicode[STIX]{x1D70C}_{k_{i-1}}-\unicode[STIX]{x1D70C}_{k_{i}}$ from the following diagram.

The map $\unicode[STIX]{x1D6FC}$ is injective because $(E_{p_{i}}^{\bot }\cap F_{q_{i}}^{\bot })\cap (E_{p_{i}}\cap F_{q_{i}}^{\bot })=E_{p_{i}}\cap F_{q_{i}}^{\bot }$ .

Condition (c5) comes from writing $E_{p_{i}}\cap F_{q_{i}}$ as the kernel of

$$\begin{eqnarray}E_{p_{i}}\rightarrow (E_{p_{i}}^{\bot }+F_{q_{i}})/F_{q_{i}}.\end{eqnarray}$$

Finally, for (c2), consider the case of a single $q_{i}=q$ ; if $E_{p}$ and $F_{q}$ are isotropic subspaces such that $\dim (E_{p}\cap F_{q})=k$ , then for generic subspaces $F_{q}\supset F_{q+1}\supset \cdots \supset F_{q+k}$ , one has

$$\begin{eqnarray}E_{p}^{\bot }\cap F_{q}^{\bot }=E_{p}^{\bot }\cap F_{q+1}^{\bot }=\cdots =E_{p}^{\bot }\cap F_{q+k}^{\bot }.\end{eqnarray}$$

For any isotropic $E_{p^{\prime }}\supseteq E_{p}$ , it follows that

$$\begin{eqnarray}E_{p^{\prime }}\cap F_{q}^{\bot }=E_{p^{\prime }}\cap F_{q+1}^{\bot }=\cdots =E_{p^{\prime }}\cap F_{q+k}^{\bot },\end{eqnarray}$$

so we should only impose rank conditions on the first of these. Writing $F_{q}^{\bot }=F_{-q+1}$ , $F_{q+1}^{\bot }=F_{-q}$ , etc., this means the values $-q,-q-1,\ldots ,-q-k+1$ should be prohibited. Accounting for intersections previously imposed leads to the above condition (c2).

Remark 2. A type C triple determines a signed permutation $w(\boldsymbol{\unicode[STIX]{x1D70F}})$ , as follows. Starting in position $p_{1}$ , first place $k_{1}$ consecutive integers in increasing order, ending with $\overline{q}_{1}$ . Then starting in position $p_{2}$ (or the next available position to the right of $p_{2}$ ), place $k_{2}-k_{1}$ integers, consecutive among those whose absolute values have not been used, ending in at most $\overline{q}_{2}$ . Continue until $k_{s}$ numbers have been placed, and finish by filling in the gaps with the smallest available positive integers.

The conditions on $\boldsymbol{\unicode[STIX]{x1D70F}}$ can be understood combinatorially: they guarantee that the length of the signed permutation $w(\boldsymbol{\unicode[STIX]{x1D70F}})$ is equal to the size of the partition $\unicode[STIX]{x1D706}(\boldsymbol{\unicode[STIX]{x1D70F}})$ ; this can be checked directly from the construction, using the combinatorial characterization of length from [Reference Björner and BrentiBB05, §8.1]. They also ensure that $\unicode[STIX]{x1D70C}_{k_{i}}$ is the number of entries of $w$ which are less than $-|q_{i}|$ ; that the $k_{a}$ entries of $w$ placed in the first $a$ steps of the above recipe are all negative; and that the entries placed after the first $a$ steps are all positive.

For example, consider $\boldsymbol{\unicode[STIX]{x1D70F}}=(\,1\,3\,5\,6\,7\,9\,,\;9\,7\,6\,5\,2\,2\,,\;6\,3\,\overline{2}\,\overline{5}\,\overline{7}\,\overline{9}\,)$ . The corresponding signed permutation is built in seven steps:

$$\begin{eqnarray}\displaystyle & & \displaystyle \cdot \;\cdot \;\cdot \;\cdot \;\cdot \;\cdot \;\cdot \;\cdot \;\overline{\mathbf{6}}\;\cdot \;,\nonumber\\ \displaystyle & & \displaystyle \cdot \;\cdot \;\cdot \;\cdot \;\cdot \;\cdot \;\overline{\mathbf{4}}\;\overline{\mathbf{3}}\;\overline{6}\;\cdot \;,\nonumber\\ \displaystyle & & \displaystyle \cdot \;\cdot \;\cdot \;\cdot \;\cdot \;\mathbf{1}\;\overline{4}\;\overline{3}\;\overline{6}\;\mathbf{2}\;,\nonumber\\ \displaystyle & & \displaystyle \cdot \;\cdot \;\cdot \;\cdot \;\mathbf{5}\;1\;\overline{4}\;\overline{3}\;\overline{6}\;2\;,\nonumber\\ \displaystyle & & \displaystyle \cdot \;\mathbf{7}\;\cdot \;\cdot \;5\;1\;\overline{4}\;\overline{3}\;\overline{6}\;2\;,\nonumber\\ \displaystyle & & \displaystyle \cdot \;7\;\mathbf{8}\;\mathbf{9}\;5\;1\;\overline{4}\;\overline{3}\;\overline{6}\;2\;,\nonumber\\ \displaystyle w(\boldsymbol{\unicode[STIX]{x1D70F}}) & = & \displaystyle \mathbf{10}\;7\;8\;9\;5\;1\;\overline{4}\;\overline{3}\;\overline{6}\;2\;.\nonumber\end{eqnarray}$$

Since $a=2$ , the $k_{a}=3$ negative entries all appear in the first and second steps. The partition $\unicode[STIX]{x1D706}(\boldsymbol{\unicode[STIX]{x1D70F}})$ is the one displayed in Figure 1, and a computation shows that $\ell (w(\boldsymbol{\unicode[STIX]{x1D70F}}))=|\unicode[STIX]{x1D706}(\boldsymbol{\unicode[STIX]{x1D70F}})|=50$ .

It would be interesting to know more about the combinatorial properties of signed permutations arising this way. For example, are they characterized by pattern avoidance?Footnote ³

Remark 3. When all $p_{i}=p$ , then a $\unicode[STIX]{x1D70C}$ -strict partition $\unicode[STIX]{x1D706}(\boldsymbol{\unicode[STIX]{x1D70F}})$ is one so that all parts of size greater than $p-1$ are distinct; that is, it is a $(p-1)$ -strict partition as defined in [Reference Buch, Kresch and TamvakisBKT17]. Geometrically, all $E_{p_{i}}$ have rank $n+1-p$ , so the locus comes from an isotropic Grassmannian. Conversely, given a $(p-1)$ -strict partition $\unicode[STIX]{x1D706}$ , following [Reference Buch, Kresch and TamvakisBKT17] define

$$\begin{eqnarray}P_{j}(\unicode[STIX]{x1D706})=n+p-1+j-\unicode[STIX]{x1D706}_{j}-\#\big\{i<j\,\big|\,\unicode[STIX]{x1D706}_{i}+\unicode[STIX]{x1D706}_{j}>2p-2+j-i\big\}.\end{eqnarray}$$

If we define a type C triple by setting $k_{j}=j$ , $p_{j}=p$ , and

$$\begin{eqnarray}\displaystyle q_{j}=\left\{\begin{array}{@{}ll@{}}n-P_{j}\quad & \text{ when }P_{j}>n,\\ n+1-P_{j}\quad & \text{ when }P_{j}\leqslant n,\end{array}\right. & & \displaystyle \nonumber\end{eqnarray}$$

one recovers $\unicode[STIX]{x1D706}=\unicode[STIX]{x1D706}(\boldsymbol{\unicode[STIX]{x1D70F}})$ . Indeed, one can check that $\unicode[STIX]{x1D70C}(\boldsymbol{\unicode[STIX]{x1D70F}})$ is given by

$$\begin{eqnarray}\displaystyle \unicode[STIX]{x1D70C}_{j} & = & \displaystyle \#\big\{i<j\,\big|\,q_{i}+q_{j}>0\big\}\nonumber\\ \displaystyle & = & \displaystyle \#\big\{i<j\,\big|\,\unicode[STIX]{x1D706}_{i}+\unicode[STIX]{x1D706}_{j}>2p-2+j-i\big\}\nonumber\end{eqnarray}$$

in this situation.

Similarly, for a triple $\boldsymbol{\unicode[STIX]{x1D70F}}$ with all $p_{i}=p$ and $k_{i}=i$ , the characteristic index $\unicode[STIX]{x1D712}$ used in [Reference Ikeda and MatsumuraIM15] is given by $\unicode[STIX]{x1D712}_{i}=q_{i}-1$ for $i\leqslant a$ , and $\unicode[STIX]{x1D712}_{i}=q_{i}$ for $i>a$ .

3 Type B: odd orthogonal bundles

A triple of type B is the same as in type C, as are the definitions of the sequence $\unicode[STIX]{x1D70C}(\boldsymbol{\unicode[STIX]{x1D70F}})$ , the partition $\unicode[STIX]{x1D706}(\boldsymbol{\unicode[STIX]{x1D70F}})$ , the raising operator $R^{(\unicode[STIX]{x1D70C},\ell )}$ , and the theta-polynomial.

The geometry starts with a vector bundle $V$ of rank $2n+1$ , equipped with a nondegenerate quadratic form. We have two flags of subbundles,

$$\begin{eqnarray}\displaystyle E_{p_{1}} & \subset & \displaystyle E_{p_{2}}\subset \cdots \subset E_{p_{s}}\subset V,\nonumber\\ \displaystyle F_{q_{1}} & \subset & \displaystyle F_{q_{2}}\subset \cdots \subset F_{q_{s}}\subset V;\nonumber\end{eqnarray}$$

the isotropicity and ranks of these are exactly as in type C. The degeneracy locus is

$$\begin{eqnarray}\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}=\{x\in X\,|\,\dim (E_{p_{i}}\cap F_{q_{i}})\geqslant k_{i}\text{ for }1\leqslant i\leqslant s\}.\end{eqnarray}$$

Let $M$ be the line bundle $\det V$ , and note that

$$\begin{eqnarray}M\cong F^{\bot }/F,\end{eqnarray}$$

for any maximal isotropic $F\subset V$ . In fact, we have $M\cong \det (D^{\bot }/D)$ for any isotropic $D\subset V$ .

Given a triple, let $\ell =k_{s}$ and $r=k_{a}$ , recalling that $a$ is the index such that $q_{a}>0>q_{a+1}$ . For $i\leqslant a$ let $c(k_{i})=c(V-E_{p_{i}}-F_{q_{i}}-M)$ , and for $i>a$ , let $c(k_{i})=c(V-E_{p_{i}}-F_{q_{i}})$ . As before, when $k_{i-1}<k\leqslant k_{i}$ , we set $c(k)=c(k_{i})$ .

The four steps of the proof are almost the same as in type C. We will indicate the differences.

3.1 Basic case

Take $s=1$ , $k_{1}=\ell =1$ , and $p_{1}=n$ , so $E_{n}$ is a line bundle and $\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}$ is the locus where $E_{n}\subseteq F_{q_{1}}$ . When $q_{1}>0$ , so $F_{q_{1}}$ is isotropic, the proposition of Appendix B gives

$$\begin{eqnarray}2\,[\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}]=c_{n+q_{1}-1}(V-F_{q_{1}}-E_{n}-M).\end{eqnarray}$$

On the other hand, when $q_{1}<0$ , so $F_{q_{1}}$ is coisotropic, the locus is defined (scheme-theoretically) by the vanishing of $E_{n}\rightarrow V/F_{q_{1}}$ , and

$$\begin{eqnarray}[\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}]=c_{n+q_{1}}(V/F_{q_{1}}\otimes E_{n}^{\ast })=c_{n+q_{1}}(V-F_{q_{1}}-E_{n})\end{eqnarray}$$

in this case.

3.2 Dominant case

Now take $k_{i}=i$ and $p_{i}=n+1-i$ , for $1\leqslant i\leqslant s$ , and write $D_{i}=E_{p_{i}}$ . As in type C, we have a filtration by $Z_{j}$ , the locus where $D_{i}\subseteq F_{q_{i}}$ for all $i\leqslant j$ , so that $Z_{0}=X$ and $Z_{s}=\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}$ . When $j\leqslant a$ (so $q_{j}>0$ ), the basic case says that

$$\begin{eqnarray}2\,[Z_{j}]=[Z_{j-1}]\cdot c_{\unicode[STIX]{x1D706}_{j}}(D_{j-1}^{\bot }/D_{j-1}-F_{q_{j}}/D_{j-1}-D_{j}/D_{j-1}-M);\end{eqnarray}$$

and when $j>a$ (so $q_{j}<0$ ),

$$\begin{eqnarray}[Z_{j}]=[Z_{j-1}]\cdot c_{\unicode[STIX]{x1D706}_{j}}(D_{\unicode[STIX]{x1D70C}_{j}}^{\bot }/D_{j-1}-F_{q_{j}}/D_{j-1}-D_{j}/D_{j-1});\end{eqnarray}$$

where in each case $\unicode[STIX]{x1D70C}_{j}$ and $\unicode[STIX]{x1D706}_{j}$ is defined as in type C. We therefore have

$$\begin{eqnarray}\displaystyle 2^{a}\,[\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}] & = & \displaystyle \biggl(\mathop{\prod }_{j=1}^{a}c_{\unicode[STIX]{x1D706}_{j}}(D_{\unicode[STIX]{x1D70C}_{j}}^{\bot }/D_{j-1}-F_{q_{j}}/D_{j-1}-D_{j}/D_{j-1}-M)\biggr)\nonumber\\ \displaystyle & & \displaystyle \times \biggl(\mathop{\prod }_{j=a+1}^{s}c_{\unicode[STIX]{x1D706}_{j}}(D_{\unicode[STIX]{x1D70C}_{j}}^{\bot }/D_{j-1}-F_{q_{j}}/D_{j-1}-D_{j}/D_{j-1})\biggr),\nonumber\end{eqnarray}$$

as before.

The rest of the proof proceeds exactly as in type C. ◻

As in type C, we recover a Pfaffian formula for vexillary signed permutations.

Corollary. If all $q_{i}>0$ , then

$$\begin{eqnarray}2^{\ell }\,[\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}]=\operatorname{Pf}_{\unicode[STIX]{x1D706}}(c(1),\ldots ,c(\ell )).\end{eqnarray}$$

4 Type D: even orthogonal bundles

A triple of type D is $\boldsymbol{\unicode[STIX]{x1D70F}}=(\mathbf{k},\mathbf{p},\mathbf{q})$ , with

$$\begin{eqnarray}\displaystyle 0<k_{1} & {<} & \displaystyle k_{2}<\cdots <k_{s},\nonumber\\ \displaystyle p_{1} & {\geqslant} & \displaystyle p_{2}\geqslant \cdots \geqslant p_{s}\geqslant 0,\nonumber\\ \displaystyle q_{1} & {\geqslant} & \displaystyle q_{2}\geqslant \cdots \geqslant q_{s}.\nonumber\end{eqnarray}$$

The value $q=-1$ is prohibited, and if $p_{s}=0$ , then all $q_{i}\geqslant 0$ . Set $a=a(\boldsymbol{\unicode[STIX]{x1D70F}})$ to be the integer such that $q_{a}>-1>q_{a+1}$ .

A quick way to characterize the further requirements on a type D triple is as follows. Form $\boldsymbol{\unicode[STIX]{x1D70F}}^{+}$ by replacing each $p_{i}$ in $\boldsymbol{\unicode[STIX]{x1D70F}}$ with $p_{i}+1$ , and replacing each $q_{i}\geqslant 0$ in $\boldsymbol{\unicode[STIX]{x1D70F}}$ with $q_{i}+1$ ; then a type D triple is one such that $\boldsymbol{\unicode[STIX]{x1D70F}}^{+}$ is a type C triple.

To be completely clear, we will spell out the conditions. Their geometric explanations are analogous to those for type C. For $i\leqslant a$ ,

(d1) $$\begin{eqnarray}\displaystyle k_{i}-k_{i-1}\leqslant (p_{i-1}-p_{i})+(q_{i-1}-q_{i}), & & \displaystyle\end{eqnarray}$$

and, as before, this is the only condition when all $q_{i}$ are nonnegative.

For each $i\leqslant a$ , let $m(i)=\min \{m\,|\,q_{i}+(k_{i}-k_{i-1})\geqslant q_{m}\}$ . The negative values

(d2) $$\begin{eqnarray}-\!q_{i}-1,\,-q_{i}-2,\,\ldots ,\,-q_{i}-(k_{i}-k_{m(i)-1})\end{eqnarray}$$

are prohibited as values of $q_{j}$ for $j>a$ .

The sequence $\unicode[STIX]{x1D70C}(\boldsymbol{\unicode[STIX]{x1D70F}})$ is defined similarly as in type C. Set $\unicode[STIX]{x1D70C}_{k}=k-1$ for $k\leqslant k_{a}$ . For $i>a$ , set $\unicode[STIX]{x1D70C}_{k_{i}}=k_{j}$ , where $j$ is the index such that $q_{j}\geqslant -q_{i}>q_{j+1}+1$ . Then fill in the other parts minimally subject to $\unicode[STIX]{x1D70C}_{k_{a}+1}\geqslant \cdots \geqslant \unicode[STIX]{x1D70C}_{k_{s}}\geqslant 0$ . We require

(d3) $$\begin{eqnarray}k_{i}-\unicode[STIX]{x1D70C}_{k_{i}}\leqslant -q_{i}\end{eqnarray}$$

for all $i>a$ .

Finally, for $i>a+1$ ,

(d4) $$\begin{eqnarray}(k_{i}-k_{i-1})+(\unicode[STIX]{x1D70C}_{k_{i-1}}-\unicode[STIX]{x1D70C}_{k_{i}})\leqslant (p_{i-1}-p_{i})+(q_{i-1}-q_{i}),\end{eqnarray}$$

and

(d5) $$\begin{eqnarray}k_{s}\geqslant -p_{s}-q_{s}+\unicode[STIX]{x1D70C}_{k_{s}}.\end{eqnarray}$$

The associated partition $\unicode[STIX]{x1D706}(\boldsymbol{\unicode[STIX]{x1D70F}})$ is defined by

$$\begin{eqnarray}\displaystyle \unicode[STIX]{x1D706}_{k_{i}}=\left\{\begin{array}{@{}ll@{}}p_{i}+q_{i}\quad & \text{if }i\leqslant a,\\ p_{i}+q_{i}+k_{i}-\unicode[STIX]{x1D70C}_{k_{i}}\quad & \text{if }i>a,\end{array}\right. & & \displaystyle \nonumber\end{eqnarray}$$

filling in the other parts minimally subject to

$$\begin{eqnarray}\displaystyle \unicode[STIX]{x1D706}_{1}>\cdots >\unicode[STIX]{x1D706}_{k_{a}}\geqslant \unicode[STIX]{x1D706}_{k_{a}+1}\geqslant \cdots \geqslant \unicode[STIX]{x1D706}_{\ell }\geqslant 0, & & \displaystyle \nonumber\end{eqnarray}$$

where $\ell =k_{s}$ . That is, $\unicode[STIX]{x1D706}(\boldsymbol{\unicode[STIX]{x1D70F}})$ is a $\unicode[STIX]{x1D70C}(\boldsymbol{\unicode[STIX]{x1D70F}})$ -strict partition.Footnote ⁴ (In contrast to type C, we allow $\unicode[STIX]{x1D706}_{k_{a}}=\unicode[STIX]{x1D706}_{k_{a}+1}$ .)

A key difference in type D is that $\unicode[STIX]{x1D706}(\boldsymbol{\unicode[STIX]{x1D70F}})$ may have a part equal to $0$ , and this is included in the data. (For example, this happens when $p_{s}=q_{s}=0$ ; in this case, the total number of parts determines the dimension of the intersection of two maximal isotropic subspaces.)

The raising operators and polynomials require some setup; details are explained in Appendix A. We will have elements $c(i)=d(i)+e(i)$ , so that $c(i)_{k}=d(i)_{k}+e(i)_{k}$ . We will also have operators $\unicode[STIX]{x1D6FF}_{i}$ , which acts on a monomial $c(1)_{\unicode[STIX]{x1D6FC}_{1}}\cdots c(\ell )_{\unicode[STIX]{x1D6FC}_{\ell }}$ by replacing $c(i)_{\unicode[STIX]{x1D6FC}_{i}}$ with $d(i)_{\unicode[STIX]{x1D6FC}_{i}}$ ; that is, $\unicode[STIX]{x1D6FF}_{i}$ sends $e(i)$ to zero and leaves everything else unchanged.

Given integers $0\leqslant r\leqslant \ell$ , and a sequence of nonnegative integers $\unicode[STIX]{x1D70C}=(\unicode[STIX]{x1D70C}_{1},\ldots ,\unicode[STIX]{x1D70C}_{\ell })$ with $\unicode[STIX]{x1D70C}_{j}<j$ and $\unicode[STIX]{x1D70C}_{j}=j-1$ for $1\leqslant j\leqslant k$ , we define the raising operator

$$\begin{eqnarray}\widetilde{R}^{(\unicode[STIX]{x1D70C},r,\ell )}=\mathop{\prod }_{j=r+1}^{\ell }\frac{\mathop{\prod }_{i=1}^{r-1}(1-R_{ij})}{\mathop{\prod }_{i=1}^{\unicode[STIX]{x1D70C}_{j}}(1+R_{ij})}\mathop{\prod }_{r\leqslant i<j\leqslant \ell }(1-R_{ij})\mathop{\prod }_{1\leqslant i<j\leqslant r}\biggl(\frac{1-\unicode[STIX]{x1D6FF}_{i}\unicode[STIX]{x1D6FF}_{j}R_{ij}}{1+\unicode[STIX]{x1D6FF}_{i}\unicode[STIX]{x1D6FF}_{j}R_{ij}}\biggr),\end{eqnarray}$$

and using these, the eta-polynomial for a $\unicode[STIX]{x1D70C}$ -strict partition is defined as

$$\begin{eqnarray}\text{H}_{\unicode[STIX]{x1D706}}^{(\unicode[STIX]{x1D70C})}(c(1),\ldots ,c(\ell ))=\widetilde{R}^{(\unicode[STIX]{x1D70C},r,\ell )}\cdot (c(1)_{\unicode[STIX]{x1D706}_{1}}\cdots c(\ell )_{\unicode[STIX]{x1D706}_{\ell }}).\end{eqnarray}$$

Here is the geometry. We have a vector bundle $V$ of rank $2n$ , equipped with a nondegenerate quadratic form taking values in the trivial bundle. There are flags of subbundles,

$$\begin{eqnarray}\displaystyle E_{p_{1}} & \subset & \displaystyle E_{p_{2}}\subset \cdots \subset E_{p_{s}}\subset V,\nonumber\\ \displaystyle F_{q_{1}} & \subset & \displaystyle F_{q_{2}}\subset \cdots \subset F_{q_{s}}\subset V;\nonumber\end{eqnarray}$$

each $E_{p}$ has rank $n-p$ and is isotropic; and each $F_{q}$ has rank $n-q$ , and is isotropic when $q\geqslant 0$ and coisotropic when $q<0$ . Note that $F_{q}^{\bot }=F_{-q}$ .

The degeneracy locus is defined as before, by

$$\begin{eqnarray}\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}=\{x\in X\,|\,\dim (E_{p_{i}}\cap F_{q_{i}})\geqslant k_{i}\text{ for }1\leqslant i\leqslant s\}.\end{eqnarray}$$

The usual type D caveat applies: this acquires its scheme structure via pullback from a Schubert bundle in a flag bundle, and even there it must be taken to mean the closure of the locus where equality holds (see, for example, [Reference Fulton and PragaczFP98, §6] or [Reference TamvakisTam16a, §6.3.2]).

Now given a type D triple, set $\ell =k_{s}$ and $r=k_{a}$ . Let $d(k_{i})=c(V-E_{p_{i}}-F_{q_{i}})$ , and for $i\leqslant a$ (so $q_{i}\geqslant 0$ ), set $e(k_{i})=e(E_{p_{i}},F_{q_{i}})$ , where the latter is defined as

$$\begin{eqnarray}e(E_{p_{i}},F_{q_{i}}):=(-1)^{\dim (E\cap F)}\,c(E/E_{p_{i}}+F/F_{q_{i}}),\end{eqnarray}$$

for some maximal isotropic bundles $E\supseteq E_{p_{i}}$ and $F\supseteq F_{q_{i}}$ . (Only the Euler class $e_{p_{i}+q_{i}}(E_{p_{i}},F_{q_{i}})$ appears in our formulas, and this is independent of the choice of such maximal $E$ and $F$ .) When $i>a$ , we set $e(k_{i})=0$ ; and as usual, when $k_{i-1}<k\leqslant k_{i}$ , we set $d(k)=d(k_{i})$ and $e(k)=e(k_{i})$ . Finally, let

$$\begin{eqnarray}c(k)=d(k)+(-1)^{k}e(k).\end{eqnarray}$$

Most of the proof proceeds exactly as in type B. We will go through the outline briefly, to point out the differences.

4.1 Basic case

Here $s=1$ , $k_{1}=\ell =1$ , and $p_{1}=n-1$ , so $E_{n-1}$ is a line bundle and $\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}$ is the locus where $E_{n-1}\subseteq F_{q_{1}}$ . Just as before, we have

$$\begin{eqnarray}\displaystyle \begin{array}{@{}rcll@{}}2\,[\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}]\,\, & =\,\, & c_{n+q_{1}-1}(V-F_{q_{1}}-E_{n-1})-e_{n+q_{1}-1}(E_{n-1},F_{q_{1}}) & \text{when }q_{1}\geqslant 0,\\ \,[\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}]\,\, & =\,\, & c_{n+q_{1}}(V-F_{q_{1}}-E_{n-1}) & \text{when }q_{1}<-1.\end{array} & & \displaystyle \nonumber\end{eqnarray}$$

The proof is the same as in type B.

4.2 Dominant case

Now $k_{i}=i$ for $1\leqslant i\leqslant s$ , and $p_{i}=n-i$ , so $D_{i}=E_{n-i}$ has rank $i$ . Let $Z_{j}$ be the locus where $D_{j}\subseteq F_{q_{j}}$ , so $Z_{0}=X$ and $Z_{s}=\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}$ . When $j\leqslant a$ , applying the basic case with $V$ replaced by $D_{j-1}^{\bot }/D_{j-1}$ , we obtain

$$\begin{eqnarray}\displaystyle 2\,[Z_{j}] & = & \displaystyle [Z_{j-1}]\cdot \big(c_{\unicode[STIX]{x1D706}_{j}}(D_{j-1}^{\bot }/D_{j-1}-F_{q_{j}}/D_{j-1}-D_{j}/D_{j-1})\nonumber\\ \displaystyle & & \displaystyle -e_{\unicode[STIX]{x1D706}_{j}}(D_{j}/D_{j-1},F_{q_{j}}/D_{j-1})\big)\nonumber\\ \displaystyle & = & \displaystyle [Z_{j-1}]\cdot \big(c_{\unicode[STIX]{x1D706}_{j}}(D_{j-1}^{\bot }/D_{j-1}-F_{q_{j}}/D_{j-1}-D_{j}/D_{j-1})\nonumber\\ \displaystyle & & \displaystyle +\,(-1)^{j}e_{\unicode[STIX]{x1D706}_{j}}(D_{j},F_{q_{j}})\big),\nonumber\end{eqnarray}$$

where $\unicode[STIX]{x1D706}_{j}=n-j+q_{j}$ .

When $j>a$ , using $D_{j-1}^{\bot }/(F_{q_{j}}\cap D_{j-1}^{\bot })=D_{\unicode[STIX]{x1D70C}_{j}}^{\bot }/F_{q_{j}}$ (by (∗) as in type C), the locus is given by the vanishing of $D_{j}/D_{j-1}\rightarrow D_{\unicode[STIX]{x1D70C}_{j}}^{\bot }/F_{q_{j}}$ , so the basic case says

$$\begin{eqnarray}[Z_{j}]=[Z_{j-1}]\cdot c_{\unicode[STIX]{x1D706}_{j}}(D_{\unicode[STIX]{x1D70C}_{j}}^{\bot }/D_{j-1}-F_{q_{j}}/D_{j-1}-D_{j}/D_{j-1}),\end{eqnarray}$$

where $\unicode[STIX]{x1D70C}_{j}$ and $\unicode[STIX]{x1D706}_{j}$ are defined as above.

It follows that

$$\begin{eqnarray}\displaystyle 2^{a}\,[\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}] & = & \displaystyle \biggl(\mathop{\prod }_{j=1}^{a}(c_{\unicode[STIX]{x1D706}_{j}}(D_{\unicode[STIX]{x1D70C}_{j}}^{\bot }/D_{j-1}-F_{q_{j}}/D_{j-1}-D_{j}/D_{j-1})+(-1)^{j}e_{\unicode[STIX]{x1D706}_{j}}(D_{j},F_{q_{j}}))\biggr)\nonumber\\ \displaystyle & & \displaystyle \times \biggl(\mathop{\prod }_{j=a+1}^{s}c_{\unicode[STIX]{x1D706}_{j}}(D_{\unicode[STIX]{x1D70C}_{j}}^{\bot }/D_{j-1}-F_{q_{j}}/D_{j-1}-D_{j}/D_{j-1})\biggr).\nonumber\end{eqnarray}$$

This leads to

(9) $$\begin{eqnarray}2^{a}\,[\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}]=\widetilde{R}^{(\unicode[STIX]{x1D70C},a,s)}c(1)_{\unicode[STIX]{x1D706}_{1}}\cdots c(s)_{\unicode[STIX]{x1D706}_{s}},\end{eqnarray}$$

with $c(j)=d(j)+(-1)^{j}e(j)$ for $j\leqslant a$ and $c(j)=d(j)$ for $j>a$ , as defined above. The deduction of (9) differs slightly from the previous cases (e.g., deducing (8) from (7)): one must verify

$$\begin{eqnarray}\displaystyle & & \displaystyle t_{i}^{b}\cdot (c_{\unicode[STIX]{x1D706}_{i}}(D_{i-1}^{\bot }/D_{i-1}-F_{q_{i}}/D_{i-1}-D_{i}/D_{i-1})+(-1)^{i}e_{\unicode[STIX]{x1D706}_{i}}(D_{i},F_{q_{i}}))\nonumber\\ \displaystyle & & \displaystyle \quad =c_{\unicode[STIX]{x1D706}_{i}+b}(D_{i-1}^{\bot }/D_{i-1}-F_{q_{i}}/D_{i-1}-D_{i}/D_{i-1})\nonumber\end{eqnarray}$$

for $i\leqslant a$ , where $t_{i}=-c_{1}(D_{i}/D_{i-1})$ , since this shows that $t_{i}$ acts as $\unicode[STIX]{x1D6FF}_{i}T_{i}$ on the factor $c(i)_{\unicode[STIX]{x1D706}_{i}}$ . To do this, by replacing $D_{i-1}^{\bot }/D_{i-1}$ with $V$ one may reduce to the case $i=1$ ; one also reduces to the case $b=1$ by applying identity (b). In this case, we compute

$$\begin{eqnarray}\displaystyle & & \displaystyle t_{1}\cdot (c_{n-1+q_{1}}(V-F_{q_{1}}-D_{1})-e_{n-1+q_{1}}(D_{1},F_{q_{1}}))\nonumber\\ \displaystyle & & \displaystyle \quad =\mathop{\sum }_{k=1}^{n+q_{1}}t_{1}^{k}\,c_{n+q_{1}-k}(V/F_{q_{1}})+(-1)^{\dim (E\cap F)}c_{n+q_{1}}(E+F/F_{q_{1}})\nonumber\\ \displaystyle & & \displaystyle \quad =c_{n+q_{1}}(V-D_{1}-F_{q_{1}})-c_{n+q_{1}}(V/F_{q_{1}})+(-1)^{\dim (E\cap F)}c_{n}(E)\,c_{q_{1}}(F/F_{q_{1}})\nonumber\\ \displaystyle & & \displaystyle \quad =c_{n+q_{1}}(V-D_{1}-F_{q_{1}})+c_{q_{1}}(F/F_{q_{1}})\big((-1)^{\dim (E\cap F)}c_{n}(E)-c_{n}(F^{\ast })\big),\nonumber\end{eqnarray}$$

and apply the relation $(-1)^{\dim (E\cap F)}c_{n}(E)=c_{n}(F^{\ast })$ , due to Edidin and Graham [Reference Edidin and GrahamEG95].

The remainder of the proof proceeds as in the other types. To deduce the general case, one needs to apply §A.3, Lemma A.2. The relations in the hypothesis of that lemma require that

$$\begin{eqnarray}(d(k)_{m}-(-1)^{\ell }e(k)_{m})(d(k)_{m}+(-1)^{\ell }e(k)_{m})+2\mathop{\sum }_{j>0}(-1)^{j}d(k)_{m+j}\,d(k)_{m-j}\end{eqnarray}$$

vanish for all $m\geqslant \unicode[STIX]{x1D706}_{k}$ . This expression may be re-written as

$$\begin{eqnarray}\displaystyle & & \displaystyle \mathop{\sum }_{j=-\infty }^{\infty }(-1)^{j}d(k)_{m+j}d(k)_{m-j}-(e(k)_{m})^{2}\nonumber\\ \displaystyle & & \displaystyle \quad =(-1)^{m}\,c_{2m}(E_{p_{i}}^{\bot }/E_{p_{i}}+(F_{q_{i}}^{\bot }/F_{q_{i}})^{\ast })-c_{m}(E/E_{p_{i}}+F/F_{q_{i}})^{2},\nonumber\end{eqnarray}$$

which vanishes if $m>\unicode[STIX]{x1D706}_{k}=p_{i}+q_{i}$ , since this is the rank of $E/E_{p_{i}}+F/F_{q_{i}}$ . When $m=\unicode[STIX]{x1D706}_{k}$ , we have $(-1)^{\unicode[STIX]{x1D706}_{k}}\,c_{2\unicode[STIX]{x1D706}_{k}}(E_{p_{i}}^{\bot }/E_{p_{i}}+(F_{q_{i}}^{\bot }/F_{q_{i}})^{\ast })=c_{\unicode[STIX]{x1D706}_{k}}(E/E_{p_{i}}+F/F_{q_{i}})^{2}$ , so the relation holds in this case as well.◻

To extract a Pfaffian from the case where $a=s$ , so all $q_{i}\geqslant 0$ , one needs a little algebra, given by the theorem of §A.2. Applying this proves the following.

Corollary. If all $q_{i}\geqslant 0$ , then

(10) $$\begin{eqnarray}\displaystyle 2^{\ell }\,[\unicode[STIX]{x1D6FA}_{\boldsymbol{\unicode[STIX]{x1D70F}}}]=\operatorname{Pf}_{\unicode[STIX]{x1D706}}(c(1),\ldots ,c(\ell )). & & \displaystyle\end{eqnarray}$$

Unpacking the definition of the $c(i)$ , the right-hand side is the Pfaffian of the matrix $(m_{ij})$ , with

$$\begin{eqnarray}\displaystyle m_{ij}=(d(i)_{\unicode[STIX]{x1D706}_{i}}-(-1)^{\ell }e(i)_{\unicode[STIX]{x1D706}_{i}})(d(j)_{\unicode[STIX]{x1D706}_{j}}+(-1)^{\ell }e(j)_{\unicode[STIX]{x1D706}_{j}})+2\mathop{\sum }_{t>0}(-1)^{t}d(i)_{\unicode[STIX]{x1D706}_{i}+t}\,d(j)_{\unicode[STIX]{x1D706}_{j}-t}, & & \displaystyle \nonumber\end{eqnarray}$$

for $1\leqslant i<j\leqslant \ell$ , and $m_{0j}=d(j)_{\unicode[STIX]{x1D706}_{j}}+e(j)_{\unicode[STIX]{x1D706}_{j}}$ for $0<j\leqslant \ell$ if $\ell$ is odd.

Remark. A Schubert variety in an orthogonal Grassmannian $\text{OG}(n-p,2n)$ is defined by a triple $\boldsymbol{\unicode[STIX]{x1D70F}}$ with all $p_{i}=p$ . To obtain all of these as degeneracy loci according to our setup, one should use two different maximal isotropic spaces in the reference flag $F_{\bullet }$ . In fact, given a complete isotropic flag, there is a unique maximal isotropic subspace $F_{1}\subset F_{0}^{\prime }\subset F_{\overline{1}}$ which is distinct from $F_{0}$ ; both $F_{0}$ and $F_{0}^{\prime }$ must be used to define Schubert varieties.

For example, consider $\boldsymbol{\unicode[STIX]{x1D70F}}=(1,\,p,\,0)$ , so $\unicode[STIX]{x1D706}=(p)$ . In the setup of [Reference Buch, Kresch and TamvakisBKT15], there are two Schubert varieties whose $p$ -strict partition is $\unicode[STIX]{x1D706}$ , given by $\dim (E_{p}\cap F_{0})\geqslant 1$ and $\dim (E_{p}\cap F_{0}^{\prime })\geqslant 1$ , respectively. (When $p=n-1$ , these are the two maximal linear spaces in the quadric.) The respective formulas are $c_{p}(V-E_{p}-F_{0})-e_{p}(E_{p},F_{0})$ and $c_{p}(V-E_{p}-F_{0}^{\prime })-e_{p}(E_{p},F_{0}^{\prime })$ . Note that $e_{p}(E_{p},F_{0}^{\prime })=-e_{p}(E_{p},F_{0})$ . (Compare [Reference Buch, Kresch and TamvakisBKT15, Example A.3].)