2.1 GM varieties

Let $V_{5}$ be a $5$ -dimensional vector space and $\mathbf{G}=\operatorname{Gr}(2,V_{5})$ the Grassmannian of $2$ -dimensional subspaces. Consider the Plücker embedding $\mathbf{G}{\hookrightarrow}\mathbf{P}(\wedge ^{2}V_{5})$ and let $\operatorname{Cone}(\mathbf{G})\subset \mathbf{P}(\mathbf{k}\oplus \wedge ^{2}V_{5})$ be the cone over $\mathbf{G}$ . Further, let

$$\begin{eqnarray}W\subset \mathbf{k}\oplus \wedge ^{2}V_{5}\end{eqnarray}$$

be a linear subspace of dimension $n+5$ with $2\leqslant n\leqslant 6$ , and $Q\subset \mathbf{P}(W)$ a quadric hypersurface. By Definition 1.1, if the intersection

(2.1) $$\begin{eqnarray}X=\operatorname{Cone}(\mathbf{G})\cap Q\end{eqnarray}$$

is smooth and transverse, then $X$ is a GM variety of dimension $n$ , and every GM variety can be written in this form.

There is a natural polarization $H$ on a GM variety $X$ , given by the restriction of the hyperplane class on $\mathbf{P}(\mathbf{k}\oplus \wedge ^{2}V_{5})$ ; we denote by ${\mathcal{O}}_{X}(1)$ the corresponding line bundle on $X$ . It is straightforward to check that

(2.2) $$\begin{eqnarray}H^{n}=10\quad \text{and}\quad -K_{X}=(n-2)H.\end{eqnarray}$$

Moreover, we have

(2.3) $$\begin{eqnarray}\begin{array}{@{}ll@{}}\text{if }\dim (X)\geqslant 3, & \text{then }\operatorname{Pic}(X)=\mathbf{Z}H,\quad \text{and}\\ \text{if }\dim (X)=2, & \text{then }(X,H)\text{ is a Brill–Noether general K3 surface}.\end{array}\end{eqnarray}$$

Conversely, by [Reference Debarre and KuznetsovDK18a, Theorem 2.16] any smooth projective polarized variety of dimension ${\geqslant}2$ satisfying (2.3) and (2.2) is a GM variety.

The intersection $\operatorname{Cone}(\mathbf{G})\cap Q$ does not contain the vertex of the cone, since $X$ is smooth. Hence projection from the vertex defines a regular map

$$\begin{eqnarray}f:X\rightarrow \mathbf{G},\end{eqnarray}$$

called the Gushel map. Let ${\mathcal{U}}$ be the rank $2$ tautological subbundle on $\mathbf{G}$ . Then ${\mathcal{U}}_{X}=f^{\ast }{\mathcal{U}}$ is a rank 2 vector bundle on $X$ , called the Gushel bundle. By [Reference Debarre and KuznetsovDK18a, § 2.1], the Gushel map and the Gushel bundle are canonically associated to $X$ , i.e. only depend on the abstract polarized variety $(X,H)$ and not on the particular realization (2.1). In particular, so is the space $V_{5}$ (being the dual of the space of sections of  ${\mathcal{U}}_{X}^{\vee }$ ), and we will sometimes write it as $V_{5}(X)$ to emphasize this. The space $W$ is also canonically associated to $X$ , since its dual is the space of global sections of the line bundle ${\mathcal{O}}_{X}(1)$ . The quadric $Q$ , however, is not canonically associated to $X$ , see § 3.1.

The intersection

$$\begin{eqnarray}M_{X}=\operatorname{Cone}(\mathbf{G})\cap \mathbf{P}(W)\end{eqnarray}$$

is called the Grassmannian hull of $X$ . Note that $X=M_{X}\cap Q$ is a quadric section of $M_{X}$ . Let  $W^{\prime }$ be the projection of $W$ to $\wedge ^{2}V_{5}$ . The intersection

(2.4) $$\begin{eqnarray}M_{X}^{\prime }=\mathbf{G}\cap \mathbf{P}(W^{\prime })\end{eqnarray}$$

is called the projected Grassmannian hull of $X$ . Again by [Reference Debarre and KuznetsovDK18a], both $M_{X}$ and $M_{X}^{\prime }$ are canonically associated to $X$ .

The Gushel map is either an embedding or a double covering of $M_{X}^{\prime }$ , according to whether the projection map $W\rightarrow W^{\prime }$ is an isomorphism or has $1$ -dimensional kernel. In the first case, $W\cong W^{\prime }$ and $M_{X}\cong M_{X}^{\prime }$ . Then considering $Q$ as a subvariety of $\mathbf{P}(W^{\prime })$ , we have

(2.5) $$\begin{eqnarray}X\cong M_{X}^{\prime }\cap Q.\end{eqnarray}$$

That is, $X$ is a quadric section of a linear section of the Grassmannian $\mathbf{G}$ . A GM variety of this type is called ordinary.

If the map $W\rightarrow W^{\prime }$ has $1$ -dimensional kernel, then we have $\mathbf{P}(W)=\operatorname{Cone}(\mathbf{P}(W^{\prime }))$ and $M_{X}=\operatorname{Cone}(M_{X}^{\prime })$ . As $Q$ does not contain the vertex of the cone (by smoothness of  $X$ ), projection from the vertex gives a double cover

(2.6) $$\begin{eqnarray}X\xrightarrow[{}]{~2:1~}M_{X}^{\prime }.\end{eqnarray}$$

That is, $X$ is a double cover of a linear section of the Grassmannian $\mathbf{G}$ . A GM variety of this type is called special.

By Lemma 2.1, if $X$ is special then $M_{X}^{\prime }$ is smooth. Further, by [Reference Debarre and KuznetsovDK18a, § 2.5] the branch divisor of the double cover (2.6) is the smooth intersection $X^{\prime }=\mathbf{G}\cap Q^{\prime }$ , where $Q^{\prime }=Q\cap \mathbf{P}(W^{\prime })$ is a quadric hypersurface in $\mathbf{P}(W^{\prime })$ . Hence, as long as $n\geqslant 3$ , the branch divisor $X^{\prime }$ of (2.6) is an ordinary GM variety of dimension $n-1$ . This gives rise to an operation taking a GM variety of one type to the opposite type, by defining in this situation

(2.7) $$\begin{eqnarray}X^{\text{op}}=X^{\prime }\quad \text{and}\quad (X^{\prime })^{\text{op}}=X.\end{eqnarray}$$

Note that we have $\dim X^{\text{op}}=\dim X\pm 1$ . The opposite GM variety is not defined for special GM surfaces.

2.2 Definition of GM categories

By the discussion in § 2.1, any GM variety $X$ of dimension $n\geqslant 3$ is obtained from a smooth linear section $M_{X}^{\prime }$ of $\mathbf{G}$ by taking a quadric section or a branched double cover. To describe a natural semiorthogonal decomposition of $\text{D}^{\text{b}}(X)$ , we first recall that $\mathbf{G}$ and its smooth linear sections of dimension at least  $3$ admit rectangular Lefschetz decompositions (in the sense of [Reference KuznetsovKuz07, § 4]) of their derived categories. Note that the bundles ${\mathcal{O}}_{\mathbf{G}},{\mathcal{U}}^{\vee }$ form an exceptional pair in $\text{D}^{\text{b}}(\mathbf{G})$ , where recall ${\mathcal{U}}$ denotes the tautological rank $2$ bundle. Let

(2.8) $$\begin{eqnarray}{\mathcal{B}}=\langle {\mathcal{O}}_{\mathbf{G}},{\mathcal{U}}^{\vee }\rangle \subset \text{D}^{\text{b}}(\mathbf{G})\end{eqnarray}$$

be the triangulated subcategory they generate. The following result holds by [Reference KuznetsovKuz06, § 6.1].

The next result gives a semiorthogonal decomposition of the derived category of a GM variety.

More explicitly, using the definition (2.8) of ${\mathcal{B}}$ , the defining semiorthogonal decomposition of a GM category ${\mathcal{A}}_{X}$ can be written as

(2.11) $$\begin{eqnarray}\text{D}^{\text{b}}(X)=\langle {\mathcal{A}}_{X},{\mathcal{O}}_{X},{\mathcal{U}}_{X}^{\vee },\ldots ,{\mathcal{O}}_{X}(n-3),{\mathcal{U}}_{X}^{\vee }(n-3)\rangle .\end{eqnarray}$$

The GM category ${\mathcal{A}}_{X}$ is the main object of study in this paper. As we will see below, its properties depend strongly on the parity of $\dim (X)$ . For this reason, we sometimes emphasize the parity of $\dim (X)$ by calling ${\mathcal{A}}_{X}$ an even or odd GM category according to whether $\dim (X)$ is even or odd.

2.3 Serre functors of GM categories

Recall from [Reference Bondal and KapranovBK90] that a Serre functor for a triangulated category ${\mathcal{T}}$ is an autoequivalence  $\text{S}_{{\mathcal{T}}}$ of  ${\mathcal{T}}$ with bifunctorial isomorphisms

$$\begin{eqnarray}\operatorname{Hom}({\mathcal{F}},\text{S}_{{\mathcal{T}}}({\mathcal{G}}))\cong \operatorname{Hom}({\mathcal{G}},{\mathcal{F}})^{\vee }\end{eqnarray}$$

for ${\mathcal{F}},{\mathcal{G}}\in {\mathcal{T}}$ . If a Serre functor exists, it is unique. If $X$ is a smooth proper variety, then $\text{D}^{\text{b}}(X)$ has a Serre functor given by the formula

(2.12) $$\begin{eqnarray}\text{S}_{\text{D}^{\text{b}}(X)}({\mathcal{F}})={\mathcal{F}}\otimes \unicode[STIX]{x1D714}_{X}[\dim X].\end{eqnarray}$$

Moreover, given an admissible subcategory ${\mathcal{A}}\subset {\mathcal{T}}$ , if ${\mathcal{T}}$ admits a Serre functor then so does  ${\mathcal{A}}$ . Using [Reference KuznetsovKuz16a], we can describe the Serre functor of a GM category.

Proposition 2.6 shows that even GM categories can be regarded as ‘non-commutative K3 surfaces’, and odd GM categories can be regarded as ‘non-commutative Enriques surfaces’. This analogy goes further than the relation between Serre functors. For instance, any Enriques surface (in characteristic $0$ ) is the quotient of a K3 surface by an involution. Similarly, the results of [Reference Kuznetsov and PerryKP17] show that odd GM categories can be described as ‘quotients’ of even GM categories by involutions. To state this precisely, recall from § 2.1 that unless $X$ is a special GM surface, there is an associated GM variety $X^{\text{op}}$ of the opposite type and parity of dimension. The following result is proved in [Reference Kuznetsov and PerryKP17, § 8.2].

Proposition 2.7. Let $X$ be a GM variety which is not a special GM surface. Then there is a  $\mathbf{Z}/2$ -action on the GM category  ${\mathcal{A}}_{X}$ such that if ${\mathcal{A}}_{X}^{\mathbf{Z}/2}$ denotes the equivariant category, then there is an equivalence

$$\begin{eqnarray}{\mathcal{A}}_{X}^{\mathbf{Z}/2}\simeq {\mathcal{A}}_{X^{\text{op}}}.\end{eqnarray}$$

If $\unicode[STIX]{x1D70E}$ is the autoequivalence generating the $\mathbf{Z}/2$ -action on ${\mathcal{A}}_{X}$ , then $\unicode[STIX]{x1D70E}$ is induced by the involution of the double covering $X\rightarrow M_{X}^{\prime }$ if $X$ is special, and $\unicode[STIX]{x1D70E}=\text{S}_{{\mathcal{A}}_{X}}\circ [-2]$ if $\dim (X)$ is odd.

2.4 Hochschild homology of GM categories

Given a suitably enhanced triangulated category ${\mathcal{A}}$ , there is an invariant $\operatorname{HH}_{\bullet }({\mathcal{A}})$ called its Hochschild homology, which is a graded $\mathbf{k}$ -vector space. We will exclusively be interested in admissible subcategories of the derived category of a smooth projective variety. For a definition of Hochschild homology in this context, see [Reference KuznetsovKuz09b].

If ${\mathcal{A}}=\text{D}^{\text{b}}(X)$ , we write $\operatorname{HH}_{\bullet }(X)$ for $\operatorname{HH}_{\bullet }({\mathcal{A}})$ . The Hochschild–Kostant–Rosenberg (HKR) isomorphism gives the following explicit description of Hochschild homology in this case [Reference MarkarianMar09]:

(2.13) $$\begin{eqnarray}\operatorname{HH}_{i}(X)\cong \bigoplus _{q-p=i}\text{H}^{q}(X,\unicode[STIX]{x1D6FA}_{X}^{p}).\end{eqnarray}$$

An important property of Hochschild homology is that it is additive under semiorthogonal decompositions.

Theorem 2.8 [Reference KuznetsovKuz09b, Theorem 7.3].

Let $X$ be a smooth projective variety. Given a semiorthogonal decomposition $\text{D}^{\text{b}}(X)=\langle {\mathcal{A}}_{1},{\mathcal{A}}_{2},\ldots ,{\mathcal{A}}_{m}\rangle$ , there is an isomorphism

$$\begin{eqnarray}\operatorname{HH}_{\bullet }(X)\cong \bigoplus _{i=1}^{m}\operatorname{HH}_{\bullet }({\mathcal{A}}_{i}).\end{eqnarray}$$

By combining this additivity property with the HKR isomorphism for GM varieties, we can compute the Hochschild homology of GM categories.

Proposition 2.9. Let $X$ be a GM variety of dimension $n$ . Then

$$\begin{eqnarray}\operatorname{HH}_{\bullet }({\mathcal{A}}_{X})\cong \left\{\begin{array}{@{}cccccl@{}}\mathbf{k}[2]\hspace{4.0pt} & \oplus \hspace{4.0pt} & \mathbf{k}^{22}\hspace{4.0pt} & \oplus \hspace{4.0pt} & \mathbf{k}[-2] & \text{if }n\text{ is even,}\\ \mathbf{k}^{10}[1]\hspace{4.0pt} & \oplus \hspace{4.0pt} & \mathbf{k}^{2}\hspace{4.0pt} & \oplus \hspace{4.0pt} & \mathbf{k}^{10}[-1] & \text{if }n\text{ is odd.}\end{array}\right.\end{eqnarray}$$

Proof. By (2.11) there is a semiorthogonal decomposition of $\text{D}^{\text{b}}(X)$ with ${\mathcal{A}}_{X}$ and  $2(n-2)$ exceptional objects as components. Since the category generated by an exceptional object is equivalent to the derived category of a point, its Hochschild homology is just $\mathbf{k}$ . Hence by additivity,

$$\begin{eqnarray}\operatorname{HH}_{\bullet }(X)\cong \operatorname{HH}_{\bullet }({\mathcal{A}}_{X})\oplus \mathbf{k}^{2(n-2)}.\end{eqnarray}$$

By (2.13) the graded dimension of $\operatorname{HH}_{\bullet }(X)$ can be computed by summing the columns of the Hodge diamond of $X$ , which looks as follows (see [Reference LogachevLog12, Reference Iliev and ManivelIM11, Reference NagelNag98, Reference Debarre and KuznetsovDK17]):

Now the lemma follows by inspection. ◻

2.5 Hochschild cohomology of GM categories

Given a suitably enhanced triangulated category ${\mathcal{A}}$ , there is also an invariant $\operatorname{HH}^{\bullet }({\mathcal{A}})$ called its Hochschild cohomology, which has the structure of a graded $\mathbf{k}$ -algebra. Again, for a definition in the case where ${\mathcal{A}}$ is an admissible subcategory of the derived category of a smooth projective variety, see [Reference KuznetsovKuz09b].

If ${\mathcal{A}}=\text{D}^{\text{b}}(X)$ , we write $\operatorname{HH}^{\bullet }(X)$ for $\operatorname{HH}^{\bullet }({\mathcal{A}})$ . There is the following version of the HKR isomorphism for Hochschild cohomology [Reference MarkarianMar09]:

(2.14) $$\begin{eqnarray}\operatorname{HH}^{i}(X)\cong \bigoplus _{p+q=i}\text{H}^{q}(X,\wedge ^{p}\text{T}_{X}).\end{eqnarray}$$

Hochschild cohomology is not additive under semiorthogonal decompositions, and so it is generally much harder to compute than Hochschild homology. There is, however, a case when the computation simplifies considerably. Recall that a triangulated category ${\mathcal{A}}$ is called $n$ -Calabi–Yau if the shift functor $[n]$ is a Serre functor for ${\mathcal{A}}$ .

Proposition 2.10 [Reference KuznetsovKuz16a, Proposition 5.2].

Let ${\mathcal{A}}$ be an admissible subcategory of $\text{D}^{\text{b}}(X)$ for a smooth projective variety $X$ . If ${\mathcal{A}}$ is an $n$ -Calabi–Yau category, then for each $i$ there is an isomorphism of vector spaces

$$\begin{eqnarray}\operatorname{HH}^{i}({\mathcal{A}})\cong \operatorname{HH}_{i-n}({\mathcal{A}}).\end{eqnarray}$$

This immediately applies to even GM categories, as by Proposition 2.6 they are $2$ -Calabi–Yau.

Corollary 2.11. Let $X$ be a GM variety of even dimension. Then

$$\begin{eqnarray}\operatorname{HH}^{\bullet }({\mathcal{A}}_{X})\cong \mathbf{k}\oplus \mathbf{k}^{22}[-2]\oplus \mathbf{k}[-4].\end{eqnarray}$$

The Hochschild cohomology of odd GM categories is significantly harder to compute. Our strategy is to exploit the fact that there is a $\mathbf{Z}/2$ -action on such a category, with invariants an even GM category. By the results of [Reference PerryPer18], this reduces us to a problem involving the Hochschild cohomology of an even GM category and the Hochschild homology of an odd GM category.

Proposition 2.12. Let $X$ be a GM variety of odd dimension. Then

$$\begin{eqnarray}\operatorname{HH}^{\bullet }({\mathcal{A}}_{X})\cong \mathbf{k}\oplus \mathbf{k}^{20}[-2]\oplus \mathbf{k}[-4].\end{eqnarray}$$

Proof. Recall that by Proposition 2.7 there is a $\mathbf{Z}/2$ -action on ${\mathcal{A}}_{X}$ such that if $\unicode[STIX]{x1D70E}:{\mathcal{A}}_{X}\rightarrow {\mathcal{A}}_{X}$ denotes the corresponding involutive autoequivalence, then:

1. (1) $\text{S}_{{\mathcal{A}}_{X}}=\unicode[STIX]{x1D70E}\circ [2]$ is a Serre functor for ${\mathcal{A}}_{X}$ ;

2. (2) ${\mathcal{A}}_{X}^{\mathbf{Z}/2}\simeq {\mathcal{A}}_{X^{\text{op}}}$ , where $X^{\text{op}}$ is the opposite variety to $X$ .

As stated, these are results at the level of triangulated categories, but they also hold at the enhanced level. Namely, in the terminology of [Reference PerryPer18], there is a $\mathbf{k}$ -linear stable $\infty$ -category $\text{D}^{\text{b}}(X)^{\text{enh}}$ (denoted $\text{Perf}(X)$ in [Reference PerryPer18]) with homotopy category $\text{D}^{\text{b}}(X)$ . The category $\text{D}^{\text{b}}(X)^{\text{enh}}$ admits a semiorthogonal decomposition of the same form as (2.10), which defines a $\mathbf{k}$ -linear stable $\infty$ -category ${\mathcal{A}}_{X}^{\text{enh}}$ whose homotopy category is ${\mathcal{A}}_{X}$ . If $\unicode[STIX]{x1D70E}^{\text{enh}}:{\mathcal{A}}_{X}^{\text{enh}}\rightarrow {\mathcal{A}}_{X}^{\text{enh}}$ denotes the corresponding involutive autoequivalence, then (1) and (2) above hold with ${\mathcal{A}}_{X}$ , $\text{S}_{{\mathcal{A}}_{X}}$ , $\unicode[STIX]{x1D70E}$ , and  ${\mathcal{A}}_{X^{\text{op}}}$ replaced by their enhanced versions, and (1) and (2) are recovered by passing to homotopy categories. The Hochschild (co)homology of ${\mathcal{A}}_{X}$ and ${\mathcal{A}}_{X^{\text{op}}}$ agree with the corresponding invariants of their enhancements. Hence [Reference PerryPer18, Corollary 1.3] gives

(2.15) $$\begin{eqnarray}\operatorname{HH}^{\bullet }({\mathcal{A}}_{X^{\text{op}}})\cong \operatorname{HH}^{\bullet }({\mathcal{A}}_{X})\oplus (\operatorname{HH}_{\bullet }({\mathcal{A}}_{X})^{\mathbf{Z}/2}[-2]),\end{eqnarray}$$

where the $\mathbf{Z}/2$ -action on $\operatorname{HH}_{\bullet }({\mathcal{A}}_{X})$ is induced by $\unicode[STIX]{x1D70E}$ .

Since $X$ has odd dimension (and hence $X^{\text{op}}$ has even dimension), by Corollary 2.11 we have

$$\begin{eqnarray}\operatorname{HH}^{\bullet }({\mathcal{A}}_{X^{\text{op}}})\cong \mathbf{k}\oplus \mathbf{k}^{22}[-2]\oplus \mathbf{k}[-4],\end{eqnarray}$$

and by Proposition 2.9 we have

$$\begin{eqnarray}\operatorname{HH}_{\bullet }({\mathcal{A}}_{X})\cong \mathbf{k}^{10}[1]\oplus \mathbf{k}^{2}\oplus \mathbf{k}^{10}[-1].\end{eqnarray}$$

Combined with (2.15), this immediately shows $\operatorname{HH}_{\bullet }({\mathcal{A}}_{X})^{\mathbf{Z}/2}$ is concentrated in degree $0$ , i.e. we have $\operatorname{HH}_{\bullet }({\mathcal{A}}_{X})^{\mathbf{Z}/2}\cong \mathbf{k}^{d}$ for some $0\leqslant d\leqslant 2$ , and

$$\begin{eqnarray}\operatorname{HH}^{\bullet }({\mathcal{A}}_{X})\cong \mathbf{k}\oplus \mathbf{k}^{22-d}[-2]\oplus \mathbf{k}[-4].\end{eqnarray}$$

To prove $d=2$ , we apply [Reference PolishchukPol14, Corollary 3.11], which gives an equality

(2.16) $$\begin{eqnarray}\mathop{\sum }_{i}(-1)^{i}\dim \operatorname{HH}^{i}({\mathcal{A}}_{X})=\mathop{\sum }_{i}(-1)^{i}\,\text{Tr}((\text{S}_{{\mathcal{A}}_{X}}^{-1})_{\ast }:\operatorname{HH}_{i}({\mathcal{A}}_{X})\rightarrow \operatorname{HH}_{i}({\mathcal{A}}_{X})).\end{eqnarray}$$

Note that since $\text{S}_{{\mathcal{A}}_{X}}=\unicode[STIX]{x1D70E}\circ [2]$ , the map $(\text{S}_{{\mathcal{A}}_{X}}^{-1})_{\ast }:\operatorname{HH}_{i}({\mathcal{A}}_{X})\rightarrow \operatorname{HH}_{i}({\mathcal{A}}_{X})$ induced by $\text{S}_{{\mathcal{A}}_{X}}^{-1}$ on Hochschild homology coincides with the map induced by $\unicode[STIX]{x1D70E}$ , and in particular squares to the identity. It follows that the right side of (2.16) is bounded above by $\sum _{i}\dim \operatorname{HH}_{i}({\mathcal{A}}_{X})=22$ . But the left side of (2.16) equals $24-d$ where $0\leqslant d\leqslant 2$ , so $d=2$ .◻

As an application, we discuss the indecomposability of GM categories. Recall that a triangulated category ${\mathcal{T}}$ is called indecomposable if it admits no non-trivial semiorthogonal decompositions, i.e. if ${\mathcal{T}}=\langle {\mathcal{A}}_{1},{\mathcal{A}}_{2}\rangle$ implies either ${\mathcal{A}}_{1}\simeq 0$ or ${\mathcal{A}}_{2}\simeq 0$ . In general, there are very few techniques for proving indecomposability of a triangulated category. However, for Calabi–Yau categories, we recall a simple criterion below.

If ${\mathcal{A}}$ is an admissible subcategory of the derived category of a smooth projective variety, we say ${\mathcal{A}}$ is connected if $\operatorname{HH}^{0}({\mathcal{A}})=\mathbf{k}$ (see [Reference KuznetsovKuz16a, § 5.2]). By Corollary 2.11 and Proposition 2.12, all GM categories are connected.

2.6 Grothendieck groups of GM categories

The Grothendieck group $\text{K}_{0}({\mathcal{T}})$ of a triangulated category ${\mathcal{T}}$ is the free group on isomorphism classes $[{\mathcal{F}}]$ of objects ${\mathcal{F}}\in {\mathcal{T}}$ , modulo the relations $[{\mathcal{G}}]=[{\mathcal{F}}]+[{\mathcal{H}}]$ for every distinguished triangle ${\mathcal{F}}\rightarrow {\mathcal{G}}\rightarrow {\mathcal{H}}$ .

Assume ${\mathcal{T}}$ is proper, i.e. that $\bigoplus _{i}\operatorname{Hom}({\mathcal{F}},{\mathcal{G}}[i])$ is finite dimensional for all ${\mathcal{F}},{\mathcal{G}}\in {\mathcal{T}}$ . For instance, this holds if ${\mathcal{T}}$ is admissible in the derived category of a smooth projective variety. Then for ${\mathcal{F}},{\mathcal{G}}\in {\mathcal{T}}$ , we set

$$\begin{eqnarray}\unicode[STIX]{x1D712}({\mathcal{F}},{\mathcal{G}})=\mathop{\sum }_{i}(-1)^{i}\dim \operatorname{Hom}({\mathcal{F}},{\mathcal{G}}[i]).\end{eqnarray}$$

This descends to a bilinear form $\unicode[STIX]{x1D712}:\text{K}_{0}({\mathcal{T}})\times \text{K}_{0}({\mathcal{T}})\rightarrow \mathbf{Z}$ , called the Euler form. In general this form is neither symmetric nor antisymmetric. However, if ${\mathcal{T}}$ admits a Serre functor (e.g. if ${\mathcal{T}}$ is admissible in the derived category of a smooth projective variety), then the left and right kernels of the form $\unicode[STIX]{x1D712}$ agree, and we denote this common subgroup of $\text{K}_{0}({\mathcal{T}})$ by $\ker (\unicode[STIX]{x1D712})$ . In this situation, the numerical Grothendieck group is the quotient

$$\begin{eqnarray}\text{K}_{0}({\mathcal{T}})_{\text{num}}=\text{K}_{0}({\mathcal{T}})/\text{ker}\,(\unicode[STIX]{x1D712}).\end{eqnarray}$$

Note that $\text{K}_{0}({\mathcal{T}})_{\text{num}}$ is torsion free, since $\ker (\unicode[STIX]{x1D712})$ is evidently saturated.

If $X$ is a smooth projective variety, we write

$$\begin{eqnarray}\text{K}_{0}(X)=\text{K}_{0}(\text{D}^{\text{b}}(X))\quad \text{and}\quad \text{K}_{0}(X)_{\text{num}}=\text{K}_{0}(\text{D}^{\text{b}}(X))_{\text{num}}.\end{eqnarray}$$

Further, let $\text{CH}(X)$ and $\text{CH}(X)_{\text{num}}$ denote the Chow rings of cycles modulo rational and numerical equivalence. The following well-known consequence of Hirzebruch–Riemann–Roch relates the (numerical) Grothendieck ring of $X$ to its (numerical) Chow ring.

Lemma 2.19. Let $X$ be a smooth projective variety. Then there are isomorphisms

$$\begin{eqnarray}\text{K}_{0}(X)\otimes \mathbf{Q}\cong \text{CH}(X)\otimes \mathbf{Q}\quad \text{and}\quad \text{K}_{0}(X)_{\text{num}}\otimes \mathbf{Q}\cong \text{CH}(X)_{\text{num}}\otimes \mathbf{Q}.\end{eqnarray}$$

The following well-known lemma says that Grothendieck groups are additive.

Lemma 2.20. Let $X$ be a smooth projective variety. Given a semiorthogonal decomposition $\text{D}^{\text{b}}(X)=\langle {\mathcal{A}}_{1},{\mathcal{A}}_{2},\ldots ,{\mathcal{A}}_{m}\rangle$ , there are isomorphisms

$$\begin{eqnarray}\text{K}_{0}(X)\cong \bigoplus _{i=1}^{m}\text{K}_{0}({\mathcal{A}}_{i})\quad \text{and}\quad \text{K}_{0}(X)_{\text{num}}\cong \bigoplus _{i=1}^{m}\text{K}_{0}({\mathcal{A}}_{i})_{\text{num}}.\end{eqnarray}$$

Now let $X$ be a GM variety. If $X$ is a surface then ${\mathcal{A}}_{X}=\text{D}^{\text{b}}(X)$ , so the Grothendieck group of ${\mathcal{A}}_{X}$ coincides with that of $X$ . Below we describe $\text{K}_{0}({\mathcal{A}}_{X})_{\text{num}}$ if $X$ is odd dimensional, or if  $X$ is a fourfold or sixfold which is not ‘Hodge-theoretically special’ in the following sense.

First, we note that if $n$ denotes the dimension of $X$ , then by Lefschetz theorems (see [Reference Debarre and KuznetsovDK17, Proposition 3.4(b)]) the Gushel map $f:X\rightarrow \mathbf{G}$ induces an injection

$$\begin{eqnarray}\text{H}^{n}(\mathbf{G},\mathbf{Q}){\hookrightarrow}\text{H}^{n}(X,\mathbf{Q}).\end{eqnarray}$$

If $n$ is odd, then $\text{H}^{n}(\mathbf{G},\mathbf{Q})$ simply vanishes. But if $n=4$ or $6$ , then $\text{H}^{n}(\mathbf{G},\mathbf{Q})=\mathbf{Q}^{2}$ is generated by Schubert cycles, and the vanishing cohomology $\text{H}_{\text{van}}^{n}(X,\mathbf{Q})$ is defined as the orthogonal to  $\text{H}^{n}(\mathbf{G},\mathbf{Q})\subset \text{H}^{n}(X,\mathbf{Q})$ with respect to the intersection form.

Definition 2.21 [Reference Debarre, Iliev and ManivelDIM15].

Let $X$ be a GM variety of dimension $n=4$ or $6$ . Then $X$ is Hodge-special if

$$\begin{eqnarray}\text{H}^{n/2,n/2}(X)\cap \text{H}_{\text{van}}^{n}(X,\mathbf{Q})\neq 0.\end{eqnarray}$$

Proof. The proof is similar to that of Proposition 2.9. First, note that by Proposition 2.3 there is a semiorthogonal decomposition of $\text{D}^{\text{b}}(X)$ with ${\mathcal{A}}_{X}$ and $2(n-2)$ exceptional objects as components. Since the category generated by an exceptional object is equivalent to the derived category of a point, both its usual and numerical Grothendieck group is $\mathbf{Z}$ . Hence by additivity,

$$\begin{eqnarray}\text{K}_{0}(X)_{\text{num}}\cong \text{K}_{0}({\mathcal{A}}_{X})_{\text{num}}\oplus \mathbf{Z}^{2(n-2)}.\end{eqnarray}$$

On the other hand, $\text{K}_{0}(X)_{\text{num}}\otimes \mathbf{Q}\cong \text{CH}(X)_{\text{num}}\otimes \mathbf{Q}$ . But under our assumptions on $X$ , the rational Hodge classes on $X$ are spanned by the restrictions of Schubert cycles on $\mathbf{G}$ . In particular, the Hodge conjecture holds for $X$ . So numerical equivalence coincides with homological equivalence, and

$$\begin{eqnarray}\textstyle \text{CH}(X)_{\text{num}}\otimes \mathbf{Q}\cong \bigoplus _{k}\text{H}^{k,k}(X,\mathbf{Q}),\end{eqnarray}$$

where $\text{H}^{k,k}(X,\mathbf{Q})=\text{H}^{k,k}(X)\,\cap \,\text{H}^{2k}(X,\mathbf{Q})$ . Thus using the Hodge diamond of $X$ (recorded in the proof of Proposition 2.9) and the assumption that $X$ is not Hodge-special if $n$ is even, we find

$$\begin{eqnarray}\dim (\text{K}_{0}(X)_{\text{num}}\otimes \mathbf{Q})=2n-2.\end{eqnarray}$$

Combined with the above, this shows the rank of $\text{K}_{0}({\mathcal{A}}_{X})_{\text{num}}$ is $2$ . Since $\text{K}_{0}({\mathcal{A}}_{X})_{\text{num}}$ is torsion free, we conclude $\text{K}_{0}({\mathcal{A}}_{X})_{\text{num}}\cong \mathbf{Z}^{2}$ .◻

Remark 2.26. Let $X$ be a GM variety of dimension $n=4$ or $6$ . The proof of the proposition shows that

$$\begin{eqnarray}\text{rank}(\text{K}_{0}({\mathcal{A}}_{X})_{\text{num}})=\dim _{\mathbf{Q}}\text{H}^{n/2,n/2}(X,\mathbf{Q})\end{eqnarray}$$

if the Hodge conjecture holds for $X$ . The Hodge conjecture holds for any uniruled smooth projective fourfold [Reference Conte and MurreCM78], so for $n=4$ the above equality is unconditional. If $n=6$ the Hodge conjecture can be proved using the correspondences studied in [Reference Debarre and KuznetsovDK17], but we do not discuss the details here.

Lemma 2.27. Let $X$ be a GM variety as in Proposition 2.25. Then in a suitable basis, the Euler form on $\text{K}_{0}({\mathcal{A}}_{X})_{\text{num}}=\mathbf{Z}^{2}$ is given by

$$\begin{eqnarray}\left(\begin{array}{@{}cc@{}}-1 & 0\\ 0 & -1\end{array}\right)\quad \text{if }n=3,\quad \left(\begin{array}{@{}cc@{}}-2 & 0\\ 0 & -2\end{array}\right)\quad \text{if }n=4.\end{eqnarray}$$

Proof. For $n=3$ , this is shown in the proof of [Reference KuznetsovKuz09a, Proposition 3.9].

For $n=4$ , we sketch the proof. First, note that any GM variety contains a line, since by taking a hyperplane section we reduce to the case of dimension $3$ , where the result is well known. Let $P\in X$ be a point, $L\subset X$ be a line, $\unicode[STIX]{x1D6F4}$ be the zero locus of a regular section of  ${\mathcal{U}}_{X}^{\vee }$ , $S$ be a complete intersection of two hyperplanes in $X$ , and $H$ be a hyperplane section of $X$ . The key claim is that

(2.17) $$\begin{eqnarray}\text{K}_{0}(X)_{\text{num}}=\mathbf{Z}\langle [{\mathcal{O}}_{P}],[{\mathcal{O}}_{L}],[{\mathcal{O}}_{\unicode[STIX]{x1D6F4}}],[{\mathcal{O}}_{S}],[{\mathcal{O}}_{H}],[{\mathcal{O}}_{X}]\rangle ,\end{eqnarray}$$

i.e. the structure sheaves of these subvarieties give an integral basis of $\text{K}_{0}(X)_{\text{num}}$ . Once this is known, as in the proof of [Reference KuznetsovKuz09a, Proposition 3.9], the lemma reduces to a (tedious) computation, which we omit.

Using [Reference KuznetsovKuz09a, Remark 5.9] it is easy to see $X$ is AK-compatible in the sense of [Reference KuznetsovKuz09a, Definition 5.1], hence to prove the claim it is enough to show that

$$\begin{eqnarray}\text{CH}(X)_{\text{num}}=\mathbf{Z}\langle [P],[L],[\unicode[STIX]{x1D6F4}],[S],[H],1\rangle .\end{eqnarray}$$

Clearly, this is equivalent to $\text{CH}^{2}(X)_{\text{num}}=\mathbf{Z}\langle [\unicode[STIX]{x1D6F4}],[S]\rangle$ . But $\text{CH}^{2}(X)_{\text{num}}$ coincides with the group $\text{CH}^{2}(X)_{\hom }\subset \text{H}^{4}(X,\mathbf{Z})$ of $2$ -cycles modulo homological equivalence (see the proof of Proposition 2.25), and $\mathbf{Z}\langle [\unicode[STIX]{x1D6F4}],[S]\rangle$ is the image of the inclusion $\text{H}^{4}(\mathbf{G},\mathbf{Z}){\hookrightarrow}\text{CH}^{2}(X)_{\hom }$ . Hence it suffices to show the cokernel of this inclusion is torsion free. Even better, the cokernel of

$$\begin{eqnarray}\text{H}^{4}(\mathbf{G},\mathbf{Z}){\hookrightarrow}\text{H}^{4}(X,\mathbf{Z})\end{eqnarray}$$

is torsion free. Indeed, we may assume $X$ is ordinary, and then the statement holds by the proof of the Lefschetz hyperplane theorem, see [Reference LazarsfeldLaz04, Example 3.1.18].◻

2.7 Geometricity of GM categories

Now we consider the question of whether ${\mathcal{A}}_{X}$ is equivalent to the derived category of a variety. The following two results show that in almost all cases, the answer is negative. In § 3.3 we will discuss a related conjecture about the rationality of GM fourfolds.

Proof. Suppose $S$ is a variety such that ${\mathcal{A}}_{X}\simeq \text{D}^{\text{b}}(S)$ . Then $S$ is smooth by [Reference KuznetsovKuz06, Lemma D.22], and proper by [Reference OrlovOrl16, Proposition 3.30]. In particular, $\text{D}^{\text{b}}(S)$ has a Serre functor given by

$$\begin{eqnarray}\text{S}_{\text{D}^{\text{b}}(S)}({\mathcal{F}})={\mathcal{F}}\otimes \unicode[STIX]{x1D714}_{S}[\dim (S)],\end{eqnarray}$$

which is unique up to isomorphism. Thus by Proposition 2.6, $S$ is a surface with trivial (if  $n$ is even) or $2$ -torsion (if $n$ is odd) canonical class. Hence $S$ is a K3, Enriques, abelian, or bielliptic surface. Using the HKR isomorphism and the Hodge diamonds of such surfaces, we find

$$\begin{eqnarray}\operatorname{HH}_{\bullet }(S)=\left\{\begin{array}{@{}cccccccccl@{}} & & \mathbf{k}[2] & \oplus & \mathbf{k}^{22} & \oplus & \mathbf{k}[-2] & & & \quad \text{if }S\text{ is K3},\\ & & & & \mathbf{k}^{12} & & & & & \quad \text{if }S\text{ is Enriques},\\ \mathbf{k}[2] & \oplus & \mathbf{k}^{4}[1] & \oplus & \mathbf{k}^{6} & \oplus & \mathbf{k}^{4}[-1] & \oplus & \mathbf{k}[-2] & \quad \text{if }S\text{ is abelian},\\ & & \mathbf{k}^{2}[1] & \oplus & \mathbf{k}^{4} & \oplus & \mathbf{k}^{2}[-1] & & & \quad \text{if }S\text{ is bielliptic}.\end{array}\right.\end{eqnarray}$$

Now parts (1) and (2) follow by comparing with $\operatorname{HH}_{\bullet }({\mathcal{A}}_{X})$ as given by Proposition 2.9. For (3) note that if ${\mathcal{A}}_{X}\simeq \text{D}^{\text{b}}(S)$ , then $\text{K}_{0}({\mathcal{A}}_{X})_{\text{num}}\cong \text{K}_{0}(S)_{\text{num}}$ . But on a projective surface powers of the hyperplane class give $3$ independent elements in  $\text{CH}(S)_{\text{num}}\otimes \mathbf{Q}\cong \text{K}_{0}(S)_{\text{num}}\otimes \mathbf{Q}$ . Hence by Proposition 2.25, $X$ is Hodge-special.◻

2.8 Self-duality of GM categories

The derived category of a smooth proper variety  $X$ is self-dual: if $\text{D}^{\text{b}}(X)^{\text{op}}$ denotes the opposite category of $\text{D}^{\text{b}}(X)$ (note that this has nothing to do with the opposite GM variety), there is an equivalence $\text{D}^{\text{b}}(X)\simeq \text{D}^{\text{b}}(X)^{\text{op}}$ given by the dualization functor ${\mathcal{F}}\mapsto \text{R}{\mathcal{H}}\mathit{om}({\mathcal{F}},{\mathcal{O}}_{X})$ . In general, this self-duality property is not inherited by semiorthogonal components of  $\text{D}^{\text{b}}(X)$ . Nonetheless, we show below that all GM categories are self-dual, which can be thought of as a weak geometricity property.

For the proof, we recall some facts about mutation functors (see [Reference BondalBon89, Reference Bondal and KapranovBK90] for more details). For any admissible subcategory ${\mathcal{A}}\subset {\mathcal{T}}$ of a triangulated category, there are associated left and right mutation functors $\text{L}_{{\mathcal{A}}}:{\mathcal{T}}\rightarrow {\mathcal{T}}$ and $\text{R}_{{\mathcal{A}}}:{\mathcal{T}}\rightarrow {\mathcal{T}}$ . These functors annihilate  ${\mathcal{A}}$ , and their restrictions $\text{L}_{{\mathcal{A}}}|_{^{\bot }{\mathcal{A}}}:^{\bot }{\mathcal{A}}\rightarrow {\mathcal{A}}^{\bot }$ and $\text{R}_{{\mathcal{A}}}|_{{\mathcal{A}}^{\bot }}:{\mathcal{A}}^{\bot }\rightarrow ^{\bot }{\mathcal{A}}$ are mutually inverse equivalences [Reference Bondal and KapranovBK90, Lemma 1.9]. If ${\mathcal{T}}=\langle {\mathcal{A}}_{1},\ldots ,{\mathcal{A}}_{n}\rangle$ is a semiorthogonal decomposition with admissible components, then for $1\leqslant i\leqslant n-1$ there are semiorthogonal decompositions

$$\begin{eqnarray}\displaystyle {\mathcal{T}} & = & \displaystyle \langle {\mathcal{A}}_{1},\ldots ,{\mathcal{A}}_{i-1},\text{L}_{{\mathcal{A}}_{i}}({\mathcal{A}}_{i+1}),{\mathcal{A}}_{i},{\mathcal{A}}_{i+2},\ldots ,{\mathcal{A}}_{n}\rangle ,\nonumber\\ \displaystyle {\mathcal{T}} & = & \displaystyle \langle {\mathcal{A}}_{1},\ldots ,{\mathcal{A}}_{i-1},{\mathcal{A}}_{i+1},\text{R}_{{\mathcal{A}}_{i+1}}({\mathcal{A}}_{i}),{\mathcal{A}}_{i+2},\ldots ,{\mathcal{A}}_{n}\rangle ,\nonumber\end{eqnarray}$$

and equivalences

(2.18) $$\begin{eqnarray}\text{L}_{{\mathcal{A}}_{i}}({\mathcal{A}}_{i+1})\simeq {\mathcal{A}}_{i+1}\quad \text{and}\quad \text{R}_{{\mathcal{A}}_{i+1}}({\mathcal{A}}_{i})\simeq {\mathcal{A}}_{i}\end{eqnarray}$$

induced by the mutation functors $\text{L}_{{\mathcal{A}}_{i}}:{\mathcal{T}}\rightarrow {\mathcal{T}}$ and $\text{R}_{{\mathcal{A}}_{i+1}}:{\mathcal{T}}\rightarrow {\mathcal{T}}$ . When ${\mathcal{T}}$ admits a Serre functor $\text{S}_{{\mathcal{T}}}$ , the effect of mutating ${\mathcal{A}}_{n}$ or ${\mathcal{A}}_{1}$ to the opposite side of the semiorthogonal decomposition of  ${\mathcal{T}}$ can be described as follows [Reference Bondal and KapranovBK90, Proposition 3.6]:

(2.19) $$\begin{eqnarray}{\mathcal{T}}=\langle \text{S}_{{\mathcal{T}}}({\mathcal{A}}_{n}),{\mathcal{A}}_{1},\ldots ,{\mathcal{A}}_{n-1}\rangle \quad \text{and}\quad {\mathcal{T}}=\langle {\mathcal{A}}_{2},\ldots ,{\mathcal{A}}_{n},\text{S}_{{\mathcal{T}}}^{-1}({\mathcal{A}}_{1})\rangle .\end{eqnarray}$$

That is, $\text{L}_{\langle {\mathcal{A}}_{1},\ldots ,{\mathcal{A}}_{n-1}\rangle }({\mathcal{A}}_{n})=\text{S}_{{\mathcal{T}}}({\mathcal{A}}_{n})$ and $\text{R}_{\langle {\mathcal{A}}_{2},\ldots ,{\mathcal{A}}_{n}\rangle }({\mathcal{A}}_{1})=\text{S}_{{\mathcal{T}}}^{-1}({\mathcal{A}}_{1})$ .

Lemma 2.30. For any GM variety $X$ the corresponding GM category ${\mathcal{A}}_{X}$ is self-dual, i.e.

$$\begin{eqnarray}{\mathcal{A}}_{X}\simeq {\mathcal{A}}_{X}^{\text{op}}.\end{eqnarray}$$

Proof. If $\dim (X)=2$ then ${\mathcal{A}}_{X}=\text{D}^{\text{b}}(X)$ , so the result holds by self-duality of $\text{D}^{\text{b}}(X)$ . Now assume $\dim (X)\geqslant 3$ . Applying the dualization functor ${\mathcal{F}}\mapsto \text{R}{\mathcal{H}}\mathit{om}({\mathcal{F}},{\mathcal{O}}_{X})$ to the semiorthogonal decomposition (2.11), we obtain a new semiorthogonal decomposition

(2.20) $$\begin{eqnarray}\text{D}^{\text{b}}(X)=\langle {\mathcal{U}}_{X}(-(n-3)),{\mathcal{O}}_{X}(-(n-3)),\ldots ,{\mathcal{U}}_{X},{\mathcal{O}}_{X},{\mathcal{A}}_{X}^{\prime }\rangle\end{eqnarray}$$

and an equivalence ${\mathcal{A}}_{X}^{\prime }\simeq {\mathcal{A}}_{X}^{\text{op}}$ . It remains to show

(2.21) $$\begin{eqnarray}{\mathcal{A}}_{X}^{\prime }\simeq {\mathcal{A}}_{X}.\end{eqnarray}$$

We mutate the subcategory $\langle {\mathcal{U}}_{X}(-(n-3)),{\mathcal{O}}_{X}(-(n-3)),\ldots ,{\mathcal{U}}_{X}\rangle$ to the far right side of (2.20). By (2.19), the formula (2.12) for the Serre functor of $\text{D}^{\text{b}}(X)$ , and the formula (2.2) for $-K_{X}$ , the result is

$$\begin{eqnarray}\text{D}^{\text{b}}(X)=\langle {\mathcal{O}}_{X},{\mathcal{A}}_{X}^{\prime },{\mathcal{U}}_{X}(1),{\mathcal{O}}_{X}(1),\ldots ,{\mathcal{U}}_{X}(n-2)\rangle .\end{eqnarray}$$

Using the isomorphism ${\mathcal{U}}_{X}(1)\cong {\mathcal{U}}_{X}^{\vee }$ and comparing this decomposition with (2.11), we deduce that ${\mathcal{A}}_{X}=\text{L}_{{\mathcal{O}}_{X}}({\mathcal{A}}_{X}^{\prime })$ . Hence ${\mathcal{A}}_{X}\simeq {\mathcal{A}}_{X}^{\prime }$ by (2.18).◻

3.1 EPW sextics and moduli of GM varieties

Let $V_{6}$ be a $6$ -dimensional vector space. Its exterior power $\wedge ^{3}V_{6}$ has a natural $\det (V_{6})$ -valued symplectic form, given by wedge product. For any Lagrangian subspace $\mathsf{A}\subset \wedge ^{3}V_{6}$ , we consider the following stratification of $\mathbf{P}(V_{6})$ :

$$\begin{eqnarray}\mathsf{Y}_{\mathsf{A}}^{{\geqslant}k}=\{v\in \mathbf{P}(V_{6})\mid \dim (\mathsf{A}\cap (v\wedge (\wedge ^{2}V_{6})))\geqslant k\}\subset \mathbf{P}(V_{6}).\end{eqnarray}$$

We write $\mathsf{Y}_{\mathsf{A}}^{k}$ for the complement of $\mathsf{Y}_{\mathsf{A}}^{{\geqslant}k+1}$ in $\mathsf{Y}_{\mathsf{A}}^{{\geqslant}k}$ , and $\mathsf{Y}_{\mathsf{A}}$ for $\mathsf{Y}_{\mathsf{A}}^{{\geqslant}1}$ . The variety $\mathsf{Y}_{\mathsf{A}}$ is called an EPW sextic (for Eisenbud, Popescu, and Walter, who first defined it), and the sequence  $\mathsf{Y}_{\mathsf{A}}^{k}$ is called the EPW stratification.

We say $\mathsf{A}$ has no decomposable vectors if $\mathbf{P}(\mathsf{A})$ does not intersect $\operatorname{Gr}(3,V_{6})\subset \mathbf{P}(\wedge ^{3}V_{6})$ . O’Grady [Reference O’GradyO’Gr06, Reference O’GradyO’Gr08, Reference O’GradyO’Gr16, Reference O’GradyO’Gr12, Reference O’GradyO’Gr13, Reference O’GradyO’Gr15] extensively investigated the geometry of EPW sextics, and proved in particular that (see also [Reference Debarre and KuznetsovDK18a, Theorem B.2]) if  $\mathsf{A}$ has no decomposable vectors, then:

• ∙ $\mathsf{Y}_{\mathsf{A}}=\mathsf{Y}_{\mathsf{A}}^{{\geqslant}1}$ is a normal irreducible sextic hypersurface, smooth along $\mathsf{Y}_{\mathsf{A}}^{1}$ ;

• ∙ $\mathsf{Y}_{\mathsf{A}}^{{\geqslant}2}=\operatorname{Sing}(\mathsf{Y}_{\mathsf{A}})$ is a normal irreducible surface of degree 40, smooth along $\mathsf{Y}_{\mathsf{A}}^{2}$ ;

• ∙ $\mathsf{Y}_{\mathsf{A}}^{3}=\operatorname{Sing}(\mathsf{Y}_{\mathsf{A}}^{{\geqslant}2})$ is finite and reduced, and for general $\mathsf{A}$ is empty;

• ∙ $\mathsf{Y}_{\mathsf{A}}^{{\geqslant}4}=\varnothing$ .

For any Lagrangian subspace $\mathsf{A}\subset \wedge ^{3}V_{6}$ , its orthogonal $\mathsf{A}^{\bot }=\ker (\wedge ^{3}V_{6}^{\vee }\rightarrow \mathsf{A}^{\vee })\subset \wedge ^{3}V_{6}^{\vee }$ is also Lagrangian, and $\mathsf{A}$ has no decomposable vectors if and only if the same is true for $\mathsf{A}^{\bot }$ . In particular, $\mathsf{A}^{\bot }$ gives rise to an EPW sequence of subvarieties of $\mathbf{P}(V_{6}^{\vee })$ , which can be written in terms of $\mathsf{A}$ as follows:

$$\begin{eqnarray}\mathsf{Y}_{\mathsf{A}^{\bot }}^{{\geqslant}k}=\{V_{5}\in \mathbf{P}(V_{6}^{\vee })\mid \dim (\mathsf{A}\cap \wedge ^{3}V_{5})\geqslant k\}\subset \mathbf{P}(V_{6}^{\vee }).\end{eqnarray}$$

This stratification has the same properties as the stratification $\mathsf{Y}_{\mathsf{A}}^{{\geqslant}k}$ . By O’Grady’s work $\mathsf{Y}_{\mathsf{A}^{\bot }}$ is projectively dual to $\mathsf{Y}_{\mathsf{A}}$ , and for this reason is called the dual EPW sextic to $\mathsf{Y}_{\mathsf{A}}$ . We note that  $\mathsf{Y}_{\mathsf{A}^{\bot }}$ is not isomorphic to $\mathsf{Y}_{\mathsf{A}}$ for general $\mathsf{A}$ (see [Reference O’GradyO’Gr08, Theorem 1.1]).

One of the main results of [Reference Debarre and KuznetsovDK18a] is the following description of the set of all isomorphism classes of smooth ordinary GM varieties. If $X\subset \mathbf{P}(W)$ is a GM variety, then the space of quadrics in $\mathbf{P}(W)$ containing $X$ is a $6$ -dimensional vector space [Reference Debarre and KuznetsovDK18a, Theorem 2.3], which we denote by $V_{6}(X)$ . The space of Plücker quadrics defining the Grassmannian $\mathbf{G}=\operatorname{Gr}(2,V_{5}(X))$ is canonically identified with $V_{5}(X)$ , so since $X\subset \operatorname{Cone}(\mathbf{G})$ we have an embedding

$$\begin{eqnarray}V_{5}(X)\subset V_{6}(X).\end{eqnarray}$$

The hyperplane $V_{5}(X)$ is called the Plücker hyperplane of $X$ and the corresponding point

$$\begin{eqnarray}\mathbf{p}_{X}\in \mathbf{P}(V_{6}(X)^{\vee })\end{eqnarray}$$

is called the Plücker point of $X$ . Furthermore, in [Reference Debarre and KuznetsovDK18a, Theorem 3.10] it is shown that there is a natural Lagrangian subspace

$$\begin{eqnarray}\mathsf{A}(X)\subset \wedge ^{3}V_{6}(X)\end{eqnarray}$$

associated to $X$ . If $X^{\text{op}}$ is the opposite variety of $X$ as defined by (2.7), then $\mathsf{A}(X^{\text{op}})=\mathsf{A}(X)$ and $\mathbf{p}_{X^{\text{op}}}=\mathbf{p}_{X}$ .

Note that by Lemma 2.1, the Grassmannian hull $M_{X}$ is automatically smooth for ordinary GM varieties of dimension $n\geqslant 3$ .

The above discussion shows the utility of the EPW stratification of $\mathbf{P}(V_{6}^{\vee })$ from the point of view of moduli. The following proposition gives a geometric interpretation of the EPW stratification of $\mathbf{P}(V_{6})$ , which will be essential later.

As mentioned before, the quadric $Q$ defining $X$ in (2.1) is not unique; such quadrics are parameterized by the affine space $\mathbf{P}(V_{6}(X))\setminus \mathbf{P}(V_{5}(X))$ of non-Plücker quadrics. In other words, a quadric $Q$ defining $X$ in (2.1) corresponds to a quadric point

$$\begin{eqnarray}\mathbf{q}\in \mathbf{P}(V_{6}(X))\end{eqnarray}$$

such that $(\mathbf{q},\mathbf{p}_{X})$ does not lie on the incidence divisor in $\mathbf{P}(V_{6}(X))\times \mathbf{P}(V_{6}(X)^{\vee })$ .

Proposition 3.4 [Reference Debarre and KuznetsovDK18a, Proposition 3.13(b)].

Let $X$ be a GM variety. Under the identification of the affine space $\mathbf{P}(V_{6}(X))\setminus \mathbf{P}(V_{5}(X))$ with the space of non-Plücker quadrics containing  $X$ , the stratum

$$\begin{eqnarray}\mathsf{Y}_{\mathsf{A}(X)}^{k}\cap (\mathbf{P}(V_{6}(X))\setminus \mathbf{P}(V_{5}(X)))\end{eqnarray}$$

corresponds to the quadrics $Q$ such that $\dim (\ker (Q))=k$ .

The symmetry between the Plücker point $\mathbf{p}_{X}$ and the quadric point $\mathbf{q}$ is the basis for the duality of GM varieties, discussed below.

3.2 The duality conjecture

The following definition extends [Reference Debarre and KuznetsovDK18a, Definitions 3.22 and 3.26].

One of the main results of [Reference Debarre and KuznetsovDK18a, § 4] is that period partners or dual GM varieties of dimension at least $3$ are birational. Our motivation for defining generalized partners and duals is the following conjecture.

By Proposition 2.6, GM varieties with equivalent GM categories must have dimensions of the same parity, which explains the parity condition in Definition 3.5. We note that part (1) of the conjecture follows from part (2), since by Definition 3.5 and Theorem 3.1 generalized period partners have a common generalized dual GM variety. For this reason, we refer to Conjecture 3.7 as the duality conjecture.

As evidence for the duality conjecture, we prove in § 4 the special case where $X_{1}$ is an ordinary GM fourfold and $X_{2}$ is a (suitably generic) generalized dual GM surface. In fact, the approach of § 4 can be used to attack the full conjecture, but is quite unwieldy to carry out in the general case. In forthcoming work, we establish the general case as a consequence of a theory of ‘categorical joins’ [Reference Kuznetsov and PerryKP18]. This approach is based on the observation from [Reference Debarre and KuznetsovDK18a, Proposition 3.28] that duality of ordinary GM varieties can be interpreted in terms of projective duality of quadrics (see also § 4.2). We show that this extends to generalized duality by replacing classical projective duality with homological projective duality.

In the rest of this subsection we discuss some consequences of the duality conjecture. We start by describing all generalized duals and partners of a given GM variety.

The argument of Lemma 3.8 shows that the set of isomorphism classes of generalized duals of  $X$ can be identified with the quotient of $\mathbf{P}(V_{6}(X))$ by the action of the finite stabilizer group of  $\mathsf{A}(X)$ in $\operatorname{PGL}(V_{6}(X))$ . Analogously, the isomorphism classes of generalized partners of $X$ are parameterized by a quotient of $\mathbf{P}(V_{6}(X)^{\vee })$ by the same group.

Let us list more explicitly the type of $X_{\mathbf{q}}^{\vee }$ according to the stratum $\mathsf{Y}_{\mathsf{A}(X)}^{k}$ of $\mathbf{q}$ and the parity of $n=\dim X$ :

Recall that the stratum $\mathsf{Y}_{\mathsf{A}(X)}^{k}$ is always non-empty for $k=0,1,2$ , generically empty for $k=3$ , and always empty for $k\geqslant 4$ (under our assumption that $\mathsf{A}(X)$ contains no decomposable vectors). In fact, the condition that $\mathsf{Y}_{\mathsf{A}(X)}^{3}$ is non-empty is divisorial in ${\mathcal{M}}_{n}$ (see Remark 4.3). In the same way, one can describe the types of generalized partners $X_{\mathbf{p}}$ of $X$ depending on the stratum $\mathsf{Y}_{\mathsf{A}(X)^{\bot }}^{k}$ of $\mathbf{p}$ and the parity of $n$ .

Conjecture 3.7 says there are equivalences

$$\begin{eqnarray}{\mathcal{A}}_{X}\simeq {\mathcal{A}}_{X_{\mathbf{p}}}\simeq {\mathcal{A}}_{X_{\mathbf{q}}^{\vee }}\end{eqnarray}$$

for every $\mathbf{p}\in \mathbf{P}(V_{6}(X)^{\vee })$ and every $\mathbf{q}\in \mathbf{P}(V_{6}(X))$ . In particular, it predicts that often GM categories are equivalent to those of lower-dimensional GM varieties, namely that:

1. (1) if $X$ is a sixfold, then its GM category is equivalent to a fourfold’s GM category;

2. (2) if $X$ is a fivefold, then its GM category is equivalent to a threefold’s GM category;

3. (3) if $X$ is a fourfold such that $\mathsf{Y}_{\mathsf{A}(X)^{\bot }}^{3}\neq \varnothing$ or $\mathsf{Y}_{\mathsf{A}(X)}^{3}\neq \varnothing$ , then its GM category is equivalent to the derived category of a GM surface.

As mentioned above, in § 4 we prove (3) in case $\mathsf{Y}_{\mathsf{A}(X)}^{3}\neq \varnothing$ and an additional genericity assumption holds, namely $\mathsf{Y}_{\mathsf{A}(X)}^{3}\not \subset \mathbf{P}(V_{5}(X))$ .

3.3 Rationality conjectures

Let us recall what is known about rationality of GM varieties. A general GM threefold is irrational by [Reference BeauvilleBea77, Theorem 5.6], while every GM fivefold or sixfold is rational by [Reference Debarre and KuznetsovDK18a, Proposition 4.2] (for a general GM fivefold or sixfold this was already known to Roth). The intermediate case of GM fourfolds is more mysterious, and closely parallels the situation for cubic fourfolds: some rational examples are known [Reference Debarre, Iliev and ManivelDIM15], but while a very general GM fourfold is expected to be irrational, it has not been proven that a single GM fourfold is irrational. Below, we analyze this state of affairs from the point of view of derived categories.

Following [Reference KuznetsovKuz16b, § 3.3], we use the following terminology.

• ∙ For a triangulated category ${\mathcal{A}}$ , the geometric dimension $\text{gdim}({\mathcal{A}})$ is defined as the minimal integer $m$ such that there exists an $m$ -dimensional connected smooth projective variety $M$ and an admissible embedding ${\mathcal{A}}{\hookrightarrow}\text{D}^{\text{b}}(M)$ .

• ∙ If $Y$ is a smooth projective variety and $\text{D}^{\text{b}}(Y)=\langle {\mathcal{A}}_{1},\ldots ,{\mathcal{A}}_{m}\rangle$ is a maximal semiorthogonal decomposition (i.e. the components are indecomposable), then ${\mathcal{A}}_{i}$ is called a Griffiths component if $\text{gdim}({\mathcal{A}}_{i})\geqslant \dim Y-1$ .

If the set of Griffiths components of $Y$ did not depend on the choice of maximal semiorthogonal decomposition, then it would be a birational invariant [Reference KuznetsovKuz16b, Lemma 3.10]; in particular, it would be empty if $Y$ is rational of dimension at least $2$ . Unfortunately, there are examples showing this is not true (see [Reference KuznetsovKuz16b, § 3.4], [Reference Böhning, Graf von Bothmer and SosnaBGS14]). It may be possible to salvage the situation by modifying the definition of a Griffiths component (some possibilities are discussed in [Reference KuznetsovKuz16b, § 3.4]), but this remains an important question.

Nonetheless, the existence of a Griffiths component appears to be related to irrationality in several examples. For instance, if $X^{\prime }\subset \mathbf{P}^{5}$ is a smooth cubic fourfold, there is a semiorthogonal decomposition

(3.1) $$\begin{eqnarray}\text{D}^{\text{b}}(X^{\prime })=\langle {\mathcal{A}}_{X^{\prime }},{\mathcal{O}}_{X^{\prime }},{\mathcal{O}}_{X^{\prime }}(1),{\mathcal{O}}_{X^{\prime }}(2)\rangle ,\end{eqnarray}$$

where ${\mathcal{A}}_{X^{\prime }}$ is a K3 category (see [Reference KuznetsovKuz04, Corollary 4.3] or [Reference KuznetsovKuz16a, Corollary 4.1]). If ${\mathcal{A}}_{X^{\prime }}$ is equivalent to the derived category of a K3 surface, then $\text{gdim}({\mathcal{A}}_{X^{\prime }})=2$ and hence (3.1) contains no Griffiths components. If ${\mathcal{A}}_{X^{\prime }}$ is not geometric (which holds for a very general cubic fourfold by an argument similar to Proposition 2.29), then we expect ${\mathcal{A}}_{X^{\prime }}$ to be a Griffiths component, although this remains an interesting open problem, cf. [Reference KuznetsovKuz16a, Conjecture 5.8]. These considerations motivated the following conjecture.

As evidence, this conjecture was proved in [Reference KuznetsovKuz10] for all rational $X^{\prime }$ known at the time. Since then, a nearly complete answer to when ${\mathcal{A}}_{X^{\prime }}$ is equivalent to the derived category of a K3 surface has been given [Reference Addington and ThomasAT14], and some new families of rational cubic fourfolds have been produced [Reference Addington, Hassett, Tschinkel and Várilly-AlvaradoAHTV16].

The same philosophy can be applied to GM fourfolds. If the GM category ${\mathcal{A}}_{X}$ of a GM fourfold $X$ is geometric, then (2.11) contains no Griffiths components, and otherwise we expect  ${\mathcal{A}}_{X}$ to be a Griffiths component. This suggests the following analogue of Conjecture 3.11.

One of the main results of this paper, Theorem 1.2 (or rather Theorem 4.1), verifies Conjecture 3.12 for a certain family of rational GM fourfolds. Another result, Theorem 1.3 (or rather Theorem 5.8), builds a bridge between Conjectures 3.12 and 3.11. Finally, recall that we proved the GM category of a very general GM fourfold is not equivalent to the derived category of a K3 surface (Proposition 2.29). Hence Conjecture 3.12 is consistent with the expectation that a very general GM fourfold is irrational.

Now we consider GM varieties of other dimensions from the perspective of derived categories. The next result shows that for a GM threefold $X$ , any maximal semiorthogonal decomposition of $\text{D}^{\text{b}}(X)$ obtained by refining (2.11) contains a Griffiths component. We view this as evidence that any smooth GM threefold is irrational.

Proof. It is easy to see that any category of geometric dimension $0$ is equivalent to $\text{D}^{\text{b}}(\operatorname{Spec}(\mathbf{k}))$ . Further, by [Reference OkawaOka11] any category of geometric dimension $1$ is equivalent to the derived category of a curve. Note that $\operatorname{HH}_{\bullet }(\operatorname{Spec}(\mathbf{k}))=\mathbf{k}$ , and if $C$ is a curve of genus $g$ then

$$\begin{eqnarray}\operatorname{HH}_{\bullet }(C)=\mathbf{k}^{g}[1]\oplus \mathbf{k}^{2}\oplus \mathbf{k}^{g}[-1].\end{eqnarray}$$

Thus if ${\mathcal{A}}_{X}$ has a semiorthogonal decomposition with all components of geometric dimension at most $1$ , Proposition 2.9 and Theorem 2.8 imply ${\mathcal{A}}_{X}\simeq \text{D}^{\text{b}}(C)$ for a genus $10$ curve $C$ . This cannot happen by Proposition 2.29.◻

If $X$ is a GM fivefold or sixfold, then by the discussion in § 3.2, $X$ has a generalized dual  $X^{\vee }$ with $\dim (X^{\vee })\leqslant \dim (X)-2$ . The duality conjecture (Conjecture 3.7(2)) predicts that  ${\mathcal{A}}_{X}\simeq {\mathcal{A}}_{X^{\vee }}$ , and hence $\text{gdim}({\mathcal{A}}_{X})\leqslant \dim (X)-2$ . So assuming the duality conjecture, we see that (2.11) has no Griffiths components, which is consistent with the rationality of $X$ .

4.1 Statement of the result

Recall that for any GM fourfold  $X$ and a quadric point $\mathbf{q}\in \mathbf{P}(V_{6}(X))$ , we associated in § 3.2 a generalized dual variety  $X_{\mathbf{q}}^{\vee }$ , which is an ordinary GM surface if  $\mathbf{q}\in \mathsf{Y}_{\mathsf{A}(X)}^{3}$ .

Theorem 4.1. Let $X$ be an ordinary GM fourfold such that

$$\begin{eqnarray}\mathsf{Y}_{\mathsf{A}(X)}^{3}\cap (\mathbf{P}(V_{6}(X))\setminus \mathbf{P}(V_{5}(X)))\neq \varnothing .\end{eqnarray}$$

Then for any point $\mathbf{q}\in \mathsf{Y}_{\mathsf{A}(X)}^{3}\cap (\mathbf{P}(V_{6}(X))\setminus \mathbf{P}(V_{5}(X)))$ , there is an equivalence

$$\begin{eqnarray}{\mathcal{A}}_{X}\simeq \text{D}^{\text{b}}(X_{\mathbf{q}}^{\vee }).\end{eqnarray}$$

The proof of this theorem takes the rest of this section. We start by noting an immediate consequence for period partners.

A key ingredient in the proof of Theorem 4.1 is the theory of homological projective duality [Reference KuznetsovKuz07]. Very roughly, this theory relates the derived categories of linear sections of an ambient variety to those of orthogonal linear sections of a ‘dual’ variety. As we explain below, the varieties $X$ and $X_{\mathbf{q}}^{\vee }$ from Theorem 4.1 can be thought of as intersections of $\mathbf{G}\subset \mathbf{P}(\wedge ^{2}V_{5})$ and its dual $\mathbf{G}^{\vee }=\operatorname{Gr}(2,V_{5}^{\vee })\subset \mathbf{P}(\wedge ^{2}V_{5}^{\vee })$ with projectively dual quadric subvarieties. To prove Theorem 1.2, we thus establish a ‘quadratic’ version of homological projective duality, in the case where the ambient variety is $\mathbf{G}$ . Much of our argument is not special to $\mathbf{G}$ , however, and should have interesting applications in other settings.

Recall from § 2.1 that if $X$ is an ordinary GM fourfold, there is a (canonical) hyperplane $W\subset \wedge ^{2}V_{5}(X)$ and a (non-canonical) quadric $Q\subset \mathbf{P}(W)$ such that $X=\mathbf{G}\cap Q$ . The fourfolds satisfying the assumption of Theorem 4.1 admit several different characterizations.

For the rest of the section, we fix an ordinary GM fourfold $X$ satisfying the equivalent conditions of Lemma 4.4 and a point $\mathbf{q}\in \mathsf{Y}_{\mathsf{A}(X)}^{3}\cap (\mathbf{P}(V_{6}(X))\setminus \mathbf{P}(V_{5}(X)))$ . Further, to ease notation, we denote the generalized dual of $X$ corresponding to the quadric point $\mathbf{q}$ (see Lemma 3.8) by

$$\begin{eqnarray}Y=X_{\mathbf{q}}^{\vee }.\end{eqnarray}$$

Note that $Y$ is a GM surface.

4.2 Setup and outline of the proof

We outline here the strategy for proving Theorem 4.1.

The starting point is the following explicit geometric relation between $X$ and  $Y$ . By Proposition 3.4, the point $\mathbf{q}$ corresponds to a rank $6$ quadric $Q$ cutting out  $X$ , and the Plücker point  $\mathbf{p}_{X}\in \mathbf{P}(V_{6}(X)^{\vee })\cong \mathbf{P}(V_{6}(Y))$ of $X$ corresponds to a quadric $Q^{\prime }$ cutting out  $Y$ . Because  $X$ and $Y$ are ordinary, we may regard $Q$ as a subvariety of $\mathbf{P}(\wedge ^{2}V_{5}(X))$ and $Q^{\prime }$ as a subvariety of $\mathbf{P}(\wedge ^{2}V_{5}(Y))$ . Then [Reference Debarre and KuznetsovDK18a, Proposition 3.28] (which is stated for dual varieties but works just as well for generalized duals) says that there is an isomorphism $V_{5}(X)\cong V_{5}(Y)^{\vee }$ identifying $Q^{\prime }\subset \mathbf{P}(\wedge ^{2}V_{5}(Y))$ with the projective dual to $Q\subset \mathbf{P}(\wedge ^{2}V_{5}(X))$ . Hence, fixing $V_{5}=V_{5}(X)$ , our setup is as follows: there is a hyperplane $W\subset \wedge ^{2}V_{5}$ and a rank  $6$ quadric $Q\subset \mathbf{P}(W)$ such that

$$\begin{eqnarray}X=\mathbf{G}\cap Q\quad \text{and}\quad Y=\mathbf{G}^{\vee }\cap Q^{\vee },\end{eqnarray}$$

where $Q^{\vee }\subset \mathbf{P}(\wedge ^{2}V_{5}^{\vee })$ is the projectively dual quadric to $Q\subset \mathbf{P}(\wedge ^{2}V_{5})$ , and

$$\begin{eqnarray}\mathbf{G}^{\vee }=\operatorname{Gr}(2,V_{5}^{\vee })\subset \mathbf{P}(\wedge ^{2}V_{5}^{\vee })\end{eqnarray}$$

is the dual Grassmannian.

From this starting point, the main steps of the proof are as follows. First, by considering families of maximal linear subspaces of $Q$ and $Q^{\vee }$ , we find $\mathbf{P}^{1}$ -bundles $\widehat{X}\rightarrow X$ and $\widehat{Y}\rightarrow Y$ , together with morphisms $\widehat{X}\rightarrow \mathbf{P}^{3}$ and $\widehat{Y}\rightarrow \mathbf{P}^{3}$ realizing $\widehat{X}$ and $\widehat{Y}$ as families of mutually orthogonal linear sections of $\mathbf{G}$ and $\mathbf{G}^{\vee }$ . This allows us to apply homological projective duality to obtain a semiorthogonal decomposition of $\text{D}^{\text{b}}(\widehat{X})$ with $\text{D}^{\text{b}}(\widehat{Y})$ as a component. By comparing this (via mutation functors) with the decomposition of $\text{D}^{\text{b}}(\widehat{X})$ coming from its  $\mathbf{P}^{1}$ -bundle structure over $X$ , we show $\text{D}^{\text{b}}(\widehat{Y})$ has a decomposition into two copies of ${\mathcal{A}}_{X}$ . On the other hand, as $\widehat{Y}\rightarrow Y$ is a $\mathbf{P}^{1}$ -bundle, $\text{D}^{\text{b}}(\widehat{Y})$ also decomposes into two copies of $\text{D}^{\text{b}}(Y)$ . We show these two decompositions of $\text{D}^{\text{b}}(\widehat{Y})$ coincide, and hence ${\mathcal{A}}_{X}\simeq \text{D}^{\text{b}}(Y)$ . Our proof gives an explicit functor inducing this equivalence, see (4.15).

4.3 Maximal linear subspaces of the quadrics

We start by discussing a geometric relation between $Q$ and $Q^{\vee }$ . Let $K\subset W$ be the kernel of  $Q$ , regarded as a symmetric linear map $W\rightarrow W^{\vee }$ . Since $\dim W=9$ and $\text{rank}(Q)=6$ , we have $\dim K=3$ . The filtration

$$\begin{eqnarray}0\subset K\subset W\subset \wedge ^{2}V_{5}\end{eqnarray}$$

induces a filtration

$$\begin{eqnarray}0\subset W^{\bot }\subset K^{\bot }\subset \wedge ^{2}V_{5}^{\vee },\end{eqnarray}$$

where $K^{\bot }$ and $W^{\bot }$ are the annihilators of $K$ and $W$ , so that $\dim K^{\bot }=7$ and $\dim W^{\bot }=1$ . The pairing between the dual spaces $\wedge ^{2}V_{5}$ and $\wedge ^{2}V_{5}^{\vee }$ induces a non-degenerate pairing between  $W/K$ and $K^{\bot }/W^{\bot }$ , and hence an isomorphism

$$\begin{eqnarray}K^{\bot }/W^{\bot }\cong (W/K)^{\vee }.\end{eqnarray}$$

The quadric $Q$ induces a smooth quadric $\bar{Q}$ in the 5-dimensional projective space $\mathbf{P}(W/K)$ . The quadric $\bar{Q}$ can be identified with the Grassmannian $\operatorname{Gr}(2,4)$ ; more precisely, we can find an isomorphism

$$\begin{eqnarray}W/K\cong \wedge ^{2}S\end{eqnarray}$$

for a 4-dimensional vector space $S$ , with an identification

$$\begin{eqnarray}\bar{Q}=\operatorname{Gr}(2,S)\subset \mathbf{P}(\wedge ^{2}S).\end{eqnarray}$$

The projective dual of $\bar{Q}$ is then the dual Grassmannian

$$\begin{eqnarray}\bar{Q}^{\vee }=\operatorname{Gr}(2,S^{\vee })\subset \mathbf{P}(\wedge ^{2}S^{\vee })=\mathbf{P}((W/K)^{\vee })=\mathbf{P}(K^{\bot }/W^{\bot }).\end{eqnarray}$$

It follows that the projective dual of

(4.1) $$\begin{eqnarray}Q=\operatorname{Cone}_{\mathbf{P}(K)}\bar{Q}\subset \mathbf{P}(\wedge ^{2}V_{5})\end{eqnarray}$$

is given by

(4.2) $$\begin{eqnarray}Q^{\vee }=\operatorname{Cone}_{\boldsymbol{ P}(W^{\bot })}\bar{Q}^{\vee }\subset \mathbf{P}(\wedge ^{2}V_{5}^{\vee }).\end{eqnarray}$$

Projective 3-space $\mathbf{P}(S)$ is (a connected component of) the space of maximal linear subspaces of the quadric $\bar{Q}=\operatorname{Gr}(2,S)$ . The universal family is the flag variety $\operatorname{Fl}(1,2;S)\rightarrow \mathbf{P}(S)$ , with fiber over a point $s\in \mathbf{P}(S)$ the plane $\mathbf{P}(s\wedge S)\subset \mathbf{P}(\wedge ^{2}S)$ . Analogously, the same flag variety $\operatorname{Fl}(2,3;S^{\vee })\cong \operatorname{Fl}(1,2;S)$ is (a connected component of) the space of maximal linear subspaces of $\bar{Q}^{\vee }=\operatorname{Gr}(2,S^{\vee })$ , this time with fiber over a point $s\in \mathbf{P}(S)$ being the plane $\mathbf{P}(\wedge ^{2}s^{\bot })\subset \mathbf{P}(\wedge ^{2}S^{\vee })$ . Note that the fibers of these two correspondences over a point $s\in \mathbf{P}(S)$ are mutually orthogonal with respect to the pairing between $\wedge ^{2}S$ and $\wedge ^{2}S^{\vee }$ . We summarize this discussion by the diagram

(4.3)

with the inner arrows being $\mathbf{P}^{2}$ -bundles with mutually orthogonal fibers (as linear subspaces of  $\mathbf{P}(\wedge ^{2}S)$ and $\mathbf{P}(\wedge ^{2}S^{\vee })$ ), and the outer arrows being $\mathbf{P}^{1}$ -bundles.

By (4.1) every maximal isotropic subspace of $\bar{Q}$ gives a maximal isotropic subspace of $Q$ by taking its preimage under the projection $W\rightarrow W/K=\wedge ^{2}S$ . Analogously, by (4.2) every maximal isotropic subspace of $\bar{Q}^{\vee }$ gives a maximal isotropic subspace of $Q^{\vee }$ by taking its preimage under the projection $K^{\bot }\rightarrow K^{\bot }/W^{\bot }=\wedge ^{2}S^{\vee }$ . Note that for the pairing between  $W$ and $K^{\bot }$ induced by the pairing between $\wedge ^{2}V_{5}$ and $\wedge ^{2}V_{5}^{\vee }$ , the subspace $K\subset W$ is the left kernel, and the subspace $W^{\bot }\subset K^{\bot }$ is the right kernel. Hence any $s\in \mathbf{P}(S)$ gives mutually orthogonal maximal isotropic spaces ${\mathcal{I}}_{s}$ and ${\mathcal{I}}_{s}^{\bot }$ of $Q$ and $Q^{\vee }$ respectively. These spaces form the fibers of vector bundles ${\mathcal{I}}$ and  ${\mathcal{I}}^{\bot }$ over $\mathbf{P}(S)$ of ranks $6$ and $4$ , which are mutually orthogonal subbundles of $\wedge ^{2}V_{5}\otimes {\mathcal{O}}_{\mathbf{P}(S)}$ and $\wedge ^{2}V_{5}^{\vee }\otimes {\mathcal{O}}_{\mathbf{P}(S)}$ . We can summarize this discussion by the following diagram.

(4.4)

Here the inner arrows are $\mathbf{P}^{5}$ - and $\mathbf{P}^{3}$ -bundles with mutually orthogonal fibers, and the outer arrows are $\mathbf{P}^{1}$ -bundles (induced by the $\mathbf{P}^{1}$ -bundles of diagram (4.3)) away from the vertices $\mathbf{P}(K)$ and $\mathbf{P}(W^{\bot })$ of the quadrics (over which the fibers are isomorphic to $\mathbf{P}(S)\cong \mathbf{P}^{3}$ ).

4.4 Families of linear sections of the Grassmannians

Now define

(4.5) $$\begin{eqnarray}\widehat{X}:=\mathbf{G}\times _{\mathbf{P}(\wedge ^{2}V_{5})}\mathbf{P}_{\mathbf{P}(S)}({\mathcal{I}})\quad \text{and}\quad \widehat{Y}:=\mathbf{G}^{\vee }\times _{\boldsymbol{ P}(\wedge ^{2}V_{5}^{\vee })}\mathbf{P}_{\mathbf{P}(S)}({\mathcal{I}}^{\bot })\end{eqnarray}$$

to be the induced families of linear sections of $\mathbf{G}$ and $\mathbf{G}^{\vee }$ . They fit into a diagram

(4.6)

with the maps induced by those in (4.4) (remember that $X=\mathbf{G}\cap Q$ and $Y=\mathbf{G}^{\vee }\cap Q^{\vee }$ ).

We will denote by $H,H^{\prime }$ , and $h$ the ample generators of $\operatorname{Pic}(\mathbf{G})$ , $\operatorname{Pic}(\mathbf{G}^{\vee })$ , and $\mathbf{P}(S)$ .

Lemma 4.5. There are rank $2$ vector bundles ${\mathcal{S}}_{X}$ and ${\mathcal{S}}_{Y}$ on $X$ and $Y$ with $\operatorname{c}_{1}({\mathcal{S}}_{X})=-H$ and $\operatorname{c}_{1}({\mathcal{S}}_{Y})=-H^{\prime }$ , and isomorphisms

$$\begin{eqnarray}\widehat{X}\cong \mathbf{P}_{X}({\mathcal{S}}_{X})\quad \text{and}\quad \widehat{Y}\cong \mathbf{P}_{Y}({\mathcal{S}}_{Y}),\end{eqnarray}$$

such that ${\mathcal{O}}_{\mathbf{P}_{X}({\mathcal{S}}_{X})}(1)=\unicode[STIX]{x1D70B}_{X}^{\ast }{\mathcal{O}}_{\mathbf{P}(S)}(h)$ and ${\mathcal{O}}_{\mathbf{P}_{Y}({\mathcal{S}}_{Y})}(1)=\unicode[STIX]{x1D70B}_{Y}^{\ast }{\mathcal{O}}_{\mathbf{P}(S)}(h)$ . In particular, $\widehat{X}$ is a smooth fivefold, $\widehat{Y}$ is a smooth threefold, and

(4.7) $$\begin{eqnarray}K_{\widehat{X}}=-H-2h\quad \text{and}\quad K_{\widehat{Y}}=H^{\prime }-2h.\end{eqnarray}$$

Proof. Since $X$ and $Y$ are smooth, they do not intersect the vertices $\mathbf{P}(K)$ and $\mathbf{P}(W^{\bot })$ of the quadrics $Q$ and $Q^{\vee }$ , hence the maps $p_{X}$ and $p_{Y}$ are $\mathbf{P}^{1}$ -fibrations induced by those in diagram (4.4). In other words, we have fiber product squares

The map $p_{\bar{Q}}$ is the projectivization of the tautological subbundle of $S\otimes {\mathcal{O}}$ on $\bar{Q}=\operatorname{Gr}(2,S)$ , and $p_{\bar{Q}^{\vee }}$ is the projectivization of the annihilator of the tautological subbundle of $S^{\vee }\otimes {\mathcal{O}}$ on the dual Grassmannian $\bar{Q}^{\vee }=\operatorname{Gr}(2,S^{\vee })$ . So we can take ${\mathcal{S}}_{X}$ and ${\mathcal{S}}_{Y}$ to be the pullbacks to $X$ and $Y$ of these bundles.

To compute the canonical classes, note that the determinant of the tautological bundle (and of its annihilator) on $\operatorname{Gr}(2,S)$ is ${\mathcal{O}}_{\operatorname{Gr}(2,S)}(-1)$ , hence $\operatorname{c}_{1}({\mathcal{S}}_{X})=-H$ and $\operatorname{c}_{1}({\mathcal{S}}_{Y})=-H^{\prime }$ . Now apply the standard formula for the canonical bundle of the projectivization of a vector bundle, taking into account that $K_{X}=-2H$ and $K_{Y}=0$ by (2.2).◻

As a byproduct of the above, we obtain the following.

4.5 Homological projective duality

Homological projective duality (HPD) is a key tool in the proof of Theorem 4.1. Very roughly, HPD relates the derived categories of linear sections of a given variety to those of orthogonal linear sections of an ‘HPD variety’. We refer to [Reference KuznetsovKuz07] for the details of this theory, and to [Reference KuznetsovKuz14] or [Reference ThomasTho17] for an overview. For us, the relevant point is that the dual Grassmannian $\mathbf{G}^{\vee }$ is HPD to $\mathbf{G}$ . We spell out the precise consequence of this that we need below.

Recall that by Lemma 2.2 there is a semiorthogonal decomposition

$$\begin{eqnarray}\text{D}^{\text{b}}(\mathbf{G})=\langle {\mathcal{B}},{\mathcal{B}}(H),{\mathcal{B}}(2H),{\mathcal{B}}(3H),{\mathcal{B}}(4H)\rangle .\end{eqnarray}$$

Let

$$\begin{eqnarray}i:\mathbf{H}(\mathbf{G},\mathbf{G}^{\vee }){\hookrightarrow}\mathbf{G}\times \mathbf{G}^{\vee }\subset \mathbf{P}(\wedge ^{2}V_{5})\times \mathbf{P}(\wedge ^{2}V_{5}^{\vee })\end{eqnarray}$$

be the incidence divisor defined by the canonical section of ${\mathcal{O}}(H+H^{\prime })$ . Recall that ${\mathcal{U}}$ denotes the tautological rank $2$ bundle on $\mathbf{G}$ , and let ${\mathcal{V}}$ denote the tautological rank $2$ bundle on $\mathbf{G}^{\vee }$ . The following was proved in [Reference KuznetsovKuz06, § 6.1]. See [Reference KuznetsovKuz07, Definition 6.1] for the definition of HPD.

Theorem 4.8. The Grassmannian $\mathbf{G}^{\vee }\rightarrow \mathbf{P}(\wedge ^{2}V_{5}^{\vee })$ is HPD to $\mathbf{G}\rightarrow \mathbf{P}(\wedge ^{2}V_{5})$ with respect to the above semiorthogonal decomposition. Moreover, the duality is implemented by a sheaf  ${\mathcal{E}}$ on  $\mathbf{H}(\mathbf{G},\mathbf{G}^{\vee })$ whose pushforward to $\mathbf{G}\times \mathbf{G}^{\vee }$ fits into an exact sequence

$$\begin{eqnarray}0\rightarrow {\mathcal{O}}_{\mathbf{G}}\boxtimes {\mathcal{V}}\rightarrow {\mathcal{U}}^{\vee }\boxtimes {\mathcal{O}}_{\boldsymbol{ G}^{\vee }}\rightarrow i_{\ast }{\mathcal{E}}\rightarrow 0.\end{eqnarray}$$

In fact, we shall only need a consequence of HPD, which we formulate below as Corollary 4.9. Note that the natural map

$$\begin{eqnarray}\widehat{X}\times _{\mathbf{P}(S)}\widehat{Y}\rightarrow X\times Y\rightarrow \mathbf{G}\times \mathbf{G}^{\vee }\end{eqnarray}$$

factors through $\mathbf{H}(\mathbf{G},\mathbf{G}^{\vee })$ . Indeed, the fiber of $\widehat{X}\times _{\mathbf{P}(S)}\widehat{Y}$ over any point $s\in \mathbf{P}(S)$ is

$$\begin{eqnarray}(\mathbf{P}({\mathcal{I}}_{s})\times \mathbf{P}({\mathcal{I}}_{s}^{\bot }))\cap (\mathbf{G}\times \mathbf{G}^{\vee })\subset \mathbf{H}(\mathbf{G},\mathbf{G}^{\vee }).\end{eqnarray}$$

Note also that

(4.8) $$\begin{eqnarray}\dim (\widehat{X}\times _{\mathbf{P}(S)}\widehat{Y})=5,\end{eqnarray}$$

since the map $\widehat{X}\times _{\mathbf{P}(S)}\widehat{Y}\rightarrow \widehat{Y}$ is flat of relative dimension 2 by Lemma 4.6, and $\dim (\widehat{Y})=3$ by Lemma 4.5.

Denote by $\widehat{{\mathcal{E}}}$ the pullback of the HPD object ${\mathcal{E}}$ to $\widehat{X}\times _{\mathbf{P}(S)}\widehat{Y}$ and by $\widehat{\unicode[STIX]{x1D6F7}}:\text{D}^{\text{b}}(\widehat{Y})\rightarrow \text{D}^{\text{b}}(\widehat{X})$ the corresponding Fourier–Mukai functor. Note that $\widehat{\unicode[STIX]{x1D6F7}}$ is $\mathbf{P}(S)$ -linear (since $\widehat{{\mathcal{E}}}$ is supported on the fiber product $\widehat{X}\times _{\mathbf{P}(S)}\widehat{Y}$ ), i.e.

$$\begin{eqnarray}\widehat{\unicode[STIX]{x1D6F7}}({\mathcal{F}}\otimes \unicode[STIX]{x1D70B}_{Y}^{\ast }{\mathcal{G}})\cong \widehat{\unicode[STIX]{x1D6F7}}({\mathcal{F}})\otimes \unicode[STIX]{x1D70B}_{X}^{\ast }{\mathcal{G}}\end{eqnarray}$$

for all ${\mathcal{F}}\in \text{D}^{\text{b}}(\widehat{Y})$ and ${\mathcal{G}}\in \text{D}^{\text{b}}(\mathbf{P}(S))$ . By Lemma 4.5 and (4.8), the families $\widehat{X}$ and $\widehat{Y}$ of linear sections of $\mathbf{G}$ and $\mathbf{G}^{\vee }$ satisfy the dimension assumptions of [Reference KuznetsovKuz07, Theorem 6.27]. Hence we obtain the following.

Corollary 4.9. The functor $\widehat{\unicode[STIX]{x1D6F7}}:\text{D}^{\text{b}}(\widehat{Y})\rightarrow \text{D}^{\text{b}}(\widehat{X})$ is fully faithful, and there is a semiorthogonal decomposition

(4.9) $$\begin{eqnarray}\text{D}^{\text{b}}(\widehat{X})=\langle \widehat{\unicode[STIX]{x1D6F7}}(\text{D}^{\text{b}}(\widehat{Y})),{\mathcal{B}}_{X}(H)\boxtimes \text{D}^{\text{b}}(\mathbf{P}(S))\rangle ,\end{eqnarray}$$

where ${\mathcal{B}}_{X}(H)\,\boxtimes \,\text{D}^{\text{b}}(\mathbf{P}(S))$ denotes the triangulated subcategory generated by objects of the form $p_{X}^{\ast }({\mathcal{F}})\otimes \unicode[STIX]{x1D70B}_{X}^{\ast }({\mathcal{G}})$ for ${\mathcal{F}}\in {\mathcal{B}}_{X}(H)$ and ${\mathcal{G}}\in \text{D}^{\text{b}}(\mathbf{P}(S))$ .

4.6 Mutations

Since $p_{X}:\widehat{X}\rightarrow X$ is a $\mathbf{P}^{1}$ -bundle (Lemma 4.5), we also have a semiorthogonal decomposition

$$\begin{eqnarray}\text{D}^{\text{b}}(\widehat{X})=\langle p_{X}^{\ast }\text{D}^{\text{b}}(X),p_{X}^{\ast }\text{D}^{\text{b}}(X)(h)\rangle .\end{eqnarray}$$

Inserting the decomposition (2.10) of $\text{D}^{\text{b}}(X)$ , we obtain

(4.10) $$\begin{eqnarray}\text{D}^{\text{b}}(\widehat{X})=\langle {\mathcal{A}}_{\widehat{X}},{\mathcal{B}},{\mathcal{B}}(H),{\mathcal{A}}_{\widehat{X}}(h),{\mathcal{B}}(h),{\mathcal{B}}(H+h)\rangle ,\end{eqnarray}$$

where to ease notation we write ${\mathcal{A}}_{\widehat{X}}$ for $p_{X}^{\ast }{\mathcal{A}}_{X}$ and simply ${\mathcal{B}}$ for $p_{X}^{\ast }{\mathcal{B}}_{X}$ . We find a sequence of mutations bringing this decomposition into the form of (4.9). In doing so we will use several times $K_{X}=-2H$ , which holds by (2.2), and $K_{\widehat{X}}=-H-2h$ , which holds by (4.7). For a brief review of mutation functors and references, see the discussion in § 2.8.

Step 1. Mutate ${\mathcal{B}}(H)$ to the left of ${\mathcal{A}}_{\widehat{X}}$ in (4.10). Since this is a mutation in $p_{X}^{\ast }\text{D}^{\text{b}}(X)$ and $K_{X}=-2H$ , by (2.19) we get

$$\begin{eqnarray}\text{D}^{\text{b}}(\widehat{X})=\langle {\mathcal{B}}(-H),{\mathcal{A}}_{\widehat{X}},{\mathcal{B}},{\mathcal{A}}_{\widehat{X}}(h),{\mathcal{B}}(h),{\mathcal{B}}(H+h)\rangle .\end{eqnarray}$$

Step 2. Mutate ${\mathcal{B}}(H+h)$ to the far left. Since $K_{\widehat{X}}=-H-2h$ , by (2.19) we get

$$\begin{eqnarray}\text{D}^{\text{b}}(\widehat{X})=\langle {\mathcal{B}}(-h),{\mathcal{B}}(-H),{\mathcal{A}}_{\widehat{X}},{\mathcal{B}},{\mathcal{A}}_{\widehat{X}}(h),{\mathcal{B}}(h)\rangle .\end{eqnarray}$$

Step 3. Mutate ${\mathcal{B}}(-H)$ to the left of ${\mathcal{B}}(-h)$ . Since these two subcategories are completely orthogonal (see the lemma below), we get

$$\begin{eqnarray}\text{D}^{\text{b}}(\widehat{X})=\langle {\mathcal{B}}(-H),{\mathcal{B}}(-h),{\mathcal{A}}_{\widehat{X}},{\mathcal{B}},{\mathcal{A}}_{\widehat{X}}(h),{\mathcal{B}}(h)\rangle .\end{eqnarray}$$