Hostname: page-component-586b7cd67f-dlnhk Total loading time: 0 Render date: 2024-11-22T15:11:10.356Z Has data issue: false hasContentIssue false

Hodge classes and the Jacquet–Langlands correspondence

Published online by Cambridge University Press:  06 September 2023

Atsushi Ichino
Affiliation:
Department of Mathematics, Kyoto University, Kitashirakawa Oiwake-cho, Sakyo-ku, Kyoto, 606-8502, Japan; E-mail: [email protected]
Kartik Prasanna*
Affiliation:
Department of Mathematics, University of Michigan, 2074 East Hall, 530 Church Street, Ann Arbor, MI 48109-1043, USA

Abstract

We prove that the Jacquet–Langlands correspondence for cohomological automorphic forms on quaternionic Shimura varieties is realized by a Hodge class. Conditional on Kottwitz’s conjecture for Shimura varieties attached to unitary similitude groups, we also show that the image of this Hodge class in $\ell $-adic cohomology is Galois invariant for all $\ell $.

Type
Number Theory
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
© The Author(s), 2023. Published by Cambridge University Press

1 Introduction

This article is motivated by the following question: Is Langlands functoriality in the case of cohomological automorphic forms on Shimura varieties induced by algebraic cycle classes? When the forms in question contribute to $H^1$ , this follows from Faltings’ theorem [Reference Faltings17] on the Tate conjecture for divisors on abelian varieties, but for higher $H^i$ it seems completely open even in the simplest of cases. Since constructing algebraic cycle classes seems extremely difficult, one can ask for the next best thing, namely to construct the associated absolute Hodge classes [Reference Deligne, Milne, Ogus and Shih15]. We study this problem in the most classical example of functoriality, namely the Jacquet–Langlands correspondence for $\mathrm {GL}_2$ and its inner forms.

1.1 The main theorem

Let F be a totally real field, $[F:\mathbb {Q}]=n$ . Denote by $\Sigma _{\infty }$ the set of infinite places of F, and for $v\in \Sigma _{\infty }$ , let $\sigma _v: F \hookrightarrow \mathbb {R}\subset \mathbb {C} $ denote the corresponding embedding of F in $\mathbb {C}$ . Let $F^c\subset \overline {\mathbb {Q}} \subset \mathbb {C}$ be the compositum of $\sigma _v (F)$ as v varies over $\Sigma _{\infty }$ . Thus, $F^c$ is the Galois closure of the image of $\sigma (F)$ for any $\sigma \in \Sigma _{\infty }$ .

Let $\pi =\otimes _v \pi _v $ be an automorphic representation of $\mathrm {GL}_2(\mathbb {A}_F)$ corresponding to a (cohomological) holomorphic Hilbert modular newform of weight $(\underline {k},r)$ , where $\underline {k}=(k_1,\ldots , k_n)$ and $k_1\equiv k_2 \equiv \cdots \equiv k_n \equiv r \bmod 2$ . For simplicity, we will assume that $\pi $ has trivial Nebentypus character so that it is self-dual up to a (Tate) twist. (See §1.3.2 for the non-self-dual case.) Moreover, in the introduction alone, we assume that $\pi $ has parallel weight two and that the Hecke eigenvalues $a_v (\pi )$ (suitably normalized) are rational; thus, $\pi $ (at least conjecturally) corresponds to an elliptic curve $A/F$ . In any case, it is known that to such a $\pi $ and every rational prime $\ell $ one can attach a two-dimensional $\ell $ -adic Galois representation $ \rho _{\pi ,\ell } $ of the Galois group $\operatorname {Gal} (\overline {\mathbb {Q}}/F)$ . The representations $\rho _{\pi ,\ell }$ (for varying $\ell $ ) form a compatible system in the sense that for all finite primes v of F not dividing $\ell $ and the conductor of $\pi $ , we have

$$\begin{align*}\operatorname{tr} \rho_{\pi,\ell} (\operatorname{Frob}_v) = a_v (\pi), \end{align*}$$

where $\operatorname {Frob}_v$ denotes a geometric Frobenius element attached to v; in particular, this trace is independent of $\ell $ .

Let $B_1$ and $B_2$ be two (nonisomorphic) quaternion algebras over F such that $\pi $ admits Jacquet–Langlands transfers to the algebraic groups $G_1 =\operatorname {Res}_{F/\mathbb {Q}} B_1^\times $ and $G_2=\operatorname {Res}_{F/\mathbb {Q}} B_2^\times $ ; we denote the corresponding automorphic representations of $G_1(\mathbb {A})$ and $G_2(\mathbb {A})$ by $\pi _1$ and $\pi _2$ , respectively. We assume that the set of infinite places of F, where $B_1$ is split agrees with the set of infinite places where $B_2$ is split, and denote this common set of infinite places by $\Sigma \subset \Sigma _{\infty }$ . Let $F_\Sigma $ be the subfield of $\mathbb {C}$ given by

$$ \begin{align*} F_\Sigma &:= \overline{\mathbb{Q}}^{\{ \sigma \in \operatorname{Gal}(\overline{\mathbb{Q}}/\mathbb{Q}) \, | \, \sigma \Sigma=\Sigma\}} = (F^c)^{\{\sigma \in \operatorname{Gal}(F^c/\mathbb{Q}) \, | \, \sigma \Sigma=\Sigma\}}. \end{align*} $$

Then $F_\Sigma $ is also characterized as the subfield of $\overline {\mathbb {Q}}$ generated (over $\mathbb {Q}$ ) by the elements

$$\begin{align*}\sum_{v\in \Sigma} \sigma_v(x), \quad x\in F \end{align*}$$

and is called the reflex field of the pair $(F, \Sigma )$ .

Let $X_{1}$ and $X_{2}$ denote the quaternionic Shimura varieties associated with $G_1$ and $G_2$ . Then $X_1$ and $X_2$ are of dimension $d:=|\Sigma |$ and have canonical models over the same reflex field $F_\Sigma \subset \overline {\mathbb {Q}} \subset \mathbb {C}$ . The Langlands–Kottwitz method can be used to study the $\ell $ -adic cohomology of the varieties $X_{1}$ and $X_{2}$ . Following the work of several authors ([Reference Langlands45], [Reference Brylinski and Labesse11], [Reference Carayol13], [Reference Reimann59], [Reference Nekovář54]), we have the following theorem: For $i=1,2$ , the $\pi _i$ -isotypic part of $H^*_{\mathrm {et}} (X_{i,\overline {\mathbb {Q}}}, \mathbb {Q}_\ell )$ is concentrated entirely in the middle degree d and moreover is isomorphic to the tensor induction

(1.1) $$ \begin{align} {\bigotimes_{v\in \Sigma}}' \ \rho_{\pi,\ell}^v, \end{align} $$

where $\rho _{\pi ,\ell }^v$ denotes the representation of $\operatorname {Gal} (\overline {\mathbb {Q}}/ \sigma _v (F))$ given by $g\mapsto \rho _{\pi ,\ell } (\sigma _v^{-1} g \sigma _v)$ . As a consequence, for all rational primes $\ell $ , we have isomorphisms

(1.2) $$ \begin{align} H^d (X_1, \mathbb{Q}_\ell)_{\pi_1} \simeq H^d (X_2, \mathbb{Q}_\ell)_{\pi_2} \end{align} $$

as representations of $\operatorname {Gal}(\overline {\mathbb {Q}}/F_\Sigma )$ . Here and henceforth, we write $H^*(X,\mathbb {Q}_\ell )$ for the $\mathbb {Q}_\ell $ -vector space $H^*_{\mathrm {et}} (X_{\overline {\mathbb {Q}}}, \mathbb {Q}_\ell )$ .

The isomorphisms (1.2) above may be viewed as giving a collection of Tate classes in

$$\begin{align*}H^{2d} (X_1\times X_2, \mathbb{Q}_\ell(d)), \end{align*}$$

and it is natural to ask if there is a single algebraic cycle $\mathcal {Z} \in \operatorname {CH}^{d} (X_1 \times X_2)$ that gives rise to this collection of Tate classes. If $p_1$ and $p_2$ are the two projections below,

the class of such a putative algebraic cycle $\mathcal {Z}$ gives rise to a map

$$\begin{align*}\mathrm{cl} (\mathcal{Z})^*: H^d(X_1) \rightarrow H^d(X_2), \quad x\mapsto p_{2,*} (\operatorname{cl}(\mathcal{Z}) \cup p_1^*(x)) \end{align*}$$

for any Weil cohomology theory, which induces isomorphisms

(1.3) $$ \begin{align} H^d(X_1)_{\pi_1} \simeq H^d(X_2)_{\pi_2}. \end{align} $$

Moreover, these isomorphisms for different Weil cohomology theories will be compatible via the usual comparison theorems.

With this motivation, we state our main theorem. We remark that our proof (of part (ii) of the theorem below) assumes the validity of Kottwitz’s conjecture characterizing the Galois representations occurring in the cohomology of Shimura varieties in the special case of Shimura varieties attached to unitary similitude groups. (See Remark 1.4 below for a more extensive discussion of the status of this conjecture.)

Theorem 1. Suppose that there is at least one infinite place of F at which $B_1$ and $B_2$ are ramified.

  1. (i) There is a nonzero Hodge class

    $$\begin{align*}\xi \in H^{2d} (X_1 \times X_2, \mathbb{Q})_{\pi_1 \boxtimes \pi_2} \end{align*}$$
    such that the induced map
    (1.4) $$ \begin{align} \xi(d)^*: H^d (X_1,\mathbb{Q})_{\pi_1} \rightarrow H^d (X_2,\mathbb{Q})_{\pi_2}, \quad x\mapsto p_{2,*} (\xi (d) \cup p_1^*(x)) \end{align} $$
    is an isomorphism of $\mathbb {Q}$ -Hodge structures. (i.e., is an isomorphism of $\mathbb {Q}$ -vector spaces that, after extending scalars to $\mathbb {C}$ , preserves the Hodge filtration.)
  2. (ii) Assume Kottwitz’s conjecture for Shimura varieties attached to unitary similitude groups. Then the Hodge class $\xi $ can be chosen such that, for all rational primes $\ell $ , the image $\xi _\ell (d)$ of (the Tate twist) $\xi (d)$ in the $\ell $ -adic étale realization

    $$\begin{align*}H^{2d} (X_1 \times X_2, \mathbb{Q}_\ell)_{\pi_1 \boxtimes \pi_2} (d) \end{align*}$$
    is $\operatorname {Gal}(\overline {\mathbb {Q}}/{F_\Sigma })$ -invariant. Consequently, the induced map
    (1.5) $$ \begin{align} \xi(d)^*_\ell: H^d (X_1,\mathbb{Q}_\ell)_{\pi_1} \simeq H^d (X_2,\mathbb{Q}_\ell)_{\pi_2},\quad x\mapsto p_{2,*} (\xi (d) \cup p_1^*(x)) \end{align} $$
    is an isomorphism of $\operatorname {Gal}(\overline {\mathbb {Q}}/{F_\Sigma })$ -modules. (Here, we view $\xi (d)$ as an étale class via the Betti-étale comparison theorems.)

Our proof does not use the previously known isomorphisms (1.2). Rather, it provides an alternate verification of these isomorphisms which may be of independent interest. We note also that the isomorphism (1.4) of Hodge structures implies relations between periods of modular forms on $B_1^\times $ and $B_2^\times $ . Such period relations have been studied previously by relating the periods to the Fourier coefficients of half-integral weight modular forms [Reference Oda55], [Reference Oda56], [Reference Murty and Ramakrishnan53]. In principle, one could use the period relations to deduce an isomorphism of Hodge structures; however, it seems very unlikely that such methods can show that this isomorphism is also Galois equivariant.

1.2 Outline of the proof

We now explain the strategy of the proof of Theorem 1. In fact, the proof in the general case is very similar to that for $F=\mathbb {Q}$ , $n=d=1$ , and so we first describe this case, even though formally speaking this case is excluded from the theorem on account of the assumption that $B_1$ and $B_2$ are ramified at some infinite place. To be precise, one should work with intersection cohomology in this case, but for simplicity we just use usual cohomology with the understanding that the proof given below is only correct once generalized to the setting where F is a totally real field and there is some infinite place where $B_1$ and $B_2$ are both ramified.

The basic idea of the proof is to embed $X_1 \times X_2$ in a larger Shimura variety X, construct a Hodge class $\xi $ on X and then show that its pullback to $X_1 \times X_2$ has the right property. The implementation of this idea is a bit involved and breaks up as follows.

1.2.1 Unitary Shimura varieties

We first replace $X_1$ and $X_2$ by closely related unitary Shimura varieties. Pick an imaginary quadratic field E that embeds in both $B_1$ and $B_2$ . Let $\mathbf {V}_1 = B_1$ and $\mathbf {V}_2 = B_2$ , viewed as (right) E-vector spaces. These are equipped with natural Hermitian forms that are of signature $(1,1)$ at the infinite place. The corresponding unitary similitude groups are given by

(1.6) $$ \begin{align} \mathrm{GU}_E(\mathbf{V}_1) \simeq (B_1^\times \times E^\times)/ F^\times, \quad \mathrm{GU}_E(\mathbf{V}_2) \simeq (B_2^\times \times E^\times)/ F^\times. \end{align} $$

Let $\mathbf {V} = \mathbf {V}_1 \oplus \mathbf {V}_2$ . Thus $\mathbf {V}$ has signature $(2,2)$ at the infinite place. Consider the maps of algebraic groups

(1.7) $$ \begin{align} B_1^\times \times B_2^\times \rightarrow \mathrm{PB}_1^\times \times \mathrm{PB}_2^\times \leftarrow \mathrm{G} (\mathrm{U}_E(\mathbf{V}_1)\times \mathrm{U}_E(\mathbf{V}_2)) /E^\times \rightarrow \mathrm{GU}_E(\mathbf{V})/E^\times. \end{align} $$

These induce maps of the associated Shimura varieties

$$\begin{align*}X_1 \times X_2 \rightarrow \tilde{X}_1 \times \tilde{X}_2 \leftarrow Y \rightarrow X \end{align*}$$

(where X is the Shimura variety associated with $\mathrm {GU}_E(\mathbf {V})/E^\times $ , etc.), which may be viewed as giving a correspondence on $(X_1 \times X_2) \times X$ . This correspondence induces a map on cohomology

$$\begin{align*}\iota^*: H^*(X) \rightarrow H^*(X_1 \times X_2). \end{align*}$$

As such, since the kernel of the map

$$\begin{align*}\mathrm{G} (\mathrm{U}_E(\mathbf{V}_1)\times \mathrm{U}_E(\mathbf{V}_2)) /E^\times \rightarrow \mathrm{PB}_1^\times \times \mathrm{PB}_2^\times \end{align*}$$

is isomorphic to

$$\begin{align*}\mathrm{G} (E^\times \times E^\times)/E^\times \simeq E^{(1)} \simeq E^\times/F^\times, \end{align*}$$

one can introduce a character $\eta $ of $E^\times /F^\times $ in the construction of the correspondence; this gives a map

$$\begin{align*}\iota_\eta^*: H^*(X) \rightarrow H^*(X_1 \times X_2). \end{align*}$$

that depends on the choice of $\eta $ .

1.2.2 Cohomological representations and Vogan–Zuckerman theory

Since the cohomology of X is given by automorphic forms [Reference Borel and Wallach10], it is natural to first look for a nontempered automorphic representation $\Pi $ of $\mathrm {GU}(\mathbf {V})$ (or say of $\mathrm {U}(\mathbf {V})$ for simplicity) which contributes to $H^2(X)$ but only to the $(1,1)$ -part. The paper of Vogan–Zuckerman [Reference Vogan and Zuckerman65] classifies cohomological representations; one finds that there is a unique nontrivial (nontempered) representation $\Pi _{\infty }^1$ of $\mathrm {U} (\mathbf {V}_{\mathbb {R}}) =\mathrm {U} (2,2)_{\mathbb {R}}$ with the property that

$$\begin{align*}H^{1,1} (\mathfrak{g}, K; \Pi_{\infty}^1) \neq 0. \end{align*}$$

The representation $\Pi _{\infty }^1$ can be realized as a cohomologically induced representation $A_{\mathfrak {q}}$ , where $\mathfrak {q}$ is a $\theta $ -stable parabolic subalgebra of $\mathfrak {g}$ with Levi component $\mathfrak {u}(1,1) \oplus \mathfrak {u}(1,1)$ . In order to construct $\Pi $ , it is first natural to look for an explicit construction of $\Pi _{\infty }^1$ which is what is accomplished in the next step.

1.2.3 An exceptional isogeny: Archimedean theta correspondence and Kudla–Millson theory

The representation $\Pi _{\infty }^1$ can be constructed as a theta lift of the trivial representation of $\mathrm {U}(1,1)$ with appropriate choices of splitting characters. However, for rather subtle reasons, this fact does not seem to be useful in our construction. Instead, we use the fact that there is an exceptional isogeny

(1.8) $$ \begin{align} \mathrm{SU}(2,2)_{\mathbb{R}} \rightarrow \mathrm{SO}(4,2)_{\mathbb{R}}. \end{align} $$

Ignoring for the moment the difference between $\mathrm {U}$ and $\mathrm {SU}$ , and between $\mathrm {O}$ and $\mathrm {SO}$ , we may view $\Pi _{\infty }^1$ as a representation of $\mathrm {O}(4,2)_{\mathbb {R}}$ , and viewed this way, the representation $\Pi _{\infty }^1$ is in fact a theta lift from $\mathrm {SL}_2$ . This fact may appear somewhat familiar to connoisseurs of Kudla–Millson theory. Indeed, Kudla–Millson theory studies certain explicit closed forms that are Poincare dual to geodesic cycles coming from embedded $\mathrm {O}(3,2)$ s in $\mathrm {O}(4,2)$ and shows that the corresponding automorphic representations of $\mathrm {O}(4,2)$ (which contribute to $H^{1,1}$ ) can be constructed as theta lifts of forms of weight $3$ on $\mathrm {SL}_2$ .

1.2.4 Inner forms

For our purposes, we need inner form versions both of the isogeny (1.8) and of the theta lift. Moreover, we need to work with similitude groups rather than isometry groups. First, the theta lift: Let B be the quaternion algebra given by $B=B_1 \cdot B_2$ in the Brauer group of $\mathbb {Q}$ . Since $B_1$ and $B_2$ are assumed to be nonisomorphic, B is a nonsplit quaternion algebra. Then there is a theta lift

where $\tilde {V}$ is a certain three-dimensional B-vector space equipped with a B-skew-Hermitian form, W is a one-dimensional B-vector space equipped with a B-Hermitian form and

denotes the space of automorphic forms on G. (To be precise, the theta lift depends on a choice of Schwartz function.) The groups $\mathrm {U}_B(W)$ and $\mathrm {U}_B(\tilde {V})$ are, respectively, the requisite inner forms of $\mathrm {SL}_2$ and $\mathrm {O}(4,2)$ . As for the isogeny, we construct (in §5) an explicit isomorphism,

(1.9) $$ \begin{align} \delta: \operatorname{PGU}_E(\mathbf{V}) \xrightarrow{\simeq} \operatorname{PGU}_B(\tilde{V})^0, \end{align} $$

which is an inner form version of equation (1.8) above for (projectivized) similitude groups.

1.2.5 The global theta lift: Schwartz forms

With this preparation, we can describe the construction of a $(1,1)$ -class on X. Let h be a modular form of weight $3$ and central character $\xi _E$ , the quadratic character associated with the extension $E/\mathbb {Q}$ , chosen such that it admits a Jacquet–Langlands transfer to $B^\times $ . Let $\tilde {\tau }_h$ be the corresponding representation of $\mathrm {GL}_2 (\mathbb {A})$ . Let $\operatorname {JL}$ denote the Jacquet–Langlands correspondence. Consider the composite maps of automorphic forms

and

We show that $\Theta \circ \operatorname {JL} (\tilde {\tau }_h)$ has trivial central character and so may be viewed as an automorphic representation of the group $\operatorname {PGU}_B(\tilde {V})^0 $ . Thus, we can consider the composite

$$\begin{align*}\Pi := \delta^{-1} \circ \Theta \circ \operatorname{JL} (\tilde{\tau}_h), \end{align*}$$

which we may view as an automorphic representation of the group $\mathrm {GU}_E (\mathbf {V}) (\mathbb {A})$ . This representation has the property that $\Pi _{\infty } \simeq \tilde {\Pi }_{\infty }^1$ , where $\tilde {\Pi }_{\infty }^1$ denotes the unique representation of $\mathrm {GU}(2,2)_{\mathbb {R}}$ with trivial central character whose restriction to $\mathrm {U}(2,2)_{\mathbb {R}}$ is isomorphic to $\Pi _{\infty }^1$ . Further, one can check that

$$\begin{align*}\dim H^{p,q} (\mathfrak{g}, K; \Pi_{\infty}) = \begin{cases} 1 & \text{if } (p,q)=(1,1) \text{ or } (3,3); \\ 2 & \text{if } (p,q)=(2,2). \end{cases} \end{align*}$$

Explicitly, we construct following the ideas of Kudla–Millson (and Funke–Millson in the higher weight case), a Schwartz form $\varphi _{\infty }$ (rather than a Schwartz function) such that with $\varphi =\varphi _{\mathrm {fin}} \otimes \varphi _{\infty }$ for any choice of a Schwartz function $\varphi _{\mathrm {fin}}$ , the theta lift

$$\begin{align*}\theta_\varphi (\phi) \end{align*}$$

may be viewed as giving a $(1,1)$ -class on X, for $\phi $ in the space of $\operatorname {JL}(\tilde {\tau }_h)$ . To be precise, the construction only depends on the restriction of $\operatorname {JL}(\tilde {\tau }_h)$ to the subgroup $\mathrm {GL}_2(\mathbb {A})^+$ (consisting of elements in $\mathrm {GL}_2(\mathbb {A})$ with positive determinant at infinity) and the vector $\phi $ must be chosen to lie in the antiholomorphic component of this restriction.

1.2.6 Nonvanishing of the restriction

Next, we show that for suitable choice of $\eta $ , h, $\varphi _{\mathrm {fin}}$ and $\phi $ , the $(1,1)$ -form $\iota _\eta ^*(\theta _\varphi (\phi ))$ is nonvanishing, when projected to the $\pi _1\boxtimes \pi _2$ -isotypic component. Let us now explain the main idea to prove this nonvanishing. Let $\omega _{f_{B_1}} $ and $\omega _{f_{B_2}}$ denote holomorphic one-forms in $H^1 (X_{B_1}, \mathbb {C})_{\pi _1}$ and $H^1 (X_{B_2}, \mathbb {C})_{\pi _2}$ , respectively. The strategy is to compute the integral

$$\begin{align*}\int_{X_{B_1} \times X_{B_2}} \iota_{\eta}^* \theta_{\varphi}(\phi) \cdot (p_1^* \omega_{f_{B_1}} \wedge \overline{p_2^* \omega_{f_{B_2}}}) \end{align*}$$

and show it is nonzero. Using the isomorphism $\delta $ from equation (1.9) and noting that the decomposition $\mathbf {V} = \mathbf {V}_1 \oplus \mathbf {V}_2$ of E-Hermitian spaces induces a decomposition $\tilde {V}=V \oplus V_0$ of B-skew-Hermitian spaces such that

$$\begin{align*}\mathrm{GU}_B(V)^0 \simeq (B_1^\times \times B_2^\times)/F^\times, \quad \mathrm{GU}_B(V_0)^0 \simeq E^\times, \end{align*}$$

and

$$\begin{align*}\delta : \mathrm{G}(\mathrm{U}_E(\mathbf{V}_1) \times \mathrm{U}_E (\mathbf{V}_2)) / E^\times \xrightarrow{\simeq} \mathrm{G}(\mathrm{U}_B(V) \times \mathrm{U}_B(V_0))^0/F^\times, \end{align*}$$

we can reduce the integral to a period on the left-hand side of the seesaw diagram below, which again involves quaternionic unitary groups:

The seesaw then implies that the period can be computed on the right where it becomes a triple product period of the form

$$\begin{align*}\int_{[\mathrm{GU}_B(W)]}\phi \cdot \overline{f_B \theta(\eta)}, \end{align*}$$

where $f_B = \theta (\overline {f_{B_1}} \boxtimes f_{B_2})$ is an automorphic form on $\mathrm {GU}_B(W) \simeq B^\times $ in the Jacquet–Langlands transfer $\pi _B$ of $\pi $ . We then show that $\eta $ , h, $\varphi _{\mathrm {fin}}$ and $\phi $ can be chosen (depending on the finite parts of $f_{B_1}$ and $f_{B_2}$ ) so as to make this triple product integral nonzero. A similar argument also shows that

$$\begin{align*}\int_{X_{B_1} \times X_{B_2}} \iota_{\eta}^* \theta_{\varphi}(\phi) \cdot (\overline{p_1^* \omega_{f_{B_1}} }\wedge p_2^* \omega_{f_{B_2}}) \end{align*}$$

is nonzero, and in fact that the induced map

$$\begin{align*}\iota_{\eta}^* \theta_{\varphi}(\phi) : H^1 (X_{B_1}, \mathbb{C}) \rightarrow H^1 (X_{B_2}, \mathbb{C}) \end{align*}$$

is an isomorphism.

1.2.7 Hodge classes

As yet we do not know that $\theta _\varphi (\phi )$ is a Hodge class. In fact, strictly speaking it is not likely to be a rational cohomology class, but we show that it lies in the $\mathbb {C}$ -span of the Hodge classes in $H^2(X)$ . The key point here is that the (expected) classification of automorphic representations implies that any automorphic representation that is nearly equivalent to $\Pi $ must have Archimedean component lying in the (unique) A-packet containing $\tilde {\Pi }_{\infty }^1$ . Moreover, this Archimedean A-packet consists of two representations $\tilde {\Pi }_{\infty }^1, \tilde {\Pi }_{\infty }^2$ and the latter contributes only to $H^4(X)$ and not $H^2(X)$ . From this, we deduce that $H^2(X,\mathbb {C})[\Pi _{\mathrm {fin}}] $ is entirely of type $(1,1)$ . (The notation $H^2(X,\mathbb {C})[\Pi _{\mathrm {fin}}] $ stands for the subspace of $H^2(X,\mathbb {C})$ on which the unramified Hecke algebra at some finite level acts by the same Hecke eigenvalues as on $\Pi _{\mathrm {fin}}$ .)

Suppose for the moment that $\Pi $ has coefficients in $\mathbb {Q}$ . Then

$$\begin{align*}H^2(X,\mathbb{Q})[\Pi_{\mathrm{fin}}] \otimes_{\mathbb{Q}} \mathbb{C} = H^2(X,\mathbb{C})[\Pi_{\mathrm{fin}}], \end{align*}$$

hence $H^2(X,\mathbb {Q})[\Pi _{\mathrm {fin}}]$ is a rational Hodge structure, pure of type $(1,1)$ . Since $\theta _\varphi (\phi )$ lies in $H^2(X,\mathbb {C})[\Pi _{\mathrm {fin}}]$ , we see that it lies in the $\mathbb {C}$ -span of $H^2(X,\mathbb {Q})[\Pi _{\mathrm {fin}}] $ and in particular is a $\mathbb {C}$ -linear combination of Hodge classes $\xi $ . We have already seen that $\theta _\varphi (\phi )\neq 0$ and moreover that its restriction to the $\pi _1 \boxtimes \pi _2$ -component of $X_1 \times X_2$ is nonzero. From this and a simple continuity argument, one deduces that there is a Hodge class $\xi \in H^2(X,\mathbb {Q})[\Pi _{\mathrm {fin}}]$ such that the induced map

$$\begin{align*}\iota_\eta ( \xi (1))^* : H^1 (X_{B_1}, \mathbb{Q})_{\pi_1} \rightarrow H^1 (X_{B_2}, \mathbb{Q})_{\pi_2} \end{align*}$$

given by

$$\begin{align*}x \mapsto p_{2,*} (p_1^* (x) \cdot \iota_{\eta}^* \xi (1)) \end{align*}$$

is an isomorphism of rational Hodge structures.

1.2.8 Galois representations

Next, we need to understand the Galois representation on $H^* (X)$ associated to the A-packet containing $\Pi $ . Again, for simplicity let us suppose $F=\mathbb {Q}$ , $d=1$ , the general case being similar. Then the expected relation between the Galois representation and the A-parameter can be deduced from Kottwitz’s conjecture (see Remark 1.4 below). In our case, we have

$$ \begin{align*} H^2(X,\mathbb{Q}_\ell)_{\Pi} &= \mathbb{Q}_\ell (-1), \\ H^4(X, \mathbb{Q}_\ell)_{\Pi} &= \mathbb{Q}_\ell(-2) \oplus \mathrm{Sym}^2(\rho_{h,\ell}) , \\ H^6 (X,\mathbb{Q}_\ell)_{\Pi} &= \mathbb{Q}_\ell (-3), \end{align*} $$

where $\rho _{h,\ell }$ is the two-dimensional $\ell $ -adic representation attached to h. From this, one deduces that as a Galois module, $H^2 (X,\mathbb {Q}_\ell )[\Pi _{\mathrm {fin}}]\simeq \mathbb {Q}_\ell (-1)^{m} $ for some integer m. For every rational prime $\ell $ , the action of $\operatorname {Gal}(\overline {\mathbb {Q}}/\mathbb {Q})$ on $\xi (1)$ is then trivial and thus $\xi (1)^*$ (viewed as acting on $\ell $ -adic cohomology via the Betti-étale comparison) is a Galois equivariant isomorphism.

1.2.9 Descending coefficients

The argument above needs a bit more care since $\Pi $ may not have coefficients in $\mathbb {Q}$ . Thus, one needs some care to ensure that the Hodge class constructed has coefficients in $\mathbb {Q}$ . This argument needed to achieve this is explained in detail in §12. Roughly, the point is to replace $H^2 (X,\mathbb {Q})[\Pi _{\mathrm {fin}}]$ by $H^2 (X,\mathbb {Q})[\mathbb {I}]$ , where $\mathbb {I}$ is the kernel of the action of the unramified Hecke algebra (with $\mathbb {Q}$ -coefficients) on $\Pi _{\mathrm {fin}}$ . Another possible source of extra coefficients is the character $\eta $ , and this needs to be handled separately.

1.2.10 The general case

This completes the outline of the proof of Theorem 1 in the case $F=\mathbb {Q}$ , $n=d=1$ , $k=2$ . The general case (assuming still that $\underline {k}=(2,\ldots ,2)$ ) is only slightly more complicated. In general, we have

$$\begin{align*}\mathrm{U}_E(\mathbf{V}) (\mathbb{R}) \simeq \mathrm{U}(2,2)^d \times \mathrm{U}(4)^{n-d}, \end{align*}$$

where the $\mathrm {U}(2,2)$ factors correspond to the places in $\Sigma $ and the $\mathrm {U}(4)$ factors to the infinite places not in $\Sigma $ . At the infinite places in $\Sigma $ , that is, where $B_1$ and $B_2$ are both split, we just imitate the constructions above. However, we need to deal as well with the infinite places where $B_1$ and $B_2$ are both ramified. At such places the representation $\Pi _{\infty }$ is trivial and the local A-packet is a singleton, consisting of just the trivial representation. This is consistent with the fact that at such places v, we have

$$\begin{align*}\mathrm{U}_E(\mathbf{V})_v\simeq \mathrm{U}(4), \quad \mathrm{U}_B(\tilde{V})_v \simeq \mathrm{O}(6), \quad \mathrm{U}_B(W)_v \simeq \mathrm{SL}_2 \end{align*}$$

and the theta lift of the weight $3$ holomorphic discrete series representation on $\mathrm {SL}_2$ is the trivial representation of $\mathrm {O}(6)$ . The conclusion then is that $H^{2d} (X,\mathbb {C})[\Pi _{\mathrm {fin}}]$ consists entirely of $(d,d)$ -classes and one can find a Hodge class $\xi \in H^{2d}(X,\mathbb {Q})[\Pi _{\mathrm {fin}}]$ such that the induced map

$$\begin{align*}\xi (d)^* : H^d (X_{B_1}, \mathbb{Q}) \rightarrow H^d (X_{B_2}, \mathbb{Q}) \end{align*}$$

given by

$$\begin{align*}x \mapsto p_{2,*} (p_1^* (x) \cdot \iota_{\eta}^* \xi (d)) \end{align*}$$

is an isomorphism of rational Hodge structures, that is also Galois invariant.

Remark 1.1. We note the following conceptual reason why we work with the group $\mathrm {U}_B(\tilde {V})$ which at Archimedean places is (almost) isomorphic to a product $\mathrm {O}(4,2)^d \times \mathrm {O}(0,6)^{n-d}$ . After all, in principle, one could also construct Kudla–Millson classes directly on the group $\mathrm {U}_B(V)$ , which at Archimedean places looks like a product $\mathrm {O}(2,2)^d \times \mathrm {O}(0,4)^{n-d}$ , by taking a lift of a form of parallel weight two. However, the issue is that on this smaller group, the Hodge classes are mixed up with other classes of the same degree, and therefore it is difficult to see that the Kudla–Millson class is in the $\mathbb {C}$ -span of the Hodge classes, except in the ‘trivial’ situation when $B_1=B_2$ ; in that case, the group $\mathrm {U}_B(V)$ is quasi-split and there are obvious ‘diagonal’ cycles in the correct degree. On the larger group, however, the Hodge classes in degree $(d,d)$ can be separated out using Hecke operators; this is the crucial idea on which the proof rests.

Remark 1.2. The assumption that $B_1$ and $B_2$ are ramified at some infinite place is made for technical reasons; it ensures that the auxiliary Shimura variety X used in the proof is compact. We believe that, with some extra work (e.g., working with intersection cohomology), this assumption could be relaxed.

Remark 1.3. Our proof of Theorem 1 requires the construction of the particular automorphic representation $\Pi $ on the unitary group $\operatorname {PGU}_E (\mathbf {V})$ and a precise characterization of the near equivalence class of this representation. We give two proofs of this characterization. The first proof uses the expected classification of nontempered automorphic representations on unitary groups (associated to Hermitian spaces over a CM field) in terms of local and global A-packets, which is work in progress of Kaletha, Minguez, Shin and White [Reference Kaletha, Minguez, Shin and White34]. The expected results from their work that we need are stated carefully in §11.1 and §11.2. But we also give another, more direct proof, of the characterization of this representation using the theta correspondence, that does not use [Reference Kaletha, Minguez, Shin and White34]. While this latter proof is unconditional, we have retained the proof using the full classification, since it provides a conceptual justification for why the method works, and since it may be useful in other situations.

Remark 1.4. As mentioned before, our proof of part (ii) of Theorem 1 is conditional on the truth of Kottwitz’s conjecture describing the Galois representations occurring in the cohomology of Shimura varieties in terms of automorphic representations. The main results on Galois representations that we need are stated in Propositions 11.8 and 11.9. In §11.5, we explain in some detail how these propositions follow from Kottwitz’s conjecture [Reference Kottwitz37].

While we do not prove any new results towards Kottwitz’s conjecture in this paper, it is an area of active investigation and the results we rely on will hopefully be available in the near future. For the benefit of the reader, we now explain what results towards this conjecture are currently available and what work still needs to be done. In loc. cit., Kottwitz outlined a strategy to prove the conjecture via establishing a stable trace formula and comparing it to the Grothendieck–Lefschetz trace formula. In the subsequent papers [Reference Kottwitz38], [Reference Kottwitz39], Kottwitz used this strategy to verify his conjecture for certain Shimura varieties of PEL type.

The Shimura varieties that we use are of abelian-type but not PEL. For abelian-type Shimura varieties, a stable trace formula and the comparison with the Grothendieck–Lefschetz trace formula has recently been established by Kisin–Shin–Zhu [Reference Kisin, Shin and Zhu35]. However, (as is explained in loc. cit. §0.2 and §9.2) two additional pieces of work need to be done to complete the characterization of Galois representations:

  1. (i) First, one needs an equality relating the stable distribution of [Reference Kisin, Shin and Zhu35] to the one in Kottwitz. This relation is encoded in the expected formula (9.2.2.1) of [Reference Kisin, Shin and Zhu35], which the authors of [Reference Kisin, Shin and Zhu35] are planning to investigate in a sequel to that paper.

  2. (ii) Second, one needs the classification of automorphic representations on unitary similitude groups in terms of A-parameters. The corresponding results for unitary groups are the subject of past and ongoing work of Kaletha–Minguez–Shin–White. The extension of these results from unitary groups to unitary similitude groups is also expected to be within reach.

Remark 1.5. At the request of one of the referees, we discuss the relation between this paper and the work of Bergeron–Millson-Mœglin (e.g., [Reference Bergeron, Millson and Mœglin6] and [Reference Bergeron, Millson and Mœglin5]), which proves many cases of the Hodge conjecture for certain orthogonal or unitary Shimura varieties. The strategy in those papers is to show that in a range of degrees, the space of Hodge classes on the varieties under consideration is spanned (for the most part) by the classes of Kudla–Millson cycles, which are linear combinations of cycle classes of sub-Shimura varieties, and are thus algebraic. (In some cases, for example $\mathrm {U}(2,2)$ , they also need to use classes that are known to be algebraic due to the Lefschetz- $(1,1)$ theorem, but are not obviously in the span of the classes of Kudla–Millson cycles.) Our work is complementary to this, and in a somewhat orthogonal direction, since in our setting, there are no obvious Kudla–Millson cycles in the degrees under consideration, nevertheless we construct interesting Hodge classes. For example, the simplest interesting setting for us (beyond $(1,1)$ -classes for $\mathrm {U}(1,1)$ which can be addressed using Lefschetz- $(1,1)$ ) is the case of $(2,2)$ -classes for $\mathrm {U}(2,2) \times \mathrm {U}(2,2)$ , which is not covered in loc. cit. Our expectation is that these Hodge classes (that represent functoriality) cannot be obtained from cycle classes of sub-Shimura varieties by any functorial process, even if one throws in classes that are known to be algebraic by the Lefschetz- $(1,1)$ theorem.

The reader may also be interested in the discussion in §1.3.3. As pointed out there, there are also situations where there are Kudla–Millson cycles in the degrees of interest, but they do not span the space of Hodge classes. Thus it seems that to understand whether these Hodge classes are algebraic requires studying algebraic cycles on Shimura varieties that do not arise from sub-Shimura varieties. This is a topic that has not seen much systematic work so far.

1.3 Extensions and generalizations

In this section, we discuss some extensions and generalizations of the main result stated above.

1.3.1 Local systems and normalizations

While we have stated the main result for trivial coefficients, it works equally well for local systems. In the main text, this more general case is treated.

We briefly mention the numerology in the case of general local systems. Suppose that the form $\pi $ has weights $\underline {k}=(k_1,\ldots , k_n)$ . Then (in the classical normalization) the Hodge structure of $H^*(X_i)_{\pi _i}$ is a tensor product over the places v in $\Sigma $ of a Hodge structure of type

$$\begin{align*}(k_v-1,0) + (0,k_v-1). \end{align*}$$

Thus, $H^*(X_1)_{\pi _1} \otimes H^*(X_2)_{\pi _2}$ is a tensor product over the places v in $\Sigma $ of a Hodge structure of type

(1.10) $$ \begin{align} (2k_v-2,0) + 2 (k_v-1,k_v-1) + (0,2k_v-2). \end{align} $$

The Hodge class in $H^*(X_1)_{\pi _1} \otimes H^*(X_2)_{\pi _2}$ should come from the tensor product over the places v in $\Sigma $ of a class of type $(k_v-1,k_v-1)$ . In our construction, we pick an auxiliary form $\tilde {\tau }$ of weights $\underline {k}+{\mathbf {1}} = (k_1+1, \ldots , k_n+1)$ . Then $\operatorname {JL}( \tilde {\tau })$ corresponds to a Hodge structure which is a tensor product over the places v in $\Sigma $ of a Hodge structure of type

$$\begin{align*}(k_v,0) + (0,k_v). \end{align*}$$

Its lift $\Pi $ to $\mathrm {U}_B(\tilde {V})$ contributes to different cohomological degrees, so there is an associated Hodge diamond which is the tensor product over the places v in $\Sigma $ of a Hodge diamond of the form

(1.11) $$ \begin{align} & \kern50pt (k_v+1,k_v+1) \\[6pt] &(2k_v,0) \quad\qquad 2 (k_v,k_v) \quad\qquad (0,2k_v) \nonumber\\[6pt] &\kern50pt (k_v-1,k_v-1).\nonumber \end{align} $$

The Hodge class in $H^*(X)_\Pi $ comes from the tensor product over the places v in $\Sigma $ of the class of type $(k_v-1,k_v-1)$ . The ‘rest’ of the Hodge structure at any place v consists of Tate twists of this $(k_v-1,k_v-1)$ class and $\mathrm {Sym}^2$ of the Hodge structure attached to $\tilde {\tau }$ .

In the main text, we use the ‘automorphic normalization’ instead of the classical normalization. This amounts to twisting the Hodge structures in equations (1.10) and (1.11) above by $(2-k_v,2-k_v)$ . This twist is therefore not visible in parallel weight $2$ .

1.3.2 The non-self-dual case

The assumption that $\pi $ has trivial central character forces $\pi $ to be self-dual and is just made for simplicity. The non-self-dual case can also be treated similarly; we do not treat this in the paper, but we outline here the main differences.

Let us denote the contragredient of $\pi $ by $\pi ^\vee $ , and let $\chi $ be the central character of $\pi $ so that $\pi \simeq \pi ^\vee \otimes \chi $ . The method described above extends to this case, except that we must choose $\eta $ such that $\eta |_{\mathbb {A}^\times _F} = \chi ^{-1} $ to compensate for the central character of $\pi $ . We remark on one unusual feature. Namely, it seems that the method outlined here naturally produces a Hodge class $\xi \in H^{2d}( X, \mathbb {Q}(\chi ^2))_\Pi $ such that

  1. (i) The induced map

    $$\begin{align*}\xi(d)^*: H^d(X_{B_1}, \mathbb{Q}(\chi^{-1}))_{\pi_1^\vee} \rightarrow H^d(X_{B_2}, \mathbb{Q}(\chi))_{\pi_2} \end{align*}$$
    is an isomorphism of $\mathbb {Q}(\chi )=\mathbb {Q}(\chi ^{-1})$ -vector spaces and preserves the Hodge filtration (on tensoring with $\mathbb {C}$ ).
  2. (ii) The Galois module $H^{2d}(X,\mathbb {Q}_\ell (\chi ^2))$ is isomorphic to (a sum of copies of) $\mathbb {Q}_\ell (-d) (\chi ^2)$ and the induced map

    $$\begin{align*}\xi(d) ^*: H^d(X_{B_1}, \mathbb{Q}_\ell(\chi^{-1}))_{\pi_1^\vee} \rightarrow H^d(X_{B_2}, \mathbb{Q}_\ell(\chi))_{\pi_2} \end{align*}$$
    satisfies the following Galois equivariance:
    $$\begin{align*}\sigma (\xi^*(x)) = \chi^2 (\sigma) \cdot \xi^* (\sigma (x)). \end{align*}$$
    We note that $\mathbb {Q}(\chi ^{-1})=\mathbb {Q}(\chi )$ and $\mathbb {Q}_\ell (\chi ^{-1})=\mathbb {Q}_\ell (\chi )$ .

The reason this is unusual is that one might expect to have a natural construction producing a rational Hodge class in $H^*(X_{B_1}, \mathbb {Q}(\chi ))_{\pi _1^\vee } \otimes H^*(X_{B_2}, \mathbb {Q}(\chi ))_{\pi _2}$ , since after all the Galois representation $H^*(X_{B_1}, \mathbb {Q}_\ell (\chi ))_{\pi _1^\vee } \otimes H^*(X_{B_2}, \mathbb {Q}_\ell (\chi ))_{\pi _2}$ always contains the trivial representation as a direct summand. Instead, our construction naturally produces a Hodge class (with coefficients in a number field) in $H^* (X_{B_1}, \mathbb {Q}(\chi ))_{\pi _1} \otimes H^*(X_{B_2}, \mathbb {Q}(\chi ))_{\pi _2}$ and then one has to ‘untwist’ it to produce the rational Hodge class that one expects to exist.

1.3.3 Absolute Hodge classes

The main theorem above is close to saying that the class $\xi $ is an absolute Hodge class in the sense of Deligne [Reference Deligne, Milne, Ogus and Shih15]. However, what is missing is the de Rham piece of the story, that is, in order to show that $\xi $ is absolutely Hodge, we would need to show in addition to the above that it is also de Rham rational and that for every embedding $\tau $ of $F_\Sigma $ in $\mathbb {C}$ , the class $\tau (\xi )$ is a Hodge class, whose image in $\ell $ -adic cohomology is Galois invariant for all $\ell $ . It seems difficult to show this directly. In a previous version of this paper, we expressed the hope that one might be able to deduce that $\xi $ is absolutely Hodge by showing that it satisfies a stronger property, namely that it is a motivated cycle in the sense of André. However, the strategy that we had in mind runs into a serious obstacle that we are unable to circumvent at the moment, so the problem of showing that $\xi $ is absolutely Hodge remains open. The obstacle is related to the following fact: There exist tempered L-packets $\Pi $ of representations on $\mathrm {U}_E(\mathbf {V})$ (with $\dim _E(\mathbf {V})=3$ ) which contribute to Hodge classes on the associated Shimura varieties such that the rank of the $\Pi $ -isotypic component of the space of algebraic cycles of group-theoretic origin (i.e., coming from embeddings $\mathrm {U}_E(\mathbf {V}') \hookrightarrow \mathrm {U}_E(\mathbf {V})$ , with $\dim _E(\mathbf {V}')=2$ ) is nonzero, yet is strictly smaller than the dimension of $\Pi $ -isotypic component of the space of Hodge classes. In particular, there exist Hodge classes on such varieties that are not represented by algebraic cycles coming from embedded unitary groups.

1.3.4 Functoriality for unitary groups

It would be very interesting to generalize the results of this paper to general unitary groups. The main obstacle to doing this seems to be understanding automorphic periods for the embedding

(1.12) $$ \begin{align} \mathrm{U}_E(\mathbf{V}_1) \times \mathrm{U}_E (\mathbf{V}_2) \hookrightarrow \mathrm{U}_E (\mathbf{V}), \end{align} $$

where $\dim _E(\mathbf {V}_1) = \dim _E(\mathbf {V}_2) = n$ say, with tempered representations $\pi _1$ , $\pi _2$ on $\mathrm {U}_E(\mathbf {V}_1)$ and $\mathrm {U}_E(\mathbf {V}_2)$ , respectively, and a nontempered representation $\pi $ on $\mathrm {U}_E (\mathbf {V})$ . In the case treated in this paper, this is accomplished for $n=2$ by using exceptional isogenies to relate the unitary groups above to inner forms of orthogonal groups and then using a seesaw to relate the requisite period integrals to triple product periods for $\mathrm {GL}_2$ , which are well understood and fall within the purview of the Gan–Gross–Prasad (GGP) conjectures. In the general case, these exceptional isogenies are not available. Thus, it seems important to formulate and prove analogs of the GGP conjecture in the setting of the equation (1.12) above.

2 Shimura varieties, local systems and motives

2.1 Realizations of motives

Some of our definitions below may be somewhat nonstandard.

2.1.1 Hodge structures

Let L be a number field given with a fixed embedding in $\mathbb {C}$ . An L-Hodge structure pure of weight n will be an L-vector space V equipped with a descending filtration $F^\cdot V_{\mathbb {C}}$ on $V_{\mathbb {C}} = V\otimes _L \mathbb {C}$ such that for $p+q=n+1$ , we have

$$\begin{align*}V_{\mathbb{C}} = F^p V_{\mathbb{C}} \oplus \overline{F^q V_{\mathbb{C}}}. \end{align*}$$

For any pair $(p,q)$ with $p+q=n$ , we set $V^{p,q} = F^p V_{\mathbb {C}} \cap \overline {F^q V_{\mathbb {C}}}$ .

2.1.2 Realizations of motives with coefficients

Let k and L be number fields. Let $\mathrm {Mot}_k^L$ denote the category of motives over k with coefficients in L. (For the moment, it is not very important what equivalence relation we use on algebraic cycles.) We are particularly interested in certain realization functors on $\mathrm {Mot}_k^L$ , assuming we are given embeddings $k\hookrightarrow \overline {\mathbb {Q}} \subset \mathbb {C}$ and $L\subset \mathbb {C}$ .

  • The Betti realization. The Betti realization $H_B (M)$ which is an L-Hodge structure.

  • The $\ell $ -adic realizations. For each rational prime $\ell $ , $H_\ell (M)$ is a free $L\otimes \mathbb {Q}_\ell $ -module, equipped with a continuous ( $L\otimes \mathbb {Q}_\ell $ -linear) action of $G_k:=\operatorname {Gal} (\overline {\mathbb {Q}}/k)$ .

We also have a natural comparison isomorphism

$$\begin{align*}H_B (M) \otimes_{\mathbb{Q}} \mathbb{Q}_\ell \simeq H_\ell (M) \end{align*}$$

of (free) $L\otimes \mathbb {Q}_\ell $ -modules.

There are other realizations which will not concern us in this paper. Thus, we define a category $\mathcal {M}_{k}^L$ as follows. The objects in this category are collections

$$\begin{align*}(V, V_\ell) \end{align*}$$

as $\ell $ varies over the primes, where V is an L-vector space equipped with an L-Hodge structure and $V_\ell $ is a free $L \otimes \mathbb {Q}_\ell $ -module with a continuous ( $L\otimes \mathbb {Q}_\ell $ -linear) action of $G_k$ , along with isomorphisms

$$\begin{align*}i_\ell: V \otimes \mathbb{Q}_\ell \simeq V_\ell. \end{align*}$$

A morphism between two such objects $(V,V_\ell , i_\ell )$ and $(V',V^{\prime }_\ell , i^{\prime }_\ell )$ is an L-linear map $j: V \rightarrow V'$ that is a morphism of L-Hodge structures such that the $L \otimes \mathbb {Q}_\ell $ -linear maps $j_\ell : V_\ell \rightarrow V^{\prime }_\ell $ defined by the commutative diagram below:

are $G_k$ -equivariant.

If $L=\mathbb {Q}$ , we omit the superscript and simply write $\mathcal {M}_k$ . Note that to any proper smooth variety X over k, we can attach objects

$$\begin{align*}{\mathcal{H}}^n (X) = \left(H^n (X (\mathbb{C}), \mathbb{Q}), H^n_{\mathrm{et}} (X_{\overline{\mathbb{Q}}}, \mathbb{Q}_\ell), i_\ell \right) \end{align*}$$

in the category $\mathcal {M}_k$ .

If $L\subset L'\subset \mathbb {C}$ , there is a natural functor $\mathcal {M}_k^L \rightarrow \mathcal {M}_k^{L'}$ , sending $(V,V_\ell )$ to $(V\otimes _L L', V_\ell \otimes _L L' = V_\ell \otimes _{L\otimes \mathbb {Q}_\ell } (L'\otimes \mathbb {Q}_\ell )).$

2.2 Shimura varieties and local systems

2.2.1 Shimura varieties

We recall some basic facts about Shimura varieties [Reference Deligne14]. Let $\mathbb {S} = \operatorname {Res}_{\mathbb {C}/\mathbb {R}} \mathbb {G}_m$ denote the Deligne torus. As usual, a Shimura datum is a pair $(G,X)$ consisting of a reductive algebraic group G over $\mathbb {Q}$ and a $G(\mathbb {R})$ -conjugacy class X of homomorphisms $h:\mathbb {S} \rightarrow G_{\mathbb {R}}$ satisfying the following conditions:

  1. (i) For h in X, the Hodge structure on the Lie algebra $\mathfrak {g}$ of $G_{\mathbb {R}}$ given by ${\operatorname {Ad}} \circ h$ is of type $(0,0) + (-1,1) + (1,-1)$ . (In particular, the restriction of such an h to $\mathbb {G}_{m,\mathbb {R}} \subset \mathbb {S}$ has image in the center of $G_{\mathbb {R}}$ .)

  2. (ii) For h in X, $({\operatorname {Ad}} \circ h)(i)$ is a Cartan involution on $G_{\mathbb {R}}^{\mathrm {ad}}$ , where $G^{\mathrm {ad}}$ is the adjoint group of G.

  3. (iii) $G^{\mathrm {ad}}$ has no factor defined over $\mathbb {Q}$ whose real points form a compact group.

These conditions imply that X has the natural structure of a disjoint union of Hermitian symmetric domains. The group $G(\mathbb {R})$ acts on X on the left by

$$\begin{align*}(g \cdot h)(z) = g\cdot h(z) \cdot g^{-1}. \end{align*}$$

Let $\mathbb {A}$ and $\mathbb {A}_f$ denote, respectively, the ring of adèles and finite adèles of $\mathbb {Q}$ . Let $\mathcal {K}$ be an open compact subgroup of $G(\mathbb {A}_f)$ . The Shimura variety associated to $(G,X,\mathcal {K})$ is the quotient

$$\begin{align*}\mathrm{Sh}_{\mathcal{K}} (G,X) = G(\mathbb{Q}) \backslash X \times G(\mathbb{A}_f)/ \mathcal{K}. \end{align*}$$

For $\mathcal {K}$ small enough, this has the natural structure of a smooth variety over $\mathbb {C}$ . The inverse limit

$$\begin{align*}\mathrm{Sh} (G,X) = \underleftarrow{\lim}_{\mathcal{K}} \ \mathrm{Sh}_{\mathcal{K}} (G,X) \end{align*}$$

is a proalgebraic variety that has a canonical model over a number field $E(G,X)$ , the reflex field of the Shimura datum $(G,X)$ . In particular, each $\mathrm {Sh}_{\mathcal {K}} (G,X)$ has a canonical model over $E(G,X)$ . For brevity of notation, we will often write simply $\mathrm {Sh}_G$ or $\mathrm {Sh}_{G,\mathcal {K}}$ since X will be understood from context.

We recall the definition of $E(G,X)$ . This field is defined to be the field of definition of the conjugacy class of cocharacters

$$\begin{align*}\mu_h: \mathbb{G}_{m,\mathbb{C}} \rightarrow \mathbb{S}_{\mathbb{C}} \rightarrow G_{\mathbb{C}}, \end{align*}$$

where the first map is $z\mapsto (z,1)$ and the second is the one induced by h.

Remark 2.1. The field $E(G,X)$ is given as a subfield of $\mathbb {C}$ , and as such has by definition a canonical embedding into $\mathbb {C}$ . When not specified below, any embedding of $E(G,X)$ in $\mathbb {C}$ will always be this canonical embedding. Indeed, we will not have use for any other embedding.

All Shimura varieties occurring in this paper will be compact, so we will assume this to be the case in the rest of this chapter.

2.2.2 Local systems and cohomology

Let $(\rho ,V)$ be a finite-dimensional representation of G defined over a number field $L\subset \mathbb {C}$ . We assume that $\rho $ factors through an action of $G/Z_s$ , where $Z_s$ is the largest subtorus of the center of G which is split over $\mathbb {R}$ but which has no subtorus split over $\mathbb {Q}$ . To the data $(G,X,\rho )$ , we can associate the following:

  1. (i) A local system $\mathbb {V}$ of L-vector spaces on $\mathrm {Sh}_G$ .

  2. (ii) For each prime $\ell $ , an $\ell $ -adic local system $\mathbb {V}_{\ell }$ (of $L\otimes \mathbb {Q}_\ell $ -vector spaces) on $\mathrm {Sh}_G$ .

Then $H^i (\mathrm {Sh}_{G,\mathcal {K}} (\mathbb {C}) , \mathbb {V})$ is an L-vector space (in fact, an L-Hodge structure) and there are natural isomorphisms of free $L\otimes \mathbb {Q}_\ell $ -modules

(2.1) $$ \begin{align} H^i (\mathrm{Sh}_{G,\mathcal{K}} ( \mathbb{C}), \mathbb{V}) \otimes_{\mathbb{Q}} \mathbb{Q}_\ell \simeq H^i_{\mathrm{et}} (\mathrm{Sh}_{G,\mathcal{K}} \otimes_{E(G,X)} \overline{\mathbb{Q}} , \mathbb{V}_{\ell}) \end{align} $$

(see [73, Exposé XI]). Note that we are using the given embedding $E(G,X) \hookrightarrow \overline {\mathbb {Q}} \subset \mathbb {C}$ on both sides of the isomorphism above. The Hecke algebra ${\mathcal {H}} (G(\mathbb {A}_f), \mathcal {K})$ acts on both sides of equation (2.1) and the isomorphism is Hecke equivariant. Taking the direct limit over $\mathcal {K}$ , we get an isomorphism,

$$\begin{align*}H^i (\mathrm{Sh}_{G} ( \mathbb{C}), \mathbb{V}) \otimes_{\mathbb{Q}} \mathbb{Q}_\ell \simeq H^i_{\mathrm{et}} (\mathrm{Sh}_{G} \otimes_{E(G,X)} \overline{\mathbb{Q}} , \mathbb{V}_{\ell}). \end{align*}$$

Let $\Pi $ be an irreducible cohomological automorphic representation of $G(\mathbb {A})$ . The $\Pi $ -isotypic component of $H^i(\mathrm {Sh}_{G} ( \mathbb {C}), \mathbb {V}_{\mathbb {C}})$ is defined to be

$$ \begin{align*} H^i (\mathrm{Sh}_{G} ( \mathbb{C}), \mathbb{V}_{\mathbb{C}})_\Pi & := \operatorname{Hom}_{{\mathcal{H}}(G(\mathbb{A}_f),\mathcal{K})} (\Pi_f^{\mathcal{K}}, H^i (\mathrm{Sh}_{G} ( \mathbb{C}), \mathbb{V}_{\mathbb{C}} )^{\mathcal{K}} ) \\ & \hspace{2pt}= \operatorname{Hom}_{{\mathcal{H}}(G(\mathbb{A}_f),\mathcal{K})} (\Pi_f^{\mathcal{K}}, H^i (\mathrm{Sh}_{G,\mathcal{K}} ( \mathbb{C}), \mathbb{V}_{\mathbb{C}} ) ) \end{align*} $$

for $\mathcal {K}$ small enough, this being independent of the choice of $\mathcal {K}$ . By Matsushima’s formula [Reference Borel and Wallach10],

$$\begin{align*}H^i (\mathrm{Sh}_{G,\mathcal{K}} ( \mathbb{C}), \mathbb{V}_{\mathbb{C}}) \simeq \bigoplus_{\pi} m(\pi) H^i (\mathfrak{g}, K; \pi_{\infty} \otimes \mathbb{V}_{\mathbb{C}}) \otimes \pi_f^{\mathcal{K}}, \end{align*}$$

where the sum is over automorphic representations $\pi = \pi _{\infty } \otimes \pi _f$ of $G(\mathbb {A})$ and $m(\pi )$ is the multiplicity of $\pi $ in the discrete spectrum of G. It follows that

$$\begin{align*}H^i (\mathrm{Sh}_{G} ( \mathbb{C}), \mathbb{V}_{\mathbb{C}})_\Pi \simeq \bigoplus_{\pi, \pi_f^{\mathcal{K}} \simeq \Pi_f^{\mathcal{K}}} m(\pi) H^i (\mathfrak{g}, K; \pi_{\infty} \otimes \mathbb{V}_{\mathbb{C}}), \end{align*}$$

where the sum is over those $\pi $ such that $\pi _f^{\mathcal {K}} \simeq \Pi _f^{\mathcal {K}}$ as ${\mathcal {H}}(G(\mathbb {A}_f),\mathcal {K})$ -modules.

2.2.3 Pullback and pushforward

Let $f:(G,X_1) \rightarrow (H,X_2)$ be a morphism of Shimura data. We assume that the reflex fields of $(G,X_1)$ and $(H,X_2)$ are the same subfield E of $\mathbb {C}$ . Let $\rho $ be a finite-dimensional representation of H defined over a number field $L\subset \mathbb {C}$ (which we can also view as a representation of G via the map $G\rightarrow H$ ) and denote by $\mathbb {V}$ the associated local systems on $\mathrm {Sh}_H$ , $\mathrm {Sh}_G$ . (Thus, the local system on $\mathrm {Sh}_G$ is just obtained by pullback from $\mathrm {Sh}_H$ .) Then there are functorial maps

$$\begin{align*}f^*: H^i(\mathrm{Sh}_H, \mathbb{V}) \rightarrow H^i(\mathrm{Sh}_G, \mathbb{V}), \end{align*}$$

defined both in Betti and $\ell $ -adic cohomology, which may be viewed as giving a morphism in the category $\mathcal {M}_E^L$ . Suppose in addition that:

  1. (i) $G\rightarrow H$ is surjective with kernel Z contained in the center of G.

  2. (ii) Z is cohomologically trivial so that $G(\mathbb {A}) \rightarrow H(\mathbb {A})$ is surjective as well.

Then there is a bijection between automorphic representations $\Pi _H$ of $H(\mathbb {A})$ and $\Pi _G$ of $G(\mathbb {A})$ on which Z acts trivially. Assuming that L contains the (common) field of definition of $\Pi _H$ and $\Pi _G$ , the map $f^*$ induces an isomorphism,

$$\begin{align*}H^i(\mathrm{Sh}_H, \mathbb{V})_{\Pi_H} \simeq H^i(\mathrm{Sh}_G, \mathbb{V})_{\Pi_G}, \end{align*}$$

in $\mathcal {M}^L_E$ .

We will also need to consider the case when the map $G\rightarrow H$ satisfies equation ((i)) but not equation ((ii)). Typically, in such cases, we will be interested in maps in the opposite direction. Indeed, there is a natural pushforward map:

$$\begin{align*}f_*: H^i(\mathrm{Sh}_{G}, \mathbb{V}) \rightarrow H^i(\mathrm{Sh}_{H}, \mathbb{V}). \end{align*}$$

If $\mathcal {K}_1$ and $\mathcal {K}_2$ are (small enough) open compact subgroups of $G(\mathbb {A}_f)$ and $H(\mathbb {A}_f)$ , respectively, the induced map

$$\begin{align*}\mathrm{Sh}_{G,\mathcal{K}_1} \rightarrow \mathrm{Sh}_{H,\mathcal{K}_2} \end{align*}$$

is finite étale onto its image which is a union of components of $\mathrm {Sh}_{H,\mathcal {K}_2}$ . Thus, $f_*$ can be defined by taking the trace to the image and then extending by zero outside the image.

Remark 2.2. To make the definition of $f_*$ independent of the choice of $\mathcal {K}_1$ and $\mathcal {K}_2$ , we need to normalize it by multiplying by the factor $\operatorname {vol} (\mathcal {K}_1)/\operatorname {vol}(\mathcal {K}_2)$ for some choice of Haar measures on $G(\mathbb {A}_f)$ and $H(\mathbb {A}_f)$ . In our application, we will implicitly make such a choice in §10.4 and §12, but the exact choice is unimportant.

3 Quaternionic Shimura varieties and the main theorem

3.1 Quaternionic Shimura varieties

Let F be a totally real field and $\Sigma _{\infty }$ the set of infinite places of F. Let B be a nonsplit quaternion algebra over F and $\Sigma $ the set of infinite places of F where B is split. Put $n = [F:\mathbb {Q}]$ and $d = |\Sigma |$ . We fix an isomorphism,

$$\begin{align*}B\otimes_F \mathbb{R} \simeq \mathrm{M}_2(\mathbb{R})^d \times {\mathbb{H}}^{n-d}, \end{align*}$$

which gives an identification

$$\begin{align*}G_B (\mathbb{R}) \simeq \mathrm{GL}_2(\mathbb{R})^d \times ({\mathbb{H}}^\times)^{n-d}, \end{align*}$$

where $G_B:=\operatorname {Res}_{F/\mathbb {Q}} B^\times $ . Let $\tau _v$ denote the composite map

$$\begin{align*}B \otimes_{F,\sigma_v} \mathbb{R} \simeq \mathrm{M}_2 (\mathbb{R}) \hookrightarrow \mathrm{M}_2 (\mathbb{C}) \end{align*}$$

for $v\in \Sigma $ and

$$\begin{align*}B \otimes_{F,\sigma_v} \mathbb{R} \simeq {\mathbb{H}} \hookrightarrow \mathrm{M}_2 (\mathbb{C}) \end{align*}$$

for $v\in \Sigma _{\infty }\smallsetminus \Sigma $ . Then $\tau _v$ may be viewed as giving a two-dimensional complex representation of $G_B$ . We identify $\mathbb {C}^\times $ with a subgroup of $\mathrm {GL}_2 (\mathbb {R})$ via

(3.1) $$ \begin{align} z=a+bi \mapsto \iota(z) := \begin{pmatrix} a& b \\ -b & a \end{pmatrix}. \end{align} $$

Let X denote the $G_B(\mathbb {R})$ -conjugacy class of

$$\begin{align*}h: \mathbb{S} \rightarrow G_{B,\mathbb{R}}, \quad h(z) = \left(\iota(z),\cdots, \iota(z), 1, \cdots ,1\right) \end{align*}$$

so that $h_v(z):=h(z)_v=\iota (z)$ for $v \in \Sigma $ and $h_v (z) = 1$ for $v \in \Sigma _{\infty } \smallsetminus \Sigma $ . We write either $\mathrm {Sh}_{G_B}$ (or for ease of notation, simply $\mathrm {Sh}_B$ ) for the associated Shimura variety. The variety $\mathrm {Sh}_B$ admits a canonical model over the reflex field $F_\Sigma $ . The Hecke algebra ${\mathcal {H}} (G_B (\mathbb {A}_f), \mathcal {K})$ acts on $\mathrm {Sh}_{B,\mathcal {K}}$ via correspondences. Moreover, the inverse limit

$$\begin{align*}\mathrm{Sh}_B = \varprojlim_{\mathcal{K}} \mathrm{Sh}_{B,\mathcal{K}} \end{align*}$$

admits a right $G_B( \mathbb {A}_f)$ -action. We refer the reader to [Reference Ichino and Prasanna30, §1] for a more detailed discussion of the Shimura varieties $\mathrm {Sh}_B$ .

3.2 Local systems

Let $\pi $ be an automorphic representation of $\mathrm {GL}_2 (\mathbb {A}_F)$ attached to a holomorphic Hilbert modular newform of weight $(\underline {k},r)$ , where $\underline {k}=(k_1, \ldots , k_n)$ is a collection of integers of the same parity and r is an integer with $k_i\equiv r \bmod 2$ . (We will often denote $k_i$ by $k_v$ if v is the ith place in the ordering and write $\underline {k} = (k_v)_{v \in \Sigma _{\infty }}$ .) We suppose that $\pi $ admits a Jacquet–Langlands transfer $\pi _B$ to $G_B(\mathbb {A})$ . We also assume that $k_i\ge 2$ for all i. This implies that the representation $\pi _{B,\infty }$ is cohomological, namely $\pi _B$ contributes to the cohomology of a local system on $\mathrm {Sh}_B$ . The local system is attached to the representation $\tau _{\underline {k},r}^\vee $ of $G_B(\mathbb {C})$ , where

$$ \begin{align*} \tau_{\underline{k},r} & := \bigotimes_{v} (\sigma_v \circ \nu)^{(r-k_v+2)/2}\mathrm{Sym}^{k_v-2} (\tau_v ) \\ &\hspace{2pt} = \bigotimes_v ({\det} \circ \tau_v )^{(r-k_v+2)/2} \mathrm{Sym}^{k_v-2} (\tau_v). \end{align*} $$

Here, $\nu $ denotes the reduced norm on B. If $k_i$ is even for all i, then by [Reference Waldspurger66] (Proposition I.3 and §II.2), the restriction of the representation $\tau _{\underline {k},r}$ to the group $G_B$ is defined over (any field L containing) $\mathbb {Q}(\underline {k})$ , where $\mathbb {Q}(\underline {k})$ is the fixed field of the subgroup

$$\begin{align*}\{ \sigma \in \operatorname{Aut}(\mathbb{C}/\mathbb{Q}) \, | \, \sigma \underline{k} = \underline{k} \} \end{align*}$$

with $\sigma \underline {k} = (k_{\sigma ^{-1} \circ v})_{v \in \Sigma _{\infty }}$ . More precisely, this representation contains an L-structure invariant by $G_B$ and that is unique up to homothety.

Remark 3.1. In this paper, we are only concerned with the case when $\pi $ has trivial central character up to twisting by a power of the reduced norm. This implies that the weights $k_i$ must all be even. Then, by twisting $\pi $ by a power of the norm character, we may assume that $r=0$ . For simplicity, we will thus make this assumption for the rest of the paper and drop r from the notation. Thus, we will just write $\tau _{\underline {k}}$ below. (For the more general case of nontrivial central characters, see §1.3.2.)

Let $L=\mathbb {Q}(\pi )$ be the field of rationality of $\pi $ , as defined in [Reference Waldspurger66], §1.8. By loc. cit. Corollary I.8.3 and Lemma I.2.3, this field contains $\mathbb {Q}(\underline {k})$ and also agrees with the field generated by (all but finitely many, in particular the unramified) Hecke eigenvalues of $\pi $ . Thus, we may view $\tau _{\underline {k}}$ as being defined over L, and then we get an associated local system of L-vector spaces $\mathbb {V}_{\underline {k}}(L)$ on $\mathrm {Sh}_B (\mathbb {C})$ and for every finite prime $\ell $ , an étale $L\otimes \mathbb {Q}_\ell $ -sheaf $\mathbb {V}_{\underline {k}}(L)_{\ell }$ on $\mathrm {Sh}_B$ . (See also [Reference Carayol13], §2.1.) Let

$$ \begin{align*} V^{\mathcal{K}}_B (L) &:=H^* (\mathrm{Sh}_{B, \mathcal{K}}(\mathbb{C}), \mathbb{V}_{\underline{k}}(L)), \\ V^{\mathcal{K}}_B(\mathbb{C})&:=H^* (\mathrm{Sh}_{B, \mathcal{K}}(\mathbb{C}), \mathbb{V}_{\underline{k}}(\mathbb{C}) ), \\ V_{B}^{\mathcal{K}}(L)_\ell&:= H^*_{\mathrm{et}} (\mathrm{Sh}_{B, \mathcal{K}} \otimes_{F_\Sigma} \overline{\mathbb{Q}}, \mathbb{V}_{\underline{k},\ell}) \end{align*} $$

so that there are canonical isomorphisms

$$\begin{align*}V^{\mathcal{K}}_B (L) \otimes_L \mathbb{C} \simeq V^{\mathcal{K}}_B(\mathbb{C}) \end{align*}$$

and

$$\begin{align*}V^{\mathcal{K}}_B (L) \otimes_{\mathbb{Q}} \mathbb{Q}_\ell \simeq V_{B}^{\mathcal{K}}(L)_\ell. \end{align*}$$

We fix an isomorphism

$$\begin{align*}B \otimes \mathbb{A}_F^S \simeq \mathrm{M}_2(\mathbb{A}_F^S), \end{align*}$$

where $\mathbb {A}_F^S$ denotes the adèles of F outside a finite set of places S containing $\Sigma _{\infty }$ and all finite places where B is ramified. This gives an isomorphism

(3.2) $$ \begin{align} B^\times (\mathbb{A}_F^S) \simeq \mathrm{GL}_2(\mathbb{A}_F^S). \end{align} $$

We assume that $\pi $ transfers to $B^\times (\mathbb {A}_F)$ , that is, there exists an automorphic representation $\pi _B= \pi _{B,\infty } \otimes \pi _B^f$ of $B^\times (\mathbb {A}_F)$ (necessarily unique by strong multiplicity one) such that $\pi _B^S \simeq \pi ^S$ via the identification (3.2) above.

For the cohomology with complex coefficients, we can define the $\pi _B$ -isotypic component by

$$\begin{align*}V_{B,\pi_B} (\mathbb{C}):= \operatorname{Hom}_{{\mathcal{H}}_{\mathbb{C}} (G_B(\mathbb{A}_f), \mathcal{K})} ((\pi_B^f)^{\mathcal{K}}, V_B^{\mathcal{K}} (\mathbb{C})), \end{align*}$$

for $\mathcal {K}$ small enough. This is concentrated in degree $2d$ and is independent of the choice of $\mathcal {K}$ . To work over the field of rationality, we note that by [Reference Waldspurger66, Lemma 1.2.2 and §II.1], the Hecke module $(\pi _B^f)^{\mathcal {K}}$ is also defined over L. More precisely, it contains an L-structure $(\pi _B^f)^{\mathcal {K}} (L)$ that is invariant by the Hecke algebra with $\mathbb {Q}$ -coefficients, ${\mathcal {H}}_{\mathbb {Q}} (G_B(\mathbb {A}_f), \mathcal {K})$ , and that is unique up to homothety. This allows us to define the $\pi _B$ -isotypic components

$$ \begin{align*} V_{B,\pi_B} & := \operatorname{Hom}_{{\mathcal{H}}_{\mathbb{Q}} (G_B(\mathbb{A}_f), \mathcal{K})} ((\pi_B^f)^{\mathcal{K}}(L), V_B^{\mathcal{K}}(L)), \\ V_{B,\pi_B, \ell} & := \operatorname{Hom}_{{\mathcal{H}}_{\mathbb{Q}} (G_B(\mathbb{A}_f), \mathcal{K})} ((\pi_B^f)^{\mathcal{K}} (L), V_{B}^{\mathcal{K}}(L)_\ell), \end{align*} $$

for $\mathcal {K}$ small enough, these being independent of the choice of $\mathcal {K}$ . Moreover, there are canonical isomorphisms of free $L\otimes \mathbb {Q}_\ell $ -modules

$$\begin{align*}V_{B,\pi_B} \otimes_{\mathbb{Q}} \mathbb{Q}_\ell \simeq V_{B,\pi_B, \ell}. \end{align*}$$

3.3 The main theorem in the general case

We can now state the main theorem in the case of general local systems. Let $B_1$ and $B_2$ be two quaternion algebras that are split at the same set of Archimedean places $\Sigma \subset \Sigma _{\infty }$ such that $\pi $ transfers to both $B_1^\times (\mathbb {A}_F)$ and $B_2^\times (\mathbb {A}_F)$ . For ease of notation, we write the transfers as $\pi _1$ and $\pi _2$ instead of $\pi _{B_1}$ and $\pi _{B_2}$ , respectively.

Theorem 3.2. Suppose that there is at least one infinite place of F at which $B_1$ and $B_2$ are ramified.

  1. (i) Let L be the coefficient field of $\pi $ . Then there is an isomorphism of L-Hodge structures

    $$\begin{align*}\iota: V_{B_1,\pi_1} \simeq V_{B_2,\pi_2}. \end{align*}$$
  2. (ii) Assume Kottwitz’s conjecture for Shimura varieties attached to unitary similitude groups. Then the isomorphism $\iota $ of part (i) can be chosen such that, for all finite primes $\ell $ , the maps $\iota _\ell $ , defined by requiring the diagram

    (3.3)
    be commutative, are $\operatorname {Gal}(\overline {\mathbb {Q}}/ F_\Sigma )$ -isomorphisms.

This theorem will be proved in §12.

4 Unitary and quaternionic unitary Shimura varieties

The proof of the main theorem will require working with several different auxiliary Shimura varieties, some that are associated with unitary groups and some with quaternionic unitary groups. In this section, we introduce the main actors and the relations between them. Many of the claims below will only be justified in the following section; however, we believe it is more transparent to introduce all the different groups up front and relegate the details of various isomorphisms and maps to §5. The reader may want to read these sections in parallel.

4.1 Unitary and quaternionic unitary groups

Let F be a totally real field. Let $B_1$ and $B_2$ be two quaternion algebras that are split at the same set of infinite places of F. Let E be a CM extension of F that embeds in both $B_1$ and $B_2$ . We fix such embeddings $E \hookrightarrow B_1$ , $E \hookrightarrow B_2$ and write

$$\begin{align*}B_1 = E + E\mathbf{j}_1, \quad B_2 = E+ E\mathbf{j}_2 \end{align*}$$

for some trace zero elements $\mathbf {j}_1 \in B^\times _1$ , $\mathbf {j}_2 \in B^\times _2$ . We write $\operatorname {pr}_i$ for the projection $B_i \rightarrow E$ onto the ‘first coordinate’ and $*_i$ for the main involution on $B_i$ . Then $\mathbf {V}_i:=B_i$ is a right Hermitian E-space, the form being given by

$$ \begin{align*}(x,y)_i = \operatorname{pr}_i (x^{*_i} y).\end{align*} $$

If $x=a+\mathbf {j}_i b$ , $y= c+\mathbf {j}_i d$ , then

$$ \begin{align*}(x,y)_i = (a + \mathbf{j}_i b, c+\mathbf{j}_i d)_i = a^\rho c - J_i b^\rho d,\end{align*} $$

where $\rho $ is the nontrivial Galois automorphism of $E/F$ and $J_i = \mathbf {j}_i^2 \in F^\times $ . This form satisfies the relations

$$ \begin{align*}(x\alpha, y\beta)_i = \alpha^\rho (x,y)_i \beta\end{align*} $$

for $\alpha ,\beta \in E$ and

$$ \begin{align*}(x,y)_i = (y,x)_i^\rho.\end{align*} $$

Then

$$\begin{align*}\mathcal{G}_1 := \mathrm{GU}_E (\mathbf{V}_1) \simeq (B_1^\times \times E^\times)/F^\times, \quad \mathcal{G}_2:=\mathrm{GU}_E (\mathbf{V}_2) \simeq (B_2^\times \times E^\times)/F^\times, \end{align*}$$

where the (inverses of these) isomorphisms are given by $(\beta , \alpha ) \mapsto (x \mapsto \beta x \alpha ^{-1})$ . Let

$$\begin{align*}\mathcal{G} = \mathrm{G} (\mathrm{U}_E (\mathbf{V}_1) \times \mathrm{U}_E (\mathbf{V}_2))/E^\times = \mathrm{G} \left( (B_1^\times \times E^\times)/F^\times \times (B_2^\times \times E^\times)/F^\times\right)/E^\times, \end{align*}$$

where $E^\times $ embeds as $\alpha \mapsto ([1,\alpha ], [1,\alpha ])$ . We define groups $\tilde {\mathcal {G}}$ and $\mathcal {G}_0$ that are closely related to $\mathcal {G}$ as follows:

$$\begin{align*}\tilde{\mathcal{G}} = \mathrm{G} (\mathrm{U}_E (\mathbf{V}_1) \times \mathrm{U}_E (\mathbf{V}_2)), \quad \mathcal{G}_0 = B_1^\times/F^\times \times B_2^\times/F^\times. \end{align*}$$

Let $\mathbf {V} = \mathbf {V}_1 \oplus \mathbf {V}_2$ , which is a four-dimensional E-Hermitian space. Also, let $\tilde {\mathbf {V}} =\wedge ^2 (\mathbf {V})$ . In §5.2, we will show that $\tilde {\mathbf {V}}$ is naturally equipped with the structure of a right B-space, where $B:=B_1 \cdot B_2$ is the quaternion algebra over F whose class in the Brauer group of F equals the product of the classes of $B_1$ and $B_2$ . (Note that B is split at all the infinite places of F.) When we want to think of $\tilde {\mathbf {V}}$ as a B-space, we will write instead $\tilde {V}$ for it. Moreover, we show that $\tilde {V}$ is equipped with a B-skew-Hermitian form such that there is a canonical isomorphism

$$\begin{align*}\mathrm{GU}_E(\mathbf{V})/E^\times=\operatorname{PGU}_E(\mathbf{V}) \simeq \operatorname{PGU}_B(\tilde{V})^0= \mathrm{GU}_B(\tilde{V})^0/F^\times. \end{align*}$$

There is also a canonical decomposition $\tilde {V} = V^\sharp \oplus V^\sharp _0$ of B-skew-Hermitian spaces. Let

We regard these as algebraic groups over $\mathbb {Q}$ by restriction of scalars. We then have the following diagram, which we also write out in gory detail below. (Here, the dual notation in the right-most column indicates also the notation used (locally) in §10.)

Here, the maps $\operatorname {pr}$ , p and q are the obvious projection maps. We write down formulas for some of the maps as well:

$$ \begin{align*} F^\times \subset (B_i^\times \times E^\times), & \quad t\mapsto (t,t), \\ E^\times \subset \mathrm{G}((B_1^\times \times E^\times)/F^\times \times (B_2^\times \times E^\times)/F^\times), & \quad \alpha \mapsto ([1,\alpha], [1,\alpha] ),\\ F^\times \subset (B_1^\times \times B_2^\times), & \quad t\mapsto (t,t^{-1}), \\ F^\times \subset \mathrm{G} ((B_1^\times \times B_2^\times)/F^\times \times E^\times), & \quad t\mapsto ([t,1],t)=([1,t],t), \\ \xi \left( [ [b_1,\alpha_1], [b_2,\alpha_2]] \right)&=[ [b_1, b_2], \nu(b_1) \alpha_1^{-1} \alpha_2)], \end{align*} $$

where the map $\xi $ is given in the diagram and $\nu $ denotes the reduced norm.

4.2 Shimura data

All the groups in the diagram have associated Shimura varieties, defined such that the maps in the diagram induce morphisms of Shimura data. It suffices to describe the Shimura datum for $\tilde {\mathcal {G}}$ and $\tilde {\mathcal {G}}_B$ since the Shimura data for all the other groups are defined by composing with the maps above. For $\tilde {\mathcal {G}}$ , this is given by

$$\begin{align*}h_v(z) = ([\iota(z),1], [\iota(z),1]) \end{align*}$$

at the infinite places $v \in \Sigma $ and

$$\begin{align*}h_v(z) = ([1,1], [1,1]) \end{align*}$$

at the other infinite places. For $\tilde {\mathcal {G}}_B$ , this is given by

$$\begin{align*}h_v(z) = ([\iota(z),\iota(z)], z\bar{z}) \end{align*}$$

at the infinite places $v \in \Sigma $ and

$$\begin{align*}h_v(z) = ([1,1], 1) \end{align*}$$

at the other infinite places. In §12.2.2, we will write out the Shimura data more explicitly for some of the other groups in the diagram.

4.3 Components

Later (in §12), we will need to use the structure of the components of $\mathrm {Sh}_{\tilde {\mathcal {G}}_B}$ . Let $G_1:=\tilde {\mathcal {G}}_B$ and $G_2:=(B_1^\times \times B_2^\times )/F^\times $ . We may consider the canonical sequences

$$\begin{align*}1 \rightarrow G_i^{\mathrm{der}} \rightarrow G_i \rightarrow T_i \rightarrow 1, \end{align*}$$

where $G_i^{\mathrm {der}}$ denotes the derived group and $T_i$ the maximal commutative quotient of $G_i$ . The map p induces a map of exact sequences as below:

The set of components of $\mathrm {Sh}_{\tilde {\mathcal {G}}_B}$ is in bijection with the Shimura variety attached to $(T_1,h_1)$ , where

$$\begin{align*}T_1= \mathrm{G} \left((F^\times \times F^\times)/F^\times \times E^\times\right) , \quad h_1 (z) =([z\bar{z},z\bar{z}], z\bar{z}). \end{align*}$$

Now,

$$\begin{align*}Z(G_1) \simeq \{ (t,\alpha) \in F^\times \times E^\times: \ t^2 = \mathrm{N} (\alpha)\}, \end{align*}$$

the inverse of this isomorphism being given by $(t,\alpha ) \mapsto ([t,1],\alpha ) = ([1,t],\alpha ) $ . The natural map $Z(G_1) \rightarrow T_1$ is given by

$$\begin{align*}(t,\alpha) \mapsto ([t^2,1], \alpha) = ([t,t],\alpha)= ([1,t^2],\alpha) \end{align*}$$

and induces an isomorphism

$$\begin{align*}Z(G_1)/\langle (-1,1) \rangle \simeq T_1. \end{align*}$$

Note that any finite order character $\eta $ of $T_1(\mathbb {Q})\backslash T_1(\mathbb {A})$ gives rise to a class in $H^0 (\mathrm {Sh}_{\tilde {\mathcal {G}}_B}, \mathbb {Q}(\eta ))$ , where $\mathbb {Q}(\eta )$ is the field generated by the values of $\eta $ . We will denote this class $c_\eta $ . Of particular interest to us are the characters obtained as follows: We fix a finite order character $\eta $ of $E^{(1)} \backslash \mathbb {A}^{(1)}_E$ and define a character of $T_1$ by

$$\begin{align*}\eta ([t_1,t_2],\alpha) = \eta ((t_1 t_2)^{-1} \alpha). \end{align*}$$

The pullback of this character to $Z(G_1)$ is given by

$$\begin{align*}\eta (t,\alpha) = \eta (t^{-1} \alpha). \end{align*}$$

4.4 Automorphic forms and cohomology of local systems

Recall that we have the following relation between unitary and quaternionic unitary groups given by the top line of the diagram above:

Since $E^\times $ and $F^\times $ are cohomologically trivial, the maps

and

are surjective. Hence, the isomorphism in the middle induces a natural bijection between automorphic representations of

with trivial central character and those of

with trivial central character. Thus, if $\Pi $ is an automorphic representation of any of the groups at the ends with trivial central character, it may be viewed as an automorphic representation of any of the other groups above; we denote all such representations by the same symbol $\Pi $ . Moreover, if $(\rho ,\mathbb {V}_\rho )$ is a finite-dimensional representation (again of one of the groups at the end but trivial on the center) defined over a field L containing the reflex field $F_\Sigma $ and $\Pi $ is defined over L, then we get a local system also denoted by $\mathbb {V}_\rho $ on each of the associated Shimura varieties and there are natural isomorphisms

in the category $\mathcal {M}_{F_\Sigma }^L$ . Note that in general, the field of definition of $\Pi $ contains the field of rationality, but it is not clear if these are equal. See [Reference Buzzard and Gee12] and [Reference Shin and Templier62] for a discussion of these issues, which are not so important for us, since we will need instead a version of the above isomorphism for eigenvectors of the unramified Hecke algebra at finite level.

Let $\mathcal {K}$ be an open compact subgroup of $G(\mathbb {A}_f)$ for G each of the end groups such that the images of the two $\mathcal {K}$ under the quotient map are identified by the isomorphism in the middle; we write $\mathcal {K}$ for this image, which is an open compact subgroup of $(G/Z)(\mathbb {A}_f) = G(\mathbb {A}_f)/Z(\mathbb {A}_f)$ . (There will be no confusion since we will always identify what the ambient group is.) Let S be a finite set of rational primes such that for all the groups

above and for all $p \notin S$ :

  • $G_p$ is unramified over $\mathbb {Q}_p$ ;

  • $\mathcal {K}_p$ is a hyperspecial maximal compact subgroup of $G_p$ ;

  • $\Pi _p$ has a nonzero $\mathcal {K}_p$ -fixed vector.

Let

be the Hecke algebra of compactly supported $\mathcal {K}^S$ -bi-invariant functions on $G(\mathbb {A}^S)$ , where $\mathbb {A}^S = {\prod ^{\prime }_{p \notin S}} \mathbb {Q}_p$ and $\mathcal {K}^S = \prod _{p \notin S} \mathcal {K}_p$ . Then

acts on $H^i(\mathrm {Sh}_{G,\mathcal {K}}, \mathbb {V}_\rho )$ . Put $\Pi ^S = \bigotimes ^{\prime }_{p \notin S} \Pi _p$ and

where $\chi $ is the character of

associated to $\Pi ^S$ . Let L be a number field such that $(\rho ,\mathbb {V}_\rho )$ is defined over L and such that L contains the values of $\chi $ . Then there are canonical isomorphisms

in the category $\mathcal {M}_{F_\Sigma }^L$ .

5 The global exceptional isomorphism

In this section, we construct the global exceptional isomorphism between a (projectivized) unitary group attached to a Hermitian (or skew-Hermitian) space $\mathbf {V}$ and the identity component of a (projectivized) quaternionic unitary group attached to a quaternionic skew-Hermitian space $\tilde {V}$ . Moreover, we study the restriction of this isomorphism to certain natural subgroups corresponding to the decomposition of $\mathbf {V}$ into the direct sum of two subspaces.

5.1 Hermitian spaces and unitary groups

Let $E/F$ be a quadratic extension of number fields and $\rho $ the nontrivial Galois automorphism of $E/F$ . Write $E = F + F\mathbf {i}$ for some trace zero element $\mathbf {i} \in E^\times $ . Let $\mathbf {V}$ be an E-Hermitian space. Thus, $\mathbf {V}$ is equipped with a nondegenerate form

$$\begin{align*}(\cdot,\cdot)_{\mathbf{V}}: \mathbf{V} \times \mathbf{V} \rightarrow E \end{align*}$$

satisfying

$$\begin{align*}(v\alpha ,w\beta )_{\mathbf{V}} = \alpha^\rho (v,w)_{\mathbf{V}} \beta, \quad (v,w)_{\mathbf{V}}=(w,v)_{\mathbf{V}}^\rho. \end{align*}$$

We denote by $\mathrm {GU}_E(\mathbf {V})$ the unitary similitude group of $\mathbf {V}$ :

$$\begin{align*}\mathrm{GU}_E(\mathbf{V}) = \{ g \in \mathrm{GL}_E(\mathbf{V}) \, | \, (gv,gw)_{\mathbf{V}} = \nu(g) \cdot (v,w)_{\mathbf{V}} \text{ for all } v, w \in \mathbf{V} \}, \end{align*}$$

where $\nu : \mathrm {GU}_E(\mathbf {V}) \rightarrow F^\times $ is the similitude character.

Proposition 5.1. Let $\mathbf {V}$ be a Hermitian space over E of dimension n, and let $g\in \mathrm {GU}_E(\mathbf {V})$ . Then

$$\begin{align*}\mathrm{N} (\det(g)) = \nu(g)^n, \end{align*}$$

where $\mathrm {N}$ denotes the norm map $E^\times \rightarrow F^\times $ .

Proof. This is obviously well known, but the proof will serve to establish some notation. Let $\mathbf {V}^*$ be the E-linear dual of $\mathbf {V}$ . First, the form $(\cdot ,\cdot )_{\mathbf {V}}$ induces an E-conjugate linear isomorphism

$$\begin{align*}\varphi: \mathbf{V} \simeq \mathbf{V}^*, \end{align*}$$

given by

$$\begin{align*}\varphi(x) (y) = ( x,y )_{\mathbf{V}}. \end{align*}$$

Then for a positive integer r,

(5.1) $$ \begin{align} {\wedge^r} \varphi: \wedge^r \mathbf{V} \rightarrow \wedge^r \mathbf{V}^* \end{align} $$

is also E-conjugate linear. Let $\iota $ be the E-linear isomorphism

(5.2) $$ \begin{align} \iota: \wedge^r (\mathbf{V}^*) \simeq (\wedge^r \mathbf{V})^*, \end{align} $$

induced by the multilinear map

(5.3) $$ \begin{align} (\mathbf{V}^*)^r \times \mathbf{V}^r \rightarrow E, \quad (\lambda_1, \ldots, \lambda_r, \mathbf{v}_1, \ldots, \mathbf{v}_r) \mapsto \det ( \lambda_i (\mathbf{v}_j)). \end{align} $$

Now, any $g\in \mathrm {GL}_E(\mathbf {V})$ acts on $\mathbf {V}^*$ via $g\lambda (\mathbf {v}) = \lambda (g^{-1}\mathbf {v})$ and $\iota $ is equivariant for this action since equation (5.3) is equivariant for the diagonal action of $\mathrm {GL}_E (\mathbf {V})$ . The composite

$$\begin{align*}\iota \circ \wedge^r \varphi: \wedge^r \mathbf{V} \rightarrow (\wedge^r \mathbf{V})^* \end{align*}$$

is an E-conjugate linear isomorphism and may be viewed as giving a Hermitian form on $\wedge ^r \mathbf {V}$ , denoted by $(\cdot , \cdot )_{\wedge ^r \mathbf {V}}$ . (That this form is conjugate symmetric follows for instance by computing it in matrix form in terms of the matrix of the form on $\mathbf {V}$ with respect to an orthogonal basis. If the matrix of the original form is the diagonal matrix with entries $a_1, \ldots , a_n$ , then the entries $a_i$ lie in F and the form on $\wedge ^r \mathbf {V}$ is represented by the diagonal matrix whose entries are products of the form $a_{i_1} \cdots a_{i_r}$ with $1\le i_1 < \cdots < i_r \le n$ .)

Now, suppose $g\in \mathrm {GU}_E (\mathbf {V})$ . Then

$$\begin{align*}\varphi (g\mathbf{v}) (\mathbf{w}) = (g\mathbf{v},\mathbf{w})_{\mathbf{V}} =\nu(g) \cdot (\mathbf{v}, g^{-1} \mathbf{w})_{\mathbf{V}} = \nu(g) \cdot (g \varphi)(\mathbf{v}) (\mathbf{w}) \end{align*}$$

so that

$$\begin{align*}\varphi \circ g = \nu(g) \cdot g \circ \varphi \end{align*}$$

and

$$\begin{align*}\wedge^r \varphi \circ g = \nu(g)^r \cdot g \circ \wedge^r \varphi. \end{align*}$$

Thus, for $\mathbf {x},\mathbf {y} \in \wedge ^r \mathbf {V}$ , we have

$$\begin{align*}(g \mathbf{x}, g \mathbf{y})_{\wedge^r \mathbf{V}} = \nu(g)^r (\mathbf{x},\mathbf{y})_{\wedge^r \mathbf{V}}. \end{align*}$$

Now, take $r=n$ . Then g acts on $\wedge ^n \mathbf {V}$ as the scalar $\det (g)$ so that $(g\mathbf {x}, g\mathbf {y})_{\wedge ^n \mathbf {V}} = \mathrm {N} (\det (g)) (\mathbf {x},y)_{\wedge ^n \mathbf {V}}$ , from which it follows that $\mathrm {N} (\det (g)) = \nu (g)^n$ .

5.2 Construction of the (global) exceptional isomorphism

Let $\mathbf {V}$ be a four-dimensional (right) E-Hermitian space. Such a $\mathbf {V}$ is classified by a collection of its determinant $\delta _v \in F_v^\times /\mathrm {N} E_v^\times $ for all places v, which equals its discriminant (see [Reference Ichino and Prasanna30, §2.1.1] for our convention) since $\dim \mathbf {V} =4$ , together with its signature at ramified Archimedean places. (For a split place, $\delta _v$ is always trivial. For a ramified Archimedean place, $\delta _v$ is trivial if the signature is either $(4,0)$ , $(2,2)$ or $(0,4)$ and is nontrivial if the signature is $(3,1)$ or $(1,3)$ .) Let B be the unique quaternion algebra over F which is ramified exactly at those places v of F at which $\delta _v$ is nontrivial. Let $*$ be the main involution on B. Then we will construct

  • a three-dimensional right B-space $\tilde {V}$ ,

  • a skew-Hermitian B-form $\langle \cdot , \cdot \rangle $ on $\tilde {V}$ , that is, a nondegenerate sesquilinear form $\langle \cdot , \cdot \rangle : \tilde {V} \times \tilde {V} \rightarrow B$ satisfying

    $$\begin{align*}\langle v\alpha ,w\beta \rangle = \alpha^* \langle v,w \rangle \beta, \quad \langle v,w \rangle = -\langle w,v \rangle^* \end{align*}$$

such that there is a natural isogeny

$$\begin{align*}\mathrm{GSU}_E (\mathbf{V}) \rightarrow \mathrm{GU}_B(\tilde{V})^0, \end{align*}$$

as well as a natural isomorphism

$$\begin{align*}\operatorname{PGU}_E( \mathbf{V}) \simeq \operatorname{PGU}_B (\tilde{V})^0. \end{align*}$$

Here, we denote by $\mathrm {GU}_B(\tilde {V})^0$ the identity component of the unitary similitude group of $\tilde {V}$ :

$$\begin{align*}\mathrm{GU}_B(\tilde{V}) = \{ g \in \mathrm{GL}_B(\tilde{V}) \, | \, \langle gv,gw \rangle = \nu(g) \cdot \langle v,w \rangle \text{ for all } v, w \in \tilde{V} \}, \end{align*}$$

where $\nu : \mathrm {GU}_B(\tilde {V}) \rightarrow F^\times $ is the similitude character, and put

$$ \begin{align*} \mathrm{GSU}_E (\mathbf{V}) & = \{ g\in \mathrm{GU}_E(\mathbf{V}) \, | \, \det (g)=\nu(g)^2 \}, \\ \operatorname{PGU}_E(\mathbf{V}) & = \mathrm{GU}_E(\mathbf{V}) / E^\times, \\ \operatorname{PGU}_B(\tilde{V})^0 & = \mathrm{GU}_B(\tilde{V})^0 / F^\times. \end{align*} $$

Let $\tilde {\mathbf {V}} = \wedge ^2 \mathbf {V}$ . This is a right E-space, and we will extend the E-action to a right B-action. To do so, we must construct an element $L\in \operatorname {End}_F (\tilde {\mathbf {V}})$ which is conjugate linear for the E-action:

$$\begin{align*}L(x \alpha) = (Lx) \alpha^\rho \end{align*}$$

for $x\in \tilde {\mathbf {V}}$ , $\alpha \in E$ .

The map L will be a composite of three maps:

  1. (i) The map

    $$\begin{align*}\wedge^2 \varphi: \wedge^2 \mathbf{V} \rightarrow \wedge^2 (\mathbf{V}^*) \end{align*}$$
    obtained by specializing equation (5.1) to $r=2$ , which is an E-conjugate linear isomorphism.
  2. (ii) The map

    $$\begin{align*}\iota: \wedge^2 (\mathbf{V}^*) \simeq (\wedge^2 \mathbf{V})^* \end{align*}$$
    obtained by specializing equation (5.2) to $r=2$ , which is an E-linear isomorphism.
  3. (iii) Here, we use that $\dim \mathbf {V} =4$ . Fix an isomorphism

    $$\begin{align*}d: \wedge^4 \mathbf{V} \simeq E. \end{align*}$$

    This is well defined up to scaling. The natural map

    (5.4) $$ \begin{align} \wedge^2 \mathbf{V} \times \wedge^2 \mathbf{V} \rightarrow \wedge^4 \mathbf{V} \simeq E \end{align} $$
    is symmetric and induces an E-linear isomorphism
    $$\begin{align*}\psi: \wedge^2 \mathbf{V} \simeq (\wedge^2 \mathbf{V})^*. \end{align*}$$

Let

$$\begin{align*}L= \psi^{-1} \circ \iota \circ \wedge^2 \varphi. \end{align*}$$

Clearly, L depends on the choice of d. If d is scaled by $\alpha $ , then $\psi $ is scaled by $\alpha $ as well and L is scaled by $\alpha ^{-1} $ . However, $L^2$ changes to

$$\begin{align*}(\alpha^{-1} L)(\alpha^{-1} L) = (\alpha^\rho \alpha)^{-1} L^2 = \mathrm{N} (\alpha)^{-1} L^2. \end{align*}$$

Thus, $L^2$ is well defined up to norms from $E^\times $ to $F^\times $ . In fact, $L^2$ turns out to be a scalar operator. To identify this scalar, we recall the following invariant attached to a Hermitian space $\mathbf {V}$ of dimension n and an isomorphism $d: \wedge ^n \mathbf {V} \simeq E$ .

Definition 5.2. Let $\mathbf {V}$ be a Hermitian space of dimension n with form H, and let $d: \wedge ^n \mathbf {V} \simeq E$ be an isomorphism. The form H induces a map

$$\begin{align*}\mathbf{V}^n \times \mathbf{V}^n \rightarrow E, \quad (\mathbf{v}_1, \ldots, \mathbf{v}_n, \mathbf{w}_1, \ldots, \mathbf{w}_n) \mapsto \det [ H(\mathbf{v}_i, \mathbf{w}_j ) ], \end{align*}$$

which factors through $\wedge ^n \mathbf {V} \times \wedge ^n \mathbf {V}$ and gives a Hermitian form

$$\begin{align*}h: \wedge^n \mathbf{V} \times \wedge^n \mathbf{V} \rightarrow E. \end{align*}$$

Let $v \in \wedge ^n \mathbf {V}$ be such that $d(v)=1$ . Then define

$$\begin{align*}\operatorname{vol} (H,d) = h (v,v). \end{align*}$$

Note that h is a Hermitian form, so $\operatorname {vol}(H,d)$ lies in $F^\times $ and its class in $F^\times /\mathrm {N} E^\times $ equals the class of the determinant of H.

Proposition 5.3. The map $L^2$ is multiplication by $\operatorname {vol} ((\cdot ,\cdot )_{\mathbf {V}},d)$ .

Proof. We will pick a suitable basis and compute. Since $L^2$ and $\operatorname {vol} ((\cdot ,\cdot )_{\mathbf {V}},d)$ scale in exactly the same way as a function of d, we can choose any convenient d as well. Let $\mathbf {v}_1, \ldots , \mathbf {v}_4$ be a basis of $\mathbf {V}$ with respect to which the form $(\cdot , \cdot )_{\mathbf {V}}$ is diagonal with entries $a_1, \ldots , a_4 \in F$ . Let $\mathbf {e}_1,\ldots , \mathbf {e}_4$ be the dual basis of $\mathbf {V}^*$ . Then

$$\begin{align*}\varphi (\mathbf{v}_i ) = a_i \mathbf{e}_i \end{align*}$$

and

$$\begin{align*}\wedge^2 (\varphi) (\mathbf{v}_i \wedge \mathbf{v}_j) = a_i a_j \mathbf{e}_i \wedge \mathbf{e}_j. \end{align*}$$

For $1\le i < j \le 4$ , let $\mathbf {v}_{ij}$ denote the element $\mathbf {v}_i \wedge \mathbf {v}_j \in \wedge ^2 \mathbf {V}$ . This collection gives a basis of $\wedge ^2 \mathbf {V}$ . We let $\{ \mathbf {e}_{ij} \} \subset (\wedge ^2 \mathbf {V})^*$ be the dual basis. Then

$$\begin{align*}\iota (\mathbf{e}_i \wedge \mathbf{e}_j) = \mathbf{e}_{ij}. \end{align*}$$

For any pair $(i,j)$ as above, let $(i',j')$ be the unique pair of elements such that $\{i,j,i',j'\} =\{ 1,2,3,4\}$ and such that $i' < j'$ . Define $\operatorname {sign}(i,j) =\pm 1$ by

$$\begin{align*}\mathbf{v}_{ij} \wedge \mathbf{v}_{i'j'}=\operatorname{sign}(i,j) \mathbf{v}_1 \wedge \mathbf{v}_2 \wedge \mathbf{v}_3 \wedge \mathbf{v}_4. \end{align*}$$

Now, choose d such that

(5.5) $$ \begin{align} d (\mathbf{v}_1 \wedge \mathbf{v}_2 \wedge \mathbf{v}_3 \wedge \mathbf{v}_4) = -1. \end{align} $$

(This choice may seem surprising, but it is made so as to agree with some conventions in [Reference Ichino and Prasanna30].) Then

$$\begin{align*}\psi^{-1} (\mathbf{e}_{ij}) =- \operatorname{sign} (i,j) \cdot \mathbf{v}_{i'j'}. \end{align*}$$

(The $-$ sign here occurs because of the choice made in equation (5.5).) Now, we can write down L explicitly in the basis $\mathbf {v}_{ij}$ . It is given by

$$ \begin{align*} \mathbf{v}_{12} & \mapsto -a_1 a_2 \mathbf{v}_{34}, \\ \mathbf{v}_{13} & \mapsto a_1 a_3 \mathbf{v}_{24}, \\ \mathbf{v}_{14} & \mapsto -a_1 a_4 \mathbf{v}_{23}, \\ \mathbf{v}_{23} & \mapsto -a_2 a_3 \mathbf{v}_{14}, \\ \mathbf{v}_{24} & \mapsto a_2 a_4 \mathbf{v}_{13}, \\ \mathbf{v}_{34} & \mapsto -a_3 a_4 \mathbf{v}_{12}. \end{align*} $$

The proposition follows from this explicit description.

Now, let us define a quaternion algebra B as follows. Let

$$\begin{align*}J:=\operatorname{vol}( (\cdot,\cdot)_{\mathbf{V}},d), \end{align*}$$

and define B by

$$\begin{align*}B := E + E\mathbf{j}, \quad \mathbf{j}^2 = J, \quad \alpha \mathbf{j} = \mathbf{j} \alpha^\rho \end{align*}$$

for all $\alpha \in E$ . Then we can define a right action of B on $\tilde {\mathbf {V}}$ by

$$\begin{align*}x \cdot \mathbf{j} = L(x). \end{align*}$$

We will denote this space by $\tilde {V}$ when we want to regard it as a B-space rather than an E-space.

As in the proof of Proposition 5.1, the composite map $\iota \circ \wedge ^2 \varphi $ is a conjugate linear isomorphism

$$\begin{align*}\wedge^2 \mathbf{V} \simeq (\wedge^2 \mathbf{V})^* \end{align*}$$

that gives rise to a Hermitian form $(\cdot , \cdot )_{\tilde {\mathbf {V}}}$ on $\tilde {\mathbf {V}} =\wedge ^2 \mathbf {V}$ . Multiplying this form by the trace zero element $\mathbf {i}$ gives a skew-Hermitian form on $\tilde {\mathbf {V}}$ , which we denote simply by $(\cdot , \cdot )$ .

Lemma 5.4. The form $(\cdot , \cdot )$ on $\tilde {\mathbf {V}}$ satisfies: For all $x,y \in \tilde {\mathbf {V}}$ ,

  1. (i) $(x\mathbf {j} , y) = (y\mathbf {j}, x)$ .

  2. (ii) $(x\mathbf {j},y\mathbf {j})^\rho = - J (x,y) $ .

Proof. Firstly,

$$ \begin{align*} (x\mathbf{j},y) &= \mathbf{i} \cdot [\iota \circ \wedge^2 (\varphi) (x\mathbf{j}) ](y) \\ &= \mathbf{i} \cdot [\psi \circ L \circ L(x)](y) \\ &= J \mathbf{i} \cdot [\psi (x) ](y). \end{align*} $$

Since equation (5.4) is symmetric, we have

$$\begin{align*}[\psi (x)] (y) = [\psi (y)](x), \end{align*}$$

from which it follows that $(x\mathbf {j},y) =(y\mathbf {j},x)$ .

Secondly, we have

$$ \begin{align*} (x\mathbf{j}, y\mathbf{j}) &= J \mathbf{i} \cdot [\psi (x) ](y\mathbf{j}) \\ &= J \mathbf{i} \cdot [\psi(y\mathbf{j})](x) \\ &= J \mathbf{i} \cdot[\psi \circ \psi^{-1} \circ \iota \circ \wedge^2 (\varphi) (y)](x) \\ &= J \cdot (y,x) \end{align*} $$

so that

$$\begin{align*}(x\mathbf{j},y\mathbf{j})^\rho = J\cdot (y,x)^\rho = -J (x,y).\\[-37pt] \end{align*}$$

Remark 5.5. In fact, (ii) above follows from (i). Indeed, assuming (i), we have

$$ \begin{align*} (x\mathbf{j}, y\mathbf{j}) &= (y\mathbf{j} \cdot \mathbf{j} , x) = J(y,x) = -J(x,y)^\rho. \end{align*} $$

Now, we can define a B-skew-Hermitian form on $\tilde {V}$ by

(5.6) $$ \begin{align} \langle x,y \rangle = (x,y) - \frac{1}{J} \cdot \mathbf{j} \cdot (x\mathbf{j},y). \end{align} $$

Proposition 5.6. The map

$$\begin{align*}\mathrm{GL}_E (\mathbf{V}) \rightarrow \mathrm{GL}_E(\tilde{\mathbf{V}}), \quad g \mapsto \wedge^2 g \end{align*}$$

induces an isogeny

$$\begin{align*}\tilde{\xi} : \mathrm{GSU}_E (\mathbf{V}) \longrightarrow \mathrm{GU}_B (\tilde{V})^0 \end{align*}$$

with kernel $\{ \pm 1 \}$ .

Proof. Let $g \in \mathrm {GU}_E (\mathbf {V})$ . We first compute the commutator of L and $\wedge ^2 g$ . Recall that

$$\begin{align*}L= \psi^{-1} \circ \iota \circ \wedge^2 \varphi. \end{align*}$$

Now, for any $g\in \mathrm {GL}_E (\mathbf {V})$ , we have

(5.7) $$ \begin{align} \iota \circ \wedge^2 g = \wedge^2 g \circ \iota \end{align} $$

and

(5.8) $$ \begin{align} \psi \circ \wedge^2 g = \det(g) \wedge^2 g \circ \psi. \end{align} $$

If further $g \in \mathrm {GU}_E(\mathbf {V})$ , then

$$\begin{align*}(gx,gy)_{\mathbf{V}} =\nu(g) (x,y)_{\mathbf{V}} \end{align*}$$

which implies that $(gx,y)_{\mathbf {V}} = \nu (g) (x, g^{-1} y)_{\mathbf {V}}$ and

$$\begin{align*}\varphi \circ g = \nu(g) g \circ \varphi. \end{align*}$$

Thus,

(5.9) $$ \begin{align} \wedge^2 \varphi \circ \wedge^2 g = \nu(g)^2 \wedge^2 g \circ \wedge^2 \varphi. \end{align} $$

It follows from equations (5.7), (5.8) and (5.9) that

(5.10) $$ \begin{align} L\circ \wedge^2 g=\nu(g)^2 \det(g)^{-1} \wedge^2 g \circ L \end{align} $$

for $g\in \mathrm {GU}_E(\mathbf {V})$ . Thus, for $g\in \mathrm {GSU}_E(\mathbf {V})$ , the endomorphism $\wedge ^2 g$ lies in $\mathrm {GU}_B (\tilde {V})$ . This gives a map

$$\begin{align*}\mathrm{GSU}_E(\mathbf{V}) \rightarrow \mathrm{GU}_B( \tilde{V}) \end{align*}$$

whose kernel is easily checked to be $\{ \pm 1\}$ . On the other hand, we have

$$ \begin{align*} \dim \mathrm{GSU}_E(\mathbf{V}) & = \dim \mathrm{GU}_E(\mathbf{V}) - 1 = \dim \mathrm{U}_E(\mathbf{V}) = 4^2 = 16, \\ \dim \mathrm{GU}_B(\tilde{V}) & = \dim \mathrm{U}_B(\tilde{V}) + 1 = 3 (2 \cdot 3-1) + 1 = 16. \end{align*} $$

Since $\mathrm {GSU}_E(\mathbf {V})$ is connected, this shows that the image of the above map is $\mathrm {GU}_B(\tilde {V})^0$ .

Proposition 5.7. There is a natural isomorphism

$$\begin{align*}\xi: \operatorname{PGU}_E(\mathbf{V}) \overset{\simeq}{\longrightarrow} \operatorname{PGU}_B(\tilde{V})^0, \end{align*}$$

where

$$\begin{align*}\operatorname{PGU}_E(\mathbf{V}) = \mathrm{GU}_E(\mathbf{V}) / E^\times, \qquad \operatorname{PGU}_B(\tilde{V})^0 = \mathrm{GU}_B(\tilde{V})^0 / F^\times. \end{align*}$$

Proof. Let $g \in \mathrm {GU}_E(\mathbf {V})$ . Put $f = \wedge ^2 g$ and $\alpha = \nu (g)^2 / \det g$ . By equation (5.10), we have

$$\begin{align*}Lf = \alpha fL. \end{align*}$$

By Proposition 5.1, we have $\mathrm {N}(\det g) = \nu (g)^4$ , so $\mathrm {N} (\alpha )=1$ and we can choose $\beta \in E^\times $ (unique up to multiplication by $F^\times $ ) such that $\alpha = \beta /\beta ^\rho $ . Then

$$\begin{align*}L \beta f = \beta^\rho L f = \beta^\rho \alpha fL = \beta fL \end{align*}$$

so that $\beta f \in \mathrm {GU}_B(\tilde {V})$ . The assignment $g \mapsto \beta f$ gives a homomorphism

$$\begin{align*}\mathrm{GU}_E(\mathbf{V}) \longrightarrow \mathrm{GU}_B(\tilde{V})/ F^\times. \end{align*}$$

It is easy to see that its kernel is the center of $\mathrm {GU}_E(\mathbf {V})$ . Indeed, if $\wedge ^2 g$ is a scalar multiplication on $\tilde {\mathbf {V}}$ , then since $\dim \mathbf {V}> 2$ , g has to be semisimple and hence is a scalar multiplication on $\mathbf {V}$ . On the other hand, as in the proof of Proposition 5.6, we have $\dim \operatorname {PGU}_E(\mathbf {V}) = \dim \operatorname {PGU}_B(\tilde {V})$ . Since $\mathrm {GU}_E(\mathbf {V})$ is connected, this shows that the image of the above map is the identity component of $\mathrm {GU}_B(\tilde {V})/F^\times $ .

Remark 5.8. The reader may note that the notation is mildly confusing here. Namely, $\beta f$ is the map given by $\beta f (x) := f(x) \beta $ since the action of E is on the right.

Remark 5.9. For $g\in \mathrm {GSU}_E(\mathbf {V})$ , we have $\alpha =1$ , so we may take $\beta =1$ as well. This implies that the maps constructed in the previous two propositions fit into the commutative diagram below, where the vertical maps are the natural homomorphisms

5.3 Subgroups

In this section, we discuss the effect of the isogeny/isomorphism of the previous section on certain natural subgroups of the unitary group obtained from a decomposition of the Hermitian space into a sum of two Hermitian spaces.

5.3.1 The sum of two two-dimensional spaces

We first discuss the case when $\mathbf {V}$ is an orthogonal direct sum of the form $\mathbf {V}=\mathbf {V}_1\oplus \mathbf {V}_2$ with $\dim _E \mathbf {V}_1 = \dim _E \mathbf {V}_2 = 2$ . Then

(5.11) $$ \begin{align} \tilde{\mathbf{V}} = \wedge^2 (\mathbf{V}_1 \oplus \mathbf{V}_2) = \wedge^2 (\mathbf{V}_1) \oplus \wedge^2 (\mathbf{V}_2) \oplus (\mathbf{V}_1 \otimes \mathbf{V}_2), \end{align} $$

where we identify $\mathbf {V}_1 \otimes \mathbf {V}_2$ with its image under the natural map to $\wedge ^2 \mathbf {V}$ (which sends $v\otimes w$ to $v\wedge w$ ).

We may assume that the basis $(\mathbf {v}_1, \mathbf {v}_2, \mathbf {v}_3, \mathbf {v}_4)$ of $\mathbf {V}$ is chosen such that $(\mathbf {v}_1,\mathbf {v}_2)$ forms a basis of $\mathbf {V}_1$ and $(\mathbf {v}_3, \mathbf {v}_4)$ a basis of $\mathbf {V}_2$ . From the explicit formulas for L, it is clear that L preserves the subspaces

$$\begin{align*}\mathbf{V}_0^\sharp= \wedge^2 (\mathbf{V}_1) \oplus \wedge^2 (\mathbf{V}_2) \quad \text{ and} \quad \mathbf{V}^\sharp= \mathbf{V}_1 \otimes \mathbf{V}_2, \end{align*}$$

so these are B-spaces that we denote by $V_0^\sharp $ and $V^\sharp $ , respectively. Since the collection $(\mathbf {v}_{ij}$ , $1\le i < j \le 4)$ forms an orthogonal basis for the form $(\cdot ,\cdot )$ , the decomposition $\tilde {\mathbf {V}} = \mathbf {V}^\sharp \oplus \mathbf {V}_0^\sharp $ is one of skew-Hermitian E-spaces. Moreover, the formula (5.6) shows that the decomposition $\tilde {V} = V^\sharp \oplus V_0^\sharp $ is one of skew-Hermitian B-spaces.

Proposition 5.10. Let H be the subgroup of $\mathrm {GSU}_E (\mathbf {V})$ given by

$$\begin{align*}H= \mathrm{GSU}_E (\mathbf{V}) \cap \mathrm{G} (\mathrm{U}_E (\mathbf{V}_1) \times \mathrm{U}_E (\mathbf{V}_2)). \end{align*}$$

Then $\tilde {\xi }$ restricts to an isogeny

(5.12) $$ \begin{align} H \rightarrow \mathrm{G}(\mathrm{U}_B(V^\sharp) \times \mathrm{U}_B (V_0^\sharp))^0, \end{align} $$

with kernel $\{ \pm 1 \}$ .

Proof. Let $g_1 \in \mathrm {GU}_E(\mathbf {V}_1)$ , $g_2 \in \mathrm {GU}_E( \mathbf {V}_2)$ be such that $g=(g_1,g_2) \in H$ . Then $\wedge ^2 g$ acts as right multiplication by $\det (g_i)$ on $\wedge ^2 \mathbf {V}_i$ and by $g_1 \otimes g_2$ on $\mathbf {V}_1 \otimes \mathbf {V}_2$ . Since H is connected, this shows that $\tilde {\xi }$ maps H into the subgroup $\mathrm {G}(\mathrm {U}_B(V^\sharp ) \times \mathrm {U}_B (V_0^\sharp ))^0$ of $\mathrm {GU}_B (\tilde {V})^0$ . On the other hand, we have

$$ \begin{align*} \dim H & = \dim \mathrm{G} (\mathrm{U}_E (\mathbf{V}_1) \times \mathrm{U}_E (\mathbf{V}_2)) - 1 \\ & = \dim \mathrm{U}_E (\mathbf{V}_1) + \dim \mathrm{U}_E (\mathbf{V}_2) \\ & = 2^2 + 2^2 = 8, \\ \dim \mathrm{G}(\mathrm{U}_B(V^\sharp) \times \mathrm{U}_B (V_0^\sharp)) & = \dim \mathrm{U}_B(V^\sharp) + \dim \mathrm{U}_B (V_0^\sharp) + 1 \\ & = 2(2 \cdot 2-1) + 1(2 \cdot 1-1) + 1 = 8. \end{align*} $$

Hence, the image of H under $\tilde {\xi }$ is $\mathrm {G}(\mathrm {U}_B(V^\sharp ) \times \mathrm {U}_B (V_0^\sharp ))^0$ .

Likewise, one has an analogous result for subgroups in the context of Proposition 5.7.

Proposition 5.11. The map $\xi $ restricts to an isomorphism

(5.13) $$ \begin{align} \mathrm{G} (\mathrm{U}_E(\mathbf{V}_1) \times \mathrm{U}_E (\mathbf{V}_2)) / E^\times \simeq \mathrm{G} (\mathrm{U}_B( V^\sharp) \times \mathrm{U}_B (V_0^\sharp))^0/F^\times. \end{align} $$

Proof. Let $g=(g_1,g_2) \in \mathrm {G} (\mathrm {U}_E(\mathbf {V}_1) \times \mathrm {U}_E (\mathbf {V}_2))$ . As in the previous proposition, the map $\wedge ^2 g$ clearly preserves the decomposition $\tilde {\mathbf {V}} = \mathbf {V}^\sharp \oplus \mathbf {V}_0^\sharp $ , hence so does $\beta \cdot \wedge ^2 g$ . Since $\beta \cdot \wedge ^2 g$ lies in $\mathrm {GU}_B (\tilde {V})$ , it must in fact lie in $\mathrm {G} (\mathrm {U}_B( V^\sharp ) \times \mathrm {U}_B (V_0^\sharp ))$ . This gives the map (5.13), which must be injective since $\xi $ is injective. From dimension considerations, it must be an isomorphism.

It is useful to write down explicitly the maps in equations (5.12) and (5.13). First for $i=1,2$ , we define a quaternion algebra $B_i$ such that $\mathbf {V}_i$ is naturally a right $B_i$ -module and is equipped with a $B_i$ -Hermitian form whose projection to E recovers the Hermitian form. Let us outline for $i=1$ , the case $i=2$ being exactly similar. The Hermitian form on $\mathbf {V}_1$ gives an E-conjugate linear isomorphism

$$\begin{align*}\varphi_1: \mathbf{V}_1 \simeq \mathbf{V}_1^*, \end{align*}$$

where $\mathbf {V}_1^*$ as usual is the E-linear dual of $\mathbf {V}_1$ . Let us fix an isomorphism

$$\begin{align*}d_1: \wedge^2 \mathbf{V}_1 \simeq E. \end{align*}$$

For definiteness, we let $d_1 (\mathbf {v}_1 \wedge \mathbf {v}_2) = 1$ . This gives a bilinear pairing

$$\begin{align*}\mathbf{V}_1 \times \mathbf{V}_1 \rightarrow E, \quad (x,y) \mapsto d_1 (x\wedge y) \end{align*}$$

and thus gives an E-linear isomorphism

$$\begin{align*}\psi_1: \mathbf{V}_1 \simeq \mathbf{V}_1^*, \quad \psi_1 (x) (y) = d_1 (x\wedge y). \end{align*}$$

Define $L_1 \in \operatorname {End}_F(\mathbf {V}_1)$ by

$$\begin{align*}L_1 = \psi_1^{-1} \circ \varphi_1. \end{align*}$$

Explicitly, we see that $L_1$ acts on $\mathbf {V}_1$ by

(5.14) $$ \begin{align} \mathbf{v}_1 & \mapsto -a_1 \mathbf{v}_2, \end{align} $$
(5.15) $$ \begin{align} \mathbf{v}_2 & \mapsto a_2 \mathbf{v}_1 \end{align} $$

so that $L_1^2 = -a_1a_2 = -\operatorname {vol} (\mathbf {V}_1, d_1)$ . Let

$$\begin{align*}J_1 = -\operatorname{vol}(\mathbf{V}_1, d_1), \end{align*}$$

and define $B_1$ by

$$\begin{align*}B_1 = E+ E\mathbf{j}_1, \quad \mathbf{j}_1^2 = J_1, \quad \alpha \mathbf{j}_1 = \mathbf{j}_1 \alpha^\rho \end{align*}$$

for all $\alpha \in E$ . Then the right E-action on $\mathbf {V}_1$ extends to a right $B_1$ -action defined by

$$\begin{align*}x \cdot \mathbf{j}_1 = L_1 (x). \end{align*}$$

When we want to think of $\mathbf {V}_1$ as a $B_1$ -space, we simply denote it $V_1$ .

Lemma 5.12. For $x,y \in \mathbf {V}_1$ , we have

  1. (i) $(x\mathbf {j}_1, y) = - (y\mathbf {j}_1, x)$ .

  2. (ii) $(x\mathbf {j}_1, y\mathbf {j}_1)^\rho = -J_1 (x,y) $ .

Proof. We have for $x,y \in \mathbf {V}_1$ ,

$$\begin{align*}(x\mathbf{j}_1, y) = \varphi_1 (x\mathbf{j}_1) (y) = \psi_1 L_1 (x\mathbf{j}_1) (y) = \psi_1 L_1^2 (x) (y) = J_1 \psi_1 (x) (y) =J_1 d_1 (x\wedge y), \end{align*}$$

and likewise $(y\mathbf {j}_1, x) = J_1 d_1 (y \wedge x)$ . It follows from this that $(x\mathbf {j}_1, y) = - (y\mathbf {j}_1,x)$ which proves (i). Now, (ii) follows from (i) since

$$\begin{align*}(x\mathbf{j}_1, y\mathbf{j}_1) = -(y\mathbf{j}_1 \cdot \mathbf{j}_1, x) = -J_1(y,x) = -J_1 (x,y)^\rho.\\[-37pt] \end{align*}$$

Using the lemma, we find that

$$\begin{align*}\langle x,y \rangle := (x,y) - \frac{1}{J_1} \mathbf{j}_1 (x\mathbf{j}_1, y) \end{align*}$$

defines a $B_1$ -Hermitian form on $V_1$ such that $\operatorname {pr} \circ \langle \cdot ,\cdot \rangle = (\cdot , \cdot )$ . Thus, there is a natural embedding

$$\begin{align*}\mathrm{GU}_{B_1}(V_1) \hookrightarrow \mathrm{GU}_E(\mathbf{V}_1) \end{align*}$$

which we can make explicit as follows. Pick a $B_1$ -basis $\mathbf {x}_1$ for $V_1$ . Then $V_1 = \mathbf {x}_1 B_1$ and for $g\in \mathrm {GU}_{B_1}(V_1)$ , let $\beta _g$ be defined by

$$\begin{align*}g\mathbf{x}_1 = \mathbf{x}_1 \beta_g. \end{align*}$$

The assignment $g\mapsto \beta _g$ gives an identification $\mathrm {GU}_{B_1} (V_1) \simeq B_1^\times $ . Indeed, this map is injective; it is also surjective since for any $\beta \in B_1^\times $ and $\alpha ,\alpha ' \in B_1$ , we have

$$\begin{align*}\langle \mathbf{x}_1 \beta \alpha ,\mathbf{x}_1 \beta \alpha' \rangle = (\beta\alpha)^\rho \langle \mathbf{x}_1,\mathbf{x}_1 \rangle \beta \alpha' = \alpha^\rho \beta^\rho \langle \mathbf{x}_1,\mathbf{x}_1 \rangle \beta \alpha' = \nu (\beta) \alpha^\rho \langle \mathbf{x}_1,\mathbf{x}_1 \rangle \alpha' = \nu(\beta) \langle \mathbf{x}_1 \alpha, \mathbf{x}_1 \alpha' \rangle. \end{align*}$$

Here we have used that $\langle \mathbf {x}_1,\mathbf {x}_1 \rangle $ lies in F, the form $\langle \cdot , \cdot \rangle $ being $B_1$ -Hermitian. Then

$$\begin{align*}\mathrm{GU}_E (\mathbf{V}_1) \simeq (B_1^\times \times E^\times)/F^\times, \end{align*}$$

where the $B_1^\times $ corresponds to the $\mathrm {GU}_{B_1} (V_1)$ action described above, the $E^\times $ -action is given by $\alpha \mapsto $ (right)-multiplication by $\alpha ^{-1}$ for $\alpha \in E^\times $ and the embedding of $F^\times $ in $B_1^\times \times E^\times $ is just the diagonal embedding $t\mapsto (t,t)$ . All of the above discussion carries over verbatim to the case $i=2$ so that after picking a $B_2$ -basis $\mathbf {x}_2$ of $V_2$ , the map $\mathrm {GU}_{B_2} (V_2) \hookrightarrow \mathrm {GU}_E (\mathbf {V}_2)$ is identified with the embedding

$$\begin{align*}B_2^\times \hookrightarrow (B_2^\times \times E^\times)/F^\times. \end{align*}$$

Next, we explicate the groups on the right of the map (5.12). First, we note that

$$\begin{align*}\mathrm{GU}_B(V_0^\sharp)^0 \simeq E^\times. \end{align*}$$

Indeed, let $\mathbf {x}$ be any nonzero vector in $\wedge ^2 \mathbf {V}_1$ so that $V_0^\sharp =\mathbf {x} B$ . Then for $\alpha \in E^\times $ , the map

$$\begin{align*}\mathbf{x} \beta \mapsto \mathbf{x} \alpha \beta \end{align*}$$

gives an element of $\mathrm {GU}_B (V_0^\sharp )$ . By dimension considerations, this gives an isomorphism $E^\times \simeq \mathrm {GU}_B(V_0^\sharp )^0$ . More precisely, it is easy to see that for any $g\in \mathrm {GU}_B (V_0^\sharp )$ , we have $g \cdot \mathbf {x} = \mathbf {x} \gamma _g$ for a unique $\gamma _g \in B^\times $ and the assignment $g\mapsto \gamma _g$ gives an isomorphism of $\mathrm {GU}_B( V_0^\sharp )$ with the semidirect product $E^\times \rtimes \langle \mathbf {j} \rangle $ , where $\mathbf {j}$ acts on $E^\times $ by conjugation in $B^\times $ . Note that the induced isomorphism $\mathrm {GU}_B( V_0^\sharp )^0 \simeq E^\times $ is independent of the choice of $\mathbf {x} \in \wedge ^2 \mathbf {V}_1$ .

As for $V^\sharp $ , we have the following proposition.

Proposition 5.13. There is an isomorphism

$$\begin{align*}\mathrm{GU}_B (V^\sharp)^0 \simeq (B_1^\times \times B_2^\times)/F^\times, \end{align*}$$

depending on the choice of a basis vector for $V_1$ (as $B_1$ -space) and for $V_2$ (as $B_2$ -space).

Proof. First, note that the restriction of the Hermitian form $(\cdot , \cdot )_{\tilde {\mathbf {V}}}$ to $\mathbf {V}_1 \otimes \mathbf {V}_2$ is just the tensor product of the Hermitian forms on $\mathbf {V}_1$ and $\mathbf {V}_2$ . Also, from equations (5.14) and (5.15), right multiplication by $\mathbf {j}_1$ and $\mathbf {j}_2$ on $\mathbf {V}_1$ and $\mathbf {V}_2$ is given explicitly by

$$\begin{align*}\mathbf{v}_1 \mathbf{j}_1 = -a_1 \mathbf{v}_2, \quad \mathbf{v}_2 \mathbf{j}_1 = a_2 \mathbf{v}_1, \quad \mathbf{v}_3 \mathbf{j}_2 = -a_3 \mathbf{v}_4, \quad \mathbf{v}_4 \mathbf{j}_2 = a_4 \mathbf{v}_3. \end{align*}$$

From this and the explicit formula for the action of $L=\mathbf {j}$ on $\tilde {\mathbf {V}}$ , we see that (right)-multiplication by $\mathbf {j}$ on $\mathbf {V}_1 \otimes \mathbf {V}_2$ is the same as (right)-multiplication by $\mathbf {j}_1 \otimes \mathbf {j}_2$ .

Choose a $B_1$ -basis $\mathbf {x}_1$ for $V_1$ and a $B_2$ -basis $\mathbf {x}_2$ for $V_2$ . Then there is an action of $B_1^\times \times B_2^\times $ on $\mathbf {V}_1 \otimes _E \mathbf {V}_2$ which on pure tensors is given by

$$\begin{align*}(\beta_1, \beta_2) \cdot (\mathbf{x}_1 \alpha_1 \otimes \mathbf{x}_2 \alpha_2) = \mathbf{x}_1 \beta_1 \alpha_1 \otimes \mathbf{x}_2 \beta_2 \alpha_2, \end{align*}$$

for any $\alpha _1 \in B_1$ , $\alpha _2 \in B_2$ . This action is clearly E-linear and also $\mathbf {j}$ -linear since $\mathbf {j}$ acts as right multiplication by $\mathbf {j}_1 \otimes \mathbf {j}_2$ , hence is in fact B-linear.

Now,

$$ \begin{align*} (\mathbf{x}_1 \beta_1 \alpha_1 \otimes \mathbf{x}_2 \beta_2 \alpha_2, \mathbf{x}_1 \beta_1 \alpha^{\prime}_1 \otimes \mathbf{x}_2 \beta_2 \alpha^{\prime}_2) &= \mathbf{i} \cdot (\mathbf{x}_1 \beta_1 \alpha_1 , \mathbf{x}_1 \beta_1 \alpha^{\prime}_1)_{\mathbf{V}_1} (\mathbf{x}_2 \beta_2 \alpha_2,\mathbf{x}_2 \beta_2 \alpha^{\prime}_2)_{\mathbf{V}_2} \\ &= \mathbf{i} \cdot \nu(\beta_1) \nu (\beta_2) \cdot (\mathbf{x}_1 \alpha_1 , \mathbf{x}_1 \alpha^{\prime}_1)_{\mathbf{V}_1} (\mathbf{x}_2 \alpha_2,\mathbf{x}_2 \alpha^{\prime}_2)_{\mathbf{V}_2} \\ &= \nu(\beta_1) \nu (\beta_2) (\mathbf{x}_1 \alpha_1 \otimes \mathbf{x}_2 \alpha_2, \mathbf{x}_1 \alpha^{\prime}_1 \otimes \mathbf{x}_2 \alpha^{\prime}_2), \end{align*} $$

which shows that $(\beta _1, \beta _2)$ gives an element in $\mathrm {GU}_E (\mathbf {V}^\sharp )$ . Since the action of $(\beta _1, \beta _2)$ commutes with $\mathbf {j}$ , we see from the formula (5.6) that it in fact defines an element of $\mathrm {GU}_B (V^\sharp )$ . This gives a map

$$\begin{align*}B_1^\times \times B_2^\times \rightarrow \mathrm{GU}_B(V^\sharp) \end{align*}$$

whose kernel is the diagonal $F^\times $ in $B_1^\times \times B_2^\times $ , embedded as $t \mapsto (t,t^{-1})$ . By dimension considerations, we see that this gives an isomorphism $(B_1^\times \times B_2^\times )/F^\times \simeq \mathrm {GU}_B( V^\sharp )^0$ .

Now, we can write down the map (5.12) explicitly. Let $h=(h_1,h_2) \in H$ with $h_1 \in \mathrm {GU}_E (\mathbf {V}_1)$ and $h_2 \in \mathrm {GU}_E (\mathbf {V}_2)$ . Then

$$\begin{align*}\nu(h_1) = \nu(h_2) \end{align*}$$

and

$$\begin{align*}\det(h_1) \det(h_2) = \nu(h_1)^2 = \nu (h_2)^2. \end{align*}$$

Moreover, $\mathrm {N} (\det h_i) = \nu (h_i)^2$ by Proposition 5.1 so that $\det (h_2) = \det (h_1)^\rho $ . Now, $\wedge ^2 h$ acts as (right)-multiplication by $\det (h_1)$ on $\wedge ^2 \mathbf {V}_1$ and by $\det (h_2)$ on $\wedge ^2 \mathbf {V}_2 = \wedge ^2 \mathbf {V}_1 \cdot \mathbf {j}$ , and thus acts on $V_0^\sharp $ as the element $\det (h_1) \in E^\times = \mathrm {GU}_B(V_0^\sharp )^0$ .

Fix a basis vector $\mathbf {x}_1$ for $V_1$ and $\mathbf {x}_2$ for $V_2$ as above. This gives identifications

$$\begin{align*}\mathrm{GU}_E (\mathbf{V}_1) = (B_1^\times \times E^\times)/F^\times, \quad \mathrm{GU}_E (\mathbf{V}_2) = (B_2^\times \times E^\times)/F^\times. \end{align*}$$

Let $g_1=[b_1,\alpha _1] \in \mathrm {GU}_E (\mathbf {V}_1)$ and $g_2=[b_2,\alpha _2] \in \mathrm {GU}_E (\mathbf {V}_2)$ . Then

$$\begin{align*}\nu(g_1) = \nu(b_1) \mathrm{N}(\alpha_1)^{-1}, \quad \det(g_1)= \nu(b_1) \cdot \alpha_1^{-2}, \end{align*}$$

and

$$\begin{align*}\nu(g_2) = \nu(b_2) \mathrm{N}(\alpha_2)^{-1}, \quad \det(g_2)= \nu(b_2) \cdot \alpha_2^{-2}. \end{align*}$$

Let $g=(g_1, g_2) \in \mathrm {G} (\mathrm {U}_E (\mathbf {V}_1) \times \mathrm {U}_E (\mathbf {V}_2))$ viewed as an element in $\mathrm {GU}_E(\mathbf {V})$ . Then

$$\begin{align*}\nu(g) = \nu(g_1) =\nu(g_2) = \nu(b_1) \mathrm{N}(\alpha_1)^{-1} = \nu(b_2) \mathrm{N}(\alpha_2)^{-1} \end{align*}$$

while

$$\begin{align*}\det (g) = \nu(b_1) \cdot \alpha_1^{-2} \cdot \nu(b_2) \cdot \alpha_2^{-2}. \end{align*}$$

Thus,

$$ \begin{align*} g \in \mathrm{GSU}_E(\mathbf{V}) & \iff \det(g) = \nu(g)^2 \\ & \iff \nu(b_1) \cdot \alpha_1^{-2} \cdot \nu(b_2) \cdot \alpha_2^{-2} = \nu(b_1) \mathrm{N}(\alpha_1)^{-1} \cdot \nu(b_2) \mathrm{N}(\alpha_2)^{-1} \\ & \iff \alpha_1 \alpha_2 = \alpha_1^\rho \alpha_2^\rho \\ & \iff \alpha_1 \alpha_2 \in F^\times. \end{align*} $$

We conclude that

$$\begin{align*}H = \left\{ \left( [b_1,\alpha_1], [b_2,\alpha_2] \right) \, | \, \nu(b_1) \mathrm{N}(\alpha_1)^{-1} = \nu(b_2) \mathrm{N} (\alpha_2)^{-1} , \ \alpha_1 \alpha_2 \in F^\times \right\}. \end{align*}$$

The action of $h= \left( [b_1,\alpha _1], [b_2,\alpha _2] \right)$ on $\mathbf {V}_1 \otimes _E \mathbf {V}_2$ is then given by

$$\begin{align*}\mathbf{x}_1 \beta_1 \otimes \mathbf{x}_2 \beta_2 \mapsto \mathbf{x}_1 b_1 \beta_1 \alpha_1^{-1} \otimes \mathbf{x}_2 b_2 \beta_2 \alpha_2^{-1} = \mathbf{x}_1 b_1 \beta_1 \alpha_1^{-1} \alpha_2^{-1} \otimes \mathbf{x}_2 b_2 \beta_2 =\mathbf{x}_1 b_1 (\alpha_1 \alpha_2)^{-1} \beta_1 \otimes \mathbf{x}_2 b_2 \beta_2 \end{align*}$$

so that the map (5.12) is given explicitly by

$$\begin{align*}h \mapsto \left([b_1 (\alpha_1 \alpha_2)^{-1}, b_2], \nu(b_1) \alpha_1^{-2} \right) \in \mathrm{G} ( (B_1^\times \times B_2^\times)/F^\times) \times E^\times). \end{align*}$$

Likewise, we can make equation (5.13) explicit. Let $g =(g_1,g_2) \in \mathrm {G} (\mathrm {U}_E(\mathbf {V}_1) \times \mathrm {U}_E (\mathbf {V}_2)) $ , with $g_1 =[b_1,\alpha _1]$ and $g_2=[b_2,\alpha _2]$ . Then

$$\begin{align*}\frac{\nu(g)^2}{\det (g)} = \frac{\nu (b_1) \mathrm{N} (\alpha_1)^{-1} \nu(b_2) \mathrm{N} (\alpha_2)^{-1} }{\nu(b_1) \nu(b_2) \alpha_1^{-2} \alpha_2^{-2}} = \frac{\alpha_1 \alpha_2}{(\alpha_1\alpha_2)^\rho}, \end{align*}$$

so we may take $\beta =\alpha _1 \alpha _2$ in the definition of $\xi (g)$ . Then the map (5.13) is given by

$$\begin{align*}g \mapsto \left( [b_1, b_2], \det(g_1) \alpha_1 \alpha_2 \right). \end{align*}$$

Example 5.14. This is the case that is of most interest to us. Instead of starting with the spaces $\mathbf {V}$ or $\mathbf {V}_1$ or $\mathbf {V}_2$ , we start with two quaternion algebras $B_1$ and $B_2$ over F containing E. Suppose that

$$\begin{align*}B_1 = E + E \mathbf{j}_1, \quad B_2 = E+ E \mathbf{j}_2, \end{align*}$$

with $\mathbf {j}_1^2 = J_1$ and $\mathbf {j}_2^2 = J_2$ . Let $\mathbf {V}_i = B_i$ considered as a (right)-E-Hermitian space with the same form as in [Reference Ichino and Prasanna30, §2.2], that is,

$$\begin{align*}(a+\mathbf{j}_i b,c+\mathbf{j}_i d)_i = a^\rho c - J_i b^\rho d. \end{align*}$$

We specialize the setup above to the case

$$\begin{align*}\mathbf{V} = \mathbf{V}_1 \oplus \mathbf{V}_2, \end{align*}$$

with Hermitian form $(\cdot , \cdot )_{\mathbf {V}}$ given by the direct sum of $(\cdot , \cdot )_1$ and $(\cdot ,\cdot )_2$ . In the basis

$$\begin{align*}\mathbf{v}_1=(1,0), \ \mathbf{v}_2=(\mathbf{j}_1,0), \ \mathbf{v}_3=(0,1), \ \mathbf{v}_4=(0,\mathbf{j}_2), \end{align*}$$

this form is diagonal with matrix

$$\begin{align*}\begin{pmatrix} a_1 & & & \\ & a_2 & & \\ & & a_3 & \\ & & & a_4 \end{pmatrix} = \begin{pmatrix} 1 & & & \\ & -J_1 & & \\ & & 1 & \\ & & & -J_2 \end{pmatrix}. \end{align*}$$

Let us pick $d:\wedge ^4 \mathbf {V} \simeq E$ such that

$$\begin{align*}d (\mathbf{v}_1 \wedge \mathbf{v}_2 \wedge \mathbf{v}_3 \wedge \mathbf{v}_4) = -1 \end{align*}$$

as in the proof of Proposition 5.3. Then $J=\operatorname {vol}( (\cdot ,\cdot )_{\mathbf {V}},d)$ is equal to $J_1 J_2$ .

With respect to the bases $(\mathbf {v}_1 \wedge \mathbf {v}_2, \mathbf {v}_3 \wedge \mathbf {v}_4 )$ of $\mathbf {V}_0^\sharp $ and $(\mathbf {v}_1 \otimes \mathbf {v}_3, \mathbf {v}_2 \otimes \mathbf {v}_3 , \mathbf {v}_1 \otimes \mathbf {v}_4, \mathbf {v}_2 \otimes \mathbf {v}_4)$ of $\mathbf {V}^\sharp $ , the matrices of these Hermitian forms are given by

$$\begin{align*}\begin{pmatrix} -J_1 & \\ & - J_2 \end{pmatrix} \quad \text{and} \quad \begin{pmatrix} 1 & & & \\ & -J_1 & & \\ & & -J_2 & \\ & & & J \end{pmatrix}, \end{align*}$$

respectively. Thus, $\mathbf {V}^\sharp $ is the tensor product of $\mathbf {V}_1$ and $\mathbf {V}_2$ as Hermitian spaces. The action of $\mathbf {j}$ on $\mathbf {V}^\sharp $ can be read off from the formulas in the proof of Proposition 5.3 and is given by

$$ \begin{align*} (\mathbf{v}_1 \otimes \mathbf{v}_3)\cdot \mathbf{j} &= \mathbf{v}_2 \otimes \mathbf{v}_4, \\ (\mathbf{v}_2 \otimes \mathbf{v}_3) \cdot \mathbf{j} &= J_1 \cdot \mathbf{v}_1 \otimes \mathbf{v}_4, \\ (\mathbf{v}_1 \otimes \mathbf{v}_4) \cdot \mathbf{j} & = J_2 \cdot \mathbf{v}_2 \otimes \mathbf{v}_3, \\ (\mathbf{v}_2 \otimes \mathbf{v}_4) \cdot \mathbf{j} &= J \cdot \mathbf{v}_1 \otimes \mathbf{v}_3. \end{align*} $$

This shows that $V^\sharp $ with its B-action and B-skew-Hermitian form is exactly the same as the space V occurring in [Reference Ichino and Prasanna30, §2.2].

5.3.2 The sum of a three-dimensional and a one-dimensional space

In this section (which is not used in this paper), we suppose that $\mathbf {V}$ is an orthogonal direct sum of the form $\mathbf {V}= \mathbf {V}_3 \oplus \mathbf {V}_4$ with $\dim _E \mathbf {V}_3 = 3$ and $\dim _E \mathbf {V}_4 =1$ . Then there is an inclusion

$$\begin{align*}\mathrm{G}(\mathrm{U}_E(\mathbf{V}_3) \times \mathrm{U}_E(\mathbf{V}_4)) \hookrightarrow \mathrm{GU}_E (\mathbf{V}), \end{align*}$$

and so we can ask for a description of how this relates to the maps $\tilde {\xi }$ , $\xi $ .

Note that

$$\begin{align*}\wedge^2 \mathbf{V} = \wedge^2 \mathbf{V}_3 \oplus (\mathbf{V}_3 \otimes \mathbf{V}_4) \end{align*}$$

as a sum of (skew)-Hermitian spaces. We may assume that the basis vectors $\mathbf {v}_i$ are chosen such that $(\mathbf {v}_1, \mathbf {v}_2, \mathbf {v}_3)$ forms a basis for $\mathbf {V}_3$ while $\mathbf {v}_4$ is a basis for $\mathbf {V}_4$ . The explicit formula for L shows that L interchanges $\wedge ^2 \mathbf {V}_3$ and $\mathbf {V}_3 \otimes \mathbf {V}_4$ . Thus, letting $\mathbf {W}:=\mathbf {V}_3 \otimes \mathbf {V}_4$ , we have

(5.16) $$ \begin{align} \tilde{V} = \mathbf{W} \otimes _E B, \end{align} $$

at least as B-spaces. The formula (5.6) shows that the restriction of the B-skew-Hermitian form $\langle \cdot , \cdot \rangle $ to $\mathbf {W}$ is the same as the restriction of the E-skew-Hermitian form $(\cdot , \cdot )$ (from $\tilde {\mathbf {V}}$ ) to $\mathbf {W}$ , from which it follows that the form $\langle \cdot , \cdot \rangle $ on $\tilde {V}$ is just the B-linear extension of $(\cdot ,\cdot )$ via the isomorphism (5.16). Thus, there is a canonical inclusion

$$\begin{align*}\mathrm{GU}_E(\mathbf{W}) \rightarrow \mathrm{GU}_B (\tilde{V}), \end{align*}$$

which must land inside $\mathrm {GU}_B(\tilde {V})^0$ since $\mathrm {GU}_E(\mathbf {W})$ is connected.

Proposition 5.15. The map $\xi $ restricts to an isomorphism

$$\begin{align*}\mathrm{G} (\mathrm{U}_E (\mathbf{V}_3) \times \mathrm{U}_E (\mathbf{V}_4))/E^\times \simeq \mathrm{GU}_E(\mathbf{W})/F^\times. \end{align*}$$

Proof. It is clear that $\xi $ restricts to an injective map between the given source and target. That it is an isomorphism follows from dimension considerations and since the target is connected.

6 The local exceptional isomorphism

In this section, we study the local exceptional isomorphism between the (projectivized) unitary group attached to the four-dimensional Hermitian space $\mathbf {V}$ and the identity component of the (projectivized) quaternionic unitary group attached to the three-dimensional quaternionic skew-Hermitian space $\tilde {V}$ . For later use, we need to consider the localizations at almost all (finite) places and at real places.

6.1 Setup

Let F be a local field of characteristic zero and E an étale quadratic algebra over F. We denote by $\rho $ the nontrivial automorphism of E over F and by $\mathrm {N} = \mathrm {N}_{E/F}$ the norm map from E to F. Fix a trace zero element $\mathbf {i} \in E^\times $ , and put $u = \mathbf {i}^2 \in F^\times $ .

We recall the construction in §5. Let $\mathbf {V}$ be a four-dimensional E-space equipped with a Hermitian form $(\cdot ,\cdot )_{\mathbf {V}}$ . (We considered a right E-space $\mathbf {V}$ earlier but regard it as a left E-space by setting $\alpha v = v \alpha $ for $\alpha \in E$ and $v \in \mathbf {V}$ .) Fix an E-linear isomorphism $d:\wedge ^4 \mathbf {V} \rightarrow E$ . Then we have a six-dimensional E-space $\tilde {\mathbf {V}} = \wedge ^2 \mathbf {V}$ equipped with a skew-Hermitian form

$$\begin{align*}(x_1 \wedge x_2, y_1 \wedge y_2) = \mathbf{i} \cdot \det \begin{pmatrix} (x_1, y_1)_{\mathbf{V}} & (x_1, y_2)_{\mathbf{V}} \\ (x_2, y_1)_{\mathbf{V}} & (x_2, y_2)_{\mathbf{V}} \end{pmatrix} \end{align*}$$

and a conjugate E-linear automorphism

$$\begin{align*}L = \psi^{-1} \circ \iota \circ \wedge^2 \varphi, \end{align*}$$

where

  • $\varphi : \mathbf {V} \rightarrow \mathbf {V}^*$ is the conjugate E-linear isomorphism induced by $(\cdot ,\cdot )_{\mathbf {V}}$ ;

  • $\iota : \wedge ^2(\mathbf {V}^*) \rightarrow (\wedge ^2 \mathbf {V})^*$ is the natural E-linear isomorphism;

  • $\psi : \wedge ^2 \mathbf {V} \rightarrow (\wedge ^2 \mathbf {V})^*$ is the E-linear isomorphism relative to d.

Note that $L^2$ is the scalar multiplication by some $J \in F^\times $ . This gives rise to a quaternion F-algebra $B = E + E \mathbf {j}$ with a trace zero element $\mathbf {j} \in B^\times $ such that $\mathbf {j}^2 = J$ and a three-dimensional right B-space $\tilde {V} = \tilde {\mathbf {V}}$ equipped with a skew-Hermitian form

$$\begin{align*}\langle x, y \rangle = (x, y) - \mathbf{j}^{-1} \cdot (L(x), y). \end{align*}$$

Moreover, we have a natural isomorphism

$$\begin{align*}\xi : \operatorname{PGU}_E(\mathbf{V}) \overset{\simeq}\longrightarrow \operatorname{PGU}_B(\tilde{V})^0 \end{align*}$$

by Proposition 5.7. Since B and $\tilde {V}$ do not depend on the choice of d, we can make a convenient choice in the following computation.

6.2 The split case

In this section, we assume that $\mathbf {V}$ is split. Fix a basis $\mathbf {v}_1, \dots , \mathbf {v}_4$ of $\mathbf {V}$ such that

$$\begin{align*}(\mathbf{v}_i, \mathbf{v}_j)_{\mathbf{V}} = \begin{cases} 1 & \text{if } (i,j) = (1,3), (2,4), (3,1),(4,2), \\ 0 & \text{otherwise.} \end{cases} \end{align*}$$

Let $\mathbf {e}_1, \dots , \mathbf {e}_4$ be its dual basis of $\mathbf {V}^*$ . We take an isomorphism $d: \wedge ^4 \mathbf {V} \rightarrow E$ such that

$$\begin{align*}d(\mathbf{v}_1 \wedge \mathbf{v}_2 \wedge \mathbf{v}_3 \wedge \mathbf{v}_4) = 1. \end{align*}$$

Let T be the maximal torus of $\mathrm {GU}_E(\mathbf {V})$ consisting of elements t such that

$$\begin{align*}t \mathbf{v}_1 = t_1 \mathbf{v}_1, \qquad t \mathbf{v}_2 = t_2 \mathbf{v}_2, \qquad t \mathbf{v}_3 = \nu (t_1^{\rho})^{-1} \mathbf{v}_3, \qquad t \mathbf{v}_4 = \nu (t_2^{\rho})^{-1} \mathbf{v}_4 \end{align*}$$

for some $t_i \in E^\times $ and $\nu \in F^\times $ . We identify T with $(E^\times )^2 \times F^\times $ via the map $t \mapsto (t_1, t_2, \nu )$ . Then the center of $\mathrm {GU}_E(\mathbf {V})$ is equal to $E^\times $ embedded into T by $z \mapsto (z, z, \mathrm {N}(z))$ .

We take a basis $\{ \mathbf {v}_{ij} \, | \, 1 \le i < j \le 4 \}$ of $\tilde {\mathbf {V}}$ given by $\mathbf {v}_{ij} = \mathbf {v}_i \wedge \mathbf {v}_j$ . Let $\{ \mathbf {e}_{ij} \, | \, 1 \le i < j \le 4 \}$ be its dual basis of $\tilde {\mathbf {V}}^*$ . Since

$$\begin{align*}\varphi(\mathbf{v}_1) = \mathbf{e}_3, \qquad \varphi(\mathbf{v}_2) = \mathbf{e}_4, \qquad \varphi(\mathbf{v}_3) = \mathbf{e}_1, \qquad \varphi(\mathbf{v}_4) = \mathbf{e}_2, \end{align*}$$

we have

$$ \begin{align*} (\iota \circ \wedge^2 \varphi)(\mathbf{v}_{12}) & = \mathbf{e}_{34}, & (\iota \circ \wedge^2 \varphi)(\mathbf{v}_{34}) & = \mathbf{e}_{12}, \\ (\iota \circ \wedge^2 \varphi)(\mathbf{v}_{13}) & = - \mathbf{e}_{13}, & (\iota \circ \wedge^2 \varphi)(\mathbf{v}_{24}) & = -\mathbf{e}_{24}, \\ (\iota \circ \wedge^2 \varphi)(\mathbf{v}_{14}) & = - \mathbf{e}_{23}, & (\iota \circ \wedge^2 \varphi)(\mathbf{v}_{23}) & = -\mathbf{e}_{14}. \end{align*} $$

Also, we have

$$ \begin{align*} \psi(\mathbf{v}_{12}) & = \mathbf{e}_{34}, & \psi(\mathbf{v}_{34}) & = \mathbf{e}_{12}, \\ \psi(\mathbf{v}_{13}) & = -\mathbf{e}_{24}, & \psi(\mathbf{v}_{24}) & = -\mathbf{e}_{13}, \\ \psi(\mathbf{v}_{14}) & = \mathbf{e}_{23}, & \psi(\mathbf{v}_{23}) & = \mathbf{e}_{14}. \end{align*} $$

Hence, we have

$$ \begin{align*} L(\mathbf{v}_{12}) & = \mathbf{v}_{12}, & L(\mathbf{v}_{34}) & = \mathbf{v}_{34}, \\ L(\mathbf{v}_{13}) & = \mathbf{v}_{24}, & L(\mathbf{v}_{24}) & = \mathbf{v}_{13}, \\ L(\mathbf{v}_{14}) & = - \mathbf{v}_{14}, & L(\mathbf{v}_{23}) & = - \mathbf{v}_{23}. \end{align*} $$

In particular, $J = 1$ . Moreover, $(\mathbf {v}_{ij}, \mathbf {v}_{i'j'})$ and $\langle \mathbf {v}_{ij}, \mathbf {v}_{i'j'} \rangle $ are given by the following tables:

$$\begin{align*}\begin{array}{c@{\quad}|@{\quad}c@{\quad}c@{\quad}c@{\quad}c@{\quad}c@{\quad}c} (\cdot, \cdot) & \mathbf {v}_{12} & \mathbf {v}_{34} & \mathbf {v}_{13} & \mathbf {v}_{24} & \mathbf {v}_{14} & \mathbf {v}_{23} \\ \hline \mathbf {v}_{12} & 0 & \mathbf {i} & 0 & 0 & 0 & 0 \\ \mathbf {v}_{34} & \mathbf {i} & 0 & 0 & 0 & 0 & 0 \\ \mathbf {v}_{13} & 0 & 0 & -\mathbf {i} & 0 & 0 & 0 \\ \mathbf {v}_{24} & 0 & 0 & 0 & -\mathbf {i} & 0 & 0 \\ \mathbf {v}_{14} & 0 & 0 & 0 & 0 & 0 & -\mathbf {i} \\ \mathbf {v}_{23} & 0 & 0 & 0 & 0 & -\mathbf {i} & 0 \\\end{array}\\ \begin{array}{c@{\quad}|@{\quad}c@{\quad}c@{\quad}c@{\quad}c@{\quad}c@{\quad}c} \langle \cdot, \cdot \rangle & \mathbf {v}_{12} & \mathbf {v}_{34} & \mathbf {v}_{13} & \mathbf {v}_{24} & \mathbf {v}_{14} & \mathbf {v}_{23} \\ \hline \mathbf {v}_{12} & 0 & \mathbf {i} + \mathbf {i}\mathbf {j} & 0 & 0 & 0 & 0 \\ \mathbf {v}_{34} & \mathbf {i} + \mathbf {i} \mathbf {j} & 0 & 0 & 0 & 0 & 0 \\ \mathbf {v}_{13} & 0 & 0 & -\mathbf {i} & -\mathbf {i}\mathbf {j} & 0 & 0 \\ \mathbf {v}_{24} & 0 & 0 & -\mathbf {i}\mathbf {j} & -\mathbf {i} & 0 & 0 \\ \mathbf {v}_{14} & 0 & 0 & 0 & 0 & 0 & -\mathbf {i}+\mathbf {i}\mathbf {j} \\ \mathbf {v}_{23} & 0 & 0 & 0 & 0 & -\mathbf {i}+\mathbf {i}\mathbf {j} & 0 \\\end{array}\end{align*}$$

We take a basis $\tilde {v}_1, \tilde {v}_2, \tilde {v}_3$ of $\tilde {V}$ over B given by

$$\begin{align*}\tilde{v}_1 = \mathbf{v}_{12} + \mathbf{v}_{14}, \qquad \tilde{v}_2 = \mathbf{v}_{34} + \mathbf{v}_{23}, \qquad \tilde{v}_3 = \mathbf{v}_{13} \end{align*}$$

so that

$$\begin{align*}(\langle \tilde{v}_i, \tilde{v}_j \rangle) = \begin{pmatrix} 0 & 2 \mathbf{i}\mathbf{j} & 0 \\ 2 \mathbf{i}\mathbf{j} & 0 & 0 \\ 0 & 0 & -\mathbf{i} \end{pmatrix}. \end{align*}$$

Let $\mathfrak {i}:B \rightarrow \mathrm {M}_2(F)$ be an isomorphism defined by

$$\begin{align*}\mathfrak{i}(a + b \mathbf{i} + c \mathbf{j} + d \mathbf{i}\mathbf{j}) = \begin{pmatrix} a+c & b-d \\ (b+d)u & a-c \end{pmatrix}. \end{align*}$$

Put

$$\begin{align*}e = \frac{1}{2}(1+\mathbf{j}), \qquad e' = \frac{1}{2}(\mathbf{i} - \mathbf{i}\mathbf{j}), \qquad e" = \frac{1}{2u}(\mathbf{i} + \mathbf{i}\mathbf{j}), \qquad e^* = \frac{1}{2}(1-\mathbf{j}) \end{align*}$$

so that

$$\begin{align*}\mathfrak{i}(e) = \begin{pmatrix} 1 & 0 \\ 0 & 0 \end{pmatrix}, \qquad \mathfrak{i}(e') = \begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix}, \qquad \mathfrak{i}(e") = \begin{pmatrix} 0 & 0 \\ 1 & 0 \end{pmatrix}, \qquad \mathfrak{i}(e^*) = \begin{pmatrix} 0 & 0 \\ 0 & 1 \end{pmatrix}. \end{align*}$$

Put $\tilde {V}^\dagger = \tilde {V} e$ . Then, by Morita theory (see [Reference Ichino and Prasanna30, §C.2] for details), $\tilde {V}^\dagger $ is a six-dimensional F-space equipped with a symmetric bilinear form $\langle \cdot , \cdot \rangle ^\dagger $ determined by

$$\begin{align*}\frac{1}{2} \langle x, y \rangle = \langle x, y \rangle^\dagger \cdot e" \end{align*}$$

for $x, y \in \tilde {V}^\dagger $ such that the restriction to $\tilde {V}^\dagger $ induces an isomorphism $\mathrm {GU}_B(\tilde {V})^0 \simeq \mathrm {GSO}(\tilde {V}^\dagger )$ (see [Reference Ichino and Prasanna30, Lemma C.2.1]). We take a basis $v_1, \dots , v_6$ of $\tilde {V}^\dagger $ over F given by

$$ \begin{align*} v_1 & = \tilde{v}_1 e = \mathbf{v}_{12}, & v_2 & = \tilde{v}_1 e" = \frac{\mathbf{i}}{u} \cdot \mathbf{v}_{14}, \\ v_3 & = \tilde{v}_2 e = \mathbf{v}_{34}, & v_4 & = \tilde{v}_2 e" = \frac{\mathbf{i}}{u} \cdot \mathbf{v}_{23}, \\ v_5 & = \tilde{v}_3 e = \frac{1}{2} (\mathbf{v}_{13} + \mathbf{v}_{24}), & v_6 & = \tilde{v}_3 e" = \frac{\mathbf{i}}{2u} (\mathbf{v}_{13} - \mathbf{v}_{24}) \end{align*} $$

so that

$$\begin{align*}(\langle v_i, v_j \rangle^\dagger) = \begin{pmatrix} 0 & 0 & u & 0 & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 & 0 \\ u & 0 & 0 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & -\frac{u}{2} & 0 \\ 0 & 0 & 0 & 0 & 0 & \frac{1}{2} \end{pmatrix}. \end{align*}$$

Let $\tilde {T}$ be the maximal torus of $\mathrm {GSO}(\tilde {V}^\dagger )$ consisting of elements $\tilde {t}$ such that

$$ \begin{align*} \tilde{t} v_1 & = \tilde{t}_1 v_1, & \tilde{t} v_2 & = \tilde{t}_2 v_2, \\ \tilde{t} v_3 & = \tilde{\nu} \tilde{t}_1^{-1} v_3, & \tilde{t} v_4 & = \tilde{\nu} \tilde{t}_2^{-1} v_4, \\ \tilde{t} v_5 & = a v_5 + bu v_6, & \tilde{t} v_6 & = b v_5 + a v_6 \end{align*} $$

for some $\tilde {t}_i \in F^\times $ and $a, b \in F$ such that $\tilde {\nu } = a^2 - b^2 u \ne 0$ . We identify $\tilde {T}$ with $(F^\times )^2 \times E^\times $ via the map $\tilde {t} \mapsto (\tilde {t}_1, \tilde {t}_2, a + b\mathbf {i})$ . Then the center of $\mathrm {GSO}(\tilde {V}^\dagger )$ is equal to $F^\times $ embedded into $\tilde {T}$ by $z \mapsto (z, z, z)$ .

Lemma 6.1. The isomorphism $\xi $ restricts to an isomorphism

$$\begin{align*}T/E^\times \overset{\simeq}\longrightarrow \tilde{T}/F^\times, \end{align*}$$

given by

$$\begin{align*}(t_1, t_2, \nu) \longmapsto (\mathrm{N}(t_1 t_2), \nu \mathrm{N}(t_1), \nu t_1 t_2^\rho). \end{align*}$$

Proof. Let $t = (t_1, t_2, \nu ) \in T \simeq (E^\times )^2 \times F^\times $ . Since

$$\begin{align*}\nu(t) = \nu, \qquad \det t = \frac{\nu^2 t_1 t_2}{t_1^\rho t_2^\rho}, \qquad \frac{\nu(t)^2}{\det t} = \frac{t_1^\rho t_2^\rho}{t_1 t_2}, \end{align*}$$

the image of t under the homomorphism $\mathrm {GU}_E(\mathbf {V}) \rightarrow \operatorname {PGU}_B(\tilde {V})$ in the proof of Proposition 5.7 is equal to the image of

$$\begin{align*}\tilde{t} = t_1^\rho t_2^\rho \cdot \wedge^2 t \end{align*}$$

in $\operatorname {PGU}_B(\tilde {V})$ . Put

$$\begin{align*}\tilde{t}_1 = \mathrm{N}(t_1 t_2), \qquad \tilde{t}_2 = \nu \mathrm{N}(t_1), \qquad a + b\mathbf{i} = \nu t_1 t_2^\rho, \qquad \tilde{\nu} = a^2 - b^2 u = \nu^2 \mathrm{N}(t_1 t_2). \end{align*}$$

Then

$$ \begin{align*} \tilde{t} v_1 & = \tilde{t} \mathbf{v}_{12} = \mathrm{N}(t_1 t_2) \mathbf{v}_{12} = \tilde{t}_1 v_1, \\ \tilde{t} v_2 & = \frac{\mathbf{i}}{u} \cdot \tilde{t} \mathbf{v}_{14} = \frac{\mathbf{i}}{u} \cdot \nu \mathrm{N}(t_1) \mathbf{v}_{14} = \tilde{t}_2 v_2, \\ \tilde{t} v_3 & = \tilde{t} \mathbf{v}_{34} = \nu^2 \mathbf{v}_{34} = \tilde{\nu} \tilde{t}_1^{-1} v_3, \\ \tilde{t} v_4 & = \frac{\mathbf{i}}{u} \cdot \tilde{t} \mathbf{v}_{23} = \frac{\mathbf{i}}{u} \cdot \nu \mathrm{N}(t_2) \mathbf{v}_{23} = \tilde{\nu} \tilde{t}_2^{-1} v_4, \\ \tilde{t} v_5 & = \frac{1}{2} (\tilde{t} \mathbf{v}_{13} + \tilde{t} \mathbf{v}_{24}) = \frac{1}{2} (\nu t_1 t_2^\rho \mathbf{v}_{13} + \nu t_1^\rho t_2 \mathbf{v}_{24}) \\ & = \frac{a}{2} (\mathbf{v}_{13} + \mathbf{v}_{24}) + \frac{b\mathbf{i}}{2} (\mathbf{v}_{13} - \mathbf{v}_{24}) = a v_5 + bu v_6, \\ \tilde{t} v_6 & = \frac{\mathbf{i}}{2u} (\tilde{t} \mathbf{v}_{13} - \tilde{t} \mathbf{v}_{24}) = \frac{\mathbf{i}}{2u} (\nu t_1 t_2^\rho \mathbf{v}_{13} - \nu t_1^\rho t_2 \mathbf{v}_{24}) \\ & = \frac{b}{2} (\mathbf{v}_{13} + \mathbf{v}_{24}) + \frac{a\mathbf{i}}{2u} (\mathbf{v}_{13} - \mathbf{v}_{24}) = b v_5 + a v_6. \end{align*} $$

Hence, the assertion follows.

6.3 The real case

In this section, we assume that $F = \mathbb {R}$ and $E = \mathbb {C}$ . Write

$$\begin{align*}\mathbf{i} = u_0 \cdot i \end{align*}$$

with $u_0 \in \mathbb {R}^\times $ so that $u = -u_0^2$ . We further assume that the signature of $\mathbf {V}$ is either $(2,2)$ or $(4,0)$ . Fix a basis $\mathbf {v}_1, \dots , \mathbf {v}_4$ of $\mathbf {V}$ such that

$$\begin{align*}(\mathbf{v}_i, \mathbf{v}_j)_{\mathbf{V}} = \begin{cases} 1 & \text{if } (i,j) = (1,1), (2,2), \\ \zeta & \text{if } (i,j) = (3,3), (4,4), \\ 0 & \text{otherwise,} \end{cases} \end{align*}$$

where $\zeta = \pm 1$ . Let $\mathbf {e}_1, \dots , \mathbf {e}_4$ be its dual basis of $\mathbf {V}^*$ . We take an isomorphism $d: \wedge ^4 \mathbf {V} \rightarrow E$ such that

$$\begin{align*}d(\mathbf{v}_1 \wedge \mathbf{v}_2 \wedge \mathbf{v}_3 \wedge \mathbf{v}_4) = 1. \end{align*}$$

Let T be the maximal torus of $\mathrm {GU}_E(\mathbf {V})$ consisting of elements t such that

$$\begin{align*}t \mathbf{v}_1 = r z_1 \mathbf{v}_1, \qquad t \mathbf{v}_2 = r z_2 \mathbf{v}_2, \qquad t \mathbf{v}_3 = r z_3 \mathbf{v}_3, \qquad t \mathbf{v}_4 = r z_4 \mathbf{v}_4 \end{align*}$$

for some $r \in \mathbb {R}_+^\times $ and $z_i \in \mathbb {C}^1$ . We identify T with $(\mathbb {C}^1)^4 \times \mathbb {R}_+^\times $ via the map $t \mapsto (z_1, z_2, z_3, z_4, r)$ . Then the center of $\mathrm {GU}_E(\mathbf {V})$ is equal to

$$\begin{align*}\{ (z,z,z,z,r) \, | \, z \in \mathbb{C}^1, \, r \in \mathbb{R}^\times_+ \} \simeq \mathbb{C}^\times. \end{align*}$$

We take a basis $\{ \mathbf {v}_{ij} \, | \, 1 \le i < j \le 4 \}$ of $\tilde {\mathbf {V}}$ given by $\mathbf {v}_{ij} = \mathbf {v}_i \wedge \mathbf {v}_j$ . Let $\{ \mathbf {e}_{ij} \, | \, 1 \le i < j \le 4 \}$ be its dual basis of $\tilde {\mathbf {V}}^*$ . Since

$$\begin{align*}\varphi(\mathbf{v}_1) = \mathbf{e}_1, \qquad \varphi(\mathbf{v}_2) = \mathbf{e}_2, \qquad \varphi(\mathbf{v}_3) = \zeta \mathbf{e}_3, \qquad \varphi(\mathbf{v}_4) = \zeta \mathbf{e}_4, \end{align*}$$

we have

$$ \begin{align*} (\iota \circ \wedge^2 \varphi)(\mathbf{v}_{12}) & = \mathbf{e}_{12}, & (\iota \circ \wedge^2 \varphi)(\mathbf{v}_{34}) & = \mathbf{e}_{34}, \\ (\iota \circ \wedge^2 \varphi)(\mathbf{v}_{13}) & = \zeta \mathbf{e}_{13}, & (\iota \circ \wedge^2 \varphi)(\mathbf{v}_{24}) & = \zeta \mathbf{e}_{24}, \\ (\iota \circ \wedge^2 \varphi)(\mathbf{v}_{14}) & = \zeta \mathbf{e}_{14}, & (\iota \circ \wedge^2 \varphi)(\mathbf{v}_{23}) & = \zeta \mathbf{e}_{23}. \end{align*} $$

Also, we have

$$ \begin{align*} \psi(\mathbf{v}_{12}) & = \mathbf{e}_{34}, & \psi(\mathbf{v}_{34}) & = \mathbf{e}_{12}, \\ \psi(\mathbf{v}_{13}) & = -\mathbf{e}_{24}, & \psi(\mathbf{v}_{24}) & = -\mathbf{e}_{13}, \\ \psi(\mathbf{v}_{14}) & = \mathbf{e}_{23}, & \psi(\mathbf{v}_{23}) & = \mathbf{e}_{14}. \end{align*} $$

Hence, we have

$$ \begin{align*} L(\mathbf{v}_{12}) & = \mathbf{v}_{34}, & L(\mathbf{v}_{34}) & = \mathbf{v}_{12}, \\ L(\mathbf{v}_{13}) & = -\zeta \mathbf{v}_{24}, & L(\mathbf{v}_{24}) & = -\zeta \mathbf{v}_{13}, \\ L(\mathbf{v}_{14}) & = \zeta \mathbf{v}_{23}, & L(\mathbf{v}_{23}) & = \zeta \mathbf{v}_{14}.\end{align*} $$

In particular, $J = 1$ . Moreover, $(\mathbf {v}_{ij}, \mathbf {v}_{i'j'})$ and $\langle \mathbf {v}_{ij}, \mathbf {v}_{i'j'} \rangle $ are given by the following tables:

$$\begin{align*}\begin{array}{c@{\quad}|@{\quad}c@{\quad}c@{\quad}c@{\quad}c@{\quad}c@{\quad}c} (\cdot, \cdot) & \mathbf {v}_{12} & \mathbf {v}_{34} & \mathbf {v}_{13} & \mathbf {v}_{24} & \mathbf {v}_{14} & \mathbf {v}_{23} \\ \hline \mathbf {v}_{12} & \mathbf {i} & 0 & 0 & 0 & 0 & 0 \\ \mathbf {v}_{34} & 0 & \mathbf {i} & 0 & 0 & 0 & 0 \\ \mathbf {v}_{13} & 0 & 0 & \zeta \mathbf {i} & 0 & 0 & 0 \\ \mathbf {v}_{24} & 0 & 0 & 0 & \zeta \mathbf {i} & 0 & 0 \\ \mathbf {v}_{14} & 0 & 0 & 0 & 0 & \zeta\mathbf {i} & 0 \\ \mathbf {v}_{23} & 0 & 0 & 0 & 0 & 0 & \zeta\mathbf {i} \\\end{array} \\\begin{array}{c@{\quad}|@{\quad}c@{\quad}c@{\quad}c@{\quad}c@{\quad}c@{\quad}c} \langle \cdot, \cdot \rangle & \mathbf {v}_{12} & \mathbf {v}_{34} & \mathbf {v}_{13} & \mathbf {v}_{24} & \mathbf {v}_{14} & \mathbf {v}_{23} \\ \hline \mathbf {v}_{12} & \mathbf {i} & \mathbf {i}\mathbf {j} & 0 & 0 & 0 & 0 \\ \mathbf {v}_{34} & \mathbf {i} \mathbf {j} & \mathbf {i} & 0 & 0 & 0 & 0 \\ \mathbf {v}_{13} & 0 & 0 & \zeta \mathbf {i} & -\mathbf {i}\mathbf {j} & 0 & 0 \\ \mathbf {v}_{24} & 0 & 0 & -\mathbf {i}\mathbf {j} & \zeta \mathbf {i} & 0 & 0 \\ \mathbf {v}_{14} & 0 & 0 & 0 & 0 & \zeta\mathbf {i} & \mathbf {i}\mathbf {j} \\ \mathbf {v}_{23} & 0 & 0 & 0 & 0 & \mathbf {i}\mathbf {j} & \zeta\mathbf {i} \\\end{array}\end{align*}$$

We take a basis $\tilde {v}_1, \tilde {v}_2, \tilde {v}_3$ of $\tilde {V}$ over B given by

$$\begin{align*}\tilde{v}_1 = \mathbf{v}_{13}, \qquad \tilde{v}_2 = \mathbf{v}_{14}, \qquad \tilde{v}_3 = \mathbf{v}_{12} \end{align*}$$

so that

$$\begin{align*}(\langle \tilde{v}_i, \tilde{v}_j \rangle) = \begin{pmatrix} \zeta \mathbf{i} & 0 & 0 \\ 0 & \zeta \mathbf{i} & 0 \\ 0 & 0 & \mathbf{i} \end{pmatrix}. \end{align*}$$

Let $\mathfrak {i}:B \rightarrow \mathrm {M}_2(F)$ be an isomorphism defined by

$$\begin{align*}\mathfrak{i}(a + b \mathbf{i} + c \mathbf{j} + d \mathbf{i}\mathbf{j}) = \begin{pmatrix} a+c & b-d \\ (b+d)u & a-c \end{pmatrix}. \end{align*}$$

Put

$$\begin{align*}e = \frac{1}{2}(1+\mathbf{j}), \qquad e' = \frac{1}{2}(\mathbf{i} - \mathbf{i}\mathbf{j}), \qquad e" = \frac{1}{2u}(\mathbf{i} + \mathbf{i}\mathbf{j}), \qquad e^* = \frac{1}{2}(1-\mathbf{j}) \end{align*}$$

so that

$$\begin{align*}\mathfrak{i}(e) = \begin{pmatrix} 1 & 0 \\ 0 & 0 \end{pmatrix}, \qquad \mathfrak{i}(e') = \begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix}, \qquad \mathfrak{i}(e") = \begin{pmatrix} 0 & 0 \\ 1 & 0 \end{pmatrix}, \qquad \mathfrak{i}(e^*) = \begin{pmatrix} 0 & 0 \\ 0 & 1 \end{pmatrix}. \end{align*}$$

Put $\tilde {V}^\dagger = \tilde {V} e$ . Then, by Morita theory (see [Reference Ichino and Prasanna30, §C.2] for details), $\tilde {V}^\dagger $ is a six-dimensional F-space equipped with a symmetric bilinear form $\langle \cdot , \cdot \rangle ^\dagger $ determined by

$$\begin{align*}\frac{1}{2} \langle x, y \rangle = \langle x, y \rangle^\dagger \cdot e" \end{align*}$$

for $x, y \in \tilde {V}^\dagger $ such that the restriction to $\tilde {V}^\dagger $ induces an isomorphism, $\mathrm {GU}_B(\tilde {V})^0 \simeq \mathrm {GSO}(\tilde {V}^\dagger )$ (see [Reference Ichino and Prasanna30, Lemma C.2.1]). We take a basis $v_1, \dots , v_6$ of $\tilde {V}^\dagger $ over F given by

$$ \begin{align*} v_1 & = \frac{\sqrt{2}}{u_0} \cdot \tilde{v}_1 e = \frac{1}{\sqrt{2} u_0} (\mathbf{v}_{13} - \zeta \mathbf{v}_{24}), & v_2 & = \sqrt{2} \cdot \tilde{v}_1 e" = -\frac{i}{\sqrt{2} u_0} (\mathbf{v}_{13} + \zeta \mathbf{v}_{24}), \\v_3 & = \frac{\sqrt{2}}{u_0} \cdot \tilde{v}_2 e = \frac{1}{\sqrt{2} u_0} (\mathbf{v}_{14} + \zeta \mathbf{v}_{23}), & v_4 & = \sqrt{2} \cdot \tilde{v}_2 e" = -\frac{i}{\sqrt{2} u_0} (\mathbf{v}_{14} - \zeta \mathbf{v}_{23}), \\v_5 & = \frac{\sqrt{2}}{u_0} \cdot \tilde{v}_3 e = \frac{1}{\sqrt{2} u_0} (\mathbf{v}_{12} + \mathbf{v}_{34}), & v_6 & = \sqrt{2} \cdot \tilde{v}_3 e" = -\frac{i}{\sqrt{2} u_0} (\mathbf{v}_{12} - \mathbf{v}_{34}) \end{align*} $$

so that

$$\begin{align*}(\langle v_i, v_j \rangle^\dagger) = \begin{pmatrix} -\zeta & 0 & 0 & 0 & 0 & 0 \\ 0 & -\zeta & 0 & 0 & 0 & 0 \\ 0 & 0 & -\zeta & 0 & 0 & 0 \\ 0 & 0 & 0 & -\zeta & 0 & 0 \\ 0 & 0 & 0 & 0 & -1 & 0 \\ 0 & 0 & 0 & 0 & 0 & -1 \end{pmatrix}. \end{align*}$$

Let $\tilde {T}$ be the maximal torus of $\mathrm {GSO}(\tilde {V}^\dagger )$ consisting of elements $\tilde {t}$ such that

$$ \begin{align*} \tilde{t} v_1 & = \tilde{r}(a_1 v_1 - b_1 v_2), & \tilde{t} v_2 & = \tilde{r}(b_1 v_1 + a_1 v_2), \\ \tilde{t} v_3 & = \tilde{r}(a_2 v_3 - b_2 v_4), & \tilde{t} v_4 & = \tilde{r}(b_2 v_3 + a_2 v_4), \\ \tilde{t} v_5 & = \tilde{r}(a_3 v_5 - b_3 v_6), & \tilde{t} v_6 & = \tilde{r}(b_3 v_5 + a_3 v_6) \end{align*} $$

for some $\tilde {r} \in \mathbb {R}^\times _+$ and $a_i, b_i \in \mathbb {R}$ such that $\tilde {z}_i = a_i + b_i i \in \mathbb {C}^1$ . We identify $\tilde {T}$ with $(\mathbb {C}^1)^3 \times \mathbb {R}^\times _+$ via the map $\tilde {t} \mapsto (\tilde {z}_1, \tilde {z}_2, \tilde {z}_3, \tilde {r})$ . Then the center of $\mathrm {GSO}(\tilde {V}^\dagger )$ is equal to

$$\begin{align*}\{ (\tilde{z},\tilde{z},\tilde{z},\tilde{r}) \, | \, \tilde{z} = \pm 1, \, \tilde{r} \in \mathbb{R}^\times_+ \} \simeq \mathbb{R}^\times. \end{align*}$$

Lemma 6.2. The isomorphism $\xi $ restricts to an isomorphism

$$\begin{align*}T/\mathbb{C}^\times \overset{\simeq}\longrightarrow \tilde{T}/\mathbb{R}^\times, \end{align*}$$

given by

$$\begin{align*}(z_1, z_2, z_3, z_4, r) \longmapsto \left( \frac{y_1y_3}{y_2y_4}, \frac{y_1y_4}{y_2y_3}, \frac{y_1y_2}{y_3y_4}, r^2 \right), \end{align*}$$

where we choose $y_i \in \mathbb {C}^1$ such that $z_i = y_i^2$ .

Proof. Let $t = (z_1, z_2, z_3, z_4, r) \in T \simeq (\mathbb {C}^1)^4 \times \mathbb {R}_+^\times $ and choose $y_i \in \mathbb {C}^1$ such that $z_i = y_i^2$ . Since

$$\begin{align*}\nu(t) = r^2, \qquad \det t = r^4 z_1 z_2 z_3 z_4, \qquad \frac{\nu(t)^2}{\det t} = \frac{1}{z_1 z_2 z_3 z_4}, \end{align*}$$

the image of t under the homomorphism $\mathrm {GU}_E(\mathbf {V}) \rightarrow \operatorname {PGU}_B(\tilde {V})$ in the proof of Proposition 5.7 is equal to the image of

$$\begin{align*}\tilde{t} = \frac{1}{y_1 y_2 y_3 y_4} \cdot \wedge^2 t \end{align*}$$

in $\operatorname {PGU}_B(\tilde {V})$ . Put $v_i' = \sqrt {2} u_0 \cdot v_i$ , and write

$$\begin{align*}\frac{y_1y_3}{y_2y_4} = a_1 + b_1 i, \qquad \frac{y_1y_4}{y_2y_3} = a_2 + b_2 i, \qquad \frac{y_1y_2}{y_3y_4} = a_3 + b_3 i \end{align*}$$

with $a_i, b_i \in \mathbb {R}$ . Then

$$ \begin{align*} \frac{1}{r^2} \cdot \tilde{t} v_1' & = \frac{1}{r^2}(\tilde{t} \mathbf{v}_{13} - \zeta \tilde{t} \mathbf{v}_{24}) = \frac{y_1y_3}{y_2y_4} \cdot \mathbf{v}_{13} - \frac{y_2y_4}{y_1y_3} \cdot \zeta \mathbf{v}_{24} \\ & = a_1 (\mathbf{v}_{13} - \zeta \mathbf{v}_{24}) + b_1 i (\mathbf{v}_{13} + \zeta \mathbf{v}_{24}) = a_1 v^{\prime}_1 - b_1 v^{\prime}_2, \\ \frac{1}{r^2} \cdot \tilde{t} v_2' & = - \frac{i}{r^2}(\tilde{t} \mathbf{v}_{13} + \zeta \tilde{t} \mathbf{v}_{24}) = - \frac{y_1y_3}{y_2y_4} \cdot i \mathbf{v}_{13} - \frac{y_2y_4}{y_1y_3} \cdot \zeta i \mathbf{v}_{24} \\ & = b_1(\mathbf{v}_{13} - \zeta \mathbf{v}_{24}) - a_1 i (\mathbf{v}_{13} + \zeta \mathbf{v}_{24}) = b_1 v^{\prime}_1 + a_1 v^{\prime}_2, \\ \frac{1}{r^2} \cdot \tilde{t} v_3' & = \frac{1}{r^2}(\tilde{t} \mathbf{v}_{14} + \zeta \tilde{t} \mathbf{v}_{23}) = \frac{y_1y_4}{y_2y_3} \cdot \mathbf{v}_{14} + \frac{y_2y_3}{y_1y_4} \cdot \zeta \mathbf{v}_{23} \\ & = a_2(\mathbf{v}_{14}+\zeta \mathbf{v}_{23}) + b_2 i (\mathbf{v}_{14} - \zeta \mathbf{v}_{23}) = a_2 v^{\prime}_3 - b_2 v^{\prime}_4, \\ \frac{1}{r^2} \cdot \tilde{t} v_4' & = -\frac{i}{r^2}(\tilde{t} \mathbf{v}_{14} - \zeta \tilde{t} \mathbf{v}_{23}) = -\frac{y_1y_4}{y_2y_3} \cdot i \mathbf{v}_{14} + \frac{y_2y_3}{y_1y_4} \cdot \zeta i \mathbf{v}_{23} \\ & = b_2 (\mathbf{v}_{14} + \zeta \mathbf{v}_{23}) - a_2 i(\mathbf{v}_{14}- \zeta \mathbf{v}_{23}) = b_2 v^{\prime}_3 + a_2 v^{\prime}_4, \\ \frac{1}{r^2} \cdot \tilde{t} v_5' & = \frac{1}{r^2}(\tilde{t} \mathbf{v}_{12} + \tilde{t} \mathbf{v}_{34}) = \frac{y_1y_2}{y_3y_4} \cdot \mathbf{v}_{12} + \frac{y_3y_4}{y_1y_2} \cdot \mathbf{v}_{34} \\ & = a_3 (\mathbf{v}_{12} + \mathbf{v}_{34}) + b_3 i (\mathbf{v}_{12} - \mathbf{v}_{34}) = a_3 v^{\prime}_5 - b_3 v^{\prime}_6, \\ \frac{1}{r^2} \cdot \tilde{t} v_6' & = -\frac{i}{r^2}(\tilde{t} \mathbf{v}_{12} - \tilde{t} \mathbf{v}_{34}) = - \frac{y_1y_2}{y_3y_4} \cdot i \mathbf{v}_{12} + \frac{y_3y_4}{y_1y_2} \cdot i \mathbf{v}_{34} \\ & = b_3 (\mathbf{v}_{12} + \mathbf{v}_{34}) - a_3 i (\mathbf{v}_{12} - \mathbf{v}_{34}) = b_3 v^{\prime}_5 + a_3 v^{\prime}_6. \end{align*} $$

Hence, the assertion follows.

Let $X^*(T/\mathbb {C}^\times )$ and $X^*(\tilde {T}/\mathbb {R}^\times )$ be the weight lattices of $T/\mathbb {C}^\times $ and $\tilde {T}/\mathbb {R}^\times $ , respectively. Then we have

$$ \begin{align*} X^*(T/\mathbb{C}^\times) & \simeq \{ (k_1, k_2, k_3, k_4) \in \mathbb{Z}^4 \, | \, k_1+k_2+k_3+k_4=0 \}, \\ X^*(\tilde{T}/\mathbb{R}^\times) & \simeq \{ (l_1, l_2, l_3) \in \mathbb{Z}^3 \, | \, l_1 + l_2 + l_3 \equiv 0 \bmod 2 \}, \end{align*} $$

where $(k_1, k_2, k_3, k_4)$ and $(l_1, l_2, l_3)$ on the right-hand sides correspond to the characters

$$\begin{align*}(z_1,z_2,z_3,z_4,r) \longmapsto z_1^{k_1} z_2^{k_2} z_3^{k_3} z_4^{k_4} \quad \text{and} \quad (\tilde{z}_1, \tilde{z}_2, \tilde{z}_3, \tilde{r}) \longmapsto \tilde{z}_1^{l_1} \tilde{z}_2^{l_2} \tilde{z}_3^{l_3}, \end{align*}$$

respectively. As an immediate consequence of Lemma 6.2, we have:

Corollary 6.3. The isomorphism $\xi $ induces an isomorphism

$$\begin{align*}X^*(\tilde{T}/\mathbb{R}^\times) \overset{\simeq}\longrightarrow X^*(T/\mathbb{C}^\times), \end{align*}$$

given by

$$\begin{align*}(l_1,l_2,l_3) \longmapsto \left( \frac{l_1+l_2+l_3}{2}, \frac{-l_1-l_2+l_3}{2}, \frac{l_1-l_2-l_3}{2}, \frac{-l_1+l_2-l_3}{2} \right) \end{align*}$$

under the above identifications.

7 Cohomological representations

In this section, we recall various facts about cohomological representations for real groups, with the goal of constructing cohomology classes on the Shimura variety attached to the group

of the previous section. Since

we will be particularly interested in the orthogonal groups $\mathrm {O}(4,2)$ and $\mathrm {O}(0,6)$ .

7.1 Cohomological representations

Let G be a connected real reductive group and K a maximal compact subgroup of G. We assume that $\operatorname {rank} G = \operatorname {rank} K$ and that $G/K$ is a Hermitian symmetric domain. Let $\mathfrak {g}_0$ and $\mathfrak {k}_0$ be the Lie algebras of G and K, respectively. Let $\theta $ be the Cartan involution of G associated to K. Then we have a Cartan decomposition

$$\begin{align*}\mathfrak{g}_0 = \mathfrak{k}_0 \oplus \mathfrak{p}_0, \end{align*}$$

where $\mathfrak {p}_0$ is the $(-1)$ -eigenspace of $\theta $ . Let J be the complex structure on $\mathfrak {p}_0$ , that is, the automorphism of $\mathfrak {p}_0$ given by the multiplication by i on the tangent space of $G/K$ at the origin. Fix a Cartan subalgebra $\mathfrak {t}_0$ of $\mathfrak {k}_0$ . Let $\mathfrak {g}$ , $\mathfrak {k}$ , $\mathfrak {p}$ , $\mathfrak {t}$ be the complexifications of $\mathfrak {g}_0$ , $\mathfrak {k}_0$ , $\mathfrak {p}_0$ , $\mathfrak {t}_0$ , respectively. Let $\mathfrak {p}^\pm $ be the $(\pm i)$ -eigenspace of J in $\mathfrak {p}$ so that

$$\begin{align*}\mathfrak{g} = \mathfrak{k} \oplus \mathfrak{p}^+ \oplus \mathfrak{p}^-. \end{align*}$$

For any subspace $\mathfrak {f}$ of $\mathfrak {g}$ stable under the adjoint action of $\mathfrak {t}$ , we denote by $\Delta (\mathfrak {f})$ the set of roots of $\mathfrak {t}$ in $\mathfrak {f}$ .

We consider an irreducible unitary $(\mathfrak {g}, K)$ -module $\pi $ such that the relative Lie algebra cohomology

$$\begin{align*}H^*(\mathfrak{g}, K; \pi \otimes F) \end{align*}$$

is nonzero for some irreducible finite-dimensional representation F of G. Such $(\mathfrak {g}, K)$ -modules are called cohomological and classified by Vogan–Zuckerman [Reference Vogan and Zuckerman65]. We also consider each piece of the Hodge decomposition

$$\begin{align*}H^i(\mathfrak{g}, K; \pi \otimes F) = \bigoplus_{p+q=i} H^{p,q}(\mathfrak{g}, K; \pi \otimes F). \end{align*}$$

Let $\mathfrak {q}$ be a $\theta $ -stable parabolic subalgebra of $\mathfrak {g}$ , that is, $\mathfrak {q}$ is the sum of nonnegative eigenspaces of $\mathrm {ad}(x)$ for some $x \in i \mathfrak {t}_0$ . Then we have a Levi decomposition

$$\begin{align*}\mathfrak{q} = \mathfrak{l} \oplus \mathfrak{u}, \end{align*}$$

where $\mathfrak {l}$ is the centralizer of x and $\mathfrak {u}$ is the unipotent radical of $\mathfrak {q}$ . Note that $\mathfrak {l}$ is the complexification of $\mathfrak {l}_0 = \mathfrak {q} \cap \mathfrak {g}_0$ and contains $\mathfrak {t}$ . Fix a positive system $\Delta ^+(\mathfrak {l} \cap \mathfrak {k})$ of $\Delta (\mathfrak {l} \cap \mathfrak {k})$ , and choose a positive system $\Delta ^+(\mathfrak {l})$ of $\Delta (\mathfrak {l})$ containing $\Delta ^+(\mathfrak {l} \cap \mathfrak {k})$ . Then

$$\begin{align*}\Delta^+(\mathfrak{k}) = \Delta^+(\mathfrak{l} \cap \mathfrak{k}) \cup \Delta(\mathfrak{u} \cap \mathfrak{k}) \quad \text{and} \quad \Delta^+(\mathfrak{g}) = \Delta^+(\mathfrak{l}) \cup \Delta(\mathfrak{u}) \end{align*}$$

are positive systems of $\Delta (\mathfrak {k})$ and $\Delta (\mathfrak {g})$ , respectively. Put

$$\begin{align*}\rho = \frac{1}{2} \sum_{\alpha \in \Delta^+(\mathfrak{g})} \alpha, \qquad \rho(\mathfrak{u} \cap \mathfrak{p}) = \frac{1}{2} \sum_{\alpha \in \Delta(\mathfrak{u} \cap \mathfrak{p})} \alpha. \end{align*}$$

Let L be the centralizer of x in G so that its Lie algebra is $\mathfrak {l}_0$ . Let $\lambda \in \mathfrak {l}^*$ be the differential of a unitary character of L such that $\langle \alpha , \lambda |_{\mathfrak {t}} \rangle \ge 0$ for all $\alpha \in \Delta (\mathfrak {u})$ . Then, by [Reference Vogan and Zuckerman65, Theorem 5.3] (see also [Reference Knapp and Vogan36]), there exists a unique irreducible unitary $(\mathfrak {g}, K)$ -module $A_{\mathfrak {q}}(\lambda )$ such that

  • $A_{\mathfrak {q}}(\lambda )$ has infinitesimal character $\lambda |_{\mathfrak {t}} + \rho $ ;

  • $A_{\mathfrak {q}}(\lambda )$ contains the K-type with highest weight $\lambda |_{\mathfrak {t}} + 2 \rho (\mathfrak {u} \cap \mathfrak {p})$ ;

  • any K-type contained in $A_{\mathfrak {q}}(\lambda )$ has highest weight of the form

    $$\begin{align*}\lambda|_{\mathfrak{t}} + 2 \rho(\mathfrak{u} \cap \mathfrak{p}) + \sum_{\alpha \in \Delta(\mathfrak{u} \cap \mathfrak{p})} n_\alpha \alpha \end{align*}$$
    for some nonnegative integers $n_\alpha $ .

Let F be an irreducible finite-dimensional representation of G with highest weight $\gamma $ . Then

$$\begin{align*}H^i(\mathfrak{g}, K; A_{\mathfrak{q}}(\lambda) \otimes F^*) \simeq \operatorname{Hom}_K(\wedge^i \mathfrak{p}, A_{\mathfrak{q}}(\lambda) \otimes F^*) \end{align*}$$

if $\gamma = \lambda |_{\mathfrak {t}}$ and

$$\begin{align*}H^i(\mathfrak{g}, K; A_{\mathfrak{q}}(\lambda) \otimes F^*) = 0 \end{align*}$$

otherwise (see [Reference Borel and Wallach10]). Now, suppose that $\gamma = \lambda |_{\mathfrak {t}}$ . Then, by [Reference Vogan and Zuckerman65, Proposition 6.19], we have

$$\begin{align*}H^{i+R^+, i+R^-}(\mathfrak{g}, K; A_{\mathfrak{q}}(\lambda) \otimes F^*) \simeq \operatorname{Hom}_{L \cap K}(\wedge^{2i}(\mathfrak{l} \cap \mathfrak{p}), \mathbb{C}), \end{align*}$$

where $R^\pm = \dim (\mathfrak {u} \cap \mathfrak {p}^\pm )$ , and

$$\begin{align*}H^{p,q}(\mathfrak{g}, K; A_{\mathfrak{q}}(\lambda) \otimes F^*) = 0 \end{align*}$$

if $p-q \ne R^+ - R^-$ .

7.2 Local theta lifts

Let the notation be as in §8 below. In particular, $G \simeq \mathrm {O}(p,q)$ and $G' \simeq \mathrm {SL}_2(\mathbb {R})$ . Let $G^0$ be the topological identity component of G. Let $\omega $ be the Weil representation of $G \times G'$ (relative to the character $x \mapsto e^{2 \pi i x}$ of $\mathbb {R}$ ). For any irreducible $(\mathfrak {g}',K')$ -module $\pi $ , the maximal $\pi ^\vee $ -isotypic quotient of $\omega $ is of the form

$$\begin{align*}\Theta(\pi) \boxtimes \pi^\vee \end{align*}$$

for some admissible $(\mathfrak {g},K)$ -module $\Theta (\pi )$ . If $\Theta (\pi )$ is nonzero, then it has a unique irreducible quotient $\theta (\pi )$ by the Howe duality [Reference Howe28].

Now, suppose that $\pi $ is a holomorphic discrete series representation of $G'$ of weight $k + 1$ (i.e., with Harish-Chandra parameter k), where k is an integer with

$$\begin{align*}k \ge 2. \end{align*}$$

For our applications, we consider the theta lifts $\theta (\pi )$ and $\theta (\pi ^\vee )$ when

$$\begin{align*}(p,q) = (4,2) \text{ or } (0,6). \end{align*}$$

7.2.1 The case $(p,q) = (4,2)$

In this case, by the result of J.-S. Li [Reference Li48, Theorem 6.2], we have

$$\begin{align*}\theta(\pi)|_{G^0} = A_{\mathfrak{q}_0}(\lambda_0), \qquad \theta(\pi^\vee)|_{G^0} = A_{\mathfrak{q}_1}(\lambda_1), \end{align*}$$

where $\mathfrak {q}_i = \mathfrak {l}_i \oplus \mathfrak {u}_i$ is the $\theta $ -stable parabolic subalgebra of $\mathfrak {g}$ with

$$ \begin{align*} \mathfrak{l}_0 & \simeq \mathfrak{so}(4) \oplus \mathfrak{so}(2), & \mathfrak{u}_0 & = \mathbb{C} X_{-\varepsilon_1 + \varepsilon_3} \oplus \mathbb{C} X_{-\varepsilon_2 + \varepsilon_3} \oplus \mathbb{C} X_{\varepsilon_1 + \varepsilon_3} \oplus \mathbb{C} X_{\varepsilon_2 + \varepsilon_3}, \\ \mathfrak{l}_1 & \simeq \mathfrak{so}(2) \oplus \mathfrak{so}(2,2), & \mathfrak{u}_1 & = \mathbb{C} X_{\varepsilon_1 - \varepsilon_2} \oplus \mathbb{C} X_{\varepsilon_1 - \varepsilon_3} \oplus \mathbb{C} X_{\varepsilon_1 + \varepsilon_2} \oplus \mathbb{C} X_{\varepsilon_1 + \varepsilon_3} \end{align*} $$

(where $X_{\pm \varepsilon _i \pm \varepsilon _j}$ is a root vector for $\pm \varepsilon _i \pm \varepsilon _j$ ), and

$$\begin{align*}\lambda_0 = (0,0,k-2), \qquad \lambda_1 = (k-2,0,0). \end{align*}$$

Since $\mathfrak {l}_0 = \mathfrak {k}$ and $\mathfrak {u}_0 = \mathfrak {p}^+$ , $A_{\mathfrak {q}_0}(\lambda _0)$ is a holomorphic discrete series representation of $G^0$ . Also, we have

$$\begin{align*}\mathfrak{u}_1 \cap \mathfrak{p}^+ = \mathbb{C} X_{\varepsilon_1 + \varepsilon_3}, \qquad \mathfrak{u}_1 \cap \mathfrak{p}^- = \mathbb{C} X_{\varepsilon_1 - \varepsilon_3} \end{align*}$$

so that $2 \rho (\mathfrak {u}_1 \cap \mathfrak {p}) = 2 \varepsilon _1$ . Hence, the minimal $K^0$ -type of $A_{\mathfrak {q}_1}(\lambda _1)$ has highest weight

$$\begin{align*}(k,0,0). \end{align*}$$

Moreover, since $\mathfrak {l}_1 \cap \mathfrak {k} \simeq \mathfrak {so}(2) \oplus \mathfrak {so}(2) \oplus \mathfrak {so}(2)$ and

$$\begin{align*}\mathfrak{l}_1 \cap \mathfrak{p} = \mathbb{C} X_{\varepsilon_2 - \varepsilon_3} \oplus \mathbb{C} X_{-\varepsilon_2 + \varepsilon_3} \oplus\mathbb{C} X_{\varepsilon_2 + \varepsilon_3} \oplus \mathbb{C} X_{-\varepsilon_2 - \varepsilon_3}, \end{align*}$$

we have

$$\begin{align*}\dim H^{i,j}(\mathfrak{g}, K^0; A_{\mathfrak{q}_1}(\lambda_1) \otimes F) = \begin{cases} 1 & \text{if } (i,j) = (1,1), (3,3), \\ 2 & \text{if } (i,j) = (2,2), \\ 0 & \text{otherwise,} \end{cases} \end{align*}$$

where F is the irreducible finite-dimensional representation of $G^0$ with highest weight $\lambda _1$ . Note that F is self-dual.

7.2.2 The case $(p,q) = (0,6)$

In this case, $\theta (\pi )|_{G^0}$ is the irreducible finite-dimensional representation of $G^0$ with highest weight

$$\begin{align*}\lambda = (k-2,0,0) \end{align*}$$

and $\theta (\pi ^\vee )$ is zero (see, e.g., [Reference Adams1, Proposition 6.5]).

8 Kudla–Millson theory

In the previous section, we studied certain cohomological representations for $G=\mathrm {O}(p,q)$ with $(p,q)=(4,2)$ or $(0,6)$ . In this section, we recall the explicit construction of $(\mathfrak {g},K)$ -cohomology classes attached to these representations using the Weil representation à la Kudla–Millson. While the original papers of Kudla and Millson considered the case of the trivial local system, the case of more general local systems was discussed in Funke–Millson. We also study the restriction of these explicit $(\mathfrak {g}, K)$ -cohomology classes to the subgroup $\mathrm {O}(2,2) \times \mathrm {O}(2,0)$ and $\mathrm {O}(0,4) \times \mathrm {O}(0,2)$ of $\mathrm {O}(4,2)$ and $\mathrm {O}(0,6)$ , respectively.

8.1 Groups and Lie algebras

Let V be an m-dimensional quadratic space over $\mathbb {R}$ of signature $(p,q)$ , where p and q are nonnegative integers such that $p+q=m$ . Namely, V is equipped with a nondegenerate symmetric bilinear form $\langle \cdot , \cdot \rangle : V \times V \rightarrow \mathbb {R}$ and an orthogonal basis $\{ e_i \, | \, 1 \le i \le m \}$ such that

$$\begin{align*}\langle e_i, e_i \rangle = \begin{cases} +1 & \text{if } 1 \le i \le p, \\ -1 & \text{if } p+1 \le i \le m. \end{cases} \end{align*}$$

We assume that p and q are even. Let $G = \mathrm {O}(V) \simeq \mathrm {O}(p,q)$ be the orthogonal group of V. Put

$$\begin{align*}V_+ = \mathbb{R} e_1 + \dots + \mathbb{R} e_p, \qquad V_- = \mathbb{R} e_{p+1} + \dots + \mathbb{R} e_m \end{align*}$$

so that $V = V_+ \oplus V_-$ . We define a Cartan involution $\theta $ of G by

$$\begin{align*}\theta(g) = I_V \cdot g \cdot I_V, \end{align*}$$

where $I_V = \mathrm {id}_{V_+} \oplus (-\mathrm {id}_{V_-})$ . Let K be the maximal compact subgroup of G with respect to $\theta $ . Then $K = \mathrm {O}(V_+) \times \mathrm {O}(V_-) \simeq \mathrm {O}(p) \times \mathrm {O}(q)$ . We define a maximal torus T of G by

$$\begin{align*}T = \mathrm{SO}(V_1) \times \dots \times \mathrm{SO}(V_r) \simeq \mathrm{SO}(2)^r, \end{align*}$$

where $V_i = \mathbb {R} e_{2i-1} + \mathbb {R} e_{2i}$ and $r = \frac {m}{2}$ .

Let $\mathfrak {g}_0$ be the Lie algebra of G. Then we have a G-equivariant isomorphism $\rho : \wedge ^2 V \rightarrow \mathfrak {g}_0$ given by

$$\begin{align*}\rho(u \wedge v)(w) = \langle u, w \rangle v - \langle v, w \rangle u. \end{align*}$$

We take a basis $\{ X_{ij} \, | \, 1 \le i < j \le m \}$ of $\mathfrak {g}_0$ given by $X_{ij} = \rho (e_i \wedge e_j)$ . Let $\{ \omega _{ij} \, | \, 1 \le i < j \le m \}$ be its dual basis of $\mathfrak {g}_0^*$ . We have a Cartan decomposition

$$\begin{align*}\mathfrak{g}_0 = \mathfrak{k}_0 \oplus \mathfrak{p}_0, \end{align*}$$

where $\mathfrak {k}_0$ and $\mathfrak {p}_0$ are the $(+1)$ -eigenspace and $(-1)$ -eigenspace of $\theta $ , respectively. Note that $\mathfrak {k}_0$ is the Lie algebra of K. Via the isomorphism $\rho $ , we have

$$\begin{align*}\mathfrak{k}_0 \simeq \wedge^2 V_+ \oplus \wedge^2 V_-, \qquad \mathfrak{p}_0 \simeq V_+ \otimes V_-. \end{align*}$$

Let $\mathfrak {t}_0$ be the Lie algebra of T. Let $\mathfrak {g}$ , $\mathfrak {k}$ , $\mathfrak {p}$ , $\mathfrak {t}$ be the complexifications of $\mathfrak {g}_0$ , $\mathfrak {k}_0$ , $\mathfrak {p}_0$ , $\mathfrak {t}_0$ , respectively. If $q=2$ , then we have a complex structure $\mathrm {id}_{V_+} \otimes J_{V_-}$ on $\mathfrak {p}_0$ and hence a decomposition

$$\begin{align*}\mathfrak{g} = \mathfrak{k} \oplus \mathfrak{p}^+ \oplus \mathfrak{p}^-, \end{align*}$$

where $J_{V_-}$ is defined by

$$\begin{align*}J_{V_-}(e_{m-1}) = -e_m, \qquad J_{V_-}(e_m) = e_{m-1}, \end{align*}$$

and $\mathfrak {p}^+$ and $\mathfrak {p}^-$ are the $(+i)$ -eigenspace and $(-i)$ -eigenspace of $\mathrm {id}_{V_+} \otimes J_{V_-}$ in $\mathfrak {p}$ , respectively.

Let W be a two-dimensional symplectic space over $\mathbb {R}$ . Namely, W is equipped with a nondegenerate skew-symmetric bilinear form $\langle \cdot , \cdot \rangle : W \times W \rightarrow \mathbb {R}$ and a basis $\{ e, f \}$ such that

$$\begin{align*}\langle e, e \rangle = \langle f, f \rangle = 0, \qquad \langle e, f \rangle = 1. \end{align*}$$

Let $G' = \mathrm {Sp}(W) \simeq \mathrm {SL}_2(\mathbb {R})$ be the symplectic group of W. Let $K' \simeq \mathrm {U}(1)$ be the standard maximal compact subgroup of $G'$ , where $\mathrm {U}(1)$ is embedded into $\mathrm {SL}_2(\mathbb {R})$ by $a + bi \mapsto \left( \begin {smallmatrix} a & b \\ -b & a \end {smallmatrix} \right)$ . Let $\mathfrak {g}_0'$ be the Lie algebra of $G'$ and $\mathfrak {g}'$ its complexification.

8.2 Finite-dimensional representations of G

Let $\{ \varepsilon _i \, | \, 1 \le i \le r \}$ be the basis of $\mathfrak {t}_0^*$ given by $\varepsilon _i(t) = t_i$ for

$$\begin{align*}t = \left( \begin{pmatrix} & t_1 \\ -t_1 & \end{pmatrix}, \dots, \begin{pmatrix} & t_r \\ -t_r & \end{pmatrix} \right). \end{align*}$$

We identify $\mathfrak {t}^*$ with $\mathbb {C}^r$ via this basis. Under this identification, the weight lattice is given by $\mathbb {Z}^r$ . We take

$$\begin{align*}\{ \varepsilon_i - \varepsilon_{i+1} \, | \, 1 \le i < r \} \cup \{ \varepsilon_{r-1} + \varepsilon_r \} \end{align*}$$

as a set of simple roots. Then $\lambda = (\lambda _1,\dots ,\lambda _r) \in \mathbb {Z}^r$ is dominant if and only if

$$\begin{align*}\lambda_1 \ge \dots \ge \lambda_{r-1} \ge |\lambda_r|. \end{align*}$$

Now, assume that $r \ge 2$ . We only consider dominant weights $\lambda $ of the form

$$\begin{align*}\lambda = (\ell,0,\dots,0) \end{align*}$$

for some nonnegative integer $\ell $ . In particular, we have $\mathbb {S}_\lambda V = \mathrm {Sym}^\ell V$ , where $\mathbb {S}_\lambda $ is the Schur functor associated to $\lambda $ . Put $\mathrm {S}^\ell V = \mathrm {Sym}^\ell V \otimes \mathbb {C}$ and equip it with a nondegenerate G-invariant bilinear pairing $\langle \cdot , \cdot \rangle : \mathrm {S}^\ell V \times \mathrm {S}^\ell V \rightarrow \mathbb {C}$ given by

$$\begin{align*}\langle v_1 \cdots v_\ell, w_1 \cdots w_\ell \rangle = \sum_{\sigma \in \mathfrak{S}_\ell} \langle v_1, w_{\sigma(1)} \rangle \cdots \langle v_\ell, w_{\sigma(\ell)} \rangle, \end{align*}$$

where $\mathfrak {S}_\ell $ is the symmetric group of degree $\ell $ . We denote by $\mathscr {H}^\ell V$ the kernel of the contraction $\mathrm {S}^{\ell } V \rightarrow \mathrm {S}^{\ell -2} V$ given by

$$\begin{align*}v_1 \cdots v_\ell \longmapsto \sum_{i<j} \langle v_i, v_j \rangle \cdot v_1 \cdots \hat{v}_i \cdots \hat{v}_j \cdots v_\ell. \end{align*}$$

Then, by [Reference Fulton and Harris18, §19.5], $\mathscr {H}^\ell V$ is the irreducible finite-dimensional representation of G with highest weight $\lambda $ , whose highest weight vector is given by

$$\begin{align*}(e_1+ie_2)^\ell. \end{align*}$$

Also, the pairing $\langle \cdot , \cdot \rangle $ induces a G-equivariant orthogonal projection

(8.1) $$ \begin{align} \mathrm{S}^\ell V \longrightarrow \mathscr{H}^\ell V. \end{align} $$

8.3 Weil representations

We recall the Schrödinger model $\mathcal {S}(V)$ of the Weil representation $\omega $ of $G \times G'$ (relative to the character $x \mapsto e^{2 \pi i x}$ of $\mathbb {R}$ ), where $\mathcal {S}(V)$ is the space of Schwartz functions on V. By [Reference Kudla40, §5], the action of G is given by

$$\begin{align*}\omega(g)\varphi(x) = \varphi(g^{-1} x), \end{align*}$$

and the action of $G'$ is given by

$$ \begin{align*} \omega \begin{pmatrix} a & \\ & a^{-1} \end{pmatrix} \varphi(x) & = a^{\frac{m}{2}} \varphi(ax), \\ \omega \begin{pmatrix} 1 & b \\ & 1 \end{pmatrix} \varphi(x) & = \varphi(x) e^{\pi i b \langle x, x \rangle}, \\ \omega \begin{pmatrix} & -1 \\ 1 & \end{pmatrix} \varphi(x) & = i^{\frac{q-p}{2}} \int_V \varphi(y) e^{-2 \pi i \langle x, y \rangle} \, dy. \end{align*} $$

Let $x_1, \dots , x_m$ be the coordinates on V with respect to the basis $\{ e_1, \dots , e_m \}$ . We denote by $S(V)$ the subspace of $\mathcal {S}(V)$ consisting of functions of the form $p \cdot \varphi _0$ , where p is a polynomial function and $\varphi _0$ is the Gaussian defined by

$$\begin{align*}\varphi_0(x_1, \dots, x_m) = e^{- \pi (x_1^2 + \dots + x_m^2)}. \end{align*}$$

We also recall the Fock model $\mathscr {P}(\mathbb {C}^m)$ of the Weil representation $\omega $ of $\mathfrak {g} \times \mathfrak {g}'$ (relative to $\lambda = 2 \pi i$ ), where $\mathscr {P}(\mathbb {C}^m)$ is the space of polynomial functions on $\mathbb {C}^m$ . We refer the reader to [Reference Kudla and Millson42, §7], [Reference Funke and Millson19, Appendix A] for details. Let $z_1, \dots , z_m$ be the coordinates on $\mathbb {C}^m$ . Then, by [19, Lemma A.3], we have a $\mathfrak {g} \times \mathfrak {g}'$ -equivariant isomorphism $\iota : S(V) \simeq \mathscr {P}(\mathbb {C}^m)$ such that $\iota (\varphi _0) = 1$ and such that

(8.2) $$ \begin{align} \begin{aligned} \iota \left( x_\alpha - \frac{1}{2 \pi} \frac{\partial}{\partial x_\alpha} \right) \iota^{-1} & = \frac{1}{2 \pi i} z_\alpha, & \iota \left( x_\alpha + \frac{1}{2 \pi} \frac{\partial}{\partial x_\alpha} \right) \iota^{-1} & = 2i \frac{\partial}{\partial z_\alpha}, \\ \iota \left( x_\mu - \frac{1}{2 \pi} \frac{\partial}{\partial x_\mu} \right) \iota^{-1} & = - \frac{1}{2 \pi i} z_\mu, & \iota \left( x_\mu + \frac{1}{2 \pi} \frac{\partial}{\partial x_\mu} \right) \iota^{-1} & = - 2i \frac{\partial}{\partial z_\mu} \end{aligned} \end{align} $$

for $1 \le \alpha \le p$ and $p+1 \le \mu \le m$ .

8.4 Schwartz forms

We now recall the Schwartz forms constructed by Kudla–Millson [Reference Kudla and Millson41] in the case of trivial coefficients and Funke–Millson [Reference Funke and Millson19] in general. Let $\ell $ be a nonnegative integer. Recall that the signature of V is $(p,q)$ . If $p \ge 1$ , then as in [Reference Funke and Millson19, §6.2], we define

$$\begin{align*}\varphi_{q,\ell} \in \mathscr{P}(\mathbb{C}^m) \otimes \wedge^q \mathfrak{p}^* \otimes \mathrm{S}^{\ell} V \end{align*}$$

by

$$\begin{align*}\varphi_{q,\ell} = \left( \frac{1}{4 \pi i} \right)^{\ell+q} \sum_{\alpha} \sum_{\beta} z_\alpha z_\beta \otimes \omega_\alpha \otimes e_\beta, \end{align*}$$

where the sums run over $\alpha = (\alpha _1, \dots , \alpha _q) \in \{ 1, \dots , p \}^q$ and $\beta = (\beta _1, \dots , \beta _\ell ) \in \{ 1, \dots , p \}^\ell $ , and

$$ \begin{align*} z_\alpha & = z_{\alpha_1} \cdots z_{\alpha_q}, & z_\beta & = z_{\beta_1} \cdots z_{\beta_\ell}, \\ \omega_\alpha & = \omega_{\alpha_1 p+1} \wedge \cdots \wedge \omega_{\alpha_q p+q}, & e_\beta & = e_{\beta_1} \cdots e_{\beta_\ell}. \end{align*} $$

(Note that we scale the Schwartz form given in [Reference Funke and Millson19, §6.2] by $2^{-\frac {q}{2}}$ and take its image under the projection $V^{\otimes \ell } \otimes \mathbb {C} \rightarrow \mathrm {S}^\ell V$ .) Then we define

$$\begin{align*}\varphi^{\prime}_{q, \ell} \in \mathscr{P}(\mathbb{C}^m) \otimes \wedge^q \mathfrak{p}^* \otimes \mathscr{H}^\ell V \end{align*}$$

as the image of $\varphi _{q,\ell }$ under the G-equivariant projection $\mathrm {S}^\ell V \rightarrow \mathscr {H}^\ell V$ as in equation (8.1). Via the isomorphism $\iota $ , we also regard $\varphi ^{\prime }_{q, \ell }$ as an element in $S(V) \otimes \wedge ^q \mathfrak {p}^* \otimes \mathscr {H}^\ell V$ . By [Reference Funke and Millson19, Theorem 5.6], $\varphi ^{\prime }_{q, \ell }$ is invariant under the diagonal action of K and

$$\begin{align*}(\omega(t) \otimes 1 \otimes 1) \varphi^{\prime}_{q, \ell} = t^{\ell + \frac{m}{2}} \varphi^{\prime}_{q, \ell} \end{align*}$$

for $t \in K' \simeq \mathrm {U}(1)$ . By [Reference Funke and Millson19, Theorem 5.7], $\varphi ^{\prime }_{q, \ell }$ defines a closed differential form on $G/K$ . Namely, $d \varphi ^{\prime }_{q, \ell } = 0$ , where

$$\begin{align*}d : (\mathscr{P}(\mathbb{C}^m) \otimes \wedge^q \mathfrak{p}^* \otimes \mathscr{H}^\ell V)^K \longrightarrow (\mathscr{P}(\mathbb{C}^m) \otimes \wedge^{q+1} \mathfrak{p}^* \otimes \mathscr{H}^\ell V)^K \end{align*}$$

is the differential as in [Reference Funke and Millson19, §5.1].

Similarly, if $p=0$ , then we define

$$\begin{align*}\varphi_\ell \in \mathscr{P}(\mathbb{C}^m) \otimes \mathrm{S}^{\ell} V \end{align*}$$

by

$$\begin{align*}\varphi_\ell = \left( \frac{1}{4 \pi i} \right)^{\ell} \sum_{\beta} z_\beta \otimes e_\beta, \end{align*}$$

where the sum runs over $\beta = (\beta _1, \dots , \beta _\ell ) \in \{ 1, \dots , m \}^\ell $ . Then we define

$$\begin{align*}\varphi_\ell' \in \mathscr{P}(\mathbb{C}^m) \otimes \mathscr{H}^{\ell} V \end{align*}$$

as the image of $\varphi _\ell $ under the G-equivariant projection $\mathrm {S}^\ell V \rightarrow \mathscr {H}^\ell V$ as in equation (8.1). Via the isomorphism $\iota $ , we also regard $\varphi ^{\prime }_\ell $ as an element in $S(V) \otimes \mathscr {H}^\ell V$ . Then $\varphi ^{\prime }_\ell $ is invariant under the diagonal action of G and

$$\begin{align*}(\omega(t) \otimes 1) \varphi^{\prime}_\ell = t^{-\ell-\frac{m}{2}} \varphi^{\prime}_\ell \end{align*}$$

for $t \in K' \simeq \mathrm {U}(1)$ .

8.5 Restrictions and contractions

For our applications, we consider a six-dimensional quadratic space $\tilde {V}$ over $\mathbb {R}$ of signature $(p,q)$ , where

$$\begin{align*}(p,q) = (4,2) \text{ or } (0,6). \end{align*}$$

Let $\tilde {G} = \mathrm {O}(\tilde {V})$ be the orthogonal group of $\tilde {V}$ . As in §8.1, we take a basis $\{e_1, \dots , e_6\}$ of $\tilde {V}$ and define a Cartan involution $\theta $ of $\tilde {G}$ . Let $\tilde {K}$ be the maximal compact subgroup of $\tilde {G}$ with respect to $\theta $ . Let $\tilde {\mathfrak {g}} = \tilde {\mathfrak {k}} \oplus \tilde {\mathfrak {p}}$ be the complexified Lie algebra of $\tilde {G}$ , where $\tilde {\mathfrak {k}}$ and $\tilde {\mathfrak {p}}$ are $(+1)$ -eigenspace and $(-1)$ -eigenspace of $\theta $ , respectively.

Put

$$\begin{align*}V = \mathbb{R} e_1 + \mathbb{R} e_2 + \mathbb{R} e_5 + \mathbb{R} e_6, \qquad V_0 = \mathbb{R} e_3 + \mathbb{R} e_4 \end{align*}$$

so that $\tilde {V} = V \oplus V_0$ . Let $G = \mathrm {O}(V)$ be the orthogonal group of V, and regard it as a subgroup of $\tilde {G}$ . The natural inclusion $V \hookrightarrow \tilde {V}$ induces a commutative diagram

(where the horizontal maps are the inclusions and the vertical maps are the contractions) and hence a G-equivariant inclusion

$$\begin{align*}\mathscr{H}^\ell V \hookrightarrow \mathscr{H}^\ell \tilde{V}. \end{align*}$$

Also, the natural projection $\tilde {V} \twoheadrightarrow V$ induces a projection $\mathrm {S}^\ell \tilde {V} \twoheadrightarrow \mathrm {S}^\ell V$ . Composing this with the inclusion $\mathscr {H}^\ell \tilde {V} \hookrightarrow \mathrm {S}^\ell \tilde {V}$ and the projection $\mathrm {S}^\ell V \twoheadrightarrow \mathscr {H}^\ell V$ , we obtain a G-equivariant projection

(8.3) $$ \begin{align} \mathscr{H}^\ell \tilde{V} \twoheadrightarrow \mathscr{H}^\ell V, \end{align} $$

which restricts to the identity on $\mathscr {H}^\ell V$ .

8.5.1 The case $(p,q)=(4,2)$

In this case, $\tilde {\mathfrak {p}}^*$ is equipped with a basis $\{\omega _{ij} \, | \, 1 \le i \le 4, \, 5 \le j \le 6 \}$ as in §8.1. Let $\tilde {\mathfrak {p}}^* \twoheadrightarrow \mathfrak {p}^*$ be the K-equivariant projection induced by the natural inclusion $\mathfrak {p} \hookrightarrow \tilde {\mathfrak {p}}$ . This together with equation (8.3) gives rise to a $K \times G'$ -equivariant map

$$\begin{align*}\operatorname{Res}: S(\tilde{V}) \otimes \wedge^2 \tilde{\mathfrak{p}}^* \otimes \mathscr{H}^\ell \tilde{V} \longrightarrow S(\tilde{V}) \otimes \wedge^2 \mathfrak{p}^* \otimes \mathscr{H}^\ell V. \end{align*}$$

Here, K acts diagonally on all three factors, while $G'$ acts only on the first factor. Choose an isomorphism $\wedge ^4 \mathfrak {p}^* \simeq \mathbb {C}$ so that $\omega _{15} \wedge \omega _{25} \wedge \omega _{16} \wedge \omega _{26} \mapsto 1$ . This induces a nondegenerate K-invariant bilinear pairing $\cdot \wedge \cdot : \wedge ^2 \mathfrak {p}^* \times \wedge ^2 \mathfrak {p}^* \rightarrow \wedge ^4 \mathfrak {p}^* \simeq \mathbb {C}$ . For $\boldsymbol {\omega } \in \wedge ^2 \mathfrak {p}^*$ and $\boldsymbol {v} \in \mathrm {S}^\ell V$ , we define a contraction

$$\begin{align*}\mathrm{C}_{\boldsymbol{\omega},\boldsymbol{v}} : S(\tilde{V}) \otimes \wedge^2 \mathfrak{p}^* \otimes \mathrm{S}^\ell V \longrightarrow S(\tilde{V}) \end{align*}$$

by $\mathrm {C}_{\boldsymbol {\omega },\boldsymbol {v}} = 1 \otimes ( \cdot \wedge \boldsymbol {\omega }) \otimes \langle \cdot , \boldsymbol {v} \rangle $ .

Let $\varphi ^{\prime }_{2,\ell } \in S(\tilde {V}) \otimes \wedge ^2 \tilde {\mathfrak {p}}^* \otimes \mathscr {H}^\ell \tilde {V}$ be the Schwartz form as in §8.4. We shall compute $\mathrm {C}_{\boldsymbol {\omega },\boldsymbol {v}}(\operatorname {Res}(\varphi ^{\prime }_{2,\ell }))$ for $\boldsymbol {\omega }$ and $\boldsymbol {v}$ given as follows. Put

$$ \begin{align*} \omega^{++} & = \omega_{15} + i \omega_{25} + i \omega_{16} - \omega_{26}, \\ \omega^{+-} & = \omega_{15} + i \omega_{25} - i \omega_{16} + \omega_{26}, \\ \omega^{-+} & = \omega_{15} - i \omega_{25} + i \omega_{16} + \omega_{26}, \\ \omega^{--} & = \omega_{15} - i \omega_{25} - i \omega_{16} - \omega_{26}. \end{align*} $$

Then for $t = (t_1, t_2) \in T \simeq \mathrm {U}(1)^2$ (where we identify $\mathrm {U}(1)$ with $\mathrm {SO}(2)$ by $a + b i \mapsto \left( \begin {smallmatrix} a & b \\ -b & a \end {smallmatrix} \right)$ ), we have

$$\begin{align*}t \cdot \omega^{\epsilon_1\epsilon_2} = t_1^{\epsilon_1} t_2^{\epsilon_2} \omega^{\epsilon_1\epsilon_2}. \end{align*}$$

In particular,

$$\begin{align*}(\mathfrak{p}^+)^* = \mathbb{C} \omega^{++} + \mathbb{C} \omega^{-+}, \qquad (\mathfrak{p}^-)^* = \mathbb{C} \omega^{+-} + \mathbb{C} \omega^{--}. \end{align*}$$

Hence, we obtain a basis of $\wedge ^2 \mathfrak {p}^*$ given by

$$ \begin{align*} \omega^{++} \wedge \omega^{-+} & \in \wedge^2 (\mathfrak{p}^+)^*, & \omega^{++} \wedge \omega^{--}, \, \omega^{-+} \wedge \omega^{+-} & \in (\mathfrak{p}^+)^* \wedge (\mathfrak{p}^-)^*, \\ \omega^{+-} \wedge \omega^{--} & \in \wedge^2 (\mathfrak{p}^-)^*, & \omega^{++} \wedge \omega^{+-}, \, \omega^{-+} \wedge \omega^{--} & \in (\mathfrak{p}^+)^* \wedge (\mathfrak{p}^-)^*. \end{align*} $$

Note that in the context of the introduction, the above elements in $\wedge ^2 (\mathfrak {p}^+)^*$ , $(\mathfrak {p}^+)^* \wedge (\mathfrak {p}^-)^*$ , $\wedge ^2 (\mathfrak {p}^-)^*$ correspond to those in

$$\begin{align*}H^{2,0}(X_1 \times X_2), \qquad H^{1,1}(X_1 \times X_2), \qquad H^{0,2}(X_1 \times X_2), \end{align*}$$

respectively, where $X_1$ and $X_2$ are quaternionic Shimura varieties. Also, the elements in $(\mathfrak {p}^+)^* \wedge (\mathfrak {p}^-)^*$ in the first row correspond to those in

$$\begin{align*}H^{1,1}(X_1) \otimes H^{0,0}(X_2), \qquad H^{0,0}(X_1) \otimes H^{1,1}(X_2), \end{align*}$$

respectively, whereas the elements in $(\mathfrak {p}^+)^* \wedge (\mathfrak {p}^-)^*$ in the second row (which are the most relevant for us) correspond to those in

$$\begin{align*}H^{1,0}(X_1) \otimes H^{0,1}(X_2), \qquad H^{0,1}(X_1) \otimes H^{1,0}(X_2), \end{align*}$$

respectively. From the representation-theoretic viewpoint, the former corresponds to the contribution of the trivial representation, whereas the latter corresponds to the contribution of holomorphic and antiholomorphic vectors in the discrete series representation. Put

$$ \begin{align*} \boldsymbol{\omega}^+ & = - \frac{1}{2i} \cdot \omega^{++} \wedge \omega^{+-} = (\omega_{15} + i \omega_{25}) \wedge (\omega_{16} + i \omega_{26}), \\ \boldsymbol{\omega}^- & = - \frac{1}{2i} \cdot \omega^{-+} \wedge \omega^{--} = (\omega_{15} - i \omega_{25}) \wedge (\omega_{16} - i \omega_{26}), \end{align*} $$

and

$$\begin{align*}\boldsymbol{v}^+ = \frac{1}{\ell!} \cdot (e_1 + i e_2)^\ell, \qquad \boldsymbol{v}^- = \frac{1}{\ell!} \cdot (e_1 - i e_2)^\ell. \end{align*}$$

Proposition 8.1. For $\epsilon = \pm $ , we have

$$\begin{align*}\mathrm{C}_{\boldsymbol{\omega}^\epsilon,\boldsymbol{v}^\epsilon}(\operatorname{Res}(\varphi^{\prime}_{2,\ell}))(x) = (x_1 + \epsilon ix_2)^{\ell+2} \cdot \varphi_0(x). \end{align*}$$

Proof. Consider the diagram

where $p,q,r,s$ are the projections. By [Reference Fulton and Harris18, §19.5], we have

$$ \begin{align*} \mathrm{S}^\ell \tilde{V} & \simeq \tau_{(\ell,0,0)} \oplus \tau_{(\ell - 2,0,0)} \oplus \cdots \oplus \tau_{(\ell-2k,0,0)}, & \mathrm{S}^\ell V & \simeq \tau_{(\ell,0)} \oplus \tau_{(\ell - 2,0)} \oplus \cdots \oplus \tau_{(\ell-2k,0)}, \\ \mathscr{H}^\ell \tilde{V} & \simeq \tau_{(\ell,0,0)}, & \mathscr{H}^\ell V & \simeq \tau_{(\ell,0)}, \end{align*} $$

where $\tau _\lambda $ denotes the irreducible representation with highest weight $\lambda $ and $k = [\frac {\ell }{2}]$ . Also, by [Reference Fulton and Harris18, §25.3], we have

$$\begin{align*}\tau_{(j,0,0)}|_{G} \simeq \tau_{(j,0)} \oplus \tau^{\oplus 2}_{(j-1,0)} \oplus \cdots \oplus \tau_{(0,0)}^{\oplus j+1}. \end{align*}$$

Hence, if $p(x) = 0$ for $x \in \mathrm {S}^\ell \tilde {V}$ , then $(q \circ r)(x) = 0$ . This implies that the above diagram is commutative. Since q is the orthogonal projection and $\boldsymbol {v}^\epsilon \in \mathscr {H}^\ell V$ , we have

$$\begin{align*}\langle (s \circ p)(x), \boldsymbol{v}^\epsilon \rangle = \langle (q \circ r)(x), \boldsymbol{v}^\epsilon \rangle = \langle r(x), \boldsymbol{v}^\epsilon \rangle \end{align*}$$

and hence

$$\begin{align*}\mathrm{C}_{\boldsymbol{\omega}^\epsilon,\boldsymbol{v}^\epsilon}(\operatorname{Res}(\varphi^{\prime}_{2,\ell})) = \mathrm{C}_{\boldsymbol{\omega}^\epsilon,\boldsymbol{v}^\epsilon}(\widetilde{\operatorname{Res}}(\varphi_{2,\ell})), \end{align*}$$

where

$$\begin{align*}\widetilde{\operatorname{Res}}: S(\tilde{V}) \otimes \wedge^2 \tilde{\mathfrak{p}}^* \otimes \mathrm{S}^\ell \tilde{V} \longrightarrow S(\tilde{V}) \otimes \wedge^2 \mathfrak{p}^* \otimes \mathrm{S}^\ell V \end{align*}$$

is the natural projection. By definition, we have

$$\begin{align*}\mathrm{C}_{\boldsymbol{\omega}^\epsilon,\boldsymbol{v}^\epsilon}(\widetilde{\operatorname{Res}}(\varphi_{2,\ell})) = \left( \frac{1}{4 \pi i} \right)^{\ell+2} \sum_{\alpha} \sum_{\beta} z_\alpha z_\beta (\omega_\alpha \wedge \boldsymbol{\omega}^\epsilon) \langle e_\beta, \boldsymbol{v}^\epsilon \rangle \end{align*}$$

in $\mathscr {P}(\mathbb {C}^6)$ , where the sums run over $\alpha = (\alpha _1, \alpha _2) \in \{ 1, 2 \}^2$ and $\beta = (\beta _1, \dots , \beta _\ell ) \in \{ 1, 2 \}^\ell $ . It is easy to see that

$$\begin{align*}\sum_{\alpha} z_\alpha (\omega_\alpha \wedge \boldsymbol{\omega}^\epsilon) = (z_1 + \epsilon i z_2)^2, \qquad \sum_{\beta} z_\beta \langle e_\beta, \boldsymbol{v}^\epsilon \rangle = (z_1 + \epsilon iz_2)^\ell \end{align*}$$

so that

$$\begin{align*}\mathrm{C}_{\boldsymbol{\omega}^\epsilon,\boldsymbol{v}^\epsilon}(\widetilde{\operatorname{Res}}(\varphi_{2,\ell})) = \left( \frac{1}{4 \pi i} \right)^{\ell+2} (z_1 + \epsilon iz_2)^{\ell+2}. \end{align*}$$

This combined with equation (8.2) gives the desired formula.

8.5.2 The case $(p,q)=(0,6)$

In this case, we have $\tilde {G} = \tilde {K}$ and $\tilde {\mathfrak {p}}^* = \{ 0 \}$ . As above, equation (8.3) gives rise to a $G \times G'$ -equivariant map,

$$\begin{align*}\operatorname{Res}: S(\tilde{V}) \otimes \mathscr{H}^\ell \tilde{V} \longrightarrow S(\tilde{V}) \otimes \mathscr{H}^\ell V. \end{align*}$$

Here, G acts diagonally on both factors, while $G'$ acts only on the first factor. For $\boldsymbol {v} \in \mathrm {S}^\ell V$ , we define a contraction

$$\begin{align*}\mathrm{C}_{\boldsymbol{v}} : S(\tilde{V}) \otimes \mathrm{S}^\ell V \longrightarrow S(\tilde{V}) \end{align*}$$

by $\mathrm {C}_{\boldsymbol {v}} = 1 \otimes \langle \cdot , \boldsymbol {v} \rangle $ .

Let $\varphi ^{\prime }_\ell \in S(\tilde {V}) \otimes \mathscr {H}^\ell \tilde {V}$ be the Schwartz form as in §8.4. Then, as in Proposition 8.1, we have:

Proposition 8.2. For $\epsilon = \pm $ , we have

$$\begin{align*}\mathrm{C}_{\boldsymbol{v}^\epsilon}(\operatorname{Res}(\varphi^{\prime}_\ell))(x) = (-1)^\ell \cdot (x_1 + \epsilon ix_2)^\ell \cdot \varphi_0(x), \end{align*}$$

where $\boldsymbol {v}^\epsilon = \frac {1}{\ell !} \cdot (e_1 + \epsilon i e_2)^\ell $ .

9 Theta lifting

In this section, we study global theta lifts for some quaternionic dual pairs. The material in this section will be needed in §10 to globalize the construction of the Kudla–Millson cohomology classes from the previous section to the group and to show the nonvanishing of their restriction to a suitable subgroup. Moreover, we will also use it in §11 to study their associated Galois representations and to show that they lie in the $\mathbb {C}$ -span of the Hodge classes.

9.1 Setup

Let F be a number field and $\mathbb {A} = \mathbb {A}_F$ the ring of adèles of F. Let B be a quaternion division algebra over F and $*$ the main involution on B. Let E be a quadratic extension of F which embeds into B. Fix a trace zero element $\mathbf {i} \in E^\times $ and write $\mathrm {N} = \mathrm {N}_{E/F}$ for the norm map from E to F. Let $\xi _E$ be the quadratic character of $\mathbb {A}^\times /F^\times $ associated to $E/F$ by class field theory.

Let V be an m-dimensional right B-space equipped with a skew-Hermitian form $\langle \cdot , \cdot \rangle : V \times V \rightarrow B$ . Let $W = B$ be a one-dimensional left B-space equipped with a Hermitian form $\langle \cdot , \cdot \rangle : W \times W \rightarrow B$ given by

$$\begin{align*}\langle x, y \rangle = x \cdot y^*. \end{align*}$$

Then $\mathrm {GU}(W) \simeq B^\times $ . Put

$$ \begin{align*} G & = \mathrm{GU}(V)^0, & G_1 & = \mathrm{U}(V)^0, \\ H & = \mathrm{GU}(W), & H_1 & = \mathrm{U}(W), \end{align*} $$

and

$$\begin{align*}R = \{ (g,h) \in G \times H \, | \, \nu(g) = \nu(h) \}, \end{align*}$$

where $\nu $ denotes the similitude character. Let $Z_G \simeq F^\times $ and $Z_H \simeq F^\times $ be the centers of $\mathrm {GU}(V)$ and $\mathrm {GU}(W)$ , respectively. Put

$$\begin{align*}(\mathbb{A}^\times)^+ = \nu(G(\mathbb{A})) \cap \nu(H(\mathbb{A})) \end{align*}$$

and

$$ \begin{align*} G(\mathbb{A})^+ & = \{ g \in G(\mathbb{A}) \, | \, \nu(g) \in (\mathbb{A}^\times)^+ \}, & G(F)^+ & = G(F) \cap G(\mathbb{A})^+, \\ H(\mathbb{A})^+ & = \{ h \in H(\mathbb{A}) \, | \, \nu(h) \in (\mathbb{A}^\times)^+ \}, & H(F)^+ & = H(F) \cap H(\mathbb{A})^+. \end{align*} $$

Let $\mathbb {V} = V \otimes _B W$ be a $4m$ -dimensional F-space equipped with a symplectic form

$$\begin{align*}\langle \hspace{-1mm} \langle \cdot, \cdot \rangle \hspace{-1mm} \rangle = \frac{1}{2} \operatorname{tr}_{B/F} \left( \langle \cdot, \cdot \rangle \otimes \langle \cdot, \cdot \rangle^* \right). \end{align*}$$

Let $\mathrm {Mp}(\mathbb {V})_{\mathbb {A}}$ be the metaplectic group

$$\begin{align*}1 \longrightarrow \mathbb{C}^1 \longrightarrow \mathrm{Mp}(\mathbb{V})_{\mathbb{A}} \longrightarrow \mathrm{Sp}(\mathbb{V})(\mathbb{A}) \longrightarrow 1. \end{align*}$$

Fix a complete polarization $\mathbb {V} = \mathbb {X} \oplus \mathbb {Y}$ . Then we can realize the Weil representation $\omega _\psi $ of $\mathrm {Mp}(\mathbb {V})_{\mathbb {A}}$ (relative to a nontrivial additive character $\psi $ of $\mathbb {A}/F$ ) on the Schwartz space $\mathcal {S}(\mathbb {X}(\mathbb {A}))$ . Assume that there exists a homomorphism $\tilde {\imath } : R(\mathbb {A}) \rightarrow \mathrm {Mp}(\mathbb {V})_{\mathbb {A}}$ such that the diagram

is commutative, where i is the canonical splitting. Then for any $\varphi \in \mathcal {S}(\mathbb {X}(\mathbb {A}))$ , we may form a theta function on $R(\mathbb {A})$ :

$$\begin{align*}\Theta_\varphi(g,h) = \sum_{x \in \mathbb{X}} \omega_\psi(\tilde{\imath}(g,h)) \varphi(x). \end{align*}$$

9.2 Theta lifts from $E^\times $ to $B^\times $

Let $V = B$ be a one-dimensional right B-space equipped with a skew-Hermitian form

$$\begin{align*}\langle x, y \rangle = x^* \cdot \kappa \mathbf{i} \cdot y \end{align*}$$

for some $\kappa \in F^\times $ . Then $\mathrm {GU}(V)^0 \simeq E^\times $ so that $(\mathbb {A}^\times )^+ = \mathrm {N}(\mathbb {A}_E^\times )$ and $G(\mathbb {A})^+ = G(\mathbb {A})$ . In Appendix A, we define a splitting $\tilde {\imath } : R(\mathbb {A}) \rightarrow \mathrm {Mp}(\mathbb {V})_{\mathbb {A}}$ as above. Let $\eta $ be a character of $\mathbb {A}_E^\times /E^\times $ . We regard $\eta $ as an automorphic character of $G(\mathbb {A})$ . For $\varphi \in \mathcal {S}(\mathbb {X}(\mathbb {A}))$ and $h \in H(\mathbb {A})^+$ , put

$$\begin{align*}\theta_\varphi(\eta)(h) = \int_{G_1(F) \backslash G_1(\mathbb{A})} \Theta_\varphi(g_1 g, h) \eta(g_1 g) \, dg_1,\end{align*}$$

where we choose $g \in G(\mathbb {A})$ such that $\nu (g) = \nu (h)$ . Since $F^\times \cap \mathrm {N}(\mathbb {A}_E^\times ) = \mathrm {N}(E^\times )$ and hence $\nu (H(F)^+) = \nu (G(F))$ , this integral defines an automorphic form $\theta _\varphi (\eta )$ on $H(\mathbb {A})^+$ . Since $[H(\mathbb {A}) : H(F) H(\mathbb {A})^+] = [\mathbb {A}^\times : F^\times \mathrm {N}(\mathbb {A}_E^\times )] = 2$ , we may extend $\theta _\varphi (\eta )$ to an automorphic form on $H(\mathbb {A})$ by the natural embedding

$$\begin{align*}H(F)^+ \backslash H(\mathbb{A})^+ \hookrightarrow H(F) \backslash H(\mathbb{A}) \end{align*}$$

and extension by zero. Let $\theta (\eta )$ be the automorphic representation of $H(\mathbb {A})$ generated by $\theta _\varphi (\eta )$ for all $\varphi \in \mathcal {S}(\mathbb {X}(\mathbb {A}))$ .

Lemma 9.1. Assume that:

  • $B_v$ is split for all Archimedean places v of F;

  • $\eta _v$ does not factor through the norm map for any place v of F such that $B_v$ is ramified.

Then we have

$$\begin{align*}\theta(\eta) = \pi(\eta)_B, \end{align*}$$

where $\pi (\eta )$ is the automorphic induction of $\eta $ from $\mathrm {GL}_1(\mathbb {A}_E)$ to $\mathrm {GL}_2(\mathbb {A})$ and $\pi (\eta )_B$ is its Jacquet–Langlands transfer to $B^\times (\mathbb {A})$ .

Proof. Suppose that $\theta (\eta )$ is nonzero. Let v be a place of F. If $B_v$ is split, then by §A.13, the splitting $\tilde {\imath } : R(F_v) \rightarrow \mathrm {Mp}(\mathbb {V}_v)$ agrees with the standard one for symplectic-orthogonal dual pairs. Hence, it follows from the local theta correspondence for unramified representations that for any irreducible component $\pi $ of $\theta (\eta )$ , we have $\pi _v \simeq \pi (\eta _v)$ for almost all v, where $\pi (\eta _v)$ is the automorphic induction of $\eta _v$ from $\mathrm {GL}_1(E_v)$ to $\mathrm {GL}_2(F_v)$ . By the strong multiplicity one theorem, $\theta (\eta )$ is irreducible and $\theta (\eta ) = \pi (\eta )_B$ .

Thus, it remains to show that $\theta (\eta )$ is nonzero. Let $\mathbf {V}$ and $\mathbf {W}$ be the one-dimensional Hermitian E-space and the two-dimensional skew-Hermitian E-space, respectively, as in §A.4. Then $\mathrm {GU}(V)^0 = \mathrm {GU}(\mathbf {V})$ and $\mathrm {GU}(W) \hookrightarrow \mathrm {GU}(\mathbf {W})$ . By §A.9, the splitting $\tilde {\imath } : R(\mathbb {A}) \rightarrow \mathrm {Mp}(\mathbb {V})_{\mathbb {A}}$ agrees with the restriction of the standard one $\mathrm {G}(\mathrm {U}(\mathbf {V}) \times \mathrm {U}(\mathbf {W}))(\mathbb {A}) \rightarrow \mathrm {Mp}(\mathbb {V})_{\mathbb {A}}$ for unitary dual pairs. Hence, it suffices to show that the global theta lift of $\chi := \eta |_{\mathbb {A}_E^1}$ (regarded as an automorphic character of $\mathrm {U}(\mathbf {V})(\mathbb {A})$ ) to $\mathrm {U}(\mathbf {W})(\mathbb {A})$ is nonzero. By assumption, we have $\chi \ne 1$ so that the standard L-function $L(s, \chi )$ is holomorphic and nonzero at $s=1$ . This together with the Rallis inner product formula [Reference Ichino29, Reference Gan, Qiu and Takeda21, Reference Yamana71] implies that the nonvanishing of the global theta lift $\theta (\chi )$ to $\mathrm {U}(\mathbf {W})(\mathbb {A})$ is equivalent to the nonvanishing of the local theta lift $\theta (\chi _v)$ to $\mathrm {U}(\mathbf {W}_v)$ for all v. If $B_v$ is split, then $\theta (\chi _v)$ is nonzero since the dual pair $(\mathrm {U}(\mathbf {V}_v), \mathrm {U}(\mathbf {W}_v))$ is in the stable range [Reference Li47]. Suppose that $B_v$ is ramified so that v is non-Archimedean. Let $r^+(\chi _v)$ and $r^-(\chi _v)$ be the first occurrence indices

$$ \begin{align*} r^+(\chi_v) & = \min \{ r \, | \, \text{the theta lift of } \chi_v \text{ to } \mathrm{U}(\mathbf{H}_v^{\oplus r}) \text{ is nonzero} \}, \\ r^-(\chi_v) & = \min \{ r \, | \, \text{the theta lift of } \chi_v \text{ to } \mathrm{U}(\mathbf{W}_v \oplus \mathbf{H}_v^{\oplus r-1}) \text{ is nonzero} \}, \end{align*} $$

where $\mathbf {H}_v$ is the hyperbolic plane over $E_v$ . Since $\chi _v \ne 1$ by assumption, we have $r^+(\chi _v) = 1$ . On the other hand, we have

$$\begin{align*}r^+(\chi_v) + r^-(\chi_v) = 2 \end{align*}$$

by the conservation relation [Reference Sun and Zhu63]. Hence, we have $r^-(\chi _v) = 1$ so that $\theta (\chi _v)$ is nonzero. This completes the proof.

9.3 Theta lifts from $B_1^\times \times B_2^\times $ to $B^\times $

Let $V = B_1 \otimes _E B_2$ be the two-dimensional skew-Hermitian right B-space as in [Reference Ichino and Prasanna30, §2.2], where $B_1$ and $B_2$ are quaternion algebras over F such that E embeds into $B_1$ and $B_2$ and such that $B_1 \cdot B_2 = B$ in the Brauer group. Then $\mathrm {GU}(V)^0 \simeq (B_1^\times \times B_2^\times ) / F^\times $ so that $(\mathbb {A}^\times )^+$ consists of elements $a \in \mathbb {A}^\times $ with $a_v> 0$ for all infinite places v such that $B_{1,v}$ or $B_{2,v}$ or $B_v$ is ramified. In [Reference Ichino and Prasanna30, Appendix C], we have defined a splitting $\tilde {\imath } : R(\mathbb {A}) \rightarrow \mathrm {Mp}(\mathbb {V})_{\mathbb {A}}$ as above. Let $\sigma _1$ and $\sigma _2$ be irreducible unitary cuspidal automorphic representations of $B_1^\times (\mathbb {A})$ and $B_2^\times (\mathbb {A})$ , respectively. We assume that they have the same central character so that we may regard $\sigma _1 \boxtimes \sigma _2$ as an automorphic representation of $G(\mathbb {A})$ . For $\varphi \in \mathcal {S}(\mathbb {X}(\mathbb {A}))$ , $f \in \sigma _1 \boxtimes \sigma _2$ and $h \in H(\mathbb {A})^+$ , put

$$\begin{align*}\theta_\varphi(f)(h) = \int_{G_1(F) \backslash G_1(\mathbb{A})} \Theta_\varphi(g_1 g, h) f(g_1 g) \, dg_1, \end{align*}$$

where we choose $g \in G(\mathbb {A})^+$ such that $\nu (g) = \nu (h)$ . Since $\nu (H(F)^+) = F^\times \cap (\mathbb {A}^\times )^+ = \nu (G(F)^+)$ by Eichler’s norm theorem, this integral defines an automorphic form $\theta _\varphi (f)$ on $H(\mathbb {A})^+$ . Since $H(F) H(\mathbb {A})^+ = H(\mathbb {A})$ , we may extend $\theta _\varphi (f)$ to an automorphic form on $H(\mathbb {A})$ . Let $\theta (\sigma _1 \boxtimes \sigma _2)$ be the automorphic representation of $H(\mathbb {A})$ generated by $\theta _\varphi (f)$ for all $\varphi \in \mathcal {S}(\mathbb {X}(\mathbb {A}))$ and $f \in \sigma _1 \boxtimes \sigma _2$ .

Lemma 9.2. Let $\pi _i$ be the Jacquet–Langlands transfer of $\sigma _i$ to $\mathrm {GL}_2(\mathbb {A})$ .

  1. (i) If $\pi _1 \ne \pi _2$ , then $\theta (\sigma _1 \boxtimes \sigma _2) = 0$ .

  2. (ii) If $\pi _1 = \pi _2$ , then $\theta (\sigma _1 \boxtimes \sigma _2)$ is the Jacquet–Langlands transfer of $\pi _i$ to $B^\times (\mathbb {A})$ (which exists).

Proof. Let $\sigma $ be an irreducible unitary cuspidal automorphic representation of $B^\times (\mathbb {A})$ and $\pi $ its Jacquet–Langlands transfer to $\mathrm {GL}_2(\mathbb {A})$ . For $\varphi \in \mathcal {S}(\mathbb {X}(\mathbb {A}))$ , $f \in \sigma _1 \boxtimes \sigma _2$ and $f' \in \bar {\sigma }$ (where $\bar {\sigma }$ is the complex conjugate of $\sigma $ ), we have a seesaw identity

$$\begin{align*}\int_{Z_H(\mathbb{A}) H(F) \backslash H(\mathbb{A})} \theta_{\varphi}(f)(h) \cdot f'(h) \, dh = \int_{Z_G(\mathbb{A}) G(F) \backslash G(\mathbb{A})} f(g) \cdot \theta_{\varphi}(f')(g) \, dg, \end{align*}$$

where $\theta _\varphi (f')$ is the theta lift of $f'$ to $G(\mathbb {A})$ as in [Reference Ichino and Prasanna30, §4]. Since the theta lift of $\bar {\sigma }$ to $G(\mathbb {A})$ is $\bar {\pi }_{B_1} \boxtimes \bar {\pi }_{B_2}$ by [Reference Ichino and Prasanna30, Proposition 4.2.3], where $\pi _{B_i}$ is the Jacquet–Langlands transfer of $\pi $ to $B_i^\times (\mathbb {A})$ (if it exists), this integral vanishes unless $\sigma _i = \pi _{B_i}$ . In particular, (i) follows. Moreover, if $\sigma _i = \pi _{B_i}$ , then we can find $\varphi $ , f, and $f'$ such that the integral is nonzero. This implies that

$$\begin{align*}\theta(\sigma_1 \boxtimes \sigma_2) = \sigma \end{align*}$$

so that (ii) follows.

9.4 Theta lifts from $B^\times $ to an inner form of $\operatorname {GSO}(4,2)$

Let V be the three-dimensional skew-Hermitian right B-space as in §5.2. Then $(\mathbb {A}^\times )^+ = \mathrm {N}(\mathbb {A}_E^\times )$ and $G(\mathbb {A})^+ = G(\mathbb {A})$ . In Appendix A, we define a splitting $\tilde {\imath } : R(\mathbb {A}) \rightarrow \mathrm {Mp}(\mathbb {V})_{\mathbb {A}}$ as above. Let $\tau $ be an irreducible unitary automorphic representation of $H(\mathbb {A})^+$ . For $\varphi \in \mathcal {S}(\mathbb {X}(\mathbb {A}))$ , $\phi \in \tau $ , and $g \in G(\mathbb {A})$ , put

$$\begin{align*}\theta_\varphi(\phi)(g) = \int_{H_1(F) \backslash H_1(\mathbb{A})} \Theta_\varphi(g, h_1 h) \phi(h_1 h) \, dh_1, \end{align*}$$

where we choose $h \in H(\mathbb {A})^+$ such that $\nu (h) = \nu (g)$ . This integral defines an automorphic form $\theta _\varphi (\phi )$ on $G(\mathbb {A})$ . Let $\theta (\tau )$ be the automorphic representation of $G(\mathbb {A})$ generated by $\theta _\varphi (\phi )$ for all $\varphi \in \mathcal {S}(\mathbb {X}(\mathbb {A}))$ and $\phi \in \tau $ .

In the rest of this section, we assume that $\theta (\tau )$ is nonzero and cuspidal. Note that $\theta (\tau )$ is automatically cuspidal if $\mathrm {U}(V)$ is anisotropic. Then:

Lemma 9.3. The global theta lift $\theta (\tau )$ is irreducible and

$$\begin{align*}\theta(\tau) \simeq \otimes_v \theta(\tau_v), \end{align*}$$

where $\theta (\tau _v)$ is the local theta lift of $\tau _v$ (see the proof for its definition).

Proof. As in [Reference Kudla and Rallis44, Corollary 7.1.3], the assertion follows from the Howe duality, which we describe below. For this, we fix a place v of F and suppress the subscript v from the notation. Also, we work with the category of Harish-Chandra modules if F is Archimedean.

Consider the compact induction

$$\begin{align*}\Omega = \operatorname{ind}^{G\times H^+}_R \omega, \end{align*}$$

where $\omega $ is the Weil representation of R (relative to a fixed nontrivial character of F and a fixed splitting over R). For any irreducible representation $\tau $ of $H^+$ , the maximal $\tau ^\vee $ -isotypic quotient of $\Omega $ is of the form

$$\begin{align*}\Theta(\tau) \boxtimes \tau^\vee \end{align*}$$

for some representation $\Theta (\tau )$ of G. Then the Howe duality asserts that

  1. (i) $\Theta (\tau )$ is of finite length;

  2. (ii) $\Theta (\tau )$ is zero or has a unique irreducible quotient $\theta (\tau )$ ;

  3. (iii) for any irreducible representations $\tau $ and $\tau '$ of $H^+$ which occur as quotients of $\Omega $ , we have

    $$\begin{align*}\theta(\tau) \simeq \theta(\tau') \Longrightarrow \tau \simeq \tau'. \end{align*}$$

This can be deduced from the Howe duality [Reference Howe28, Reference Waldspurger68, Reference Gan and Takeda23, Reference Gan and Sun22] for $(\mathrm {U}(V), \mathrm {U}(W))$ as follows.

We first show that the Howe duality for $(\mathrm {U}(V)^0, \mathrm {U}(W))$ follows from the Howe duality for $(\mathrm {U}(V), \mathrm {U}(W))$ . If B is ramified, then there is nothing to prove since $\mathrm {U}(V)^0(F) = \mathrm {U}(V)(F)$ . If B is split, then we have $\mathrm {U}(V) \simeq \mathrm {O}(V^\dagger )$ and $\mathrm {U}(W) \simeq \mathrm {Sp}(W^\dagger )$ , where $V^\dagger $ and $W^\dagger $ are the six-dimensional quadratic F-space and the two-dimensional symplectic F-space, respectively, associated to V and W by Morita theory. For brevity, we write $\mathsf {G} = \mathrm {O}(V^\dagger )$ and $\mathsf {G}^0 = \mathrm {SO}(V^\dagger )$ . Let $\sigma _0$ be an irreducible representation of $\mathsf {G}^0$ . Then $\sigma _0$ is an irreducible component of $\sigma |_{\mathsf {G}^0}$ for some irreducible representation $\sigma $ of $\mathsf {G}$ . Note that $\sigma $ is not necessarily uniquely determined. Namely, $\sigma _0$ is also an irreducible component of $(\sigma \otimes \operatorname {sgn})|_{\mathsf {G}^0}$ , where $\operatorname {sgn}$ denotes the unique nontrivial character of $\mathsf {G}$ trivial on $\mathsf {G}^0$ . Fix $\varepsilon \in \mathsf {G} \smallsetminus \mathsf {G}^0$ , and put $\sigma _0^\varepsilon (g) = \sigma _0(\varepsilon g \varepsilon ^{-1})$ for $g \in \mathsf {G}^0$ . We have

$$\begin{align*}\sigma_0 \simeq \sigma_0^\varepsilon \Longleftrightarrow \sigma \not\simeq \sigma \otimes \operatorname{sgn}, \end{align*}$$

and

  • if $\sigma \not \simeq \sigma \otimes \operatorname {sgn}$ , then

    $$\begin{align*}\sigma|_{\mathsf{G}^0} = \sigma_0, \qquad \operatorname{Ind}^{\mathsf{G}}_{\mathsf{G}^0} \sigma_0 = \sigma \oplus (\sigma \otimes \operatorname{sgn}); \end{align*}$$
  • if $\sigma \simeq \sigma \otimes \operatorname {sgn}$ , then

    $$\begin{align*}\sigma|_{\mathsf{G}^0} = \sigma_0 \oplus \sigma_0^\varepsilon, \qquad \operatorname{Ind}^{\mathsf{G}}_{\mathsf{G}^0} \sigma_0 = \sigma. \end{align*}$$

Then, by the conservation relation [Reference Sun and Zhu63], we have

  • if $\sigma \not \simeq \sigma \otimes \operatorname {sgn}$ , then at most one of $\sigma $ and $\sigma \otimes \operatorname {sgn}$ occurs as a quotient of $\omega $ ;

  • if $\sigma \simeq \sigma \otimes \operatorname {sgn}$ , then $\sigma $ does not occur as a quotient of $\omega $ .

This reduces the Howe duality for $(\mathrm {U}(V)^0, \mathrm {U}(W))$ to the Howe duality for $(\mathrm {U}(V), \mathrm {U}(W))$ .

Finally, as in [Reference Roberts61], [Reference Gan and Tantono24, §3] (noting that the projections $R \rightarrow G$ and $R\rightarrow H^+$ are surjective), the Howe duality for $(\mathrm {GU}(V)^0, \mathrm {GU}(W)^+)$ follows from the Howe duality for $(\mathrm {U}(V)^0, \mathrm {U}(W))$ . This completes the proof.

Now, we explicate the local theta lift $\theta (\tau _v)$ in the unramified case. Fix a non-Archimedean place v of F such that:

  • $F_v$ is of odd residual characteristic;

  • $E_v$ is unramified over $F_v$ ;

  • $B_v$ is split over $F_v$ ;

  • $V_v^\dagger $ has a self-dual $\mathcal {O}_{F_v}$ -lattice;

  • $\psi _v$ is of order zero;

  • $\tau _v$ is unramified.

Here, $V_v^\dagger $ is the six-dimensional quadratic $F_v$ -space associated to $V_v$ . For the moment, we suppress the subscript v from the notation. We may take a trace zero element $\mathbf {i} \in E^\times $ such that $u = \mathbf {i}^2 \in \mathcal {O}_F^\times $ and identify G with the group

$$\begin{align*}\{ g \in \mathrm{GL}_6(F) \, | \, {}^t g \mathcal{Q} g = \nu(g) \cdot \mathcal{Q}, \, \det g = \nu(g)^3 \}, \end{align*}$$

where

$$\begin{align*}\mathcal{Q} = \begin{pmatrix} & & & & & 1 \\ & & & & 1 & \\ & & 1 & & & \\ & & & -u & & \\ & 1 & & & & \\ 1 & & & & & \end{pmatrix}. \end{align*}$$

Let $B_G$ be a Borel subgroup of G and T a maximal torus of G given by

$$\begin{align*}B_G = \left\{ \begin{pmatrix} * & * & * & * & * & * \\ & * & * & * & * & * \\ & & * & * & * & * \\ & & * & * & * & * \\ & & & & * & * \\ & & & & & * \end{pmatrix} \right\} \quad \supset \quad T = \left\{ \begin{pmatrix} * & & & & & \\ & * & & & & \\ & & * & * & & \\ & & * & * & & \\ & & & & * & \\ & & & & & * \end{pmatrix} \right\}. \end{align*}$$

Then we have an isomorphism $T \simeq (F^\times )^2 \times E^\times $ defined by

$$\begin{align*}(t_1, t_2, a + b\mathbf{i}) \longmapsto \begin{pmatrix} t_1 & & & & & \\ & t_2 & & & & \\ & & a & bu & & \\ & & b & a & & \\ & & & & \nu t_2^{-1} & \\ & & & & & \nu t_1^{-1} \end{pmatrix}, \end{align*}$$

where $\nu = a^2 - b^2 u$ . Also, we may identify H with $\mathrm {GL}_2(F)$ so that

$$\begin{align*}H^+ = \{ h \in \mathrm{GL}_2(F) \, | \, \det h \in \mathrm{N}(E^\times) \}. \end{align*}$$

Let $B_H$ be the Borel subgroup of H consisting of upper triangular matrices. Recall that $\tau $ is an irreducible unramified representation of $H^+$ . Then $\tau $ is an irreducible component of

$$\begin{align*}\operatorname{Ind}^H_{B_H}(\chi_1 \otimes \chi_2)|_{H^+} \end{align*}$$

for some unramified characters $\chi _1, \chi _2$ of $F^\times $ . Note that $\chi _1, \chi _2$ are not necessarily uniquely determined. Namely, $\tau $ is also an irreducible component of $\operatorname {Ind}^H_{B_H}(\chi _1 \xi _E \otimes \chi _2 \xi _E)|_{H^+}$ . Then:

Lemma 9.4. The local theta lift $\theta (\tau )$ is an irreducible component of

$$\begin{align*}\operatorname{Ind}^G_{B_G}(\chi_1 \chi_2^{-1} \xi_E \otimes |\cdot| \otimes (\chi_2 |\cdot|^{-\frac{1}{2}}) \circ \mathrm{N} ). \end{align*}$$

Proof. By §A.13, the splitting $\tilde {\imath } : R \rightarrow \mathrm {Mp}(\mathbb {V})$ agrees with the standard one for symplectic-orthogonal dual pairs. Hence, the assertion follows from the standard unramified computation. We omit the details.

Suppose again that F is a number field. We further assume that $B_v$ is split for all Archimedean places v of F and that $\mathrm {U}(V)$ is anisotropic over F. Then we show that the near equivalence class of $\theta (\tau )$ consists of automorphic representations $\theta (\tau ')$ , where $\tau '$ runs over automorphic representations in the near equivalence class of $\tau $ . Namely, we have:

Proposition 9.5. Let $\pi $ be an irreducible unitary automorphic representation of $G(\mathbb {A})$ such that $\pi _v \simeq \theta (\tau _v)$ for almost all v. Then there exists an irreducible automorphic representation $\tau '$ of $H(\mathbb {A})^+$ such that $\tau _v' \simeq \tau _v$ for almost all v and such that

$$\begin{align*}\pi = \theta(\tau'). \end{align*}$$

To prove this proposition, we consider the theta lift in the opposite direction. For $\varphi \in \mathcal {S}(\mathbb {X}(\mathbb {A}))$ , $f \in \pi $ and $h \in H(\mathbb {A})^+$ , put

$$\begin{align*}\theta_\varphi(\bar{f})(h) = \int_{G_1(F) \backslash G_1(\mathbb{A})} \Theta_\varphi(g_1 g, h) \overline{f(g_1 g)} \, dg_1, \end{align*}$$

where we choose $g \in G(\mathbb {A})$ such that $\nu (g) = \nu (h)$ . This integral defines an automorphic form $\theta _\varphi (\bar {f})$ on $H(\mathbb {A})^+$ . Let $\theta (\bar {\pi })$ be the automorphic representation of $H(\mathbb {A})^+$ generated by $\theta _\varphi (\bar {f})$ for all $\varphi \in \mathcal {S}(\mathbb {X}(\mathbb {A}))$ and $f \in \pi $ .

Lemma 9.6. We have

$$\begin{align*}\theta(\bar{\pi}) \ne 0. \end{align*}$$

Now, Proposition 9.5 is an immediate consequence of Lemma 9.6. Indeed, as in Lemma 9.3, it follows from the Howe duality that $\theta (\bar {\pi })$ is irreducible and $\theta (\bar {\pi }) \simeq \otimes _v \theta (\bar {\pi }_v)$ , where $\theta (\bar {\pi }_v)$ is the local theta lift of $\bar {\pi }_v$ . Since $\pi _v \simeq \theta (\tau _v)$ for almost all v by assumption, we have

$$\begin{align*}\theta(\bar{\pi}_v) \simeq \bar{\tau}_v \end{align*}$$

for almost all v. Hence, $\tau ' = \overline {\theta (\bar {\pi })}$ satisfies the desired condition.

Lemma 9.6 can be deduced from the Rallis inner product formula as follows.

9.5 Proof of Lemma 9.6

9.5.1 Doubled spaces

Let $V^\square = V \oplus V$ and $\mathbb {V}^\square = \mathbb {V} \oplus \mathbb {V} = V^\square \otimes _B W$ be the doubled spaces as in §A.3. Let $\iota : \mathrm {U}(V) \times \mathrm {U}(V) \hookrightarrow \mathrm {U}(V^\square )$ be the natural embedding. Let $\mathbb {V}^\square = \mathbb {X}^\square \oplus \mathbb {Y}^\square = \mathbb {V}^{\bigtriangledown } \oplus \mathbb {V}^{\triangle }$ be complete polarizations defined by

$$ \begin{align*} \mathbb{X}^\square & = \mathbb{X} \oplus \mathbb{X}, & \mathbb{V}^{\bigtriangledown} & = V^{\bigtriangledown} \otimes_B W, & V^{\bigtriangledown} & = \{ (v,-v) \, | \, v \in V \}, \\ \mathbb{Y}^\square & = \mathbb{Y} \oplus \mathbb{Y}, & \mathbb{V}^{\triangle} & = V^{\triangle} \otimes_B W, & V^{\triangle} & = \{ (v,v) \, | \, v \in V \}. \end{align*} $$

Let $\omega _\psi ^\square $ be the Weil representation of $\mathrm {U}(V^\square )(\mathbb {A}) \times \mathrm {U}(W)(\mathbb {A})$ relative to $\psi $ realized on the Schwartz space $\mathcal {S}(\mathbb {V}^\bigtriangledown (\mathbb {A}))$ as in [Reference Kudla40, §5]. Then for any $\varphi \in \mathcal {S}(\mathbb {V}^\bigtriangledown (\mathbb {A}))$ , we may form a theta function on $\mathrm {U}(V^\square )(\mathbb {A}) \times \mathrm {U}(W)(\mathbb {A})$ :

$$\begin{align*}\Theta_\varphi^\square(g,h) = \sum_{x \in \mathbb{V}^\bigtriangledown(F)} \omega_\psi^\square(g,h) \varphi(x). \end{align*}$$

We define a partial Fourier transform,

as in [Reference Ichino and Prasanna30, §4.1.1]. It follows from the definition of $\tilde {\imath }$ , combined with the analog of [Reference Harris, Kudla and Sweet27, Proposition 2.2] for $(\mathrm {U}(V), \mathrm {U}(W))$ , that

induces an isomorphism

$$\begin{align*}(\omega_\psi \circ \tilde{\imath}) \otimes (\bar{\omega}_\psi \circ \tilde{\imath}) \simeq \omega_\psi^\square \circ (\iota \otimes \mathrm{id}) \end{align*}$$

as representations of $G_1(\mathbb {A}) \times G_1(\mathbb {A}) \times H_1(\mathbb {A})$ . Hence, we have

(9.1) $$ \begin{align} \Theta_{\varphi}^\square(\iota(g_1,g_2), h) = \Theta_{\varphi_1}(g_1, h) \overline{\Theta_{\varphi_2}(g_2, h)} \end{align} $$

for

with $\varphi _1, \varphi _2 \in \mathcal {S}(\mathbb {X}(\mathbb {A}))$ , $g_1, g_2 \in G_1(\mathbb {A})$ and $h \in H_1(\mathbb {A})$ .

9.5.2 Degenerate principal series representations

Write

$$\begin{align*}\mathrm{U}(V^\square) = \left\{ g \in \mathrm{GL}_6(B) \, \left| \, g \begin{pmatrix} & {\mathbf{1}}_3 \\ -{\mathbf{1}}_3 & \end{pmatrix} {}^t g^* = \begin{pmatrix} & {\mathbf{1}}_3 \\ -{\mathbf{1}}_3 & \end{pmatrix} \right. \right\} \end{align*}$$

as in §A.3, and put

$$\begin{align*}G_1^\square = \mathrm{U}(V^\square)^0. \end{align*}$$

Let P be the Siegel parabolic subgroup of $G_1^\square $ stabilizing $V^\triangle $ :

$$\begin{align*}P = \left\{ \left. \begin{pmatrix} a & * \\ & ({}^t a^*)^{-1} \end{pmatrix} \, \right| \, a \in \mathrm{GL}_3(B) \right\}. \end{align*}$$

For $s \in \mathbb {C}$ , let $\mathcal {I}(s)$ be the degenerate principal series representation of $G_1^\square (\mathbb {A})$ consisting of smooth functions $\mathcal {F}$ on $G_1^\square (\mathbb {A})$ which satisfy

$$\begin{align*}\mathcal{F} \left( \begin{pmatrix} a & * \\ & ({}^t a^*)^{-1} \end{pmatrix} g \right) = |\nu(a)|^{s+\frac{5}{2}} \cdot \mathcal{F}(g). \end{align*}$$

For a holomorphic section $\mathcal {F} = \mathcal {F}(\cdot , s)$ of $\mathcal {I}(s)$ , we define an Eisenstein series $E(s,\mathcal {F})$ on $G_1^\square (\mathbb {A})$ by (the meromorphic continuation of)

$$\begin{align*}E(g, s,\mathcal{F}) = \sum_{\gamma \in P(F) \backslash G_1^\square(F)} \mathcal{F}(\gamma g, s). \end{align*}$$

For each place v of F, let $\mathcal {I}_v(s)$ be the degenerate principal series representation of $G_{1,v}^\square $ given similarly as above. We define an intertwining operator

$$\begin{align*}M_v(s) : \mathcal{I}_v(s) \longrightarrow \mathcal{I}_v(-s) \end{align*}$$

by (the meromorphic continuation of)

$$\begin{align*}(M_v(s) \mathcal{F})(g,-s) = \int_{U_v} \mathcal{F} \left( \begin{pmatrix} & -{\mathbf{1}}_3 \\ {\mathbf{1}}_3 & \end{pmatrix} ug, s \right) du, \end{align*}$$

where U is the unipotent radical of P. Let

$$\begin{align*}M^*_v(s) = \frac{b_v(s)}{a_v(s)} \cdot M_v(s) \end{align*}$$

be the normalized intertwining operator, where

$$ \begin{align*} a_v(s) & = \zeta_v(2s) \zeta_v(2s-2) \zeta_v(2s-4), \\ b_v(s) & = \zeta_v(2s+1) \zeta_v(2s+3) \zeta_v(2s+5). \end{align*} $$

By [Reference Yamana70, Proposition 4.11(2)], $M^*_v(s)$ is holomorphic for $\operatorname {Re} s \ge 0$ .

Recall the Weil representation $\omega ^\square _{\psi _v}$ of $\mathrm {U}(V_v^\square ) \times \mathrm {U}(W_v)$ relative to $\psi _v$ realized on the Schwartz space $\mathcal {S}(\mathbb {V}_v^\bigtriangledown )$ . For $\varphi \in \mathcal {S}(\mathbb {V}_v^\bigtriangledown )$ , we define $\mathcal {F}_\varphi \in \mathcal {I}_v(-\frac {3}{2})$ by

$$\begin{align*}\mathcal{F}_\varphi(g) = \omega^\square_{\psi_v}(g) \varphi(0). \end{align*}$$

We denote by $\mathcal {R}(W_v)$ the subspace of $\mathcal {I}_v(-\frac {3}{2})$ spanned by $\mathcal {F}_\varphi $ for all $\varphi \in \mathcal {S}(\mathbb {V}_v^\bigtriangledown )$ .

Lemma 9.7. We have

$$\begin{align*}\operatorname{Im} M^*_v(\tfrac{3}{2}) = \mathcal{R}(W_v). \end{align*}$$

Proof. If $B_v$ is ramified (so that v is non-Archimedean by assumption and $\mathrm {U}(V_v^\square ) = \mathrm {U}(V_v^\square )^0$ ), then the assertion is proved in [Reference Yamana69, Theorem 1.3]. Assume that $B_v$ is split. Let $\tilde {W}_v$ be the unique four-dimensional Hermitian left $B_v$ -space and define the subspace $\mathcal {R}(\tilde {W}_v)$ of $\mathcal {I}_v(\frac {3}{2})$ similarly as above. Then we have

$$\begin{align*}\mathcal{R}(\tilde{W}_v) = \mathcal{I}_v(\tfrac{3}{2}), \qquad M^*_v(\tfrac{3}{2}) \mathcal{R}(\tilde{W}_v) = \mathcal{R}(W_v) \end{align*}$$

by [Reference Yamana69, Theorem 1.6], [Reference Loke50, Appendix A], [Reference Yamana70, Proposition 4.11(3)]. We remark that, in these references, the results are stated for the degenerate principal series representation of $\mathrm {U}(V_v^\square )$ , but the above equalities can be deduced by restriction to $\mathrm {U}(V_v^\square )^0$ . This completes the proof.

9.5.3 The doubling method

We denote by $\operatorname {Res}^{G}_{G_1}(\pi )$ the restriction of $\pi $ to $G_1(\mathbb {A})$ as functions. Fix an irreducible component $\sigma $ of $\operatorname {Res}^{G}_{G_1}(\pi )$ . Note that $\sigma _v$ is the irreducible unramified component of

(9.2) $$ \begin{align} \operatorname{Ind}^{G_{1,v}}_{B_{G_{1,v}}}(\chi_{1,v} \chi_{2,v}^{-1} \xi_{E_v} \otimes |\cdot|_v \otimes 1) \end{align} $$

for almost all v, where $B_{G_{1,v}}$ is a Borel subgroup of $G_{1,v}$ with Levi component $F_v^\times \times F_v^\times \times E^1_v$ . Let $\langle \cdot , \cdot \rangle $ be the Petersson inner product on $\sigma $ given by

$$\begin{align*}\langle f_1, f_2 \rangle = \int_{G_1(F) \backslash G_1(\mathbb{A})} f_1(g) \overline{f_2(g)} \, dg, \end{align*}$$

where $dg$ is the Tamagawa measure on $G_1(\mathbb {A})$ . Fix decompositions $\langle \cdot , \cdot \rangle = \prod _v \langle \cdot , \cdot \rangle _v$ and $dg = \prod _v dg_v$ , where $\langle \cdot , \cdot \rangle _v$ is an invariant Hermitian inner product on $\sigma _v$ and $dg_v$ is a Haar measure on $G_{1,v}$ .

We now consider the doubling zeta integral of Piatetski-Shapiro and Rallis [Reference Piatetski-Shapiro and Rallis57, Reference Li49, Reference Lapid and Rallis46, Reference Kakuhama33] given by

$$\begin{align*}Z(s,\mathcal{F},f_1,f_2) = \int_{G_1(F) \backslash G_1(\mathbb{A})} \int_{G_1(F) \backslash G_1(\mathbb{A})} E(\iota(g_1,g_2), s, \mathcal{F}) \overline{f_1(g_1)} f_2(g_2) \, dg_1 \, dg_2 \end{align*}$$

for a holomorphic section $\mathcal {F} = \otimes _v \mathcal {F}_v$ of $\mathcal {I}(s)$ and $f_1 = \otimes _v f_{1,v}, f_2 = \otimes _v f_{2,v} \in \sigma $ . Recalling equation (9.2), we have

(9.3) $$ \begin{align} Z(s,\mathcal{F},f_1,f_2) = \frac{L^S(s+\frac{1}{2}, \operatorname{ad} \tau \times \xi_E) \zeta^S(s+\frac{3}{2}) \zeta^S(s+\frac{1}{2}) \zeta^S(s-\frac{1}{2})}{\zeta^S(2s+1) \zeta^S(2s+3) \zeta^S(2s+5)} \cdot \prod_{v \in S} Z_v(s,\mathcal{F}_v,f_{1,v},f_{2,v}), \end{align} $$

where S is a sufficiently large finite set of places of F and $Z_v(s,\mathcal {F}_v,f_{1,v},f_{2,v})$ is the local zeta integral given by

$$\begin{align*}Z_v(s,\mathcal{F}_v,f_{1,v},f_{2,v}) = \int_{G_{1,v}} \mathcal{F}_v(\iota(g_v, 1), s) \overline{\langle \sigma_v(g_v) f_{1,v}, f_{2,v} \rangle_v} \, dg_v. \end{align*}$$

Moreover, as in [Reference Kudla and Rallis43, Theorem 3.2.2], [Reference Kudla and Rallis44, Proposition 7.2.1], we can prove the following.

Lemma 9.8. There exist a holomorphic section $\mathcal {F}_v$ of $\mathcal {I}_v(s)$ and $f_{1,v}, f_{2,v} \in \sigma _v$ such that $Z_v(s,\mathcal {F}_v,f_{1,v},f_{2,v})$ is holomorphic and nonzero at $s=\frac {3}{2}$ .

9.5.4 The Rallis inner product formula

By equation (9.3) and Lemma 9.8, there exist a holomorphic section $\mathcal {F} = \otimes _v \mathcal {F}_v$ of $\mathcal {I}(s)$ and $f_1 = \otimes _v f_{1,v}, f_2 = \otimes _v f_{2,v} \in \sigma $ such that

$$\begin{align*}\operatorname{Res}_{s=\frac{3}{2}} Z(s,\mathcal{F},f_1,f_2) \ne 0. \end{align*}$$

In fact, $E(s,\mathcal {F})$ has a simple pole at $s=\frac {3}{2}$ by [Reference Yamana70, Theorem 3.1] and its residue can be described as follows. By Lemma 9.7, we have $M_v^*(\frac {3}{2}) \mathcal {F}_v = \mathcal {F}_{\varphi _v}$ for some $\varphi _v \in \mathcal {S}(\mathbb {V}_v^\bigtriangledown )$ . Put $\varphi = \otimes _v \varphi _v \in \mathcal {S}(\mathbb {V}^\bigtriangledown (\mathbb {A}))$ . We define an automorphic form $I(\varphi )$ on $G_1^\square (\mathbb {A})$ by

$$\begin{align*}I(g,\varphi) = \int_{H_1(F) \backslash H_1(\mathbb{A})} \Theta_{\varphi}^\square(g,h) \, dh. \end{align*}$$

Then, by the Siegel–Weil formula [Reference Yamana70, Theorem 7.11], we have

$$\begin{align*}\operatorname{Res}_{s=\frac{3}{2}} E(s,\mathcal{F}) = I(\varphi) \end{align*}$$

up to a nonzero constant. Hence, we have

$$\begin{align*}\int_{G_1(F) \backslash G_1(\mathbb{A})} \int_{G_1(F) \backslash G_1(\mathbb{A})} I(\iota(g_1,g_2), \varphi) \overline{f_1(g_1)} f_2(g_2) \, dg_1 \, dg_2 \ne 0. \end{align*}$$

We may further assume that for some $\varphi _1, \varphi _2 \in \mathcal {S}(\mathbb {X}(\mathbb {A}))$ . Then, by equation (9.1), the left-hand side is equal to

$$ \begin{align*} \int_{G_1(F) \backslash G_1(\mathbb{A})} \int_{G_1(F) \backslash G_1(\mathbb{A})} \int_{H_1(F) \backslash H_1(\mathbb{A})} &\Theta_{\varphi_1}(g_1, h) \overline{\Theta_{\varphi_2}(g_2, h)} \overline{f_1(g_1)} f_2(g_2) \, dh \, dg_1 \, dg_2 \\ &\qquad\qquad\qquad = \int_{H_1(F) \backslash H_1(\mathbb{A})} \theta_{\varphi_1}(\bar{\tilde{f}}_1)(h) \overline{\theta_{\varphi_2}(\bar{\tilde{f}}_2)(h)} \, dh, \end{align*} $$

where we choose $\tilde {f}_i \in \pi $ such that $\tilde {f}_i|_{G_1(\mathbb {A})} = f_i$ . Hence, we have $\theta _{\varphi _i}(\bar {\tilde {f}}_i) \ne 0$ . This completes the proof of Lemma 9.6 and hence of Proposition 9.5.

10 Construction of the cohomology class and nonvanishing of its restriction

Notation

For any reductive algebraic group G over a number field F, we denote by the space of automorphic forms on $G(\mathbb {A})$ .

10.1 Groups

Let F be a totally real number field. We denote by $\mathbb {A} = \mathbb {A}_F$ and $\mathbb {A}_f = \mathbb {A}_{F,f}$ the rings of adèles and finite adèles of F, respectively. Let B be a quaternion division algebra over F. We assume that $B_v$ is split for all real places v of F. Let E be a totally imaginary quadratic extension of F which embeds into B. We write $E = F + F \mathbf {i}$ and $B = E + E\mathbf {j}$ for some trace zero elements $\mathbf {i} \in E^\times $ and $\mathbf {j} \in B^\times $ . Put $u = \mathbf {i}^2 \in F^\times $ and $J = \mathbf {j}^2 \in F^\times $ . Let $\tilde {V} = V^\sharp \oplus V_0^\sharp $ be the three-dimensional skew-Hermitian right B-space as in §5.2, where $V^\sharp $ and $V_0^\sharp $ are the two- and one-dimensional subspaces as in §5.3.1, respectively. To ease notation, we write $V = V^\sharp $ and $V_0 = V_0^\sharp $ . Recall from Example 5.14 and [Reference Ichino and Prasanna30, §2.2] that we may write $V = B_1 \otimes _E B_2$ for some quaternion algebras $B_1$ and $B_2$ over F such that $B_1 \cdot B_2 = B$ in the Brauer group and such that E embeds into $B_1$ and $B_2$ . In particular, $B_1$ and $B_2$ act on V by left multiplication. Put

$$\begin{align*}\tilde{G} = \mathrm{GU}(\tilde{V})^0, \qquad G = \mathrm{GU}(V)^0 \simeq (B_1^\times \times B_2^\times)/F^\times, \qquad G_0 = \mathrm{GU}(V_0)^0 \simeq E^\times. \end{align*}$$

Let $\tilde {Z} \simeq F^\times $ and $Z \simeq F^\times $ be the centers of $\tilde {G}$ and G, respectively. We define a subgroup $\mathbf {G}$ of $G \times G_0$ by

$$\begin{align*}\mathbf{G} = \mathrm{G}(\mathrm{U}(V) \times \mathrm{U}(V_0))^0 = \{ (g, \alpha) \, | \, \nu(g) = \mathrm{N}(\alpha) \}, \end{align*}$$

where $\nu $ is the similitude character and $\mathrm {N} = \mathrm {N}_{E/F}$ is the norm map. We also regard $\mathbf {G}$ as a subgroup of $\tilde {G}$ via the natural embedding. Let $\mathbf {Z} \subset Z \times G_0$ be the center of $\mathbf {G}$ :

$$\begin{align*}\mathbf{Z} \simeq \{ (z, \alpha) \, | \, z^2 = \mathrm{N}(\alpha) \}. \end{align*}$$

Then we have a natural embedding $\tilde {Z} \hookrightarrow \mathbf {Z}$ and an exact sequence

$$\begin{align*}1 \longrightarrow \tilde{Z} \longrightarrow \mathbf{Z} \overset{\mathbf{p}}\longrightarrow E^1 \longrightarrow 1, \end{align*}$$

where $\mathbf {p}(z,\alpha ) = z^{-1} \alpha $ .

Let W be the one-dimensional Hermitian left B-space as in §9.1. Put

$$\begin{align*}H = \mathrm{GU}(W) \simeq B^\times. \end{align*}$$

Let $Z_H \simeq F^\times $ be the center of H.

10.2 Weil representations

Let $\tilde {\mathbb {V}} = \mathbb {V} \oplus \mathbb {V}_0$ be the $12$ -dimensional symplectic F-space given by

$$\begin{align*}\tilde{\mathbb{V}} = \tilde{V} \otimes_B W, \qquad \mathbb{V} = V \otimes_B W, \qquad \mathbb{V}_0 = V_0 \otimes_B W. \end{align*}$$

As in §A.1, we take complete polarizations

$$\begin{align*}\tilde{\mathbb{V}} = \tilde{\mathbb{X}} \oplus \tilde{\mathbb{Y}}, \qquad \mathbb{V} = \mathbb{X} \oplus \mathbb{Y}, \qquad \mathbb{V}_0 = \mathbb{X}_0 \oplus \mathbb{Y}_0 \end{align*}$$

such that

$$\begin{align*}\tilde{\mathbb{X}} = \mathbb{X} \oplus \mathbb{X}_0, \qquad \tilde{\mathbb{Y}} = \mathbb{Y} \oplus \mathbb{Y}_0. \end{align*}$$

By Appendix A and [Reference Ichino and Prasanna30, Appendix C], we may define Weil representations $\omega $ (relative to the standard additive character $\psi $ of $\mathbb {A}/F$ ) of

$$\begin{align*}\mathrm{G}(\mathrm{U}(\tilde{V}) \times \mathrm{U}(W))^0(\mathbb{A}), \qquad \mathrm{G}(\mathrm{U}(V) \times \mathrm{U}(W))^0(\mathbb{A}), \qquad \mathrm{G}(\mathrm{U}(V_0) \times \mathrm{U}(W))^0(\mathbb{A}) \end{align*}$$

on

$$\begin{align*}\mathcal{S}(\tilde{\mathbb{X}}(\mathbb{A})), \qquad \mathcal{S}(\mathbb{X}(\mathbb{A})), \qquad \mathcal{S}(\mathbb{X}_0(\mathbb{A})), \end{align*}$$

respectively, satisfying various compatibilities.

10.3 Real groups

Let $\Sigma _{\infty }$ be the set of real places of F and $\Sigma $ the subset of $v \in \Sigma _{\infty }$ such that $B_{1,v}$ and $B_{2,v}$ are split. We assume that $\Sigma \ne \Sigma _{\infty }$ . Put $d = |\Sigma |$ . For any $v \in \Sigma _{\infty }$ , we may write $J = t_v^2$ for some $t_v \in F_v^\times $ since $B_v$ is split. We define an isomorphism $\mathfrak {i}_v:B_v \rightarrow \mathrm {M}_2(F_v)$ of quaternion $F_v$ -algebras by

$$\begin{align*}\mathfrak{i}_v(a + b \mathbf{i} + c \mathbf{j} + d \mathbf{i} \mathbf{j}) = \begin{pmatrix} a + ct_v & b - dt_v \\ (b + dt_v)u & a - ct_v \end{pmatrix}. \end{align*}$$

Put

$$\begin{align*}e_v = \frac{1}{2} + \frac{t_v}{2J} \mathbf{j}, \qquad e_v' = \frac{1}{2} \mathbf{i} - \frac{t_v}{2J} \mathbf{i} \mathbf{j}, \qquad e_v" = \frac{1}{2u} \mathbf{i} + \frac{t_v}{2uJ} \mathbf{i} \mathbf{j}, \qquad e_v^* = \frac{1}{2} - \frac{t_v}{2J} \mathbf{j} \end{align*}$$

so that

$$\begin{align*}\mathfrak{i}_v(e_v) = \begin{pmatrix} 1 & 0 \\ 0 & 0 \end{pmatrix}, \qquad \mathfrak{i}_v(e_v') = \begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix}, \qquad \mathfrak{i}_v(e_v") = \begin{pmatrix} 0 & 0 \\ 1 & 0 \end{pmatrix}, \qquad \mathfrak{i}_v(e_v^*) = \begin{pmatrix} 0 & 0 \\ 0 & 1 \end{pmatrix}. \end{align*}$$

Note that

$$\begin{align*}\begin{bmatrix} e_v \cdot x \\ e_v' \cdot x \end{bmatrix} = \mathfrak{i}_v(x) \cdot \begin{bmatrix} e_v \\ e_v' \end{bmatrix} \end{align*}$$

for $x \in B_v$ .

Let $v \in \Sigma _{\infty }$ . Let $\tilde {V}_v^\dagger = V_v^\dagger \oplus V_{0,v}^\dagger $ be the six-dimensional quadratic $F_v$ -space as in [Reference Ichino and Prasanna30, §C.2] associated to the $B_v$ -space $\tilde {V}_v = V_v \oplus V_{0,v}$ . By §6.3, the signature of $\tilde {V}_v^\dagger $ is equal to

$$\begin{align*}\begin{cases} (4,2) & \text{if } v \in \Sigma; \\ (0,6) & \text{if } v \in \Sigma_{\infty} \smallsetminus \Sigma. \end{cases} \end{align*}$$

As in §8.1, we take a basis of $\tilde {V}_v^\dagger $ so that we have identifications

$$\begin{align*}\tilde{G}_v = \mathrm{GSO}(4,2), \qquad G_v = \mathrm{GSO}(2,2), \qquad G_{0,v} = \mathrm{GSO}(2,0) \end{align*}$$

if $v \in \Sigma $ and

$$\begin{align*}\tilde{G}_v = \mathrm{GSO}(0,6), \qquad G_v = \mathrm{GSO}(0,4), \qquad G_{0,v} = \mathrm{GSO}(0,2) \end{align*}$$

if $v \in \Sigma _{\infty } \smallsetminus \Sigma $ . Here,

$$\begin{align*}\mathrm{GSO}(p,q) = \{ g \in \mathrm{GL}_{p+q}(\mathbb{R}) \, | \, {}^t g I_{p,q} g = \nu(g) \cdot I_{p,q}, \, \det g = \nu(g)^{\frac{p+q}{2}} \} \end{align*}$$

with

$$\begin{align*}I_{p,q} = \begin{pmatrix} {\mathbf{1}}_p & \\ & -{\mathbf{1}}_q \end{pmatrix} \end{align*}$$

if $p+q$ is even. Let $\tilde {\mathfrak {g}}_v = \tilde {\mathfrak {k}}_v \oplus \tilde {\mathfrak {p}}_v$ and $\mathfrak {g}_v = \mathfrak {k}_v\oplus \mathfrak {p}_v$ be the complexified Lie algebra of $\tilde {G}_v$ and $G_v$ , respectively, where $\tilde {\mathfrak {k}}_v$ and $\mathfrak {k}_v$ (resp. $\tilde {\mathfrak {p}}_v$ and $\mathfrak {p}_v$ ) are the $(+1)$ -eigenspaces (resp. the $(-1)$ -eigenspaces) of the Cartan involutions as in §8.1. Put

$$\begin{align*}\tilde{\mathfrak{p}} = \prod_{v \in \Sigma} \tilde{\mathfrak{p}}_v, \qquad \mathfrak{p} = \prod_{v \in \Sigma} \mathfrak{p}_v. \end{align*}$$

Let

$$\begin{align*}\iota_v:\mathbb{C}^\times \times \mathbb{C}^\times \simeq E_v^\times \times E_v^\times \longrightarrow (B_{1,v}^\times \times B_{2,v}^\times)/F_v^\times \simeq G_v \end{align*}$$

be a map induced by the isomorphism $E_v \simeq \mathbb {C}$ given by $a+b\mathbf {i} \mapsto a + b |u|_{F_v}^{\frac {1}{2}} i$ for $a, b \in F_v = \mathbb {R}$ and the fixed embeddings $\iota _1:E \hookrightarrow B_1$ and $\iota _2:E \hookrightarrow B_2$ . We explicate $\iota _v$ below. Recall from Example 5.14 that $V = \mathbf {e}_1 B + \mathbf {e}_2 B$ is equipped with a skew-Hermitian form

$$\begin{align*}\langle \mathbf{e}_1 x_1 + \mathbf{e}_2 x_2, \mathbf{e}_1 y_1 + \mathbf{e}_2 y_2 \rangle = x_1^* \cdot \mathbf{i} \cdot y_1 - x_2^* \cdot J_1 \mathbf{i} \cdot y_2, \end{align*}$$

where $\mathbf {e}_1 = 1 \otimes 1$ and $\mathbf {e}_2 = \mathbf {j}_1 \otimes 1$ . We take a basis

$$ \begin{align*} e_{1,v} & = \sqrt{2} \cdot |uJ_1|_{F_v}^{-\frac{1}{2}} \cdot \mathbf{e}_2 e_v, & e_{2,v} & = \sqrt{2} \cdot |J_1|_{F_v}^{-\frac{1}{2}} \cdot \mathbf{e}_2 e^{\prime\prime}_v, \\ e_{3,v} & = \sqrt{2} \cdot |u|_{F_v}^{-\frac{1}{2}} \cdot \mathbf{e}_1 e_v, & e_{4,v} & = \sqrt{2} \cdot \mathbf{e}_1 e^{\prime\prime}_v \end{align*} $$

of $V_v^\dagger $ so that

$$\begin{align*}(\langle e_{i,v}, e_{j,v} \rangle^\dagger) = \begin{cases} I_{2,2} & \text{if } v \in \Sigma; \\ I_{0,4} & \text{if } v \in \Sigma_{\infty} \smallsetminus \Sigma. \end{cases} \end{align*}$$

Since

$$ \begin{align*} \iota_1(\mathbf{i}) \mathbf{e}_1 & = \mathbf{e}_1 \mathbf{i}, & \iota_1(\mathbf{i}) \mathbf{e}_2 & = - \mathbf{e}_2 \mathbf{i}, \\ \iota_2(\mathbf{i}) \mathbf{e}_1 & = \mathbf{e}_1 \mathbf{i}, & \iota_2(\mathbf{i}) \mathbf{e}_2 & = \mathbf{e}_2 \mathbf{i} \end{align*} $$

and

$$\begin{align*}\mathbf{i} e_v = u e_v", \qquad \mathbf{i} e_v" = e_v, \end{align*}$$

we have

(10.1) $$ \begin{align} \begin{aligned} \iota_1(i) e_{1,v} & = e_{2,v}, \qquad & \iota_1(i) e_{2,v} & = - e_{1,v}, \qquad & \iota_1(i) e_{3,v} & = - e_{4,v}, \qquad & \iota_1(i) e_{4,v} & = e_{3,v}, \\ \iota_2(i) e_{1,v} & = - e_{2,v}, \qquad & \iota_2(i) e_{2,v} & = e_{1,v}, \qquad & \iota_2(i) e_{3,v} & = - e_{4,v}, \qquad & \iota_2(i) e_{4,v} & = e_{3,v}, \end{aligned} \end{align} $$

where $i = |u|_{F_v}^{-\frac {1}{2}} \cdot \mathbf {i}$ . Hence,

$$\begin{align*}\iota_v(a_1 + b_1 i, a_2 + b_2 i) = \begin{pmatrix} a_1 & -b_1 & & \\ b_1 & a_1 & & \\ & & a_1 & b_1 \\ & & -b_1 & a_1 \end{pmatrix} \begin{pmatrix} a_2 & b_2 & & \\ -b_2 & a_2 & & \\ & & a_2 & b_2 \\ & & -b_2 & a_2 \end{pmatrix}. \end{align*}$$

Also, let $W_v^\dagger $ be the two-dimensional symplectic $F_v$ -space as in [Reference Ichino and Prasanna30, §C.2] associated to the $B_v$ -space $W_v$ . Using a basis $e_v, e_v'$ of $W_v^\dagger $ , we identify $H_v$ with $\mathrm {GL}_2(\mathbb {R})$ . We embed $\mathbb {C}^\times $ into $H_v = \mathrm {GL}_2(\mathbb {R})$ by

$$\begin{align*}a+bi \longmapsto \begin{pmatrix} a & b \\ -b & a \end{pmatrix}. \end{align*}$$

Since $\tilde {\mathbb {V}}_v = \tilde {V}_v^\dagger \otimes _{F_v} W_v^\dagger $ , etc., we have identifications

$$\begin{align*}\tilde{\mathbb{X}}_v = \tilde{V}_v^\dagger, \qquad \mathbb{X}_v = V_v^\dagger, \qquad \mathbb{X}_{0,v} = V_{0,v}^\dagger. \end{align*}$$

For any $v \in \Sigma _{\infty }$ and any nonnegative integer $\ell $ , we put $\mathrm {S}^\ell \tilde {V}_v = \mathrm {Sym}^\ell \tilde {V}_v^\dagger \otimes _{F_v} \bar {F}_v$ and denote by $\mathscr {H}^\ell \tilde {V}_v$ the kernel of the contraction $\mathrm {S}^\ell \tilde {V}_v \rightarrow \mathrm {S}^{\ell -2} \tilde {V}_v$ (see §8.2). We define $\mathrm {S}^\ell V_v$ and $\mathscr {H}^\ell V_v$ similarly.

10.4 Construction

For $v \in \Sigma _{\infty }$ , let $k_v \ge 2$ be a positive even integer and put $\ell _v = k_v-2$ . Put $\underline {\ell } = (\ell _v)_{v \in \Sigma _{\infty }}$ and

$$\begin{align*}\mathscr{H}^{\underline{\ell}} \tilde{V} = \bigotimes_{v \in \Sigma_{\infty}} \mathscr{H}^{\ell_v} \tilde{V}_v, \qquad \mathscr{H}^{\underline{\ell}} V = \bigotimes_{v \in \Sigma_{\infty}} \mathscr{H}^{\ell_v} V_v. \end{align*}$$

We consider a Schwartz form

$$\begin{align*}\tilde{\varphi} = \otimes_v \tilde{\varphi}_v \in \mathcal{S}(\tilde{\mathbb{X}}(\mathbb{A})) \otimes \wedge^{2d} \tilde{\mathfrak{p}}^* \otimes \mathscr{H}^{\underline{\ell}} \tilde{V} \end{align*}$$

such that

$$\begin{align*}\tilde{\varphi}_v = \begin{cases} \varphi^{\prime}_{2, \ell_v} & \text{if } v \in \Sigma; \\ \varphi^{\prime}_{\ell_v} & \text{if } v \in \Sigma_{\infty} \smallsetminus \Sigma \end{cases} \end{align*}$$

(see §8.4; note that $\bigotimes _{v \in \Sigma } \wedge ^2 \tilde {\mathfrak {p}}_v^* \subset \wedge ^{2d} \tilde {\mathfrak {p}}^*$ ). Then we have a theta form

$$\begin{align*}\Theta_{\tilde{\varphi}}(\tilde{g},h) = \sum_{x \in \tilde{\mathbb{X}}(F)} (\omega(\tilde{g},h) \otimes 1 \otimes 1) \tilde{\varphi}(x), \end{align*}$$

on $\mathrm {G}(\mathrm {U}(\tilde {V}) \times \mathrm {U}(W))^0(\mathbb {A})$ , where we regard $\tilde {\varphi }$ as a $\wedge ^{2d} \tilde {\mathfrak {p}}^* \otimes \mathscr {H}^{\underline {\ell }} \tilde {V}$ -valued function on $\tilde {\mathbb {X}}(\mathbb {A})$ . Let $\tau $ be an irreducible unitary automorphic representation of $H(\mathbb {A})^+$ with central character $\xi _E$ such that:

  • $\tau _v$ is the antiholomorphic discrete series representation of $\mathrm {GL}_2(\mathbb {R})^+$ of weight $-k_v-1$ if $v \in \Sigma $ ;

  • $\tau _v$ is the holomorphic discrete series representation of $\mathrm {GL}_2(\mathbb {R})^+$ of weight $k_v+1$ if $v \in \Sigma _{\infty } \smallsetminus \Sigma $ ,

where

$$ \begin{align*} H(\mathbb{A})^+ & = \{ h \in H(\mathbb{A}) \, | \, \nu(h) \in \mathrm{N}(\mathbb{A}_E^\times) \}, \\ \mathrm{GL}_2(\mathbb{R})^+ & = \{ h \in \mathrm{GL}_2(\mathbb{R}) \, | \, \det h> 0 \}. \end{align*} $$

Let $\phi = \otimes _v \phi _v \in \tau $ be a nonzero vector such that

$$\begin{align*}\tau_v(z) \phi_v = \begin{cases} z^{-k_v-1} \cdot \phi_v & \text{if } v \in \Sigma; \\ z^{k_v+1} \cdot \phi_v & \text{if } v \in \Sigma_{\infty} \smallsetminus \Sigma \end{cases} \end{align*}$$

for $z \in \mathbb {C}^1$ , where we embed $\mathbb {C}^\times $ into $\mathrm {GL}_2(\mathbb {R})$ as in §10.3. We define a theta lift

by

$$\begin{align*}\theta_{\tilde{\varphi}}(\phi)(\tilde{g}) = \int_{\mathrm{U}(W)(F) \backslash \mathrm{U}(W)(\mathbb{A})} \Theta_{\tilde{\varphi}}(\tilde{g}, h_1 h) \phi(h_1 h) \, dh_1 \end{align*}$$

for $\tilde {g} \in \tilde {G}(\mathbb {A})$ , where we choose $h \in H(\mathbb {A})^+$ such that $\nu (h) = \nu (\tilde {g})$ but the integral is independent of the choice of h. By Proposition A.1, $\theta _{\tilde {\varphi }}(\phi )$ has trivial central character.

Next, we take the image $\tilde {\Xi } := \operatorname {res}(\theta _{\tilde {\varphi }}(\phi ))$ of $\theta _{\tilde {\varphi }}(\phi )$ under the map

induced by the restriction

and the projections $\wedge ^{2d} \tilde {\mathfrak {p}}^* \rightarrow \wedge ^{2d} \mathfrak {p}^*$ and $\mathscr {H}^{\underline {\ell }} \tilde {V} \rightarrow \mathscr {H}^{\underline {\ell }} V$ (see §8.5). For any character $\eta $ of $\mathbb {A}_E^\times /E^\times $ such that $\eta |_{\mathbb {A}^\times } = 1$ , we define the $\eta $ -component

of $\tilde {\Xi }$ by

$$\begin{align*}\tilde{\Xi}_\eta(\mathbf{g}) = \int_{\tilde{Z}(\mathbb{A}) \mathbf{Z}(F) \backslash \mathbf{Z}(\mathbb{A})} \tilde{\Xi}(\mathbf{z} \mathbf{g}) \cdot (\eta \circ \mathbf{p})(\mathbf{z}) \, d \mathbf{z}, \end{align*}$$

where the Haar measure $d \mathbf {z}$ is normalized so that $\operatorname {vol}(\tilde {Z}(\mathbb {A}) \mathbf {Z}(F) \backslash \mathbf {Z}(\mathbb {A}))=1$ . Furthermore, we define its pushforward

by the first projection $\operatorname {pr}:\mathbf {G}(\mathbb {A}) \rightarrow G(\mathbb {A})$ as follows. Let $G(\mathbb {A})^+$ be the image of $\operatorname {pr}$ , that is,

$$\begin{align*}G(\mathbb{A})^+ = \{g \in G(\mathbb{A}) \, | \, \nu(g) \in \mathrm{N}(\mathbb{A}_E^\times) \}. \end{align*}$$

Note that $Z(\mathbb {A}) \subset G(\mathbb {A})^+$ and $[G(\mathbb {A}):G(F)G(\mathbb {A})^+] = [\mathbb {A}^\times : F^\times \mathrm {N}(\mathbb {A}_E^\times )] = 2$ . For $g \in G(\mathbb {A})^+$ , choose $\alpha _g \in \mathbb {A}_E^\times $ such that $\nu (g) = \mathrm {N}(\alpha _g)$ and put

$$\begin{align*}\operatorname{pr}_*(\tilde{\Xi}_\eta)(g) = \tilde{\Xi}_\eta(g, \alpha_g) \cdot \eta(\alpha_g), \end{align*}$$

which is independent of the choice of $\alpha _g$ . Then we extend $\operatorname {pr}_*(\tilde {\Xi }_\eta )$ to a $\wedge ^{2d} \mathfrak {p}^* \otimes \mathscr {H}^{\underline {\ell }} V$ -valued automorphic form on $G(\mathbb {A})$ by the natural embedding

$$\begin{align*}G(F)^+ \backslash G(\mathbb{A})^+ \hookrightarrow G(F) \backslash G(\mathbb{A}) \end{align*}$$

and extension by zero, where $G(F)^+ = G(F) \cap G(\mathbb {A})^+$ . Note that $\operatorname {pr}_*(\tilde {\Xi }_\eta )$ has trivial central character.

Finally, for any open compact subgroup $\mathcal {K}$ of $Z(\mathbb {A}_f) \backslash G(\mathbb {A}_f)$ , we define the $\mathcal {K}$ -invariant projection

of $\Xi := \operatorname {pr}_*(\tilde {\Xi }_\eta )$ by

$$\begin{align*}\Xi_{\mathcal{K}}(g) = \int_{\mathcal{K}} \Xi(gk) \, dk, \end{align*}$$

where the Haar measure $dk$ is normalized so that $\operatorname {vol}(\mathcal {K})=1$ .

10.5 Nonvanishing

Let $\pi $ be an irreducible unitary cuspidal automorphic representation of $\mathrm {GL}_2(\mathbb {A})$ with trivial central character such that:

  • $\pi _v$ is the discrete series representation of $\mathrm {GL}_2(\mathbb {R})$ of even weight $k_v$ if $v \in \Sigma _{\infty }$ .

We assume that $\pi $ has the Jacquet–Langlands transfers $\pi _{B_1}$ and $\pi _{B_2}$ to $B_1^\times (\mathbb {A})$ and $B_2^\times (\mathbb {A})$ , respectively. We regard $\pi _{B_1} \boxtimes \pi _{B_2}$ as an irreducible unitary automorphic representation of $G(\mathbb {A})$ with trivial central character.

For $\epsilon = (\epsilon _v)_{v \in \Sigma _{\infty }}$ with $\epsilon _v = \pm $ , let

$$\begin{align*}f_1^\epsilon = \Big( \bigotimes_{v \in \Sigma_{\infty}} f_{1,v}^{\epsilon_v} \Big) \otimes \Big( \bigotimes_{v \notin \Sigma_{\infty}} f_{1,v} \Big) \in \pi_{B_1}, \qquad f_2^\epsilon = \Big( \bigotimes_{v \in \Sigma_{\infty}} f_{2,v}^{\epsilon_v} \Big) \otimes \Big( \bigotimes_{v \notin \Sigma_{\infty}} f_{2,v} \Big) \in \pi_{B_2} \end{align*}$$

be nonzero vectors such that:

  • if $v \in \Sigma $ , then

    (10.2) $$ \begin{align} \pi_{B_1,v}(z) f_{1,v}^{\epsilon_v} = z^{\epsilon_v k_v} \cdot f_{1,v}^{\epsilon_v}, \qquad \pi_{B_2,v}(z) f_{2,v}^{\epsilon_v} = z^{-\epsilon_v k_v} \cdot f_{2,v}^{\epsilon_v} \end{align} $$
    for $z \in \mathbb {C}^1$ (such $f_{i,v}^{\epsilon _v}$ is unique up to scalars);
  • if $v \in \Sigma _{\infty } \smallsetminus \Sigma $ , then

    (10.3) $$ \begin{align} \pi_{B_1,v}(z) f_{1,v}^{\epsilon_v} = z^{\epsilon_v(k_v-2)} \cdot f_{1,v}^{\epsilon_v}, \qquad \pi_{B_2,v}(z) f_{2,v}^{\epsilon_v} = z^{-\epsilon_v(k_v-2)} \cdot f_{2,v}^{\epsilon_v} \end{align} $$
    for $z \in \mathbb {C}^1$ (such $f_{i,v}^{\epsilon _v}$ is unique up to scalars);
  • if $v \notin \Sigma _{\infty }$ , then $f_{i,v}$ does not depend on $\epsilon $ .

Here, for $v \in \Sigma _{\infty }$ , we embed $\mathbb {C}^\times $ into $B_{i,v}^\times $ via the isomorphism $\mathbb {C} \simeq E_v$ as in §10.3 and the fixed embedding $E \hookrightarrow B_i$ . We regard $f^\epsilon := f_1^\epsilon \boxtimes f_2^\epsilon $ as an automorphic form on $G(\mathbb {A})$ with trivial central character. Put

with

$$\begin{align*}\boldsymbol{\omega}^\epsilon = \bigotimes_{v \in \Sigma} \boldsymbol{\omega}_v^{\epsilon_v}, \qquad \boldsymbol{v}^\epsilon = \bigotimes_{v \in \Sigma_{\infty}} \boldsymbol{v}_v^{\epsilon_v}, \end{align*}$$

where $\underline {\ell } = (\ell _v)_{v \in \Sigma _{\infty }}$ with $\ell _v = k_v-2$ , and $\boldsymbol {\omega }_v^{\epsilon _v} \in \wedge ^2 \mathfrak {p}_v^*$ and $\boldsymbol {v}_v^{\epsilon _v} \in \mathscr {H}^{\ell _v} V_v$ are as in §8.5.

Finally, let $(\cdot , \cdot )$ be the nondegenerate bilinear pairing on $\wedge ^{2d} \mathfrak {p}^* \otimes \mathscr {H}^{\underline {\ell }} V$ induced by

  • the bilinear pairing $\cdot \wedge \cdot : \wedge ^2 \mathfrak {p}_v^* \times \wedge ^2 \mathfrak {p}_v^* \rightarrow \wedge ^4 \mathfrak {p}_v^* \simeq \mathbb {C}$ as in §8.5;

  • the bilinear pairing $\langle \cdot , \cdot \rangle : \mathrm {S}^{\ell _v} V_v \times \mathrm {S}^{\ell _v} V_v \rightarrow \mathbb {C}$ as in §8.2.

Proposition 10.1. Suppose that $f_1^\epsilon $ and $f_2^\epsilon $ as above are given. Let $\mathcal {K} = \prod _v \mathcal {K}_v$ be an open compact subgroup of $Z(\mathbb {A}_f) \backslash G(\mathbb {A}_f)$ such that $f_{1,v} \boxtimes f_{2,v}$ is $\mathcal {K}_v$ -fixed for all $v \notin \Sigma _{\infty }$ . Assume further that there exists a finite place $v_0$ of F such that

  1. (i) $E_{v_0}/F_{v_0}$ is ramified;

  2. (ii) $B_{1,v_0}$ and $B_{2,v_0}$ are split;

  3. (iii) $\mathcal {K}_{v_0}$ is a hyperspecial maximal compact subgroup of $Z_{v_0} \backslash G_{v_0}$ .

Then there exist $\tilde {\varphi }, \tau , \phi , \eta $ as in §10.4 such that

$$\begin{align*}(\Xi_{\mathcal{K}}, \boldsymbol{f}^\epsilon) := \int_{Z(\mathbb{A}) G(F) \backslash G(\mathbb{A})} (\Xi_{\mathcal{K}}(g), \boldsymbol{f}^\epsilon(g)) \, dg \ne 0 \end{align*}$$

for all $\epsilon $ , where $\Xi = \operatorname {pr}_*(\tilde {\Xi }_\eta )$ with $\tilde {\Xi } = \operatorname {res}(\theta _{\tilde {\varphi }}(\phi ))$ and $\boldsymbol {f}^\epsilon = f^\epsilon \otimes \boldsymbol {\omega }^\epsilon \otimes \boldsymbol {v}^\epsilon $ with $f^\epsilon = f_1^\epsilon \boxtimes f_2^\epsilon $ .

The rest of this section is devoted to the proof of Proposition 10.1.

10.6 Reduction to triple product integrals

Let $\tilde {\varphi } = \otimes _v \tilde {\varphi }_v$ be a Schwartz form as in §10.4. For any finite place v of F, we assume that $\tilde {\varphi }_v \in \mathcal {S}(\tilde {\mathbb {X}}_v)$ is a Schwartz function of the form

$$\begin{align*}\tilde{\varphi}_v = \varphi_v \otimes \varphi_{0,v} \end{align*}$$

for some $\varphi _v \in \mathcal {S}(\mathbb {X}_v)$ and $\varphi _{0,v} \in \mathcal {S}(\mathbb {X}_{0,v})$ . For any real place v of F, we define Schwartz functions $\varphi _v^{\epsilon _v} \in \mathcal {S}(\mathbb {X}_v)$ and $\varphi _{0,v} \in \mathcal {S}(\mathbb {X}_{0,v})$ by

(10.4) $$ \begin{align} \varphi_v^{\epsilon_v}(x_1,x_2,x_5,x_6) & = \begin{cases} (x_1 + \epsilon_v i x_2)^{k_v} \cdot e^{-\pi(x_1^2 + x_2^2 + x_5^2 + x_6^2)} & \text{if } v \in \Sigma; \\ (x_1 + \epsilon_v i x_2)^{k_v-2} \cdot e^{-\pi(x_1^2 + x_2^2 + x_5^2 + x_6^2)} & \text{if } v \in \Sigma_{\infty} \smallsetminus \Sigma, \end{cases} \end{align} $$
(10.5) $$ \begin{align} \varphi_{0,v}(x_3,x_4) & = e^{-\pi(x_3^2+x_4^2)}, \end{align} $$

where $x_1, \dots , x_6$ are the coordinates on $\tilde {\mathbb {X}}_v = \tilde {V}_v^\dagger $ as in §8.5. Put

$$\begin{align*}\varphi^\epsilon = \Big( \bigotimes_{v \in \Sigma_{\infty}} \varphi_v^{\epsilon_v} \Big) \otimes \Big( \bigotimes_{v \notin \Sigma_{\infty}} \varphi_v \Big) \in \mathcal{S}(\mathbb{X}(\mathbb{A})), \qquad \varphi_0 = \bigotimes_v \varphi_{0,v} \in \mathcal{S}(\mathbb{X}_0(\mathbb{A})) \end{align*}$$

so that $\varphi ^\epsilon \otimes \varphi _0 \in \mathcal {S}(\tilde {\mathbb {X}}(\mathbb {A}))$ .

Lemma 10.2. We have

(10.6) $$ \begin{align} (\Xi_{\mathcal{K}}, \boldsymbol{f}^\epsilon) = \int_{\tilde{Z}(\mathbb{A}) \mathbf{G}(F) \backslash \mathbf{G}(\mathbb{A})} \theta_{\varphi^\epsilon \otimes \varphi_0}(\phi)(\mathbf{g}) \cdot (f^\epsilon \boxtimes \eta)(\mathbf{g}) \, d\mathbf{g}, \end{align} $$

where $\theta _{\varphi ^\epsilon \otimes \varphi _0}(\phi )$ is the theta lift as defined in §9.4 and $f^\epsilon \boxtimes \eta $ is regarded as an automorphic form on $\mathbf {G}(\mathbb {A})$ .

Proof. If $v \in \Sigma $ , then by Proposition 8.1, we have

$$\begin{align*}\mathrm{C}_{v, \boldsymbol{\omega}_v^{\epsilon_v}, \boldsymbol{v}_v^{\epsilon_v}}(\operatorname{Res}_v(\tilde{\varphi}_v)) = \varphi_v^{\epsilon_v} \otimes \varphi_{0,v}, \end{align*}$$

where

$$ \begin{align*} \operatorname{Res}_v & : S(\tilde{\mathbb{X}}_v) \otimes \wedge^2 \tilde{\mathfrak{p}}_v^* \otimes \mathscr{H}^\ell \tilde{V}_v \longrightarrow S(\tilde{\mathbb{X}}_v) \otimes \wedge^2 \mathfrak{p}^*_v \otimes \mathscr{H}^\ell V_v, \\ \mathrm{C}_{v, \boldsymbol{\omega}_v^{\epsilon_v}, \boldsymbol{v}_v^{\epsilon_v}} & : S(\tilde{\mathbb{X}}_v) \otimes \wedge^2 \mathfrak{p}_v^* \otimes \mathrm{S}^\ell V_v \longrightarrow S(\tilde{\mathbb{X}}_v) \end{align*} $$

are the restriction and the contraction as in §8.5.1. Also, if $v \in \Sigma _{\infty } \smallsetminus \Sigma $ , then by Proposition 8.2, we have

$$\begin{align*}\mathrm{C}_{v, \boldsymbol{v}_v^{\epsilon_v}}(\operatorname{Res}_v(\tilde{\varphi}_v)) = \varphi_v^{\epsilon_v} \otimes \varphi_{0,v}, \end{align*}$$

where

$$ \begin{align*} \operatorname{Res}_v: S(\tilde{\mathbb{X}}_v) \otimes \mathscr{H}^\ell \tilde{V}_v & \longrightarrow S(\tilde{\mathbb{X}}_v) \otimes \mathscr{H}^\ell V_v, \\ \mathrm{C}_{v, \boldsymbol{v}_v^{\epsilon_v}} : S(\tilde{\mathbb{X}}_v) \otimes \mathrm{S}^\ell V_v & \longrightarrow S(\tilde{\mathbb{X}}_v) \end{align*} $$

are the restriction and the contraction as in §8.5.2. This implies that

$$\begin{align*}(\operatorname{Res}(\Theta_{\tilde{\varphi}}(\tilde{g},h)), \boldsymbol{\omega}^\epsilon \otimes \boldsymbol{v}^\epsilon) = \Theta_{\varphi^\epsilon \otimes \varphi_0}(\tilde{g},h), \end{align*}$$

where

$$\begin{align*}\operatorname{Res} : \wedge^{2d} \tilde{\mathfrak{p}}^* \otimes \mathscr{H}^{\underline{\ell}} \tilde{V} \longrightarrow \wedge^{2d} \mathfrak{p}^* \otimes \mathscr{H}^{\underline{\ell}} V \end{align*}$$

is the projection. Hence, we have

$$\begin{align*}(\operatorname{Res}(\theta_{\tilde{\varphi}}(\phi)(\tilde{g})), \boldsymbol{\omega}^\epsilon \otimes \boldsymbol{v}^\epsilon) = \theta_{\varphi^\epsilon \otimes \varphi_0}(\phi)(\tilde{g}) \end{align*}$$

so that the right-hand side of equation (10.6) is equal to

$$\begin{align*}\int_{\tilde{Z}(\mathbb{A}) \mathbf{G}(F) \backslash \mathbf{G}(\mathbb{A})} (\tilde{\Xi}(\mathbf{g}), \boldsymbol{\omega}^\epsilon \otimes \boldsymbol{v}^\epsilon) \cdot (f^\epsilon \boxtimes \eta)(\mathbf{g}) \, d\mathbf{g}. \end{align*}$$

This integral is equal to

$$ \begin{align*} &\hspace{-12pt} \int_{\mathbf{Z}(\mathbb{A}) \mathbf{G}(F) \backslash \mathbf{G}(\mathbb{A})} \int_{\tilde{Z}(\mathbb{A}) \mathbf{Z}(F) \backslash \mathbf{Z}(\mathbb{A})} (\tilde{\Xi}(\mathbf{z} \mathbf{g}), \boldsymbol{\omega}^\epsilon \otimes \boldsymbol{v}^\epsilon) \cdot (f^\epsilon \boxtimes \eta)(\mathbf{z} \mathbf{g}) \, d \mathbf{z} \, d\mathbf{g} \\ & = \int_{\mathbf{Z}(\mathbb{A}) \mathbf{G}(F) \backslash \mathbf{G}(\mathbb{A})} (\tilde{\Xi}_\eta(\mathbf{g}), \boldsymbol{\omega}^\epsilon \otimes \boldsymbol{v}^\epsilon) \cdot (f^\epsilon \boxtimes \eta)(\mathbf{g}) \, d\mathbf{g} \\ & = \int_{Z(\mathbb{A}) G(F)^+ \backslash G(\mathbb{A})^+} (\operatorname{pr}_*(\tilde{\Xi}_\eta)(g), \boldsymbol{\omega}^\epsilon \otimes \boldsymbol{v}^\epsilon) \cdot f^\epsilon(g) \, dg \\ & = \int_{Z(\mathbb{A}) G(F) \backslash G(\mathbb{A})} (\operatorname{pr}_*(\tilde{\Xi}_\eta)(g), \boldsymbol{\omega}^\epsilon \otimes \boldsymbol{v}^\epsilon) \cdot f^\epsilon(g) \, dg \\ & = \int_{Z(\mathbb{A}) G(F) \backslash G(\mathbb{A})} (\Xi(g), \boldsymbol{f}^\epsilon(g)) \, dg \\ & = \int_{Z(\mathbb{A}) G(F) \backslash G(\mathbb{A})} (\Xi_{\mathcal{K}}(g), \boldsymbol{f}^\epsilon(g)) \, dg, \end{align*} $$

noting that $\operatorname {pr}_*(\tilde {\Xi }_\eta )$ is supported in $G(F) G(\mathbb {A})^+$ and $\boldsymbol {f}^\epsilon $ is $\mathcal {K}$ -fixed.

We now consider the seesaw diagram

Then the seesaw identity (combined with Lemma 10.2) says that

(10.7) $$ \begin{align} \begin{aligned} (\Xi_{\mathcal{K}}, \boldsymbol{f}^\epsilon) & = \int_{\tilde{Z}(\mathbb{A}) \mathbf{G}(F) \backslash \mathbf{G}(\mathbb{A})} \theta_{\varphi^\epsilon \otimes \varphi_0}(\phi)(\mathbf{g}) \cdot (f^\epsilon \boxtimes \eta)(\mathbf{g}) \, d\mathbf{g} \\ & = \int_{Z_H(\mathbb{A}) H(F)^+ \backslash H(\mathbb{A})^+} \theta_{\varphi^\epsilon}(f^\epsilon)(h) \cdot \theta_{\varphi_0}(\eta)(h) \cdot \phi(h) \, dh, \end{aligned} \end{align} $$

where $H(F)^+ = H(F) \cap H(\mathbb {A})^+$ , and $\theta _{\varphi ^\epsilon }(f^\epsilon )$ and $\theta _{\varphi _0}(\eta )$ are the theta lifts as defined in §9.3 and §9.2, respectively. Hence, to prove Proposition 10.1, it suffices to find $\varphi ^\epsilon , \varphi _0, \eta , \tau , \phi $ such that the right-hand side of equation (10.7) is nonzero for all $\epsilon $ .

10.7 Choosing $\varphi ^\epsilon $

Let $\pi _B$ be the Jacquet–Langlands transfer of $\pi $ to $B^\times (\mathbb {A})$ (which exists since $\pi _{B_1}$ and $\pi _{B_2}$ exist by assumption and $B_1 \cdot B_2 = B$ in the Brauer group). Note that:

  • $\pi _B$ has trivial central character;

  • $\pi _{B,v}$ is the discrete series representation of $\mathrm {GL}_2(\mathbb {R})$ of weight $k_v$ if $v \in \Sigma _{\infty }$ .

By Lemma 9.2, we have a nonzero equivariant map

$$\begin{align*}\theta: \mathcal{S}(\mathbb{X}(\mathbb{A})) \otimes (\pi_{B_1} \boxtimes \pi_{B_2}) \longrightarrow \pi_B \end{align*}$$

given by $\varphi \otimes f \mapsto \theta _\varphi (f)$ .

Lemma 10.3. Let $\varphi ^\epsilon = \left( \bigotimes _{v \in \Sigma _{\infty }} \varphi _v^{\epsilon _v} \right) \otimes \left( \bigotimes _{v \notin \Sigma _{\infty }} \varphi _v \right) \in \mathcal {S}(\mathbb {X}(\mathbb {A}))$ be a Schwartz function such that $\varphi _v^{\epsilon _v}$ is as in equation (10.4) for all $v \in \Sigma _{\infty }$ . Then

(10.8) $$ \begin{align} \pi_{B,v}(z) \theta_{\varphi^\epsilon}(f^\epsilon) = \begin{cases} z^{k_v} \cdot \theta_{\varphi^\epsilon}(f^\epsilon) & \text{if } v \in \Sigma; \\ z^{-k_v} \cdot \theta_{\varphi^\epsilon}(f^\epsilon) & \text{if } v \in \Sigma_{\infty} \smallsetminus \Sigma \end{cases} \end{align} $$

for $z \in \mathbb {C}^1$ . Moreover, $\theta _{\varphi ^\epsilon }(f^\epsilon )$ is nonzero for some such $\varphi ^\epsilon $ .

Proof. We have

(10.9) $$ \begin{align} \omega_v(t, z) \varphi_v^{\epsilon_v} = \begin{cases} t_1^{\epsilon_v k_v} \cdot z^{k_v} \cdot \varphi_v^{\epsilon_v} & \text{if } v \in \Sigma; \\ t_1^{\epsilon_v (k_v-2)} \cdot z^{-k_v} \cdot \varphi_v^{\epsilon_v} & \text{if } v \in \Sigma_{\infty} \smallsetminus \Sigma \end{cases} \end{align} $$

for $t = (t_1, t_2) \in \mathrm {U}(V_v)^0$ with $t_i \in \mathrm {SO}(2) \simeq \mathbb {C}^1$ and $z \in \mathrm {U}(W_v)$ with $z \in \mathbb {C}^1$ . This proves equation (10.8).

By the Howe duality for $(\mathrm {GU}(V_v)^0, \mathrm {GU}(W_v)^+)$ (see the proof of Lemma 9.3), we have a decomposition

$$\begin{align*}\theta = \bigotimes_v \theta_v, \end{align*}$$

where

$$\begin{align*}\theta_v :\mathcal{S}(\mathbb{X}_v) \otimes (\pi_{B_1, v} \boxtimes \pi_{B_2, v}) \longrightarrow \pi_{B,v} \end{align*}$$

is the unique (up to scalars) nonzero $\mathrm {G}(\mathrm {U}(V_v) \times \mathrm {U}(W_v))^0$ -equivariant map. Since $\pi _{B_1, v} \boxtimes \pi _{B_2, v}$ is irreducible, we may choose $\varphi _v$ so that $\theta _v(\varphi _v \otimes (f_{1,v} \boxtimes f_{2,v})) \ne 0$ for $v \notin \Sigma _{\infty }$ .

It remains to show that $\theta _v(\varphi _v^{\epsilon _v} \otimes (f_{1,v}^{\epsilon _v} \boxtimes f_{2,v}^{\epsilon _v})) \ne 0$ for $v \in \Sigma _{\infty }$ , where $\varphi _v^{\epsilon _v}$ is as in equation (10.4). Let

$$\begin{align*}{}^t\theta_v :\mathcal{S}(\mathbb{X}_v) \otimes \pi_{B,v}^\vee \longrightarrow (\pi_{B_1, v} \boxtimes \pi_{B_2, v})^\vee \end{align*}$$

be the $\mathrm {G}(\mathrm {U}(V_v) \times \mathrm {U}(W_v))^0$ -equivariant map induced by $\theta _v$ . Let $w_v \in \pi _{B,v}^\vee $ be the unique (up to scalars) nonzero vector such that

(10.10) $$ \begin{align} \pi_{B,v}^\vee(z) w_v = \begin{cases} z^{-k_v} \cdot w_v & \text{if } v \in \Sigma; \\ z^{k_v} \cdot w_v & \text{if } v \in \Sigma_{\infty} \smallsetminus \Sigma \end{cases} \end{align} $$

for $z \in \mathbb {C}^1$ . Then, by equation (10.9), ${}^t\theta _v(\varphi _v^{\epsilon _v} \otimes w_v)$ is a scalar multiple of the unique (up to scalars) nonzero vector $\mathfrak {F}_v^{\epsilon _v} \in (\pi _{B_1, v} \boxtimes \pi _{B_2, v})^\vee $ such that

(10.11) $$ \begin{align} (\pi_{B_1, v} \boxtimes \pi_{B_2, v})^\vee(t) \mathfrak{F}_v^{\epsilon_v} = \begin{cases} t_1^{\epsilon_v k_v} \cdot \mathfrak{F}_v^{\epsilon_v} & \text{if } v \in \Sigma; \\ t_1^{\epsilon_v (k_v-2)} \cdot \mathfrak{F}_v^{\epsilon_v} & \text{if } v \in \Sigma_{\infty} \smallsetminus \Sigma \end{cases} \end{align} $$

for $t = (t_1, t_2) \in \mathrm {U}(V_v)^0$ with $t_i \in \mathrm {SO}(2) \simeq \mathbb {C}^1$ . Since

$$\begin{align*}\langle \theta_v(\varphi_v^{\epsilon_v} \otimes (f_{1,v}^{\epsilon_v} \boxtimes f_{2,v}^{\epsilon_v})), w_v \rangle = \langle f_{1,v}^{\epsilon_v} \boxtimes f_{2,v}^{\epsilon_v}, {}^t\theta_v(\varphi_v^{\epsilon_v} \otimes w_v) \rangle \end{align*}$$

and $\langle f_{1,v}^{\epsilon _v} \boxtimes f_{2,v}^{\epsilon _v}, \mathfrak {F}_v^{\epsilon _v} \rangle \ne 0$ by equations (10.2), (10.3) and (10.11), where $\langle \cdot , \cdot \rangle $ denotes the natural pairing, it suffices to show that ${}^t\theta _v(\varphi _v^{\epsilon _v} \otimes w_v) \ne 0$ .

For this, we realize ${}^t\theta _v$ explicitly as follows. Recall that we write $J = t_v^2$ for some $t_v \in F^\times _v$ in §10.3. We define an isomorphism $\mathfrak {i}^{\prime }_v: B_{1,v} \rightarrow B_{2,v}$ of quaternion $F_v$ -algebras by

$$\begin{align*}\mathfrak{i}^{\prime}_v(a + bi + cj_1 + dij_1) = a + bi + cj_2 + dij_2, \end{align*}$$

where

$$\begin{align*}i = |u|_{F_v}^{-\frac{1}{2}} \cdot \mathbf{i}, \qquad j_1 = |J_1|_{F_v}^{-\frac{1}{2}} \cdot \mathbf{j}_1, \qquad j_2 = \zeta_v t_v^{-1} \cdot |J_1|_{F_v}^{\frac{1}{2}} \cdot \mathbf{j}_2 \end{align*}$$

with

$$\begin{align*}\zeta_v = \begin{cases} +1 & \text{if } v \in \Sigma; \\ -1 & \text{if } v \in \Sigma_{\infty} \smallsetminus \Sigma. \end{cases} \end{align*}$$

Since $\mathbf {j}_1 \mathbf {e}_2 = J_1 \mathbf {e}_1$ , $\mathbf {j}_2 \mathbf {e}_2 = \mathbf {e}_1 \mathbf {j}$ and $\mathbf {j} e_v" = -t_v e_v"$ , we have

$$\begin{align*}j_1 e_{2,v} = \zeta_v e_{4,v}, \qquad j_2 e_{2,v} = -\zeta_v e_{4,v}. \end{align*}$$

From this and equation (10.1), we deduce that

$$\begin{align*}x^* e_{2,v} = \mathfrak{i}^{\prime}_v(x) e_{2,v} \end{align*}$$

for all $x \in B_{1,v}$ , where $*$ is the main involution on $B_{1,v}$ . In particular, if we define a subgroup $\varDelta _v$ of $G_v = (B_{1,v}^\times \times B_{2,v}^\times )/F_v^\times $ by

$$\begin{align*}\varDelta_v = \{ ((x^*)^{-1}, \mathfrak{i}^{\prime}_v(x)) \, | \, x \in B_{1,v}^\times \} / F_v^\times, \end{align*}$$

then $e_{2,v}$ is $\varDelta _v$ -fixed. We now realize $\pi _{B,v}^\vee $ on the Whittaker model $\mathcal {W}(\pi _{B,v}^\vee )$ with respect to the character $\left( \begin {smallmatrix} 1 & x \\ & 1 \end {smallmatrix} \right) \mapsto e^{-2 \pi i \zeta _v x}$ and define a map

$$\begin{align*}\tilde{\mathcal{B}}_v: \mathcal{S}(\mathbb{X}_v) \otimes \mathcal{W}(\pi_{B,v}^\vee) \longrightarrow \mathbb{C} \end{align*}$$

by

$$\begin{align*}\tilde{\mathcal{B}}_v(\Phi \otimes W) = \int_{N_v \backslash \mathrm{SL}_2(F_v)} \omega_v(h) \Phi(\sqrt{2} e_{2,v}) W(h) \, dh, \end{align*}$$

where $N_v$ is the group of unipotent upper triangular matrices in $H_v = \mathrm {GL}_2(F_v)$ and the integral is absolutely convergent by [Reference Waldspurger67, Lemme 5]. For $\mathfrak {F} \in (\pi _{B_1, v} \boxtimes \pi _{B_2, v})^\vee $ , put

$$\begin{align*}\mathcal{B}_v(\mathfrak{F}) = \tilde{\mathcal{B}}_v(\tilde{\mathfrak{F}}), \end{align*}$$

where we choose $\tilde {\mathfrak {F}} \in \mathcal {S}(\mathbb {X}_v) \otimes \mathcal {W}(\pi _{B,v}^\vee )$ such that ${}^t\theta _v(\tilde {\mathfrak {F}}) = \mathfrak {F}$ . Then, by [Reference Waldspurger67, Lemme 6], this does not depend on the choice of $\tilde {\mathfrak {F}}$ and defines a $\varDelta _v$ -invariant map $\mathcal {B}_v: (\pi _{B_1, v} \boxtimes \pi _{B_2, v})^\vee \rightarrow \mathbb {C}$ so that

$$\begin{align*}\tilde{\mathcal{B}}_v = \mathcal{B}_v \circ {}^t \theta_v. \end{align*}$$

Note that the representation $x \mapsto \pi _{B_1, v}((x^*)^{-1})$ is isomorphic to $\pi _{B_1,v}^\vee $ . Thus, it suffices to show that $\tilde {\mathcal {B}}_v(\varphi _v^{\epsilon _v} \otimes w_v) \ne 0$ . By equation (10.10), we may normalize $w_v$ so that

$$\begin{align*}w_v \! \begin{pmatrix} a & \\ & a^{-1} \end{pmatrix} = a^{k_v} e^{-2 \pi a^2}. \end{align*}$$

If $v \in \Sigma $ , then

$$ \begin{align*} \tilde{\mathcal{B}}_v(\varphi_v^{\epsilon_v} \otimes w_v) & = \int_0^\infty a^2 \varphi_v^{\epsilon_v}(a \sqrt{2} e_{2,v}) \cdot w_v \! \begin{pmatrix} a & \\ & a^{-1} \end{pmatrix} \cdot a^{-2} \, d^\times a \\ & = (\epsilon_v i \sqrt{2})^{k_v} \cdot \int_0^\infty a^{2k_v} e^{-4 \pi a^2} \, d^\times a \\ & = (\epsilon_v i \sqrt{2})^{k_v} \cdot (4 \pi)^{-k_v} \cdot 2^{-1} \cdot \int_0^\infty a^{k_v} e^{-a} \, d^\times a \\ & = (\epsilon_v i \sqrt{2})^{k_v} \cdot (4 \pi)^{-k_v} \cdot 2^{-1} \cdot \Gamma(k_v), \end{align*} $$

where $d^\times a = da/a$ . Similarly, if $v \in \Sigma _{\infty } \smallsetminus \Sigma $ , then

$$\begin{align*}\tilde{\mathcal{B}}_v(\varphi_v^{\epsilon_v} \otimes w_v) = (\epsilon_v i \sqrt{2})^{k_v-2} \cdot (4 \pi)^{-k_v+1} \cdot 2^{-1} \cdot \Gamma(k_v-1). \end{align*}$$

This completes the proof.

By Lemma 10.3, we may choose $\varphi ^\epsilon $ so that $\theta _{\varphi ^\epsilon }(f^\epsilon )$ is nonzero. Moreover, by replacing $f^\epsilon $ by its scalar multiple if necessary, we may assume that $\theta _{\varphi ^\epsilon }(f^\epsilon )$ does not depend on $\epsilon $ .

Lemma 10.4. There exists an element $(g_0, h_0) \in \mathrm {G}(\mathrm {U}(V_{v_0}) \times \mathrm {U}(W_{v_0}))^0$ such that the restriction of $\theta _{\omega (g_0,h_0) \varphi ^\epsilon }(f^\epsilon )$ to $H(\mathbb {A})^+$ is nonzero.

Proof. Since $H(F) H(\mathbb {A})^+$ is the kernel of $\xi _E \circ \nu $ and $E_{v_0}$ is a ramified quadratic extension of $F_{v_0}$ , we have

$$\begin{align*}H(\mathbb{A}) = H(F) H(\mathbb{A})^+ \bigsqcup H(F) H(\mathbb{A})^+ h_0 \end{align*}$$

for some $h_0 \in H_{v_0}$ such that $\nu (h_0) \in \mathcal {O}_{F_{v_0}}^\times \smallsetminus \mathrm {N}(\mathcal {O}_{E_{v_0}}^\times )$ . Then there exists an element $g_0 \in G_{v_0}$ such that $\nu (g_0) = \nu (h_0)$ and such that the image of $g_0$ in $Z_{v_0} \backslash G_{v_0}$ belongs to $\mathcal {K}_{v_0}$ . Since $f^\epsilon $ is $\mathcal {K}_{v_0}$ -fixed, we have

$$ \begin{align*} \theta_{\varphi^\epsilon}(f^\epsilon)(h h_0) & = \int_{\mathrm{U}(V)^0(F) \backslash \mathrm{U}(V)^0(\mathbb{A})} \Theta_{\varphi^\epsilon}(g_1 g g_0, h h_0) f^\epsilon(g_1 g g_0) \, dg_1 \\ & = \int_{\mathrm{U}(V)^0(F) \backslash \mathrm{U}(V)^0(\mathbb{A})} \Theta_{\varphi^\epsilon}(g_1 g g_0, h h_0) f^\epsilon(g_1 g) \, dg_1 \\ & = \theta_{\omega(g_0,h_0) \varphi^\epsilon}(f^\epsilon)(h) \end{align*} $$

for $h \in H(\mathbb {A})^+$ , where we choose $g \in G(\mathbb {A})$ such that $\nu (g) = \nu (h)$ . Hence, $\theta _{\varphi ^\epsilon }(f^\epsilon )(h)$ or $\theta _{\omega (g_0,h_0) \varphi ^\epsilon }(f^\epsilon )(h)$ is nonzero for some $h \in H(\mathbb {A})^+$ .

By Lemma 10.4, we may assume that the restriction of $\theta _{\varphi ^\epsilon }(f^\epsilon )$ to $H(\mathbb {A})^+$ is nonzero.

10.8 Choosing $\eta $ and $\varphi _0$

We choose $\eta $ satisfying the conditions of the following lemma.

Lemma 10.5. There exists a character $\eta $ of $\mathbb {A}_E^\times /E^\times $ such that:

  • $\eta |_{\mathbb {A}^\times } = 1$ ;

  • $\eta _v = 1$ for all real places v of F;

  • $\eta _v$ does not factor through the norm map if $B_v$ is ramified.

Proof. By Hilbert 90, the map $x \mapsto x/x^\rho $ induces an isomorphism $E^\times / F^\times \simeq E^1$ . Hence, it suffices to find a character $\chi $ of $\mathbb {A}_E^1/E^1$ such that:

  • $\chi _v = 1$ for all real places v of F;

  • $\chi _v^2 \ne 1$ if $B_v$ is ramified.

Since $E_v$ is nonsplit if either v is real or $B_v$ is ramified, it remains to show the following: if S is a finite set of places of F such that $E_v$ is nonsplit for all $v \in S$ and $\chi _S$ is a character of $E_S^1 = \prod _{v \in S} E_v^1$ , then there exists a character $\chi $ of $\mathbb {A}_E^1/E^1$ such that $\chi |_{E^1_S} = \chi _S$ . But this assertion follows from the fact that $E_S^1$ is compact and hence the image of the natural continuous injective homomorphism

$$\begin{align*}E_S^1 \longrightarrow \mathbb{A}_E^1 \longrightarrow \mathbb{A}_E^1/E^1 \end{align*}$$

is closed.

Let $\pi (\eta )$ be the automorphic induction of $\eta $ to $\mathrm {GL}_2(\mathbb {A})$ and $\pi (\eta )_B$ its Jacquet–Langlands transfer to $B^\times (\mathbb {A})$ (which exists since $\eta _v$ does not factor through the norm map if $B_v$ is ramified). Note that:

  • $\pi (\eta )_B$ has central character $\xi _E$ ;

  • $\pi (\eta )_{B,v}$ is the limit of discrete series representation of $\mathrm {GL}_2(\mathbb {R})$ of weight $1$ if $v \in \Sigma _{\infty }$ .

By Lemma 9.1, we have a nonzero equivariant map

$$\begin{align*}\theta: \mathcal{S}(\mathbb{X}_0(\mathbb{A})) \longrightarrow \pi(\eta)_B \end{align*}$$

given by $\varphi _0 \mapsto \theta _{\varphi _0}(\eta )$ .

Lemma 10.6. Let $\varphi _0 = \bigotimes _v \varphi _{0,v} \in \mathcal {S}(\mathbb {X}_0(\mathbb {A}))$ be a Schwartz function such that $\varphi _{0,v}$ is as in equation (10.5) for all $v \in \Sigma _{\infty }$ . Then

(10.12) $$ \begin{align} \pi(\eta)_{B,v}(z) \theta_{\varphi_0}(\eta) = \begin{cases} z \cdot \theta_{\varphi_0}(\eta) & \text{if } v \in \Sigma; \\ z^{-1} \cdot \theta_{\varphi_0}(\eta) & \text{if } v \in \Sigma_{\infty} \smallsetminus \Sigma \end{cases} \end{align} $$

for $z \in \mathbb {C}^1$ . Moreover, $\theta _{\varphi _0}(\eta )$ is nonzero for some such $\varphi _0$ .

Proof. We have

$$\begin{align*}\omega_v(t, z) \varphi_{0,v} = \begin{cases} z \cdot \varphi_{0,v} & \text{if } v \in \Sigma; \\ z^{-1} \cdot \varphi_{0,v} & \text{if } v \in \Sigma_{\infty} \smallsetminus \Sigma \end{cases} \end{align*}$$

for $t \in \mathrm {U}(V_{0,v})^0$ and $z \in \mathrm {U}(W_v)$ with $z \in \mathbb {C}^1$ . This proves equation (10.12).

As explained in the proof of Lemma 9.1, we may regard $\theta _{\varphi _0}(\eta )$ as the theta lift of $\eta $ (regarded as an automorphic character of $\mathrm {GU}(\mathbf {V})(\mathbb {A})$ ) to $\mathrm {GU}(\mathbf {W})(\mathbb {A})$ , where $\mathbf {V}$ and $\mathbf {W}$ are the one-dimensional Hermitian E-space and the two-dimensional skew-Hermitian E-space, respectively, as in §A.4. Hence, by the Howe duality for $(\mathrm {GU}(\mathbf {V}_v), \mathrm {GU}(\mathbf {W}_v)^+)$ , we have a decomposition

$$\begin{align*}\theta = \bigotimes_v \theta_v, \end{align*}$$

where

$$\begin{align*}\theta_v : \mathcal{S}(\mathbb{X}_{0,v}) \longrightarrow \pi(\eta)_{B,v} \end{align*}$$

is the unique (up to scalars) nonzero $\mathrm {G}(\mathrm {U}(\mathbf {V}_v) \times \mathrm {U}(\mathbf {W}_v))$ -equivariant map. Here, $(g, [h,\alpha ]) \in \mathrm {G}(\mathrm {U}(\mathbf {V}_v) \times \mathrm {U}(\mathbf {W}_v))$ with $h \in B_v^\times $ and $\alpha \in E_v^\times $ acts as $\omega _v(g,[h,\alpha ]) \otimes \eta _v(g)$ on the left-hand side and as $\pi (\eta )_{B,v}(h) \otimes \eta _v(\alpha )^{-1}$ on the right-hand side. For $v \notin \Sigma _{\infty }$ , we may choose $\varphi _{0,v}$ so that $\theta _v(\varphi _{0,v}) \ne 0$ .

It remains to show that $\theta _v(\varphi _{0,v}) \ne 0$ for $v \in \Sigma _{\infty }$ , where $\varphi _{0,v}$ is as in equation (10.5). Since $\eta _v=1$ , $\theta _v$ can be realized by

$$\begin{align*}\theta_v(\Phi) = \mathcal{F}_\Phi, \qquad \mathcal{F}_\Phi(h) = \omega_v(g,h) \Phi(0) \end{align*}$$

for $\Phi \in \mathcal {S}(\mathbb {X}_{0,v})$ and $h \in H_v^+$ , where we choose $g \in G_{0,v}$ such that $\nu (g) = \nu (h)$ and regard $\pi (\eta )_{B,v}$ as a subrepresentation of some unitary principal series representation. Then, noting that $\varphi _{0,v}(0) = 1$ , we have $\theta _v(\varphi _{0,v}) \ne 0$ .

By Lemma 10.6, we may choose $\varphi _0$ so that $\theta _{\varphi _0}(\eta )$ is nonzero. Since $\theta _{\varphi _0}(\eta )$ is supported in $H(F) H(\mathbb {A})^+$ by definition, its restriction to $H(\mathbb {A})^+$ is also nonzero.

10.9 Choosing $\tau $ and $\phi $

Lemma 10.7. Let $\psi $ be the restriction of $\theta _{\varphi ^\epsilon }(f^\epsilon ) \cdot \theta _{\varphi _0}(\eta )$ to $H(\mathbb {A})^+$ . For $v \in \Sigma _{\infty }$ , let $\sigma _v$ be the representation of $\mathrm {GL}_2(\mathbb {R})^+$ generated by $\psi $ . Then

(10.13) $$ \begin{align} \sigma_v(z) \psi = \begin{cases} z^{k_v+1} \cdot \psi & \text{if } v \in \Sigma; \\ z^{-k_v-1} \cdot \psi & \text{if } v \in \Sigma_{\infty} \smallsetminus \Sigma \end{cases} \end{align} $$

for $z \in \mathbb {C}^1$ . Moreover, if $\psi $ is nonzero, then

  • $\sigma _v$ is the holomorphic discrete series representation of $\mathrm {GL}_2(\mathbb {R})^+$ of weight $k_v+1$ if $v \in \Sigma $ ;

  • $\sigma _v$ is the antiholomorphic discrete series representation of $\mathrm {GL}_2(\mathbb {R})^+$ of weight $-k_v-1$ if $v \in \Sigma _{\infty } \smallsetminus \Sigma $ .

Proof. We only consider the case $v \in \Sigma $ ; the other case is similar. Let $\psi '$ and $\psi "$ be the restrictions of $\theta _{\varphi ^\epsilon }(f^\epsilon )$ and $\theta _{\varphi _0}(\eta )$ to $H(\mathbb {A})^+$ , respectively. Since $\psi = \psi ' \cdot \psi "$ , equation (10.13) follows from equations (10.8) and (10.12).

Let $\sigma ^{\prime }_v$ and $\sigma ^{\prime \prime }_v$ be the representations of $\mathrm {GL}_2(\mathbb {R})^+$ generated by $\psi '$ and $\psi "$ , respectively. Then $\sigma ^{\prime }_v \simeq \mathrm {HDS}_{k_v}$ and $\sigma ^{\prime \prime }_v \simeq \mathrm {HDS}_1$ , where for any positive integer k, $\mathrm {HDS}_k$ denotes the (limit of) holomorphic discrete series representation of $\mathrm {GL}_2(\mathbb {R})^+$ of weight k with central character trivial on $\mathbb {R}^\times _+$ . Since $\psi = \psi ' \cdot \psi "$ , $\sigma _v$ is a subquotient of $\sigma ^{\prime }_v \otimes \sigma _v"$ . However, we have

$$\begin{align*}\mathrm{HDS}_{k_v} \otimes \mathrm{HDS}_1 \simeq \bigoplus_{i=0}^\infty \mathrm{HDS}_{k_v+1+2i} \end{align*}$$

by [Reference Repka60, Theorem 8.1]. Hence, if $\psi $ is nonzero, then equation (10.13) forces $\sigma _v \simeq \mathrm {HDS}_{k_v+1}$ . This completes the proof.

Lemma 10.8. There exists an element $(g_0, h_0) \in \mathrm {G}(\mathrm {U}(V_0) \times \mathrm {U}(W))^0(\mathbb {A}_f)$ such that the restriction of $\theta _{\varphi ^\epsilon }(f^\epsilon ) \cdot \theta _{\omega (g_0, h_0)\varphi _0}(\eta )$ to $H(\mathbb {A})^+$ is nonzero.

Proof. Let $\psi '$ and $\psi "$ be the restrictions of $\theta _{\varphi ^\epsilon }(f^\epsilon )$ and $\theta _{\varphi _0}(\eta )$ to $H(\mathbb {A})^+$ , respectively. Choose an open compact subgroup $\mathcal {K}_H^+$ of $H(\mathbb {A}_f)^+$ so that $\psi '$ and $\psi "$ are $\mathcal {K}_H^+$ -fixed. Since $Z_H(\mathbb {A}) H(F)^+ \backslash H(\mathbb {A})^+$ is compact, we have a finite decomposition

$$\begin{align*}H(\mathbb{A})^+ = \bigsqcup_i H(F)^+ H(F_{\infty})^+ h_i \mathcal{K}_H^+ \end{align*}$$

for some $h_i \in H(\mathbb {A}_f)^+$ , where $F_{\infty } = F \otimes _{\mathbb {Q}} \mathbb {R}$ . This gives rise to a natural identification

$$\begin{align*}H(F)^+ \backslash H(\mathbb{A})^+ / K_{H, \infty} \mathcal{K}_H^+ = \bigsqcup_i \Gamma_i \backslash \mathfrak{h}^n, \end{align*}$$

where $K_{H, \infty } = \prod _{v \in \Sigma _{\infty }} \mathbb {R}^\times \cdot \mathrm {SO}(2)$ is a maximal compact modulo center subgroup of $H(F_{\infty })^+$ , $\mathfrak {h}$ is the upper half plane, $n = [F:\mathbb {Q}]$ , and $\Gamma _i = H(F)^+ \cap h_i \mathcal {K}_H h_i^{-1}$ . Hence, the restrictions of $\psi '$ and $\psi "$ to $H(F_{\infty })^+ h_i$ descend to analytic functions $\Psi _i'$ and $\Psi _i"$ on $\mathfrak {h}^n$ (regarded as a real analytic manifold), respectively, satisfying some equivariance properties relative to the action of $\Gamma _i$ on $\mathfrak {h}^n$ . Since $\psi '$ and $\psi "$ are nonzero, so are $\Psi ^{\prime }_i$ and $\Psi ^{\prime \prime }_j$ for some i and j. Then the product $\Psi ^{\prime }_i \cdot \Psi ^{\prime \prime }_j$ is also nonzero. Namely, if we put $h_0 = h_i^{-1} h_j \in H(\mathbb {A}_f)^+$ , then

$$\begin{align*}\psi'(h) \cdot \psi"(h h_0) \ne 0 \end{align*}$$

for some $h \in H(F_{\infty })^+ h_i$ . Choose $g_0 \in G_0(\mathbb {A}_f)$ such that $\nu (g_0) = \nu (h_0)$ . Then

$$ \begin{align*} \theta_{\varphi_0}(\eta)(h h_0) & = \int_{\mathrm{U}(V_0)^0(F) \backslash \mathrm{U}(V_0)^0(\mathbb{A})} \Theta_{\varphi_0}(g_1 g g_0, h h_0) \eta(g_1 g g_0) \, dg_1 \\ & = \eta(g_0) \cdot \int_{\mathrm{U}(V_0)^0(F) \backslash \mathrm{U}(V_0)^0(\mathbb{A})} \Theta_{\varphi_0}(g_1 g g_0, h h_0) \eta(g_1 g) \, dg_1 \\ & = \eta(g_0) \cdot \theta_{\omega(g_0, h_0)\varphi_0}(\eta)(h) \end{align*} $$

for $h \in H(\mathbb {A})^+$ , where we choose $g \in G_0(\mathbb {A})$ such that $\nu (g) = \nu (h)$ . Hence,

$$\begin{align*}\theta_{\varphi^\epsilon}(f^\epsilon)(h) \cdot \theta_{\omega(g_0, h_0)\varphi_0}(\eta)(h) = \eta(g_0)^{-1} \cdot \psi'(h) \cdot \psi"(h h_0) \ne 0 \end{align*}$$

for some $h \in H(\mathbb {A})^+$ .

By Lemma 10.8, we may assume that the restriction of $\theta _{\varphi ^\epsilon }(f^\epsilon ) \cdot \theta _{\varphi _0}(\eta )$ to $H(\mathbb {A})^+$ is nonzero. Then, noting that $Z_H(\mathbb {A}) H(F)^+ \backslash H(\mathbb {A})^+$ is compact, we deduce from the spectral decomposition together with Lemma 10.7 that

$$\begin{align*}\int_{Z_H(\mathbb{A}) H(F)^+ \backslash H(\mathbb{A})^+} \theta_{\varphi^\epsilon}(f^\epsilon)(h) \cdot \theta_{\varphi_0}(\eta)(h) \cdot \phi(h) \, dh \ne 0 \end{align*}$$

for some nonzero vector $\phi $ in some irreducible automorphic representation $\tau $ of $H(\mathbb {A})^+$ as in §10.4. This completes the proof of Proposition 10.1.

11 Arthur packets, Galois representations and Hodge classes

11.1 Classification

Let F be a totally real number field and E a totally imaginary quadratic extension of F. Let $\mathbf {V}$ be an n-dimensional Hermitian E-space and $G = \mathrm {U}(\mathbf {V})$ the unitary group of $\mathbf {V}$ . In this section, we recall the classification of automorphic representations of $G(\mathbb {A}_F)$ , which has been established by Mok [Reference Mok52] in the quasi-split case, following Arthur’s book [Reference Arthur4], and has been extended to the general case by Kaletha–Minguez–Shin–White [Reference Kaletha, Minguez, Shin and White34].

More precisely, let $L^2_{\mathrm {disc}}(G)$ be the discrete spectrum of the unitary representation of $G(\mathbb {A}_F)$ on the Hilbert space $L^2(G(F) \backslash G(\mathbb {A}_F))$ . Then the decomposition of $L^2_{\mathrm {disc}}(G)$ into near equivalence classes is described as follows. We say that an irreducible cuspidal automorphic representation $\pi $ of $\mathrm {GL}_m(\mathbb {A}_E)$ is conjugate-self-dual if $\pi ^\rho \simeq \pi ^\vee $ , where $\pi ^\rho $ and $\pi ^\vee $ are the Galois conjugate and the contragredient of $\pi $ , respectively. In this case, exactly one of the Asai L-functions $L(s, \pi , \mathrm {As}^+)$ and $L(s, \pi , \mathrm {As}^-)$ has a pole at $s=1$ (see [Reference Gan, Gross and Prasad20, §7] for the definition of the Asai representations $\mathrm {As}^\pm $ ). For $\epsilon = \pm $ , we say that $\pi $ has sign $\epsilon $ if $L(s, \pi , \mathrm {As}^\epsilon )$ has a pole at $s=1$ . We also say that $\pi $ is conjugate-orthogonal (resp. conjugate-symplectic) if it is conjugate-self-dual with sign $+$ (resp. $-$ ). Consider a formal unordered finite direct sum

$$\begin{align*}\psi = \bigoplus_i \pi_i \boxtimes \mathrm{Sym}^{d_i-1}, \end{align*}$$

where

  • $\pi _i$ is an irreducible cuspidal automorphic representation of $\mathrm {GL}_{m_i}(\mathbb {A}_E)$ ;

  • $\mathrm {Sym}^{d_i-1}$ is the irreducible representation of $\mathrm {SL}_2(\mathbb {C})$ of dimension $d_i$ .

Then $\psi $ is called an elliptic A-parameter for G if

  • $\sum _i m_i d_i = n$ ;

  • if $d_i$ is odd, then $\pi _i$ is conjugate-self-dual with sign $(-1)^{n-1}$ ;

  • if $d_i$ is even, then $\pi _i$ is conjugate-self-dual with sign $(-1)^n$ ;

  • if $(\pi _i, d_i) = (\pi _j, d_j)$ , then $i=j$ .

We attach to $\psi $ an automorphic representation

$$\begin{align*}\pi_\psi = {\huge\mathop{\mathchoice{\raise-0.1em\hbox{\(\boxplus\)}}{\Large\raise-0.15em\hbox{\(\boxplus\)}}{\large\hbox{\(\boxtimes\)}}{\boxtimes}}}_i \left( \pi_i |\det|^{\frac{d_i-1}{2}} \boxplus \pi_i |\det|^{\frac{d_i-3}{2}} \boxplus \cdots \boxplus \pi_i |\det|^{-\frac{d_i-1}{2}} \right) \end{align*}$$

of $\mathrm {GL}_n(\mathbb {A}_E)$ , where $\boxplus $ denotes the isobaric sum. Then the result of Kaletha–Minguez–Shin–White [Reference Kaletha, Minguez, Shin and White34, Theorem ${}^{\ast }$ 1.7.1] (which is proved in [Reference Kaletha, Minguez, Shin and White34] partially and will be completed in its sequels) says that

$$\begin{align*}L^2_{\mathrm{disc}}(G) = \bigoplus_\psi L^2_\psi(G), \end{align*}$$

where $\psi $ runs over elliptic A-parameters for G and $L^2_\psi (G)$ is the near equivalence class of irreducible subrepresentations $\pi $ of $L^2_{\mathrm {disc}}(G)$ such that for almost all places v of F, the base change of $\pi _v$ to $\mathrm {GL}_n(E_v)$ is isomorphic to $\pi _{\psi ,v}$ .

We next describe this decomposition in terms of L-groups. Recall that the L-group of $G = \mathrm {U}(\mathbf {V})$ is given by

$$\begin{align*}{}^L G = \mathrm{GL}_n(\mathbb{C}) \rtimes \operatorname{Gal}(\overline{\mathbb{Q}}/F), \end{align*}$$

where $\operatorname {Gal}(\overline {\mathbb {Q}}/E)$ acts trivially on $\mathrm {GL}_n(\mathbb {C})$ and the nontrivial element in $\operatorname {Gal}(E/F)$ acts as the automorphism $\theta _n$ defined by

Put $\tilde {G} = \operatorname {Res}_{E/F} \mathrm {GL}_n$ so that its L-group is given by

$$\begin{align*}{}^L \tilde{G} = \left( \mathrm{GL}_n(\mathbb{C}) \times \mathrm{GL}_n(\mathbb{C}) \right) \rtimes \operatorname{Gal}(\overline{\mathbb{Q}}/F), \end{align*}$$

where $\operatorname {Gal}(\overline {\mathbb {Q}}/E)$ acts trivially on $\mathrm {GL}_n(\mathbb {C}) \times \mathrm {GL}_n(\mathbb {C})$ and the nontrivial element in $\operatorname {Gal}(E/F)$ acts as the automorphism $(g_1,g_2) \mapsto (g_2,g_1)$ . Then the base change L-homomorphism $\operatorname {BC}:{}^L G \rightarrow {}^L \tilde {G}$ is given by

$$\begin{align*}\operatorname{BC}(g \rtimes \sigma) = (g, \theta_n(g)) \rtimes \sigma. \end{align*}$$

Let $\psi $ be an elliptic A-parameter for G. Then the local Langlands correspondence induces an (equivalence class of) A-parameter $\tilde {\psi }_v : \mathcal {L}_{F_v} \times \mathrm {SL}_2(\mathbb {C}) \rightarrow {}^L \tilde {G}$ for any place v of F, where

$$\begin{align*}\mathcal{L}_{F_v} = \begin{cases} \text{the Weil group of } F_v & \text{if } v \text{ is real;} \\ \text{the Weil--Deligne group of } F_v & \text{if } v \text{ is finite.} \end{cases} \end{align*}$$

Moreover, by [Reference Gan, Gross and Prasad20, Theorem 8.1], there exists a unique (equivalence class of) A-parameter $\psi _v : \mathcal {L}_{F_v} \times \mathrm {SL}_2(\mathbb {C}) \rightarrow {}^L G$ such that $\tilde {\psi }_v = {\operatorname {BC}} \circ \psi _v$ . We associate to $\psi _v$ an L-parameter $\phi _{\psi _v}: \mathcal {L}_{F_v} \rightarrow {}^L G$ by

$$\begin{align*}\phi_{\psi_v}(w) = \psi_v \! \left( w, \begin{pmatrix} |w|_v^{\frac{1}{2}} & \\ & |w|_v^{-\frac{1}{2}} \end{pmatrix} \right). \end{align*}$$

Then $L^2_\psi (G)$ consists of irreducible subrepresentations $\pi $ of $L^2_{\mathrm {disc}}(G)$ such that the L-parameter of $\pi _v$ is $\phi _{\psi _v}$ for almost all v.

For our applications, we will consider elliptic A-parameters with $n=4$ of the form

(11.1) $$ \begin{align} \psi = \pi_E \boxtimes \mathrm{Sym}^1, \end{align} $$

where

  • $\pi $ is an irreducible cuspidal automorphic representation of $\mathrm {GL}_2(\mathbb {A}_F)$ with central character $\xi _E$ ;

  • $\pi _E$ is the base change of $\pi $ to $\mathrm {GL}_2(\mathbb {A}_E)$ .

We note the following:

Lemma 11.1.

  1. (i) If $\pi _E$ is cuspidal, then $\pi _E$ is conjugate-orthogonal.

  2. (ii) If $\pi _E$ is not cuspidal, then $\pi _E = \chi \boxplus \chi ^{-1}$ for some conjugate-orthogonal character $\chi $ of $\mathbb {A}_E^\times /E^\times $ such that $\chi ^2 \ne 1$ .

Proof. First, assume that $\pi _E$ is not cuspidal. Then $\pi $ is the automorphic induction of some character $\chi $ of $\mathbb {A}_E^\times /E^\times $ so that $\pi _E = \chi \boxplus \chi ^\rho $ . Since $\pi $ is cuspidal, we have $\chi ^\rho \ne \chi $ . Also, since the central character of $\pi $ is $\xi _E$ , we have $\chi |_{\mathbb {A}_F^\times } = 1$ , that is, $\chi $ is conjugate-orthogonal. Hence, the assertion follows.

Next, assume that $\pi _E$ is cuspidal. Put $H = \mathrm {GL}_2$ and $\tilde {H} = \operatorname {Res}_{E/F} \mathrm {GL}_2$ so that

$$\begin{align*}{}^L H = \mathrm{GL}_2(\mathbb{C}) \times \operatorname{Gal}(\overline{\mathbb{Q}}/F), \qquad {}^L \tilde{H} = \left( \mathrm{GL}_2(\mathbb{C}) \times \mathrm{GL}_2(\mathbb{C}) \right) \rtimes \operatorname{Gal}(\overline{\mathbb{Q}}/F). \end{align*}$$

Then the base change L-homomorphism $\operatorname {BC}:{}^L H \rightarrow {}^L \tilde {H}$ is given by

$$\begin{align*}\operatorname{BC}(h \times \sigma) = (h, h) \rtimes \sigma. \end{align*}$$

Recall that $\mathrm {As}^+$ is the representation of ${}^L \tilde {H}$ on $\mathbb {C}^2 \otimes \mathbb {C}^2$ defined by

$$ \begin{align*} \mathrm{As}^+((h_1, h_2) \rtimes 1)(x \otimes y) & = h_1 x \otimes h_2 y, \\ \mathrm{As}^+((1, 1) \rtimes \sigma)(x \otimes y) & = \begin{cases} x \otimes y & \text{if } \sigma \in \operatorname{Gal}(\overline{\mathbb{Q}}/E); \\ y \otimes x & \text{if } \sigma \notin \operatorname{Gal}(\overline{\mathbb{Q}}/E). \end{cases} \end{align*} $$

Then we have

$$\begin{align*}\mathrm{As}^+ \circ \operatorname{BC} \simeq \mathrm{Sym}^2 \oplus (\wedge^2 \otimes \xi_E) \end{align*}$$

as representations of ${}^L H$ . Since the central character of $\pi $ is $\xi _E$ , it follows that

$$\begin{align*}L(s, \pi_E, \mathrm{As}^+) = L(s, \pi, \mathrm{Sym}^2) \cdot \zeta_F(s). \end{align*}$$

This implies the assertion.

11.2 Local A-packets

Let $\psi $ be an elliptic A-parameter for $G = \mathrm {U}(\mathbf {V})$ and $L^2_\psi (G)$ the near equivalence class associated to $\psi $ . Then the result of Kaletha–Minguez–Shin–White [Reference Kaletha, Minguez, Shin and White34, Theorem ${}^{\ast }$ 1.7.1] also describes the local-global structure of $L^2_\psi (G)$ with a multiplicity formula. In particular, if $\pi $ is an irreducible summand of $L^2_\psi (G)$ , then for any place v of F, the local component $\pi _v$ is an irreducible summand of some representation in $\Pi _{\psi _v}$ . Here, $\Pi _{\psi _v}$ is the local A-packet associated to $\psi _v$ consisting of certain semisimple representations of $G_v$ of finite length.

Suppose that v is real. If $G_v$ is quasi-split and $\psi _v$ is ‘cohomological’, then it follows from the result of Arancibia–Mœglin–Renard [Reference Arancibia, Mœglin and Renard3] (the unitary group case had already been treated by Johnson [Reference Johnson32]) that $\Pi _{\psi _v}$ agrees with the packet constructed by Adams–Johnson [Reference Adams and Johnson2], which we recall below. From now on, we suppress the subscript v from the notation.

Let $\mathbf {V}$ be an n-dimensional Hermitian space over $\mathbb {C}$ of signature $(p,q)$ . Choosing a basis of $\mathbf {V}$ , we may identify the unitary group $G = \mathrm {U}(\mathbf {V})$ with

$$\begin{align*}\mathrm{U}(p,q) = \{ g \in \mathrm{GL}_n(\mathbb{C}) \, | \, {}^t \bar{g} I_{p,q} g = I_{p,q} \}, \qquad I_{p,q} = \begin{pmatrix} {\mathbf{1}}_p & \\ & - {\mathbf{1}}_q \end{pmatrix}. \end{align*}$$

We define a Cartan involution $\theta $ of G by $\theta (g) = {}^t \bar {g}^{-1}$ . Let $K \simeq \mathrm {U}(p) \times \mathrm {U}(q)$ be the maximal compact subgroup of G with respect to $\theta $ and $T \simeq \mathrm {U}(1)^n$ the maximal torus of G consisting of diagonal matrices. Let $B_{G_{\mathbb {C}}}$ be the Borel subgroup of $G_{\mathbb {C}} \simeq \mathrm {GL}_n(\mathbb {C})$ (which is not defined over $\mathbb {R}$ ) consisting of upper triangular matrices.

Let $\mathfrak {g}_0$ , $\mathfrak {k}_0$ , $\mathfrak {t}_0$ be the Lie algebras of G, K, T, respectively. We have a Cartan decomposition $\mathfrak {g}_0 = \mathfrak {k}_0 \oplus \mathfrak {p}_0$ , where $\mathfrak {p}_0$ is the $(-1)$ -eigenspace of $\theta $ . Let $\mathfrak {g}$ , $\mathfrak {k}$ , $\mathfrak {p}$ , $\mathfrak {t}$ be the complexifications of $\mathfrak {g}_0$ , $\mathfrak {k}_0$ , $\mathfrak {p}_0$ , $\mathfrak {t}_0$ , respectively. Let $\mathfrak {p}^{\pm }$ be the $(\pm i)$ -eigenspace of the complex structure on $\mathfrak {p}$ defined by

$$\begin{align*}X \longmapsto J X J^{-1}, \qquad J = \begin{pmatrix} e^{\frac{\pi i}{4}} {\mathbf{1}}_p & \\ & e^{-\frac{\pi i}{4}} {\mathbf{1}}_q \end{pmatrix}. \end{align*}$$

More explicitly, we have $\mathfrak {g} = \mathfrak {k} \oplus \mathfrak {p}^+ \oplus \mathfrak {p}^-$ with

$$ \begin{align*} \mathfrak{k} & = \left\{ \left. \begin{pmatrix} A & 0 \\ 0 & D \end{pmatrix} \, \right| \, A \in \mathrm{M}_p(\mathbb{C}), \, D \in \mathrm{M}_q(\mathbb{C}) \right\}, \\ \mathfrak{p}^+ & = \left\{ \left. \begin{pmatrix} 0 & B \\ 0 & 0 \end{pmatrix} \, \right| \, B \in \mathrm{M}_{p,q}(\mathbb{C}) \right\}, \\ \mathfrak{p}^- & = \left\{ \left. \begin{pmatrix} 0 & 0 \\ C & 0 \end{pmatrix} \, \right| \, C \in \mathrm{M}_{q,p}(\mathbb{C}) \right\}. \end{align*} $$

The packet constructed by Adams–Johnson [Reference Adams and Johnson2] consists of certain unitary representations $\pi $ such that $H^*(\mathfrak {g}, K; \pi \otimes F) \ne 0$ for some irreducible finite-dimensional representation F of G. Let $\lambda \in \mathfrak {t}^* \simeq \mathbb {C}^n$ be the highest weight of $F^*$ relative to $B_{G_{\mathbb {C}}}$ . We may write

$$\begin{align*}\lambda = (\underbrace{\lambda_1, \dots, \lambda_1}_{n_1}, \underbrace{\lambda_2, \dots, \lambda_2}_{n_2}, \dots, \underbrace{\lambda_r, \dots, \lambda_r}_{n_r}) \in \mathbb{Z}^n \end{align*}$$

with $\lambda _1 \ge \lambda _2 \ge \dots \ge \lambda _r$ . Then we consider the A-parameter $\psi : \mathcal {L}_{\mathbb {R}} \times \mathrm {SL}_2(\mathbb {C}) \rightarrow {}^LG$ whose restriction to $\mathbb {C}^\times \times \mathrm {SL}_2(\mathbb {C})$ is equal to

$$\begin{align*}\left( \chi_{\lambda_1 + \rho_1} \boxtimes \mathrm{Sym}^{n_1-1} \right) \oplus \dots \oplus \left( \chi_{\lambda_r + \rho_r} \boxtimes \mathrm{Sym}^{n_r-1} \right), \end{align*}$$

where

  • $\rho _i = \frac {1}{2}(- n_1 - \dots - n_{i-1} + n_{i+1} + \dots + n_r)$ ;

  • $\chi _\kappa $ is the character of $\mathbb {C}^\times $ defined by $\chi _\kappa (z) = (z/\bar {z})^\kappa $ ;

  • $\mathrm {Sym}^{d-1}$ is the irreducible d-dimensional representation of $\mathrm {SL}_2(\mathbb {C})$ .

This defines a parabolic subgroup Q of $G_{\mathbb {C}}$ (which is not defined over $\mathbb {R}$ ) containing $B_{G_{\mathbb {C}}}$ with Levi component L (which is defined over $\mathbb {R}$ ) such that

$$\begin{align*}\psi = \xi \circ \psi_L, \end{align*}$$

where $\xi : {}^L L \rightarrow {}^L G$ is the canonical embedding and $\psi _L :\mathcal {L}_{\mathbb {R}} \times \mathrm {SL}_2(\mathbb {C}) \rightarrow {}^L L$ is an A-parameter such that the A-packet $\Pi _{\psi _L}$ consists of a single one-dimensional representation of L. Note that

$$\begin{align*}L = G \cap \left( \mathrm{GL}_{n_1}(\mathbb{C}) \times \dots \times \mathrm{GL}_{n_r}(\mathbb{C}) \right). \end{align*}$$

Put

$$\begin{align*}S = W(L,T) \backslash W(G,T) / W_{\mathbb{R}}(G,T) \simeq \left( \mathfrak{S}_{n_1} \times \dots \times \mathfrak{S}_{n_r}\right) \backslash \mathfrak{S}_n / \left( \mathfrak{S}_p \times \mathfrak{S}_q \right), \end{align*}$$

where W and $W_{\mathbb {R}}$ denote the absolute and relative Weyl groups, respectively. As a set of representatives for S, we can take the set of $w \in \mathfrak {S}_n$ such that

  • $w^{-1}(i) < w^{-1}(j)$ for $n_1 + \dots + n_{k-1}+1 \le i < j \le n_1 + \dots + n_k$ for $1 \le k \le r$ ;

  • $w(i) < w(j)$ for $1 \le i < j \le p$ and for $p+1 \le i < j \le n$ .

For any $w \in S$ , we have a $\theta $ -stable parabolic subgroup $Q_w = w^{-1} Q w$ of $G_{\mathbb {C}}$ with Levi component $L_w = w^{-1} L w$ . Let $\mathfrak {q}_w$ be the Lie algebra of $Q_w$ . Then the Adams–Johnson packet $\Pi ^{\mathrm {AJ}}_\psi $ is given by

$$\begin{align*}\Pi^{\mathrm{AJ}}_\psi = \left\{ A_{\mathfrak{q}_w}(w^{-1} \lambda) \, | \, w \in S \right\}. \end{align*}$$

We now explicate the A-packet $\Pi _\psi $ when $G = \mathrm {U}(2,2)$ and $\psi $ is the localization of a global A-parameter as in equation (11.1). More precisely, we start with the discrete series representation of $\mathrm {GL}_2(\mathbb {R})$ of weight $k+1$ with an even integer $k \ge 2$ . Since its base change to $\mathrm {GL}_2(\mathbb {C})$ is the principal series representation $\operatorname {Ind}(\chi _{\frac {k}{2}} \otimes \chi _{-\frac {k}{2}})$ , the associated A-parameter $\psi $ is given by

$$\begin{align*}\psi = \big( \chi_{\frac{k}{2}} \boxtimes \mathrm{Sym}^1 \big) \oplus \big( \chi_{-\frac{k}{2}} \boxtimes \mathrm{Sym}^1 \big) \end{align*}$$

so that we need to take

$$\begin{align*}\lambda = \big( \tfrac{k}{2}-1, \tfrac{k}{2}-1, -\tfrac{k}{2}+1, -\tfrac{k}{2}+1 \big). \end{align*}$$

In this case, we have $S = \{ w_0, w_1, w_2 \}$ with

$$\begin{align*}w_0 = 1, \qquad w_1 = (23), \qquad w_2 = (13)(24). \end{align*}$$

Then

$$\begin{align*}\Pi_\psi = \Pi^{\mathrm{AJ}}_\psi = \{ A_{\mathfrak{q}_i}(w_i^{-1} \lambda) \, | \, 0 \le i \le 2 \}, \end{align*}$$

where $\mathfrak {q}_i = \mathfrak {q}_{w_i}$ .

Proposition 11.2. We have

$$ \begin{align*} \dim H^{p,q}(\mathfrak{g}, K; A_{\mathfrak{q}_0}(w_0^{-1} \lambda) \otimes F) & = \begin{cases} 1 & \text{if } (p,q) = (4,0), \\ 0 & \text{otherwise,} \end{cases} \\ \dim H^{p,q}(\mathfrak{g}, K; A_{\mathfrak{q}_1}(w_1^{-1} \lambda) \otimes F) & = \begin{cases} 1 & \text{if } (p,q) = (1,1), (3,3), \\ 2 & \text{if } (p,q) = (2,2), \\ 0 & \text{otherwise,} \end{cases} \\ \dim H^{p,q}(\mathfrak{g}, K; A_{\mathfrak{q}_2}(w_2^{-1} \lambda) \otimes F) & = \begin{cases} 1 & \text{if } (p,q) = (0,4), \\ 0 & \text{otherwise,} \end{cases} \end{align*} $$

where F is the irreducible finite-dimensional representation of G with highest weight $\lambda $ .

Proof. First, note that F is self-dual, that is, $F^*$ has highest weight $\lambda $ . Also, if we write $\mathfrak {q}_i = \mathfrak {l}_i \oplus \mathfrak {u}_i$ with Levi component $\mathfrak {l}_i$ and unipotent radical $\mathfrak {u}_i$ , then

$$ \begin{align*} \mathfrak{l}_0 & = \left\{ \begin{pmatrix} * & * & 0 & 0 \\ * & * & 0 & 0 \\ 0 & 0 & * & * \\ 0 & 0 & * & * \end{pmatrix} \right\}, & \mathfrak{u}_0 & = \left\{ \begin{pmatrix} 0 & 0 & * & * \\ 0 & 0 & * & * \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{pmatrix} \right\}, \\ \mathfrak{l}_1 & = \left\{ \begin{pmatrix} * & 0 & * & 0 \\ 0 & * & 0 & * \\ * & 0 & * & 0 \\ 0 & * & 0 & * \end{pmatrix} \right\}, & \mathfrak{u}_1 & = \left\{ \begin{pmatrix} 0 & * & 0 & * \\ 0 & 0 & 0 & 0 \\ 0 & * & 0 & * \\ 0 & 0 & 0 & 0 \end{pmatrix} \right\}, \\ \mathfrak{l}_2 & = \left\{ \begin{pmatrix} * & * & 0 & 0 \\ * & * & 0 & 0 \\ 0 & 0 & * & * \\ 0 & 0 & * & * \end{pmatrix} \right\}, & \mathfrak{u}_2 & = \left\{ \begin{pmatrix} 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ * & * & 0 & 0 \\ * & * & 0 & 0 \end{pmatrix} \right\}. \end{align*} $$

Hence, the assertion follows from [Reference Vogan and Zuckerman65, Proposition 6.19].

We also note the following:

Lemma 11.3. Let

$$\begin{align*}\delta = \begin{pmatrix} & {\mathbf{1}}_2 \\ {\mathbf{1}}_2 & \end{pmatrix} \in \mathrm{GU}(2,2). \end{align*}$$

Then we have

$$ \begin{align*} A_{\mathfrak{q}_0}(w_0^{-1} \lambda) \circ \operatorname{Ad}(\delta) & = A_{\mathfrak{q}_2}(w_2^{-1} \lambda), \\ A_{\mathfrak{q}_1}(w_1^{-1} \lambda) \circ \operatorname{Ad}(\delta) & = A_{\mathfrak{q}_1}(w_1^{-1} \lambda). \end{align*} $$

Proof. The lemma immediately follows from the characterization of the cohomological representation as described in §7.1.

11.3 The Hodge structure

Suppose again that F is a totally real number field and E is a totally imaginary quadratic extension of F. Let $\mathbf {V}$ be a four-dimensional Hermitian E-space. We now change notation and write $G = \mathrm {GU}(\mathbf {V})$ and $G' = \mathrm {U}(\mathbf {V})$ for the unitary similitude and unitary groups of $\mathbf {V}$ , respectively. We assume that $\mathrm {disc} \mathbf {V} \notin \mathrm {N}(E^\times )$ but $\mathrm {disc} \mathbf {V}_v \in \mathrm {N}(E_v^\times )$ for all $v \in \Sigma _{\infty }$ . Then we may identify $G_v$ with the group

$$\begin{align*}\{ g \in \mathrm{GL}_4(\mathbb{C}) \, | \, {}^t \bar{g} I_v g = \nu(g) \cdot I_v \}, \end{align*}$$

where

$$\begin{align*}I_v = \begin{cases} \begin{pmatrix} {\mathbf{1}}_2 & \\ & -{\mathbf{1}}_2 \end{pmatrix} & \text{if } v \in \Sigma; \\ {\mathbf{1}}_4 & \text{if } v \in \Sigma_{\infty} \smallsetminus \Sigma \end{cases} \end{align*}$$

for some subset $\Sigma $ of $\Sigma _{\infty }$ . We further assume that $\Sigma \ne \Sigma _{\infty }$ so that $G'$ is anisotropic. For $v \in \Sigma _{\infty }$ , we define a maximal compact subgroup $K_v'$ of $G_v'$ by

$$\begin{align*}K_v' = \left\{ \left. \begin{pmatrix} a & \\ & d \end{pmatrix} \, \right| \, a, d \in \mathrm{U}(2) \right\} \end{align*}$$

if $v \in \Sigma $ and $K_v' = G_v'$ if $v \in \Sigma _{\infty } \smallsetminus \Sigma $ . Put $K_v = F_v^\times \cdot K_v'$ , where we regard $F_v^\times \simeq \mathbb {R}^\times $ as a subgroup of $G_v$ via the map $z \mapsto z {\mathbf {1}}_4$ . Then $K_v$ is a maximal connected compact modulo center subgroup of $G_v$ . Put

$$ \begin{align*} G_{\infty} & = \prod_{v \in \Sigma_{\infty}} G_v, & K_{\infty} & = \prod_{v \in \Sigma_{\infty}} K_v, \\ G_{\infty}' & = \prod_{v \in \Sigma_{\infty}} G^{\prime}_v, & K_{\infty}' & = \prod_{v \in \Sigma_{\infty}} K_v'. \end{align*} $$

Let $\mathfrak {g}$ and $\mathfrak {g}'$ be the complexified Lie algebras of $G_{\infty }$ and $G_{\infty }'$ , respectively. Let $\mathbb {S} = \operatorname {Res}_{\mathbb {C}/\mathbb {R}} \mathbb {G}_m$ and $G_0 = \operatorname {Res}_{F/\mathbb {Q}} G$ . We define a homomorphism $h: \mathbb {S} \rightarrow G_{0,\mathbb {R}}$ by $h(z) = (h_v(z))_{v \in \Sigma _{\infty }}$ with

$$\begin{align*}h_v(z) = \begin{cases} \begin{pmatrix} z {\mathbf{1}}_2 & \\ & \bar{z} {\mathbf{1}}_2 \end{pmatrix} & \text{if } v \in \Sigma; \\ {\mathbf{1}}_4 & \text{if } v \in \Sigma_{\infty} \smallsetminus \Sigma. \end{cases} \end{align*}$$

Let X be the $G_0(\mathbb {R})$ -conjugacy class of homomorphisms $\mathbb {S} \rightarrow G_{0,\mathbb {R}}$ containing h. Then we have an identification

$$\begin{align*}X = G_{\infty} / K_{\infty}. \end{align*}$$

For any $v \in \Sigma _{\infty }$ and any even integer $k \ge 2$ , let $(\rho _{v,k}, V_{v,k})$ be the irreducible algebraic representation of $G_v$ such that

  • $\rho _{v,k}$ has trivial central character;

  • $\rho _{v,k}' = \rho _{v,k}|_{G_v'}$ is the irreducible finite-dimensional representation of $G_v'$ with highest weight

    $$\begin{align*}\lambda_v = \big( \tfrac{k}{2}-1, \tfrac{k}{2}-1, -\tfrac{k}{2}+1, -\tfrac{k}{2}+1 \big). \end{align*}$$

Let $\underline {k} = (k_v)_{v \in \Sigma _{\infty }}$ be a tuple of even integers $k_v \ge 2$ , and put

$$\begin{align*}\rho_{\underline{k}} = \bigotimes_{v \in \Sigma_{\infty}} \rho_{v,k_v}, \qquad V_{\underline{k}} = \bigotimes_{v \in \Sigma_{\infty}} V_{v, k_v}. \end{align*}$$

For any open compact subgroup $\mathcal {K}$ of $G(\mathbb {A}_{F,f})$ (where $\mathbb {A}_{F,f}$ denotes the ring of finite adèles of F), let $\mathrm {Sh}_{\mathcal {K}}$ be the Shimura variety associated to $(G_0, X, \mathcal {K})$ :

$$\begin{align*}\mathrm{Sh}_{\mathcal{K}} = G(F) \backslash X \times G(\mathbb{A}_{F,f}) / \mathcal{K}. \end{align*}$$

Then $(\rho _{\underline {k}}, V_{\underline {k}})$ gives rise to a local system $\mathbb {V}_{\underline {k}}$ on $\mathrm {Sh}_{\mathcal {K}}$ . We have the Hodge decomposition

$$\begin{align*}H^i(\mathrm{Sh}_{\mathcal{K}}, \mathbb{V}_{\underline{k}}) = \bigoplus_{p+q=i} H^{p,q}(\mathrm{Sh}_{\mathcal{K}}, \mathbb{V}_{\underline{k}}). \end{align*}$$

In §5.2, we have associated to $\mathbf {V}$ a quaternion F-algebra B and a three-dimensional skew-Hermitian right B-space $\tilde {V}$ such that $\operatorname {PGU}_E(\mathbf {V}) \simeq \operatorname {PGU}_B(\tilde {V})^0$ . By the above assumption on $\mathbf {V}$ , B is division but $B_v$ is split for all $v \in \Sigma _{\infty }$ . Let W be the one-dimensional Hermitian left B-space as in §9.1. Then $\mathrm {GU}(W) \simeq B^\times $ . Let $\tau $ be an irreducible unitary automorphic representation of $\mathrm {GU}(W)(\mathbb {A}_F)^+$ with central character $\xi _E$ such that:

  • $\tau _v$ is the antiholomorphic discrete series representation of $\mathrm {GL}_2(\mathbb {R})^+$ of weight $-k_v-1$ if $v \in \Sigma $ ;

  • $\tau _v$ is the holomorphic discrete series representation of $\mathrm {GL}_2(\mathbb {R})^+$ of weight $k_v+1$ if $v \in \Sigma _{\infty } \smallsetminus \Sigma $ .

Let $\Pi = \theta (\tau )$ be the global theta lift of $\tau $ to $\mathrm {GU}(\tilde {V})(\mathbb {A}_F)$ relative to the standard additive character $\psi $ of $\mathbb {A}_F/F$ (i.e., the additive character $\psi $ such that $\psi _v(x) = e^{2 \pi i x}$ for $v \in \Sigma _{\infty }$ ). We assume that $\Pi $ is nonzero. Then:

  • $\Pi $ is irreducible by Lemma 9.3;

  • $\Pi $ has trivial central character by Proposition A.1.

Hence, we may regard $\Pi $ as an irreducible unitary automorphic representation of $G(\mathbb {A}_F)$ with trivial central character. Let S be a finite set of rational primes such that for all $p \notin S$ and all places v of F above p:

  • $G_v$ is unramified over $F_v$ ;

  • $\mathcal {K}_v$ is a hyperspecial maximal compact subgroup of $G_v$ ;

  • $\Pi _v$ has a nonzero $\mathcal {K}_v$ -fixed vector.

Let

be the Hecke algebra of compactly supported $\mathcal {K}^S$ -bi-invariant functions on $G(\mathbb {A}_F^S)$ , where $\mathbb {A}_F^S = {\prod ^{\prime }_{p \notin S}} \prod _{v \mid p} F_v$ and $\mathcal {K}^S = \prod _{p \notin S} \prod _{v \mid p} \mathcal {K}_v$ . Then

acts on $H^i(\mathrm {Sh}_{\mathcal {K}}, \mathbb {V}_{\underline {k}})$ . Put $\Pi ^S = \bigotimes ^{\prime }_{p \notin S} \bigotimes _{v \mid p} \Pi _v$ and

where $\chi $ is the character of

associated to $\Pi ^S$ . We define $H^{p,q}(\mathrm {Sh}_{\mathcal {K}}, \mathbb {V}_{\underline {k}})[\Pi ^S]$ similarly.

Proposition 11.4. We have

$$\begin{align*}H^{2d}(\mathrm{Sh}_{\mathcal{K}}, \mathbb{V}_{\underline{k}})[\Pi^S] = H^{d,d}(\mathrm{Sh}_{\mathcal{K}}, \mathbb{V}_{\underline{k}})[\Pi^S], \end{align*}$$

where $d = |\Sigma |$ .

Proof. By Matsushima’s formula [Reference Borel and Wallach10, VII.5.2], we have

$$\begin{align*}H^{2d}(\mathrm{Sh}_{\mathcal{K}}, \mathbb{V}_{\underline{k}}) \simeq \bigoplus_\pi m(\pi) H^{2d}(\mathfrak{g}, K_{\infty}; \pi_{\infty} \otimes \rho_{\underline{k}}) \otimes \pi_f^{\mathcal{K}}, \end{align*}$$

where $\pi = \pi _{\infty } \otimes \pi _f$ runs over equivalence classes of irreducible admissible representations of $(\mathfrak {g}, K_{\infty }) \times G(\mathbb {A}_{F,f})$ , $m(\pi )$ is the multiplicity of $\pi $ in the space of automorphic forms on $G(\mathbb {A}_F)$ and $\pi _f^{\mathcal {K}}$ is the space of $\mathcal {K}$ -fixed vectors in $\pi _f$ . Since this isomorphism is compatible with the Hodge decompositions, it suffices to prove the following: If $m(\pi )> 0$ and $\pi _v \simeq \Pi _v$ for almost all v, then

(11.2) $$ \begin{align} H^{2d}(\mathfrak{g}, K_{\infty}; \pi_{\infty} \otimes \rho_{\underline{k}}) = H^{d,d}(\mathfrak{g}, K_{\infty}; \pi_{\infty} \otimes \rho_{\underline{k}}). \end{align} $$

Under this assumption, $\pi _v$ has trivial central character for almost all v, and hence so is $\pi $ since it is automorphic. Since $\pi _{\infty }$ and $\rho _{\underline {k}}$ have trivial central character, we have

$$ \begin{align*} H^{2d}(\mathfrak{g}, K_{\infty}; \pi_{\infty} \otimes \rho_{\underline{k}}) & = H^{2d}(\mathfrak{g}', K^{\prime}_{\infty}; \pi_{\infty}' \otimes \rho_{\underline{k}}'), \\ H^{d,d}(\mathfrak{g}, K_{\infty}; \pi_{\infty} \otimes \rho_{\underline{k}}) & = H^{d,d}(\mathfrak{g}', K^{\prime}_{\infty}; \pi_{\infty}' \otimes \rho_{\underline{k}}'), \end{align*} $$

where $\pi _{\infty }' = \pi _{\infty }|_{(\mathfrak {g}', K^{\prime }_{\infty })}$ and $\rho ^{\prime }_{\underline {k}} = \rho _{\underline {k}}|_{G_{\infty }'}$ . Note that $\pi _{\infty }'$ and $\rho ^{\prime }_{\underline {k}}$ remain irreducible.

Fix a realization of $\pi $ in the space of automorphic forms on $G(\mathbb {A}_F)$ . Let be the restriction of to $G'(\mathbb {A}_F)$ as functions so that is a nonzero subspace of the space of automorphic forms on $G'(\mathbb {A}_F)$ . Fix an irreducible component $\pi '$ of . Since the natural surjective map is $(\mathfrak {g}', K^{\prime }_{\infty }) \times G'(\mathbb {A}_{F,f})$ -equivariant, $\pi ^{\prime }_v$ is an irreducible component of $\pi _v|_{G_v'}$ (resp. $\pi ^{\prime }_v = \pi _v|_{(\mathfrak {g}_v', K_v')}$ ) if v is finite (resp. if v is real).

We now compute the A-parameter $\psi $ of $\pi '$ . Choose an irreducible unitary cuspidal automorphic representation $\tilde {\tau }$ of $\mathrm {GL}_2(\mathbb {A}_F)$ so that $\tau $ is an irreducible component of $\tilde {\tau }^B|_{B^\times (\mathbb {A}_F)^+}$ , where $\tilde {\tau }^B$ is the Jacquet–Langlands transfer of $\tilde {\tau }$ to $B^\times (\mathbb {A}_F)$ . Let $\tau _E$ be the base change of $\tilde {\tau }$ to $\mathrm {GL}_2(\mathbb {A}_E)$ . Note that $\tau _E$ does not depend on the choice of $\tilde {\tau }$ . For almost all v, $\tilde {\tau }_v$ is a principal series representation $\operatorname {Ind}(\chi _v \otimes \chi _v^{-1} \xi _{E_v})$ of $\mathrm {GL}_2(F_v)$ for some unramified character $\chi _v$ of $F_v^\times $ . Hence,

$$\begin{align*}\tau_{E,v} = \operatorname{Ind}(\eta_v \otimes \eta_v^{-1}) \end{align*}$$

for almost all v, where $\eta _v = \chi _v \circ \mathrm {N}_{E_v/F_v}$ . On the other hand, by Lemmas 6.1 and 9.4, $\pi _v$ is the unique irreducible unramified subquotient of

$$\begin{align*}\operatorname{Ind}^{G_v}_{B_{G_v}}(\eta_v |\cdot|_{E_v}^{\frac{1}{2}} \otimes \eta_v |\cdot|_{E_v}^{-\frac{1}{2}} \otimes \chi_v^{-2}) \end{align*}$$

for almost all v, where $B_{G_v}$ is the standard Borel subgroup of $G_v$ containing the maximal torus $T_v \simeq (E_v^\times )^2 \times F_v^\times $ as in §9.4. Hence, $\pi ^{\prime }_v$ is the unique irreducible unramified subquotient of

$$\begin{align*}\operatorname{Ind}^{G^{\prime}_v}_{B_{G^{\prime}_v}}(\eta_v |\cdot|_{E_v}^{\frac{1}{2}} \otimes \eta_v |\cdot|_{E_v}^{-\frac{1}{2}}) \end{align*}$$

for almost all v, where $B_{G^{\prime }_v} = B_{G_v} \cap G_v'$ is the standard Borel subgroup of $G_v'$ containing the maximal torus $T_v \cap G_v' \simeq (E_v^\times )^2$ . Namely, we have

$$\begin{align*}\psi = \tau_E \boxtimes \mathrm{Sym}^1. \end{align*}$$

Thus, by the classification of automorphic representations of $G'(\mathbb {A}_F)$ , $\pi _v'$ is an irreducible summand of some representation in the local A-packet $\Pi _{\psi _v}$ for all v. In particular, if $v \in \Sigma $ , then $\pi _v'$ is one of the representations $A_{\mathfrak {q}_i}(w_i^{-1} \lambda _v)$ as in §11.2, and hence we have

$$\begin{align*}H^i(\mathfrak{g}_v',K^{\prime}_v; \pi_v' \otimes \rho^{\prime}_{v, k_v}) = 0 \end{align*}$$

for $i<2$ and

$$\begin{align*}H^2(\mathfrak{g}_v',K^{\prime}_v; \pi_v' \otimes \rho^{\prime}_{v, k_v}) = H^{1,1}(\mathfrak{g}_v',K^{\prime}_v; \pi_v' \otimes \rho^{\prime}_{v, k_v}) \end{align*}$$

by Proposition 11.2. From this, we deduce that

$$ \begin{align*} H^{2d}(\mathfrak{g}', K^{\prime}_{\infty}; \pi_{\infty}' \otimes \rho_{\underline{k}}') & = \bigotimes_{v \in \Sigma} H^2(\mathfrak{g}_v', K^{\prime}_v; \pi_v' \otimes \rho_{v, k_v}') \bigotimes_{v \in \Sigma_{\infty} \smallsetminus \Sigma} H^0(\mathfrak{g}_v', K^{\prime}_v; \pi_v' \otimes \rho_{v, k_v}') \\ & = \bigotimes_{v \in \Sigma} H^{1,1}(\mathfrak{g}_v', K^{\prime}_v; \pi_v' \otimes \rho_{v, k_v}') \bigotimes_{v \in \Sigma_{\infty} \smallsetminus \Sigma} H^0(\mathfrak{g}_v', K^{\prime}_v; \pi_v' \otimes \rho_{v, k_v}') \\ & = H^{d,d}(\mathfrak{g}', K^{\prime}_{\infty}; \pi_{\infty}' \otimes \rho_{\underline{k}}'). \end{align*} $$

This completes the proof.

Remark 11.5. The classification of automorphic representations is used in the proof of Proposition 11.4, but in fact, we can avoid appealing to the result of Kaletha–Minguez–Shin–White [Reference Kaletha, Minguez, Shin and White34] as follows. Let $\pi $ be an irreducible automorphic representation of $G(\mathbb {A}_F)$ such that $\pi _v \simeq \Pi _v$ for almost all v. Then, by Proposition 9.5, we may write $\pi $ as a global theta lift. Hence, if $\pi ^{\prime }_v$ is an irreducible component of $\pi _v|_{(\mathfrak {g}_v', K_v')}$ for $v \in \Sigma $ , then it follows from the description of local theta lifts [Reference Li48] (see also §7.2 and Lemma 11.3) that $\pi ^{\prime }_v = A_{\mathfrak {q}_i}(w_i^{-1} \lambda _v)$ for some i. This implies equation (11.2).

Proposition 11.6. The Hodge structure $H^{2d}(\mathrm {Sh}_{\mathcal {K}}, \mathbb {V}_{\underline {k}})[\Pi ^S]$ is purely of type $(d,d)$ .

Proof. From the proof of Proposition 11.4, we need to compute the Hodge structure on

$$\begin{align*}H^{d,d}(\mathfrak{g}', K^{\prime}_{\infty}; \pi_{\infty}' \otimes \rho_{\underline{k}}') \simeq \bigotimes_{v \in \Sigma} H^{1,1}(\mathfrak{g}_v', K^{\prime}_v; \pi_v' \otimes \rho_{v, k_v}') \bigotimes_{v \in \Sigma_{\infty} \smallsetminus \Sigma} H^0(\mathfrak{g}_v', K^{\prime}_v; \pi_v' \otimes \rho_{v, k_v}'). \end{align*}$$

The term $H^0(\mathfrak {g}_v', K^{\prime }_v; \pi _v' \otimes \rho _{v, k_v}')$ (for $v\in \Sigma _{\infty } \smallsetminus \Sigma $ ) is clearly of type $(0,0)$ , so we are reduced to showing that $H^{1,1}(\mathfrak {g}_v', K^{\prime }_v; \pi _v' \otimes \rho _{v, k_v}')$ (for $v\in \Sigma $ ) is of type $(1,1)$ . The Hodge type can be computed using [Reference Zucker72], keeping in mind that loc. cit. gives the Hodge numbers in the standard normalization; they need to be twisted appropriately to get the Hodge type in the automorphic (unitary) normalization that we are using. For ease of comparison with [Reference Zucker72], we temporarily change notation to match that reference. (See also [Reference Faltings16], [Reference Harris25].)

Fix $v \in \Sigma $ for the rest of the proof. Let $W \simeq \mathfrak {S}_4$ and $W_c \simeq \mathfrak {S}_2 \times \mathfrak {S}_2$ be the Weyl groups of $\mathfrak {g}^{\prime }_v = \mathfrak {gl}(4,\mathbb {C})$ and $\mathfrak {k}^{\prime }_v = \mathfrak {gl}(2,\mathbb {C}) \oplus \mathfrak {gl}(2,\mathbb {C})$ , respectively. Let $W^1$ be the set of representatives for $W_c \backslash W$ given by

$$\begin{align*}\{ w \in W \, | \, w^{-1}(\Delta_c^+) \subset \Delta^+ \}, \end{align*}$$

where $\Delta _c^+$ and $\Delta ^+$ are the sets of positive roots in $\mathfrak {k}^{\prime }_v$ and $\mathfrak {g}^{\prime }_v$ , respectively. Put

$$\begin{align*}W^1(p) = \{ w \in W^1 \, | \, \ell(w) = p \}, \end{align*}$$

where $\ell (w)$ is the length of w. Then we can enumerate the elements in $W^1$ as follows:

$$\begin{align*}\begin{array}{c|cccccc} p & 0 & 1 & 2 & 2 & 3 & 4 \\ \hline w^{-1} & 1 & (23) & (243) & (123) & (1243) & (13)(24). \end{array} \end{align*}$$

Recall that $k_v$ is a positive even integer and put

$$\begin{align*}\Lambda = \tfrac{1}{2}(k_v-2,k_v-2,-k_v+2,-k_v+2). \end{align*}$$

Let $\rho $ be half the sum of positive roots in $\mathfrak {g}_v'$ :

$$\begin{align*}\rho = \tfrac{1}{2}(3,1,-1,-3). \end{align*}$$

Let $Z=\{t_\alpha := (\alpha , \alpha ^{-1} ) \in \mathrm {U}(1)\times \mathrm {U}(1) \} \subset \mathrm {U}(2) \times \mathrm {U}(2)$ . As in [Reference Zucker72], §1 and §4, let $\mu $ and $\lambda $ denote the highest characters of Z appearing in the adjoint representation of $\mathrm {U}(2,2)$ and in the representation $\rho ^{\prime }_{v,k_v}$ . Then $\mu $ is just the action on $(\mathfrak {p}_v')^+$ and is explicitly given by

$$\begin{align*}\mu (t_\alpha) = \alpha^2. \end{align*}$$

As for $\lambda $ , it agrees with the action of Z on the irreducible representation of $\mathrm {U}(2) \times \mathrm {U}(2)$ with highest weight $\Lambda $ ; since this representation is just $\det ^{\frac {k_v}{2}-1} \boxtimes \det ^{-\frac {k_v}{2}+1}$ , we see that

$$\begin{align*}\lambda(t_\alpha) = (\alpha^2)^{\frac{k_v}{2}-1} (\alpha^{-2})^{-\frac{k_v}{2}+1} = \alpha^{2k_v-4}. \end{align*}$$

Since $\rho ^{\prime }_{v,k_v}$ is self-dual, the lowest character of Z appearing in $\rho ^{\prime }_{v,k_v}$ is simply $\lambda ^{-1}$ ; thus, $m= 2k_v-4$ , where m is defined as in loc. cit. equation (4.8). Here, m is the total weight of the Hodge structure on the fiber of the local system. (To convert to our normalization, where $\rho _{v,k_v}$ has trivial central character and hence the total weight on the fiber is zero, we must therefore twist the Hodge numbers below by $(2-k_v,2-k_v)$ .)

For completeness, we consider not just $H^{1,1}$ but all the nonzero $H^{p,q}(\mathfrak {g}_v', K^{\prime }_v; \pi _v' \otimes \rho _{v, k_v}')$ , where $\pi _v'$ is chosen such that $H^{p,q} \neq 0$ . This space is then the sum of components of multidegree $(p,q);(r,s)$ , where $(r,s)$ with $r+s=m$ is the bidegree coming from the Hodge structure on the fiber. By [Reference Zucker72], §5, the $(p,q);(k-p,m+p-k)$ component can only be nonzero if the action $\tau _Z$ of Z on the irreducible representation $\tau $ of $\mathrm {U}(2) \times \mathrm {U}(2)$ with highest weight $w(\Lambda + \rho ) - \rho $ is $\lambda - k\mu $ for some $w \in W^1 (p)$ . Thus, we just need to run through the different choices of $(p,q)$ and $w\in W^1 (p)$ .

  • If $(p,q)=(0,4)$ and $w^{-1}=1$ , then

    $$ \begin{align*} w(\Lambda+\rho) & = \tfrac{1}{2}(k_v+1,k_v-1,-k_v+1,-k_v-1), \\ w(\Lambda+\rho) - \rho & = \tfrac{1}{2}(k_v-2,k_v-2,-k_v+2,-k_v+2) \end{align*} $$
    so that
    $$ \begin{align*} \tau & = (\mathrm{Sym}^{0} \otimes {\det}^{\frac{k_v}{2}-1}) \boxtimes (\mathrm{Sym}^{0} \otimes {\det}^{-\frac{k_v}{2}+1}), \\ \tau_Z & : t_\alpha \mapsto (\alpha^2)^{\frac{k_v}{2}-1} (\alpha^{-2})^{-\frac{k_v}{2}+1}=\alpha^{2k_v-4}. \end{align*} $$
    Then $k=0$ , so the Hodge type is
    $$\begin{align*}(0,4) + (0,2k_v-4) = (0,2k_v). \end{align*}$$
  • If $(p,q)=(1,1)$ and $w^{-1}=(23)$ , then

    $$ \begin{align*} w(\Lambda+\rho) & = \tfrac{1}{2}(k_v+1,-k_v+1,k_v-1,-k_v-1), \\ w(\Lambda+\rho) - \rho & = \tfrac{1}{2}(k_v-2,-k_v,k_v,-k_v+2) \end{align*} $$
    so that
    $$ \begin{align*} \tau & = (\mathrm{Sym}^{k_v-1} \otimes {\det}^{-\frac{k_v}{2}}) \boxtimes (\mathrm{Sym}^{k_v-1} \otimes {\det}^{-\frac{k_v}{2}+1}), \\ \tau_Z & : t_\alpha \mapsto \alpha^{k_v-1} (\alpha^2)^{-\frac{k_v}{2}}(\alpha^{-1})^{k_v-1} (\alpha^{-2})^{-\frac{k_v}{2}+1}=\alpha^{-2}. \end{align*} $$
    Then $k=k_v-1$ , so the Hodge type is
    $$\begin{align*}(1,1) + (k_v-1-1, 2k_v-4 +1-(k_v-1) = (k_v-1,k_v-1). \end{align*}$$
  • If $(p,q)=(2,2)$ and $w^{-1}=(243)$ , then

    $$ \begin{align*} w(\Lambda+\rho) & = \tfrac{1}{2}(k_v+1,-k_v-1,k_v-1,-k_v+1), \\ w(\Lambda+\rho) - \rho & = \tfrac{1}{2}(k_v-2,-k_v-2,k_v,-k_v+4) \end{align*} $$
    so that
    $$ \begin{align*} \tau & = (\mathrm{Sym}^{k_v} \otimes {\det}^{-\frac{k_v}{2}-1}) \boxtimes (\mathrm{Sym}^{k_v-2} \otimes {\det}^{-\frac{k_v}{2}+2}), \\ \tau_Z & : t_\alpha \mapsto \alpha^{k_v} (\alpha^2)^{-\frac{k_v}{2}-1}(\alpha^{-1})^{k_v-2}(\alpha^{-2})^{-\frac{k_v}{2}+2}=\alpha^{-4}. \end{align*} $$
    Then $k=k_v$ , so the Hodge type is
    $$\begin{align*}(2,2) + (k_v-2, 2k_v-4 + 2-k_v) = (k_v, k_v). \end{align*}$$
  • If $(p,q)=(2,2)$ and $w^{-1}=(123)$ , then

    $$ \begin{align*} w(\Lambda+\rho) & = \tfrac{1}{2}(k_v-1,-k_v+1,k_v+1,-k_v-1), \\ w(\Lambda+\rho) - \rho & = \tfrac{1}{2}(k_v-4,-k_v,k_v+2,-k_v+2) \end{align*} $$
    so that
    $$ \begin{align*} \tau & = (\mathrm{Sym}^{k_v-2} \otimes {\det}^{-\frac{k_v}{2}}) \boxtimes (\mathrm{Sym}^{k_v} \otimes {\det}^{-\frac{k_v}{2}+1}), \\ \tau_Z & : t_\alpha \mapsto \alpha^{k_v-2} (\alpha^2)^{-\frac{k_v}{2}}(\alpha^{-1})^{k_v}(\alpha^{-2})^{-\frac{k_v}{2}+1}=\alpha^{-4}. \end{align*} $$
    Then $k=k_v$ , so the Hodge type is
    $$\begin{align*}(2,2) + (k_v-2, 2k_v-4 + 2-k_v) = (k_v, k_v). \end{align*}$$
  • If $(p,q)=(3,3)$ and $w^{-1}=(1243)$ , then

    $$ \begin{align*} w(\Lambda+\rho) & = \tfrac{1}{2}(k_v-1,-k_v-1,k_v+1,-k_v+1), \\ w(\Lambda+\rho) - \rho & = \tfrac{1}{2}(k_v-4,-k_v-2,k_v+2,-k_v+4) \end{align*} $$
    so that
    $$ \begin{align*} \tau & = (\mathrm{Sym}^{k_v-1} \otimes {\det}^{-\frac{k_v}{2}-1}) \boxtimes (\mathrm{Sym}^{k_v-1} \otimes {\det}^{-\frac{k_v}{2}+2}), \\ \tau_Z & : t_\alpha \mapsto \alpha^{k_v-1} (\alpha^2)^{-\frac{k_v}{2}-1}(\alpha^{-1})^{k_v-1}(\alpha^{-2})^{-\frac{k_v}{2}+2}=\alpha^{-6}. \end{align*} $$
    Then $k=k_v+1$ , so the Hodge type is
    $$\begin{align*}(3,3) + (k_v+1-3, 2k_v-4 + 3-(k_v+1)) = (k_v+1, k_v+1). \end{align*}$$
  • If $(p,q)=(4,0)$ and $w^{-1}=(13)(24)$ , then

    $$ \begin{align*} w(\Lambda+\rho) & = \tfrac{1}{2}(-k_v+1,-k_v-1,k_v+1,k_v-1), \\ w(\Lambda+\rho) - \rho & = \tfrac{1}{2}(-k_v-2,-k_v-2,k_v+2,k_v+2) \end{align*} $$
    so that
    $$ \begin{align*} \tau & = (\mathrm{Sym}^{0} \otimes {\det}^{-\frac{k_v}{2}-1}) \boxtimes (\mathrm{Sym}^{0} \otimes {\det}^{\frac{k_v}{2}+1}), \\ \tau_Z & : t_\alpha \mapsto (\alpha^2)^{-\frac{k_v}{2}-1}(\alpha^{-2})^{\frac{k_v}{2}+1}=\alpha^{-2k_v-4}. \end{align*} $$
    Then $k=2k_v$ , so the Hodge type is
    $$\begin{align*}(4,0) + (2k_v-4, 2k_v-4 + 4-2k_v) = (2k_v, 0). \end{align*}$$

The relevant case for us is the case $(p,q)=(1,1)$ in which case the overall Hodge type $(k_v-1,k_v-1)$ ; twisting it by $(2-k_v, 2-k_v)$ , we see that $H^{1,1}(\mathfrak {g}_v', K^{\prime }_v; \pi _v' \otimes \rho _{v, k_v}')$ has Hodge type $(1,1)$ as expected.

Finally, we note that by §4.4, we may regard $\Pi $ as an automorphic representation of any of the groups , , , . Let S and $\mathcal {K}$ be as in §4.4 as well. Then we get:

Corollary 11.7. The Hodge structure on is purely of type $(d,d)$ .

11.4 Galois representations

Finally, we state the main result we need on Galois representations.

Proposition 11.8. Assume Kottwitz’s conjecture for Shimura varieties attached to unitary similitude groups. Then the action of $\operatorname {Gal}(\overline {\mathbb {Q}}/F_\Sigma )$ on

is trivial.

The proposition above encodes the expected relation between the automorphic form $\Pi $ and the cohomology of the Shimura variety and is consistent with Corollary 11.7. As such, it is an immediate consequence of the following special case of Kottwitz’s conjecture [Reference Kottwitz37, §10]:

Proposition 11.9. Assume Kottwitz’s conjecture for Shimura varieties attached to unitary similitude groups. Then the action of $\operatorname {Gal}(\overline {\mathbb {Q}}/F_\Sigma )$ on the semisimplification of

is trivial.

Remark 11.10. Proposition 11.8 follows directly from Proposition 11.9 by a standard argument using the finiteness of the class number of $F_\Sigma $ (as in [Reference Nekovář54, §5.13]). For convenience of the reader, we include the argument here. Recall that any subquotient of the representation of $\operatorname {Gal}(\overline {\mathbb {Q}}/F_\Sigma )$ on (regarded as a $\mathbb {Q}_\ell $ -vector space) is unramified at almost all places and is de Rham at all places dividing $\ell $ . Thus, it suffices to show that $H^1_g(k,\mathbb {Q}_\ell ) = 0$ for any number field k. Here, $H^1_g(k,\mathbb {Q}_\ell )$ is the Bloch–Kato Selmer group [Reference Bloch and Kato8] consisting of elements $x \in H^1(k,\mathbb {Q}_\ell )$ such that

  • $x_v \in H^1_f(k_v, \mathbb {Q}_\ell )$ for almost all v;

  • $x_v \in H^1_g(k_v, \mathbb {Q}_\ell )$ for all v dividing $\ell $ ,

where for any finite place v of k, $x_v$ denotes the restriction of x to $H^1(k_v,\mathbb {Q}_\ell )$ . By class field theory, we identify

$$\begin{align*}H^1(k_v,\mathbb{Q}_\ell) = \operatorname{Hom}_{\mathrm{cont}}(\operatorname{Gal}(\overline{k_v}/k_v), \mathbb{Q}_\ell) = \operatorname{Hom}_{\mathrm{cont}}(k_v^\times, \mathbb{Q}_\ell). \end{align*}$$

Under this identification, we have $H^1_f(k_v,\mathbb {Q}_\ell ) = H^1(k_v,\mathbb {Q}_\ell ) = \operatorname {Hom}(k_v^\times /\mathfrak {o}_v^\times , \mathbb {Q}_\ell )$ if v does not divide $\ell $ , and $H^1_g(k_v,\mathbb {Q}_\ell ) = \operatorname {Hom}(k_v^\times /\mathfrak {o}_v^\times , \mathbb {Q}_\ell )$ if v divides $\ell $ by [Reference Bloch and Kato8, Example 3.9]. Here, $\mathfrak {o}_v$ is the ring of integers of $k_v$ . From this, we deduce that

$$\begin{align*}H^1_g(k,\mathbb{Q}_\ell) = \operatorname{Hom}(\mathbb{A}_k^\times / k^\times k_{\infty}^\times \widehat{\mathfrak{o}}^\times, \mathbb{Q}_\ell) \end{align*}$$

with $k_{\infty } = k \otimes _{\mathbb {Q}} \mathbb {R}$ and $\widehat {\mathfrak {o}} = \prod _v \mathfrak {o}_v$ . But since $\mathbb {A}_k^\times / k^\times k_{\infty }^\times \widehat {\mathfrak {o}}^\times $ is finite, we have $H^1_g(k,\mathbb {Q}_\ell ) = 0$ .

Remark 11.11. We remark that Kottwitz’s conjecture for should follow from the stable trace formula for Shimura varieties of abelian type established by Kisin–Shin–Zhu [Reference Kisin, Shin and Zhu35] but is conditional on the classification of automorphic representations on unitary similitude groups and the equality [Reference Kisin, Shin and Zhu35, (9.2.2.1)] of certain stable distributions. This is explained in more detail in Remark 1.4 in the introduction.

In the next section, we explain how to deduce Proposition 11.9 from Kottwitz’s conjecture.

11.5 Kottwitz’s conjecture

Put $\Gamma _k = \operatorname {Gal}(\overline {\mathbb {Q}}/k)$ for a number field k. Then $\Sigma _{\infty }$ (regarded as the set of embeddings of F in $\mathbb {C}$ ) admits a natural action of $\Gamma _{\mathbb {Q}}$ induced by the inclusion $\overline {\mathbb {Q}} \hookrightarrow \mathbb {C}$ . We identify $\Sigma _{\infty }$ with $\Gamma _{\mathbb {Q}}/\Gamma _F$ so that the fixed embedding $F \hookrightarrow \overline {\mathbb {Q}}$ corresponds to the trivial coset $\Gamma _F$ . Choose a set of representatives $\{ \sigma _1, \dots , \sigma _n \}$ for $\Gamma _{\mathbb {Q}}/\Gamma _F$ so that $\Sigma = \{\sigma _1 \Gamma _F, \dots , \sigma _d \Gamma _F\}$ , where $n = [F:\mathbb {Q}]$ and $d = |\Sigma |$ . Define an action of $\Gamma _{\mathbb {Q}}$ on $\{1, \dots , n\}$ so that

$$\begin{align*}\gamma \sigma_i \Gamma_F = \sigma_{\gamma(i)} \Gamma_F \end{align*}$$

for $\gamma \in \Gamma _{\mathbb {Q}}$ . We denote by $F_\Sigma $ the fixed field of the subgroup

$$\begin{align*}\{\sigma \in \Gamma_{\mathbb{Q}} \, | \, \sigma \Sigma = \Sigma \}. \end{align*}$$

Recall that $G = \mathrm {GU}(\mathbf {V})$ with

$$\begin{align*}G_v \simeq \begin{cases} \mathrm{GU}(2,2) & \text{if } v \in \Sigma; \\ \mathrm{GU}(4) & \text{if } v \in \Sigma_{\infty} \smallsetminus \Sigma \end{cases} \end{align*}$$

and $G_0 = \operatorname {Res}_{F/\mathbb {Q}} G$ . (Note that with the notation of §4.1.) Then we have

$$\begin{align*}{}^L G = \hat{G} \rtimes \Gamma_F, \qquad \hat{G} = \mathrm{GL}_4(\mathbb{C}) \times \mathbb{C}^\times, \end{align*}$$

where $\Gamma _E$ acts trivially on $\hat {G}$ and the nontrivial element in $\operatorname {Gal}(E/F)$ acts as the automorphism $\hat {\theta }$ defined by

$$\begin{align*}\hat{\theta}(g,\nu) = (\theta_4(g), \nu \cdot \det g). \end{align*}$$

Also, by [Reference Borel9, §5], we have

$$\begin{align*}{}^LG_0 = \hat{G}_0 \rtimes \Gamma_{\mathbb{Q}}, \qquad \hat{G}_0 = (\hat{G})^n, \end{align*}$$

where $\gamma \in \Gamma _{\mathbb {Q}}$ acts on $\hat {G}_0$ as the automorphism

$$\begin{align*}(g_1, \dots, g_n) \mapsto (\gamma_1 \cdot g_{\gamma^{-1}(1)}, \dots, \gamma_n \cdot g_{\gamma^{-1}(n)}) \end{align*}$$

with

$$\begin{align*}\gamma_i = \sigma_i^{-1} \gamma \sigma_{\gamma^{-1}(i)} \in \Gamma_F. \end{align*}$$

To describe the Galois representation on the cohomology of the Shimura variety , we need to introduce some representation of the L-group. Following [Reference Blasius and Rogawski7, §5.1], we recall its definition. Let $\mu : \mathbb {G}_{m,\mathbb {C}} \rightarrow \mathbb {S}_{\mathbb {C}} \rightarrow G_{0,\mathbb {C}}$ be the cocharacter associated to the homomorphism $h: \mathbb {S} \rightarrow G_{0,\mathbb {R}}$ as in §11.3. More explicitly, we have $\mu (z) = (\mu _v(z))_{v \in \Sigma _{\infty }}$ with

$$\begin{align*}\mu_v(z) = \begin{cases} \begin{pmatrix} z {\mathbf{1}}_2 & \\ & {\mathbf{1}}_2 \end{pmatrix} \times z & \text{if } v \in \Sigma; \\ {\mathbf{1}}_4 \times 1 & \text{if } v \in \Sigma_{\infty} \smallsetminus \Sigma. \end{cases} \end{align*}$$

From this, we see that the reflex field of the Shimura datum $(G_0,X)$ is $F_\Sigma $ . We also identify $\mu $ with a character of the standard maximal torus of $\hat {G}_0$ . Let $r_0$ be the irreducible algebraic representation of $\hat {G}_0$ with extreme weight $-\mu $ , which can be explicated as follows. Let $\wedge ^2 \mathbb {C}^4$ be the exterior square of the standard representation of $\mathrm {GL}_4(\mathbb {C})$ and regard it as a representation of $\hat {G}$ by letting $\nu \in \mathbb {C}^\times $ act as the scalar $\nu $ . We denote by r its contragredient on $\mathcal {V} = (\wedge ^2 \mathbb {C}^4)^*$ . Then $r_0$ is the representation of $\hat {G}_0$ on $\mathcal {V}_0 = \mathcal {V}^{\otimes d}$ given by

$$\begin{align*}r_0(g_1, \dots, g_n) = r(g_1) \otimes \dots \otimes r(g_d). \end{align*}$$

On the other hand, since $(\wedge ^2 \mathbb {C}^4)^* \otimes \det \simeq \wedge ^2 \mathbb {C}^4$ as representations of $\mathrm {GL}_4(\mathbb {C})$ , there exists a unique automorphism A of $\mathcal {V}$ such that

$$\begin{align*}r(\hat{\theta}(g)) \circ A = A \circ r(g) \end{align*}$$

for all $g \in \hat {G}$ and such that A fixes the highest weight vector (unique up to scalars) in $\mathcal {V}$ with respect to the standard Borel subgroup of $\hat {G}$ . Then we can extend r to ${}^L G$ by setting

$$\begin{align*}r(1 \rtimes \sigma) = \begin{cases} \mathrm{id} & \text{if } \sigma \in \Gamma_E; \\ A & \text{otherwise} \end{cases} \end{align*}$$

and hence $r_0$ to $\hat {G}_0 \rtimes \Gamma _{F_\Sigma }$ by setting

$$\begin{align*}r_0(1 \rtimes \gamma)(x_1 \otimes \dots \otimes x_d) = r(1 \rtimes \gamma_1) x_{\gamma^{-1}(1)} \otimes \dots \otimes r(1 \rtimes \gamma_d) x_{\gamma^{-1}(d)}. \end{align*}$$

We also need to introduce the expected classification of automorphic representations of $G(\mathbb {A}_F)$ . Let $L^2_{\mathrm {disc}}(G)$ be the discrete spectrum of the unitary representation of $G(\mathbb {A}_F)$ on the Hilbert space $L^2(A_G(F_{\infty })^0 G(F) \backslash G(\mathbb {A}_F))$ , where $A_G$ is the split component of the center of G and $F_{\infty } = F\otimes _{\mathbb {Q}} \mathbb {R}$ . We say that a pair $(\psi ',\chi )$ is an elliptic A-parameter for G if

  • $\psi '$ is an elliptic A-parameter for $G'$ ;

  • $\chi $ is a character of $\mathbb {A}_E^\times /E^\times $ such that $\chi ^\rho /\chi $ is equal to the central character of $\pi _{\psi '}$ .

Then one expects the decomposition

$$\begin{align*}L^2_{\mathrm{disc}}(G) = \bigoplus_\psi L^2_\psi(G), \end{align*}$$

where $\psi = (\psi ',\chi )$ runs over elliptic A-parameters for G and $L^2_\psi (G)$ is the near equivalence class of irreducible subrepresentations $\pi $ of $L^2_{\mathrm {disc}}(G)$ such that for almost all places v of F, the base change of $\pi _v$ to $\mathrm {GL}_4(E_v) \times E_v^\times $ is isomorphic to $\pi _{\psi ',v} \boxtimes \chi _v$ .

To compute the Galois representation, it is convenient to introduce the hypothetical Langlands group $\mathcal {L}_k$ of a number field k equipped with a surjective homomorphism $\operatorname {pr}: \mathcal {L}_k \twoheadrightarrow \Gamma _k$ . Let $\psi $ be an elliptic A-parameter for G and regard it as an L-homomorphism $\psi : \mathcal {L}_F \times \mathrm {SL}_2(\mathbb {C}) \rightarrow {}^L G$ . Let $\phi _\psi : \mathcal {L}_F \rightarrow {}^L G$ be the L-parameter associated to $\psi $ , that is,

$$\begin{align*}\phi_{\psi}(w) = \psi \! \left( w, \begin{pmatrix} |w|^{\frac{1}{2}} & \\ & |w|^{-\frac{1}{2}} \end{pmatrix} \right). \end{align*}$$

Then we have a representation $r(\psi ) = (r \circ \phi _{\psi }) \otimes |\cdot |^{-2}$ of $\mathcal {L}_F$ on $\mathcal {V}$ equipped with a decomposition

$$\begin{align*}\mathcal{V} = \bigoplus_i \mathcal{V}^i, \end{align*}$$

where

$$\begin{align*}\mathcal{V}^i = \left\{ v \in \mathcal{V} \, \left| \, (r \circ \psi) \left( 1, \left( \begin{smallmatrix} t & \\ & t^{-1} \end{smallmatrix} \right) \right) v = t^{-i} v \text{ for all } t \in \mathbb{C}^\times \right. \right\}. \end{align*}$$

Similarly, if $\psi _0: \mathcal {L}_{\mathbb {Q}} \times \mathrm {SL}_2(\mathbb {C}) \rightarrow {}^L G_0$ is the A-parameter induced by $\psi $ , then we have a representation $r(\psi _0) = (r_0 \circ \phi _{\psi _0}) \otimes |\cdot |^{-2d}$ of $\mathcal {L}_{F_\Sigma }$ on $\mathcal {V}_0$ equipped with a decomposition

$$\begin{align*}\mathcal{V}_0 = \bigoplus_i \mathcal{V}^i_0. \end{align*}$$

More explicitly, we have

$$\begin{align*}\psi_0(\gamma, h) = (g(\gamma_1, h), \dots, g(\gamma_d, h)) \rtimes \operatorname{pr}(\gamma), \end{align*}$$

where we write $\psi (\sigma , h) = g(\sigma , h) \rtimes \operatorname {pr}(\sigma )$ and $\gamma _i \in \mathcal {L}_F$ is defined similarly as above, and

$$\begin{align*}\mathcal{V}_0^i = \bigoplus_{i=i_1+\dots+i_d} \mathcal{V}^{i_1} \otimes \dots \otimes \mathcal{V}^{i_d}. \end{align*}$$

We write $r^i_\ell (\psi _0)$ for the $\ell $ -adic representation of $\Gamma _{F_\Sigma }$ which should correspond to the representation of $\mathcal {L}_{F_\Sigma }$ on $\mathcal {V}_0^i$ . Finally, let $\pi ^S$ be the irreducible unramified representation of $G(\mathbb {A}_F^S)$ associated to $\psi $ , where S is a sufficiently large finite set of rational primes. Then it follows from Kottwitz’s conjecture [Reference Kottwitz37, §10] that the $\ell $ -adic representation of $\Gamma _{F_\Sigma }$ on the semisimplification of

is isomorphic to a subrepresentation of $r^{i-4d}_\ell (\psi _0)^{\oplus m}$ for some integer m.

To deduce Proposition 11.9 from Kottwitz’s conjecture, we now suppose that $\psi = (\psi ',\chi )$ with $\psi ' = \pi _E \boxtimes \mathrm {Sym}^1$ as in equation (11.1) and $\chi = 1$ . It suffices to show that $r_\ell ^{-2d}(\psi _0)(d)$ is trivial. Let $\rho $ be the two-dimensional representation of $\mathcal {L}_F$ (conjecturally) associated to $\pi $ and put $\rho _E = \rho |_{\mathcal {L}_E}$ . (We can justify the formal computation by using the $\ell $ -adic representation of $\Gamma _F$ associated to $\pi $ , but we omit the details.) Since $\pi _E$ has trivial central character, $\rho _E$ is self-dual. Let $\mathcal {W}$ be the four-dimensional representation of $\mathcal {L}_E$ induced by $\phi _\psi |_{\mathcal {L}_E}$ so that $\mathcal {W} = \mathcal {W}^1 \oplus \mathcal {W}^{-1}$ with $\mathcal {W}^{\pm 1} = \rho _E \otimes |\cdot |^{\mp \frac {1}{2}}$ . Then, noting that $\mathcal {W}$ is self-dual, we have

$$\begin{align*}\mathcal{V} = (\wedge^2 \mathcal{W})^* \otimes |\cdot|^{-2} = \wedge^2 \mathcal{W} \otimes |\cdot|^{-2} = \mathcal{V}^2 \oplus \mathcal{V}^0 \oplus \mathcal{V}^{-2} \end{align*}$$

as representations of $\mathcal {L}_F$ , where

$$ \begin{align*} \mathcal{V}^2 & = \wedge^2 \mathcal{W}^1 \otimes |\cdot|^{-2} = |\cdot|^{-3}, \\ \mathcal{V}^0 & = \mathcal{W}^1 \otimes \mathcal{W}^{-1} \otimes |\cdot|^{-2} = \mathrm{As}^+(\rho_E) \otimes |\cdot|^{-2} = |\cdot|^{-2} \oplus \left( \mathrm{Sym}^2(\rho) \otimes |\cdot|^{-2} \right), \\ \mathcal{V}^{-2} & = \wedge^2 \mathcal{W}^{-1} \otimes |\cdot|^{-2} = |\cdot|^{-1}. \end{align*} $$

(Note that in the context of §1.2.8 when $F=\mathbb {Q}$ and $d=1$ , $\mathcal {V}^i$ corresponds to the $\ell $ -adic representation on $H^{i+4}$ .) Hence,

$$\begin{align*}\mathcal{V}_0^{-2d} \otimes |\cdot|^d = (\mathcal{V}^{-2})^{\otimes d} \otimes |\cdot|^d \end{align*}$$

is the trivial representation of $\mathcal {L}_{F_\Sigma }$ as desired.

12 Hodge–Tate classes and the proof of the main theorem

12.1 Hodge–Tate classes

We make the following definition. Recall that the category $\mathcal {M}_k^L$ that is used below was defined in §2.1.

Definition 12.1. Let $(V,V_\ell , i_\ell )$ denote a (pure) object in $\mathcal {M}_k^L$ . A class $c\in V$ is said to be a Hodge–Tate class (HT in brief) if c is a Hodge class in V and $i_\ell (c)$ is a Tate class in $V_\ell $ for all $\ell $ . Thus, c is required to lie in $V^{0,0}$ and $i_\ell (c)$ is required to be $G_k$ -invariant for all $\ell $ .

We let $\mathcal {HT} (V)$ denote the L-subspace of HT-classes in V and $\mathcal {HT}(V)_{\mathbb {C}}$ its $\mathbb {C}$ -span. (This notation is slightly ambiguous since it does not keep track of the isomorphisms $i_\ell $ ; this will typically not cause a problem since the maps $i_\ell $ will be understood from the context.) Clearly, any morphism from $(V,V_\ell , i_\ell )$ to $(V', V^{\prime }_\ell , i^{\prime }_\ell )$ induces maps $\mathcal {HT}(V) \rightarrow \mathcal {HT}(V')$ and $\mathcal {HT}(V)_{\mathbb {C}} \rightarrow \mathcal {HT}(V')_{\mathbb {C}}$ .

If $L\subset L'\subset \mathbb {C}$ , the natural functor $\mathcal {M}_k^L \rightarrow \mathcal {M}_k^{L'}$ carries $\mathcal {HT}(V)$ into $\mathcal {HT}(V_{L'})$ , where we write $V_{L'}$ for $V\otimes _{L} L'$ .

12.2 The construction of a cohomology class

While some aspects of the construction have been described previously at various points in the paper, we now collect in a single place the entire construction, which also makes clear the dependence on various auxiliary choices.

12.2.1 Spaces and groups

Choose a CM quadratic extension $E/F$ that embeds in both $B_1$ and $B_2$ . (Later, we will be more careful about the choice of E.) Fix embeddings $E\hookrightarrow B_1$ and $E\hookrightarrow B_2$ . Let $\mathbf {V}_1=B_1$ and $\mathbf {V}_2=B_2$ , viewed as Hermitian E-spaces with the canonical Hermitian form, as in [Reference Ichino and Prasanna30, §2.2], and let $\mathbf {V}=\mathbf {V}_1 \oplus \mathbf {V}_2$ be their direct sum, viewed as a four-dimensional Hermitian E-space. To the space $\mathbf {V}$ , we can associate the skew-Hermitian B-space $\tilde {V}$ as in §5.2. Further, as in §5.3.1, the decomposition $\mathbf {V}=\mathbf {V}_1 \oplus \mathbf {V}_2$ of E-Hermitian spaces induces a decomposition $\tilde {V} = V \oplus V_0$ as the sum of skew-Hermitian B-spaces of dimensions two and one, respectively. We then get a collection of groups and maps between them as described in §4, and the reader is referred especially to the diagrams of groups in that section, which will be used often in the construction below.

12.2.2 Shimura data

Fix isomorphisms

$$\begin{align*}B_i \otimes_{F,\sigma_v} \mathbb{R} \simeq \mathrm{M}_2(\mathbb{R}) \ \ \text{for }v\in \Sigma; \quad B_i \otimes_{F,\sigma_v} \mathbb{R} \simeq {\mathbb{H}} \ \ \text{for }v\in \Sigma_{\infty}\smallsetminus \Sigma. \end{align*}$$

For concreteness, we can fix isomorphisms as follows:

$$\begin{align*}\mathbf{i} \mapsto \begin{pmatrix} 0 & 1 \\ \sigma_v(u) & 0 \end{pmatrix}, \quad \mathbf{j}_i \mapsto \begin{pmatrix} \sqrt{\sigma_v(J_i)} & 0 \\ 0 & -\sqrt{\sigma_v(J_i)}\end{pmatrix}, \quad v\in \Sigma_{\infty}, \end{align*}$$

where for $v\in \Sigma _{\infty }\smallsetminus \Sigma $ , the notation $\sqrt {\sigma _v(J_i)}$ stands for $\sqrt {|\sigma _v(J_i)|}i$ . We will use these isomorphisms to identify

$$\begin{align*}G_{B_i} (\mathbb{R}) \simeq \prod_{v\in\Sigma} \mathrm{GL}_2(\mathbb{R}) \times \prod_{v\in \Sigma_{\infty}\smallsetminus \Sigma} {\mathbb{H}}^\times. \end{align*}$$

Define Shimura data associated to $B_1$ and $B_2$ as in §3. Namely, take the $G_{B_i}(\mathbb {R})$ -conjugacy class of the homomorphisms

$$\begin{align*}h_i: \mathbb{S} \rightarrow G_{B_i} (\mathbb{R}), \quad (h_i(z))_v= \iota (z) \text{ for }v\in \Sigma; \quad (h_i(z))_v =1 \text{ for }v\in \Sigma_{\infty} \smallsetminus \Sigma, \end{align*}$$

where $\iota : \mathbb {C}^\times \rightarrow \mathrm {GL}_2(\mathbb {R})$ is defined as in equation (3.1). Note that for $v\in \Sigma $ , $h_{i,v}$ is $(B_i\otimes _{F,\sigma _v}\mathbb {R})^\times $ -conjugate to the embedding $\iota ^{\prime }_v:\mathbb {C}^\times \simeq (E\otimes _{F,\sigma _v} \mathbb {R})^\times \subset (B\otimes _{F,\sigma _v} \mathbb {R})^\times $ , where the first of these isomorphisms is induced from the map $E\otimes _{F,\sigma _v} \mathbb {R} \simeq \mathbb {C}$ sending $\mathbf {i} \mapsto \sqrt {|\sigma _v(u)|}i$ . We denote this latter isomorphism by $\sigma _v$ as well.

Let $X_{B_1}$ and $X_{B_2}$ denote the associated Shimura varieties. The Shimura data for the other groups are defined as follows. For $(B_i^\times \times E^\times )/F^\times \simeq \mathrm {GU}_E(\mathbf {V}_i)$ with $(\beta ,\alpha )\mapsto (x \mapsto \beta x \alpha ^{-1})$ ,

$$\begin{align*}(h_i(z))_v= (\iota (z),1) \text{ for }v\in \Sigma; \quad (h_i(z))_v =1 \text{ for }v\in \Sigma_{\infty} \smallsetminus \Sigma. \end{align*}$$

In the basis $(1_{B_i},\mathbf {j}_i)$ of $\mathbf {V}_i$ , the map $(B_i^\times \times E^\times )/F^\times \rightarrow \mathrm {GU}_E(\mathbf {V}_i)$ is given by

$$\begin{align*}(\alpha + \beta \mathbf{j}_i , \delta) \mapsto \delta^{-1} \begin{pmatrix} \alpha & J_i \beta \\ \bar{\beta} & \bar{\alpha} \end{pmatrix} \in \mathrm{GL}_2(E), \quad \alpha, \beta, \delta \in E. \end{align*}$$

In the basis $(\mathbf {w}_{i1}, \mathbf {w}_{i2})= (1_{B_i},\frac {1}{\sqrt {|\sigma _v(J_i)|}}\mathbf {j}_i)$ of $\mathbf {V}_{i,v} :=\mathbf {V}_i \otimes _{E,\sigma _v} \mathbb {C}$ , the Hermitian form is diagonal, given by the matrix $\begin {pmatrix} 1 & \\ & \pm 1 \end {pmatrix}$ with the sign being $- 1$ (resp. $+1$ ) if $v \in \Sigma $ (resp. $v \in \Sigma _{\infty } \smallsetminus \Sigma $ ).

Let $v\in \Sigma $ . The map $(B_i^\times \times E^\times )/F^\times \rightarrow \mathrm {GU}_E(\mathbf {V}_i)(\mathbb {R})_v$ is given by

$$\begin{align*}(\alpha + \beta \mathbf{j}_i , \delta) \mapsto \sigma_v(\delta)^{-1} \begin{pmatrix} \sigma_v(\alpha) &\sqrt{\sigma_v( J_i)} \sigma_v(\beta) \\ \sqrt{\sigma_v( J_i)} \sigma_v(\bar{\beta}) & \sigma_v(\bar{\alpha}) \end{pmatrix} \in \mathrm{GU}(1,1), \quad \alpha, \beta, \delta \in E. \end{align*}$$

In particular, the map $h_{i,v}: \mathbb {C}^\times \rightarrow \mathrm {GU}(1,1)$ is $\mathrm {GU}(1,1)$ -conjugate to the map $z\mapsto \begin {pmatrix} z & 0 \\ 0 & \bar {z} \end {pmatrix}$ .

For $\tilde {\mathcal {G}}=\mathrm {G} (\mathrm {U}_E(\mathbf {V}_1) \times \mathrm {U}_E(\mathbf {V}_2))$ , let h be defined by $h(z) = (h_1(z), h_2(z))$ . For , let h be defined by composing the map h for $\tilde {\mathcal {G}}$ with the inclusion . In the basis

$$\begin{align*}(\mathbf{w}_{11}, \mathbf{w}_{21}, \mathbf{w}_{12}, \mathbf{w}_{22})= (1_{B_1}, 1_{B_2}, \frac{1}{\sqrt{\sigma_v(J_1)}}\mathbf{j}_1, \frac{1}{\sqrt{\sigma_v(J_2)}}\mathbf{j}_2) \end{align*}$$

of $\mathbf {V}_v= \mathbf {V} \otimes _{E,\sigma _v} \mathbb {C}$ , the Hermitian form is $\mathrm {diag} (1,1,\pm 1,\pm 1)$ with the sign being $- 1$ (resp. $+1$ ) if $v \in \Sigma $ (resp. $v \in \Sigma _{\infty } \smallsetminus \Sigma $ ). For $v\in \Sigma $ , $h_v$ is -conjugate to the map

$$\begin{align*}z \mapsto \begin{pmatrix} z {\mathbf{1}}_2 & \\ & \bar{z} {\mathbf{1}}_2 \end{pmatrix}, \end{align*}$$

while for $v\in \Sigma _{\infty }\smallsetminus \Sigma $ , $h_v$ is trivial.

For $\tilde {\mathcal {G}}_B=\mathrm {G} (\mathrm {U}_B(V) \times \mathrm {U}_B (V_0))=\mathrm {G} ((B_1^\times \times B_2^\times )/F^\times \times E^\times )$ , we take

$$\begin{align*}h(z)_v =((\iota(z),\iota(z)),z\bar{z}) \end{align*}$$

for $v \in \Sigma $ and $h(z)_v =1$ otherwise. For

, we take h to be the map obtained by composing h for $\tilde {\mathcal {G}}_B$ with the inclusion

. Thus, for $v\in \Sigma $ , the action of $h(z)_v$ on $\tilde {V}_v= V_v \oplus V_{0,v} = (\mathbf {V}_{1,v} \otimes _{\mathbb {C}} \mathbf {V}_{2,v}) \oplus (\wedge ^2_{\mathbb {C}} \mathbf {V}_{1,v} \oplus \wedge ^2_{\mathbb {C}} \mathbf {V}_{2,v})$ is given by $\iota (z) \otimes \iota (z) \oplus z\bar {z}$ . To compute the

-conjugacy class, we may replace $\iota (z)$ by $\iota ^{\prime }_v (z)$ . The action of $\iota ^{\prime }_v(z)$ on $\mathbf {V}_{i,v}$ is diagonalizable, given by $\begin {pmatrix} z & \\ & \bar {z}\end {pmatrix}$ in the basis $(\mathbf {w}_{i1},\mathbf {w}_{i2})$ , that is,

$$\begin{align*}\iota^{\prime}_v(z) \mathbf{w}_{i1} =\mathbf{w}_{i1} z, \quad \iota^{\prime}_v(z)\mathbf{w}_{i2} =\mathbf{w}_{i2} \bar{z}. \end{align*}$$

Now, $(\mathbf {w}_{11} \otimes \mathbf {w}_{21}, \mathbf {w}_{12} \otimes \mathbf {w}_{21}, \mathbf {w}_{11} \wedge \mathbf {w}_{12})$ is a $B\otimes _{F,\sigma _v} \mathbb {R}$ -basis of $\tilde {V}_v$ and in this basis, the action of $h(z)_v$ is diagonal, given by $\mathrm {diag}(z^2, z\bar {z}, z \bar {z})$ . From this description, it is clear that under the canonical isomorphism

the chosen Shimura data are identified.

12.2.3 Local systems

We consider a local system $\tilde {\mathbb {V}}_\rho $ on

associated with a finite-dimensional representation $\rho $ of

. To construct this local system, we first fix isomorphisms $B\otimes _F F_v \simeq \mathrm {M}_2 (\mathbb {R})$ for all infinite places v of F. Then to each infinite place v, as in §6 (and [Reference Ichino and Prasanna30, §C.2]), we can associate orthogonal spaces $\tilde {V}^\dagger _v = V^\dagger _v \oplus V^\dagger _{0,v}$ such that

$$\begin{align*}\mathrm{GU}_{B_v}(\tilde{V}_v)^0 \simeq \mathrm{GSO} (\tilde{V}^\dagger_v), \quad \mathrm{GU}_{B_v}(V_v)^0 \simeq \mathrm{GSO} (V^\dagger_v), \quad \mathrm{GU}_{B_v}(V_{0,v})^0 \simeq \mathrm{GSO} (V^\dagger_{0,v}). \end{align*}$$

Recall that

where for $p+q$ even,

$$\begin{align*}\mathrm{GSO}(p,q) = \{ g \in \mathrm{GL}_{p+q}(\mathbb{R}) \, | \, {}^t g I_{p,q} g = \nu(g) \cdot I_{p,q}, \, \det g = \nu(g)^{\frac{p+q}{2}} \} \end{align*}$$

with

$$\begin{align*}I_{p,q} = \begin{pmatrix} {\mathbf{1}}_p & \\ & -{\mathbf{1}}_q \end{pmatrix}. \end{align*}$$

The local system is then associated to the algebraic representation $\tilde {\mathbb {V}}_{\rho ,\mathbb {C}}:=\otimes _v \mathscr {H}^{k_v-2} (\tilde {V}_v^\dagger )$ of

, where for $\ell $ even, we have

$$\begin{align*}\mathscr{H}^{\ell}:= \ker(\mathrm{Sym}^\ell \rightarrow \mathrm{Sym}^{\ell-2}) \otimes \nu(\cdot)^{-\ell/2}. \end{align*}$$

Note that by §8.2 (for $\ell =k-2$ ) the restriction of this representation to $\mathrm {SO}(4,2)$ is irreducible with highest weight $(k-2,0,0)$ . Via the isomorphism given by Corollary 6.3, this corresponds (for $\ell =k-2$ ) to the irreducible representation of $\mathrm {U}(2,2)$ with highest weight

$$\begin{align*}\left( \tfrac{k}{2}-1, \tfrac{k}{2}-1, -\tfrac{k}{2}+1,-\tfrac{k}{2}+1\right). \end{align*}$$

A similar statement holds for the places $v\in \Sigma _{\infty } \smallsetminus \Sigma $ , and for the pair $(\mathrm {SO}(0,6), \mathrm {U}(4,0))$ . Thus, the local system $\tilde {\mathbb {V}}_{\rho ,\mathbb {C}}$ is isomorphic to the local system $\mathbb {V}_{\underline {k},\mathbb {C}}$ considered in §11.3.

Proposition 12.2. The $\mathbb {C}$ -vector space $\tilde {\mathbb {V}}_{\rho ,\mathbb {C}}$ contains a $\mathbb {Q}(\underline {k})$ -subspace $\tilde {\mathbb {V}}_\rho $ that is stable by the action of and such that $\tilde {\mathbb {V}}_\rho \otimes _{\mathbb {Q}(\underline {k})} \mathbb {C} =\tilde {\mathbb {V}}_{\rho ,\mathbb {C}}$ . Moreover, such a subspace is unique up to homothety.

Proof. Fix an infinite place v of F. Then the representation $\mathscr {H}^{k_v-2} (\tilde {V}_v^\dagger )$ is defined over $\sigma _v(F)$ by [Reference Tits64], Théorème 3.3, since the highest weight is both invariant by $\operatorname {Gal}(\overline {\mathbb {Q}}/\sigma _v(F))$ and lies in the root lattice. Taking the tensor induction over all places $v\in \Sigma _{\infty }$ yields a $\mathbb {Q}(\underline {k})$ -structure on $\tilde {\mathbb {V}}_{\rho ,\mathbb {C}}$ . The uniqueness up to homothety follows from the irreducibility of $\tilde {\mathbb {V}}_{\rho ,\mathbb {C}}$ .

12.2.4 Auxiliary modular form

Let $\tilde {\tau }$ be an irreducible automorphic representation of $B^\times (\mathbb {A}) $ corresponding to a holomorphic Hilbert modular form of weights $(\underline {k} + {\mathbf {1}},r)$ (with some odd integer r) and central character $\xi _E$ . Let $B^\times (\mathbb {A})^+$ denote the subgroup of $B^\times (\mathbb {A})$ consisting of elements with positive reduced norm at every infinite place. The restriction of $\tilde {\tau }$ to $B^\times (\mathbb {A})^+$ splits up as a sum of $2^{[F:\mathbb {Q}]}$ representations, characterized by the local component at the $[F:\mathbb {Q}]$ infinite places being either holomorphic or antiholomorphic discrete series. We let $\tau $ be the irreducible summand whose local component is antiholomorphic for $v\in \Sigma $ and holomorphic for $v\not \in \Sigma $ , twisted by a (half-integer power of) the norm character to make it unitary.

12.2.5 Theta lift to

Let $\theta _{\tilde {\varphi }} (\phi )$ denote the element in

constructed in §10.4, with an element $\phi $ in the space $\tau $ and a Schwartz form $\tilde {\varphi }$ . (Note that in that section, the group

is simply denoted by $\tilde {G}$ . Then $\theta _{\tilde {\varphi }} (\phi )$ corresponds to a class

via the isomorphism

12.2.6 Pull back to $\tilde {\mathcal {G}}_B$

Pull back $\xi _\tau $ to a class $ i^* \xi _\tau $ in $H^{2d}(\mathrm {Sh}_{\tilde {\mathcal {G}}_B}, \tilde {\mathbb {V}}_{\rho ,\mathbb {C}})$ . Decompose $\tilde {\mathbb {V}}_{\rho ,\mathbb {C}}$ into a sum of irreducibles (as a representation of $\tilde {\mathcal {G}}_B(\mathbb {R})$ ), and project to the irreducible component $\mathbb {V}_{\rho ,\mathbb {C}}:=\otimes _v \mathscr {H}^{k_v-2} (V_v)$ , as in equation (8.3). Thus, $ i^* \xi _\tau $ can be viewed as an element of $H^{2d}(\mathrm {Sh}_{\tilde {\mathcal {G}}_B}, \mathbb {V}_{\rho ,\mathbb {C}})$ . Note that the $\mathbb {Q}(\underline {k})$ -rational structure on $\tilde {\mathbb {V}}_{\rho ,\mathbb {C}}$ can be chosen such that the projection map carries it into the $\mathbb {Q}(\underline {k})$ -rational structure on $\mathbb {V}_{\rho ,\mathbb {C}}$ .

12.2.7 Auxiliary character

For a finite order Hecke character $\eta $ of $T_1(\mathbb {A})$ (see §4.3), we take the class $c_\eta \in H^0 (\mathrm {Sh}_{\tilde {\mathcal {G}}_B}, \mathbb {Q}(\eta ))$ and cup it with $ i^* \xi _\tau $ , to get

$$\begin{align*}\tilde{\xi}_{\tau,\eta} :=i^*\xi_\tau \cup c_\eta \in H^{2d}(\mathrm{Sh}_{\tilde{\mathcal{G}}_B}, \mathbb{V}_{\rho,\mathbb{C}}). \end{align*}$$

12.2.8 Push forward to $\mathrm {Sh}_{G}$ and $\mathcal {K}$ -invariant projection

Push forward the class $\tilde {\xi }_{\tau ,\eta }$ to $\mathrm {Sh}_{G}$ . Pick an open compact $\mathcal {K}$ of $Z(\mathbb {A}_f)\backslash G(\mathbb {A}_f)$ , and take the $\mathcal {K}$ -invariant projection $\tilde {\xi }_{\tau ,\eta ,\mathcal {K}}$ .

12.2.9 Pull back to $\mathrm {Sh}_{B_1} \times \mathrm {Sh}_{B_2}$

Take an open compact subgroup $\mathcal {K}_1 \times \mathcal {K}_2 \subset B_1(\mathbb {A}_f) \times B_2 (\mathbb {A}_f)$ whose image under the natural map to $Z(\mathbb {A}_f)\backslash G(\mathbb {A}_f)$ is contained in $\mathcal {K}$ . Then pull back to $\mathrm {Sh}_{B_1,\mathcal {K}_1} \times \mathrm {Sh}_{B_2,\mathcal {K}_2}$ to get the class

$$\begin{align*}\xi_{\tau,\eta} := j^* p_{1,*} \tilde{\xi}_{\tau,\eta,\mathcal{K}} \in H^{2d} (\mathrm{Sh}_{B_1,\mathcal{K}_1} \times \mathrm{Sh}_{B_2,\mathcal{K}_2}, \mathbb{V}_{\rho,\mathbb{C}}). \end{align*}$$

12.2.10 Project to $[\pi _1 , \pi _2] $ -component

On $\mathrm {Sh}_{B_1} \times \mathrm {Sh}_{B_2}$ , we have

$$\begin{align*}\mathbb{V}_{\rho} \simeq \mathbb{V}_{\underline{k}} \boxtimes \mathbb{V}_{\underline{k}}. \end{align*}$$

Thus

$$\begin{align*}H^{2d} (X_{B_1,\mathcal{K}_1} \times X_{B_2,\mathcal{K}_2}, \mathbb{V}_{\rho,\mathbb{C}}) = \bigoplus_{\tilde{\pi}_1, \tilde{\pi}_2} {\mathcal{H}}_{\mathcal{K}_1,\mathcal{K}_2}^{2d} [\tilde{\pi}_1,\tilde{\pi}_2], \end{align*}$$

where

$$\begin{align*}{\mathcal{H}}_{\mathcal{K}_1,\mathcal{K}_2}^{2d} [\tilde{\pi}_1,\tilde{\pi}_2]= ( \tilde{\pi}_{1,f}^{\mathcal{K}_1}\otimes \tilde{\pi}_{2,f}^{\mathcal{K}_2} ) \otimes H^{2d} (X_{B_1,\mathcal{K}_1} \times X_{B_2,\mathcal{K}_2} , \mathbb{V}_{\underline{k},\mathbb{C}} \boxtimes \mathbb{V}_{\underline{k},\mathbb{C}} )_{\tilde{\pi}_1 \boxtimes \tilde{\pi}_2} \end{align*}$$

and $\tilde {\pi }_1, \tilde {\pi }_2$ range over automorphic representations of $B_1^\times (\mathbb {A})$ and $B_2^\times (\mathbb {A})$ such that $\tilde {\pi }_1 \boxtimes \tilde {\pi }_2$ contributes to the cohomology of the local system $V_{\rho ,\mathbb {C}}$ . Then

$$\begin{align*}\epsilon_\pi (\xi_{\tau,\eta}) \in {\mathcal{H}}_{\mathcal{K}_1,\mathcal{K}_2}^{2d} [\pi_1,\pi_2] \end{align*}$$

is defined to be the projection to the $[\pi _1 , \pi _2]$ -component. Note that

$$\begin{align*}H^{2d} (X_{B_1,\mathcal{K}_1} \times X_{B_2,\mathcal{K}_2} , \mathbb{V}_{\underline{k},\mathbb{C}} \boxtimes \mathbb{V}_{\underline{k},\mathbb{C}} )_{\pi_1 \boxtimes \pi_2} = H^d (X_{B_1,\mathcal{K}_1}, \mathbb{V}_{\underline{k},\mathbb{C}})_{\pi_1} \otimes H^d (X_{B_2,\mathcal{K}_2}, \mathbb{V}_{\underline{k},\mathbb{C}})_{\pi_2}. \end{align*}$$

12.2.11 Contraction with $v_1 \otimes v_2$

Though we are assuming that $\pi ^\vee \simeq \pi $ , below we distinguish between them for purposes of clarity. Pick nonzero elements $v_1 \in (\pi _1^{\vee })^{f,\mathcal {K}_1}$ , $v_2 \in (\pi _2^\vee )^{f,\mathcal {K}_2}$ such that $v_1 \otimes v_2$ is $\mathcal {K}$ -invariant. Then contracting $v_1 \otimes v_2$ with $\epsilon _\pi (\xi _\eta )$ gives an element

$$\begin{align*}\xi := \langle \epsilon_\pi(\xi_{\tau,\eta}), v_1 \otimes v_2 \rangle \in H^d (\mathrm{Sh}_{B_1} , \mathbb{V}_{\underline{k},\mathbb{C}})_{\pi_1} \otimes H^d (\mathrm{Sh}_{B_2} , \mathbb{V}_{\underline{k},\mathbb{C}})_{\pi_2}. \end{align*}$$

12.3 The construction of a Hodge–Tate class

Note that $\xi $ induces a map (for the moment of $\mathbb {C}$ -vector spaces!)

$$\begin{align*}I_\xi: H^d (\mathrm{Sh}_{B_1} , \mathbb{V}_{\underline{k},\mathbb{C}})_{\pi_1^\vee} \simeq H^d (\mathrm{Sh}_{B_1} , \mathbb{V}_{\underline{k},\mathbb{C}})_{\pi_1}^\vee \xrightarrow{{} \cdot \xi} H^d (\mathrm{Sh}_{B_2} , \mathbb{V}_{\underline{k},\mathbb{C}})_{\pi_2}. \end{align*}$$

Note also that $\xi $ depends on the choices of the following data:

$$\begin{align*}\Upsilon: = (E,\tilde{\tau}, \phi, \tilde{\varphi}, \eta, \mathcal{K}, \mathcal{K}_1, \mathcal{K}_2, v_1 ,v_2 ). \end{align*}$$

Proposition 12.3. There exists a choice of data $\Upsilon $ such that $I_\xi $ is an isomorphism of $\mathbb {C}$ -vector spaces:

$$\begin{align*}H^d (\mathrm{Sh}_{B_1} , \mathbb{V}_{\underline{k},\mathbb{C}})_{\pi_1^\vee} \simeq H^d (\mathrm{Sh}_{B_2} , \mathbb{V}_{\underline{k},\mathbb{C}})_{\pi_2}. \end{align*}$$

Proof. By Matsushima’s formula, there are canonical isomorphisms:

$$ \begin{align*} H^d (\mathfrak{g}_1, K_1; \pi_{B_1,\infty}^\vee \otimes \rho_{\underline{k}} ) &\simeq H^d (\mathrm{Sh}_{B_1} , \mathbb{V}_{\underline{k}})_{\pi_1^\vee}, \\ H^d (\mathfrak{g}_2, K_2; \pi_{B_2,\infty} \otimes \rho_{\underline{k}} ) &\simeq H^d (\mathrm{Sh}_{B_2} , \mathbb{V}_{\underline{k}})_{\pi_2}, \end{align*} $$

and so we just need to check that the data $\Upsilon $ can be picked so that the induced map

$$\begin{align*}I_\xi: H^d (\mathfrak{g}_1, K_1; \pi_{B_1,\infty}^\vee \otimes \rho_{\underline{k}} ) ^\vee \rightarrow H^d (\mathfrak{g}_2, K_2; \pi_{B_2,\infty} \otimes \rho_{\underline{k}} ) \end{align*}$$

is an isomorphism. But this is exactly the content of Proposition 10.1. The only point to note is that one can in fact pick a CM extension $E/F$ satisfying the conditions (i) through (iii) in the statement of the proposition. But equations (ii) and (iii) hold for all but a finite number of finite places, so it is obvious that there exists E satisfying the needed conditions.

We now come to the proof of the main theorem (Theorem 1 of the introduction and its generalization, Theorem 3.2.)

Theorem 12.4.

  1. (i) There exists a Hodge class $\xi _0 \in V_{B_1,\pi _1} \otimes V_{B_2,\pi _2}(d) $ such that the induced map

    $$\begin{align*}V_{B_1,\pi_1} \simeq V_{B_1,\pi_1}^\vee (-d) \xrightarrow{{} \cdot \xi_0} V_{B_2,\pi_2} \end{align*}$$
    is an isomorphism of L-Hodge structures.
  2. (ii) Assume Kottwitz’s conjecture for Shimura varieties attached to unitary similitude groups. Then the Hodge class $\xi _0$ can be chosen such that it belongs to $\mathcal {HT} (V_{B_1,\pi _1} \otimes V_{B_2,\pi _2} (d))$ so that the induced map

    $$\begin{align*}V_{B_1,\pi_1} \otimes \mathbb{Q}_\ell \simeq (V_{B_1,\pi_1} \otimes \mathbb{Q}_\ell) ^\vee (-d) \xrightarrow{{} \cdot \xi_0} V_{B_2,\pi_2} \otimes \mathbb{Q}_\ell \end{align*}$$
    is an isomorphism of $G_{F_\Sigma }$ -modules for all rational primes $\ell $ .

Proof. The construction outlined in §12.2.1 to §12.2.11 above gives a map (for any open compact subgroup $\tilde {\mathcal {K}} $ of

)

such that $\operatorname {Res}_{\mathbb {C}}$ sends

$$\begin{align*}\xi_\tau (d) \mapsto \xi(d) \mapsto I_{\xi (d)}. \end{align*}$$

Let $\mathbb {I}$ be the kernel of the unramified part of the Hecke algebra (at level $\tilde {\mathcal {K}}$ ) acting on $\Pi ^{\tilde {\mathcal {K}}}$ so that

where $\sigma $ ranges over a set of automorphisms of $\mathbb {C}/\mathbb {Q}$ such that the $\Pi ^\sigma $ are the distinct conjugates of $\Pi $ . Since $\tilde {\mathbb {V}}_\rho $ is defined over $\mathbb {Q}(\underline {k})$ , we may consider the $\mathbb {Q}(\underline {k})$ -Hodge structure

Now,

and $\operatorname {Res}_{\mathbb {C}} (\xi _\tau (d))$ is an isomorphism; hence, there exists an element

such that $\operatorname {Res}(\Xi _\tau (d))$ is an isomorphism. Indeed, if we pick a $\mathbb {Q}(\underline {k})$ -basis $(x_1, \ldots , x_r)$ for

, then this is also a $\mathbb {C}$ -basis for

. Expanding $\xi _\tau $ in this basis:

$$\begin{align*}\xi_\tau = a_1x_1 + \cdots + a_r x_r \end{align*}$$

we see that the $(a_1, \ldots , a_r)\in \mathbb {C}^r$ satisfies $\det (I_{\sum _i a_i x_i} (d) ) \neq 0 $ . Since this is a polynomial function in the $a_i$ , it follows that there exist $b_i\in \mathbb {Q}(\underline {k})$ with $\det (I_{\sum _i b_i x_i} (d) ) \neq 0 $ . Taking $\Xi _\tau = \sum _i b_i x_i$ , we see that $\operatorname {Res}(\Xi _\tau (d))$ is an isomorphism. By a similar argument using a determinant, we can replace the class $c_\eta $ in the original construction by some $\mathbb {Q}$ -linear combination c of the fundamental classes of the components of $\mathrm {Sh}_{\tilde {\mathcal {G}}_B}$ . (We note that since the action of $\operatorname {Gal} (\overline {\mathbb {Q}}/F_\Sigma )$ on the components of $\mathrm {Sh}_{\tilde {\mathcal {G}}_B}$ is trivial, the class c is $\operatorname {Gal} (\overline {\mathbb {Q}}/F_\Sigma )$ -invariant.) The class $\operatorname {Res}(\Xi _\tau (d))$ then has coefficients in L. (The only step where the coefficients might be enlarged is the projection to the $\pi _1 \boxtimes \pi _2$ -component, and the coefficient field L of $\pi $ contains $\mathbb {Q}(\underline {k})$ .) By Corollary 11.7, the class $\Xi _\tau (d)$ is a $\mathbb {Q}(\underline {k})$ -rational Hodge class; hence, $\xi _0 := \operatorname {Res}(\Xi _\tau (d))$ is an L-rational Hodge class. Assuming Kottwitz’s conjecture, by Proposition 11.8, the action of $\operatorname {Gal} (\overline {\mathbb {Q}}/F_\Sigma )$ on $\Xi _\tau (d)$ is trivial and so the same is true for $\xi _0$ .

A Splittings

A.1 Setup

Let F be a number field and B a quaternion division algebra over F. Let E be a quadratic extension of F which embeds into B. Let $*$ be the main involution on B and $\rho $ the nontrivial Galois automorphism of E over F. We write $E = F + F \mathbf {i}$ and $B = E + E \mathbf {j}$ for some trace zero elements $\mathbf {i} \in E^\times $ and $\mathbf {j} \in B^\times $ . Let $\operatorname {pr} : B \rightarrow E$ be the associated projection. Put $u = \mathbf {i}^2 \in F^\times $ and $J = \mathbf {j}^2 \in F^\times $ . Fix a nontrivial additive character $\psi $ of $\mathbb {A}/F$ and a character $\chi $ of $\mathbb {A}_E^\times / E^\times $ such that $\chi |_{\mathbb {A}^\times } = \xi _E$ , where $\xi _E$ is the quadratic character of $\mathbb {A}^\times / F^\times $ associated to $E/F$ by class field theory.

We consider an m-dimensional right B-space V equipped with a skew-Hermitian form $\langle \cdot , \cdot \rangle : V \times V \rightarrow B$ given by

(A.1) $$ \begin{align} \langle e_1 x_1 + \dots + e_m x_m, e_1 y_1 + \dots + e_m y_m \rangle = x_1^* \cdot \kappa_1 \mathbf{i} \cdot y_1 + \dots + x_m^* \cdot \kappa_m \mathbf{i} \cdot y_m \end{align} $$

for some basis $e_1,\dots ,e_m$ of V and some $\kappa _1, \dots , \kappa _m \in F^\times $ . We denote by $\mathrm {GU}(V)$ the unitary similitude group of V and by $\nu : \mathrm {GU}(V) \rightarrow F^\times $ the similitude character

$$\begin{align*}\mathrm{GU}(V) = \{ g \in \mathrm{GL}(V) \, | \, \langle gv, gv' \rangle = \nu(g) \cdot \langle v, v' \rangle \text{ for all } v, v' \in V \}, \end{align*}$$

where $\mathrm {GL}(V)$ acts on V on the left. We have a natural embedding

$$\begin{align*}E^\times \hookrightarrow \mathrm{GU}(V), \end{align*}$$

where we may regard $\alpha \in E^\times $ as an element in $\mathrm {GU}(V)$ given by $e_i \mapsto e_i \alpha $ for all i.

Let $W = B$ be a left B-space equipped with a Hermitian form $\langle \cdot , \cdot \rangle : W \times W \rightarrow B$ given by

$$\begin{align*}\langle x, y \rangle = x \cdot y^*. \end{align*}$$

We denote by $\mathrm {GU}(W)$ the unitary similitude group of W and by $\nu : \mathrm {GU}(W) \rightarrow F^\times $ the similitude character

$$\begin{align*}\mathrm{GU}(W) = \{ h \in \mathrm{GL}(W) \, | \, \langle wh, w'h \rangle = \nu(h) \cdot \langle w, w' \rangle \text{ for all } w, w' \in W \}, \end{align*}$$

where $\mathrm {GL}(W)$ acts on W on the right. Then we have $\mathrm {GU}(W) \simeq B^\times $ .

Let $\mathbb {V} = V \otimes _B W$ be a $4m$ -dimensional F-space equipped with a symplectic form

$$\begin{align*}\langle \hspace{-1mm} \langle \cdot, \cdot \rangle \hspace{-1mm} \rangle := \frac{1}{2} \operatorname{tr}_{B/F} \left( \langle \cdot, \cdot \rangle \otimes \langle \cdot, \cdot \rangle^* \right). \end{align*}$$

Then we have a natural homomorphism

(A.2) $$ \begin{align} \mathrm{G}(\mathrm{U}(V) \times \mathrm{U}(W)) \longrightarrow \mathrm{Sp}(\mathbb{V}), \end{align} $$

where

$$\begin{align*}\mathrm{G}(\mathrm{U}(V) \times \mathrm{U}(W)) = \{ (g, h) \in \mathrm{GU}(V) \times \mathrm{GU}(W) \, | \, \nu(g) = \nu(h) \} \end{align*}$$

and $\mathrm {GL}(V) \times \mathrm {GL}(W)$ acts on $\mathbb {V}$ on the right:

$$\begin{align*}(v \otimes w) \cdot (g, h) := g^{-1} v \otimes w h. \end{align*}$$

Let $\mathcal {G}$ be a subgroup of $\mathrm {G}(\mathrm {U}(V) \times \mathrm {U}(W))$ defined by

$$\begin{align*}\mathcal{G} = \{ (g, h) \in \mathrm{GU}(V)^0 \times \mathrm{GU}(W) \, | \, \nu(g) = \nu(h) \in \mathrm{N}_{E/F}(E^\times) \}. \end{align*}$$

We take a complete polarization $\mathbb {V} = \mathbb {X} \oplus \mathbb {Y}$ defined by

$$ \begin{align*} \mathbb{X} & = F \cdot e_1 \otimes 1 + \dots + F \cdot e_m \otimes 1 + F \cdot e_1 \otimes \mathbf{j} + \dots + F \cdot e_m \otimes \mathbf{j}, \\ \mathbb{Y} & = F \cdot e_1 \otimes \mathbf{i} + \dots + F \cdot e_m \otimes \mathbf{i} + F \cdot e_1 \otimes \mathbf{i}\mathbf{j} + \dots + F \cdot e_m \otimes \mathbf{i}\mathbf{j}. \end{align*} $$

A.2 Splitting over $\mathcal {G}$

For each place v of F, let $\mathrm {Mp}(\mathbb {V}_v)$ be the metaplectic group over $F_v$ :

$$\begin{align*}1 \longrightarrow \mathbb{C}^1 \longrightarrow \mathrm{Mp}(\mathbb{V}_v) \longrightarrow \mathrm{Sp}(\mathbb{V}_v) \longrightarrow 1. \end{align*}$$

Then $\mathrm {Mp}(\mathbb {V}_v)$ can be realized by a $2$ -cocycle $z_{\mathbb {Y}_v}$ relative to $\mathbb {Y}_v$ and $\psi _v$ (see, e.g., [Reference Ranga Rao58], [Reference Ichino and Prasanna30, §3.2.2]). For almost all v, there exists a map $s_{\mathbb {Y}_v}:K_v \rightarrow \mathbb {C}^1$ , where $K_v$ is the standard maximal compact subgroup of $\mathrm {Sp}(\mathbb {V}_v)$ , such that

$$\begin{align*}z_{\mathbb{Y}_v}(k_1,k_2) = \frac{s_{\mathbb{Y}_v}(k_1 k_2)}{s_{\mathbb{Y}_v}(k_1) s_{\mathbb{Y}_v}(k_2)} \end{align*}$$

for $k_1, k_2 \in K_v$ (see, e.g., [Reference Ichino and Prasanna30, §3.2.3]).

Proposition A.1. For all v, there exists a map $s_v:\mathcal {G}_v \rightarrow \mathbb {C}^1$ satisfying the following conditions:

  1. (i) For $\mathbf {g}_1, \mathbf {g}_2 \in \mathcal {G}_v$ , we have

    $$\begin{align*}z_{\mathbb{Y}_v}(\mathbf{g}_1,\mathbf{g}_2) = \frac{s_v(\mathbf{g}_1 \mathbf{g}_2)}{s_v(\mathbf{g}_1) s(\mathbf{g}_2)}. \end{align*}$$
    Here, by abuse of notation, we write $\mathbf {g}_i$ on the left-hand side for the image of $\mathbf {g}_i$ in $\mathrm {Sp}(\mathbb {V}_v)$ under equation (A.2).
  2. (ii) For $\mathbf {z} = (z, z)$ with $z \in F_v^\times $ and $\mathbf {g} \in \mathcal {G}_v$ , we have

    $$\begin{align*}s_v(\mathbf{z} \mathbf{g}) = \xi_{E_v}(z)^m \cdot s_v(\mathbf{g}). \end{align*}$$
  3. (iii) For almost all v, we have

    $$\begin{align*}s_v|_{\mathcal{G}_v \cap K_v} = s_{\mathbb{Y}_v}|_{\mathcal{G}_v \cap K_v}. \end{align*}$$
  4. (iv) For $\gamma \in \mathcal {G}(F)$ , we have

    $$\begin{align*}\prod_v s_v(\gamma) = 1. \end{align*}$$

As in [Reference Ichino and Prasanna30, §3.3], Proposition A.1 enable us to define a Weil representation $\omega _\psi $ of $\mathcal {G}(\mathbb {A})$ on the Schwartz space $\mathcal {S}(\mathbb {X}(\mathbb {A}))$ . Moreover, for any $\varphi \in \mathcal {S}(\mathbb {X}(\mathbb {A}))$ , the associated theta function

$$\begin{align*}\Theta_{\varphi}(\mathbf{g}) := \sum_{x \in \mathbb{X}} \omega_{\psi}(\mathbf{g}) \varphi(x) \end{align*}$$

on $\mathcal {G}(\mathbb {A})$ is left $\mathcal {G}(F)$ -invariant.

Remark A.2. Suppose that V is the three-dimensional skew-Hermitian right B-space as in §5.2. Then V satisfies the condition (A.1) and we may apply the above construction. Note that $\nu (\mathrm {GU}(V_v)^0) = \mathrm {N}_{E_v/F_v}(E_v^\times )$ for all v so that

$$\begin{align*}\mathcal{G}(\mathbb{A}) = \mathrm{G}(\mathrm{U}(V) \times \mathrm{U}(W))^0(\mathbb{A}). \end{align*}$$

The proof of Proposition A.1 will be given in §A.3–§A.6 below. From now on, we fix a place v of F and suppress the subscript v from the notation.

A.3 The doubling method for $\mathrm {U}(V)$

We consider the doubled space $V^\square = V \oplus V$ equipped with a skew-Hermitian form:

$$\begin{align*}\langle (v_1,v_2), (v_1',v_2') \rangle := \langle v_1, v_1' \rangle - \langle v_2, v_2' \rangle. \end{align*}$$

Then we have a natural embedding

$$\begin{align*}\iota : \mathrm{G}(\mathrm{U}(V) \times \mathrm{U}(V)) \longrightarrow \mathrm{GU}(V^\square). \end{align*}$$

If $\mathbb {V}^\square = \mathbb {V} \oplus \mathbb {V}$ is the doubled space equipped with a symplectic form defined similarly as above, then we have a natural embedding

$$\begin{align*}\iota : \mathrm{Sp}(\mathbb{V}) \times \mathrm{Sp}(\mathbb{V}) \longrightarrow \mathrm{Sp}(\mathbb{V}^\square) \end{align*}$$

and an identification

$$\begin{align*}\mathbb{V}^\square = V^\square \otimes_B W. \end{align*}$$

We take a complete polarization $\mathbb {V}^\square = \mathbb {V}^\bigtriangledown \oplus \mathbb {V}^\triangle $ defined by

$$\begin{align*}\mathbb{V}^{\bigtriangledown} = \{ (x,-x) \, | \, x \in \mathbb{V} \}, \qquad \mathbb{V}^{\triangle} = \{ (x,x) \, | \, x \in \mathbb{V} \}. \end{align*}$$

Under the above identification, we have

$$\begin{align*}\mathbb{V}^{\bigtriangledown} = V^{\bigtriangledown} \otimes_B W, \qquad \mathbb{V}^{\triangle} = V^{\triangle} \otimes_B W, \end{align*}$$

where $V^\square = V^\bigtriangledown \oplus V^\triangle $ is the complete polarization over B defined similarly as above.

Now, we recall Kudla’s splitting over $\mathrm {U}(V^\square )$ , where we regard $\mathrm {U}(V^\square )$ as a subgroup of $\mathrm {Sp}(\mathbb {V}^\square )$ via the natural embedding. As in [Reference Ichino and Prasanna30, §C.3], we regard $V^\square $ as a left B-space and let $\mathrm {GL}(V^\square )$ act on $V^\square $ on the right:

$$ \begin{align*} x \cdot v & := v \cdot x^*, & x & \in B, \\ v \cdot g & := g^{-1} \cdot v, & g & \in \mathrm{GL}(V^\square). \end{align*} $$

Similarly, we regard W as a right B-space and let $\mathrm {GL}(W)$ act on W on the left. Then we have an identification

$$\begin{align*}\mathbb{V}^\square = W \otimes_B V^\square. \end{align*}$$

Put

$$\begin{align*}\mathbf{v}_i = \frac{1}{2 \kappa_i \mathbf{i}} \cdot (e_i, -e_i), \qquad \mathbf{v}_i^* = (e_i, e_i) \end{align*}$$

so that $\langle \mathbf {v}_i, \mathbf {v}_j^* \rangle = \delta _{ij}$ . Using a basis $\mathbf {v}_1, \dots , \mathbf {v}_m, \mathbf {v}_1^*, \dots , \mathbf {v}_m^*$ of $V^\square $ , we identify $\mathrm {U}(V^\square )$ with

$$\begin{align*}\left\{ g \in \mathrm{GL}_{2m}(B) \, \left| \, g \begin{pmatrix} & {\mathbf{1}}_m \\ -{\mathbf{1}}_m & \end{pmatrix} {}^t g^* = \begin{pmatrix} & {\mathbf{1}}_m \\ -{\mathbf{1}}_m & \end{pmatrix} \right. \right\}. \end{align*}$$

Let $P_{V^\triangle }$ be the maximal parabolic subgroup of $\mathrm {U}(V^\square )$ stabilizing $V^\triangle $ :

$$\begin{align*}P_{V^\triangle} = \left\{ \left. \begin{pmatrix} a & * \\ & ({}^t a^*)^{-1} \end{pmatrix} \, \right| \, a \in \mathrm{GL}_m(B) \right\}. \end{align*}$$

We define a map

$$\begin{align*}\hat{s}_1 : \mathrm{U}(V^\square) \longrightarrow \mathbb{C}^1 \end{align*}$$

as follows:

  • If B is split, then we set

    $$\begin{align*}\hat{s}_1(g) = 1 \end{align*}$$
    for $g \in \mathrm {U}(V^\square )$ .
  • If B is ramified, then we set

    $$\begin{align*}\hat{s}_1(g) = (-1)^j \end{align*}$$
    for $g \in P_{V^\triangle } \tau _j P_{V^\triangle }$ with
    $$\begin{align*}\tau_j = \begin{pmatrix} {\mathbf{1}}_{m-j} & & & \\ & & & -{\mathbf{1}}_j \\ & & {\mathbf{1}}_{m-j} & \\ & {\mathbf{1}}_j & & \end{pmatrix}. \end{align*}$$

By [Reference Kudla40, Theorem 3.1, cases ${1}_{-}$ and ${2}_{+}$ ], we have

(A.3) $$ \begin{align} z_{\mathbb{V}^\triangle}(g_1,g_2) = \frac{\hat{s}_1(g_1 g_2)}{\hat{s}_1(g_1) \hat{s}_1(g_2)} \end{align} $$

for $g_1, g_2 \in \mathrm {U}(V^\square )$ .

Lemma A.3. For $\alpha \in E^{\times }$ and $g \in \mathrm {U}(V^\square )$ , we have

$$\begin{align*}\hat{s}_1(\alpha g \alpha^{-1}) = \hat{s}_1(g). \end{align*}$$

Proof. Since $\alpha p \alpha ^{-1} \in P_{V^\triangle }$ for $p \in P_{V^\triangle }$ and $\alpha \tau _j \alpha ^{-1} = \tau _j$ , the assertion follows.

Lemma A.4. Let $\alpha \in E^1$ . Then we have

$$\begin{align*}\hat{s}_1(\iota(\alpha, 1)) = 1 \end{align*}$$

if B is split, and

$$\begin{align*}\hat{s}_1(\iota(\alpha, 1)) = \begin{cases} 1 & \text{if } \alpha = 1, \\ (-1)^m & \text{if } \alpha \ne 1 \end{cases} \end{align*}$$

if B is ramified.

Proof. We may assume that B is ramified and $\alpha \ne 1$ . Then we have

$$\begin{align*}\begin{bmatrix} \mathbf{v}_i \cdot \iota(\alpha,1) \\ \mathbf{v}_i^* \cdot \iota(\alpha,1) \end{bmatrix} = A \cdot \begin{bmatrix} \mathbf{v}_i \\ \mathbf{v}_i^* \end{bmatrix}, \end{align*}$$

where

$$ \begin{align*} A & = \begin{pmatrix} \frac{1}{2}(\alpha+1) & \frac{1}{4 \kappa_i \mathbf{i}}(\alpha-1) \\ \kappa_i \mathbf{i} (\alpha-1) & \frac{1}{2}(\alpha+1) \end{pmatrix} \\ & = \begin{pmatrix} -\frac{1}{\kappa_i \mathbf{i} (\alpha^\rho - 1)} & \frac{1}{2} (\alpha+1) \\ & \kappa_i \mathbf{i} (\alpha-1) \\ \end{pmatrix} \cdot \begin{pmatrix} & -1 \\ 1 & \end{pmatrix} \cdot \begin{pmatrix} 1 & \frac{\alpha+1}{2 \kappa_i \mathbf{i} (\alpha-1)} \\ & 1 \end{pmatrix}. \end{align*} $$

This implies the assertion.

A.4 The doubling method for $\mathrm {U}(\mathbf {W})$

We consider a two-dimensional left E-space $\mathbf {W} = B$ equipped with a skew-Hermitian form

$$\begin{align*}(x, y) = -\mathbf{i} \cdot \operatorname{pr} (x \cdot y^*). \end{align*}$$

Then we have a natural embedding

$$\begin{align*}\mathrm{GU}(W) \hookrightarrow \mathrm{GU}(\mathbf{W}) \end{align*}$$

and an isomorphism $\mathrm {GU}(\mathbf {W}) \simeq (B^{\times } \times E^{\times }) / F^{\times }$ , where $B^{\times } \times E^{\times }$ acts on $\mathbf {W}$ by

$$\begin{align*}x \cdot (h, \alpha) = \alpha^{-1} \cdot x \cdot h. \end{align*}$$

We write $[h,\alpha ]$ for the image of $(h,\alpha )$ in $\mathrm {GU}(\mathbf {W})$ . Also, we consider the doubled space $\mathbf {W}^\square = \mathbf {W} \oplus \mathbf {W}$ equipped with a skew-Hermitian form

$$\begin{align*}((w_1,w_2), (w_1',w_2')) := (w_1, w_1') - (w_2, w_2'). \end{align*}$$

Then we have a natural embedding

$$\begin{align*}\iota : \mathrm{G}(\mathrm{U}(\mathbf{W}) \times \mathrm{U}(\mathbf{W})) \longrightarrow \mathrm{GU}(\mathbf{W}^\square). \end{align*}$$

Let $\mathbf {V} = e_1 E + \dots + e_m E$ be an m-dimensional right E-space equipped with a Hermitian form

$$\begin{align*}(e_1 x_1 + \dots + e_m x_m, e_1 y_1 + \dots + e_m y_m) = x_1^\rho \cdot \kappa_1 \cdot y_1 + \dots + x_m^\rho \cdot \kappa_m \cdot y_m. \end{align*}$$

Let $f: \mathbf {V} \otimes _E \mathbf {W} \rightarrow V \otimes _B W$ be the natural isomorphism. Then we have

(A.4) $$ \begin{align} f(v \otimes (w \cdot [h,\alpha])) = f(v \otimes w) \cdot (\alpha, h) \end{align} $$

for $h \in B^\times $ and $\alpha \in E^\times $ , and

$$\begin{align*}\langle \hspace{-1mm} \langle \cdot, \cdot \rangle \hspace{-1mm} \rangle \circ (f \times f) = \frac{1}{2} \operatorname{tr}_{E/F} ( (\cdot, \cdot) \otimes (\cdot, \cdot)^{\rho} ). \end{align*}$$

Hence, we may identify $\mathbf {V} \otimes _E \mathbf {W}$ with $\mathbb {V}$ and omit f from the notation. Similarly, we identify $\mathbf {V} \otimes _E \mathbf {W}^\square $ with $\mathbb {V}^\square $ .

Now, we recall Kudla’s splitting over $\mathrm {U}(\mathbf {W}^\square )$ , where we regard $\mathrm {U}(\mathbf {W}^\square )$ as a subgroup of $\mathrm {Sp}(\mathbb {V}^\square )$ via the natural embedding. We take a complete polarization $\mathbf {W}^\square = \mathbf {W}^\bigtriangledown \oplus \mathbf {W}^\triangle $ over E defined by

$$\begin{align*}\mathbf{W}^{\bigtriangledown} = \{ (w,-w) \, | \, w \in \mathbf{W} \}, \qquad \mathbf{W}^{\triangle} = \{ (w,w) \, | \, w \in \mathbf{W} \}. \end{align*}$$

Put

$$\begin{align*}\mathbf{w}_1 = - \frac{1}{2\mathbf{i}} \cdot (1, -1), \qquad \mathbf{w}_2 = \frac{1}{2 J \mathbf{i}} \cdot (\mathbf{j}, -\mathbf{j}), \qquad \mathbf{w}_1^* = (1, 1), \qquad \mathbf{w}_2^* = (\mathbf{j}, \mathbf{j}) \end{align*}$$

so that $(\mathbf {w}_i, \mathbf {w}_j^*) = \delta _{ij}$ . Using a basis $\mathbf {w}_1, \mathbf {w}_2, \mathbf {w}_1^*, \mathbf {w}_2^*$ of $\mathbf {W}^\square $ , we identify $\mathrm {U}(\mathbf {W}^\square )$ with

$$\begin{align*}\left\{ h \in \mathrm{GL}_4(E) \, \left| \, h \begin{pmatrix} & {\mathbf{1}}_2 \\ -{\mathbf{1}}_2 & \end{pmatrix} {}^t h^\rho = \begin{pmatrix} & {\mathbf{1}}_2 \\ -{\mathbf{1}}_2 & \end{pmatrix} \right. \right\}. \end{align*}$$

Let $P_{\mathbf {W}^\triangle }$ be the maximal parabolic subgroup of $\mathrm {U}(\mathbf {W}^\square )$ stabilizing $\mathbf {W}^\triangle $ :

$$\begin{align*}P_{\mathbf{W}^\triangle} = \left\{ \left. \begin{pmatrix} a & * \\ & ({}^t a^\rho)^{-1} \end{pmatrix} \, \right| \, a \in \mathrm{GL}_2(E) \right\}. \end{align*}$$

We define a map

$$\begin{align*}\hat{s}_2 : \mathrm{U}(\mathbf{W}^\square) \longrightarrow \mathbb{C}^1 \end{align*}$$

by setting

$$\begin{align*}\hat{s}_2(h) = \chi(x(h))^m \cdot \gamma^{-j} \end{align*}$$

for $h = p_1 \tau _j p_2$ with

$$\begin{align*}p_i = \begin{pmatrix} a_i & * \\ & ({}^t a_i^\rho)^{-1} \end{pmatrix} \in P_{\mathbf{W}^\triangle}, \qquad \tau_j = \begin{pmatrix} {\mathbf{1}}_{2-j} & & & \\ & & & -{\mathbf{1}}_j \\ & & {\mathbf{1}}_{2-j} & \\ & {\mathbf{1}}_j & & \end{pmatrix}, \end{align*}$$

where

$$\begin{align*}x(h) = \det(a_1 a_2)\quad \mod \mathrm{N}_{E/F}(E^\times) \end{align*}$$

and

$$\begin{align*}\gamma = (u, \det \mathbf{V})_F \cdot \gamma_F(-u, \tfrac{1}{2} \psi)^m \cdot \gamma_F(-1, \tfrac{1}{2} \psi)^{-m}. \end{align*}$$

Here, $(\cdot , \cdot )_F$ is the quadratic Hilbert symbol of F and $\gamma _F(\cdot ,\frac {1}{2}\psi )$ is the Weil index as in [Reference Ranga Rao58, Appendix], [Reference Ichino and Prasanna30, §3.1.1]. By [Reference Kudla40, Theorem 3.1, cases ${3}_{+}$ ], we have

(A.5) $$ \begin{align} z_{\mathbb{V}^\triangle}(h_1,h_2) = \frac{\hat{s}_2(h_1 h_2)}{\hat{s}_2(h_1) \hat{s}_2(h_2)} \end{align} $$

for $h_1, h_2 \in \mathrm {U}(\mathbf {W}^\square )$ .

Lemma A.5. For $\alpha \in E^{\times }$ and $h \in \mathrm {U}(\mathbf {W}^\square )$ , we have

$$\begin{align*}\hat{s}_2(\iota([\alpha,1], [\alpha,1]) \cdot h \cdot \iota([\alpha,1], [\alpha,1])^{-1}) = \hat{s}_2(h). \end{align*}$$

Proof. Put $h_\alpha = \iota ([\alpha ,1], [\alpha ,1])$ . Since

$$\begin{align*}\begin{bmatrix} \mathbf{w}_1 \cdot h_\alpha \\ \mathbf{w}_2 \cdot h_\alpha \\ \mathbf{w}_1^* \cdot h_\alpha \\ \mathbf{w}_2^* \cdot h_\alpha \end{bmatrix} = \begin{pmatrix} \alpha & & & \\ & \alpha^\rho & & \\ & & \alpha & \\ & & & \alpha^\rho \end{pmatrix} \cdot \begin{bmatrix} \mathbf{w}_1 \\ \mathbf{w}_2 \\ \mathbf{w}_1^* \\ \mathbf{w}_2^* \end{bmatrix}, \end{align*}$$

we have $x(h_\alpha p h_\alpha ^{-1}) = x(p)$ for $p \in P_{\mathbf {W}^\triangle }$ and $h_\alpha \tau _j h_\alpha ^{-1} = \tau _j$ . Hence, the assertion follows.

Lemma A.6. For $\alpha \in E^\times $ , we have

$$\begin{align*}\hat{s}_2(\iota([\alpha,\alpha], [\alpha,\alpha])) = \chi(\alpha)^{-2m}. \end{align*}$$

Proof. Put $h_\alpha = \iota ([\alpha ,\alpha ], [\alpha ,\alpha ])$ . Since

$$\begin{align*}\begin{bmatrix} \mathbf{w}_1 \cdot h_\alpha \\ \mathbf{w}_2 \cdot h_\alpha \\ \mathbf{w}_1^* \cdot h_\alpha \\ \mathbf{w}_2^* \cdot h_\alpha \end{bmatrix} = \begin{pmatrix} 1 & & & \\ & \alpha^{-1} \alpha^\rho & & \\ & & 1 & \\ & & & \alpha^{-1} \alpha^\rho \end{pmatrix} \cdot \begin{bmatrix} \mathbf{w}_1 \\ \mathbf{w}_2 \\ \mathbf{w}_1^* \\ \mathbf{w}_2^* \end{bmatrix} \end{align*}$$

and $\chi (\alpha ^\rho ) = \chi (\alpha )^{-1}$ , the assertion follows.

Lemma A.7. Let $\alpha \in E^1$ . Then we have

$$\begin{align*}\hat{s}_2(\iota([1,\alpha], 1)) = \chi(\alpha)^{-m} \end{align*}$$

if B is split, and

$$\begin{align*}\hat{s}_2(\iota([1,\alpha], 1)) = \chi(\alpha)^{-m} \times \begin{cases} 1 & \text{if } \alpha = 1, \\ (-1)^m & \text{if } \alpha \ne 1 \end{cases} \end{align*}$$

if B is ramified.

Proof. We may assume that $\alpha \ne 1$ . Then we have $\operatorname {tr}_{E/F}(\alpha ) \ne 2$ and hence $\alpha -1 \in E^\times $ . As in the proof of Lemma A.4, we have

$$\begin{align*}\begin{bmatrix} \mathbf{w}_1 \cdot \iota([1,\alpha],1) \\ \mathbf{w}_2 \cdot \iota([1,\alpha],1) \\ \mathbf{w}_1^* \cdot \iota([1,\alpha],1) \\ \mathbf{w}_2^* \cdot \iota([1,\alpha],1) \end{bmatrix} = A \cdot \begin{bmatrix} \mathbf{w}_1 \\ \mathbf{w}_2 \\ \mathbf{w}_1^* \\ \mathbf{w}_2^* \end{bmatrix}, \end{align*}$$

where

$$ \begin{align*} A & = \begin{pmatrix} \frac{1}{2}(\alpha^{-1}+1) & & - \frac{1}{4 \mathbf{i}}(\alpha^{-1}-1) & \\ & \frac{1}{2}(\alpha^{-1}+1) & & \frac{1}{4 J \mathbf{i}}(\alpha^{-1}-1) \\ - \mathbf{i} (\alpha^{-1}-1) & & \frac{1}{2}(\alpha^{-1}+1) & \\ & J \mathbf{i} (\alpha^{-1}-1) & & \frac{1}{2}(\alpha^{-1}+1) \end{pmatrix} \\ & = \begin{pmatrix} \frac{1}{\mathbf{i} (\alpha - 1)} & & * & \\ & -\frac{1}{J \mathbf{i} (\alpha - 1)} & & * \\ & & - \mathbf{i} (\alpha^\rho-1) & \\ & & & J \mathbf{i} (\alpha^\rho-1) \end{pmatrix} \cdot \tau_2 \cdot \begin{pmatrix} 1 & & * & \\ & 1 & & * \\ & & 1 & \\ & & & 1 \end{pmatrix}. \end{align*} $$

Hence, we have

$$\begin{align*}x(\iota([1,\alpha],1)) = - \frac{1}{u J (\alpha-1)^2} \equiv \frac{J}{(\alpha-1)^2} \equiv - \frac{J}{\alpha}\quad \mod \mathrm{N}_{E/F}(E^\times) \end{align*}$$

so that

$$ \begin{align*} \chi(x(\iota([1,\alpha],1))) & = \chi(\alpha)^{-1} \cdot \xi_E(-J) \\ & = \chi(\alpha)^{-1} \cdot \xi_E(-1) \times \begin{cases} 1 & \text{if } B \text{ is split,} \\ -1 & \text{if } B \text{ is ramified.} \end{cases} \end{align*} $$

Also, we have

$$\begin{align*}\gamma^2 = (-1, -u)_F^m \cdot (-1, -1)_F^m = (-1,u)_F^m = \xi_E(-1)^m. \end{align*}$$

This implies the assertion.

A.5 Splitting over $\mathcal {G}^\sharp $

Let $\mathcal {G}^\sharp $ be a subgroup of $\mathrm {GU}(V) \times \mathrm {GU}(W) \times \mathrm {G}\mathrm {U}(V) \times \mathrm {GU}(W)$ defined by

$$\begin{align*}\mathcal{G}^\sharp = \{ (g,h,\alpha,\alpha) \in \mathrm{GU}(V)^0 \times \mathrm{GU}(W) \times E^\times \times E^\times \, | \, \nu(g) = \nu(h) = \mathrm{N}_{E/F}(\alpha) \}. \end{align*}$$

Then we have a natural homomorphism

(A.6) $$ \begin{align} \mathcal{G}^\sharp \subset \mathrm{G}(\mathrm{U}(V) \times \mathrm{U}(W)) \times \mathrm{G}(\mathrm{U}(V) \times \mathrm{U}(W)) \longrightarrow \mathrm{Sp}(\mathbb{V}) \times \mathrm{Sp}(\mathbb{V}) \subset \mathrm{Sp}(\mathbb{V}^\square). \end{align} $$

We define a map

$$\begin{align*}\hat{s}^\sharp: \mathcal{G}^\sharp \longrightarrow \mathbb{C}^1 \end{align*}$$

by setting

$$\begin{align*}\hat{s}^\sharp(g,h,\alpha,\alpha) = \chi(\alpha)^{-m} \cdot \hat{s}_1(\iota(g \alpha^{-1},1)) \cdot \hat{s}_2(\iota(h \alpha^{-1}, 1)) \cdot z_{\mathbb{V}^\triangle}(\iota(g \alpha^{-1},1), \iota(h \alpha^{-1},1)). \end{align*}$$

Lemma A.8. For $\mathbf {g}_1, \mathbf {g}_2 \in \mathcal {G}^\sharp $ , we have

$$\begin{align*}z_{\mathbb{V}^\triangle}(\mathbf{g}_1,\mathbf{g}_2) = \frac{\hat{s}^\sharp(\mathbf{g}_1 \mathbf{g}_2)}{\hat{s}^\sharp(\mathbf{g}_1) \hat{s}^\sharp(\mathbf{g}_2)}. \end{align*}$$

Here, by abuse of notation, we write $\mathbf {g}_i$ on the left-hand side for the image of $\mathbf {g}_i$ in $\mathrm {Sp}(\mathbb {V}^\square )$ under equation (A.6).

Proof. Write $\mathbf {g}_i = (g_i,h_i,\alpha _i,\alpha _i)$ . If $\alpha _1 = \alpha _2 = 1$ , then the assertion follows from equations (A.3) and (A.5) and [Reference Mœglin, Vignéras and Waldspurger51, Chapitre 2, II.5]. If $\alpha _1$ and $\alpha _2$ are arbitrary, put $\mathbf {h}_i = (g_i \alpha _i^{-1}, h_i \alpha _i^{-1}, 1,1)$ and $\boldsymbol {\alpha }_i = (\alpha _i,\alpha _i,\alpha _i,\alpha _i)$ . Let $P_{\mathbb {V}^\triangle }$ be the maximal parabolic subgroup of $\mathrm {Sp}(\mathbb {V}^\square )$ stabilizing $\mathbb {V}^\triangle $ . Then it follows from [Reference Ranga Rao58, Theorem 4.1] that

(A.7) $$ \begin{align} z_{\mathbb{V}^\triangle}(p_1 \sigma p, p^{-1} \sigma' p_2) = z_{\mathbb{V}^\triangle}(\sigma, \sigma') \end{align} $$

for $p_1, p_2, p \in P_{\mathbb {V}^\triangle }$ and $\sigma , \sigma ' \in \mathrm {Sp}(\mathbb {V}^\square )$ (see also [Reference Ichino and Prasanna30, §3.1.1]). Since $\mathbf {g}_1\mathbf {g}_2 = \mathbf {h}_1 \cdot \boldsymbol {\alpha }_1 \mathbf {h}_2 \boldsymbol {\alpha }_1^{-1} \cdot \boldsymbol {\alpha }_1 \boldsymbol {\alpha }_2$ and the image of $\boldsymbol {\alpha }_i$ in $\mathrm {Sp}(\mathbb {V}^\square )$ belongs to $P_{\mathbb {V}^\triangle }$ , we have

$$\begin{align*}z_{\mathbb{V}^\triangle}(\mathbf{g}_1, \mathbf{g}_2) = z_{\mathbb{V}^\triangle}(\mathbf{h}_1, \boldsymbol{\alpha}_1 \mathbf{h}_2 \boldsymbol{\alpha}_1^{-1}) = \frac{\hat{s}^\sharp(\mathbf{h}_1 \boldsymbol{\alpha}_1 \mathbf{h}_2 \boldsymbol{\alpha}_1^{-1})}{\hat{s}^\sharp(\mathbf{h}_1) \hat{s}^\sharp(\boldsymbol{\alpha}_1\mathbf{h}_2\boldsymbol{\alpha}_1^{-1})}. \end{align*}$$

On the other hand, by definition, we have

$$\begin{align*}\frac{\hat{s}^\sharp(\mathbf{g}_1 \mathbf{g}_2)}{\hat{s}^\sharp(\mathbf{g}_1) \hat{s}^\sharp(\mathbf{g}_2)} = \frac{\hat{s}^\sharp(\mathbf{g}_1 \mathbf{g}_2 (\boldsymbol{\alpha}_1 \boldsymbol{\alpha}_2)^{-1})}{\hat{s}^\sharp(\mathbf{g}_1 \boldsymbol{\alpha}_1^{-1}) \hat{s}^\sharp(\mathbf{g}_2 \boldsymbol{\alpha}_2^{-1})} = \frac{\hat{s}^\sharp(\mathbf{h}_1 \boldsymbol{\alpha}_1 \mathbf{h}_2 \boldsymbol{\alpha}_1^{-1})}{\hat{s}^\sharp(\mathbf{h}_1) \hat{s}^\sharp(\mathbf{h}_2)}. \end{align*}$$

It follows from Lemmas A.3 and A.5, combined with equation (A.7), that

$$\begin{align*}\hat{s}^\sharp(\boldsymbol{\alpha}_1 \mathbf{h}_2 \boldsymbol{\alpha}_1^{-1}) = \hat{s}^\sharp(\mathbf{h}_2). \end{align*}$$

This completes the proof.

Lemma A.9. For $\alpha \in E^1$ , we have

$$\begin{align*}\hat{s}^\sharp(1,1,\alpha,\alpha) = \hat{s}_2(\iota(1, [\alpha,\alpha])). \end{align*}$$

Proof. Put

$$ \begin{align*} g_\alpha & = \iota(\alpha, 1) \in \mathrm{U}(V^\square), \\ h_\alpha & = \iota([\alpha, 1], 1) \in \mathrm{U}(\mathbf{W}^\square), \\ k_\alpha & = \iota([1, \alpha], 1) \in \mathrm{U}(\mathbf{W}^\square), \\ m_\alpha & = \iota([\alpha, \alpha], [\alpha, \alpha]) \in \mathrm{U}(\mathbf{W}^\square). \end{align*} $$

By definition, we have

$$\begin{align*}\hat{s}^\sharp(1,1,\alpha,\alpha) = \chi(\alpha)^{-m} \cdot \hat{s}_1(g_\alpha^{-1}) \cdot \hat{s}_2(h_\alpha^{-1}) \cdot z_{\mathbb{V}^\triangle}(g_\alpha^{-1}, h_\alpha^{-1}). \end{align*}$$

Since $m_\alpha = h_\alpha \cdot k_\alpha \cdot \iota (1, [\alpha ,\alpha ])$ and the image of $m_\alpha $ in $\mathrm {Sp}(\mathbb {V}^\square )$ belongs to $P_{\mathbb {V}^\triangle }$ , it follows from equations (A.5) and (A.7) and Lemma A.6 that

$$ \begin{align*} \hat{s}_2(\iota(1, [\alpha,\alpha])) & = \hat{s}_2(k_\alpha^{-1} h_\alpha^{-1}) \cdot \hat{s}_2(m_\alpha) \cdot z_{\mathbb{V}^\triangle}(k_\alpha^{-1} h_\alpha^{-1}, m_\alpha) \\ & = \hat{s}_2(k_\alpha^{-1} h_\alpha^{-1}) \cdot \chi(\alpha)^{-2m} \\ & = \chi(\alpha)^{-2m} \cdot \hat{s}_2(k_\alpha^{-1}) \cdot \hat{s}_2(h_\alpha^{-1}) \cdot z_{\mathbb{V}^\triangle}(k_\alpha^{-1}, h_\alpha^{-1}). \end{align*} $$

By Lemmas A.4 and A.7, we have

$$\begin{align*}\hat{s}_2(k_\alpha^{-1}) = \chi(\alpha)^m \cdot \hat{s}_1(g_\alpha^{-1}). \end{align*}$$

By equation (A.4), the image of $k_\alpha $ in $\mathrm {Sp}(\mathbb {V}^\square )$ agrees with that of $g_\alpha $ so that

$$\begin{align*}z_{\mathbb{V}^\triangle}(k_\alpha^{-1}, h_\alpha^{-1}) = z_{\mathbb{V}^\triangle}(g_\alpha^{-1}, h_\alpha^{-1}). \end{align*}$$

This completes the proof.

A.6 Proof of Proposition A.1

Now, we take a complete polarization $\mathbb {V}^\square = \mathbb {X}^\square \oplus \mathbb {Y}^\square $ defined by

$$\begin{align*}\mathbb{X}^\square = \mathbb{X} \oplus \mathbb{X}, \qquad \mathbb{Y}^\square = \mathbb{Y} \oplus \mathbb{Y}. \end{align*}$$

As in [Reference Ichino and Prasanna30, §D.3], we have

$$\begin{align*}z_{\mathbb{Y}^\square}(\iota(\sigma_1,\sigma_2), \iota(\sigma_1',\sigma_2')) = z_{\mathbb{Y}}(\sigma_1, \sigma_1') \cdot z_{\mathbb{Y}}(\sigma_2, \sigma_2')^{-1} \end{align*}$$

for $\sigma _i, \sigma _i' \in \mathrm {Sp}(\mathbb {V})$ . Fix $\sigma _0 \in \mathrm {Sp}(\mathbb {V}^\square )$ such that $\mathbb {V}^\bigtriangledown = \mathbb {X}^\square \cdot \sigma _0$ and $\mathbb {V}^\triangle = \mathbb {Y}^\square \cdot \sigma _0$ . Put

$$\begin{align*}\mu(\sigma) = z_{\mathbb{Y}^\square}(\sigma_0, \sigma)^{-1} \cdot z_{\mathbb{Y}^\square}(\sigma_0 \sigma \sigma_0^{-1}, \sigma_0) \end{align*}$$

for $\sigma \in \mathrm {Sp}(\mathbb {V}^\square )$ . Note that $\mu $ does not depend on the choice of $\sigma _0$ . Then, by [Reference Kudla40, Lemma 4.2], we have

(A.8) $$ \begin{align} z_{\mathbb{Y}^\square}(\sigma, \sigma') = z_{\mathbb{V}^\triangle}(\sigma, \sigma') \cdot \frac{\mu(\sigma \sigma')}{\mu(\sigma) \mu(\sigma')} \end{align} $$

for $\sigma , \sigma ' \in \mathrm {Sp}(\mathbb {V}^\square )$ .

Put $s^\sharp = \hat {s}^\sharp \cdot \mu $ and $s_2 = \hat {s}_2 \cdot \mu $ . By Lemma A.8 and equation (A.5), we have

$$\begin{align*}z_{\mathbb{Y}^\square}(\mathbf{g}_1, \mathbf{g}_2) = \frac{s^\sharp(\mathbf{g}_1 \mathbf{g}_2)}{s^\sharp(\mathbf{g}_1) s^\sharp(\mathbf{g}_2)} \end{align*}$$

for $\mathbf {g}_1,\mathbf {g}_2 \in \mathcal {G}^\sharp $ and

$$\begin{align*}z_{\mathbb{Y}^\square}(h_1, h_2) = \frac{s_2(h_1 h_2)}{s_2(h_1) s_2(h_2)} \end{align*}$$

for $h_1, h_2 \in \mathrm {U}(\mathbf {W}^\square )$ . We define a map

$$\begin{align*}s: \mathcal{G} \longrightarrow \mathbb{C}^1 \end{align*}$$

by setting

$$\begin{align*}s(g,h) = \frac{s^\sharp(g,h,\alpha,\alpha)}{s_2(\iota(1, [\alpha,\alpha]))}, \end{align*}$$

where we choose $\alpha \in E^\times $ such that $\nu (g) = \nu (h) = \mathrm {N}_{E/F}(\alpha )$ .

Lemma A.10. The map s is well defined, that is, for $(g,h,\alpha ,\alpha ) \in \mathcal {G}^\sharp $ and $\beta \in E^1$ , we have

$$\begin{align*}\frac{s^\sharp(g,h,\alpha \beta, \alpha \beta)}{s_2(\iota(1, [\alpha \beta, \alpha \beta]))} = \frac{s^\sharp(g,h,\alpha, \alpha)}{s_2(\iota(1, [\alpha, \alpha]))}. \end{align*}$$

Proof. First, note that, by equation (A.4), the image of $(\alpha , \alpha ) \in \mathrm {G}(\mathrm {U}(V) \times \mathrm {U}(W))$ in $\mathrm {Sp}(\mathbb {V})$ agrees with that of $[\alpha , \alpha ] \in \mathrm {U}(\mathbf {W})$ . We have

$$ \begin{align*} s^\sharp(g,h,\alpha \beta, \alpha \beta) & = s^\sharp(g,h,\alpha,\alpha) \cdot s^\sharp(1,1,\beta,\beta) \cdot z_{\mathbb{Y}^\square}((g,h,\alpha,\alpha), (1,1,\beta,\beta)) \\ & = s^\sharp(g,h,\alpha,\alpha) \cdot s^\sharp(1,1,\beta,\beta) \cdot z_{\mathbb{Y}}((\alpha,\alpha), (\beta,\beta))^{-1} \end{align*} $$

and

$$ \begin{align*} s_2(\iota(1, [\alpha \beta, \alpha \beta])) & = s_2(\iota(1, [\alpha, \alpha])) \cdot s_2(\iota(1, [\beta, \beta])) \cdot z_{\mathbb{Y}^\square}(\iota(1, [\alpha, \alpha]), \iota(1, [\beta, \beta])) \\ & = s_2(\iota(1, [\alpha, \alpha])) \cdot s_2(\iota(1, [\beta, \beta])) \cdot z_{\mathbb{Y}}([\alpha, \alpha], [\beta, \beta])^{-1}. \end{align*} $$

By Lemma A.9, we have

$$\begin{align*}s^\sharp(1,1,\beta,\beta) = s_2(\iota(1, [\beta, \beta])). \end{align*}$$

This implies the assertion.

Lemma A.11. The map s satisfies the condition (i) of Proposition A.1, that is, for $\mathbf {g}_1, \mathbf {g}_2 \in \mathcal {G}$ , we have

$$\begin{align*}z_{\mathbb{Y}}(\mathbf{g}_1,\mathbf{g}_2) = \frac{s(\mathbf{g}_1\mathbf{g}_2)}{s(\mathbf{g}_1) s(\mathbf{g}_2)}. \end{align*}$$

Proof. Write $\mathbf {g}_i = (g_i, h_i)$ , and choose $\alpha _i \in E^\times $ such that $\nu (g_i) = \nu (h_i) = \mathrm {N}_{E/F}(\alpha _i)$ . Then we have

$$ \begin{align*} \frac{s(\mathbf{g}_1\mathbf{g}_2)}{s(\mathbf{g}_1) s(\mathbf{g}_2)} & = \frac{s^\sharp(g_1 g_2, h_1 h_2, \alpha_1 \alpha_2, \alpha_1 \alpha_2)}{s^\sharp(g_1, h_1, \alpha_1, \alpha_1) s^\sharp(g_2, h_2, \alpha_2, \alpha_2)} \cdot \frac{s_2(\iota(1, [\alpha_1, \alpha_1])) s_2(\iota(1, [\alpha_2, \alpha_2]))}{s_2(\iota(1, [\alpha_1 \alpha_2, \alpha_1 \alpha_2]))} \\ & = z_{\mathbb{Y}^\square}((g_1, h_1, \alpha_1, \alpha_1), (g_2, h_2, \alpha_2, \alpha_2)) \cdot z_{\mathbb{Y}^\square}(\iota(1, [\alpha_1, \alpha_1]), \iota(1, [\alpha_2, \alpha_2]))^{-1} \\ & = z_{\mathbb{Y}}((g_1, h_1), (g_2, h_2)) \cdot z_{\mathbb{Y}}((\alpha_1, \alpha_1), (\alpha_2, \alpha_2))^{-1} \cdot z_{\mathbb{Y}}([\alpha_1, \alpha_1], [\alpha_2, \alpha_2]) \\ & = z_{\mathbb{Y}}((g_1, h_1), (g_2, h_2)). \end{align*} $$

This completes the proof.

By definition, the map s also satisfies the other conditions of Proposition A.1. This completes the proof of Proposition A.1.

A.7 Independence of the choice of $\chi $

To define the map s, we have used the fixed character $\chi $ of $E^\times $ such that $\chi |_{F^\times } = \xi _E$ . However, we have:

Lemma A.12. The map s defined as above does not depend on the choice of $\chi $ .

Proof. Let $\chi _1$ and $\chi _2$ be two characters of $E^\times $ such that $\chi _1|_{F^\times } = \chi _2|_{F^\times } = \xi _E$ . We will write $s = s_{\chi _i}$ , and so on, to indicate the dependence on $\chi _i$ . For $(g,h) \in \mathcal {G}$ , choose $\alpha \in E^\times $ such that $\nu (g) = \nu (h) = \mathrm {N}_{E/F}(\alpha )$ . Then, by definition, we have

$$ \begin{align*} \frac{s_{\chi_1}(g,h)}{s_{\chi_2}(g,h)} & = \frac{s^\sharp_{\chi_1}(g,h,\alpha,\alpha)}{s^\sharp_{\chi_2}(g,h,\alpha,\alpha)} \cdot \frac{s_{2,\chi_2}(\iota(1, [\alpha,\alpha]))}{s_{2,\chi_1}(\iota(1, [\alpha,\alpha]))} \\ & = \frac{\hat{s}^\sharp_{\chi_1}(g,h,\alpha,\alpha)}{\hat{s}^\sharp_{\chi_2}(g,h,\alpha,\alpha)} \cdot \frac{\hat{s}_{2,\chi_2}(\iota(1, [\alpha,\alpha]))}{\hat{s}_{2,\chi_1}(\iota(1, [\alpha,\alpha]))} \\ & = \eta(\alpha)^{-m} \cdot \frac{\hat{s}_{2,\chi_1}(\iota(h \alpha^{-1}, 1))}{\hat{s}_{2,\chi_2}(\iota(h \alpha^{-1}, 1))} \cdot \frac{\hat{s}_{2,\chi_2}(\iota(1, [\alpha,\alpha]))}{\hat{s}_{2,\chi_1}(\iota(1, [\alpha,\alpha]))}, \end{align*} $$

where $\eta = \chi _1/\chi _2$ . On the other hand, for $k \in \mathrm {U}(\mathbf {W}^\square )$ , we have

$$\begin{align*}\frac{\hat{s}_{2,\chi_1}(k)}{\hat{s}_{2,\chi_2}(k)} = \eta(x(k))^m = \tilde{\eta}(\det k)^m, \end{align*}$$

where $\tilde {\eta }$ is the character of $E^1$ such that $\tilde {\eta }(x/x^\rho ) = \eta (x)$ for $x \in E^\times $ . Since

$$ \begin{align*} \det \iota(h \alpha^{-1}, 1) & = \nu(h) \mathrm{N}_{E/F}(\alpha)^{-1} = 1, \\ \det \iota(1, [\alpha,\alpha]) & = \mathrm{N}_{E/F}(\alpha) \alpha^{-2} = \alpha^{-1} \alpha^\rho, \end{align*} $$

we have

$$\begin{align*}\frac{s_{\chi_1}(g,h)}{s_{\chi_2}(g,h)} = \eta(\alpha)^{-m} \cdot \tilde{\eta}(\alpha^{-1} \alpha^\rho)^{-m} = 1. \end{align*}$$

This completes the proof.

A.8 Compatibility with seesaws

We write $V = V' \oplus V"$ as an orthogonal direct sum of skew-Hermitian right B-spaces

$$\begin{align*}V' = e_1 B \oplus \dots \oplus e_{m'} B, \qquad V" = e_{m'+1} B \oplus \dots \oplus e_m B. \end{align*}$$

Let $\mathbb {V}' = V' \otimes _B W$ and $\mathbb {V}" = V" \otimes _B W$ be the symplectic F-spaces as in §A.1. Then we have $\mathbb {V} = \mathbb {V}' \oplus \mathbb {V}"$ , which gives rise to a seesaw diagram

Let $\mathcal {G}'$ and $\mathcal {G}"$ be the subgroups of $\mathrm {G}(\mathrm {U}(V') \times \mathrm {U}(W))$ and $\mathrm {G}(\mathrm {U}(V") \times \mathrm {U}(W))$ , respectively, as in §A.1. Put

$$\begin{align*}\mathcal{G}"' = \{ (g', g", h) \in \mathrm{GU}(V')^0 \times \mathrm{GU}(V")^0 \times \mathrm{GU}(W) \, | \, \nu(g') = \nu(g") = \nu(h) \in \mathrm{N}_{E/F}(E^\times) \}. \end{align*}$$

We regard $\mathcal {G}"'$ as subgroups of $\mathcal {G}$ and $\mathcal {G}' \times \mathcal {G}"$ via the above seesaw diagram. We take the complete polarizations $\mathbb {V}' = \mathbb {X}' \oplus \mathbb {Y}'$ and $\mathbb {V}" = \mathbb {X}" \oplus \mathbb {Y}"$ as in §A.1 so that

$$\begin{align*}\mathbb{X} = \mathbb{X}' \oplus \mathbb{X}", \qquad \mathbb{Y} = \mathbb{Y}' \oplus \mathbb{Y}". \end{align*}$$

Let $s': \mathcal {G}' \rightarrow \mathbb {C}^1$ and $s":\mathcal {G}" \rightarrow \mathbb {C}^1$ be the maps trivializing $z_{\mathbb {Y}'}$ and $z_{\mathbb {Y}"}$ , respectively, defined similarly as above. Then, by construction, we have

$$\begin{align*}s = s' \otimes s" \end{align*}$$

on $\mathcal {G}"'$ .

A.9 Compatibility with [Reference Ichino and Prasanna31]

In this section, we compare the splitting s with the standard one for unitary dual pairs when $\dim V = 1$ . In this case, using the notation of §A.4, we have a seesaw diagram

We define a map

$$\begin{align*}s^\natural : \mathrm{G}(\mathrm{U}(V) \times \mathrm{U}(W))^0 \longrightarrow \mathbb{C}^1 \end{align*}$$

by setting

$$\begin{align*}s^\natural(\alpha, h) = s_2(\iota([h, \alpha], 1)) \end{align*}$$

for $(\alpha , h) \in \mathrm {G}(\mathrm {U}(V) \times \mathrm {U}(W))^0$ , where $[h, \alpha ] \in \mathrm {U}(\mathbf {W})$ and $s_2 = \hat {s}_2 \cdot \mu $ . Then $s^\natural $ trivializes $z_{\mathbb {Y}}$ by equation (A.4). This splitting will be used in [Reference Ichino and Prasanna31].

Lemma A.13. We have

$$\begin{align*}s^\natural(\alpha, h) = s(\alpha, h) \cdot \chi(\alpha)^{-1}. \end{align*}$$

Proof. Recall that

$$\begin{align*}s(\alpha, h) = \frac{s^\sharp(\alpha, h, \alpha, \alpha)}{s_2(\iota(1, [\alpha, \alpha]))}. \end{align*}$$

We have

$$ \begin{align*} s^\sharp(\alpha, h, \alpha, \alpha) & = s^\sharp(1, h \alpha^{-1}, 1, 1) \cdot s^\sharp(\alpha, \alpha, \alpha, \alpha) \cdot z_{\mathbb{Y}^\square}((1, h \alpha^{-1}, 1, 1), (\alpha, \alpha, \alpha, \alpha)) \\ & = s^\sharp(1, h \alpha^{-1}, 1, 1) \cdot s^\sharp(\alpha, \alpha, \alpha, \alpha) \cdot z_{\mathbb{Y}}((1, h \alpha^{-1}), (\alpha, \alpha)). \end{align*} $$

By definition and Lemma A.6, we have

$$\begin{align*}s^\sharp(1, h \alpha^{-1}, 1, 1) = s_2(\iota([h \alpha^{-1}, 1], 1)) \end{align*}$$

and

$$ \begin{align*} s^\sharp(\alpha, \alpha, \alpha, \alpha) & = \chi(\alpha) \cdot s_2(\iota([\alpha, \alpha], [\alpha, \alpha])) \\ & = \chi(\alpha) \cdot s_2(\iota([\alpha, \alpha], 1)) \cdot s_2(\iota(1, [\alpha, \alpha])) \cdot z_{\mathbb{Y}^\square}(\iota([\alpha, \alpha], 1), \iota(1, [\alpha, \alpha])) \\ & = \chi(\alpha) \cdot s_2(\iota([\alpha, \alpha], 1)) \cdot s_2(\iota(1, [\alpha, \alpha])). \end{align*} $$

Hence, we have

$$ \begin{align*} s(\alpha, h) & = \chi(\alpha) \cdot s_2(\iota([h \alpha^{-1}, 1], 1)) \cdot s_2(\iota([\alpha, \alpha], 1)) \cdot z_{\mathbb{Y}}([h \alpha^{-1}, 1], [\alpha, \alpha]) \\ & = \chi(\alpha) \cdot s_2(\iota([h \alpha^{-1}, 1], 1)) \cdot s_2(\iota([\alpha, \alpha], 1)) \cdot z_{\mathbb{Y}^\square}(\iota([h \alpha^{-1}, 1], 1), \iota([\alpha, \alpha], 1)) \\ & = \chi(\alpha) \cdot s_2(\iota([h, \alpha], 1)).\\[-37pt] \end{align*} $$

A.10 Compatibility with [Reference Ichino and Prasanna30]

In this section, we compare the splitting s with the one defined in [Reference Ichino and Prasanna30, Appendix C]. Suppose again that F is a number field. Let $V = B_1 \otimes _E B_2$ be the two-dimensional skew-Hermitian right B-space as in [Reference Ichino and Prasanna30, §2.2], where $B_1$ and $B_2$ are quaternion algebras over F such that E embeds into $B_1$ and $B_2$ , and such that $B_1 \cdot B_2 = B$ in the Brauer group. We write $B_i = E + E \mathbf {j}_i$ for some trace zero element $\mathbf {j}_i \in B_i^\times $ and put $J_i = \mathbf {j}_i^2 \in F^\times $ . We may assume that

$$\begin{align*}J_1 \cdot J_2 = J. \end{align*}$$

Then the skew-Hermitian form on V is given by equation (A.1) with

$$ \begin{align*} e_1 & = 1 \otimes 1, & \kappa_1 & = 1, \\ e_2 & = \mathbf{j}_1 \otimes 1, & \kappa_2 & = -J_1. \end{align*} $$

Recall the exact sequence

$$\begin{align*}1 \longrightarrow F^\times \longrightarrow B_1^\times \times B_2^\times \longrightarrow \mathrm{GU}(V)^0 \longrightarrow 1, \end{align*}$$

where $F^\times $ embeds into $B_1^\times \times B_2^\times $ by $z \mapsto (z, z^{-1})$ and $B_1^\times \times B_2^\times $ acts on V on the left by

$$\begin{align*}(g_1, g_2) \cdot (x_1 \otimes x_2) = g_1 x_1 \otimes g_2 x_2. \end{align*}$$

We write $[g_1,g_2]$ for the image of $(g_1,g_2)$ in $\mathrm {GU}(V)^0$ . If we put

$$\begin{align*}\tilde{\mathcal{G}} = \{ (g_1, g_2, h) \in B_1^\times \times B_2^\times \times B^\times \, | \, \nu(g_1) \nu(g_2) = \nu(h) \in \mathrm{N}_{E/F}(E^\times) \}, \end{align*}$$

where $\nu $ denotes the reduced norm, then we have a natural surjective map $\tilde {\mathcal {G}} \twoheadrightarrow \mathcal {G}$ . We take the complete polarization $\mathbb {V} = \mathbb {X} \oplus \mathbb {Y}$ as in §A.1, which agrees with the one given in [Reference Ichino and Prasanna30, §2.2]. For each place v of F, let

$$\begin{align*}\tilde{s}_v: \mathrm{GU}(V_v)^0 \times \mathrm{GU}(W_v) \longrightarrow \mathbb{C}^1 \end{align*}$$

be the map trivializing $z_{\mathbb {Y}_v}$ defined in [Reference Ichino and Prasanna30, Appendix C]. Since both $\tilde {s}_v$ and $s_v$ trivialize $z_{\mathbb {Y}_v}$ , there exists a continuous character $\boldsymbol {\chi }$ of $\mathcal {G}(\mathbb {A})$ such that

$$\begin{align*}\tilde{s}_v|_{\mathcal{G}_v} = s_v \cdot \boldsymbol{\chi}_v \end{align*}$$

for all v. Since both $\tilde {s}_v$ and $s_v$ satisfy the product formula, $\boldsymbol {\chi }$ is trivial on $\mathcal {G}(F)$ . We regard $\boldsymbol {\chi }$ as a character of $\tilde {\mathcal {G}}(\mathbb {A})$ .

Proposition A.14. Assume that

  • F is totally real;

  • E is totally imaginary;

  • $B_{1,v}$ and $B_{2,v}$ are split for some real place v of F.

Then, for $(g_1, g_2, h) \in \tilde {\mathcal {G}}(\mathbb {A})$ , we have

$$\begin{align*}\boldsymbol{\chi}(g_1, g_2, h) = 1. \end{align*}$$

Namely, we have

$$\begin{align*}\tilde{s}_v|_{\mathcal{G}_v} = s_v \end{align*}$$

for all v.

Proof. We define a homomorphism $\tilde {\nu }: \tilde {\mathcal {G}}(\mathbb {A}) \rightarrow \mathbb {A}^\times $ by

$$\begin{align*}\tilde{\nu}(g_1, g_2, h) = \nu(g_1). \end{align*}$$

Then the image of $\tilde {\nu }$ consists of elements $a \in \mathbb {A}^\times $ with $a_v> 0$ for all infinite places v such that $B_{1,v}$ or $B_{2,v}$ or $B_v$ is ramified. Also, putting $\tilde {\mathcal {G}}^{(1)} = B_1^{(1)} \times B_2^{(1)} \times B^{(1)}$ , we have

$$ \begin{align*} \ker \tilde{\nu} & = \tilde{\mathcal{G}}^{(1)}(\mathbb{A}) \cdot \{ (1, \alpha,\alpha) \, | \, \alpha \in \mathbb{A}_E^\times \}, \\ \tilde{\nu}^{-1}(\mathrm{N}_{E/F}(\mathbb{A}_E^\times)) & = \ker \tilde{\nu} \cdot \{ (\alpha,\alpha^{-1},1) \, | \, \alpha \in \mathbb{A}_E^\times \}, \\ \tilde{\nu}^{-1}(F^\times) & = \ker \tilde{\nu} \cdot \tilde{\mathcal{G}}(F), \end{align*} $$

where we have used Eichler’s norm theorem in the last equality. Since $\tilde {\nu }^{-1}(F^\times \mathrm {N}_{E/F}(\mathbb {A}_E^\times ))$ is the kernel of $\xi _E \circ \tilde {\nu }$ , it is a subgroup of $\tilde {\mathcal {G}}(\mathbb {A})$ of index $2$ and does not contain any element $(g_{1,v}, g_{2,v}, h_v) \in \tilde {\mathcal {G}}_v$ such that $\nu (g_{i,v}) \notin \mathrm {N}_{E_v/F_v}(E_v^\times )$ .

Now, we show that $\boldsymbol {\chi }$ is trivial. Since $\boldsymbol {\chi }$ is automorphic, it is trivial on $\tilde {\mathcal {G}}^{(1)}(\mathbb {A})$ . Moreover, in §A.11 below, we will prove the following:

  • For $\alpha \in \mathbb {A}_E^\times $ , we have

    (A.9) $$ \begin{align} \boldsymbol{\chi}(1,\alpha,\alpha) = 1. \end{align} $$
  • For $\alpha \in \mathbb {A}_E^\times $ , we have

    (A.10) $$ \begin{align} \boldsymbol{\chi}(\alpha, \alpha^{-1}, 1) = 1. \end{align} $$
  • Let v be a real place of F such that $B_{1,v}$ and $B_{2,v}$ are split. Choose $t_{i,v} \in F_v^\times $ such that $J_i = t_{i,v}^2$ . Then we have

    (A.11) $$ \begin{align} \boldsymbol{\chi}_v(t_{1,v}^{-1} \cdot \mathbf{j}_1, t_{2,v}^{-1} \cdot \mathbf{j}_2, 1) = 1. \end{align} $$
    Note that $\nu (t_{i,v}^{-1} \cdot \mathbf {j}_i) = -1 \notin \mathrm {N}_{E_v/F_v}(E_v^\times )$ .

This implies the assertion.

A.11 Computation of splittings

We retain the notation of §A.10. We fix a place v of F and suppress the subscript v from the notation. Recall that $\boldsymbol {\chi }$ is a continuous character of $\tilde {\mathcal {G}}$ such that

$$\begin{align*}\boldsymbol{\chi}(g_1,g_2,h) = \frac{\tilde{s}(g_1,g_2,h)}{s(g_1,g_2,h)}, \end{align*}$$

where we regard $\tilde {s}$ and s as maps on $\tilde {\mathcal {G}}$ . To compute $\boldsymbol {\chi }$ explicitly, we need to introduce more notation.

A.11.1 Notation

We denote by $\mathrm {GSp}_{2n}(F)$ the symplectic similitude group and by $\nu : \mathrm {GSp}_{2n}(F) \rightarrow F^\times $ the similitude character:

$$\begin{align*}\mathrm{GSp}_{2n}(F) = \left\{ \sigma \in \mathrm{GL}_{2n}(F) \, \left| \, \sigma \begin{pmatrix} & {\mathbf{1}}_n \\ -{\mathbf{1}}_n & \end{pmatrix} {}^t \sigma = \nu(\sigma) \cdot \begin{pmatrix} & {\mathbf{1}}_n \\ -{\mathbf{1}}_n & \end{pmatrix} \right. \right\}. \end{align*}$$

Let $\mathrm {Sp}_{2n}(F) = \ker \nu $ be the symplectic group and P the standard maximal parabolic subgroup of $\mathrm {Sp}_{2n}(F)$ :

$$\begin{align*}P = \{ \mathbf{m}(\mathbf{a}) \mathbf{n}(\mathbf{b}) \, | \, \mathbf{a} \in \mathrm{GL}_n(F), \mathbf{b} \in \mathrm{Sym}_n(F) \}, \end{align*}$$

where

$$\begin{align*}\mathbf{m}(\mathbf{a}) = \begin{pmatrix} \mathbf{a} & \\ & {}^t \mathbf{a}^{-1} \end{pmatrix}, \qquad \mathbf{n}(\mathbf{b}) = \begin{pmatrix} {\mathbf{1}}_n & \mathbf{b} \\ & {\mathbf{1}}_n \end{pmatrix}. \end{align*}$$

Put

$$\begin{align*}d(\nu) = \begin{pmatrix} {\mathbf{1}}_n & \\ & \nu \cdot {\mathbf{1}}_n \end{pmatrix}, \qquad \tau_j = \begin{pmatrix} {\mathbf{1}}_{n-j} & & & \\ & & & -{\mathbf{1}}_j \\ & & {\mathbf{1}}_{n-j} & \\ & {\mathbf{1}}_j & & \end{pmatrix}. \end{align*}$$

If $\sigma = p_1 \tau _j p_2 \in \mathrm {Sp}_{2n}(F)$ with $p_i = \mathbf {m}(\mathbf {a}_i) \mathbf {n}(\mathbf {b}_i) \in P$ , put

$$\begin{align*}x(\sigma) = \det(\mathbf{a}_1 \mathbf{a}_2)\ \mod (F^{\times})^2, \qquad j(\sigma) = j. \end{align*}$$

Note that

(A.12) $$ \begin{align} x(d(\nu) \cdot \sigma \cdot d(\nu)^{-1}) = \nu^{j(\sigma)} \cdot x(\sigma), \qquad j(d(\nu) \cdot \sigma \cdot d(\nu)^{-1}) = j(\sigma). \end{align} $$

We define a map

$$\begin{align*}v: \mathrm{Sp}_{2n}(F) \times F^\times \longrightarrow \mathbb{C}^1 \end{align*}$$

by setting

$$\begin{align*}v(\sigma, \nu) = (x(\sigma), \nu)_F \cdot \gamma_F(\nu, \tfrac{1}{2} \psi)^{-j(\sigma)}, \end{align*}$$

where $(\cdot , \cdot )_F$ is the quadratic Hilbert symbol of F and $\gamma _F(\cdot ,\frac {1}{2}\psi )$ is the Weil index as in [Reference Ranga Rao58, Appendix], [Reference Ichino and Prasanna30, §3.1.1]. Let z be the $2$ -cocycle on $\mathrm {Sp}_{2n}(F)$ realizing the metaplectic group (see e.g. [Reference Ranga Rao58], [Reference Ichino and Prasanna30, §3.2.2]). By [Reference Ranga Rao58, Theorem 4.1 and Corollary 4.2], we have:

  • $z(\sigma , \sigma ^{-1}) = 1$ for $\sigma \in \mathrm {Sp}_{2n}(F)$ ;

  • $z(p_1 \sigma p, p^{-1} \sigma ' p_2) = z(\sigma , \sigma ')$ for $p_1, p_2, p \in P$ and $\sigma , \sigma ' \in \mathrm {Sp}_{2n}(F)$ ;

  • $z(\tau _i, \tau _j) = 1$ ;

  • $z(\tau _n, \mathbf {n}(\mathbf {b}) \tau _n) = \gamma _F(\tfrac {1}{2} \psi )^n \cdot \gamma _F(\det \mathbf {b}, \tfrac {1}{2} \psi ) \cdot h_F(\mathbf {b})$ for $\mathbf {b} \in \mathrm {Sym}_n(F) \cap \mathrm {GL}_n(F)$ , where $h_F(\mathbf {b})$ is the Hasse invariant of the nondegenerate symmetric bilinear form associated to $\mathbf {b}$ .

We may extend z to a $2$ -cocycle on $\mathrm {GSp}_{2n}(F)$ (see, e.g., [Reference Ichino and Prasanna30, Appendix B]). Then, for $\sigma , \sigma ' \in \mathrm {GSp}_{2n}(F)$ with $\nu (\sigma ) = \nu $ and $\nu (\sigma ') = \nu ^{-1}$ , we have

$$\begin{align*}z(\sigma, \sigma') = z(\sigma \cdot d(\nu)^{-1}, d(\nu) \cdot \sigma') \cdot v(\sigma' \cdot d(\nu), \nu). \end{align*}$$

Recall that $\mathbb {V} = V \otimes _B W$ is an eight-dimensional symplectic F-space. Let $\mathbf {e}_1, \dots , \mathbf {e}_4, \mathbf {e}_1^*, \dots , \mathbf {e}_4^*$ be the basis of $\mathbb {V}$ given in [Reference Ichino and Prasanna30, §2.2]. Then we have

$$\begin{align*}\mathbb{X} = F \mathbf{e}_1 + \dots + F \mathbf{e}_4, \qquad \mathbb{Y} = F \mathbf{e}_1^* + \dots + F \mathbf{e}_4^*, \qquad \langle \hspace{-1mm} \langle \mathbf{e}_i, \mathbf{e}_j^* \rangle \hspace{-1mm} \rangle = \delta_{ij}. \end{align*}$$

Using this basis, we identify $\mathrm {GSp}(\mathbb {V})$ with $\mathrm {GSp}_8(F)$ . Under this identification, we write $P_{\mathbb {Y}}$ and $z_{\mathbb {Y}}$ for P and z, respectively. We refer to [Reference Ichino and Prasanna30, §C.1] for an explicit description of the image of $B_1^\times \times B_2^\times \times B^\times $ in $\mathrm {GSp}(\mathbb {V})$ . Also, using a basis

$$\begin{align*}(\mathbf{e}_1,0), \dots, (\mathbf{e}_4, 0), (0,\mathbf{e}_1), \dots, (0,\mathbf{e}_4), (\mathbf{e}_1^*,0), \dots, (\mathbf{e}_4^*, 0), (0,-\mathbf{e}_1^*), \dots, (0,-\mathbf{e}_4^*) \end{align*}$$

of $\mathbb {V}^{\square }$ , we identify $\mathrm {GSp}(\mathbb {V}^{\square })$ with $\mathrm {GSp}_{16}(F)$ . For $1 \le i \le 4$ , put

$$ \begin{align*} \mathbb{X}_i & = F \mathbf{e}_i, & \mathbb{X}_i^{\square} & = \mathbb{X}_i \oplus \mathbb{X}_i, \\ \mathbb{Y}_i & = F \mathbf{e}_i^*, & \mathbb{Y}_i^{\square} & = \mathbb{Y}_i \oplus \mathbb{Y}_i, \\ \mathbb{V}_i & = \mathbb{X}_i \oplus \mathbb{Y}_i, & \mathbb{V}_i^{\square} & = \mathbb{V}_i \oplus \mathbb{V}_i. \end{align*} $$

Then we have a natural embedding

$$\begin{align*}\iota_i: \mathrm{Sp}(\mathbb{V}_i^\square) \longrightarrow \mathrm{Sp}(\mathbb{V}^\square). \end{align*}$$

Using a basis $(\mathbf {e}_i,0), (0,\mathbf {e}_i), (\mathbf {e}_i^*,0), (0,-\mathbf {e}_i^*)$ of $\mathbb {V}^{\square }_i$ , we identify $\mathrm {GSp}(\mathbb {V}^{\square }_i)$ with $\mathrm {GSp}_4(F)$ . Put

$$\begin{align*}\sigma_0 = \begin{pmatrix} \frac{1}{2} {\mathbf{1}}_4 & - \frac{1}{2} {\mathbf{1}}_4 & & \\ & & \frac{1}{2} {\mathbf{1}}_4 & \frac{1}{2} {\mathbf{1}}_4 \\ & & {\mathbf{1}}_4 & -{\mathbf{1}}_4 \\ -{\mathbf{1}}_4 & -{\mathbf{1}}_4 & & \end{pmatrix} \in \mathrm{Sp}(\mathbb{V}^{\square}). \end{align*}$$

Then we have

$$\begin{align*}\begin{bmatrix} \frac{1}{2}(\vec{\mathbf{e}}, -\vec{\mathbf{e}}) \\ \frac{1}{2}(\vec{\mathbf{e}}^*, -\vec{\mathbf{e}}^*) \\ (\vec{\mathbf{e}}^*, \vec{\mathbf{e}}^*) \\ (-\vec{\mathbf{e}}, -\vec{\mathbf{e}}) \end{bmatrix} = \sigma_0 \cdot \begin{bmatrix} (\vec{\mathbf{e}}, 0) \\ (0, \vec{\mathbf{e}}) \\ (\vec{\mathbf{e}}^*, 0) \\ (0, -\vec{\mathbf{e}}^*) \end{bmatrix}, \qquad \vec{\mathbf{e}} = \begin{bmatrix} \mathbf{e}_1 \\ \mathbf{e}_2 \\ \mathbf{e}_3 \\ \mathbf{e}_4 \end{bmatrix}, \qquad \vec{\mathbf{e}}^* = \begin{bmatrix} \mathbf{e}_1^* \\ \mathbf{e}_2^* \\ \mathbf{e}_3^* \\ \mathbf{e}_4^* \end{bmatrix} \end{align*}$$

so that $\mathbb {V}^{\bigtriangledown } = \mathbb {X}^{\square } \cdot \sigma _0$ and $\mathbb {V}^{\triangle } = \mathbb {Y}^{\square } \cdot \sigma _0$ .

A.11.2 Proof of equation (A.9)

In this section, we will show that

$$\begin{align*}\boldsymbol{\chi}(1, \alpha, \alpha) = 1 \end{align*}$$

for $\alpha \in E^\times $ . We write $\alpha = a + b \mathbf {i}$ with $a, b \in F$ and put $\nu = a^2 - b^2 u$ . Since $\boldsymbol {\chi }$ is continuous, we may assume that

$$\begin{align*}a \ne 0, \qquad b \ne 0. \end{align*}$$

Lemma A.15. We have

$$\begin{align*}\tilde{s}(1, \alpha, \alpha) = \gamma_F(J_1, \tfrac{1}{2} \psi) \cdot (-2abJ_2, J_1)_F. \end{align*}$$

Proof. Put $g = [1, \alpha ] \in \mathrm {GU}(V)^0$ and $h = \alpha \in \mathrm {GU}(W)$ . Then we have $\tilde {s}(1, \alpha , \alpha ) = \tilde {s}(g) \cdot \tilde {s}(h) \cdot z_{\mathbb {Y}}(g, h)$ . By [Reference Ichino and Prasanna30, Proposition C.4.2], we have

$$\begin{align*}\tilde{s}(g) = (- \nu J_2, J_1)_F, \qquad \tilde{s}(h) = (J_2, J_1)_F. \end{align*}$$

It remains to compute $z_{\mathbb {Y}}(g, h)$ .

Recall that

$$\begin{align*}z_{\mathbb{Y}}(g, h) = z_{\mathbb{Y}}(g \cdot d(\nu), d(\nu)^{-1} \cdot h) \cdot v(h \cdot d(\nu)^{-1}, \nu^{-1}). \end{align*}$$

We have

$$\begin{align*}g = \nu^{-1} \cdot \begin{pmatrix} a \cdot {\mathbf{1}}_4 & -bu \cdot \mathbf{J}_2 \\ -b \cdot \mathbf{J}_2^{-1} & a \cdot {\mathbf{1}}_4 \end{pmatrix}, \qquad h = \begin{pmatrix} a \cdot {\mathbf{1}}_4 & bu \cdot \mathbf{J} \\ b \cdot \mathbf{J}^{-1} & a \cdot {\mathbf{1}}_4 \end{pmatrix} \end{align*}$$

in $\mathrm {GSp}(\mathbb {V})$ , where

$$\begin{align*}\mathbf{J}_2 = \begin{pmatrix} 1 & & & \\ & -J_1 & & \\ & & J_2 & \\ & & & -J \end{pmatrix}, \qquad \mathbf{J} = \begin{pmatrix} 1 & & & \\ & -J_1 & & \\ & & -J_2 & \\ & & & J \end{pmatrix}. \end{align*}$$

Since

$$ \begin{align*} g & = \begin{pmatrix} - b^{-1} \cdot \mathbf{J}_2 & \nu^{-1} a \cdot {\mathbf{1}}_4 \\ & - \nu^{-1} b \cdot \mathbf{J}_2^{-1} \end{pmatrix} \cdot \tau_4 \cdot \mathbf{n}(- a b^{-1} \cdot \mathbf{J}_2), \\ h & = \mathbf{n}(a b^{-1} \cdot \mathbf{J}) \cdot \tau_4 \cdot \begin{pmatrix} b \cdot \mathbf{J}^{-1} & a \cdot {\mathbf{1}}_4 \\ & \nu b^{-1} \cdot \mathbf{J} \end{pmatrix}, \end{align*} $$

we have

$$\begin{align*}z_{\mathbb{Y}}(g \cdot d(\nu), d(\nu)^{-1} \cdot h) = z_{\mathbb{Y}}(\tau_4 \cdot \mathbf{n}(- \nu a b^{-1} \cdot \mathbf{J}_2), \mathbf{n}(\nu a b^{-1} \cdot \mathbf{J}) \cdot \tau_4) = z_{\mathbb{Y}}(\tau_4, \mathbf{n}(\mathbf{b}) \cdot \tau_4), \end{align*}$$

where

$$\begin{align*}\mathbf{b} = - \nu a b^{-1} \cdot \mathbf{J}_2 + \nu a b^{-1} \cdot \mathbf{J} = 2 \nu a b^{-1} \cdot \begin{pmatrix} 0 & & & \\ & 0 & & \\ & & -J_2 & \\ & & & J \end{pmatrix}. \end{align*}$$

If we put

$$\begin{align*}\mathbf{b}' = 2 \nu a b^{-1} \cdot \begin{pmatrix} - J_2 & \\ & J \end{pmatrix}, \end{align*}$$

then we have

$$\begin{align*}z_{\mathbb{Y}}(\tau_4, \mathbf{n}(\mathbf{b}) \cdot \tau_4) = \gamma_F(\tfrac{1}{2} \psi)^2 \cdot \gamma_F(\det \mathbf{b}', \tfrac{1}{2} \psi) \cdot h_F(\mathbf{b}'). \end{align*}$$

Hence, since $\det \mathbf {b}' \equiv -J_1 \bmod (F^{\times })^2$ and

$$\begin{align*}h_F(\mathbf{b}') = (-2 \nu a b^{-1} J_2, 2 \nu a b^{-1} J)_F = (-2 \nu a b J_2, J_1)_F, \end{align*}$$

we have

$$ \begin{align*} z_{\mathbb{Y}}(g \cdot d(\nu), d(\nu)^{-1} \cdot h) & = \gamma_F(-1, \tfrac{1}{2} \psi)^{-1} \cdot \gamma_F(-J_1, \tfrac{1}{2} \psi) \cdot (-2 \nu a b J_2, J_1)_F \\ & = \gamma_F(J_1, \tfrac{1}{2} \psi) \cdot (2 \nu a b J_2, J_1)_F. \end{align*} $$

On the other hand, since $x(h \cdot d(\nu )^{-1}) \equiv 1 \bmod (F^{\times })^2$ and $j(h \cdot d(\nu )^{-1}) = 4$ , we have

$$\begin{align*}v(h \cdot d(\nu)^{-1}, \nu^{-1}) = 1. \end{align*}$$

Thus, we obtain

$$\begin{align*}z_{\mathbb{Y}}(g, h) = \gamma_F(J_1, \tfrac{1}{2} \psi) \cdot (2 \nu a b J_2, J_1)_F. \end{align*}$$

This completes the proof.

Now, we compute $s(1, \alpha , \alpha )$ . Note that $[1, \alpha ] \in \mathrm {GU}(V)^0$ is the image of $\alpha $ under the embedding $E^\times \hookrightarrow \mathrm {GU}(V)$ as in §A.1. Hence, by definition, we have

$$\begin{align*}s(1,\alpha,\alpha) = \frac{s^\sharp(\alpha, \alpha, \alpha, \alpha)}{s_2(\iota(1, [\alpha, \alpha]))}, \end{align*}$$

where $[\alpha , \alpha ] \in \mathrm {U}(\mathbf {W})$ . Since

$$ \begin{align*} s_2(\iota([\alpha, \alpha], [\alpha, \alpha])) & = s_2(\iota([\alpha, \alpha], 1)) \cdot s_2(\iota(1, [\alpha, \alpha])) \cdot z_{\mathbb{Y}^\square}(\iota([\alpha, \alpha], 1), \iota(1, [\alpha, \alpha])) \\ & = s_2(\iota([\alpha, \alpha], 1)) \cdot s_2(\iota(1, [\alpha, \alpha])), \end{align*} $$

we have

$$ \begin{align*} s(1,\alpha,\alpha) & = \frac{s^\sharp(\alpha, \alpha, \alpha, \alpha)}{s_2(\iota([\alpha, \alpha], [\alpha, \alpha]))} \cdot s_2(\iota([\alpha, \alpha], 1)) \\ & = \frac{\hat{s}^\sharp(\alpha, \alpha, \alpha, \alpha)}{\hat{s}_2(\iota([\alpha, \alpha], [\alpha, \alpha]))} \cdot \hat{s}_2(\iota([\alpha, \alpha], 1)) \cdot \mu(\iota([\alpha, \alpha], 1)). \end{align*} $$

Hence, by definition and Lemma A.6, we have

$$\begin{align*}s(1, \alpha, \alpha) = \chi(\alpha)^2 \cdot \hat{s}_2(\iota([\alpha, \alpha], 1)) \cdot \mu(\iota([\alpha, \alpha], 1)). \end{align*}$$

Lemma A.16. We have

$$\begin{align*}\hat{s}_2(\iota([\alpha, \alpha], 1)) = \chi(\alpha)^{-2} \cdot (u, J_1)_F. \end{align*}$$

Proof. Put $h = \iota ([\alpha , \alpha ], 1) \in \mathrm {U}(\mathbf {W}^\square )$ and $\beta = \alpha ^{-1} \alpha ^\rho $ so that $\beta - 1 \in E^\times $ . As in the proof of Lemma A.4, we have

$$\begin{align*}\begin{bmatrix} \mathbf{w}_1 \cdot h \\ \mathbf{w}_2 \cdot h \\ \mathbf{w}_1^* \cdot h \\ \mathbf{w}_2^* \cdot h \end{bmatrix} = A \cdot \begin{bmatrix} \mathbf{w}_1 \\ \mathbf{w}_2 \\ \mathbf{w}_1^* \\ \mathbf{w}_2^* \end{bmatrix}, \end{align*}$$

where

$$ \begin{align*} A & = \begin{pmatrix} 1 & & & \\ & \frac{1}{2} (\beta+1) & & \frac{1}{4 J \mathbf{i}} (\beta-1) \\ & & 1 & \\ & J \mathbf{i} (\beta-1) & & \frac{1}{2} (\beta+1) \end{pmatrix} \\ & = \begin{pmatrix} 1 & & & \\ & -\frac{1}{J \mathbf{i} (\beta^\rho - 1)} & & * \\ & & 1 & \\ & & & J \mathbf{i} (\beta-1) \end{pmatrix} \cdot \tau_1 \cdot \begin{pmatrix} 1 & & & \\ & 1 & & * \\ & & 1 & \\ & & & 1 \end{pmatrix}. \end{align*} $$

Hence, we have

$$\begin{align*}\hat{s}_2(\iota([\alpha, \alpha], 1)) = \chi(J \mathbf{i} (\beta -1))^2 \cdot \gamma^{-1}, \end{align*}$$

where

$$ \begin{align*} \gamma & = (u, \det \mathbf{V})_F \cdot \gamma_F(-u, \tfrac{1}{2} \psi)^2 \cdot \gamma_F(-1, \tfrac{1}{2} \psi)^{-2} \\ & = (u, -J_1)_F \cdot (-1, -u)_F \cdot (-1, -1)_F \\ & = (u, J_1)_F. \end{align*} $$

Since $\beta -1 = \alpha ^{-1}(\alpha ^\rho - \alpha ) = - 2b \mathbf {i} \alpha ^{-1}$ , we have $\chi (J \mathbf {i} (\beta -1))^2 = \chi (- 2b uJ \alpha ^{-1})^2 = \chi (\alpha )^{-2}$ . This completes the proof.

Lemma A.17. We have

$$\begin{align*}\mu(\iota([\alpha, \alpha], 1)) = \gamma_F(J_1, \tfrac{1}{2} \psi) \cdot (- 2abuJ_2, J_1)_F. \end{align*}$$

Proof. We write $\alpha ^{-1} \alpha ^\rho = c + d \mathbf {i}$ with $c, d \in F$ so that

$$\begin{align*}c = \frac{a^2 + b^2 u}{a^2 - b^2 u} \ne \pm 1, \qquad d = - \frac{2ab}{a^2 - b^2 u} \ne 0. \end{align*}$$

Recall that

$$\begin{align*}\mu(\iota([\alpha, \alpha], 1)) = z_{\mathbb{Y}^\square}(\sigma_0, \sigma)^{-1} \cdot z_{\mathbb{Y}^\square}(\sigma_0 \sigma \sigma_0^{-1}, \sigma_0), \end{align*}$$

where $\sigma $ is the image of $\iota ([\alpha , \alpha ], 1)$ in $\mathrm {Sp}(\mathbb {V}^\square )$ . We have

$$\begin{align*}\sigma_0 = \prod_{i=1}^4 \iota_i(\tau_1 \cdot \mathbf{m}(\mathbf{a}_1)), \qquad \sigma = \prod_{i=3}^4 \iota_i(\sigma_i), \end{align*}$$

where

$$\begin{align*}\mathbf{a}_1 = \begin{pmatrix} \frac{1}{2} & - \frac{1}{2} \\ - 1 & - 1 \end{pmatrix}, \qquad \sigma_i = \begin{pmatrix} c & & d k_i u & \\ & 1 & & \\ \frac{d}{k_i} & & c & \\ & & & 1 \end{pmatrix}, \qquad k_i = \begin{cases} J_2 & \text{if } i=3, \\ -J & \text{if } i=4. \end{cases} \end{align*}$$

Since $\tau _1 \cdot \mathbf {m}(\mathbf {a}_1) \in P_{\mathbb {Y}_i^\square } \cdot \tau _1 \cdot \mathbf {m}(\mathbf {a}_2)$ and $\sigma _i \in \mathbf {n}(\mathbf {b}_{1,i}) \cdot \tau \cdot P_{\mathbb {Y}_i^\square }$ , where

$$\begin{align*}\mathbf{a}_2 = \begin{pmatrix} 1 & \\ 1 & 1 \end{pmatrix}, \qquad \mathbf{b}_{1,i} = \frac{c k_i}{d} \cdot \begin{pmatrix} 1 & \\ & 0 \end{pmatrix}, \qquad \tau = \begin{pmatrix} & & -1 & \\ & 1 & & \\ 1 & & & \\ & & & 1 \end{pmatrix}, \end{align*}$$

we have

$$ \begin{align*} z_{\mathbb{Y}^\square}(\sigma_0, \sigma) & = \prod_{i=3}^4 z_{\mathbb{Y}_i^\square}(\tau_1 \cdot \mathbf{m}(\mathbf{a}_1), \sigma_i) \\ & = \prod_{i=3}^4 z_{\mathbb{Y}_i^\square}(\tau_1, \mathbf{m}(\mathbf{a}_2) \cdot \mathbf{n}(\mathbf{b}_{1,i}) \cdot \tau). \end{align*} $$

If we put

$$\begin{align*}\mathbf{b}_{2,i} = \frac{c k_i}{d} \cdot \begin{pmatrix} 1 & 1 \\ 1 & \end{pmatrix}, \qquad \mathbf{b}_{3,i} = \frac{c k_i}{d} \cdot \begin{pmatrix} 0 & \\ & 1 \end{pmatrix}, \end{align*}$$

then we have $\mathbf {m}(\mathbf {a}_2) \cdot \mathbf {n}(\mathbf {b}_{1,i}) = \mathbf {n}(\mathbf {b}_{2,i}) \cdot \mathbf {n}(\mathbf {b}_{3,i}) \cdot \mathbf {m}(\mathbf {a}_2)$ and hence

$$\begin{align*}z_{\mathbb{Y}^\square}(\sigma_0, \sigma) = \prod_{i=3}^4 z_{\mathbb{Y}_i^\square}(\tau_1 \cdot \mathbf{n}(\mathbf{b}_{2,i}), \mathbf{n}(\mathbf{b}_{3,i}) \cdot \mathbf{m}(\mathbf{a}_2) \cdot \tau). \end{align*}$$

Since $\tau _1 \cdot \mathbf {n}(\mathbf {b}_{2,i}) \in P_{\mathbb {Y}_i^\square } \cdot \tau _1$ and $\mathbf {n}(\mathbf {b}_{3,i}) \cdot \mathbf {m}(\mathbf {a}_2) \cdot \tau \in \tau \cdot P_{\mathbb {Y}_i^\square }$ , we have

$$\begin{align*}z_{\mathbb{Y}^\square}(\sigma_0, \sigma) = \prod_{i=3}^4 z_{\mathbb{Y}_i^\square}(\tau_1, \tau) = 1. \end{align*}$$

On the other hand, we have

$$\begin{align*}\sigma_0 \sigma \sigma_0^{-1} = \prod_{i=3}^4 \iota_i(\sigma_i'), \end{align*}$$

where

$$\begin{align*}\sigma_i' = \begin{pmatrix} \frac{1}{2} (c+1) & \frac{d k_i u}{2} & \frac{d k_i u}{4} & -\frac{1}{4} (c-1) \\ \frac{d}{2 k_i} & \frac{1}{2} (c+1) & \frac{1}{4} (c-1) & - \frac{d}{4 k_i} \\ \frac{d}{k_i} & c-1 & \frac{1}{2} (c+1) & -\frac{d}{2 k_i} \\ -c+1 & - d k_i u & - \frac{d k_i u}{2} & \frac{1}{2} (c+1) \end{pmatrix}. \end{align*}$$

Since $\sigma _i' \in P_{\mathbb {Y}_i^\square } \cdot \tau _2 \cdot \mathbf {n}(\mathbf {b}_{4,i}) \cdot \mathbf {n}(\mathbf {b}_{5,i})$ , where

$$\begin{align*}\mathbf{b}_{4,i} = - \frac{d}{2(c-1) k_i} \cdot \begin{pmatrix} 0 & \\ & 1 \end{pmatrix}, \qquad \mathbf{b}_{5,i} = \frac{d k_i u}{2(c-1)} \cdot \begin{pmatrix} 1 & \\ & 0 \end{pmatrix}, \end{align*}$$

we have

$$ \begin{align*} z_{\mathbb{Y}^\square}(\sigma_0 \sigma \sigma_0^{-1}, \sigma_0) & = \prod_{i=3}^4 z_{\mathbb{Y}_i^\square}(\sigma_i', \tau_1 \cdot \mathbf{m}(\mathbf{a}_1)) \\ & = \prod_{i=3}^4 z_{\mathbb{Y}_i^\square}(\tau_2, \mathbf{n}(\mathbf{b}_{4,i}) \cdot \mathbf{n}(\mathbf{b}_{5,i}) \cdot \tau_1). \end{align*} $$

Hence, since $\mathbf {n}(\mathbf {b}_{5,i}) \cdot \tau _1 \in \tau _1 \cdot P_{\mathbb {Y}_i^{\square }}$ and

$$\begin{align*}\frac{d}{c-1} = - \frac{a}{bu}, \end{align*}$$

we have

$$ \begin{align*} z_{\mathbb{Y}^\square}(\sigma_0 \sigma \sigma_0^{-1}, \sigma_0) & = \prod_{i=3}^4 z_{\mathbb{Y}_i^\square}(\tau_2, \mathbf{n}(\mathbf{b}_{4,i}) \cdot \tau_1) \\ & = \prod_{i=3}^4 \left[ \gamma_F(\tfrac{1}{2} \psi) \cdot \gamma_F(2ab k_i u, \tfrac{1}{2} \psi) \right] \\ & = \gamma_F(-1, \tfrac{1}{2} \psi)^{-1} \cdot \gamma_F(k_3 k_4, \tfrac{1}{2} \psi) \cdot (2ab k_3 u, 2ab k_4 u)_F \\ & = \gamma_F(-k_3 k_4, \tfrac{1}{2} \psi) \cdot (-2ab k_3 u, -k_3 k_4)_F. \end{align*} $$

This completes the proof.

By Lemmas A.16 and A.17, we have

$$\begin{align*}s(1, \alpha, \alpha) = \gamma_F(J_1, \tfrac{1}{2} \psi) \cdot (- 2abJ_2, J_1)_F. \end{align*}$$

Now, equation (A.9) follows from this and Lemma A.15.

A.11.3 Proof of equation (A.10)

In this section, we will show that

$$\begin{align*}\boldsymbol{\chi}(\alpha, \alpha^{-1}, 1) = 1 \end{align*}$$

for $\alpha \in E^\times $ . We write $\alpha = a + b \mathbf {i}$ with $a, b \in F$ and put $\nu = a^2 - b^2 u$ . Since $\boldsymbol {\chi }$ is continuous, we may assume that

$$\begin{align*}a \ne 0, \qquad b \ne 0. \end{align*}$$

Lemma A.18. We have

$$\begin{align*}\tilde{s}(\alpha, \alpha^{-1}, 1) = \gamma_F(J, \tfrac{1}{2} \psi) \cdot (-2abJ_1, J)_F. \end{align*}$$

Proof. Put $g_1 = [\alpha , 1] \in \mathrm {GU}(V)^0$ and $g_2 = [1, \alpha ^{-1}] \in \mathrm {GU}(V)^0$ . Then we have $\tilde {s}(\alpha , \alpha ^{-1}, 1) = \tilde {s}(g_1) \cdot \tilde {s}(g_2) \cdot z_{\mathbb {Y}}(g_1, g_2)$ . By [Reference Ichino and Prasanna30, Proposition C.4.2], we have

$$\begin{align*}\tilde{s}(g_1) = (- \nu J_1, J_2)_F, \qquad \tilde{s}(g_2) = (- \nu J_2, J_1)_F. \end{align*}$$

We have

$$\begin{align*}g_1 = \nu^{-1} \cdot \begin{pmatrix} a \cdot {\mathbf{1}}_4 & -bu \cdot \mathbf{J}_1 \\ -b \cdot \mathbf{J}_1^{-1} & a \cdot {\mathbf{1}}_4 \end{pmatrix}, \qquad g_2 = \begin{pmatrix} a \cdot {\mathbf{1}}_4 & bu \cdot \mathbf{J}_2 \\ b \cdot \mathbf{J}_2^{-1} & a \cdot {\mathbf{1}}_4 \end{pmatrix} \end{align*}$$

in $\mathrm {GSp}(\mathbb {V})$ , where

$$\begin{align*}\mathbf{J}_1 = \begin{pmatrix} 1 & & & \\ & J_1 & & \\ & & -J_2 & \\ & & & -J \end{pmatrix}, \qquad \mathbf{J}_2 = \begin{pmatrix} 1 & & & \\ & -J_1 & & \\ & & J_2 & \\ & & & -J \end{pmatrix}. \end{align*}$$

Hence, as in the proof of Lemma A.15, we have

$$\begin{align*}z_{\mathbb{Y}}(g_1, g_2) = \gamma_F(J, \tfrac{1}{2} \psi) \cdot (2 \nu abJ_1, J)_F. \end{align*}$$

This completes the proof.

Now, we compute $s(\alpha , \alpha ^{-1}, 1)$ . By definition, we have

$$ \begin{align*} s(\alpha,\alpha^{-1}, 1) & = s^\sharp([\alpha, \alpha^{-1}], 1, 1, 1) \\ & = \hat{s}^\sharp([\alpha, \alpha^{-1}], 1, 1, 1) \cdot \mu(\iota([\alpha, \alpha^{-1}], 1)) \\ & = \hat{s}_1(\iota([\alpha, \alpha^{-1}], 1)) \cdot \mu(\iota([\alpha, \alpha^{-1}], 1)), \end{align*} $$

where $[\alpha , \alpha ^{-1}] \in \mathrm {U}(V)^0$ .

Lemma A.19. We have

$$\begin{align*}\hat{s}_1(\iota([\alpha, \alpha^{-1}], 1)) = (u, J)_F. \end{align*}$$

Proof. Put $g = \iota ([\alpha , \alpha ^{-1}], 1) \in \mathrm {U}(V^\square )$ and $\beta = \alpha ^{-1} \alpha ^\rho $ so that $\beta - 1 \in E^\times $ . As in the proof of Lemma A.4, we have

$$\begin{align*}\begin{bmatrix} \mathbf{v}_1 \cdot g \\ \mathbf{v}_2 \cdot g \\ \mathbf{v}_1^* \cdot g \\ \mathbf{v}_2^* \cdot g \end{bmatrix} = A \cdot \begin{bmatrix} \mathbf{v}_1 \\ \mathbf{v}_2 \\ \mathbf{v}_1^* \\ \mathbf{v}_2^* \end{bmatrix}, \end{align*}$$

where

$$ \begin{align*} A & = \begin{pmatrix} 1 & & & \\ & \frac{1}{2} (\beta+1) & & - \frac{1}{4 J_1 \mathbf{i}} (\beta-1) \\ & & 1 & \\ & -J_1 \mathbf{i} (\beta-1) & & \frac{1}{2} (\beta+1) \end{pmatrix} \\ & = \begin{pmatrix} 1 & & & \\ & \frac{1}{J_1 \mathbf{i} (\beta^\rho - 1)} & & * \\ & & 1 & \\ & & & -J_1 \mathbf{i} (\beta-1) \end{pmatrix} \cdot \tau_1 \cdot \begin{pmatrix} 1 & & & \\ & 1 & & * \\ & & 1 & \\ & & & 1 \end{pmatrix}. \end{align*} $$

Hence, we have

$$\begin{align*}\hat{s}_1(g) = \begin{cases} 1 & \text{if } B \text{ is split,} \\ -1 & \text{if } B \text{ is ramified.} \end{cases} \end{align*}$$

This completes the proof.

Lemma A.20. We have

$$\begin{align*}\mu(\iota([\alpha, \alpha^{-1}], 1)) = \gamma_F(J, \tfrac{1}{2} \psi) \cdot (-2abu J_1, J)_F. \end{align*}$$

Proof. Recall that

$$\begin{align*}\mu(\iota([\alpha, \alpha^{-1}], 1)) = z_{\mathbb{Y}^\square}(\sigma_0, \sigma)^{-1} \cdot z_{\mathbb{Y}^\square}(\sigma_0 \sigma \sigma_0^{-1}, \sigma_0), \end{align*}$$

where $\sigma $ is the image of $\iota ([\alpha , \alpha ^{-1}], 1)$ in $\mathrm {Sp}(\mathbb {V}^\square )$ . If we write $\alpha ^{-1} \alpha ^\rho = c + d \mathbf {i}$ with $c, d \in F$ , then we have

$$\begin{align*}\sigma = \prod_{i=2}^3 \iota_i(\sigma_i), \end{align*}$$

where

$$\begin{align*}\sigma_i = \begin{pmatrix} c & & d k_i u & \\ & 1 & & \\ \frac{d}{k_i} & & c & \\ & & & 1 \end{pmatrix}, \qquad k_i = \begin{cases} J_1 & \text{if } i=2, \\ -J_2 & \text{if } i=3. \end{cases} \end{align*}$$

Hence, as in the proof of Lemma A.17, we have $z_{\mathbb {Y}^\square }(\sigma _0, \sigma ) = 1$ and

$$\begin{align*}z_{\mathbb{Y}^\square}(\sigma_0 \sigma \sigma_0^{-1}, \sigma_0) = \gamma_F(-k_2 k_3, \tfrac{1}{2} \psi) \cdot (-2ab k_2 u, -k_2 k_3)_F. \end{align*}$$

This completes the proof.

By Lemmas A.19 and A.20, we have

$$\begin{align*}s(\alpha, \alpha^{-1}, 1) = \gamma_F(J, \tfrac{1}{2} \psi) \cdot (-2ab J_1, J)_F. \end{align*}$$

Now, equation (A.10) follows from this and Lemma A.18.

A.12 Proof of equation (A.11)

Assume that $J_1, J_2 \in (F^\times )^2$ . Choose $t_i \in F^\times $ such that $J_i = t_i^2$ , and put $\mathbf {j}_i^\natural = t_i^{-1} \cdot \mathbf {j}_i$ . In this section, we will show that

$$\begin{align*}\boldsymbol{\chi}(\mathbf{j}_1^\natural, \mathbf{j}_2^\natural, 1) = 1. \end{align*}$$

Lemma A.21. We have

$$\begin{align*}\tilde{s}(\mathbf{j}_1^\natural, \mathbf{j}_2^\natural, 1) = 1. \end{align*}$$

Proof. Put $g_1 = [\mathbf {j}_1^\natural , 1] \in \mathrm {GU}(V)^0$ and $g_2 = [1, \mathbf {j}_2^\natural ] \in \mathrm {GU}(V)^0$ . Then we have $\tilde {s}(\mathbf {j}_1^\natural , \mathbf {j}_2^\natural , 1) = \tilde {s}(g_1) \cdot \tilde {s}(g_2) \cdot z_{\mathbb {Y}}(g_1, g_2)$ . By [Reference Ichino and Prasanna30, Proposition C.4.2], we have

$$\begin{align*}\tilde{s}(g_1) = \tilde{s}(g_2) = 1. \end{align*}$$

It remains to compute $z_{\mathbb {Y}}(g_1, g_2)$ .

Noting that $\nu (g_1) = \nu (g_2) = -1$ , we have

$$\begin{align*}z_{\mathbb{Y}}(g_1, g_2) = z_{\mathbb{Y}}(g_1 \cdot d(-1), d(-1) \cdot g_2) \cdot v(g_2 \cdot d(-1), -1). \end{align*}$$

We have

$$\begin{align*}g_1 = \mathbf{m}(\mathbf{a}_1) \cdot d(-1), \qquad g_2 = \mathbf{m}(\mathbf{a}_2) \cdot d(-1) \end{align*}$$

in $\mathrm {GSp}(\mathbb {V})$ , where

$$\begin{align*}\mathbf{a}_1 = \begin{pmatrix} & \frac{1}{t_1} & & \\ t_1 & & & \\ & & & \frac{1}{t_1} \\ & & t_1 & \end{pmatrix}, \qquad \mathbf{a}_2 = \begin{pmatrix} & & \frac{1}{t_2} & \\ & & & \frac{1}{t_2} \\ t_2 & & & \\ & t_2 & & \end{pmatrix}. \end{align*}$$

Hence, we have

$$\begin{align*}z_{\mathbb{Y}}(g_1 \cdot d(-1), d(-1) \cdot g_2) = 1. \end{align*}$$

On the other hand, since $x(g_2 \cdot d(-1)) = 1$ and $j(g_2 \cdot d(-1)) = 0$ , we have

$$\begin{align*}v(g_2 \cdot d(-1), -1) = 1. \end{align*}$$

Thus, we obtain $z_{\mathbb {Y}}(g_1, g_2) = 1$ . This completes the proof.

Now, we compute $s(\mathbf {j}_1^\natural , \mathbf {j}_2^\natural , 1)$ . By definition, we have

$$ \begin{align*} s(\mathbf{j}_1^\natural, \mathbf{j}_2^\natural, 1) & = s^\sharp([\mathbf{j}_1^\natural, \mathbf{j}_2^\natural], 1, 1, 1) \\ & = \hat{s}^\sharp([\mathbf{j}_1^\natural, \mathbf{j}_2^\natural], 1, 1, 1) \cdot \mu(\iota([\mathbf{j}_1^\natural, \mathbf{j}_2^\natural], 1)) \\ & = \hat{s}_1(\iota([\mathbf{j}_1^\natural, \mathbf{j}_2^\natural], 1)) \cdot \mu(\iota([\mathbf{j}_1^\natural, \mathbf{j}_2^\natural], 1)), \end{align*} $$

where $[\mathbf {j}_1^\natural , \mathbf {j}_2^\natural ] \in \mathrm {U}(V)^0$ .

Lemma A.22. We have

$$\begin{align*}\hat{s}_1(\iota([\mathbf{j}_1^\natural, \mathbf{j}_2^\natural], 1)) = 1. \end{align*}$$

Proof. Since B is split, the assertion follows.

Lemma A.23. We have

$$\begin{align*}\mu(\iota([\mathbf{j}_1^\natural, \mathbf{j}_2^\natural], 1)) = 1. \end{align*}$$

Proof. Recall that

$$\begin{align*}\mu(\iota([\mathbf{j}_1^\natural, \mathbf{j}_2^\natural], 1)) = z_{\mathbb{Y}^\square}(\sigma_0, \sigma)^{-1} \cdot z_{\mathbb{Y}^\square}(\sigma_0 \sigma \sigma_0^{-1}, \sigma_0), \end{align*}$$

where $\sigma $ is the image of $\iota ([\mathbf {j}_1^\natural , \mathbf {j}_2^\natural ], 1)$ in $\mathrm {Sp}(\mathbb {V}^\square )$ . Since

$$\begin{align*}\sigma = \mathbf{m}(\mathbf{a}_1), \end{align*}$$

where

$$\begin{align*}\mathbf{a}_1 = \begin{pmatrix} \mathbf{a} & \\ & {\mathbf{1}}_4 \end{pmatrix}, \qquad \mathbf{a} = \begin{pmatrix} & & & \frac{1}{t_1 t_2} \\ & & \frac{t_1}{t_2} & \\ & \frac{t_2}{t_1} & & \\ t_1 t_2 & & & \end{pmatrix}, \end{align*}$$

we have $z_{\mathbb {Y}^\square }(\sigma _0, \sigma ) = 1$ . On the other hand, we have

$$\begin{align*}\sigma_0 \sigma \sigma_0^{-1} = \begin{pmatrix} \tfrac{1}{2} ({\mathbf{1}}_4 + \mathbf{a}) & & & \tfrac{1}{4} ({\mathbf{1}}_4 - \mathbf{a}) \\ & \tfrac{1}{2} ({\mathbf{1}}_4 + {}^t \mathbf{a}) & - \tfrac{1}{4} ({\mathbf{1}}_4 - {}^t \mathbf{a}) & \\ & - {\mathbf{1}}_4 + {}^t \mathbf{a} & \tfrac{1}{2} ({\mathbf{1}}_4 + {}^t \mathbf{a}) & \\ {\mathbf{1}}_4 - \mathbf{a} & & & \tfrac{1}{2} ({\mathbf{1}}_4 + \mathbf{a}) \end{pmatrix}. \end{align*}$$

Since $\sigma _0 \sigma \sigma _0^{-1} \in P_{\mathbb {Y}^\square } \cdot \tau \cdot \mathbf {m}(\mathbf {a}_2)$ and $\sigma _0 \in \tau _4 \cdot P_{\mathbb {Y}^\square }$ , where

$$\begin{align*}\mathbf{a}_2 = \begin{pmatrix} \mathbf{a}' & \\ & \mathbf{a}" \end{pmatrix}, \qquad \mathbf{a}' = \begin{pmatrix} 1 & & & \\ & 1 & & \\ & -\frac{t_2}{t_1} & 1 & \\ -t_1 t_2 & & & 1 \end{pmatrix}, \qquad \mathbf{a}" = \begin{pmatrix} 1 & & & \\ & 1 & & \\ & -\frac{t_1}{t_2} & 1 & \\ -\frac{1}{t_1 t_2} & & & 1 \end{pmatrix}, \end{align*}$$

and

$$\begin{align*}\tau = \begin{pmatrix} {\mathbf{1}}_2 & & & & & & & \\ & {\mathbf{0}}_2 & & & & -{\mathbf{1}}_2 & & \\ & & {\mathbf{1}}_2 & & & & & \\ & & & {\mathbf{0}}_2 & & & & -{\mathbf{1}}_2 \\ & & & & {\mathbf{1}}_2 & & & \\ & {\mathbf{1}}_2 & & & & {\mathbf{0}}_2 & & \\ & & & & & & {\mathbf{1}}_2 & \\ & & & {\mathbf{1}}_2 & & & & {\mathbf{0}}_2 \end{pmatrix}, \end{align*}$$

we have

$$\begin{align*}z_{\mathbb{Y}^\square}(\sigma_0 \sigma \sigma_0^{-1}, \sigma_0) = z_{\mathbb{Y}^\square}(\tau, \mathbf{m}(\mathbf{a}_2) \cdot \tau_4). \end{align*}$$

Hence, since $\mathbf {m}(\mathbf {a}_2) \cdot \tau _4 \in \tau _4 \cdot P_{\mathbb {Y}^\square }$ , we have

$$\begin{align*}z_{\mathbb{Y}^\square}(\sigma_0 \sigma \sigma_0^{-1}, \sigma_0) = z_{\mathbb{Y}^\square}(\tau, \tau_4) = 1. \end{align*}$$

This completes the proof.

By Lemmas A.22 and A.23, we have

$$\begin{align*}s(\mathbf{j}_1^\natural, \mathbf{j}_2^\natural, 1) = 1. \end{align*}$$

Now, equation (A.11) follows from this and Lemma A.21.

A.13 Compatibility with [Reference Harris and Kudla26]

Suppose that F is local. In this section, we compare the splitting s with the standard one for symplectic-orthogonal dual pairs when B is split. In this case, we have $J \in \mathrm {N}_{E/F}(E^\times )$ so that we may write $J = k^2 - l^2 u$ for some $k, l \in F$ . We define an isomorphism $\mathfrak {i}:B \rightarrow \mathrm {M}_2(F)$ of quaternion F-algebras by

$$\begin{align*}\mathfrak{i}(a + b \mathbf{i} + c \mathbf{j} + d \mathbf{i} \mathbf{j}) = \begin{pmatrix} a & b \\ bu & a \end{pmatrix} + \begin{pmatrix} c & d \\ du & c \end{pmatrix} \begin{pmatrix} k & -l \\ lu & -k \end{pmatrix}. \end{align*}$$

Put

$$ \begin{align*} e & = \frac{1}{2} + \frac{k}{2J} \mathbf{j} - \frac{l}{2J} \mathbf{i} \mathbf{j}, & e' & = \frac{1}{2} \mathbf{i} + \frac{lu}{2J} \mathbf{j} - \frac{k}{2J} \mathbf{i} \mathbf{j}, \\ e" & = \frac{1}{2u} \mathbf{i} - \frac{l}{2J} \mathbf{j} + \frac{k}{2uJ} \mathbf{i} \mathbf{j}, & e^* & = \frac{1}{2} - \frac{k}{2J} \mathbf{j} + \frac{l}{2J} \mathbf{i} \mathbf{j} \end{align*} $$

so that

$$\begin{align*}\mathfrak{i}(e) = \begin{pmatrix} 1 & 0 \\ 0 & 0 \end{pmatrix}, \qquad \mathfrak{i}(e') = \begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix}, \qquad \mathfrak{i}(e") = \begin{pmatrix} 0 & 0 \\ 1 & 0 \end{pmatrix}, \qquad \mathfrak{i}(e^*) = \begin{pmatrix} 0 & 0 \\ 0 & 1 \end{pmatrix}. \end{align*}$$

In particular, we have

$$\begin{align*}\begin{bmatrix} e \cdot x \\ e' \cdot x \end{bmatrix} = \mathfrak{i}(x) \cdot \begin{bmatrix} e \\ e' \end{bmatrix} \end{align*}$$

for $x \in B$ .

Let V be an m-dimensional skew-Hermitian right B-space as in equation (A.1). We consider the $2m$ -dimensional quadratic F-space $V^\dagger := V e$ given by Morita theory (see [Reference Ichino and Prasanna30, §C.2] for details). With respect to a basis $e_1 e, e_1 e", \dots , e_m e, e_m e"$ of $V^\dagger $ , the symmetric bilinear form on $V^\dagger $ is associated to

$$\begin{align*}\frac{1}{2} \cdot \begin{pmatrix} \kappa_1 u & & & & \\ & - \kappa_1& & & \\ & & \ddots & & \\ & & & \kappa_m u & \\ & & & & - \kappa_m \end{pmatrix}. \end{align*}$$

Similarly, we consider the two-dimensional symplectic F-space $W^\dagger := e W$ . Then, by [Reference Ichino and Prasanna30, Lemma C.2.2], we have an identification

$$\begin{align*}\mathbb{V} = V^\dagger \otimes_F W^\dagger. \end{align*}$$

We take a complete polarization $W^\dagger = X \oplus Y$ defined by

$$\begin{align*}X = F e, \qquad Y = F e'. \end{align*}$$

This induces a complete polarization $\mathbb {V} = \mathbb {X}' \oplus \mathbb {Y}'$ , where

$$\begin{align*}\mathbb{X}' = V^\dagger \otimes_F X, \qquad \mathbb{Y}' = V^\dagger \otimes_F Y. \end{align*}$$

More explicitly, we have

$$ \begin{align*} \mathbb{X}' & = F \cdot e_1 \otimes e + \dots + F \cdot e_m \otimes e + F \cdot e_1 \otimes e" + \dots + F \cdot e_m \otimes e", \\ \mathbb{Y}' & = F \cdot e_1 \otimes e' + \dots + F \cdot e_m \otimes e' + F \cdot e_1 \otimes e^* + \dots + F \cdot e_m \otimes e^*. \end{align*} $$

Now, we recall the splitting defined in [Reference Harris and Kudla26, §5.1]. Using a basis $e, e'$ of $W^\dagger $ , we identify $\mathrm {GSp}(W^\dagger )$ with $\mathrm {GSp}_2(F) = \mathrm {GL}_2(F)$ . We define a map

$$\begin{align*}s^\dagger: \mathrm{Sp}(W^\dagger) \longrightarrow \mathbb{C}^1 \end{align*}$$

by setting

$$\begin{align*}s^\dagger(h) = \xi_E(x(h))^m \cdot \gamma^{\prime -j(h)}, \end{align*}$$

where $x(h)$ and $j(h)$ are as in §A.11.1, and

$$\begin{align*}\gamma' = \gamma_F(\tfrac{1}{2} \psi)^{2m} \cdot \gamma_F(\det V^\dagger, \tfrac{1}{2} \psi) \cdot h_F(V^\dagger). \end{align*}$$

We extend $s^\dagger $ to a map

$$\begin{align*}s^\dagger: \mathrm{G}(\mathrm{O}(V^\dagger) \times \mathrm{Sp}(W^\dagger)) \longrightarrow \mathbb{C}^1 \end{align*}$$

so that

$$\begin{align*}s^\dagger(\mathbf{g}) = s^\dagger(h \cdot d(\nu(h))^{-1}) \end{align*}$$

for $\mathbf {g} = (g, h) \in \mathrm {G}(\mathrm {O}(V^\dagger ) \times \mathrm {Sp}(W^\dagger ))$ .

Lemma A.24. For $\mathbf {g}_1, \mathbf {g}_2 \in \mathrm {G}(\mathrm {O}(V^\dagger ) \times \mathrm {Sp}(W^\dagger ))$ , we have

$$\begin{align*}z_{\mathbb{Y}'}(\mathbf{g}_1, \mathbf{g}_2) = \frac{s^\dagger(\mathbf{g}_1 \mathbf{g}_2)}{s^\dagger(\mathbf{g}_1) s^\dagger(\mathbf{g}_2)}. \end{align*}$$

Proof. If $\mathbf {g}_1, \mathbf {g}_2 \in \mathrm {Sp}(W^\dagger )$ , then the assertion follows from [Reference Kudla40, Theorem 3.1, cases ${1}_{+}$ ]. By [Reference Harris and Kudla26, §5.1], this implies the general case. Nevertheless, we include a direct argument for the convenience of the reader.

Let $\mathbf {g}_i = (g_i, h_i) \in \mathrm {G}(\mathrm {O}(V^\dagger ) \times \mathrm {Sp}(W^\dagger ))$ . Recall that

$$\begin{align*}z_{\mathbb{Y}'}(\mathbf{g}_1, \mathbf{g}_2) = \gamma_F(\tfrac{1}{2} \psi \circ q(\mathbb{Y}' \cdot \mathbf{g}_1^{-1}, \mathbb{Y}' \cdot \mathbf{g}_2^{-1} \mathbf{g}_1^{-1}, \mathbb{Y}')), \end{align*}$$

where q denotes the Leray invariant (see, e.g., [Reference Ranga Rao58], [Reference Ichino and Prasanna30, §3.1.2]). Put

$$\begin{align*}\nu_i = \nu(h_i), \qquad h_i' = h_i \cdot d(\nu_i)^{-1}, \qquad h_2" = d(\nu_1) \cdot h_2' \cdot d(\nu_1)^{-1} \end{align*}$$

so that

$$\begin{align*}h_1 h_2 = h_1' \cdot d(\nu_1) \cdot h_2' \cdot d(\nu_2) = h_1' h_2" \cdot d(\nu_1 \nu_2). \end{align*}$$

Since $\mathbb {Y}' \cdot (g, d(\nu )) = \mathbb {Y}'$ for $g \in \mathrm {GO}(V^\dagger )$ and $\nu \in F^{\times }$ , we have $\mathbb {Y}' \cdot \mathbf {g}_1^{-1} = \mathbb {Y}' \cdot h_1^{\prime -1}$ and $\mathbb {Y}' \cdot \mathbf {g}_2^{-1} \mathbf {g}_1^{-1} = \mathbb {Y}' \cdot h_2^{\prime \prime -1} h_1^{\prime -1}$ so that

$$\begin{align*}q(\mathbb{Y}' \cdot \mathbf{g}_1^{-1}, \mathbb{Y}' \cdot \mathbf{g}_2^{-1} \mathbf{g}_1^{-1}, \mathbb{Y}') = q(\mathbb{Y}' \cdot h_1^{\prime -1}, \mathbb{Y}' \cdot h_2^{\prime\prime -1} h_1^{\prime -1}, \mathbb{Y}'). \end{align*}$$

Hence, we have

$$\begin{align*}z_{\mathbb{Y}'}(\mathbf{g}_1, \mathbf{g}_2) = z_{\mathbb{Y}'}(h_1', h_2") = \frac{s^\dagger(h^{\prime}_1 h^{\prime\prime}_2)}{s^\dagger(h^{\prime}_1) s^\dagger(h^{\prime\prime}_2)}. \end{align*}$$

On the other hand, by definition, we have $s^\dagger (\mathbf {g}_i) = s^\dagger (h^{\prime }_i)$ and $s^\dagger (\mathbf {g}_1 \mathbf {g}_2) = s^\dagger (h_1' h_2")$ . By equation (A.12), and noting that $\nu (\mathrm {GO}(V^\dagger )) = \mathrm {N}_{E/F}(E^\times )$ if m is odd, we have

$$\begin{align*}s^\dagger(h_2") = \xi_E(\nu_1)^{j(h_2') m} \cdot s^\dagger(h_2') = s^\dagger(h_2'). \end{align*}$$

This completes the proof.

Fix $\varsigma _0 \in \mathrm {Sp}(\mathbb {V})$ such that $\mathbb {X} = \mathbb {X}' \cdot \varsigma _0$ and $\mathbb {Y} = \mathbb {Y}' \cdot \varsigma _0$ . Put

$$\begin{align*}\mu_0(\sigma) = z_{\mathbb{Y}'}(\varsigma_0, \sigma) \cdot z_{\mathbb{Y}'}(\varsigma_0 \sigma \varsigma_0^{-1}, \varsigma_0)^{-1} \end{align*}$$

for $\sigma \in \mathrm {Sp}(\mathbb {V})$ . Note that $\mu _0$ does not depend on the choice of $\varsigma _0$ . Then, by [Reference Kudla40, Lemma 4.2], we have

$$\begin{align*}z_{\mathbb{Y}}(\sigma, \sigma') = z_{\mathbb{Y}'}(\sigma, \sigma') \cdot \frac{\mu_0(\sigma \sigma')}{\mu_0(\sigma) \mu_0(\sigma')} \end{align*}$$

for $\sigma , \sigma ' \in \mathrm {Sp}(\mathbb {V})$ .

Put $s_0 = s^\dagger \cdot \mu _0$ . Via the canonical isomorphisms $\mathrm {GU}(V) \simeq \mathrm {GO}(V^\dagger )$ and $\mathrm {GU}(W) \simeq \mathrm {GSp}(W^\dagger )$ , we regard $s_0$ as a map

$$\begin{align*}s_0 : \mathrm{G}(\mathrm{U}(V) \times \mathrm{U}(W)) \longrightarrow \mathbb{C}^1. \end{align*}$$

By Lemma A.24, we have

$$\begin{align*}z_{\mathbb{Y}}(\mathbf{g}_1, \mathbf{g}_2) = \frac{s_0(\mathbf{g}_1 \mathbf{g}_2)}{s_0(\mathbf{g}_1) s_0(\mathbf{g}_2)} \end{align*}$$

for $\mathbf {g}_1, \mathbf {g}_2 \in \mathrm {G}(\mathrm {U}(V) \times \mathrm {U}(W))$ .

Proposition A.25. We have

$$\begin{align*}s_0|_{\mathcal{G}} = s. \end{align*}$$

The rest of this section is devoted to the proof of Proposition A.25.

As in §A.3, we define the doubled space $W^{\dagger \square } = W^\dagger \oplus W^\dagger $ and take the complete polarization $W^{\dagger \square } = W^{\dagger \bigtriangledown } \oplus W^{\dagger \triangle }$ . Then we have identifications

$$\begin{align*}\mathbb{V}^\square = V^\dagger \otimes_F W^{\dagger \square}, \qquad \mathbb{V}^\bigtriangledown = V^\dagger \otimes_F W^{\dagger \bigtriangledown}, \qquad \mathbb{V}^\triangle = V^\dagger \otimes_F W^{\dagger \triangle}. \end{align*}$$

We also take complete polarizations $W^{\dagger \square } = X^\square \oplus Y^\square $ and $\mathbb {V}^\square = \mathbb {X}^{\prime \square } \oplus \mathbb {Y}^{\prime \square }$ , where

$$ \begin{align*} X^\square & = X \oplus X, & \mathbb{X}^{\prime\square} & = \mathbb{X}' \oplus \mathbb{X}' = V^\dagger \otimes_F X^\square, \\ Y^\square & = Y \oplus Y, & \mathbb{Y}^{\prime\square} & = \mathbb{Y}' \oplus \mathbb{Y}' = V^\dagger \otimes_F Y^\square. \end{align*} $$

As in [Reference Ichino and Prasanna30, §D.3], we have

$$\begin{align*}z_{\mathbb{Y}^{\prime\square}}(\iota(\sigma_1,\sigma_2), \iota(\sigma_1',\sigma_2')) = z_{\mathbb{Y}'}(\sigma_1, \sigma_1') \cdot z_{\mathbb{Y}'}(\sigma_2, \sigma_2')^{-1} \end{align*}$$

for $\sigma _i, \sigma _i' \in \mathrm {Sp}(\mathbb {V})$ . Using a basis $(e,0), (0,e), (e',0), (0,-e')$ of $W^{\dagger \square }$ , we identify $\mathrm {GSp}(W^{\dagger \square })$ with $\mathrm {GSp}_4(F)$ . Put

$$\begin{align*}h_0 = \begin{pmatrix} \frac{1}{2} & -\frac{1}{2} & & \\ & & \frac{1}{2} & \frac{1}{2} \\ & & 1 & -1 \\ -1 & -1 & & \end{pmatrix} \in \mathrm{Sp}(W^{\dagger \square}). \end{align*}$$

Then we have

$$\begin{align*}\begin{bmatrix} \frac{1}{2}(e, -e) \\ \frac{1}{2}(e', -e') \\ (e', e') \\ (-e, -e) \end{bmatrix} = h_0 \cdot \begin{bmatrix} (e, 0) \\ (0, e) \\ (e', 0) \\ (0, -e') \end{bmatrix} \end{align*}$$

so that $W^{\dagger \bigtriangledown } = X^\square \cdot h_0$ and $W^{\dagger \triangle } = Y^\square \cdot h_0$ . Put $\mathbf {h}_0 = \mathrm {id} \otimes h_0 \in \mathrm {Sp}(\mathbb {V}^\square )$ and

$$\begin{align*}\mu'(\sigma) = z_{\mathbb{Y}^{\prime\square}}(\mathbf{h}_0, \sigma)^{-1} \cdot z_{\mathbb{Y}^{\prime\square}}(\mathbf{h}_0 \sigma \mathbf{h}_0^{-1}, \mathbf{h}_0) \end{align*}$$

for $\sigma \in \mathrm {Sp}(\mathbb {V}^\square )$ . Since $\mathbb {V}^\bigtriangledown = \mathbb {X}^{\prime \square } \cdot \mathbf {h}_0$ and $\mathbb {V}^\triangle = \mathbb {Y}^{\prime \square } \cdot \mathbf {h}_0$ , we have

(A.13) $$ \begin{align} z_{\mathbb{Y}^{\prime\square}}(\sigma, \sigma') = z_{\mathbb{V}^\triangle}(\sigma, \sigma') \cdot \frac{\mu'(\sigma \sigma')}{\mu'(\sigma) \mu'(\sigma')} \end{align} $$

for $\sigma , \sigma ' \in \mathrm {Sp}(\mathbb {V}^\square )$ .

As in §A.6, we put $s^{\sharp \prime } = \hat {s}^\sharp \cdot \mu '$ and $s_2' = \hat {s}_2 \cdot \mu '$ and define a map

$$\begin{align*}s': \mathcal{G} \longrightarrow \mathbb{C}^1 \end{align*}$$

by setting

$$\begin{align*}s'(g,h) = \frac{s^{\sharp \prime}(g,h,\alpha,\alpha)}{s^{\prime}_2(\iota(1, [\alpha,\alpha]))}, \end{align*}$$

where we choose $\alpha \in E^\times $ such that $\nu (g) = \nu (h) = \mathrm {N}_{E/F}(\alpha )$ . As in Lemma A.10, $s'$ is well defined. Moreover, as in Lemma A.11, we have

$$\begin{align*}z_{\mathbb{Y}'}(\mathbf{g}_1,\mathbf{g}_2) = \frac{s'(\mathbf{g}_1\mathbf{g}_2)}{s'(\mathbf{g}_1) s'(\mathbf{g}_2)} \end{align*}$$

for $\mathbf {g}_1, \mathbf {g}_2 \in \mathcal {G}$ .

Lemma A.26. We have

$$\begin{align*}s = s' \cdot \mu_0. \end{align*}$$

Proof. Put $\varsigma _{00} = \iota (\varsigma _0, \varsigma _0) \in \mathrm {Sp}(\mathbb {V}^\square )$ and

$$\begin{align*}\mu_{00}(\sigma) = z_{\mathbb{Y}^{\prime\square}}(\varsigma_{00}, \sigma) \cdot z_{\mathbb{Y}^{\prime\square}}(\varsigma_{00} \sigma \varsigma_{00}^{-1}, \varsigma_{00})^{-1} \end{align*}$$

for $\sigma \in \mathrm {Sp}(\mathbb {V}^\square )$ . Since $\mathbb {X}^\square = \mathbb {X}^{\prime \square } \cdot \varsigma _{00}$ and $\mathbb {Y}^\square = \mathbb {Y}^{\prime \square } \cdot \varsigma _{00}$ , we have

(A.14) $$ \begin{align} z_{\mathbb{Y}^\square}(\sigma, \sigma') = z_{\mathbb{Y}^{\prime\square}}(\sigma, \sigma') \cdot \frac{\mu_{00}(\sigma \sigma')}{\mu_{00}(\sigma) \mu_{00}(\sigma')} \end{align} $$

for $\sigma , \sigma ' \in \mathrm {Sp}(\mathbb {V}^\square )$ . Then it follows from equations (A.8), (A.13) and (A.14) that

$$\begin{align*}\frac{\mu_{00}(\sigma \sigma')}{\mu_{00}(\sigma) \mu_{00}(\sigma')} = \frac{\mu(\sigma \sigma')}{\mu(\sigma) \mu(\sigma')} \cdot \frac{\mu'(\sigma) \mu'(\sigma')}{\mu'(\sigma \sigma')} \end{align*}$$

for $\sigma , \sigma ' \in \mathrm {Sp}(\mathbb {V}^\square )$ . Namely, $\mu _{00} \cdot \mu ' / \mu $ is a character of $\mathrm {Sp}(\mathbb {V}^\square )$ . Since $[\mathrm {Sp}(\mathbb {V}^\square ), \mathrm {Sp}(\mathbb {V}^\square )] = \mathrm {Sp}(\mathbb {V}^\square )$ , this character must be trivial and hence

$$\begin{align*}\mu_{00} = \mu/\mu'. \end{align*}$$

Since

$$ \begin{align*} \mu_{00}(\iota(\sigma,1)) & = z_{\mathbb{Y}^{\prime\square}}(\iota(\varsigma_0, \varsigma_0), \iota(\sigma, 1)) \cdot z_{\mathbb{Y}^{\prime\square}}(\iota(\varsigma_0 \sigma \varsigma_0^{-1}, 1), \iota(\varsigma_0, \varsigma_0))^{-1} \\ & = z_{\mathbb{Y}'}(\varsigma_0, \sigma) \cdot z_{\mathbb{Y}'}(\varsigma_0 \sigma \varsigma_0^{-1}, \varsigma_0)^{-1} \\ & = \mu_0(\sigma) \end{align*} $$

for $\sigma \in \mathrm {Sp}(\mathbb {V})$ , it suffices to show that

$$\begin{align*}\frac{s(\mathbf{g})}{\mu(\iota(\mathbf{g}, 1))} = \frac{s'(\mathbf{g})}{\mu'(\iota(\mathbf{g}, 1))} \end{align*}$$

for $\mathbf {g} \in \mathcal {G}$ . Here, by abuse of notation, we write $\mathbf {g}$ in the denominator for the image of $\mathbf {g}$ in $\mathrm {Sp}(\mathbb {V})$ under equation (A.2) so that $\iota (\mathbf {g}, 1) \in \mathrm {Sp}(\mathbb {V}^\square )$ .

For $\mathbf {g} = (g, h) \in \mathcal {G}$ , choose $\alpha \in E^\times $ such that $\nu (g) = \nu (h) = \mathrm {N}_{E/F}(\alpha )$ . Put $\mathbf {g}^\sharp = (g,h,\alpha ,\alpha ) \in \mathcal {G}^\sharp $ and $\boldsymbol {\alpha } = [\alpha , \alpha ] \in \mathrm {U}(\mathbf {W})$ . Note that the image of $\mathbf {g}^\sharp $ in $\mathrm {Sp}(\mathbb {V}^\square )$ agrees with $\iota (\mathbf {g}, 1) \cdot \iota (1,\boldsymbol {\alpha })$ . By definition, we have

$$\begin{align*}s(\mathbf{g}) = \frac{\hat{s}^\sharp(\mathbf{g}^\sharp)}{\hat{s}_2(\iota(1, \boldsymbol{\alpha}))} \cdot \frac{\mu(\mathbf{g}^\sharp)}{\mu(\iota(1, \boldsymbol{\alpha}))}, \qquad s'(\mathbf{g}) = \frac{\hat{s}^\sharp(\mathbf{g}^\sharp)}{\hat{s}_2(\iota(1, \boldsymbol{\alpha}))} \cdot \frac{\mu'(\mathbf{g}^\sharp)}{\mu'(\iota(1, \boldsymbol{\alpha}))}. \end{align*}$$

Thus, it remains to show that

$$\begin{align*}\frac{\mu(\mathbf{g}^\sharp)}{\mu(\iota(\mathbf{g},1)) \cdot \mu(\iota(1, \boldsymbol{\alpha}))} = \frac{\mu'(\mathbf{g}^\sharp)}{\mu'(\iota(\mathbf{g},1)) \cdot \mu'(\iota(1, \boldsymbol{\alpha}))}. \end{align*}$$

But the left-hand side is equal to

$$\begin{align*}\frac{z_{\mathbb{Y}^\square}(\iota(\mathbf{g}, 1), \iota(1, \boldsymbol{\alpha}))}{z_{\mathbb{V}^\triangle}(\iota(\mathbf{g}, 1), \iota(1, \boldsymbol{\alpha}))} = \frac{1}{z_{\mathbb{V}^\triangle}(\iota(\mathbf{g}, 1), \iota(1, \boldsymbol{\alpha}))} \end{align*}$$

by equation (A.8), whereas the right-hand side is equal to

$$\begin{align*}\frac{z_{\mathbb{Y}^{\prime\square}}(\iota(\mathbf{g}, 1), \iota(1, \boldsymbol{\alpha}))}{z_{\mathbb{V}^\triangle}(\iota(\mathbf{g}, 1), \iota(1, \boldsymbol{\alpha}))} = \frac{1}{z_{\mathbb{V}^\triangle}(\iota(\mathbf{g}, 1), \iota(1, \boldsymbol{\alpha}))} \end{align*}$$

by equation (A.13). This completes the proof.

Thus, to finish the proof of Proposition A.25, it remains to prove the following.

Lemma A.27. We have

$$\begin{align*}s^\dagger|_{\mathcal{G}} = s'. \end{align*}$$

Proof. Since both $s^\dagger $ and $s'$ trivialize $z_{\mathbb {Y}'}$ , there exists a continuous character $\boldsymbol {\chi }'$ of $\mathcal {G}$ such that

$$\begin{align*}s^\dagger|_{\mathcal{G}} = s' \cdot \boldsymbol{\chi}'. \end{align*}$$

We will show that $\boldsymbol {\chi }'$ is trivial. Since $[\mathrm {Sp}(W^\dagger ), \mathrm {Sp}(W^\dagger )] = \mathrm {Sp}(W^\dagger )$ , $\boldsymbol {\chi }'$ is trivial on $\mathrm {U}(W) \simeq \mathrm {Sp}(W^\dagger )$ . Let $\mathbf {g} = (g,1) \in \mathcal {G}$ with $g \in \mathrm {U}(V)^0 \simeq \mathrm {SO}(V^\dagger )$ . By definition, we have $s^\dagger (\mathbf {g}) = 1$ and

$$\begin{align*}s'(\mathbf{g}) = s^{\sharp \prime}(g,1,1,1) = \hat{s}^\sharp(g,1,1,1) \cdot \mu'(\iota(\mathbf{g}, 1)) = \mu'(\iota(\mathbf{g}, 1)). \end{align*}$$

Since $\iota (\mathbf {g}, 1)$ belongs to $P_{\mathbb {Y}^{\prime \square }}$ and commutes with $\mathbf {h}_0$ , we have

$$\begin{align*}\mu'(\iota(\mathbf{g}, 1)) = z_{\mathbb{Y}^{\prime\square}}(\mathbf{h}_0, \iota(\mathbf{g},1))^{-1} \cdot z_{\mathbb{Y}^{\prime\square}}(\iota(\mathbf{g}, 1), \mathbf{h}_0) = 1. \end{align*}$$

Hence, we have $s'(\mathbf {g}) = 1$ so that $\boldsymbol {\chi }'(\mathbf {g}) = 1$ .

Thus, it remains to show that

$$\begin{align*}\boldsymbol{\chi}'(\mathbf{g}) = 1 \end{align*}$$

for $\mathbf {g} = (\alpha ,\alpha ) \in \mathcal {G}$ with $\alpha \in E^\times $ . We write $\alpha = a + b \mathbf {i}$ with $a, b \in F$ and put $\nu = a^2 - b^2 u$ . Since $\boldsymbol {\chi }'$ is continuous, we may assume that

$$\begin{align*}a \ne 0, \qquad b \ne 0. \end{align*}$$

By definition, we have $s^\dagger (\mathbf {g}) = s^\dagger (h)$ , where

$$\begin{align*}h = \begin{pmatrix} a & b \\ bu & a \end{pmatrix} \cdot \begin{pmatrix} 1 & \\ & \nu^{-1} \end{pmatrix}. \end{align*}$$

Since

$$\begin{align*}h = \begin{pmatrix} \frac{1}{bu} & a \\ & bu \end{pmatrix} \cdot \begin{pmatrix} & -1 \\ 1 & \end{pmatrix} \cdot \begin{pmatrix} 1 & \frac{a}{\nu bu} \\ & 1 \end{pmatrix}, \end{align*}$$

we have

$$\begin{align*}s^\dagger(h) = \xi_E(bu)^m \cdot \gamma^{\prime -1} = (-b, u)_F^m \cdot \gamma^{\prime -1}. \end{align*}$$

Recall that

$$ \begin{align*} \gamma^{\prime -1} & = \gamma_F(\tfrac{1}{2} \psi)^{-2m} \cdot \gamma_F(\det V^\dagger, \tfrac{1}{2} \psi)^{-1} \cdot h_F(V^\dagger) \\ & = \gamma_F(-1, \tfrac{1}{2} \psi)^m \cdot \gamma_F((-u)^m, \tfrac{1}{2} \psi)^{-1} \cdot h_F(V^\dagger) \\ & = \gamma_F(-1, \tfrac{1}{2} \psi)^m \cdot \gamma_F(-u, \tfrac{1}{2} \psi)^{-m} \cdot (-u,-u)_F^{(m-1)m/2} \cdot h_F(V^\dagger) \\ & = \gamma_F(u, \tfrac{1}{2} \psi)^m \cdot (-u,-u)_F^{(m-1)m/2} \cdot h_F(V^\dagger). \end{align*} $$

If we put $V_i^\dagger = F e_i e + F e_i e"$ , then we have

$$ \begin{align*} h_F(V^\dagger) & = \prod_{i=1}^m h_F(V_i^\dagger) \cdot \prod_{1 \le i < j \le m} (\det V_i^\dagger, \det V_j^\dagger)_F \\ & = \prod_{i=1}^m (- 2\kappa_i, u)_F \cdot (-u, -u)_F^{(m-1)m/2}. \end{align*} $$

Hence, we have

$$\begin{align*}s^\dagger(\mathbf{g}) = \gamma_F(u, \tfrac{1}{2} \psi)^m \cdot \prod_{i=1}^m (2 b \kappa_i, u)_F. \end{align*}$$

On the other hand, by definition, we have

$$ \begin{align*} s'(\mathbf{g}) & = \frac{s^{\sharp \prime}(\alpha,\alpha,\alpha,\alpha)}{s_2'(\iota(1, \boldsymbol{\alpha}))} \\ & = \frac{s^{\sharp \prime}(\alpha,\alpha,\alpha,\alpha)}{s_2'(\iota(\boldsymbol{\alpha}, \boldsymbol{\alpha}))} \cdot \frac{s_2'(\iota(\boldsymbol{\alpha}, \boldsymbol{\alpha}))}{s_2'(\iota(1, \boldsymbol{\alpha}))} \\ & = \frac{s^{\sharp \prime}(\alpha,\alpha,\alpha,\alpha)}{s_2'(\iota(\boldsymbol{\alpha}, \boldsymbol{\alpha}))} \cdot s_2'(\iota(\boldsymbol{\alpha}, 1)) \cdot z_{\mathbb{Y}^{\prime\square}}(\iota(\boldsymbol{\alpha}, 1), \iota(1, \boldsymbol{\alpha})) \\ & = \frac{s^{\sharp \prime}(\alpha,\alpha,\alpha,\alpha)}{s_2'(\iota(\boldsymbol{\alpha}, \boldsymbol{\alpha}))} \cdot s_2'(\iota(\boldsymbol{\alpha}, 1)) \\ & = \frac{\hat{s}^\sharp(\alpha,\alpha,\alpha,\alpha)}{\hat{s}_2(\iota(\boldsymbol{\alpha}, \boldsymbol{\alpha}))} \cdot \hat{s}_2(\iota(\boldsymbol{\alpha}, 1)) \cdot \mu'(\iota(\boldsymbol{\alpha}, 1)), \end{align*} $$

where $\boldsymbol {\alpha } = [\alpha , \alpha ] \in \mathrm {U}(\mathbf {W})$ . By definition and Lemma A.6, we have

$$\begin{align*}\frac{\hat{s}^\sharp(\alpha,\alpha,\alpha,\alpha)}{\hat{s}_2(\iota(\boldsymbol{\alpha}, \boldsymbol{\alpha}))} = \chi(\alpha)^m. \end{align*}$$

Also, as in Lemma A.16, we have

$$\begin{align*}\hat{s}_2(\iota(\boldsymbol{\alpha}, 1)) = \chi(J \mathbf{i} (\beta - 1))^m \cdot \gamma^{-1} = \chi(2 b \alpha^{-1})^m \cdot \gamma^{-1}, \end{align*}$$

where $\beta = \alpha ^{-1} \alpha ^\rho $ and

$$ \begin{align*} \gamma^{-1} & = (\det \mathbf{V}, u)_F \cdot \gamma_F(-u, \tfrac{1}{2} \psi)^{-m} \cdot \gamma_F(-1, \tfrac{1}{2} \psi)^m \\ & = \gamma(u, \tfrac{1}{2} \psi)^m \cdot \prod_{i=1}^m (\kappa_i, u)_F. \end{align*} $$

Hence, noting that the image of $\mathbf {g}$ in $\mathrm {Sp}(\mathbb {V})$ agrees with that of $\boldsymbol {\alpha }$ , we have

$$\begin{align*}s'(\mathbf{g}) = \gamma(u, \tfrac{1}{2} \psi)^m \cdot \prod_{i=1}^m (2b \kappa_i, u)_F \cdot \mu'(\iota(\mathbf{g}, 1)). \end{align*}$$

Thus, we are reduced to showing that

$$\begin{align*}\mu'(\iota(\mathbf{g}, 1)) = 1 \end{align*}$$

for $\mathbf {g} = (\alpha , \alpha ) \in \mathcal {G}$ with $\alpha = a + b \mathbf {i} \in E^\times $ such that $a \ne 0$ and $b \ne 0$ . This is further reduced to the case $\dim V = 1$ . Then we may identify $V^\dagger $ with the F-space $Fe + Fe"$ equipped with a symmetric bilinear form

$$\begin{align*}\langle x_1 e + x_2 e", y_1 e + y_2 e" \rangle^\dagger = \kappa u \cdot x_1 y_1 - \kappa \cdot x_2 y_2, \end{align*}$$

where $\kappa = \kappa _1/2$ . We take a basis

$$ \begin{align*} \mathbf{x}_1 & = e \otimes (e, 0), & \mathbf{y}_1 & = \frac{1}{\kappa u} \cdot e \otimes (e', 0), \\ \mathbf{x}_2 & = e" \otimes (e, 0), & \mathbf{y}_2 & = - \frac{1}{\kappa} \cdot e" \otimes (e', 0), \\ \mathbf{x}_3 & = e \otimes (0, e), & \mathbf{y}_3 & = \frac{1}{\kappa u} \cdot e \otimes (0, -e'), \\ \mathbf{x}_4 & = e" \otimes (0, e), & \mathbf{y}_4 & = - \frac{1}{\kappa} \cdot e" \otimes (0, -e') \end{align*} $$

of $\mathbb {V}^\square = V^\dagger \otimes _F W^{\dagger \square }$ so that

$$\begin{align*}\mathbb{X}^{\prime\square} = F \mathbf{x}_1 + \dots + F \mathbf{x}_4, \qquad \mathbb{Y}^{\prime\square} = F \mathbf{y}_1 + \dots + F \mathbf{y}_4, \qquad \langle \hspace{-1mm} \langle \mathbf{x}_i, \mathbf{y}_j \rangle \hspace{-1mm} \rangle = \delta_{ij}. \end{align*}$$

Using this basis, we identify $\mathrm {Sp}(\mathbb {V}^\square )$ with $\mathrm {Sp}_8(F)$ . Then we have

$$ \begin{align*} \mathbf{h}_0 & = \begin{pmatrix} \frac{1}{2} & & -\frac{1}{2} & & & & & \\ & \frac{1}{2} & & -\frac{1}{2} & & & & \\ & & & & \frac{\kappa u}{2} & & \frac{\kappa u}{2} & \\ & & & & & -\frac{\kappa}{2} & & -\frac{\kappa}{2} \\ & & & & 1 & & -1 & \\ & & & & & 1 & & -1 \\ -\frac{1}{\kappa u} & & -\frac{1}{\kappa u} & & & & & \\ & \frac{1}{\kappa} & & \frac{1}{\kappa} & & & & \end{pmatrix}, \\\iota(\mathbf{g}, 1) & = \begin{pmatrix} \frac{1}{2}(c+1) & \frac{du}{2} & & & -\frac{d\kappa u}{2} & \frac{\kappa}{2}(c-1) & & \\ \frac{d}{2} & \frac{1}{2}(c+1) & & & - \frac{\kappa}{2}(c-1) & \frac{d\kappa}{2} & & \\ & & 1 & & & & & \\ & & & 1 & & & & \\ -\frac{d}{2\kappa} & -\frac{1}{2\kappa}(c-1) & & & \frac{1}{2}(c+1) & -\frac{d}{2} & & \\ \frac{1}{2\kappa}(c-1) & \frac{du}{2\kappa} & & & -\frac{du}{2} & \frac{1}{2}(c+1) & & \\ & & & & & & 1 & \\ & & & & & & & 1 \end{pmatrix}, \end{align*} $$

where

$$\begin{align*}c = \frac{a^2 + b^2 u}{a^2 - b^2 u} \ne \pm 1, \qquad d = - \frac{2ab}{a^2 - b^2 u} \ne 0 \end{align*}$$

so that $c^2 - d^2 u = 1$ . Recall that

$$\begin{align*}\mu'(\iota(\mathbf{g}, 1)) = z_{\mathbb{Y}^{\prime\square}}(\mathbf{h}_0, \iota(\mathbf{g}, 1))^{-1} \cdot z_{\mathbb{Y}^{\prime\square}}(\mathbf{h}_0 \cdot \iota(\mathbf{g}, 1) \cdot \mathbf{h}_0^{-1}, \mathbf{h}_0). \end{align*}$$

Since $\mathbf {h}_0 \in P_{\mathbb {Y}^{\prime \square }} \cdot \tau _2 \cdot \mathbf {m}(\mathbf {a}_1)$ and $\iota (\mathbf {g}, 1) = \mathbf {n}(\mathbf {b}_1) \cdot \tau \cdot P_{\mathbb {Y}^{\prime \square }}$ , where

$$\begin{align*}\mathbf{a}_1 = \begin{pmatrix} 1 & & & \\ & 1 & & \\ 1 & & 1 & \\ & 1 & & 1 \end{pmatrix}, \qquad \mathbf{b}_1 = \frac{d \kappa}{c-1} \cdot \begin{pmatrix} -u & & & \\ & 1 & & \\ & & 0 & \\ & & & 0 \end{pmatrix}, \qquad \tau = \begin{pmatrix} & & -{\mathbf{1}}_2 & \\ & {\mathbf{1}}_2 & & \\ {\mathbf{1}}_2 & & & \\ & & & {\mathbf{1}}_2 \end{pmatrix}, \end{align*}$$

we have

$$ \begin{align*} z_{\mathbb{Y}^{\prime\square}}(\mathbf{h}_0, \iota(\mathbf{g}, 1)) & = z_{\mathbb{Y}^{\prime\square}}(\tau_2 \cdot \mathbf{m}(\mathbf{a}_1), \mathbf{n}(\mathbf{b}_1) \cdot \tau) \\ & = z_{\mathbb{Y}^{\prime\square}}(\tau_2, \mathbf{m}(\mathbf{a}_1) \cdot \mathbf{n}(\mathbf{b}_1) \cdot \tau). \end{align*} $$

If we put

$$\begin{align*}\mathbf{b}_2 = \frac{d \kappa}{c-1} \cdot \begin{pmatrix} -u & & -u & \\ & 1 & & 1 \\ -u & & & \\ & 1 & & \end{pmatrix}, \qquad \mathbf{b}_3 = \frac{d \kappa}{c-1} \cdot \begin{pmatrix} 0 & & & \\ & 0 & & \\ & & -u & \\ & & & 1 \end{pmatrix}, \end{align*}$$

then we have $\mathbf {m}(\mathbf {a}_1) \cdot \mathbf {n}(\mathbf {b}_1) = \mathbf {n}(\mathbf {b}_2) \cdot \mathbf {n}(\mathbf {b}_3) \cdot \mathbf {m}(\mathbf {a}_1)$ and hence

$$\begin{align*}z_{\mathbb{Y}^{\prime\square}}(\mathbf{h}_0, \iota(\mathbf{g}, 1)) = z_{\mathbb{Y}^{\prime\square}}(\tau_2 \cdot \mathbf{n}(\mathbf{b}_2), \mathbf{n}(\mathbf{b}_3) \cdot \mathbf{m}(\mathbf{a}_1) \cdot \tau). \end{align*}$$

Since $\tau _2 \cdot \mathbf {n}(\mathbf {b}_2) \in P_{\mathbb {Y}^{\prime \square }} \cdot \tau _2$ and $\mathbf {n}(\mathbf {b}_3) \cdot \mathbf {m}(\mathbf {a}_1) \cdot \tau \in \tau \cdot P_{\mathbb {Y}^{\prime \square }}$ , we have

$$\begin{align*}z_{\mathbb{Y}^{\prime\square}}(\mathbf{h}_0, \iota(\mathbf{g}, 1)) = z_{\mathbb{Y}^{\prime\square}}(\tau_2, \tau) = 1. \end{align*}$$

On the other hand, we have $\mathbf {h}_0 \in \tau _2 \cdot P_{\mathbb {Y}^{\prime \square }}$ and

$$ \begin{align*} & \mathbf{h}_0 \cdot \iota(\mathbf{g}, 1) \cdot \mathbf{h}_0^{-1} \\ & = \begin{pmatrix} \frac{1}{4}(c+3) & \frac{du}{4} & -\frac{d}{4} & -\frac{1}{4}(c-1) & -\frac{d \kappa u}{8} & \frac{\kappa}{8}(c-1) & -\frac{\kappa u}{8}(c-1) & \frac{d \kappa u}{8} \\ \frac{d}{4} & \frac{1}{4}(c+3) & -\frac{1}{4u}(c-1) & -\frac{d}{4} & -\frac{\kappa}{8}(c-1) & \frac{d \kappa}{8} & -\frac{d \kappa u}{8} & \frac{\kappa}{8}(c-1) \\ -\frac{du}{4} & -\frac{u}{4}(c-1) & \frac{1}{4}(c+3) & \frac{du}{4} & \frac{\kappa u}{8}(c-1) & -\frac{d \kappa u}{8} & \frac{d \kappa u^2}{8} & -\frac{\kappa u}{8}(c-1) \\ -\frac{1}{4}(c-1) & -\frac{du}{4} & \frac{d}{4} & \frac{1}{4}(c+3) & \frac{d \kappa u}{8} & -\frac{\kappa}{8}(c-1) & \frac{\kappa u}{8}(c-1) & -\frac{d \kappa u}{8} \\ -\frac{d}{2 \kappa} & -\frac{1}{2 \kappa}(c-1) & \frac{1}{2 \kappa u}(c-1) & \frac{d}{2 \kappa} & \frac{1}{4}(c+3) & -\frac{d}{4} & \frac{du}{4} & -\frac{1}{4}(c-1) \\ \frac{1}{2 \kappa}(c-1) & \frac{du}{2 \kappa} & -\frac{d}{2 \kappa} & -\frac{1}{2 \kappa}(c-1) & -\frac{du}{4} & \frac{1}{4}(c+3) & -\frac{u}{4}(c-1) & \frac{du}{4} \\ -\frac{1}{2 \kappa u}(c-1) & -\frac{d}{2 \kappa} & \frac{d}{2 \kappa u} & \frac{1}{2 \kappa u}(c-1) & \frac{d}{4} & -\frac{1}{4 u}(c-1) & \frac{1}{4}(c+3) & -\frac{d}{4} \\ \frac{d}{2 \kappa} & \frac{1}{2 \kappa}(c-1) & -\frac{1}{2 \kappa u}(c-1) & -\frac{d}{2 \kappa} & -\frac{1}{4}(c-1) & \frac{d}{4} & -\frac{du}{4} & \frac{1}{4}(c+3) \end{pmatrix} \\ & \in P_{\mathbb{Y}^{\prime\square}} \cdot \tau \cdot \mathbf{m}(\mathbf{a}_2) \cdot \mathbf{n}(\mathbf{b}_4), \end{align*} $$

where

$$\begin{align*}\mathbf{a}_2 = \begin{pmatrix} 1 & & & -1 \\ & 1 & -\frac{1}{u} & \\ & & 1 & \\ & & & 1 \end{pmatrix}, \qquad \mathbf{b}_4 = \frac{(c+1)\kappa}{d} \cdot \begin{pmatrix} 0 & & & \\ & 0 & & \\ & & u & \\ & & & -1 \end{pmatrix}. \end{align*}$$

Since $\tau \cdot \mathbf {n}(\mathbf {b}_4) \in P_{\mathbb {Y}^{\prime \square }} \cdot \tau $ and $\mathbf {n}(\mathbf {b}_4)^{-1} \cdot \mathbf {m}(\mathbf {a}_2) \cdot \mathbf {n}(\mathbf {b}_4) \cdot \tau _2 \in \tau _2 \cdot P_{\mathbb {Y}^{\prime \square }}$ , we have

$$ \begin{align*} z_{\mathbb{Y}^{\prime\square}}(\mathbf{h}_0 \cdot \iota(\mathbf{g}, 1) \cdot \mathbf{h}_0^{-1}, \mathbf{h}_0) & = z_{\mathbb{Y}^{\prime\square}}(\tau \cdot \mathbf{m}(\mathbf{a}_2) \cdot \mathbf{n}(\mathbf{b}_4), \tau_2) \\ & = z_{\mathbb{Y}^{\prime\square}}(\tau \cdot \mathbf{n}(\mathbf{b}_4), \mathbf{n}(\mathbf{b}_4)^{-1} \cdot \mathbf{m}(\mathbf{a}_2) \cdot \mathbf{n}(\mathbf{b}_4) \cdot \tau_2) \\ & = z_{\mathbb{Y}^{\prime\square}}(\tau, \tau_2) \\ & = 1. \end{align*} $$

This completes the proof.

Acknowledgments

We would like to thank the referees for many useful suggestions that improved the article. K.P. also thanks the Institute for Advanced Study for its support in 2014-15 as a von Neumann fellow, when the work on this paper was initiated.

Competing interest

The authors have no competing interest to declare.

Financial support

During the preparation of this article, A.I. was partially supported by JSPS KAKENHI Grant Number 26287003, and K.P. was partially supported by NSF grants DMS 1160720, DMS 1600494, DMS 2001293 and a grant from the Simons Foundation (# 305784).

References

Adams, J., ‘The theta correspondence over $\mathbb{R}$ ’, in Harmonic Analysis, Group Representations, Automorphic Forms and Invariant Theory, Lect. Notes Ser. Inst. Math. Sci. Natl. Univ. Singap., 12 (World Sci. Publ., Hackensack, NJ, 2007), 139.Google Scholar
Adams, J. and Johnson, J. F., ‘Endoscopic groups and packets of nontempered representations’, Compositio Math. 64(3) (1987), 271309.Google Scholar
Arancibia, N., Mœglin, C. and Renard, D., ‘Paquets d’Arthur des groupes classiques et unitaires’, Ann. Fac. Sci. Toulouse Math. (6) 27(5) (2018), no. 5, 10231105.CrossRefGoogle Scholar
Arthur, J., The Endoscopic Classification of Representations. Orthogonal and Symplectic Groups, American Mathematical Society Colloquium Publications, 61 (American Mathematical Society, Providence, RI, 2013).Google Scholar
Bergeron, N., Millson, J. and Mœglin, C., ‘The Hodge conjecture and arithmetic quotients of complex balls’, Acta Math. 216(1) (2016), 1125.CrossRefGoogle Scholar
Bergeron, N., Millson, J. and Mœglin, C., ‘Hodge type theorems for arithmetic manifolds associated to orthogonal groups’, Int. Math. Res. Not. IMRN (15) (2017), 44954624.Google Scholar
Blasius, D. and Rogawski, J. D., ‘Zeta functions of Shimura varieties’, Motives (Seattle, WA, 1991), Proc. Sympos. Pure Math., 55, Part 2, (Amer. Math. Soc., Providence, RI, 1994), 525571.CrossRefGoogle Scholar
Bloch, S. and Kato, K., ‘ $L$ -functions and Tamagawa numbers of motives’, The Grothendieck Festschrift, Vol. I, Progr. Math., 86 (Birkhäuser Boston, Boston, MA, 1990), 333400.Google Scholar
Borel, A., ‘Automorphic $L$ -functions’, Automorphic Forms, Representations and $L$ -Functions (Proc. Sympos. Pure Math., Oregon State Univ., Corvallis, Ore., 1977), Part 2, Proc. Sympos. Pure Math., XXXIII (Amer. Math. Soc., Providence, RI, 1979), 2761.Google Scholar
Borel, A. and Wallach, N., Continuous Cohomology, Discrete Subgroups, and Representations of Reductive Groups, second edn., Mathematical Surveys and Monographs, 67 (American Mathematical Society, Providence, RI, 2000).CrossRefGoogle Scholar
Brylinski, J.-L. and Labesse, J.-P., ‘Cohomologie d’intersection et fonctions L de certaines variétés de Shimura’, Ann. Sci. École Norm. Sup. (4) 17(3) (1984), 361412. French.CrossRefGoogle Scholar
Buzzard, K. and Gee, T., ‘The conjectural connections between automorphic representations and Galois representations’, Automorphic Forms and Galois Representations. Vol. 1, London Math. Soc. Lecture Note Ser., 414 (Cambridge Univ. Press, Cambridge, 2014), 135187.CrossRefGoogle Scholar
Carayol, H., ‘Sur les représentations $\ell$ -adiques associées aux formes modulaires de Hilbert’, Ann. Sci. École Norm. Sup. (4) 19(3) (1986), 409468. French.CrossRefGoogle Scholar
Deligne, P., ‘Variétés de Shimura: Interprétation modulaire, et techniques de construction de modèles canoniques’, Automorphic Forms, Representations and L-Functions (Proc. Sympos. Pure Math., Oregon State Univ., Corvallis, Ore., 1977), Part 2, Proc. Sympos, Pure Math., XXXIII (Amer. Math. Soc., Providence, RI, 1979), 247289. French.CrossRefGoogle Scholar
Deligne, P., Milne, J. S., Ogus, A. and Shih, K.-y., Hodge Cycles, Motives, and Shimura Varieties, Lecture Notes in Mathematics, 900 (Springer-Verlag, Berlin-New York, 1982).CrossRefGoogle Scholar
Faltings, G., ‘On the cohomology of locally symmetric Hermitian spaces’, Paul Dubreil and Marie-Paule Malliavin Algebra Seminar, 35th Year (Paris, 1982), Lecture Notes in Math., 1029 (Springer, Berlin, 1983), 5598.Google Scholar
Faltings, G., ‘Endlichkeitssätze für abelsche Varietäten über Zahlkörpern’, Invent. Math. 73(3) (1983), 349366. German.CrossRefGoogle Scholar
Fulton, W. and Harris, J., Representation Theory. A First Course, Graduate Texts in Mathematics, 129 Readings in Mathematics. (Springer-Verlag, New York, 1991), xvi+551.Google Scholar
Funke, J. and Millson, J., ‘Cycles with local coefficients for orthogonal groups and vector-valued Siegel modular forms’, Amer. J. Math. 128(4) (2006), 899948.CrossRefGoogle Scholar
Gan, W. T., Gross, B. H. and Prasad, D., ‘Symplectic local root numbers, central critical $L$ values, and restriction problems in the representation theory of classical groups’, Sur les conjectures de Gross et Prasad. I. Astérisque (346) (2012), 1109.Google Scholar
Gan, W. T., Qiu, Y. and Takeda, S., ‘The regularized Siegel–Weil formula (the second term identity) and the Rallis inner product formula’, Invent. Math. 198(3) (2014), 739831.CrossRefGoogle Scholar
Gan, W. T. and Sun, B., ‘The Howe duality conjecture: quaternionic case’, in Representation Theory, Number Theory, and Invariant Theory, Progr. Math., 323 (Birkhäuser/Springer, Cham, 2017), 175192.CrossRefGoogle Scholar
Gan, W. T. and Takeda, S., ‘A proof of the Howe duality conjecture’, J. Amer. Math. Soc. 29(2) (2016), 473493.CrossRefGoogle Scholar
Gan, W. T. and Tantono, W., ‘The local Langlands conjecture for $GSp(4)$ , II: The case of inner forms’, Amer. J. Math. 136(3) (2014), 761805.CrossRefGoogle Scholar
Harris, M., ‘Hodge–de Rham structures and periods of automorphic forms’, in Motives (Seattle, WA, 1991), Proc. Sympos. Pure Math., 55, Part 2, (Amer. Math. Soc., Providence, RI, 1994), 573624.Google Scholar
Harris, M. and Kudla, S. S., ‘Arithmetic automorphic forms for the nonholomorphic discrete series of $\mathit{\mathsf{GSp}}(2)$ ’, Duke Math. J. 66(1) (1992), 59121.CrossRefGoogle Scholar
Harris, M., Kudla, S. S. and Sweet, W. J., ‘Theta dichotomy for unitary groups’, J. Amer. Math. Soc. 9(4) (1996), 9411004.CrossRefGoogle Scholar
Howe, R., ‘Transcending classical invariant theory’, J. Amer. Math. Soc. 2(3) (1989), 535552.CrossRefGoogle Scholar
Ichino, A., ‘On the Siegel–Weil formula for unitary groups’, Math. Z. 255(4) (2007), 721729.CrossRefGoogle Scholar
Ichino, A. and Prasanna, K., Periods of Quaternionic Shimura Varieties. I, Contemporary Mathematics, 762 (American Mathematical Society, Providence, RI, 2021).CrossRefGoogle Scholar
Ichino, A. and Prasanna, K., ‘Periods of quaternionic Shimura varieties. II’, In preparation.Google Scholar
Johnson, J. F., ‘Stable base change of certain derived functor modules’, Math. Ann. 287(3) (1990), 467493.CrossRefGoogle Scholar
Kakuhama, H., ‘On the local factors of irreducible representations of quaternionic unitary groups’, Manuscripta Math. 163(1–2) (2020), 5786.CrossRefGoogle Scholar
Kaletha, T., Minguez, A., Shin, S. W. and White, P.-J., ‘Endoscopic classification of representations: inner forms of unitary groups’, Preprint, 2014, arXiv:1409.3731.Google Scholar
Kisin, M., Shin, S. W. and Zhu, Y., ‘The stable trace formula for Shimura varieties of abelian type’, Preprint, 2021, arXiv:2110.05381.Google Scholar
Knapp, A. W. and Vogan, D. A. Jr., Cohomological Induction and Unitary Representations, Princeton Mathematical Series, 45 (Princeton University Press, Princeton, NJ, 1995).CrossRefGoogle Scholar
Kottwitz, R. E., ‘Shimura varieties and $\lambda$ -adic representations’, in Automorphic Forms, Shimura Varieties, and $L$ -Functions, Vol. I (Ann Arbor, MI, 1988), Perspect. Math., 10 (Academic Press, Boston, 1990), 161209.Google Scholar
Kottwitz, R. E., ‘Points on some Shimura varieties over finite fields’, J. Amer. Math. Soc. 5(2) (1992), 373444.CrossRefGoogle Scholar
Kottwitz, R. E., ‘On the $\lambda$ -adic representations associated to some simple Shimura varieties’, Invent. Math. 108(3) (1992), 653665.CrossRefGoogle Scholar
Kudla, S. S., ‘Splitting metaplectic covers of dual reductive pairs’, Israel J. Math. 87(1–3) (1994), 361401.CrossRefGoogle Scholar
Kudla, S. S. and Millson, J. J., ‘The theta correspondence and harmonic forms. I’, Math. Ann. 274(3) (1986), 353378.CrossRefGoogle Scholar
Kudla, S. S. and Millson, J. J., ‘Intersection numbers of cycles on locally symmetric spaces and Fourier coefficients of holomorphic modular forms in several complex variables’, Inst. Hautes Études Sci. Publ. Math. (71) (1990), 121172.CrossRefGoogle Scholar
Kudla, S. S. and Rallis, S., Poles of Eisenstein Series and $L$ -Functions. Festschrift in honor of I. I. Piatetski-Shapiro on the occasion of his sixtieth birthday, Part II (Ramat Aviv, 1989), Israel Math. Conf. Proc., 3 (Weizmann, Jerusalem, 1990), 81110.Google Scholar
Kudla, S. S. and Rallis, S., ‘A regularized Siegel–Weil formula: The first term identity’, Ann. of Math.(2) 140(1) (1994), 180.CrossRefGoogle Scholar
Langlands, R. P., R. P. Modular forms and $\ell$ -adic representations’, in Modular Functions of One Variable , II. (Proc. Internat. Summer School, Univ. Antwerp, Antwerp, 1972), Lecture Notes in Math., Vol. 349, (Springer, Berlin, 1973), 361500.CrossRefGoogle Scholar
Lapid, E. M. and Rallis, S., ‘On the local factors of representations of classical groups’, Automorphic Representations, $L$ -Functions and Applications: Progress and Prospects, Ohio State Univ. Math. Res. Inst. Publ., 11 (de Gruyter, Berlin, 2005), 309359.Google Scholar
Li, J.-S., ‘Singular unitary representations of classical groups’, Invent. Math. 97(2) (1989), 237255.CrossRefGoogle Scholar
Li, J.-S., ‘Theta lifting for unitary representations with nonzero cohomology’, Duke Math. J. 61(3) (1990), 913937.CrossRefGoogle Scholar
Li, J.-S., ‘Nonvanishing theorems for the cohomology of certain arithmetic quotients’, J. Reine Angew. Math. 428 (1992), 177217.Google Scholar
Loke, H. Y., ‘Howe quotients of unitary characters and unitary lowest weight modules. With an appendix by Soo Teck Lee’, Represent. Theory 10 (2006), 2147.CrossRefGoogle Scholar
Mœglin, C., Vignéras, M.-F. and Waldspurger, J.-L., Correspondances de Howe sur un corps $p$ -adique, Lecture Notes in Mathematics, 1291 (Springer-Verlag, Berlin, 1987).Google Scholar
Mok, C. P., ‘Endoscopic classification of representations of quasi-split unitary groups’, Mem. Amer. Math. Soc. 235(1108) (2015).Google Scholar
Murty, V. K. and Ramakrishnan, D., ‘Period relations and the Tate conjecture for Hilbert modular surfaces’, Invent. Math. 89(2) (1987), 319345.CrossRefGoogle Scholar
Nekovář, J., ‘Eichler–Shimura relations and semisimplicity of étale cohomology of quaternionic Shimura varieties’, Ann. Sci. Éc. Norm. Supér. (4) 51(5) (2018), 11791252.CrossRefGoogle Scholar
Oda, T., Periods of Hilbert Modular Surfaces, Progress in Mathematics, 19 (Birkhäuser, Boston, 1982).CrossRefGoogle Scholar
Oda, T., ‘Hodge structures of Shimura varieties attached to the unit groups of quaternion algebras. Galois Groups and Their Representations (Nagoya, 1981), Adv. Stud. Pure Math., 2 (North-Holland, Amsterdam, 1983), 15–36.Google Scholar
Piatetski-Shapiro, I. I. Rallis, S., ‘L-functions for the classical groups’, in Explicit Constructions of Automorphic $L$ -Functions, Lecture Notes in Mathematics, 1254 (Springer-Verlag, 1987), 152.Google Scholar
Ranga Rao, R., ‘On some explicit formulas in the theory of Weil representation’, Pacific J. Math. 157(2) (1993), 335371.CrossRefGoogle Scholar
Reimann, H., The Semi-Simple Zeta Function of Quaternionic Shimura Varieties, Lecture Notes in Mathematics, 1657 (Springer-Verlag, Berlin, 1997).CrossRefGoogle Scholar
Repka, J., ‘Tensor products of unitary representations of ${SL}_2(\mathit{\mathsf{R}})$ ’, Amer. J. Math. 100(4) (1978), 747774.CrossRefGoogle Scholar
Roberts, B., ‘The theta correspondence for similitudes’, Israel J. Math. 94 (1996), 285317.CrossRefGoogle Scholar
Shin, S. W. and Templier, N., ‘On fields of rationality for automorphic representations’, Compos. Math. 150(12) (2014), 20032053.CrossRefGoogle Scholar
Sun, B. and Zhu, C.-B., ‘Conservation relations for local theta correspondence’, J. Amer. Math. Soc. 28(4) (2015), 939983.CrossRefGoogle Scholar
Tits, J., ‘Représentations linéaires irréductibles d’un groupe réductif sur un corps quelconque’, J. Reine Angew. Math. 247 (1971), 196220. French.Google Scholar
Vogan, D. A. and Zuckerman, G. J., ‘Unitary representations with nonzero cohomology’, Compositio Math. 53(1) (1984), 5190.Google Scholar
Waldspurger, J.-L., ‘Quelques propriétés arithmétiques de certaines formes automorphes sur GL(2)’, Compositio Math. 54(2) (1985), 121171. French.Google Scholar
Waldspurger, J.-L., ‘Sur les valeurs de certaines fonctions $L$ automorphes en leur centre de symétrie’, Compositio Math. 54(2) (1985), 173242.Google Scholar
Waldspurger, J.-L., Démonstration d’une conjecture de dualité de Howe dans le cas $p$ -adique, $p\ne 2$ . Festschrift in honor of I. I. Piatetski-Shapiro on the occasion of his sixtieth birthday, Part I (Ramat Aviv, 1989), Israel Math. Conf. Proc., 2 (Weizmann, Jerusalem, 1990), 267324.Google Scholar
Yamana, S., ‘Degenerate principal series representations for quaternionic unitary groupsIsrael J. Math. 185 (2011), 77124.CrossRefGoogle Scholar
Yamana, S., ‘On the Siegel-Weil formula for quaternionic unitary groups’, Amer. J. Math. 135(5) (2013), 13831432.CrossRefGoogle Scholar
Yamana, S., ‘L-functions and theta correspondence for classical groups’, Invent. Math. 196(3) (2014), 651732.CrossRefGoogle Scholar
Zucker, S., ‘Locally homogeneous variations of Hodge structure’, Enseign. Math. (2) 27(3–4) (1981), 243276 (1982).Google Scholar
Théorie des topos et cohomologie étale des schémas. Tome 3. (French) Séminaire de Géométrie Algébrique du Bois-Marie 1963–1964 (SGA 4). Dirigé par M. Artin, A. Grothendieck et J. L. Verdier. Avec la collaboration de P. Deligne et B. Saint-Donat. Lecture Notes in Mathematics, vol. 305 (Springer-Verlag, Berlin-New York, 1973).Google Scholar