2.1 The setting

Let ${\mathfrak g}$ be a symmetrisable Kac–Moody algebra with generalised Cartan matrix $\left (a_{ij}\right )_{i,j\in I}$, where I is a finite set. Let $\{d_i\mid i\in I\}$ be a set of relatively prime positive integers such that the matrix $\left (d_ia_{ij}\right )$ is symmetric. Let $\Pi =\{\alpha _i\mid i\in I\}$ be the set of simple roots for ${\mathfrak g}$ and let $Q={\mathbb Z}\Pi $ be the root lattice. Consider the symmetric bilinear form $(\cdot ,\cdot ):Q\times Q \rightarrow {\mathbb Z}$ defined by $\left (\alpha _i,\alpha _j\right )=d_ia_{ij}$ for all $i,j\in I$. Let W be the corresponding Weyl group with simple reflections $s_i$ for $i\in I$. Let ${\mathfrak g}'=[{\mathfrak g},{\mathfrak g}]$ be the derived subalgebra of ${\mathfrak g}$.

Throughout the paper, we fix a field k of characteristic $0$ and set ${\mathbb K}=k(q)$. The quantised enveloping algebra $U=U_q({\mathfrak g}')$ is the ${\mathbb K}$-algebra with generators $E_i, F_i, K_i^{\pm 1}$ for $i \in I$ and defining relations given in [Reference LusztigLus94, 3.1]. In particular, the generators $E_i, F_i$ satisfy the quantum Serre relations

$$ \begin{align*} S_{ij}\left(E_i,E_j\right)=0=S_{ij}\left(F_i,F_j\right) \end{align*} $$

for all $i,j\in I$, where

(2.1)$$ \begin{align} S_{ij}(x,y)=\sum_{\ell=0}^{1-a_{ij}}(-1)^\ell \genfrac{[}{]}{0pt}{}{1-a_{ij}}{\ell}_{q_i}x^{1-a_{ij}-\ell}y x^\ell \in {\mathbb K}[x,y], \end{align} $$

with $q_i=q^{d_i}$ denoting the quantum Serre polynomial [Reference LusztigLus94, Corollary 33.1.5]. The algebra U is a Hopf algebra with coproduct ${\varDelta }$ given by

(2.2)$$ \begin{align} {\varDelta}(E_i){=}E_i\otimes 1+ K_i\otimes E_i, \qquad {\varDelta}(F_i){=}F_i\otimes K_i^{-1}+1 \otimes F_i, \qquad {\varDelta}(K_i){=}K_i\otimes K_i, \end{align} $$

for all $i\in I$. For $\beta =\sum _{i\in I}n_i \alpha _i\in Q$, write $K_\beta =\prod _{i\in I}K_i^{n_i}$.

We write ${\mathbb N}=\{0,1,2,\dotsc \}$ and let $Q^+={\mathbb N}\Pi $ be the positive cone in the root lattice. For any $\mu \in Q^+$ we let $U^+_\mu =\mathrm {span}_{\mathbb K}\left \{E_{i_1} \dotsm E_{i_\ell }\mid \sum _{j=1}^\ell \alpha _{i_j}=\mu \right \}$ be the corresponding root space, where $\mathrm {span}_{\mathbb K}$ denotes the ${\mathbb K}$-linear span. Moreover, define $U^-_{-\mu }=\omega \left (U^+_\mu \right )$, where $\omega :U\rightarrow U$ is the algebra automorphism given in [Reference LusztigLus94, 3.1.3]. We will use the Lusztig–Kashiwara skew derivations $\partial ^R_i, \partial ^L_i:U^\pm \rightarrow U^\pm $ for $i\in I$, which are uniquely determined by

(2.3)$$ \begin{align} E_i y - y E_i &=\frac{K_i \partial^L_i(y) - \partial^R_i(y)K_i^{-1}}{q_i-q_i^{-1}} , \end{align} $$

(2.4)$$ \begin{align} F_i x - x F_i &= \frac{K_i^{-1} \partial^L_i(x) - \partial^R_i(x)K_i}{q_i-q_i^{-1}}, \end{align} $$

for $x\in U^+$ and $y\in U^-$ [Reference LusztigLus94, Proposition 3.1.6]. The skew derivations $\partial _i^L$ and $\partial _i^R$ satisfy the relations

(2.5)$$ \begin{align} \partial_i^L(fg)&=\partial_i^L(f)g +q^{\left(\alpha_i,\mu\right)}f \partial_i^L(g), \end{align} $$

(2.6)$$ \begin{align} \partial_i^R(fg)&=q^{\left(\alpha_i,\nu\right)}\partial_i^R(f)g +f \partial_i^R(g), \end{align} $$

for all $f\in U^-_{-\mu }$, $g\in U^-_{-\nu }$ and for all $f\in U^+_{\mu }$, $g\in U^+_{\nu }$.

For any $i\in I$, let $T_i:U\rightarrow U$ be the algebra automorphism denoted by $T^{\prime \prime }_{i,1}$ in [Reference LusztigLus94, 37.1]. The skew derivations $\partial _i^L$ and $\partial _i^R$ detect elements in the intersection $U^\pm \cap T_i\left (U^{\pm }\right )$. More precisely, for $x\in U^+$ and $y\in U^-$ we have the following equivalences [Reference LusztigLus94, 38.1.6, 37.2.4]:

(2.7)$$ \begin{align} x&\in T_i^{-1}(U^+) & &\Longleftrightarrow & \partial_i^L(x)&=0,\end{align} $$

(2.8)$$ \begin{align} x&\in T_i(U^+) & &\Longleftrightarrow & \partial_i^R(x)&=0,\end{align} $$

(2.9)$$ \begin{align} y&\in T_i^{-1}(U^-) & &\Longleftrightarrow & \partial_i^L(y)&=0,\end{align} $$

(2.10)$$ \begin{align} y&\in T_i(U^-) & &\Longleftrightarrow & \partial_i^R(y)&=0,\end{align} $$

Given a subset $X \subseteq I$, denote

$$\begin{align*}Q_X=\sum_{j\in X} {\mathbb Z} \alpha_j \quad \text{and} \quad Q_X^+=\sum_{j\in X}{\mathbb N} \alpha_j. \end{align*}$$

We write ${\mathcal M}_X$ to denote the subalgebra of U generated by $\left \{E_j, F_j, K_j^{\pm 1}\mid j\in X\right \}$, and ${\mathcal M}^+_X$ and ${\mathcal M}^-_X$ for the subalgebras generated by $\left \{E_j\mid j\in X\right \}$ and $\left \{F_j\mid j\in X\right \}$, respectively. We write $G_X^+$ and $G_X^-$ to denote the subalgebras of U generated by $\left \{E_jK_j^{-1}\mid j\in X\right \}$ and $\left \{F_jK_j\mid j\in X\right \}$, respectively. Moreover, we write $G^\pm $ for $G^\pm _I$.

2.2 Parabolic subalgebras

Let $X\subseteq I$ be a subset of finite type. The parabolic subalgebra

$$ \begin{align*} {\mathcal P}_X={\mathbb K}\left\langle F_i, E_j, K_i^{\pm 1}\mid i\in I, j\in X\right\rangle \end{align*} $$

is a Hopf subalgebra of U, and the corresponding Levi factor

$$ \begin{align*} {\mathcal L}_X={\mathbb K}\left\langle F_j, E_j, K_i^{\pm 1}\mid i\in I, j\in X\right\rangle \end{align*} $$

is a Hopf subalgebra of ${\mathcal P}_X$. The parabolic subalgebra and the Levi factor have respective triangular decompositions

(2.11)$$ \begin{align} {\mathcal P}_X\cong {\mathcal M}_X^+ \otimes U^0 \otimes U^- \quad \text{and} \quad {\mathcal L}_X \cong {\mathcal M}_X^+\otimes U^0 \otimes {\mathcal M}_X^-. \end{align} $$

There is a surjective Hopf algebra homomorphism $\pi _X:{\mathcal P}_X\rightarrow {\mathcal L}_X$ defined by

$$ \begin{align*} \pi_X\rvert_{{\mathcal L}_X}={\mathrm{id}}_{{\mathcal L}_X} \quad \text{and} \quad \pi_X(F_i)= 0 \qquad \text{for all}\ i\in I\setminus X. \end{align*} $$

The structure of Hopf algebras with a projection onto a Hopf subalgebra was investigated in detail by D. Radford in [Reference RadfordRad85]. In the following we recall some of his results in our setting. All the material in this section is known to experts, but we include some proofs for the convenience of the reader.

Consider the left coaction of ${\mathcal L}_X$ on ${\mathcal P}_X$ given by

$$ \begin{align*} {\varDelta}_{{\mathcal L}_X}=(\pi_X \otimes {\mathrm{id}})\circ {\varDelta}: {\mathcal P}_X \rightarrow {\mathcal L}_X\otimes {\mathcal P}_X \end{align*} $$

and define a subalgebra ${\mathcal R}_X\subset {\mathcal P}_X$ by

$$ \begin{align*} {\mathcal R}_X={}^{{\mathcal L}_X}{{\mathcal P}_X}=\{a\in {\mathcal P}_X\mid {\varDelta}_{{\mathcal L}_X}(a)=1 \otimes a\}. \end{align*} $$

The algebra ${\mathcal R}_X$ is a right ${\mathcal L}_X$-module algebra under the right adjoint action given by ${\mbox{ad}}_r(h)(a)= S\left (h_{(1)}\right )a h_{(2)}$ for all $a\in {\mathcal R}_X, h\in {\mathcal L}_X$. Hence we can form the smash product ${\mathcal L}_X\otimes {\mathcal R}_X$, which is an associative algebra with the multiplication

$$ \begin{align*} (h\otimes a)(h'\otimes a') = h h^{\prime}_{(1)} \otimes {\mbox{ad}}_r\left(h^{\prime}_{(2)}\right)(a)a'. \end{align*} $$

The following theorem is obtained by translating [Reference RadfordRad85, Theorem 3(d)] from left to right:

Formula (2.2) for the coproduct of $F_i$ implies that $F_i\in {\mathcal R}_X$ for all $i\in I\setminus X$. Moreover, as ${\mathcal R}_X$ is invariant under the right adjoint action of ${\mathcal L}_X$, we obtain ${\mbox{ad}}_r({\mathcal L}_X)(F_i) \subseteq {\mathcal R}_X$ for all $i\in I\setminus X$.

For all $j\in X$, $i\in I\setminus X$ one has ${\mbox{ad}}_r\left (E_j\right )(F_i)=-K_j^{-1}\left [E_j,F_i\right ]=0$. Hence by the triangular decomposition (2.11) for ${\mathcal L}_X$ we get ${\mbox{ad}}_r({\mathcal L}_X)(F_i)={\mbox{ad}}_r\left ({\mathcal M}^-_X\right )(F_i)\subset U^-$. By Theorem 2.1, Corollary 2.3 and the triangular decompositions (2.11), we get that the multiplication map

(2.12)$$ \begin{align} {\mathcal M}_X^- \otimes {\mathcal R}_X \rightarrow U^- \end{align} $$

is an isomorphism of algebras.

We can also describe the subalgebra ${\mathcal R}_X\subset U^-$ in terms of the Lusztig automorphisms:

Lemma 2.4. We have

$$ \begin{align*} {\mathcal R}_X=\left\{a\in U^-\mid T_j^{-1}(a)\in U^- \text{ for all}\ j\in X\right\}=\bigcap_{j\in X} \left(U^-\cap T_j\left(U^-\right)\right). \end{align*} $$

Proof. Let $j\in X$. We first show that $T_j^{-1}({\mathcal R}_X)\subset U^-$. By Corollary 2.3 it suffices to show that $T_j^{-1}\left ({\mbox{ad}}_r\left ({\mathcal M}_X^-\right )(F_i)\right )\subset U^-$ for any $i\in I\setminus X$. We have $T_j^{-1}(F_i)\in U^-$. Now we proceed by induction. Assume that $u\in {\mathcal R}_X$ satisfies $T_j^{-1}(u)\in U^-$ and let $\ell \in X$. Then we get

$$ \begin{align*} T_j^{-1}({\mbox{ad}}_r(F_\ell)(u))&=T_j^{-1}\left(-F_\ell K_\ell u K_{\ell}^{-1} + u F_\ell\right), \end{align*} $$

which by the induction hypothesis lies in $U^-$ if $\ell \neq j$. In the case $\ell =j$, this formula becomes

$$ \begin{align*} T_j^{-1}({\mbox{ad}}_r(F_\ell)(u))=\left[T_\ell^{-1}(u), E_\ell\right] K_\ell \stackrel{\text{(2.9)}}{=} \frac{1}{q_\ell-q_\ell^{-1}}\partial_\ell^R\left(T_\ell^{-1}(u)\right)\in U^-. \end{align*} $$

This implies that $T_j^{-1}\left ({\mbox{ad}}_r\left ({\mathcal M}_X^-\right )(F_i)\right )\subseteq U^-$, as required.

Conversely, assume that $u\in U^-$ satisfies $T_j^{-1}(u)\in U^-$ for all $j\in X$. We want to show that $u\in {\mathcal R}_X$. For any $j\in X$ the decomposition (2.12) implies that the multiplication map

(2.13)$$ \begin{align} {\mathbb K}\left[F_j\right]\otimes \left({\mathcal M}_X^-\cap T_j\left({\mathcal M}_X^-\right)\right)\otimes {\mathcal R}_X\rightarrow U^- \end{align} $$

is a linear isomorphism. Write $u=\sum _m h_m\otimes v_m$ with linearly independent $h_m\in {\mathcal M}_X^-\cong {\mathbb K}\left [F_j\right ]\otimes \left ({\mathcal M}_X^-\cap T_j\left ({\mathcal M}_X^-\right )\right )$ and $v_m\in {\mathcal R}_X$. Then the relations $T_j^{-1}(u)\in U^-$ and $T_j^{-1}(v_m)\in U^-$ imply that $h_m\in \left ({\mathcal M}_X^-\cap T_j\left ({\mathcal M}_X^-\right )\right )$. By [Reference LusztigLus94, Lemma 1.2.15] we have

$$ \begin{align*} \bigcap_{j\in X} \left({\mathcal M}_X^-\cap T_j\left({\mathcal M}_X^-\right)\right)\stackrel{\text{(2.10)}}{=} \left\{y\in {\mathcal M}_X^-\mid\partial^R_j(y)=0 \text{ for all}\ j\in X\right\} = {\mathbb K}. \end{align*} $$

Hence all $h_m$ are scalars, which implies that $u\in {\mathcal R}_X$.

Remark 2.5. In formula (2.13) we used the decomposition ${\mathcal M}_X^-\cong {\mathbb K}\left [F_j\right ]\otimes \left ({\mathcal M}_X^-\cap T_j\left ({\mathcal M}_X^-\right )\right )$, which holds by the Poincaré–Birkhoff–Witt theorem for the finite type quantum enveloping algebra ${\mathcal M}_X$. By Lemma 2.4 and the decomposition (2.12) in the case where $\lvert X\rvert =1$, we now have the same decomposition

(2.14)$$ \begin{align} U^-\cong {\mathbb K}[F_i] \otimes \left(U^-\cap T_i\left(U^-\right)\right) \end{align} $$

for any $i\in I$ also in infinite type.

To rewrite the statement of Lemma 2.4 we note the following fact:

Lemma 2.6. Let $w\in W$ be an element with reduced expression $w=s_{i_1}s_{i_2} \dotsm s_{i_m}$. Then the following relations hold:

(2.15)$$ \begin{align} U^- \cap T_w\left(U^-\right)&= U^- \cap T_{i_1}\left(U^-\right)\cap T_{i_1} T_{i_2}\left(U^-\right) \cap \dotsb \cap T_w\left(U^-\right), \end{align} $$

(2.16)$$ \begin{align} U^- \cap T_w^{-1}\left(U^-\right)&= U^- \cap T^{-1}_{i_m}\left(U^-\right)\cap T^{-1}_{i_m} T^{-1}_{i_{m-1}}\left(U^-\right) \cap \dotsb \cap T^{-1}_w\left(U^-\right). \end{align} $$

Proof. It suffices to prove equation (2.16), as equation (2.15) then follows by application of $T_w$. We prove equation (2.16) by induction on the length $l(w)=m$ of w. For $m=1$ there is nothing to show. Now assume that $w=s_i w'$, where $l(w)=l(w')+1$. Then $T_w=T_i T_w'$, and formula (2.14) gives us

$$ \begin{align*} T^{-1}_w\left(U^-\right)&=T^{-1}_{w'} T^{-1}_i \left({\mathbb K}[F_i] \otimes \left(U^-\cap T_i\left(U^-\right)\right)\right)\\ &= T_{w'}^{-1}\left({\mathbb K}[E_iK_i] \otimes \left(T_i^{-1}\left(U^-\right)\cap U^-\right) \right)\\ &= {\mathbb K}\left[T_{w'}^{-1}(E_iK_i)\right] \otimes \left(T_w^{-1}\left(U^-\right) \cap T_{w'}^{-1}\left(U^-\right)\right). \end{align*} $$

By [Reference LusztigLus94, Proposition 40.2.1] we have $T_{w'}^{-1}(E_i)\in U^+$, and hence the triangular decomposition $U^+\otimes U^0 \otimes U^-\cong U$ implies that

$$ \begin{align*} U^- \cap T^{-1}_w\left(U^-\right) = U^- \cap T_w^{-1}\left(U^-\right) \cap T_{w'}^{-1}\left(U^-\right). \end{align*} $$

Now equation (2.16) follows by the induction hypothesis.

Let $w_X$ denote the longest element in the finite parabolic subgroup $W_X$ of the Weyl group W. There is a diagram automorphism $\tau _X:X\rightarrow X$ such that $w_X\left (\alpha _j\right )=-\alpha _{\tau _X\left (j\right )}$ for all $j\in X$.

Proof. For any $j\in X$, we can write $w_X=s_j w'$ for some $w'\in W_X$ with $l(w')=l(w_X)-1$. Hence equation (2.15) and Lemma 2.4 imply that

$$ \begin{align*} U^-\cap T_{w_X}\left(U^-\right) \subseteq \bigcap_{j\in X} \left(U^- \cap T_j\left(U^-\right)\right)={\mathcal R}_X. \end{align*} $$

The converse inclusion is verified similarly to the first part of the proof of Lemma 2.4. Again it suffices to show that ${\mbox{ad}}_r\left ({\mathcal M}_X^-\right )(F_i) \subseteq U^-\cap T_{w_X}\left (U^-\right )$ for all $i\in I\setminus X$. By [Reference LusztigLus94, Proposition 40.2.1] we have $T_{w_X}^{-1}(F_i)\in U^-$, and hence $F_i\in U^-\cap T_{w_X}\left (U^-\right )$. If $u\in U^-\cap T_{w_X}\left (U^-\right )$ and $\ell \in X$, then

$$ \begin{align*} T_{w_X}^{-1}({\mbox{ad}}_r(F_\ell)(u))=\left[T_{w_X}^{-1}(u), E_{\tau_X(\ell)}\right]K_{\tau_X(\ell)} \stackrel{\text{(2.9)}}{=} \frac{1}{q_\ell-q_\ell^{-1}} \partial_{\tau_X(\ell)}^R\left(T_{w_X}^{-1}(u)\right)\in U^-. \end{align*} $$

This implies inductively that $T_{w_X}^{-1}\left ({\mbox{ad}}_r\left ({\mathcal M}_X^-\right )(F_i)\right )\subset U^-$ and completes the proof of the corollary.

2.3 Partial parabolic subalgebras

Consider a subset of finite type $X\subseteq I$ and a map $\tau :I\rightarrow I$ which is an involutive diagram automorphism such that $\tau (X)=X$ and $w_X\left (\alpha _j\right )=-\alpha _{\tau \left (j\right )}$ for all $j\in X$. Following [Reference Regelskis and VlaarRV20], we call a pair $(X,\tau )$ with these properties a compatible decoration. The map $\Theta =-w_X\circ \tau :Q\rightarrow Q$ is an involutive automorphism of the root lattice. Define $Q^\Theta =\{\beta \in Q\mid \Theta (\beta )=\beta \}$ and $U^0_\Theta ={\mathbb K}\left \langle K_\beta \mid \beta \in Q^\Theta \right \rangle $. As $Q^\Theta $ is generated by the elements of the set $\left \{\alpha _i-\alpha _{\tau \left (i\right )},\alpha _j\mid i\in I\setminus X, j\in X\right \}$, we see that

$$ \begin{align*} U^0_\Theta={\mathbb K} \left\langle K_j^{\pm 1}, K_i K_{\tau\left(i\right)}^{-1}\mid j\in X, i\in I \right\rangle. \end{align*} $$

Consider the subalgebra ${\mathcal A}={\mathcal A}(X,\tau )\subset U$ defined by

$$ \begin{align*} {\mathcal A}={\mathbb K}\left\langle F_i, E_j, K_\beta\mid i\in I, j\in X, \Theta(\beta)=\beta\right\rangle. \end{align*} $$

We call ${\mathcal A}$ the partial parabolic subalgebra corresponding to the pair $(X,\tau )$. By definition, ${\mathcal A}$ is contained in the standard parabolic subalgebra ${\mathcal P}_X$, with the only difference being that ${\mathcal A}\cap U^0=U^0_\Theta $ is not the whole torus. In particular, ${\mathcal A}$ has the triangular decomposition

(2.17)$$ \begin{align} {\mathcal A}\cong {\mathcal M}_X^+ \otimes U^0_\Theta \otimes U^-. \end{align} $$

Additionally, consider the subalgebra ${\mathcal H}={\mathcal H}(X,\tau )\subset U$ defined by

$$ \begin{align*} {\mathcal H}={\mathbb K} \left\langle E_j, F_j, K_\beta\mid j\in X, \Theta(\beta)=\beta\right\rangle={\mathcal M}_X U^0_\Theta. \end{align*} $$

We call ${\mathcal H}$ the partial Levi factor corresponding to the generalised Satake diagram $(X,\tau )$. The partial Levi factor ${\mathcal H}$ has a triangular decomposition

(2.18)$$ \begin{align} {\mathcal H} \cong {\mathcal M}^+_X \otimes U_\Theta^0 \otimes {\mathcal M}^-_X. \end{align} $$

The partial parabolic subalgebra ${\mathcal A}$ is not a subbialgebra of U. However, the partial Levi factor ${\mathcal H}$ is a Hopf subalgebra of U. Comparison of the triangular decompositions (2.17) and (2.18) with the decomposition (2.12) implies that the multiplication map

$$ \begin{align*} {\mathcal H} \otimes {\mathcal R}_X \rightarrow {\mathcal A} \end{align*} $$

is an isomorphism.

2.4 Quantum symmetric pairs

Let $(X,\tau )$ be a generalised Satake diagram. Following [Reference Regelskis and VlaarRV20], this means that $(X,\tau )$ is a compatible decoration, as described at the beginning of Section 2.3, and additionally that for $i\in I\setminus X$ and $j\in X$ the implication

(2.19)$$ \begin{align} \tau(i)=i \text{ and } a_{ji}=-1 \Longrightarrow \Theta(\alpha_i)\neq -\alpha_i-\alpha_j \end{align} $$

holds. The condition (2.19) is weaker than condition (3) in [Reference KolbKol14, Definition 2.3]. As explained in [Reference Regelskis and VlaarRV20, Section 4.1], the results of [Reference KolbKol14] remain valid in this more general setting. Following [Reference KolbKol14] we recall the definition of the quantum symmetric pair coideal subalgebra corresponding to the generalised Satake diagram $(X,\tau )$. Recall the set of parameters ${\mathcal C}$ defined by equation (1.2). For any ${\mathbf {c}}=(c_i)_{i\in I\setminus X}\in {\mathcal C}$, let ${\mathcal B}_{\mathbf {c}}$ denote the subalgebra of U generated by the partial Levi factor ${\mathcal H}$ and the elements

(2.20)$$ \begin{align} B_i=F_i - c_i T_{w_X}\left(E_{\tau(i)}\right) K_i^{-1} \quad \text{for all }{i\in I\setminus X}. \end{align} $$

Recall that the algebra ${\mathcal B}_{\mathbf {c}}$ has a natural ${\mathbb N}$-filtration ${\mathcal F}_\ast $ defined via the degree function (1.3). Hence ${\mathcal F}_n({\mathcal B}_{\mathbf {c}})$ is the ${\mathbb K}$-linear span of all monomials in the generators of ${\mathcal B}_{\mathbf {c}}$, where each monomial contains at most n factors $B_i$, with $i\in I\setminus X$. Let ${\mathrm {gr}}({\mathcal B}_{\mathbf {c}})$ be the associated graded algebra. It follows from [Reference KolbKol14, Theorem 7.1] that there exists a surjective algebra homomorphism

(2.21)$$ \begin{align} \varphi:{\mathcal A} \rightarrow {\mathrm{gr}}({\mathcal B}_{\mathbf{c}}) \end{align} $$

given by $\varphi (F_i)=B_i$ for all $i\in I\setminus X$ and $\varphi (h)=h$ for all $h\in {\mathcal H}$. Moreover, the triangular decomposition (2.17) for ${\mathcal A}$ and [Reference KolbKol14, Proposition 6.2] imply that $\varphi $ is an isomorphism.

The partial parabolic subalgebra ${\mathcal A}$ is ${\mathbb N}$-graded via the degree function given by

$$ \begin{align*} \deg(F_i)&=1 \qquad \text{for all}\ i\in I\setminus X,\\ \deg(h)&=0 \qquad \text{for all}\ h\in {\mathcal H}. \end{align*} $$

With this grading, the map $\varphi $ in formula (2.21) is an isomorphism of graded algebras. For any $n\in {\mathbb N}$ we let ${\mathcal A}_n$ denote the n-th graded component of ${\mathcal A}$ and we write ${\mathcal A}_{<n}=\bigoplus _{m=0}^{n-1}{\mathcal A}_m$.

2.5 The Letzter map

Let $U^{\mathrm {poly}}$ denote the subalgebra of U generated by ${\mathcal A}$ and the elements $\widetilde {E}_i=E_iK_i^{-1}$, $K_i^{-1}$ for all $i\in I\setminus X$. Observe that ${\mathcal B}_{\mathbf {c}}\subset U^{\mathrm {poly}}$, as $T_{w_X}\left (E_{\tau (i)}\right )K_i^{-1}\in {\mathcal A} \widetilde {E}_{\tau (i)}{\mathcal A}$. Let $I_\tau $ be a set of representatives of $\tau $-orbits in $I\setminus X$. In analogy to the definition of $V_X$ in the proof of Corollary 2.3, let $V_X^+\subset {\mathcal P}_X$ be the subalgebra generated by the subspaces ${\mbox{ad}}_r({\mathcal L}_X)\left (\widetilde {E}_i\right )={\mbox{ad}}_r\left ({\mathcal M}_X^+\right )\left (\widetilde {E}_i\right )$ for $i\in I\setminus X$. The triangular decomposition

$$ \begin{align*} U\cong U^- \otimes U^0 \otimes G^+ \cong {\mathcal A} \otimes {\mathbb K}\left[K_i^{\pm 1}\mid i\in I_\tau\right]\otimes V_X^+ \end{align*} $$

implies that there is a direct sum decomposition of vector spaces

(2.22)$$ \begin{align} U^{\mathrm{poly}} \cong {\mathcal A} \oplus U^{\mathrm{poly}} \mathrm{span}_{\mathbb K}\left\{\widetilde{E}_i, K_i^{-1}\mid i\in I\setminus X\right\} {\mathcal H}. \end{align} $$

Let

(2.23)$$ \begin{align} \psi:U^{\mathrm{poly}} \rightarrow {\mathcal A} \end{align} $$

denote the ${\mathbb K}$-linear projection with respect to the direct sum decomposition (2.22).

As ${\mathcal B}_{\mathbf {c}}$ is a subalgebra of $U^{\mathrm {poly}}$, we can restrict the map $\psi $ to ${\mathcal B}_{\mathbf {c}}$. We call the map $\psi :{\mathcal B}_{\mathbf {c}}\rightarrow {\mathcal A}$ the Letzter map. The algebra ${\mathcal B}_{\mathbf {c}}$ is filtered by the filtration ${\mathcal F}_\ast $ given in Section 2.4. The algebra ${\mathcal A}$ is graded and hence also filtered. By construction, the Letzter map $\psi :{\mathcal B}_{\mathbf {c}}\rightarrow {\mathcal A}$ is a linear map of filtered vector spaces. Let

$$ \begin{align*} {\mathrm{gr}}(\psi):{\mathrm{gr}}({\mathcal B}_{\mathbf{c}})\rightarrow {\mathrm{gr}}({\mathcal A})\cong {\mathcal A} \end{align*} $$

be the associated graded map, and recall the isomorphism $\varphi $ from formula (2.21). The composition ${\mathrm {gr}}(\psi )\circ \varphi :{\mathcal A} \rightarrow {\mathcal A}$ is ${\mathcal H}$-linear and satisfies

$$ \begin{align*} {\mathrm{gr}}(\psi)\circ \varphi\left(F_{i_1}\dotsm F_{i_\ell}\right)= F_{i_1}\dotsm F_{i_\ell} \end{align*} $$

for all $i_1,\dotsc , i_\ell \in I$. Hence ${\mathrm {gr}}(\psi )\circ \varphi ={\mathrm {id}}_{\mathcal A}$. This implies that ${\mathrm {gr}}(\psi )=\varphi ^{-1}$ is a linear isomorphism. This proves the following lemma which is also contained in [Reference LetzterLet19, Corollary 4.4] for ${\mathfrak g}$ of finite type:

We collect some additional properties of the Letzter map:

2.6 An antilinear isomorphism of quantum symmetric pair coideals

Let $\overline {\phantom {m}}:U\rightarrow U$, $u\mapsto \overline {u}$, be the Lusztig bar involution defined in [Reference LusztigLus94, 3.1.12]. Denote by $\rho _X$ and $\rho _X^\vee $ the half sum of positive roots and of coroots, respectively, for the root system corresponding to $X \subseteq I$. For any ${\mathbf {c}}=(c_i)_{i\in I\setminus X}\in {\mathbb K}^{I\setminus X}$, define ${\mathbf {c}}'=\left (c_i'\right )\in {\mathbb K}^{I\setminus X}$ by

(2.24)$$ \begin{align} c_i'=(-1)^{2\alpha_i\left(\rho_X^\vee\right)}q^{\left(\alpha_i,\Theta\left(\alpha_i\right)-2\rho_X\right)} \overline{c_{\tau(i)}} \end{align} $$

for all $i\in I\setminus X$. Note that ${\mathbf {c}}\in {\mathcal C}$ if and only if ${\mathbf {c}}'\in {\mathcal C}$. In the following we will compare the quantum symmetric pair coideal subalgebras ${\mathcal B}_{\mathbf {c}}$ and ${\mathcal B}_{{\mathbf {c}}'}$. To indicate the parameters, we write $B_i'$ for the generators (2.20) of ${\mathcal B}_{{\mathbf {c}}'}$ for $i\in I\setminus X$.

In [Reference Appel and VlaarAV20, Theorem 7.4], Appel and Vlaar considered an extended version of the quasi K-matrix constructed in [Reference Balagović and KolbBK19]. Using this extended version of the quasi K-matrix, one obtains the following theorem, a proof of which in our notation can be found in [Reference KolbKol21, Corollary 4.1]:

Theorem 2.12. For any ${\mathbf {c}}\in {\mathcal C}$ there exists a k-algebra isomorphism $\Phi :{\mathcal B}_{\mathbf {c}}\rightarrow {\mathcal B}_{{\mathbf {c}}'}$ such that

$$ \begin{align*} \Phi\rvert_{\mathcal H}=\overline{\phantom{m}}\rvert_{\mathcal H} \quad \text{and} \quad \Phi(B_i)=B_i' \qquad \text{for all}\ i\in I\setminus X. \end{align*} $$

In particular, we have $\Phi (q)=q^{-1}$ and $\Phi (K_\beta )=K_{-\beta }$ for all $\beta \in Q^\Theta $.

3.1 Star products

We will extensively use the notion of star product algebras and their properties from [Reference Kolb and YakimovKY20]. For an ${\mathbb N}$-graded ${\mathbb K}$-algebra $A=\bigoplus _{j\in {\mathbb N}} A_j$ and $m\in {\mathbb N}$, denote $A_{<m}=\bigoplus _{j=0}^{m-1}A_j$ and $A_{\le m}=\bigoplus _{j=0}^{m}A_j$.

Definition 3.1. Assume that $A=\bigoplus _{j\in {\mathbb N}} A_j$ is an ${\mathbb N}$-graded ${\mathbb K}$-algebra. A star product on A is an associative bilinear operation $* : A \times A \to A$ such that

$$\begin{align*}a * b - ab \in A_{< m+ n} \qquad \text{for all}\ a \in A_m, b \in A_n. \end{align*}$$

A star product $\ast $ on A is called $0$-equivariant if

$$\begin{align*}a * h = ah \quad \text{and} \quad h * a= h a \qquad \text{for all}\ h \in A_0, a \in A. \end{align*}$$

In this setting, $(A,\ast )$ is a filtered algebra with ${\mathcal F}_m (A) := A_{\leq m}$ and

$$\begin{align*}{\mathrm{gr}} (A, *) \cong A. \end{align*}$$

If A is generated in degrees $0$ and $1$, then $(A,*)$ is generated by ${\mathcal F}_1 (A)$ and we have the following properties [Reference Kolb and YakimovKY20, Lemma 5.2.(ii)]:

3.2 The pullback of the algebra structure on ${\mathcal B}_{\mathbf {c}}$

We can use the Letzter map $\psi :{\mathcal B}_{\mathbf {c}}\rightarrow {\mathcal A}$ to define a new algebra structure $\ast $ on ${\mathcal A}$ by

$$ \begin{align*} a\ast b = \psi(\psi^{-1}(a)\psi^{-1}(b)) \qquad \text{for all}\ a,b\in {\mathcal A}. \end{align*} $$

By Lemma 2.11(4) the binary operation $\ast $ defines a star product on ${\mathcal A}$, and by Lemma 2.11(2) this star product is $0$-equivariant. Hence Lemma 3.2 implies that the star product $\ast $ on ${\mathcal A}$ is uniquely determined by the family of maps $\mu ^L_f\in {\mathrm {Hom}}_{{\mathbb K}}({\mathcal R}_X,{\mathcal A})$ for all $f\in ({\mathcal R}_X)_1={\mathcal R}_X\cap {\mathcal A}_1$ defined by

$$ \begin{align*} \mu_f^L(u) = f\ast u - fu \qquad \text{for all}\ u\in {\mathcal R}_X. \end{align*} $$

As $\mu ^L_{{\mbox{ad}}_r(h)(f)}(u)=S\left (h_{(1)}\right )\mu _f^L\left (h_{(2)}u\right )$ for all $f\in ({\mathcal R}_X)_1$, $h\in {\mathcal H}$ and $u\in {\mathcal R}_X$, the $0$-equivariant star product $\ast $ on ${\mathcal A}$ is uniquely determined by the maps $\mu _{F_i}^L$ for $i\in I\setminus X$. In the following lemma, we determine these maps:

Lemma 3.3. For all $i\in I\setminus X$ and all $u\in {\mathcal R}_X$, we have

(3.1)$$ \begin{align} F_i\ast u = F_i u - c_i \frac{q^{-\left(\alpha_i,\Theta\left(\alpha_i\right)\right)}}{q_i-q_i^{-1}}K_{-\alpha_i-\Theta\left(\alpha_i\right)} T_{w_X} \circ \partial_{\tau(i)}^L\circ T_{w_X}^{-1}(u). \end{align} $$

Proof. Write $u=\psi (b)$ for some $b\in {\mathcal B}_{\mathbf {c}}$. Using Lemma 2.11(1) and (3), we calculate

$$ \begin{align*} F_i\ast u &= \psi(B_i b)\\ &= \psi\left(\left(F_i - c_i T_{w_X}\left(E_{\tau(i)}\right)K_i^{-1}\right)\psi(b) \right)\\ &= F_i u - c_i q^{-\left(\alpha_i,\Theta\left(\alpha_i\right)\right)} \psi\left(K_i^{-1}\left[T_{w_X}\left(E_{\tau(i)}\right),u\right] \right)\\ &= F_i u - c_i q^{-\left(\alpha_i,\Theta\left(\alpha_i\right)\right)} T_{w_X}\left(\psi\left(K_{w_X\left(\alpha_i\right)}^{-1}\left[E_{\tau(i)}, T_{w_X}^{-1}(u)\right] \right)\right). \end{align*} $$

Using equation (2.3) and the definition of the projection $\psi $, we obtain

$$ \begin{align*} F_i\ast u &= F_i u - c_i \frac{q^{-\left(\alpha_i,\Theta\left(\alpha_i\right)\right)}}{q_i-q_i^{-1}} T_{w_X} \left(K_{w_X\left(\alpha_i\right)}^{-1} K_{\tau(i)} \partial^L_{\tau(i)}\left(T_{w_X}^{-1}(u)\right) \right)\\ &= F_i u - c_i \frac{q^{-\left(\alpha_i,\Theta\left(\alpha_i\right)\right)}}{q_i-q_i^{-1}} K_{-\alpha_i - \Theta\left(\alpha_i\right)} T_{w_X} \left( \partial^L_{\tau(i)}\left( T_{w_X}^{-1}(u)\right)\right), \end{align*} $$

which gives the desired formula.

3.3 Twisted skew derivations

Recall Corollary 2.7 and the subalgebra $G_X^+$ defined in Section 2.1. The decomposition (2.12) implies that $T_{w_X}:U^-\rightarrow G_X^+\otimes {\mathcal R}_X$ is an algebra isomorphism. To shorten notation, define twisted skew derivations $\partial ^L_{i,X}, \partial ^R_{i,X}:G_X^+\otimes {\mathcal R}_X \rightarrow G_X^+\otimes {\mathcal R}_X$ by

$$ \begin{align*} \partial^{L}_{i,X}=T_{w_X} \circ \partial^{L}_i \circ T_{w_X}^{-1}, \qquad \partial^{R}_{i,X}=T_{w_X} \circ \partial^{R}_i \circ T_{w_X}^{-1}, \end{align*} $$

for all $i\in I$. The skew derivation properties (2.5) and (2.6) of $\partial ^L_i$ and $\partial ^R_i$ imply

(3.2)$$ \begin{align} \partial_{i,X}^L(fg)&=\partial_{i,X}^L(f)g +q^{\left(w_X\left(\alpha_i\right),\mu\right)}f \partial_{i,X}^L(g) \end{align} $$

(3.3)$$ \begin{align} \partial_{i,X}^R(fg)&=q^{\left(w_X\left(\alpha_i\right),\nu\right)}\partial_{i,X}^R(f)g +f \partial_{i,X}^R(g) \end{align} $$

for all $f\in \left (G_X^+\otimes {\mathcal R}_X\right )_{-\mu }$, $g\in \left (G_X^+\otimes {\mathcal R}_X\right )_{-\nu }$. To understand the star product better, we need to know the action of the twisted skew derivation $\partial ^L_{i,X}$ on the elements of ${\mathcal R}_X$.

Lemma 3.4. For any $i\in I \setminus X$ we have

(3.4)$$ \begin{align} \partial^L_{i,X}(F_i) = q^{\left(\alpha_i,\alpha_i-w_X\left(\alpha_i\right)\right)} K_{\alpha_i-w_X\left(\alpha_i\right)} \partial_i^R\left(T_{w_X}(E_i)\right). \end{align} $$

Proof. We apply $T_{w_X}^{-1}$ to equation (2.4) for $x=T_{w_X}(E_i)$ and compare with equation (2.3) to obtain

$$ \begin{align*} &\frac{T_{w_X}^{-1}\left(\partial_i^R\left(T_{w_X}(E_i)\right)K_i\right) - T_{w_X}^{-1}\left(K_i^{-1}\partial^L_i\left(T_{w_X}(E_i)\right)\right)}{q_i-q_i^{-1}}\\ &\quad=\left[E_i, T_{w_X}^{-1}(F_i)\right] =\frac{K_i\partial^L_i\left(T_{w_X}^{-1}(F_i)\right)-\partial_i^R\left(T_{w_X}^{-1}(F_i)\right)K_i^{-1}}{q_i-q_i^{-1}}. \end{align*} $$

Comparing the summands involving $K_i$, we obtain

$$ \begin{align*} T_{w_X}^{-1}\left(\partial_i^R\left(T_{w_X}(E_i)\right)K_i\right)=K_i\partial^L_i\left(T_{w_X}^{-1}(F_i)\right) \end{align*} $$

and hence

$$ \begin{align*} T_{w_X}\left(\partial_i^L\left(T_{w_X}^{-1}(F_i)\right) \right)&= K_{w_X\left(\alpha_i\right)}^{-1}\partial_i^R\left(T_{w_X}(E_i)\right)K_i\\ &=q^{\left(\alpha_i,\alpha_i-w_X\left(\alpha_i\right)\right)} K_{\alpha_i-w_X\left(\alpha_i\right)}\partial_i^R\left(T_{w_X}(E_i)\right), \end{align*} $$

which confirms the statement of the lemma.

For simplicity, define

$$\begin{align*}Z_i=\partial_i^R\left(T_{w_X}(E_i)\right)\in U^+_{w_X\left(\alpha_i\right)-\alpha_i} \qquad \text{for any}\ i\in I\setminus X. \end{align*}$$

As $w_X(\alpha _i)-\alpha _i\in Q_X^+$, we get $Z_i\in {\mathcal M}_X^+$ and hence we have

(3.5)$$ \begin{align} \left[Z_i,F_j\right]=0 \qquad \text{for all}\ i,j\in I \setminus X. \end{align} $$

Recall the nonsymmetric quantum integer $(n)_p$ defined by $(n)_p=\sum _{j=0}^{n-1}p^j$ for any $n\in {\mathbb N}$, $p\in {\mathbb K}$.

Lemma 3.5. For any $i\in I\setminus X$ and $n\in {\mathbb N}$, we have

(3.6)$$ \begin{align} \partial_{i,X}^L\left(F_i^n\right)=q^{\left(\alpha_i,\alpha_i-w_X\left(\alpha_i\right)\right)}(n)_{q_i^2}K_{\alpha_i-w_X\left(\alpha_i\right)} Z_i F_i^{n-1}. \end{align} $$

Proof. For $n=1$, equation (3.6) holds by Lemma 3.4. We proceed by induction on n. For $n>1$ we calculate

$$ \begin{align*} \partial_{i,X}^L\left(F_i^n\right) &\stackrel{\text{(3.2)}}{=}\partial_{i,X}^L(F_i)F_i^{n-1} + q^{\left(w_X\left(\alpha_i\right),\alpha_i\right)} F_i\partial_{i,X}^L\left(F_i^{n-1}\right)\\ &\stackrel{\text{(3.4)}}{=}q^{\left(\alpha_i,\alpha_i-w_X\left(\alpha_i\right)\right)} K_{\alpha_i-w_X\left(\alpha_i\right)}Z_i F_i^{n-1} + F_i q^{\left(\alpha_i,\alpha_i\right)} (n-1)_{q_i^2} K_{\alpha_i-w_X\left(\alpha_i\right)} Z_i F_i^{n-2}\\ &\stackrel{\phantom{\text{(3.2)}}}{=} q^{\left(\alpha_i,\alpha_i-w_X\left(\alpha_i\right)\right)} (n)_{q_i^2} K_{\alpha_i-w_X\left(\alpha_i\right)} Z_i F_i^{n-1}, \end{align*} $$

where we also used the fact that $Z_i$ and $F_i$ commute by equation (3.5).

We will need similar relations for ${\mbox{ad}}_r\left (F_j\right )\left (F_i^n\right )$, which lies in ${\mathcal R}_X$ for $j\in X$.

Lemma 3.6. For any $u\in {\mathcal R}_X$, $j\in X$ and $i\in I\setminus X$, we have

(3.7)$$ \begin{align} \partial_{i,X}^L\left({\mbox{ad}}_r\left(F_j\right)(u)\right) = - \frac{1}{q_j-q_j^{-1}} \partial_{\tau(j),X}^R \circ \partial_{i,X}^L(u). \end{align} $$

Proof. For $u\in ({\mathcal R}_X)_{-\mu }$ and $j\in X$ we calculate

(3.8)$$ \begin{align} T_{w_X}^{-1}\left({\mbox{ad}}_r\left(F_j\right)(u)\right)& \stackrel{\phantom{\text{(2.9)}}}{=}T_{w_X}^{-1}\left(u F_j - q^{-\left(\alpha_j,\mu\right)}F_j u\right)\nonumber\\ &\stackrel{\phantom{\text{(2.9)}}}{=} - T_{w_X}^{-1}(u) E_{\tau\left(j\right)} K_{\tau\left(j\right)} + q^{-\left(\alpha_j,\mu\right)} E_{\tau\left(j\right)} K_{\tau\left(j\right)} T_{w_X}^{-1}(u)\nonumber\\ &\stackrel{\phantom{\text{(2.9)}}}{=} \left[E_{\tau\left(j\right)}, T_{w_X}^{-1}(u)\right] K_{\tau\left(j\right)}\nonumber\\ &\stackrel{\text{(2.9)}}{=} - \frac{1}{q_j-q_j^{-1}} \partial^R_{\tau\left(j\right)}\circ T_{w_X}^{-1}(u). \end{align} $$

Hence we obtain

$$ \begin{align*} \partial_{i,X}^L\left({\mbox{ad}}_r\left(F_j\right)(u)\right) &= - \frac{1}{q_j-q_j^{-1}} T_{w_X}\circ \partial^L_i \circ \partial^R_{\tau\left(j\right)}\circ T_{w_X}^{-1}(u)\\ &= - \frac{1}{q_j-q_j^{-1}} T_{w_X}\circ \partial^R_{\tau\left(j\right)} \circ \partial^L_i \circ T_{w_X}^{-1}(u)\\ &= - \frac{1}{q_j-q_j^{-1}} \partial_{\tau\left(j\right),X}^R \circ \partial_{i,X}^L(u), \end{align*} $$

which is the desired formula.

By equation (3.7), we will need to understand the action of $\partial _{j,X}^R$ on $G_X^+\otimes {\mathcal R}_X$. Equation (3.8) implies that

(3.9)$$ \begin{align} \partial_{j,X}^R(u)=-\left(q_j-q_j^{-1}\right) {\mbox{ad}}_r\left(F_{\tau\left(j\right)}\right)(u) \qquad \text{for all}\ u\in {\mathcal R}_X, j\in X. \end{align} $$

Moreover, by Lemma 3.4 we have

(3.10)$$ \begin{align} T_{w_X}^{-1}\left(K_{\alpha_i-w_X\left(\alpha_i\right)}Z_i\right) = q^{\left(\alpha_i, w_X\left(\alpha_i\right)-\alpha_i\right)} \partial_i^L\circ T_{w_X}^{-1}(F_i), \end{align} $$

and hence

$$ \begin{align*} \partial_{\tau\left(j\right),X}^R\left(K_{\alpha_i-w_X\left(\alpha_i\right)}Z_i\right)= q^{\left(\alpha_i, w_X\left(\alpha_i\right)-\alpha_i\right)} \partial_{\tau\left(j\right),X}^R \circ \partial_{i,X}^L (F_i), \end{align*} $$

which implies that $\partial _{\tau \left (j\right ),X}^R\left (K_{\alpha _i-w_X\left (\alpha _i\right )}Z_i\right )\in \left (G_X^+\right )_{w_X\left (\alpha _i\right )-\alpha _i-\alpha _j}$.

Lemma 3.7. For any $i\in I\setminus X$, $j\in X$ and $1\le n\in {\mathbb N}$, we have

(3.11)$$ \begin{align} \partial^L_{i,X}\left({\mbox{ad}}_r\left(F_j\right)\left(F_i^n\right)\right)&= \frac{q^{\left(\alpha_i+\alpha_j,\alpha_i+\alpha_j-w_X\left(\alpha_i\right)\right)-n\left(\alpha_i,\alpha_j\right)}}{q_j-q_j^{-1}} (n)_{q_i^2} K_{\alpha_i+\alpha_j-w_X\left(\alpha_i\right)} \partial^R_j(Z_i) F_i^{n-1}\nonumber\\ & \quad + q^{\left(\alpha_i,\alpha_i-w_X\left(\alpha_i\right)\right)}(n)_{q_i^2} K_{\alpha_i-w_X\left(\alpha_i\right)}Z_i {\mbox{ad}}_r\left(F_j\right)\left(F_i^{n-1}\right). \end{align} $$

Proof. By equation (2.3) we have

$$ \begin{align*} \!\left[E_{\tau\left(j\right)},T_{w_X}^{-1}\left(K_{\alpha_i-w_X\left(\alpha_i\right)}Z_i\right)\right]=\frac{K_{\tau\left(j\right)}\partial^L_{\tau\left(j\right)}\left(T_{w_X}^{-1}\left(K_{\alpha_i-w_X\left(\alpha_i\right)}Z_i\right)\right)-\partial^R_{\tau\left(j\right)}\left(T_{w_X}^{-1}\left(K_{\alpha_i-w_X\left(\alpha_i\right)}Z_i\right)\right)K_{\tau\left(j\right)}^{-1}}{q_j-q_j^{-1}}. \end{align*} $$

Applying $T_{w_X}$ to this equation, we obtain

$$ \begin{align*} &\frac{K_j^{-1}\partial_{\tau\left(j\right),X}^L\left(K_{\alpha_i-w_X\left(\alpha_i\right)}Z_i\right)-\partial_{\tau\left(j\right),X}^R\left(K_{\alpha_i-w_X\left(\alpha_i\right)}Z_i\right)K_{j}}{q_j-q_j^{-1}}\\ &\quad =-\left[F_j K_j, K_{\alpha_i-w_X\left(\alpha_i\right)}Z_i\right]\\ &\quad = - K_{\alpha_i-w_X\left(\alpha_i\right)}\left[F_j,Z_i\right]K_j\\ &\quad = - K_{\alpha_i-w_X\left(\alpha_i\right)} \frac{K_j^{-1}\partial_j^L(Z_i) - \partial_j^R(Z_i)K_j}{q_j-q_j^{-1}} K_j. \end{align*} $$

Using the fact that $\partial _{\tau (i)}^R\left (K_{\alpha _i-w_X\left (\alpha _i\right )}Z_i\right ) \in \left (G_X^+\right )_{w_X\left (\alpha _i\right )-\alpha _i-\alpha _j}$ proved before Lemma 3.7, and collecting terms in $U^+K_{\alpha _i+2\alpha _j-w_X\left (\alpha _i\right )}$, we obtain

$$ \begin{align*} \partial_{\tau\left(j\right),X}^R\left(K_{\alpha_i-w_X\left(\alpha_i\right)} Z_i\right) =-q^{-\left(\alpha_j,w_X\left(\alpha_i\right)-\alpha_i-\alpha_j\right)} K_{\alpha_i-w_X\left(\alpha_i\right)+\alpha_j}\partial^R_j(Z_i). \end{align*} $$

We can use this formula and the skew derivation property (3.3) to calculate

$$ \begin{align*} \partial^L_{i,X}\left({\mbox{ad}}_r\left(F_j\right)\left(F_i^n\right)\right) &\stackrel{\text{(3.7)}}{=} \frac{-1}{q_j-q_j^{-1}} \partial_{\tau\left(j\right),X}^R \circ \partial_{i,X}^L\left(F_i^n\right)\\ &\stackrel{\text{(3.6)}}{=} \frac{-1}{q_j-q_j^{-1}} \partial_{\tau\left(j\right),X}^R \left(q^{\left(\alpha_i,\alpha_i-w_X\left(\alpha_i\right)\right)}(n)_{q_i^2}K_{\alpha_i-w_X\left(\alpha_i\right)} Z_i F_i^{n-1} \right)\\ &\stackrel{\text{\phantom{(3.7)}}}{=} \frac{q^{\left(\alpha_i+\alpha_j,\alpha_i+\alpha_j-w_X\left(\alpha_i\right)\right)-n\left(\alpha_i,\alpha_j\right)}}{q_j-q_j^{-1}} (n)_{q_i^2} K_{\alpha_i+\alpha_j-w_X\left(\alpha_i\right)} \partial_j^R(Z_i) F_i^{n-1}\\ & \qquad \qquad \quad + q^{\left(\alpha_i,\alpha_i-w_X\left(\alpha_i\right)\right)}(n)_{q_i^2} K_{\alpha_i-w_X\left(\alpha_i\right)}Z_i {\mbox{ad}}_r\left(F_j\right)\left(F_i^{n-1}\right), \end{align*} $$

where we also used equation (3.9) in the last step.

4.1 Bivariate continuous q-Hermite polynomials

Recall the univariate continuous q-Hermite polynomials $H_m(x;q)$ defined recursively for all $m\in {\mathbb Z}$ by $H_{-m}(x;q)=0$ for $m<0$, $H_{0}(x;q)=1$ and

(4.1)$$ \begin{align} 2x H_{m}(x;q) = H_{m+1}(x;q) + (1-q^m) H_{m-1}(x;q) \end{align} $$

(see [Reference Koekoek, Lesky and SwarttouwKLS10, 14.26]). The following family of bivariate continuous q-Hermite polynomials $H_{m,n}(x,y;q,r)$ was introduced in [Reference Casper, Kolb and YakimovCKY21]:

Definition 4.1. Let $r\in {\mathbb K}$. The bivariate continuous q-Hermite polynomials $H_{m,n}(x,y;q,r)$ are defined for all $m,n\in {\mathbb Z}$ by $H_{0,n}(x,y;q,r)=H_n(y;q)$ and the recursion

(4.2)$$ \begin{align} 2x H_{m,n}(x,y;q,r) &= H_{m+1,n}(x,y;q,r) + (1-q^m)H_{m-1,n}(x,y;q,r)\nonumber\\ &\quad + q^m(1-q^n)r H_{m,n-1}(x,y;q,r), \end{align} $$

where we set $H_{-m,n}(x,y;q,r) =0$ for $m>0, n \in {\mathbb Z}$ and $H_{m,-n}(x,y;q,r)=0\ m \in {\mathbb Z}, n>0$.

It was shown in [Reference Casper, Kolb and YakimovCKY21] that the bivariate continuous q-Hermite polynomials form a family of orthogonal polynomials with many desirable properties. In particular, they satisfy the symmetry condition

$$ \begin{align*} H_{m,n}(x,y;q,r) = H_{n,m}(y,x;q,r) \qquad \text{for all}\ m,n\in {\mathbb Z}. \end{align*} $$

Moreover, by [Reference Casper, Kolb and YakimovCKY21, (3.4)] the bivariate q-Hermite polynomials $H_{m,n}(x,y;q,r)$ can be expressed in terms of the univariate continuous q-Hermite polynomials by the formula

(4.3)$$ \begin{align} H_{m,n}(x,y;q,r)=\sum_{k=0}^{\min\left(m,n\right)}\frac{(-1)^kq^{{k\choose 2}}(q;q)_m (q;q)_n r^k}{(q;q)_{m-k}(q;q)_{n-k}(q;q)_k} H_{m-k}(x;q) H_{n-k}(y;q), \end{align} $$

where $(t;q)_k=\prod _{j=0}^{k-1}\left (1-tq^j\right )$ denotes the q-Pochhammer symbol. Note that the coefficient inside the sum in equation (4.3) is a polynomial in q.

In the following we will encounter rescaled versions of the continuous q-Hermite polynomials with coefficients in an associative algebra ${\mathcal M}$ over ${\mathbb K}$. Let ${\mathcal M}[x]={\mathcal M}\otimes _{\mathbb K} {\mathbb K}[x]$ and ${\mathcal M}[x,y]={\mathcal M}\otimes _{\mathbb K} {\mathbb K}[x,y]$ be the algebras of polynomials in one or two variables with coefficients in ${\mathcal M}$.

Proof. The recursion (4.2) implies that $H_{m,n}\left (x,y;q^2,r\right )$ is a polynomial with leading term $x^my^n$, and that if a monomial $x^iy^j$ appears in $H_{m,n}\left (x,y;q^2,r\right )$ with nonzero coefficient, then $i+j\equiv m+n \bmod 2$. This implies that equations (4.4) indeed define polynomials in ${\mathcal M}[x]$ and ${\mathcal M}[x,y]$. This proves (1). Properties (2) and (3) are obtained by replacing x and y by $x/b$ and $y/b$, respectively, and q by $q^2$ in the recursions (4.1) and (4.2). Property (4) follows from equation (4.3) and the relations

$$ \begin{align*} \left(q^2,q^2\right)_k&=(-1)^k q^{k(k+1)/2}\left(q-q^{-1}\right)^k[k]_q^!, \qquad \frac{\left(q^2;q^2\right)_n}{\left(q^2;q^2\right)_m \left(q^2;q^2\right)_{n-m}}&= q^{m(n-m)}\begin{bmatrix}n \\ m \end{bmatrix}_{q} \end{align*} $$

for $n\ge m$.

4.2 A relation between q-Hermite polynomials for $q^2$ and $q^{-2}$

In the following we fix $b^2\in {\mathcal M}$ and we let $w_m(x)\in {\mathcal M}[x]$ be the polynomial defined by equation (4.4). Moreover, set

(4.8)$$ \begin{align} v_m(x)=\sum_{k=0}^{\left\lfloor m/2 \right\rfloor} (-1)^k q^k \eta(k) \frac{[m]^!_q}{[m-2k]^!_q [k]^!_q} w_{m-2k}(x) \in {\mathcal M}[x], \end{align} $$

where, as before, $\eta (k)=\left (q-q^{-1}\right )^k q^{-k(k+1)/2} \left (b^2/4\right )^k$. In the following we set $w_k(x)=0$ for all $k< 0$, and hence in the definition (4.8) of $v_m(x)$ we may take the sum over all nonnegative integers k.

Lemma 4.3. The polynomials $v_m(x)$ satisfy the recursion

(4.9)$$ \begin{align} x v_m(x)= v_{m+1}(x) + \frac{b^2}{4}\left(1-q^{-2m}\right) v_{m-1}(x) \end{align} $$

with $v_0(x)=1$ and $v_1(x)=x$. Hence we have

(4.10)$$ \begin{align} v_m(x) = (b/2)^m H_m\left(\frac{x}{b};q^{-2}\right) \end{align} $$

for all $m\in {\mathbb N}$.

Proof. Using the definition (4.8) of $v_m(x)$ and the recursion (4.5), we calculate

$$ \begin{align*} xv_m(x) &- \frac{b^2}{4}\left(1-q^{-2m}\right)v_{m-1}(x)\\ &=\sum_{k=0}^{\left\lfloor m/2 \right\rfloor} (-1)^k q^k \eta(k) \frac{[m]^!_q}{[m-2k]^!_q [k]^!_q} \left(w_{m-2k+1}(x) + \frac{b^2}{4} \left(1-q^{2(m-2k)}\right)w_{m-2k-1}(x)\right)\\ & \quad-\frac{b^2}{4}\left(1-q^{-2m}\right)\sum_{k=0}^{\left\lfloor (m-1)/2 \right\rfloor} (-1)^k q^k \eta(k) \frac{[m-1]^!_q}{[m-1-2k]^!_q [k]^!_q}w_{m-1-2k}(x)\\ &= w_{m+1}(x) +\sum_{k\ge 1} (-1)^k q^k \eta(k) \frac{[m]^!_q}{[m+1-2k]^!_q [k]^!_q} \cdot\\ & \quad \cdot\left([m+1-2k]_q + [k]_q\left(q^{m-k+1} + q^{-m+k-1}\right) \right)w_{m+1-2k}(x)\\ &=v_{m+1}(x). \end{align*} $$

This confirms the recursion (4.9). The second statement holds by Lemma 4.2(1) and (2) with q replaced by $q^{-1}$.

4.3 Serre combinations of bivariate continuous q-Hermite polynomials

The following identity is the crucial ingredient needed in the following sections to express the quantum Serre relations for quantum symmetric pairs in terms of univariate continuous q-Hermite polynomials:

Proposition 4.4. Let $a\in -{\mathbb N}$ and assume that $r=q^a$. Then the polynomials $w_m(x), v_n(x)\in {\mathcal M}[x]$ and $w_{m,n}(x,y)\in {\mathcal M}[x,y]$ defined by equations (4.4) and (4.8) satisfy the relation

$$ \begin{align*} \sum_{n=0}^{1-a} (-1)^n \begin{bmatrix} 1-a\\ n \end{bmatrix}_q w_{1-a-n,n}(x,y) &= \sum_{n=0}^{1-a} (-1)^n \begin{bmatrix} 1-a\\ n \end{bmatrix}_q w_{1-a-n}(x)\,v_n(y) \\ &= \sum_{n=0}^{1-a} (-1)^n \begin{bmatrix} 1-a\\ n \end{bmatrix}_q v_{1-a-n}(x)\,w_{n}(y). \end{align*} $$

Proof. For $r=q^a$ and any $b^2\in {\mathcal M}$, we use equation (4.7) to calculate

$$ \begin{align*} &\sum_{n=0}^{1-a} (-1)^n \begin{bmatrix} 1-a\\ n \end{bmatrix}_q w_{1-a-n,n}(x,y)\\ & \qquad\qquad\qquad = \sum_{n=0}^{1-a} (-1)^n \begin{bmatrix} 1-a\\ n \end{bmatrix}_q \sum_{k\ge 0} q^{k}\eta(k)\begin{bmatrix} 1-a-n\\ k \end{bmatrix}_q \begin{bmatrix} n\\ k \end{bmatrix}_q [k]^!_q w_{1-a-n-k}(x)w_{n-k}(y), \end{align*} $$

where again $w_n(x)=0$ for $n<0$. Setting $\ell =n+k$ and $m=k$, we obtain

$$ \begin{align*} \sum_{n=0}^{1-a} (-1)^n &\begin{bmatrix} 1-a\\ n \end{bmatrix}_q w_{1-a-n,n}(x,y)\\ &= \sum_{\ell=0}^{1-a} (-1)^\ell \begin{bmatrix} 1-a\\ \ell \end{bmatrix}_q \sum_{m\ge 0} (-1)^mq^m \eta(m)\frac{[\ell]^!_q}{[\ell-2m]^!_q [m]^!_q} w_{1-a-\ell}(x) w_{\ell-2m}(y)\\ &=\sum_{\ell=0}^{1-a} (-1)^\ell \begin{bmatrix} 1-a\\ \ell \end{bmatrix}_q w_{1-a-\ell}(x) v_{\ell}(y), \end{align*} $$

which proves the first identity of the proposition. On the other hand, setting $\ell =1-a-n+k$ and $m=k$, we obtain

$$ \begin{align*} \sum_{n=0}^{1-a}& (-1)^n \begin{bmatrix} 1-a\\ n \end{bmatrix}_q w_{1-a-n,n}(x,y)\\ &=(-1)^{1-a} \sum_{\ell=0}^{1-a} (-1)^\ell \begin{bmatrix} 1-a\\ \ell \end{bmatrix}_q \sum_{m\ge 0} (-1)^mq^m \eta(m)\frac{[\ell]^!_q}{[\ell-2m]^!_q [m]^!_q} w_{\ell-2m}(x) w_{1-a-\ell}(y)\\ &=(-1)^{1-a}\sum_{\ell=0}^{1-a} (-1)^\ell \begin{bmatrix} 1-a\\ \ell \end{bmatrix}_q v_{\ell}(x) w_{1-a-\ell}(y) =\sum_{\ell=0}^{1-a} (-1)^\ell \begin{bmatrix} 1-a\\ \ell \end{bmatrix}_q v_{1-a-\ell}(x) w_{\ell}(y), \end{align*} $$

which proves the second identity.

4.4 Bar invariance of deformed quantum Serre polynomials

Assume that the ${\mathbb K}$-algebra ${\mathcal M}$ has a bar involution

$$ \begin{align*} \overline{\phantom{m}}^{\mathcal M}:{\mathcal M} \rightarrow {\mathcal M}, \qquad m\mapsto \overline{m}^{\mathcal M}, \end{align*} $$

which is a k-algebra isomorphism such that $\overline {q}^{\mathcal M}=q^{-1}$. We extend $\overline {\phantom {m}}^{\mathcal M}$ to a bar involution on ${\mathcal M}[x]$ and ${\mathcal M}[x,y]$ by application of $\overline {\phantom {m}}^{\mathcal M}$ to the coefficients of any polynomial. Again, we consider the polynomials $w_m(x), v_m(x)$ and $w_{m,n}(x,y)$ formed with respect to a fixed element $b^2\in {\mathcal M}\setminus \{0\}$. Define $(b')^2=\overline {b^2}^{\mathcal M}$ and let $w_m'(x), v_m'(x)$ and $w_{m,n}'(x,y)$ be the corresponding polynomials formed with respect to the element $(b')^2$.

Lemma 4.5. Retain the setting of Proposition 4.4 and set $(b')^2=\overline {b^2}^{\mathcal M}$. Then the relations

(4.11)$$ \begin{align} \overline{w_n(x)}^{\mathcal M}=v_n'(x) \qquad \text{and} \qquad \overline{v_n(x)}^{\mathcal M}=w_n'(x) \end{align} $$

hold in ${\mathcal M}[x]$ for all $n\in {\mathbb N}$.

This lemma and Proposition 4.4 allow us to describe the behaviour of the deformed quantum Serre polynomial under the bar involution.

Corollary 4.6. Retain the setting of Proposition 4.4 and set $(b')^2=\overline {b^2}^{\mathcal M}$. Then the relation

$$ \begin{align*} \overline{\sum_{n=0}^{1-a}(-1)^n \begin{bmatrix} 1-a\\ n \end{bmatrix}_q w_{1-a-n,n}(x,y)}^{\mathcal M} = \sum_{n=0}^{1-a}(-1)^n \begin{bmatrix} 1-a\\ n \end{bmatrix}_q w_{1-a-n,n}'(x,y) \end{align*} $$

holds in ${\mathcal M}[x,y]$.

Proof. By Lemma 4.5 and Proposition 4.4, we have

$$ \begin{align*} \overline{\sum_{n=0}^{1-a} (-1)^n \begin{bmatrix} 1-a\\ n \end{bmatrix}_q w_{1-a-n,n}(x,y)}^{\mathcal M} &= \sum_{n=0}^{1-a} (-1)^n \begin{bmatrix} 1-a\\ n \end{bmatrix}_q \overline{w_{1-a-n}(x)}^{\mathcal M}\overline{v_n(y)}^{\mathcal M}\\ &=\sum_{n=0}^{1-a} (-1)^n \begin{bmatrix} 1-a\\ n \end{bmatrix}_q v_{1-a-n}'(x)\,w_n'(y)\\ &= \sum_{n=0}^{1-a} (-1)^n \begin{bmatrix} 1-a\\ n \end{bmatrix}_q w_{1-a-n,n}'(x,y), \end{align*} $$

as desired.

4.5 Deformed Chebyshev polynomials of the second kind

For any $r\in {\mathbb K}$ consider the sequence of polynomials given by the recursion formula

(4.12)$$ \begin{align} C_{n+1}(x; q, r) = 2x C_{n}(x; q, r) - \frac{r^{-1}-q^{n+1}}{1 - q^{n+1}} C_{n-1}(x; q, r), \qquad n \geq 0, \end{align} $$

subject to the initial conditions

(4.13)$$ \begin{align} C_0(x; q, r) =1, \qquad C_1(x; q, r) =2x. \end{align} $$

It is clear from the recursion and the initial conditions that in the case $r=1$ we recover the Chebyshev polynomials of the second kind:

$$\begin{align*}C_n(x; q, 1) = U_n(x). \end{align*}$$

For background on the classical Chebyshev polynomials, see, for example, [Reference Magnus, Oberhettinger and SoniMOS69, Sect. 5.7].

Example 4.7. By direct computation, one obtains

(4.14)$$ \begin{align} C_2(x;q,r)&= 4 x^2- \frac{r^{-1}-q^2}{\left(1-q^2\right)}, \end{align} $$

(4.15)$$ \begin{align} C_3(x;q,r)&=8x^3 - 2x\left( \frac{r^{-1}-q^2}{1-q^2} + \frac{r^{-1}-q^3}{1-q^3} \right). \end{align} $$

By Favard’s theorem, $\{C_n(x;q,r)\}_{n=0}^\infty $ is a sequence of orthogonal polynomials with respect to a Borel measure when q and r are real and $\lvert q\rvert < 1 \leq r^{-1}$. We will call them deformed Chebyshev polynomials of the second kind. If q and r are viewed as indeterminates, then

$$\begin{align*}\left(1-q^2\right) \dotsm (1-q^n) C_n(x; q, r) \in {\mathbb Z}\left[q,r^{-1}\right] \qquad \text{for}\ n \geq 2. \end{align*}$$

The rescaled polynomials

$$\begin{align*}\widetilde{C}_n(x; q, r) = r^{n/2} C_n\left(r^{-1/2} x; q, r\right) \end{align*}$$

satisfy the recursion

(4.15a)$$\begin{align}\widetilde{C}_{n+1}(x; q, r) = 2x \widetilde{C}_{n}(x; q, r) - \frac{1- r q^{n+1}}{1 - q^{n+1}} \widetilde{C}_{n-1}(x; q, r), \qquad n \geq 0 \end{align}$$

and the same initial conditions as $C_{n}(x; q, r)$. In our recursion for the deformed Chebyshev polynomials (4.12) we used $r^{-1}$ instead of r because the recursion (4.15a) naturally leads to expressions involving q-Pochhammer symbols.

Next we provide explicit formulas for two generating functions for the deformed Chebyshev polynomials. In an appropriate field extension of ${\mathbb K}=k(q)$, define

(4.16)$$ \begin{align} x_{1,2}=x\pm \sqrt{x^2-1}, \qquad a_{1,2}=x\pm \sqrt{x^2-r^{-1}}, \qquad \widetilde{a}_{1,2} = r^{-1}\left(x\pm \sqrt{x^2-r}\right), \end{align} $$

which satisfy the relations

$$ \begin{align*} 1-2xs+s^2&=(1- x_1s)(1-x_2s), \\ 1-2xs+r^{-1}s^2&=(1-a_1s)(1-a_2s),\\ 1-2r^{-1} xs+r^{-1}s^2&=\left(1-\widetilde{a}_1s\right)\left(1-\widetilde{a}_2s\right). \end{align*} $$

Recall the definition of the basic hypergeometric function ${}_{3}\phi _2$[Reference Koekoek, Lesky and SwarttouwKLS10, 1.10], and define

(4.17)$$ \begin{align} \eta\left(\begin{matrix} x \\s \end{matrix};q\right)&=\frac{1}{1-2xs+r^{-1}s^2} {}_{3}\phi_2\left(\begin{array}{c} q,x_1s,x_2s \\qa_1s,qa_2s\end{array};q,q^2\right)\\ &=\sum_{k=0}^\infty \frac{(x_1s;q)_k(x_2s;q)_k}{(a_1s;q)_{k+1}(a_2s;q)_{k+1}}q^{2k}. \nonumber \end{align} $$

As the basic hypergeometric function ${}_{3}\phi _2\left (\begin {array}{c} q,x_1s,x_2s \\qa_1s,qa_2s\end {array};q,q^2\right )$ is analytic at $s=0$, so is $\eta \left (\begin {matrix} x \\s \end {matrix};q\right )$. Analogously,

$$\begin{align*}\widetilde{\eta} \left(\begin{matrix} x \\s \end{matrix};q\right)=\frac{1}{1-2xs+s^2} {}_{3}\phi_2\left(\begin{array}{c} q, q \widetilde{a}_1s, q \widetilde{a}_2 s \\qx_1s,qx_2s\end{array};q,q r\right) \end{align*}$$

is analytic at $s=0$.

Proposition 4.8. We have

$$ \begin{align*} \eta\left(\begin{matrix} x \\s \end{matrix};q, r\right)&= \sum_{n=0}^\infty C_n(x;q,r) \frac{s^n}{1-q^{n+2}} \end{align*} $$

and

$$ \begin{align*} \widetilde{\eta}\left(\begin{matrix} x \\s \end{matrix};q, r\right)&= \sum_{n=0}^\infty \frac{\left(q^2;q\right)_n}{(qr;q)_{n+1}} \widetilde{C}_n(x;q,r) s^n = \sum_{n=0}^\infty \frac{\left(q^2;q\right)_n}{(qr;q)_{n+1}} C_n\left(r^{-1/2}x;q,r\right) \left(r^{1/2} s\right)^n. \end{align*} $$

The first generating function will play a key role in Sections 5.5 and 5.6. In the special case $r=1$, the second one reduces to the standard generating function for the Chebyshev polynomials of the second kind:

$$\begin{align*}\sum_{n=0}^\infty U_n(x) s^n = \left(1 - 2xs +s^2\right)^{-1}. \end{align*}$$

Proof. For simplicity of notation, we suppress the arguments s and q of the functions $\eta $ and $\widetilde {\eta }$. Consider the Taylor expansion

$$\begin{align*}\eta(x)= \sum_{n=0}^\infty f_n(x;q) s^n \end{align*}$$

in the variable s with Taylor coefficients $f_n(x;q)$. We have

$$ \begin{align*} \eta(qs) &= \sum_{k=0}^\infty \frac{(qx_1s;q)_k(qx_2s;q)_k}{(qa_1s;q)_{k+1}(qa_2s;q)_{k+1}}q^{2k} \\ &= q^{-2} \frac{(1-a_1s)(1-a_2s)}{(1-x_1s)(1-x_2s)} \sum_{k=1}^\infty \frac{(x_1s;q)_k(x_2s;q)_k}{(a_1s;q)_{k+1}(a_2s;q)_{k+1}}q^{2k}\\ &= q^{-2} \frac{(1-a_1s)(1-a_2s)}{(1-x_1s)(1-x_2s)} \left(\eta(s)-\frac{1}{(1-a_1s)(1-a_2s)}\right)\\ &= q^{-2} \frac{1-2xs+r^{-1}s^2}{1-2xs+s^2} \left(\eta(s)-\frac{1}{1-2xs+r^{-1}s^2}\right), \end{align*} $$

and thus

$$ \begin{align*} q^2\left(1-2xs+s^2\right) \eta(qs) = \left(1-2xs+r^{-1}s^2\right) \eta(s)-1. \end{align*} $$

Comparing the coefficients of $s^{n+1}$ in the Taylor series expansion of both sides at $s=0$ gives

$$ \begin{align*} &q^{n+3} f_{n+1}(x;q) -2 q^{n+2}x f_{n}(x;q) + q^{n+1} f_{n-1}(x;q) \\ &\quad = f_{n+1}(x;q) - 2x f_{n}(x;q) + r^{-1}f_{n-1}(x;q) - \delta_{n,-1}. \end{align*} $$

Hence, $f_n(x;q)$ are uniquely determined by the recursion

$$ \begin{align*} \left(1-q^{n+3}\right) f_{n+1}(x;q) = 2x\left(1-q^{n+2}\right) f_{n}(x;q) - \left(r^{-1}-q^{n+1}\right)f_{n-1}(x;q) + \delta_{n,-1}, \end{align*} $$

with the initial condition $f_n(x;q)=0$ for $n<0$. Comparing this to the recursion (4.12) and the initial conditions (4.13) gives

$$\begin{align*}f_n(x;q) = C_n(x; q, r)/\left(1-q^{n+2}\right) \qquad \text{for}\ n \geq 0, \end{align*}$$

which proves the first generating function identity. Similarly, for the second identity, one first shows that

$$ \begin{align*} \widetilde{\eta}(qs) = (qr)^{-1} \frac{1-2xs+s^2}{1-2 r^{-1} q xs+ r^{-1} q^2 s^2} \left(\widetilde{\eta}(s)-\frac{1}{1-2xs+s^2}\right), \end{align*} $$

and then proceeds as before.

5.1 ${\mathcal M}_X^+$-valued orthogonal polynomials

For a suitable choice of $b^2\in {\mathcal M}_X^+$, the recursions (4.5) and (4.6) translate into the recursions given in A) and B) in Section 1.4. Indeed, recall that ${\mathcal Z}_i=q_ic_i \partial _{\tau (i)}^R\left (T_{w_X\left (E_{\tau (i)}\right )}\right )$ for $i\in I\setminus X$ and set

$$ \begin{align*} b_i^2= \frac{4}{\left(q_i-q_i^{-1}\right)^2}{\mathcal Z}_i\in {\mathcal M}_X^+. \end{align*} $$

For $b^2=b_i^2$, the rescaled continuous q-Hermite polynomials $w_m(x)$ defined for $m\in {\mathbb N}$ by

(5.1)$$ \begin{align} w_m(x) = (b_i/2)^m H_m\left(\frac{x}{b_i};q_i^2\right) \in {\mathbb K}\left[b_i^2,x\right] \subset {\mathcal M}_X^+[x] \end{align} $$

satisfy the initial conditions and recursion A) in Section 1.4. Hence we get $w_m(x)=w_m\left (x,q_i^2\right )$. Similarly,

(5.2)$$ \begin{align} v_m(x) = (b_i/2)^m H_m\left(\frac{x}{b_i};q_i^{-2}\right) \in {\mathbb K}\left[b_i^2,x\right] \subset {\mathcal M}_X^+[x] \end{align} $$

satisfy recursion A) in Section 1.4 with $q_i^{2m}$ replaced by $q_i^{-2m}$. Hence we get $v_m(x)=w_m\left (x,q_i^{-2}\right )$. Finally, for $i\in I\setminus X$ and $j\in I$, the rescaled bivariate continuous q-Hermite polynomials

(5.3)$$ \begin{align} w_{m,n}(x,y) = (b_i/2)^{m+n} H_{m,n}\left(\frac{x}{b_i},\frac{y}{b_i};q_i^2,q_i^{a_{ij}}\right) \in {\mathbb K}\left[b_i^2,x,y\right] \subset {\mathcal M}_X^+[x,y] \end{align} $$

satisfy the initial conditions and recursion B) in Section 1.4. Hence we get $w_{m,n}(x,y)=w_{m,n}\left (x,y;q_i^2,q_i^{a_{ij}}\right )$. By Lemma 4.2 we have

(5.4)$$ \begin{align} w_{0,n}(x,y)= w_n(y) \end{align} $$

for all $n\in {\mathbb N}$. All through Section 5, the notations $w_m(x)$, $v_m(x)$ and $w_{m,n}(x,y)$ will refer to these special instances of the polynomials investigated in Section 4. In particular, we may freely use the results of Section 4. To keep notation short we will suppress the dependence on i and j.

We will evaluate the polynomials $w_m(x)$, $v_m(x)$ and $w_{m,n}(x,y)$ on elements in the algebra $({\mathcal A},\ast )$. To distinguish the different algebra structures on ${\mathcal A}$, we introduce some notation. For any $u\in {\mathcal A}$ and any $n\in {\mathbb N}$, we write

$$ \begin{align*} u^{\ast n}= \underbrace{u\ast u\ast \dotsb \ast u}_{n\ \text{factors}}. \end{align*} $$

For any polynomial $w(x)=\sum _{n}\lambda _n x^n\in {\mathcal A}[x]$ and any $u\in {\mathcal A}$, we write

(5.5)$$ \begin{align} w(u)^\ast = \sum_n \lambda_n \ast u^{\ast n}. \end{align} $$

For any polynomial $w(x,y)=\sum _{s,t}\lambda _{st} x^s y^t\in {\mathcal A}[x,y]$ and any $a_1, a_2, a_3 \in {\mathcal A}$, we write

(5.6)$$ \begin{align} a_3 \curvearrowright w(a_1 \stackrel{\ast}{,} a_2) = \sum_{s,t} a_1^{\ast s} \ast a_3 \ast a_2^{\ast t}\ast \lambda_{st}. \end{align} $$

5.2 The powers of $F_i$ for $\tau (i)=i$

Fix $i\in I\setminus X$ with $\tau (i)=i$. The following proposition expresses $F_i^n$ for $n\in {\mathbb N}$ in terms of the star product $\ast $ on ${\mathcal A}$. Recall equations (5.1) and (5.5) and the recursion (4.5).

Proposition 5.2. Let $i\in I\setminus X$ with $\tau (i)=i$. The relation

(5.7)$$ \begin{align} F_i^m = w_m(F_i)^\ast \end{align} $$

holds in ${\mathcal A}$ for any $m\in {\mathbb N}$.

Proof. The relation (5.7) holds for $m=0$ and $m=1$. We proceed by induction on m. By Lemmas 3.3 and 3.5 we have

$$ \begin{align*} F_i\ast F_i^m&\stackrel{\phantom{\text{(3.6)}}}{=} F_i^{m+1} - c_i \frac{q^{\left(\alpha_i,w_X\left(\alpha_i\right)\right)}}{q_i-q_i^{-1}} K_{w_X\left(\alpha_i\right)-\alpha_i} \partial_{i,X}^L\left(F_i^{m}\right)\\ &\stackrel{\text{(3.6)}}{=} F_i^{m+1} - \frac{c_i q_i^2}{q_i-q_i^{-1}} (m)_{q_i^2} Z_i F_i^{m-1}\\ &\stackrel{\phantom{\text{(3.6)}}}{=} F_i^{m+1} + \frac{b_i^2}{4} \left(1-q_i^{2m}\right) F_i^{m-1}. \end{align*} $$

Hence by the induction hypothesis and equation (4.5), we get

$$ \begin{align*} F_i^{m+1} = F_i\ast w_m(F_i)^\ast - \frac{b_i^2}{4} \left(1-q_i^{2m}\right) w_{m-1}(F_i)^\ast = w_{m+1}(F_i)^\ast, \end{align*} $$

which completes the induction step.

5.3 The quantum Serre relation for $\tau (i)=i\neq j$ where $i,j\in I\setminus X$

We now want to rewrite the quantum Serre relation $S_{ij}\left (F_i,F_j\right )=0$ for $\tau (i)=i$, with $i,j\in I\setminus X$, in terms of the star product on the algebra ${\mathcal A}$. Recall the rescaled bivariate q-Hermite polynomials (5.3), the insertion operator defined by equation (5.6) and the recursion (4.6).

Proposition 5.3. Let $i,j\in I\setminus X$ with $\tau (i)=i\neq j$. Then the relation

(5.8)$$ \begin{align} F_i^m F_j F_i^n = F_j \curvearrowright w_{m,n}(F_i\stackrel{\ast}{,}F_i) \end{align} $$

holds in ${\mathcal A}$ for any $m,n\in {\mathbb N}$.

Proof. We prove equation (5.8) by induction on m. For $m=0$ the relation holds by Proposition 5.2 and equation (5.4). Now fix $m,n\in {\mathbb N}$. Equations (3.1), (3.2) and (3.6) imply

(5.9)$$ \begin{align} F_i\ast F_i^mF_jF_i^n &= F_i^{m+1} F_j F_i^n - \frac{c_i q^{\left(\alpha_i,w_X\left(\alpha_i\right)\right)}}{q_i-q_i^{-1}} K_{w_X\left(\alpha_i\right)-\alpha_i} \partial^L_{i,X}\left(F_i^mF_jF_i^n\right)\nonumber\\ &= F_i^{m+1} F_j F_i^n - \frac{c_i q^{\left(\alpha_i,\alpha_i\right)}}{q_i-q_i^{-1}} (m)_{q_i^2} Z_i F_i^{m-1}F_j F_i^n \nonumber\\ &\quad - \frac{c_i q^{\left(\alpha_i,(m+1)\alpha_i+\alpha_j\right)}}{q_i-q_i^{-1}} (n)_{q_i^2} Z_i F_i^{m}F_j F_i^{n-1} \nonumber\\ &= F_i^{m+1} F_j F_i^n + \frac{b_i^2}{4} \left(1-q_i^{2m}\right) F_i^{m-1}F_j F_i^n \nonumber\\ &\quad + \frac{b_i^2}{4} q_i^{2m+a_{ij}}\left(1-q_i^{2n}\right) F_i^{m}F_j F_i^{n-1}. \end{align} $$

Hence by the induction hypothesis and equation (4.6) for $r=q^{a_{ij}}$ we get

$$ \begin{align*} F_i^{m+1}F_j F_i^n &= F_i\ast \left( F_j \curvearrowright w_{m,n}(F_i\stackrel{\ast}{,}F_i)\right) - \frac{b_i^2}{4}\left(1-q_i^{2m}\right)F_j \curvearrowright w_{m-1,n}(F_i\stackrel{\ast}{,}F_i)\\ & \quad -\frac{b_i^2}{4} q_i^{2m+a_{ij}}\left(1-q_i^{2n}\right)F_j \curvearrowright w_{m,n-1}(F_i\stackrel{\ast}{,}F_i)\\ &= F_j\curvearrowright \left(x\cdot w_{m,n} - \frac{b_i^2}{4}\left(1-q_i^{2m}\right) w_{m-1,n} \right.\\ &\quad \left. -\frac{b_i^2}{4} q_i^{2m+a_{ij}}\left(1-q_i^{2n}\right) w_{m,n-1} \right)(F_i\stackrel{\ast}{,}F_i)\\ &=F_j \curvearrowright w_{m+1,n}(F_i\stackrel{\ast}{,}F_i), \end{align*} $$

which completes the induction step.

With this proposition we are able to express the quantum Serre relation $S_{ij}\left (F_i,F_j\right )=0$ in ${\mathcal A}$ in terms of the star product. Recall equations (5.1)–(5.3).

Theorem 5.4. Let $i,j\in I\setminus X$ with $\tau (i)=i\neq j$. Then the relation

$$ \begin{align*} \sum_{n=0}^{1-a_{ij}}(-1)^n \begin{bmatrix} 1-a_{ij}\\ n \end{bmatrix}_{q_i} F_j \curvearrowright w_{1-a_{ij}-n,n}(F_i \stackrel{\ast}{,}F_i) =0 \end{align*} $$

holds in the algebra $({\mathcal A},\ast )$. This relation can be rewritten as

$$ \begin{align*} \sum_{n=0}^{1-a_{ij}}(-1)^n \begin{bmatrix} 1-a_{ij}\\ n \end{bmatrix}_{q_i} w_{1-a_{ij}-n}(F_i)^\ast \ast F_j \ast v_{n}(F_i)^\ast =0. \end{align*} $$

In view of the isomorphism $\psi :{\mathcal B}_{\mathbf {c}}\rightarrow ({\mathcal A},\ast )$, Theorem 5.4 proves case (I) of Theorem 1.2.

5.4 A recursive formula in the case $\tau (i)=i\neq j$ where $i\in I\setminus X$ and $j\in X$

Assume that $i\in I\setminus X$ with $\tau (i)=i$ and $j\in X$. Recall from equation (1.5) that in this case we write

(5.10)$$ \begin{align} &{\mathcal Z}_i=c_iq_i Z_i=c_iq_i \partial_i^R\left(T_{w_X}(E_i)\right). \end{align} $$

Moreover, we set

(5.11)$$ \begin{align} d_{ij}=\partial_j^R({\mathcal Z}_i)K_j, \qquad \widetilde{d_{ij}}=K_j^{-1}\partial_j^L({\mathcal Z}_i), \end{align} $$

so that equation (2.4) now reads

(5.12)$$ \begin{align} \left[F_j,{\mathcal Z}_i\right] = \frac{\widetilde{d_{ij}}-d_{ij}}{q_j-q_j^{-1}}. \end{align} $$

Recall from Section 2.6 that it is convenient to simultaneously consider the parameters ${\mathbf {c}}=(c_i)_{i\in I\setminus X}\in {\mathcal C}$ and ${\mathbf {c}}'=\left (c_i'\right )_{i\in I\setminus X}\in {\mathcal C}$ related by equation (2.24). We write ${\mathcal Z}_i'$, $d^{\prime }_{ij}$, $\widetilde {d^{\prime }_{ij}}$ to denote the elements (5.10) and (5.11) corresponding to the parameter family ${\mathbf {c}}'$. It follows from [Reference Balagović and KolbBK15, Lemma 2.9] and [Reference Bao and WangBW21, Theorem 4.1] that

(5.13)$$ \begin{align} \overline{{\mathcal Z}_i}=\overline{c_i} q_i (-1)^{2\alpha_i\left(\rho_X^\vee\right)}q^{\left(\alpha_i,\Theta\left(\alpha_i\right)-2\rho_X\right)} \partial_i^R\left(T_{w_X}(E_i)\right)={\mathcal Z}^{\prime}_i. \end{align} $$

Moreover, we have $d_{ij}\in K_j{\mathcal M}_X^+$ and $\widetilde {d_{ij}}\in K_j^{-1}{\mathcal M}_X^+$. Hence, applying the bar involution to equation (5.12) gives us

$$ \begin{align*} \overline{\widetilde{d_{ij}}}=d^{\prime}_{ij}, \qquad \overline{d_{ij}}=\widetilde{d^{\prime}_{ij}}. \end{align*} $$

These relations will be used in the proof of Theorem 5.10.

The following lemma provides a formula similar to equation (5.9) in the present setting:

Lemma 5.5. Let $i\in I\setminus X$, with $\tau (i)=i$ and $j\in X$. Then the relation

(5.14)$$ \begin{align} F_i\ast F_i^mF_j F_i^n &= F_i^{m+1} F_j F_i^n \nonumber\\ &\quad + \left(1-q_i^{2m}\right) F_i^{m-1} F_j F_i^n \frac{b_i^2}{4} + q_i^{2 m + a_{ij}} \left(1-q_i^{2n}\right) F_i^m F_j F_i^{n-1}\frac{b_i^2}{4} \nonumber\\ & \quad+ \frac{q_i^{(m-1)a_{ij}} \left(1-q_i^{2m}\right)}{\lambda_{ij}} d_{ij} F_i^{m+n-1} -\frac{q_i^{-(m-1)a_{ij}} \left(1-q_i^{2(m+n)}\right)}{\lambda_{ij}} \widetilde{d_{ij}} F_i^{m+n-1} \end{align} $$

holds in ${\mathcal A}$ for all $m,n\in {\mathbb N}$ with $b_i^2=4 {\mathcal Z}_i/\left (q_i-q_i^{-1}\right )^2$ and $\lambda _{ij}=\left (q_i-q_i^{-1}\right )^2\left (q_j-q_j^{-1}\right )$.

Proof. For all $m,n\in {\mathbb N}$, we have

(5.15)$$ \begin{align} F_i^m F_j F_i^n= q^{n\left(\alpha_j,\alpha_i\right)} F_i^{m+n} F_j - q^{n\left(\alpha_j,\alpha_i\right)} F_i^m {\mbox{ad}}_r\left(F_j\right)\left(F_i^n\right), \end{align} $$

and hence equation (3.1) implies

(5.16)$$ \begin{align} F_i\ast F_i^mF_j F_i^n &= F_i^{m+1} F_j F_i^n - \frac{c_i q^{\left(\alpha_i,w_X\left(\alpha_i\right)+n\alpha_j\right)}}{q_i-q_i^{-1}} K_{w_X\left(\alpha_i\right)-\alpha_i} \partial_{i,X}^L\left(F_i^{m+n}\right) F_j\nonumber\\ & \quad + \frac{c_i q^{\left(\alpha_i,w_X\left(\alpha_i\right)+n\alpha_j\right)}}{q_i-q_i^{-1}} K_{w_X\left(\alpha_i\right)-\alpha_i} \partial_{i,X}^L\left(F_i^m {\mbox{ad}}_r\left(F_j\right)\left(F_i^n\right)\right). \end{align} $$

In view of the skew derivation property (3.2), equations (3.6) and (3.11) allow us to rewrite this as

$$ \begin{align*} F_i\ast F_i^mF_j F_i^n &= F_i^{m+1} F_j F_i^n - \frac{c_i q^{\left(\alpha_i,\alpha_i+n\alpha_j\right)}}{q_i-q_i^{-1}} (m+n)_{q_i^2} Z_i F_i^{m+n-1} F_j\\ & \quad + \frac{c_i q^{\left(\alpha_i,\alpha_i+n\alpha_j\right)}}{q_i-q_i^{-1}} (m)_{q_i^2} Z_i F_i^{m-1} {\mbox{ad}}_r\left(F_j\right)\left(F_i^n\right)\\ & \quad +\frac{c_i q^{(m+1)\left(\alpha_i,\alpha_i+\alpha_j\right)-\left(\alpha_j,w_X\left(\alpha_i\right)-\alpha_i-\alpha_j\right)}}{\left(q_i-q_i^{-1}\right)\left(q_j-q_j^{-1}\right)} (n)_{q_i^2} K_j \partial_j^R(Z_i) F_i^{m+n-1}\\ & \quad + \frac{c_i q^{\left(\alpha_i,(m+1)\alpha_i+n\alpha_j\right)}}{q_i-q_i^{-1}} (n)_{q_i^2} Z_i F_i^m {\mbox{ad}}_r\left(F_j\right)\left(F_i^{n-1}\right). \end{align*} $$

Using the relation $(m+n)_{q_i^2}=(m)_{q_i^2} + q_i^{2m}(n)_{q_i^2}$ and equation (5.15) for the third and fifth terms, we obtain

$$ \begin{align*} &F_i\ast F_i^mF_j F_i^n = F_i^{m+1} F_j F_i^n - \frac{c_i q_i^2}{q_i-q_i^{-1}}\left( (m)_{q_i^2} Z_i F_i^{m-1} F_j F_i^n + \right. \\ &\qquad\qquad\qquad\left. + q_i^{2 m + a_{ij}} (n)_{q_i^2} Z_i F_i^m F_j F_i^{n-1}\right) +\frac{c_i q_i^{(2m+2)-(n-2)a_{ij}}}{\left(q_i-q_i^{-1}\right)\left(q_j-q_j^{-1}\right)} (n)_{q_i^2} F_i^{m+n-1} \partial_j^R(Z_i) K_j. \end{align*} $$

Using the notation of equations (5.10) and (5.11) and the commutation relation (5.12), one transforms this equation into equation (5.14).

Recall the insertion operation defined by equation (5.6) and the rescaled bivariate continuous q-Hermite polynomials $w_{m,n}(x,y)$ defined by equation (5.3). The recursion (5.14) implies that there exist polynomials $\rho _{m,n}(x), \sigma _{m,n}(x)\in {\mathcal M}_X^+[x]$ such that

(5.17)$$ \begin{align} F_i^m F_j F_i^n = F_j\curvearrowright w_{m,n}(F_i\stackrel{\ast}{,}F_i) + d_{ij} \rho_{m,n}(F_i)^\ast + \widetilde{d_{ij}} \sigma_{m,n}(F_i)^\ast. \end{align} $$

For $m=0$, we have $F_jF_i^n=F_j\curvearrowright w_n(F_i)^\ast $ and hence $\rho _{0,n}(x)=\sigma _{0,n}(x)=0$ for all $n\in {\mathbb N}$. We can translate the recursion (5.14) into recursive formulas for $\rho _{m,n}(x)$ and $\sigma _{m,n}(x)$.

Lemma 5.6. With the ansatz (5.17), the recursion (5.14) is equivalent to the recursions

(5.18)$$ \begin{align} q_i^{a_{ij}} x\rho_{m,n}(x)&= \rho_{m+1,n}(x) +\left(1-q_i^{2m}\right)\rho_{m-1,n}(x) \frac{b_i^2}{4}\nonumber\\& \quad + \left(1-q_i^{2n}\right)q_i^{2m+a_{ij}} \rho_{m,n-1}(x) \frac{b_i^2}{4} + \frac{q_i^{(m-1)a_{ij}}\left(1-q_i^{2m}\right)}{\lambda_{ij}} w_{m+n-1}(x),\\q_i^{-a_{ij}} x\sigma_{m,n}(x)&= \sigma_{m+1,n}(x) +\left(1-q_i^{2m}\right) \sigma_{m-1,n}(x) \frac{b_i^2}{4}\nonumber\\& \quad + \left(1-q_i^{2n}\right)q_i^{2m+a_{ij}} \sigma_{m,n-1}(x) \frac{b_i^2}{4} - \frac{q_i^{-(m-1)a_{ij}}\left(1-q_i^{2(m+n)}\right)}{\lambda_{ij}} w_{m+n-1}(x)\nonumber \end{align} $$

for all $m,n\in {\mathbb N}$.

From now on we focus on the polynomials $\rho _{m,n}$ only. In Section 5.6 we will determine the Serre combination of the polynomials $\rho _{m,n}$. The Serre combination of the polynomials $\sigma _{m,n}$ can then be obtained with the help of the isomorphism $\Phi $ from Section 2.6.

We produce an unscaled version of the recursion (5.18). Define polynomials $U_{m,n}(x;q,r)\in {\mathbb K}[x]$ recursively by $U_{0,n}(x;q,r)=0$ for all $n\in {\mathbb N}$ and

(5.19)$$ \begin{align} 2x U_{m,n}(x)&= U_{m+1,n}(x) + (1-q^m) r^{-1} U_{m-1,n}(x) + q^m(1-q^n)U_{m,n-1}(x) \nonumber\\ & \quad +(1-q^m) H_{m+n-1}(x). \end{align} $$

The following statement is an immediate consequence of equation (5.18):

Lemma 5.7. The relation

$$ \begin{align*} \rho_{m,n}(x)=\frac{q_i^{(m-2)a_{ij}}}{\lambda_{ij}} \left(b_i/2\right)^{m+n-2} U_{m,n}\left(\frac{x}{b_i};q_i^2,q_i^{2a_{ij}}\right) \end{align*} $$

holds for all $m,n\in {\mathbb N}$ with $b_i^2=4{\mathcal Z}_i/\left (q_i-q_i^{-1}\right )^2$.

5.5 A generating function approach for $\tau (i)=i\neq j$ where $i\in I\setminus X$ and $j\in X$

Define a generating function

$$ \begin{align*} \psi\left(\begin{matrix} x \\s,t\end{matrix};q\right)=\sum_{m,n\ge 0} \frac{H_{m+n}(x;q)}{(q;q)_m(q;q)_n}s^mt^n. \end{align*} $$

Using the recursions (4.1) and (4.2) and induction over m, one obtains

$$ \begin{align*} H_{m,n}(x,x;q,1)=H_{m+n}(x;q). \end{align*} $$

Hence [Reference Casper, Kolb and YakimovCKY21, Theorem 2.7] implies that

$$ \begin{align*} \psi\left(\begin{matrix} x \\s,t\end{matrix};q\right) = \frac{(st;q)_\infty}{\left\lvert\left(se^{i\theta},te^{i\theta};q\right)_\infty\right\rvert^2} \end{align*} $$

for $x=\cos (\theta )$. Following [Reference Casper, Kolb and YakimovCKY21, Section 3.1], we have

$$ \begin{align*} \psi\left(\begin{matrix} x \\qs,t\end{matrix};q\right) = \frac{1-2xs+s^2}{1-ts} \psi\left(\begin{matrix} x \\s,t\end{matrix};q\right). \end{align*} $$

In terms of $x_{1,2}=x\pm \sqrt {x^2-1}$, defined in equation (4.16), this can be rewritten as

(5.20)$$ \begin{align} \psi\left(\begin{matrix} x \\qs,t\end{matrix};q\right) = \frac{(1-x_1s)(1-x_2s)}{1-ts} \psi\left(\begin{matrix} x \\s,t\end{matrix};q\right). \end{align} $$

Now recall the polynomials $U_{m,n}(x;q,r)$ defined by the recursion (5.19) and consider the generating function

(5.21)$$ \begin{align} \phi\left(\begin{matrix} x \\s,t\end{matrix};q\right) = \sum_{m,n\ge 0} \frac{U_{m,n}(x;q,r)}{(q;q)_m(q;q)_n}s^mt^n. \end{align} $$

The initial condition $U_{0,n}(x;q,r)=0$ implies that

(5.22)$$ \begin{align} \phi\left(\begin{matrix} x \\0,t\end{matrix};q\right)=0. \end{align} $$

Finally, recall the definition (4.17) of the function $\eta \left (\begin {matrix} x \\s \end {matrix};q\right )$, which is analytic at $s=0$.

Lemma 5.8. The relation

(5.23)$$ \begin{align} \phi\left(\begin{matrix} x \\s,t\end{matrix};q\right)=-s^2 \eta\left(\begin{matrix} x \\s \end{matrix};q\right) \psi\left(\begin{matrix} x \\s,t\end{matrix};q\right) \end{align} $$

holds as an identity of formal power series in s and t with coefficients in ${\mathbb K}[x]$.

Proof. The recursion (5.19) implies that

$$ \begin{align*} 2x \phi\left(\begin{matrix} x \\s,t\end{matrix};q\right)&= \frac{1}{s}\left( \phi\left(\begin{matrix} x \\s,t\end{matrix};q\right) - \phi\left(\begin{matrix} x \\qs,t\end{matrix};q\right)\right)\\ &\quad + s r^{-1} \phi\left(\begin{matrix} x \\s,t\end{matrix};q\right) + t \phi\left(\begin{matrix} x \\qs,t\end{matrix};q\right) +s \psi\left(\begin{matrix} x \\s,t\end{matrix}; q\right), \end{align*} $$

and hence

$$ \begin{align*} (1-ts) \phi\left(\begin{matrix} x \\qs,t\end{matrix};q\right)=\left(1-2xs + r^{-1}s^2\right) \phi\left(\begin{matrix} x \\s,t\end{matrix};q\right) + s^2 \psi\left(\begin{matrix} x \\s,t\end{matrix};q\right). \end{align*} $$

This relation can be rewritten as

$$ \begin{align*} \phi\left(\begin{matrix} x \\s,t\end{matrix};q\right)= \frac{1-ts}{(1-a_1s)(1-a_2s)} \phi\left(\begin{matrix} x \\qs,t\end{matrix};q\right) - \frac{s^2}{(1-a_1s)(1-a_2s)} \psi\left(\begin{matrix} x \\s,t\end{matrix};q\right), \end{align*} $$

where $a_{1,2}=x\pm \sqrt {x^2-r^{-1}}$ satisfy the relation $1-2xs+r^{-1}s^2=(1-a_1s)(1-a_2s)$, see equation (4.16). By induction over n, this formula together with equation (5.20) gives

$$ \begin{align*} \phi\left(\begin{matrix} x \\s,t\end{matrix};q\right)&= \frac{(ts;q)_n}{(a_1s;q)_n(a_2s;q)_n} \phi\left(\begin{matrix} x \\q^ns,t\end{matrix};q\right)\\ &\quad - s^2 \sum_{k=0}^{n-1}\frac{q^{2k}(x_1s;q)_k(x_2s;q)_k}{(a_1s;q)_{k+1}(a_2s;q)_{k+1}} \psi\left(\begin{matrix} x \\s,t\end{matrix};q\right). \end{align*} $$

Now we argue analytically for $q\in {\mathbb C}$ with $\lvert q\rvert <1$. In the limit $n\to \infty $, the first term in this expression vanishes by equation (5.22), and hence we get the desired formula (5.23).

5.6 The quantum Serre combination of the polynomials $\rho _{m,n}$

To simplify notation, set $N=1-a_{ij}$ and $q=q_i$. By equation (5.17) and Lemma 5.7, we need to determine the following polynomial:

(5.24)$$ \begin{align} \sum_{m=0}^N(-1)^m\begin{bmatrix}N\\m\end{bmatrix}_{q}\rho_{N-m,m}(x) &=(-1)^N \sum_{m=0}^N(-1)^m\begin{bmatrix}N\\m\end{bmatrix}_{q}\rho_{m,N-m}(x)\\ &=(-1)^N \frac{q^{2(N-1)}}{\lambda_{ij}}\left(b_i/2\right)^{N-2}P_N(x/b_i;q), \nonumber \end{align} $$

where

(5.25)$$ \begin{align} P_N(x;q)=\sum_{m=0}^N (-1)^m\begin{bmatrix}N\\m\end{bmatrix}_q q^{(1-N)m} U_{m,N-m}\left(x;q^2,q^{2(1-N)}\right). \end{align} $$

Up to an overall factor, the polynomial $P_N(x;q)$ is a deformed Chebyshev polynomial of the second kind, as defined in Section 4.5.

Lemma 5.9. For any $N\in {\mathbb N}$, we have

(5.26)$$ \begin{align} P_N(x;q)=(-1)^{N-1} q^{(1-N)N}\left(q^2;q^2\right)_{N-1} C_{N-2}\left(x;q^2, q^{2(1-N)}\right). \end{align} $$

Proof. By Proposition 4.8, Lemma 5.8 and equation (5.21) we have

(5.27)$$ \begin{align} P_N(x;q)&=\sum_{m=0}^N(-1)^{m-1}\begin{bmatrix}N\\m\end{bmatrix}_q q^{(1-N)m}\left(q^2;q^2\right)_m \cdot\nonumber\\ & \quad \cdot\sum_{k=2}^m \frac{C_{k-2}\left(x;q^2, q^{2(1-N)}\right)}{1-q^{2k}} \frac{H_{N-k}\left(x;q^2\right)}{\left(q^2,q^2\right)_{m-k}}\nonumber\\ &=\sum_{k=2}^N \frac{\omega_{N,k}(q)}{1-q^{2k}} C_{k-2}\left(x;q^2, q^{2(1-N)}\right) H_{N-k}\left(x;q^2\right), \end{align} $$

where

$$ \begin{align*} \omega_{N,k}(q)=\sum_{m=k}^N (-1)^{m-1}\begin{bmatrix}N\\m\end{bmatrix}_q \frac{\left(q^2;q^2\right)_m}{\left(q^2;q^2\right)_{m-k}}q^{(1-N)m}. \end{align*} $$

Using the relation $\left (q^2;q^2\right )_m=(-1)^m q^{m(m+1)/2}\left (q-q^{-1}\right )^m [m]^!_q$ and setting $\ell =m-k$, we obtain

$$ \begin{align*} \omega_{N,k}(q)=- q^{k(k+1)/2}q^{(1-N)k}\left(q-q^{-1}\right)^k \frac{[N]^!_q}{[N-k]^!_q} \sum_{\ell=0}^{N-k}(-1)^\ell \begin{bmatrix}N-k\\ \ell\end{bmatrix}_q q^{\ell(k-N+1)}. \end{align*} $$

By [Reference LusztigLus94, 1.3.4], we obtain

$$ \begin{align*} \omega_{N,k}(q)=\begin{cases} 0 & \text{if}\ k\neq N,\\ (-1)^{N-1} \left(q^2;q^2\right)_N q^{(1-N)N}& \text{if}\ k=N. \end{cases} \end{align*} $$

Inserting this into equation (5.27), we obtain the desired formula.

Inserting equation (5.26) into equation (5.24), we obtain

(5.28)$$ \begin{align} &\sum_{m=0}^N(-1)^m\begin{bmatrix}N\\m\end{bmatrix}_{q}\rho_{N-m,m}(x) \nonumber\\ &\quad =- \frac{q^{(N-2)(1-N)}}{\lambda_{ij}}\left(\frac{b_i}{2}\right)^{N-2} \left(q^2;q^2\right)_{N-1} C_{N-2}\left(x/b_i;q^2, q^{2(1-N)}\right). \end{align} $$

5.7 The quantum Serre relation for $\tau (i)=i$ where $i\in I\setminus X$ and $j\in X$

We are now in a position to write down the deformed quantum Serre relation (1.4) in the case $i\in I\setminus X$, $\tau (i)=i$ and $j\in X$. Recall the antilinear algebra isomorphism $\Phi :{\mathcal B}_{\mathbf {c}}\rightarrow {\mathcal B}_{{\mathbf {c}}'}$ from Theorem 2.12. Using the parameters ${\mathbf {c}}$ and ${\mathbf {c}}'$, we obtain two star products $\ast $ and $\ast '$ on ${\mathcal A}$ such that ${\mathcal B}_{\mathbf {c}}\cong ({\mathcal A},\ast )$ and ${\mathcal B}_{{\mathbf {c}}'}\cong ({\mathcal A},\ast ')$, respectively. Under these identifications we may consider $\Phi $ as an antilinear algebra isomorphism

$$ \begin{align*} \Phi:({\mathcal A},\ast) \rightarrow ({\mathcal A},\ast'). \end{align*} $$

Also recall the elements $b_i^2=4{\mathcal Z}_i/\left (q_i-q_i^{-1}\right )^2$ and write $\left (b_i^2\right )'=4{\mathcal Z}_i'/\left (q_i-q_i^{-1}\right )^2$. By equation (5.13) we have

(5.29)$$ \begin{align} \Phi\left(b_i^2\right)=\overline{b_i^2}=\left(b_i'\right)^2 \end{align} $$

With these notational preliminaries, we are ready to prove our main result:

Theorem 5.10. Let $i\in I\setminus X$ with $\tau (i)=i$ and $j\in X$. Then the relation

$$ \begin{align*} \sum_{n=0}^{1-a_{ij}}(-1)^n \begin{bmatrix} 1-a_{ij}\\ n \end{bmatrix}_{q_i} F_j \curvearrowright w_{1-a_{ij}-n,n}(F_i\stackrel{\ast}{,}F_i) + C + D =0 \end{align*} $$

holds in the algebra $({\mathcal A},\ast )$, where

(5.30)$$ \begin{align}C&= -\frac{d_{ij}}{\lambda_{ij}} q_i^{-a_{ij}\left(a_{ij} +1\right)} \left(q_i^2; q_i^2\right)_{-a_{ij}} \left(\frac{b_i}{2}\right)^{-a_{ij}-1} C_{-a_{ij}-1} \left(\frac{F_i}{b_i}; q_i^2, q_i^{2 a_{ij}}\right)^\ast, \end{align} $$

(5.31)$$ \begin{align}D&= \frac{\widetilde{d_{ij}}}{\lambda_{ij}}q_i^{a_{ij}\left(a_{ij} +1\right)} \left(q_i^{-2}; q_i^{-2}\right)_{-a_{ij}} \left(\frac{b_i}{2}\right)^{-a_{ij}-1} C_{-a_{ij}-1} \left(\frac{F_i}{b_i}; q_i^{-2}, q_i^{-2 a_{ij}}\right)^\ast. \end{align} $$

Proof. By equation (5.17) we have the relation

(5.32)$$ \begin{align} 0=\sum_{n=0}^{1-a_{ij}} (-1)^n \begin{bmatrix}1-a_{ij} \\ n \end{bmatrix}_{q_i}F_i^{1-a_{ij} - n} F_j F_i^n = A+C+D \end{align} $$

in ${\mathcal A}$, with

(5.33)$$ \begin{align} A&= \sum_{n=0}^{1-a_{ij}} (-1)^n \begin{bmatrix}1-a_{ij} \\ n \end{bmatrix}_{q_i} F_j \curvearrowright w_{1- a_{ij} -n, n}(F_i\stackrel{\ast}{,}F_i) \\C&= d_{ij} \sum_{n=0}^{1-a_{ij}} (-1)^n \begin{bmatrix}1-a_{ij} \\ n \end{bmatrix}_{q_i} \rho_{1-a_{ij} - n, n}(F_i)^*,\nonumber \\D&= \widetilde{d_{ij}} \sum_{n=0}^{1-a_{ij}} (-1)^n \begin{bmatrix}1-a_{ij} \\ n \end{bmatrix}_{q_i} \sigma_{1-a_{ij} - n, n}(F_i)^*.\nonumber \end{align} $$

By equation (5.28) we obtain

$$ \begin{align*} C&= -\frac{d_{ij}}{\lambda_{ij}} \left( \frac{b_i}{2} \right)^{-a_{ij}-1} q_i^{-a_{ij}\left(a_{ij} +1\right)} \left(q_i^2; q_i^2\right)_{-a_{ij}} C_{-a_{ij}-1} \left(\frac{F_i}{b_i}; q_i^2, q_i^{2 a_{ij}}\right)^\ast, \end{align*} $$

which proves equation (5.30). For the parameters ${\mathbf {c}}'\in {\mathcal C}$ defined by equation (2.24), we write equation (5.32) as

$$ \begin{align*} A'+C'+D'=0. \end{align*} $$

Equation (5.29) and Corollary 4.6 imply that $\Phi (A)=A'$. Hence we obtain

(5.34)$$ \begin{align} \Phi(C) + \Phi(D) = C' + D'. \end{align} $$

Relation (5.11) implies that

$$ \begin{gather*} C= K_j p_C(F_i)^\ast, \qquad C'= K_j p_C'(F_i)^\ast, \\ D= K_j^{-1} p_D(F_i)^\ast, \qquad D'= K_j^{-1} p_D'(F_i)^\ast, \end{gather*} $$

for some polynomials $p_C(x), p_D(x), p_C'(x), p_D'(x)\in {\mathcal M}_X^+[x]$. As $\Phi (K_j)=K_j^{-1}$, equation (5.34) hence implies that $\Phi (C)=D'$ and $\Phi (D)=C'$. This gives us

$$ \begin{align*} D&=\Phi^{-1}(C')\\ &=\frac{\widetilde{d_{ij}}}{\lambda_{ij}}\left( \frac{b_i}{2} \right)^{-a_{ij}-1} q_i^{a_{ij}\left(a_{ij} +1\right)} \left(q_i^{-2}; q_i^{-2}\right)_{-a_{ij}} C_{-a_{ij}-1} \left(\frac{F_i}{b_i}; q_i^{-2}, q_i^{-2 a_{ij}}\right)^\ast, \end{align*} $$

which proves equation (5.31).

For $n\in {\mathbb N}$, set

$$ \begin{align*} u_n\left(x;q_i^2,q_i^{2a_{ij}}\right)=\left(\frac{b_i}{2}\right)^n C_n\left(\frac{x}{b_i};q_i^2,q_i^{2a_{ij}}\right)\in {\mathcal M}_X^+[x]. \end{align*} $$

The polynomials $u_n\left (x;q_i^2,q_i^{2a_{ij}}\right )$ satisfy the initial conditions and recursion given in C) in Section (1.4). With this notation we get

$$ \begin{align*} C&= -\frac{d_{ij}}{\lambda_{ij}} q_i^{-a_{ij}\left(a_{ij} +1\right)} \left(q_i^2; q_i^2\right)_{-a_{ij}} u_{-a_{ij}-1} \left(F_i; q_i^2, q_i^{2 a_{ij}}\right)^\ast,\\ D&= \frac{\widetilde{d_{ij}}}{\lambda_{ij}}q_i^{a_{ij}\left(a_{ij} +1\right)} \left(q_i^{-2}; q_i^{-2}\right)_{-a_{ij}} u_{-a_{ij}-1} \left(F_i; q_i^{-2}, q_i^{-2 a_{ij}}\right)^\ast. \end{align*} $$

Inserting these expressions for C and D and equation (5.11) into the formula in Theorem 5.10, we obtain case (II) of Theorem 1.2 under the isomorphism $\psi :{\mathcal B}_{\mathbf {c}}\rightarrow ({\mathcal A},\ast )$.

Remark 5.11. We can use the formula from Proposition 4.4 to rewrite the term A given by equation (5.33) as

$$ \begin{align*} A = \sum_{n=0}^{1-a_{ij}}(-1)^n \begin{bmatrix} 1-a_{ij}\\ n \end{bmatrix}_{q_i} w_{1-a_{ij}-n}(F_i)^\ast \ast F_j \ast v_{n}(F_i)^\ast. \end{align*} $$

However, following Remark 5.1, this formula needs to be interpreted such that any coefficients $b_i^k\in {\mathcal M}_X^+$ coming from $w_{1-a_{ij}-n}(F_i)^\ast $ are moved to the right hand side of the factor $F_j$.

Example 5.12. The formulas in Theorem 5.10 allow us to write down explicit expressions for the deformed quantum Serre relations in the case $\tau (i)=i\in I\setminus X$ and $j\in X$. We obtain

$$\begin{align*}\sum_{n=0}^{1-a_{ij}} (-1)^n \begin{bmatrix}1-a_{ij} \\ n \end{bmatrix}_{q_i} F_j \curvearrowright w_{1- a_{ij} -n, n}(F_i\stackrel{\ast}{,}F_i) =\begin{cases} 0 & \text{if}\ a_{ij}=0,\\ -\displaystyle\frac{q_i d_{ij} + q_i^{-1}\widetilde{d_{ij}}}{\left(q_i-q_i^{-1}\right)\left(q_j-q_j^{-1}\right)}& \text{if}\ a_{ij}=-1,\\ [2]_{q_i}\displaystyle\frac{q_id_{ij} - q_i^{-1}\widetilde{d_{ij}}}{q_j-q_j^{-1}} F_i & \mbox{if}\ a_{ij}=-2. \end{cases} \end{align*}$$

Under the identification ${\mathcal B}_{\mathbf {c}}\cong ({\mathcal A},\ast )$, these relations coincide with the relations given in [Reference KolbKol14, Theorem 7.8] in the reformulation given in [Reference Balagović and KolbBK15, Theorem 3.9]. Note that ${\mathcal Z}_i$ in the present paper coincides with $-q_ic_i{\mathcal Z}_i$ in [Reference Balagović and KolbBK15]. For $a_{ij}=-3$, we obtain the new relation

$$ \begin{align*} & \sum_{n=0}^{1-a_{ij}} (-1)^n \begin{bmatrix}1-a_{ij} \\ n \end{bmatrix}_{q_i} F_j \curvearrowright w_{1- a_{ij} -n, n}(F_i\stackrel{\ast}{,}F_i)\\ &\quad = - [2]_{q_i} \frac{q_i^3-q_i^{-3}}{q_j-q_j^{-1}} \left(d_{ij}+\widetilde{d_{ij}}\right) F_i^{\ast 2} - \frac{q_i^3-q_i^{-3}}{\left(q_i-q_i^{-1}\right)^2\left(q_j-q_j^{-1}\right)} \left(q_i^3d_{ij}+q_i^{-3}\widetilde{d_{ij}}\right) {\mathcal Z}_i. \end{align*} $$

In the special case where $a_{ji}=-1$ with $q_i=q$ and $q_j=q^3$, this formula reproduces the formula given in [Reference Regelskis and VlaarRV20, (4.19)].

5.8 The quantum Serre relation for $\tau (i)=j$ where $i,j\in I\setminus X$

All through this subsection we assume that $i,j\in I\setminus X$ with $\tau (i)=j\neq i$. In this case, equation (3.1) implies that

(5.35)$$ \begin{align} F_i\ast u = F_i u - c_i\frac{q^{\left(\alpha_i,w_X\left(\alpha_j\right)\right)}}{q_i-q_i^{-1}} K_{w_X\left(\alpha_j\right)-\alpha_i} \partial_{j,X}^L(u) \end{align} $$

for all $u\in {\mathcal R}_X$. Hence for any $n\in {\mathbb N}$, we have $F_i^{\ast n}=F_i^n$ and

(5.36)$$ \begin{align} F_j\ast F_i^n &\stackrel{\phantom{\text{(3.6)}}}{=}F_j F_i^n - c_j\frac{q^{\left(\alpha_j,w_X\left(\alpha_i\right)\right)}}{q_i-q_i^{-1}} K_{w_X\left(\alpha_i\right)-\alpha_j} \partial_{i,X}^L\left(F_i^n\right)\nonumber \\ &\stackrel{\text{(3.6)}}{=} F_j F_i^n - c_j\frac{q^{\left(\alpha_i,\alpha_j\right)}}{q_i-q_i^{-1}} (n)_{q_i^2}K_i K_j^{-1} Z_i F_i^{n-1}\nonumber\\ &\stackrel{\phantom{\text{(3.6)}}}{=}F_j F_i^n -c_j\frac{q_i^{na_{ij}-2n+2}}{q_i-q_i^{-1}} (n)_{q_i^2} F_i^{n-1} K_i K_j^{-1}Z_i. \end{align} $$

Moreover, one shows by induction on m, using equations (5.35), (3.2) and (3.4), that

(5.37)$$ \begin{align} F_i^{\ast m} \ast F_j F_i^n = F_i^m F_j F_i^n - c_i \frac{q_i^{2n-(n-1)a_{ij}}}{q_i-q_i^{-1}} (m)_{q_i^2}F_i^{m+n-1} K_j K_i^{-1} Z_j. \end{align} $$

Inserting equation (5.36) into equation (5.37), we obtain

(5.38)$$ \begin{align} F_i^m F_j F_i^n &= F_i^{\ast m}\ast F_j \ast F_i^{\ast n} + c_i \frac{q_i^{2n-(n-1)a_{ij}}}{q_i-q_i^{-1}} (m)_{q_i^2} F_i^{m+n-1} K_j K_i^{-1} Z_j \nonumber\\ & \quad + c_j\frac{q_i^{na_{ij}-2n+2}}{q_i-q_i^{-1}} (n)_{q_i^2} F_i^{m+n-1} K_i K_j^{-1}Z_i. \end{align} $$

Using the relations

$$ \begin{align*} \sum_{n=0}^\ell (-1)^n \begin{bmatrix}\ell \\ n \end{bmatrix}_{q} q^{n(\ell + 1)} =\left(q^2;q^2\right)_\ell, \qquad \sum_{n=0}^\ell (-1)^n \begin{bmatrix}\ell \\ n \end{bmatrix}_{q} q^{n(\ell-1)}=0, \end{align*} $$

which hold for all $\ell \in {\mathbb N}$, one shows that

(5.39)$$ \begin{align} \sum_{n=0}^{1-a_{ij}} (-1)^n \begin{bmatrix}1-a_{ij} \\ n \end{bmatrix}_{q_i} q_i^{n\left(2-a_{ij}\right)}\left(1-a_{ij}-n\right)_{q_i^2} &=-\frac{q_i^{-1}\left(q_i^{2};q_i^{2}\right)_{1-a_{ij}}}{q_i-q_i^{-1}}, \end{align} $$

(5.40)$$ \begin{align} \sum_{n=0}^{1-a_{ij}} (-1)^n \begin{bmatrix}1-a_{ij} \\ n \end{bmatrix}_{q_i} q_i^{n\left(a_{ij}-2\right)}(n)_{q_i^2} &=-\frac{q_i^{-1}\left(q_i^{-2};q_i^{-2}\right)_{1-a_{ij}}}{q_i-q_i^{-1}}. \end{align} $$

Using equations (5.38), (5.39) and (5.40), we can now rewrite the quantum Serre relation $S_{ij}\left (F_i,F_j\right )=0$ in terms of the star product $\ast $ on ${\mathcal A}$. One obtains the following result:

Theorem 5.13. Let $i,j\in I\setminus X$ with $\tau (i)=j\neq i$ and set $N=1-a_{ij}$. Then the relation

$$ \begin{align*} & \sum_{n=0}^{N}(-1)^n\begin{bmatrix}N \\ n \end{bmatrix}_{q_i} F_i^{\ast (N-n)}\ast F_j \ast F_i^{\ast n}\\ &\quad =\frac{c_i q_i^{-N} \left(q_i^{2};q_i^{2}\right)_{N}}{\left(q_i-q_i^{-1}\right)^2} F_i^{\ast(N-1)} K_j K_i^{-1}Z_j + \frac{c_jq_i \left(q_i^{-2};q_i^{-2}\right)_{N}}{\left(q_i-q_i^{-1}\right)^2}F_i^{\ast(N-1)} K_i K_j^{-1} Z_i \end{align*} $$

holds in the algebra $({\mathcal A},\ast )$.

With the relation ${\mathcal Z}_i=c_iq_iZ_j$, this theorem turns into case (III) of Theorem 1.2 under the isomorphism $\psi :{\mathcal B}_{\mathbf {c}}\rightarrow ({\mathcal A},\ast )$.