Proof. For simplicity, assume $\beta =1$ . Then $ \widehat c_{\lambda ,(n)}^\nu $ is the coefficient of $G\hspace {-0.2mm}Q_{\lambda }(\mathbf {x})G\hspace {-0.2mm}P_{(n)}(\mathbf {y})$ in $G\hspace {-0.2mm}Q_\nu (\mathbf {x}, \mathbf {y})$ . Since $G\hspace {-0.2mm}P_\mu (y_1,0,0,\dots )$ is zero if $\ell (\mu ) \geq 2$ and equal to $y_1^{\mu _1}$ if $\ell (\mu ) \leq 1$ , the number $ \widehat c_{\lambda ,(n)}^\nu $ is also the coefficient of $G\hspace {-0.2mm}Q_{\lambda }(\mathbf {x}) t^n$ in the power series $G\hspace {-0.2mm}Q_\nu (\mathbf {x};t) $ formed by setting $y_1= t$ and $y_2=y_3=\dots =0$ in $G\hspace {-0.2mm}Q_\nu (\mathbf {x}, \mathbf {y})$ . As $G\hspace {-0.2mm}Q_\nu = \sum _{T \in \mathsf {ShSVT}_Q(\nu )} \mathbf {x}^{T}$ is symmetric, it holds that

$$\begin{align*}\begin{aligned} G\hspace{-0.2mm}Q_\nu(\mathbf{x};t) & = \sum_{\lambda \subseteq \nu} \sum_{T \in \mathsf{ShRibbons}_P(\nu/\hspace{-1mm}/\lambda)} G\hspace{-0.2mm}Q_\lambda(\mathbf{x}) t^{|T|} = \sum_{n\geq 0} \sum_{\lambda \subseteq \nu} | \mathsf{ShRibbons}_Q(\nu/\hspace{-1mm}/\lambda;n)| G\hspace{-0.2mm}Q_\lambda(\mathbf{x}) t^{n},\end{aligned} \end{align*}$$

and so we have $ \widehat c_{\lambda ,(n)}^\nu =|\mathsf {ShRibbons}_Q(\nu /\hspace {-1mm}/\lambda; n)|.$ The argument to show that we likewise have $\widehat b_{\lambda ,(n)}^\nu =|\mathsf {ShRibbons}_P(\nu /\hspace {-1mm}/\lambda; n)|$ is similar, as $\widehat b_{\lambda ,(n)}^\nu $ is the coefficient of $G\hspace {-0.2mm}P_{\lambda }(\mathbf {x}) t^n$ in $G\hspace {-0.2mm}P_\nu (\mathbf {x};t) $ .

Proof. Let T be a semistandard ShBT. Then $\mathsf {swap}_{1\leftrightarrow 2'}(T)$ has the same shape and weight as T by construction. Since the operation $\mathsf {swap}_{1\leftrightarrow 2'}$ affects only the boxes in T filled with $1$ or $2'$ , the result is valid for all semistandard T filled with $\{1', 1, 2', 2\}$ if and only if it is valid for those semistandard T filled with only $\{1, 2'\}$ . So without loss of generality, assume that the only entries of T are $1$ and $2'$ . Such a tableau T contains no $2 \times 2$ square of entries, as there is no way to complete the diagram to a semistandard ShBT. In particular, the only way that T can contain one box strictly northeast of another is if it contains two consecutive boxes on the diagonal:

Thus, the partial order $\preceq $ on the bars in T where one bar is smaller than (‘before’) another if the first lies entirely (weakly) southeast of the latter is either a total order (if T does not contain two diagonal boxes) or has a unique incomparable pair that come after all other bars of T. This order is compatible with the orders ${\mathcal {H}}_1, \ldots , {\mathcal {H}}_k$ and ${\mathcal {V}}_1, \ldots , {\mathcal {V}}_l$ in Definition 3.3. Moreover, this order extends to every $\{1, 2'\}$ -filled ShBT (semistandard, sorted or otherwise) with the same shape as T, and to every tableau appearing as an intermediate stage in the computation of $\mathsf {swap}_{1\leftrightarrow 2'}(T)$ .

Suppose the bars of $1$ s in T are ${\mathcal {H}}_1, \ldots , {\mathcal {H}}_k$ and the bars of $2'$ s in T are ${\mathcal {V}}_1, \ldots , {\mathcal {V}}_l$ , as in Definition 3.3. Applying any number of $\mathsf {swap}_{1\leftrightarrow 2'}$ operations to T results in a tableau with k bars of $1$ s and l bars of $2'$ s. For $i = 1, \ldots , l$ , let $ T^{(i)} := \mathsf {swap}_{1\leftrightarrow 2'}( \cdots \mathsf {swap}_{1\leftrightarrow 2'}(T, {\mathcal {V}}_1) \cdots , {\mathcal {V}}_i), $ and let the bars of $1$ s and $2'$ s in $T^{(i)}$ be, respectively, ${\mathcal {H}}^{(i)}_1, \ldots , {\mathcal {H}}^{(i)}_k$ and ${\mathcal {V}}^{(i)}_1, \ldots , {\mathcal {V}}^{(i)}_l$ .

Given a tableau, we will say that a pair of bars A, B is bad if A is filled with $1$ , B is filled with $2'$ , and A contains a box either directly west or directly south of a box in B as in these pictures:

Each move in (a) and (b) of Definition 3.2 replaces an adjacent pair of bad bars with a non-bad pair. We claim that in the tableau $T^{(i)}$ , there are no bad pairs involving any of the bars ${\mathcal {V}}_1^{(i)}, \ldots , {\mathcal {V}}_i^{(i)}$ , except possibly in the case that T has two diagonal boxes and $i = l$ .

For $i = 0$ , this is vacuously true; we proceed by induction. By hypothesis, there are no bad pairs involving ${\mathcal {V}}_1^{(i - 1)}, \ldots , {\mathcal {V}}_i^{(i - 1)}$ in $T^{(i)}$ . Suppose that ${\mathcal {V}}_i = {\mathcal {V}}_i^{(i - 1)}$ lies in column c, and choose j such that ${\mathcal {H}}_1^{(i - 1)}, \ldots , {\mathcal {H}}_j^{(i - 1)}$ lie in columns whose indices are greater than or equal to c (i.e., they lie weakly southeast of ${\mathcal {V}}_i$ in $T^{(i - 1)}$ ), while ${\mathcal {H}}_{j + 1} = {\mathcal {H}}_{j + 1}^{(i - 1)}$ , …, ${\mathcal {H}}_{k} = {\mathcal {H}}_{k}^{(i - 1)}$ lie in columns whose indices are strictly smaller than c. In this setup, it is possible that a single bar of $1$ s lies in columns with indices both smaller than and greater than or equal to c, but only if $i = l$ and ${\mathcal {V}}_i$ consists of a single diagonal box, as in .

The first $j - 1$ swaps in the computation of $\mathsf {swap}_{1\leftrightarrow 2'}(T^{(i - 1)}, {\mathcal {V}}_i)$ do nothing. The jth swap either does nothing (if ${\mathcal {H}}_j^{(i - 1)}$ does not contain any boxes in column c) or performs a swap of the type described in Definition 3.2(a). Following this swap, the resulting two bars $\tilde {{\mathcal {V}}}_i, \tilde {{\mathcal {H}}}_j$ no longer form a bad pair. Moreover, although the bar $\tilde {{\mathcal {V}}}_i$ may extend one row below ${\mathcal {V}}_i$ , this cannot introduce any new bad pairs involving the bar $\tilde {{\mathcal {V}}}_i$ unless there is a $1$ to the left of the first box in ${\mathcal {H}}_j^{(i - 1)}$ , which only happens if $i = l$ and ${\mathcal {V}}_i$ consists of a single diagonal box, as in these examples:

If, at this stage, there are any further bad pairs involving $\tilde {{\mathcal {V}}}_i$ , they must come from $1$ s in boxes directly west of a box in $\tilde {{\mathcal {V}}}_i$ . This can happen if $i = l$ and $\tilde {{\mathcal {V}}}_i$ contains a diagonal box , but otherwise the bar ${\mathcal {H}}_{k + 1}$ of $1$ s in the bad pair must be directly west of and adjacent to the last box in $\tilde {{\mathcal {V}}}_i$ . If $\tilde {{\mathcal {V}}}_i$ has more than one box, then the immediate next swap (of the type in Definition 3.2(b)) removes the box from $\tilde {{\mathcal {V}}}_i$ that is involved in the bad pair, and the remaining swaps (with ${\mathcal {H}}_{j + 2}$ , …) do nothing, leaving a tableau in which the new bar ${\mathcal {V}}^{(i)}_i$ is not involved in any bad pairs. If $\tilde {{\mathcal {V}}}_i$ only has a single box, then possibly several moves of type Definition 3.2(b) are required; after these moves, the box occupied by ${\mathcal {V}}^{(i)}_i$ was previously occupied by a box labeled $1$ , so there cannot be any other $1$ s in the same column, and so also in this case, ${\mathcal {V}}^{(i)}_i$ is not involved in any bad pairs.

Finally, we observe that none of these moves changes any bar of $1$ s that is east of ${\mathcal {V}}^{(i - 1)}_{i - 1}$ (because they are all strictly southeast of ${\mathcal {V}}_i$ ) and none of these moves causes any $1$ s to appear in a lower row than they previously appeared, so no bad pairs involving ${\mathcal {V}}^{(i - 1)}_1 = {\mathcal {V}}^{(i)}_1$ , …, ${\mathcal {V}}^{(i - 1)}_{i - 1} = {\mathcal {V}}^{(i)}_{i - 1}$ are created. This completes the proof of the claim.

It follows from the preceding argument that either $\mathsf {swap}_{1\leftrightarrow 2'}(T)$ has no bad pairs at all or its shape has two diagonal boxes and there is a bad pair involving the bar of $2'$ s in the upper diagonal box. By comparing with the definition of sorted tableaux (see Definition 3.1), we find that this is precisely the desired conclusion.

Proof. Let $\tilde T$ be a sorted shifted bar tableau. By construction, $\tilde T$ and $\mathsf {unswap}_{1\leftrightarrow 2'}(\tilde T)$ have the same shape and same weight. First, suppose that $\tilde T$ does not contain two diagonal boxes. In this case, by comparing the definitions of sorted and semistandard ShBT, we see that rotating $\tilde T$ by $180^\circ $ produces a semistandard ShBT $\tilde T^r$ . Moreover, by comparing Definitions 3.2, 3.3 with Definitions 3.6, 3.7, we see that $\mathsf {swap}_{1\leftrightarrow 2'}(\tilde T^r) = \mathsf {unswap}_{1\leftrightarrow 2'}(\tilde T)^r$ , and consequently that $\mathsf {unswap}_{1\leftrightarrow 2'}(\tilde T) = \mathsf {swap}_{1\leftrightarrow 2'}(\tilde T^r)^r$ is a semistandard ShBT.

Otherwise, suppose $\tilde T$ has two diagonal boxes. There are four possible arrangements of the entries the first two diagonals of $\tilde T$ , which we illustrate as follows; in all cases, $\tilde {{\mathcal {V}}}_l$ is colored in pink and $\tilde {{\mathcal {H}}}_k$ is colored in cyan:

• (1)

• (2)

• (3)

• (4)

In the first two cases, one has $\mathsf {unswap}_{1\leftrightarrow 2'}(\tilde T, \tilde {{\mathcal {V}}}_l) = \tilde T$ . In these cases, deleting the unique box of $\tilde {{\mathcal {V}}}_l$ from $\tilde T$ produces a sorted tableau $\tilde T - \tilde {{\mathcal {V}}}_l$ with only one diagonal box. Therefore, by the argument in the previous paragraph, $\mathsf {unswap}_{1\leftrightarrow 2'}(\tilde T - \tilde {{\mathcal {V}}}_l)$ is a semistandard tableau. This tableau becomes $\mathsf {unswap}_{1\leftrightarrow 2'}(\tilde T) = \mathsf {unswap}_{1\leftrightarrow 2'}(\tilde T - \tilde {{\mathcal {V}}}_l) \cup \{\tilde {{\mathcal {V}}}_l\}$ when we add back the unique box of $\tilde {{\mathcal {V}}}_l$ at the end. Since the added box always contains $2'$ , the result is semistandard, as needed.

In the latter two cases, one has $\mathsf {unswap}_{1\leftrightarrow 2'}(\tilde T, \tilde {{\mathcal {V}}}_l, \tilde {{\mathcal {H}}}_k) = (\tilde T, \tilde {{\mathcal {V}}}_l)$ (i.e., the very first swap belongs to case (c) of Definition 3.6 and so does not change anything). In these cases, deleting the unique box of $\tilde {{\mathcal {H}}}_k$ produces a sorted tableau $\tilde T - \tilde {{\mathcal {H}}}_k$ with only one diagonal box. Again by the argument in the first paragraph of this proof, we deduce that $\mathsf {unswap}_{1\leftrightarrow 2'}(\tilde T - \tilde {{\mathcal {H}}}_k)$ is a semistandard tableau. This tableau becomes $\mathsf {unswap}_{1\leftrightarrow 2'}(\tilde T) = \mathsf {unswap}_{1\leftrightarrow 2'}(\tilde T - \tilde {{\mathcal {H}}}_k) \cup \{\tilde {{\mathcal {H}}}_k\}$ when we add back the unique box of $\tilde {{\mathcal {H}}}_k$ at the end, which preserves semistandard-ness. This completes the proof.

Proof. By Propositions 3.5 and 3.9, the two maps are shape- and weight-preserving and they have the correct domain and codomain. Since each swap preserves the relative orders of the ${\mathcal {V}}_i$ and of the ${\mathcal {H}}_j$ , it is immediate from the definitions that $\mathsf {unswap}_{1\leftrightarrow 2'}(\mathsf {swap}_{1\leftrightarrow 2'}(T)) = T$ for any semistandard ShBT T and $\mathsf {swap}_{1\leftrightarrow 2'}(\mathsf {unswap}_{1\leftrightarrow 2'}(\tilde T)) = \tilde T$ for any sorted ShBT $\tilde T$ .

Proof. First, consider the equivalence classes that consist of all unprimed bars. In this case, if $\mathcal {A} \approx \mathcal {B}$ , then the relations satisfied by the entries of a sorted ShBT imply that when the bar $\mathcal {A}$ is filled with $1$ and lies in row i, the bar $\mathcal {B}$ must be filled with $2$ s and lie in row $i + 1$ , and conversely. Hence, the entire equivalence class is restricted to two consecutive rows i and $i + 1$ , with row i containing $1$ s and row $i + 1$ containing $2$ s. Furthermore, row $i + 1$ cannot extend to the right of row i because the only way to complete the diagram

to a valid sorted ShBT is if the box is filled with a $1$ . Similarly, row i cannot extend to the left of row $i + 1$ except in the case

where it does so at the diagonal. The case of equivalence classes containing all primed bars is the conjugate of this case.

We now consider the case of equivalence classes that contain both primed and unprimed bars. In this case, the exceptional rule defining $\approx $ must come into play: the tableau T must have two diagonal boxes $(i, i)$ and $(i + 1, i + 1)$ for some i, at least one of which must be a bar unto itself. Any skew shifted shape that contains $(i, i)$ and $(i + 1, i + 1)$ also contains $(i, i + 1)$ ; up to conjugacy, we may assume that the box $(i, i + 1)$ is filled with an unprimed entry. In fact, this unprimed entry must be $1$ : no tableau of the form with entries in $\{1', 2', 1, 2\}$ is a sorted ShBT. Let $\mathcal {A}$ be the bar containing the box $(i, i + 1)$ . By hypothesis, one of the two boxes $(i, i)$ and $(i + 1, i + 1)$ must belong to a one-box bar that is filled with either $1'$ or $2'$ .

First, suppose that the box $(i, i)$ is a singleton bar $\mathcal {B}$ . In this case, its only relation under $\approx $ is $\mathcal {A} \approx \mathcal {B}$ , and the box $(i + 1, i + 1)$ could either be filled with $2'$ (if $\mathcal {B}$ is also filled with $2'$ ) or with $2$ . In the case that $(i + 1, i + 1)$ is filled with $2'$ , its unique relation under $\approx $ is with $\mathcal {A}$ , and so in either case, the logic of the previous paragraph applies to conclude that all boxes in the equivalence class belong to rows i and $i + 1$ .

Second, suppose that box $(i + 1, i + 1)$ is filled with a primed entry. By Definition 3.1, it follows that this entry is $2'$ and that $(i, i)$ is either filled with $1$ or with $2'$ . In the former case, the equivalence class again belongs to two rows, for the same reasons. In the latter case, it turns out that the box $(i, i)$ must also be a bar unto itself because it is not possible to complete the diagram

to a sorted ShBT. This establishes all of our claims about two-row-groups. The case of two-column-groups is equivalent after taking conjugates everywhere.

Proof. The case of equivalence classes that do not contain any diagonal box is straightforward from the definition. If a tableau T contains two diagonal boxes $(i, i)$ and $(i + 1, i + 1)$ , then at least one of them belongs to a two-row- or two-column-group with $(i, i + 1)$ and so does not belong to any one-row- or one-column-group. Thus, we may restrict our attention to the case of tableaux with a single diagonal box, and the equivalence class that contains that box. In particular, we must show that the diagonal box cannot be in the same group with its neighbors to the east and south simultaneously. It is a simple finite computation to check that in all $30$ sorted ShBT of shape

at least one of the neighbors of the diagonal box belongs to a two-row- or two-column-group. Then the result follows immediately.

Proof. By construction, every bar in a sorted ShBT belongs to a group, and each local transformation (toggling the labels on one-row- and one-column-groups, toggling the bar divisions on two-row- and two-column-groups that contain at most one diagonal box, applying rules (R1)–(R6) and (C1)–(C6) to two-row- and two-column-groups that contain two diagonal boxes) reverses the weight and preserves the shape of the group on which it acts, so the map $\mathsf {reverse\_weight}$ is weight-reversing and shape-preserving. It remains to show first that $\mathsf {reverse\_weight}$ is well defined (i.e., that for any sorted ShBT T, the image $\mathsf {reverse\_weight}(T)$ is also a sorted ShBT), and second that the groups of $\mathsf {reverse\_weight}(T)$ are supported on the same boxes as the groups of T. Since each local transformation is an involution by definition, this will establish that $\mathsf {reverse\_weight}$ is an involution.

Given any group in T, its image under the appropriate local transformation is compatible with the definition of sorted tableaux. Thus, to check that $\mathsf {reverse\_weight}$ is well defined as a map from sorted ShBT to sorted ShBT, we must establish that (1) adjacent boxes in different groups satisfy the correct order relation, and (2) if there are two diagonal boxes, then they and the box between them are validly filled.

We begin with some basic observations. All of the local transformations have the property that if a nondiagonal box $(i, j)$ in a sorted ShBT T is filled with a primed (respectively, unprimed) entry, then the box $(i, j)$ in $\mathsf {reverse\_weight}(T)$ is also filled with a primed (respectively, unprimed) entry. Moreover, every nondiagonal box of a one- or two-column-group is filled with a primed entry, and every nondiagonal box of a one- or two-row-group is filled with an unprimed entry. Therefore, to prove that $\mathsf {reverse\_weight}(T)$ is a sorted ShBT, we need only check conditions at the diagonal and between groups of the same ‘type’ (both row or both column). We handle the second case first.

In the case of two-row-groups that do not contain two diagonal boxes, each box is filled with the same number after toggling the bar divisions as it was before (only the division of these boxes into bars changes), and the new bars are still connected under $\approx $ (a division between bars that fails to disconnect the group still fails to disconnect the group when it switches rows). Thus, the operation $\mathsf {reverse\_weight}$ preserves the order relation between groups of this type.

Two one-row-groups cannot have any adjacent boxes: if a box in one were directly below a box in the other, then these two boxes would belong to a two-row-group (and so, by definition, could not belong to one-row-groups), while if a box in one were directly to the right of a box in the other, then these two boxes would belong to the same one-row-group.

Finally, suppose that $\mathcal {A}$ is a two-row-group occupying rows i and $i + 1$ and $\mathcal {B}$ is a one-row-group that contains a box adjacent to a box in $\mathcal {A}$ . These adjacent boxes must lie in the same row (rather than the same column) because otherwise the box in the one-row-group would belong instead to a two-row-group. It follows as in the proof of Proposition 3.12 that if the boxes in the one-row-group $\mathcal {B}$ are to the right of the boxes in $\mathcal {A}$ , then it must lie in row i, because there is no way to complete

to a sorted ShBT. Similarly, if the boxes in $\mathcal {B}$ lie to the left of the boxes in $\mathcal {A}$ , then they must lie in row $i + 1$ . (In the latter case, there is an additional consideration concerning the situation in which the one-row-block consists of a single diagonal box, but in this case, the putative one-row-block would actually be part of the two-row-block.) In the first situation, after applying $\mathsf {reverse\_weight}$ , the box in the two-row-group is still filled with $1$ and still to the left of the box in the one-row-group (which will either end up filled with $1$ or $2$ ), and in the second case, the box in the two-row-group is still filled with $2$ and still to the right of the box in the one-row-group. Thus, the order relations are preserved in both cases. The situation for one- and two-column-groups is precisely the conjugate. This completes the proof that $\mathsf {reverse\_weight}(T)$ obeys the order relations of a sorted ShBT among all pairs of adjacent boxes except possibly those at the diagonal (in the case that T contains two diagonal boxes).

Now consider the collection of sorted ShBT with two diagonal boxes in positions $(i, i)$ and $(i + 1, i + 1)$ . There are twelve possible fillings of the three boxes $(i, i)$ , $(i, i + 1)$ , $(i + 1, i + 1)$ : six of them are shown in the statement of Proposition 3.12, and the other six are conjugate to these. In the cases

and their conjugates, it is easy to see that the three boxes must belong to a single two-row-group. (The nontrivial cases are the first and fourth; the proof of Proposition 3.12 shows explicitly that in the box $(i, i)$ must be a one-bar group in the fourth case, and the proof in the first case is identical.) The boxes directly below this group (if there are any) can only be filled with primed entries; after applying $\mathsf {reverse\_weight}$ (checking the six cases (R1)–(R6)), those boxes will still be filled with primed entries and so the necessary order relations are respected. Similarly, the boxes directly to the right of the group (if there are any) will be filled with unprimed entries, and any box in row $i + 1$ will be part of a two-row-group and filled with $2$ ; after applying $\mathsf {reverse\_weight}$ , we again have that all order relations are respected. The same holds for the conjugates of these cases after swapping the words ‘row’ with ‘column’, ‘primed’ with ‘unprimed’, ‘below’ with ‘right’, and ‘ $(i, i)$ ’ with ‘ $(i + 1, i + 1)$ ’. This leaves the two mutually conjugate cases

Here, we have two possibilities to consider: box $(i, i + 1)$ is automatically in the same two-row- or two-column-group as one of the other diagonal boxes, but the odd box out could be part of a bar with more than one box. For example, in the tableau the unprimed bars are part of a two-row-group and the primed bars are part of a two-column-group. If the two diagonal boxes are in different groups, then the three boxes are unchanged by the operation $\mathsf {reverse\_weight}$ , all boxes below them (if any) are primed and stay this way after applying $\mathsf {reverse\_weight}$ , and all boxes to their right (if any) are unprimed and stay this way after applying $\mathsf {reverse\_weight}$ , so all order relations involving these boxes are preserved. If the two diagonal boxes are part of the same group, and when we apply $\mathsf {reverse\_weight}$ , we fall into case (R4) or (C4), and the rest of the argument is essentially the same as for the cases discussed above.

Proof. Let T be a semistandard shifted bar tableau. In view of Theorems 3.10 and 3.15, it remains only to prove our claim that $\tau ^{\mathrm {ShBT}}(T)$ and T have the same number of primed diagonal entries.

By considering the various cases in Definition 3.2, we see that the only swaps in the initial application of $\mathsf {swap}_{1\leftrightarrow 2'}$ that can change a diagonal box are moves of the form

(3.8)

or its conjugate, where the leftmost (respectively, topmost) affected box belongs to the diagonal. In either case, if such a swap is applied, then it will be the last swap in the computation of $\mathsf {swap}_{1\leftrightarrow 2'}(T)$ that changes anything (i.e., any remaining swaps will fall into case (c) of Definition 3.2). Similarly, the only swaps in the final application of $\mathsf {unswap}_{1\leftrightarrow 2'}$ that can change a diagonal box are moves that are the reverse of (3.8) or its conjugate, and when such swaps are applied, they occur as the first swap in the computation when $\mathsf {unswap}_{1\leftrightarrow 2'}$ is applied to $\mathsf {reverse\_weight}(\mathsf {swap}_{1\leftrightarrow 2'}(T))$ . We now consider exhaustively the possible changes that can occur at the diagonal.

First, suppose T contains a single diagonal box $(i,i)$ , and assume that this box of T is filled with $1'$ or $2$ . Then $\mathsf {swap}_{1\leftrightarrow 2'}(T)$ also possesses a single diagonal box filled with the same entry. In the sorted ShBT $\mathsf {swap}_{1\leftrightarrow 2'}(T)$ , this box belongs either to a one-row- or one-column-group, or to a two-row- or two-column-group that contains exactly one diagonal box. On groups of this type, the operation $\mathsf {reverse\_weight}$ preserves the entry in the diagonal box with a unique exception – namely, if the diagonal box belongs to a one-row- or one-column-group whose entries are all equal (as in the last figure in (3.7)).

In the non-exceptional case, $\mathsf {reverse\_weight}(\mathsf {swap}_{1\leftrightarrow 2'}(T))$ has its unique diagonal box $(i, i)$ filled with the same value ( $1'$ or $2$ ) as T. Applying $\mathsf {unswap}_{1\leftrightarrow 2'}$ preserves this property since it only affects boxes containing $1$ and $2'$ , so $\tau ^{\mathrm {ShBT}}(T)$ also has its diagonal box filled with the same value as T.

Assume we are in the exceptional case, so that the unique diagonal box $(i, i)$ of $\mathsf {swap}_{1\leftrightarrow 2'}(T)$ is part of a one-row- or one-column-group whose entries are all equal. Without loss of generality (up to conjugation) we can assume that it is a one-row-group whose entries are all $2$ , and so that the diagonal box in $\mathsf {reverse\_weight}(\mathsf {swap}_{1\leftrightarrow 2'}(T))$ is filled with $1$ . Possibly the box $(i - 1, i)$ directly below the diagonal is empty (i.e., it does not belong to T). Then the very first step of the application of $\mathsf {unswap}_{1\leftrightarrow 2'}$ cannot be a move of the form

(3.9)

and hence, the resulting tableau $\tau ^{\mathrm {ShBT}}(T)$ still has unprimed diagonal entry.

If, however, the box $(i - 1, i)$ is not empty, then in $\mathsf {swap}_{1\leftrightarrow 2'}(T)$ it cannot be filled with $2$ (because $\mathsf {swap}_{1\leftrightarrow 2'}(T)$ is sorted), and it cannot be filled with $1$ (because $(i, i)$ does not belong to a two-row-group), so it must be filled with a primed entry. Furthermore, we claim that $(i - 1, i)$ cannot be filled with $2'$ : if box $(i-1,i)$ had $2'$ and the bar of $(i,i)$ had multiple boxes, then $(i-1,i+1)$ would be filled with $1$ to be sorted and $(i,i)$ would be in a two-row-group, whereas if box $(i-1,i)$ had $2'$ and the bar of $(i,i)$ had just one box, then $(i-1,i)$ would part of a one-column-group that includes $(i,i)$ , which is a contradiction in either case.

Thus, when the box $(i - 1, i)$ belongs to T, it must be filled with $1'$ in $\mathsf {swap}_{1\leftrightarrow 2'}(T)$ , and this $1'$ must be part of a two-column group. After applying $\mathsf {reverse\_weight}$ to $\mathsf {swap}_{1\leftrightarrow 2'}(T)$ , it follows that the diagonal box will be filled with the value $1$ and the box $(i - 1, i)$ will be filled with $1'$ . This means that applying $\mathsf {unswap}_{1\leftrightarrow 2'}$ to $\mathsf {reverse\_weight}(\mathsf {unswap}_{1\leftrightarrow 2'}(T))$ cannot begin with a move of the form (3.9), and hence, the resulting tableau $\tau ^{\mathrm {ShBT}}(T)$ still has unprimed diagonal entry. This completes the claim in the case that T has one diagonal box and it is filled with $1'$ or $2$ .

Continue to assume that T has a single diagonal box $(i,i)$ , but now suppose this box is filled with $1$ or $2'$ . Up to conjugacy, we may assume that the box is filled with $1$ , so that T looks like

where possibly the bar containing the diagonal box extends to the right. Our objective is to show that the diagonal box $(i,i)$ is still unprimed in $\tau ^{\mathrm {ShBT}}(T)$ . To this end, we first observe that the bar tableau $\mathsf {swap}_{1\leftrightarrow 2'}(T)$ must look like

(3.10)

where again ‘the bar of $1$ s’ may extend to the right. Since $(i, i)$ is the unique diagonal box of T, it must be the case that T does not contain any boxes in row $i + 1$ , and consequently in all four cases ‘the bar of $1$ s’ in $\mathsf {swap}_{1\leftrightarrow 2'}(T)$ is not related by $\approx $ to any other box. In the second, third and fourth cases of (3.10), it follows that this bar is in a one-row-group; in the first case, it might be part of either a one-row- or one-column-group, depending on the presence and contents of the cells marked ‘ $??$ ’ in $\mathsf {swap}_{1\leftrightarrow 2'}(T)$ . We consider the cases in order.

In the first case, suppose first that the diagonal box $(i, i)$ is part of a one-row-group in $\mathsf {swap}_{1\leftrightarrow 2'}(T)$ . Then the bar containing $(i - 1, i)$ that is filled with $1'$ must belong to a two-column-group in $\mathsf {swap}_{1\leftrightarrow 2'}(T)$ (or else the two bars would be related by $\sim $ ). Therefore, after applying $\mathsf {reverse\_weight}$ , the diagonal box in $\mathsf {reverse\_weight}(\mathsf {swap}_{1\leftrightarrow 2'}(T))$ will be unprimed and the $(i - 1, i)$ box will still be filled with $1'$ . The operation $\mathsf {unswap}_{1\leftrightarrow 2'}$ can only convert an unprimed diagonal box into a primed diagonal box by an application of the operation

but this cannot happen when the box $(i - 1, i)$ is filled with $1'$ . Thus, the diagonal box in $\tau ^{\mathrm {ShBT}}(T)$ is unprimed in this case.

Continuing with the first case, suppose instead that the diagonal box $(i, i)$ is part of a one-column-group in $\mathsf {swap}_{1\leftrightarrow 2'}(T)$ . Since $(i - 1, i)$ is filled with $1'$ in $\mathsf {swap}_{1\leftrightarrow 2'}(T)$ , the entire one-column-group is filled with $1'$ and $1$ . Therefore, after toggling the labels of the group, the diagonal box in $\mathsf {reverse\_weight}(\mathsf {swap}_{1\leftrightarrow 2'}(T))$ will be filled with $2$ (and the rest of the one-column-group with $2'$ ). No step in the application of $\mathsf {unswap}_{1\leftrightarrow 2'}$ moves a box filled with $2$ ; hence, the diagonal box in $\tau ^{\mathrm {ShBT}}(T)$ is unprimed in this case.

In the second case, the diagonal box in $\mathsf {swap}_{1\leftrightarrow 2'}(T)$ can only belong to a one-row-group, so the diagonal box in $\mathsf {reverse\_weight}(\mathsf {swap}_{1\leftrightarrow 2'}(T))$ is unprimed. The operation $\mathsf {unswap}_{1\leftrightarrow 2'}$ can only convert an unprimed diagonal box into a primed diagonal box by an application of the operation

but this cannot happen when the box $(i - 1, i)$ is not part of the shape. Thus, the diagonal box in $\tau ^{\mathrm {ShBT}}(T)$ is also unprimed in this case.

In the third case, the diagonal one-box-bar filled with $2'$ belongs to the same one-row-group in $\mathsf {swap}_{1\leftrightarrow 2'}(T)$ as ‘the bar of $1$ s’. These two bars are filled the same in $\mathsf {reverse\_weight}(\mathsf {swap}_{1\leftrightarrow 2'}(T))$ as in $\mathsf {swap}_{1\leftrightarrow 2'}(T)$ since when we apply $\mathsf {reverse\_weight}$ , we are in the case of toggling the labels with $p, q> 0$ in the sense of (3.6). Then the very first step of $\mathsf {unswap}_{1\leftrightarrow 2'}$ applies the rule

so the diagonal box in $\tau ^{\mathrm {ShBT}}(T)$ is unprimed in this case.

Finally, in the fourth case, we have by sortedness that ‘the bar of $1$ s’ must be a singleton bar, and the box filled with ‘ $??$ ’ must actually be filled with $2'$ , so $\mathsf {swap}_{1\leftrightarrow 2'}(T)$ is of the form

In this case, the bars containing boxes $(i - 1, i)$ and $(i - 1, i + 1)$ in $\mathsf {swap}_{1\leftrightarrow 2'}(T)$ belong to a two-column-group, and so the one-box bars $(i, i)$ and $(i, i + 1)$ form a one-row-group in $\mathsf {swap}_{1\leftrightarrow 2'}(T)$ . It follows that $\mathsf {reverse\_weight}(\mathsf {swap}_{1\leftrightarrow 2'}(T))$ has the same entries in these four boxes. Then, as in the third case, the very first step of $\mathsf {unswap}_{1\leftrightarrow 2'}$ applies the rule

and so the diagonal box in $\tau ^{\mathrm {ShBT}}(T)$ is unprimed in this case.

Finally, we consider the case when T has two diagonal boxes. If both diagonal boxes are unprimed, then there are three ways that they and the box between them can be filled in T:

After applying $\mathsf {swap}_{1\leftrightarrow 2'}$ to such a tableau, we might end up with one of the near-diagonal arrangements

where in the last case, necessarily the diagonal box forms a one-box bar (following a move of the form (3.8)). In the first case, applying $\mathsf {reverse\_weight}$ toggles the bar divisions and so does not change the contents of the diagonal boxes or the box $(i, i + 1)$ , and applying $\mathsf {unswap}_{1\leftrightarrow 2'}$ to the result preserves the diagonal boxes. The second and third cases are precisely those to which the rule (R1) in the definition of $\mathsf {reverse\_weight}$ applies; they are interchanged, and then $\mathsf {unswap}_{1\leftrightarrow 2'}$ restores the unprimed diagonal boxes.

The argument when both diagonal boxes of T are primed is precisely the conjugate of the previous case. This leaves the situation when T has exactly two diagonal boxes, only one of which is primed. Then there are six possible arrangements for the two diagonal boxes and the box between them, in three conjugate pairs:

Suppose T belongs to one of the first two cases. Then $\mathsf {swap}_{1\leftrightarrow 2'}(T)$ still has the near-diagonal arrangement or (possibly reversed from what it had before); up to conjugacy, we assume the former. If the diagonal boxes $\mathsf {swap}_{1\leftrightarrow 2'}(T)$ belong to different groups, then after applying $\mathsf {reverse\_weight}$ , the near-diagonal arrangement is unchanged, and then $\mathsf {unswap}_{1\leftrightarrow 2'}$ does not affect the diagonal boxes. Suppose instead that the two diagonal boxes belong to the same group in $\mathsf {swap}_{1\leftrightarrow 2'}(T)$ . Depending on whether the bars containing boxes $(i, i + 1)$ and $(i + 1, i + 1)$ extend to the right, applying $\mathsf {reverse\_weight}$ to $\mathsf {swap}_{1\leftrightarrow 2'}(T)$ belongs to either case (R2) or (R4), and the image necessarily has the near-diagonal arrangement

Applying $\mathsf {unswap}_{1\leftrightarrow 2'}$ to such a tableau results in a tableau with near-diagonal arrangement

and so we conclude that $\tau ^{\mathrm {ShBT}}(T)$ has exactly one primed diagonal box, as needed.

The remaining four cases are, up to conjugacy, precisely the reverse direction of the cases just considered; since $\tau ^{\mathrm {ShBT}}$ is an involution, it follows that its action in these cases also preserves the number of primed diagonal boxes. This completes the proof.

Proof. The weight of a general ShBT is the tuple $a=(a_1,a_2,\dots )$ , where $a_i$ is the number of bars that contain either i or $i'$ . Let $|a| = a_1+a_2+\dots $ . The coefficient of $(-\beta )^{|\lambda /\mu |-|a|} x_1^{a_1} \cdots x_i^{a_i} x_{i + 1}^{a_{i + 1}} \cdots $ in $j\hspace {-0.1mm}q^{\mathrm {comb}}_{\lambda /\mu }$ is the number of semistandard ShBT’s of shape $\lambda /\mu $ with weight $a=(a_1,a_2,\dots )$ . By the first part of Theorem 3.16, applying the map $\tau ^{\mathrm {ShBT}}$ to the subtableaux generated by the entries $i', i, i' + 1, i + 1$ gives a bijection with the ShBT’s counted by the coefficient of $(-\beta )^{|\lambda /\mu |-|a|} x_1^{a_1} \cdots x_i^{a_{i + 1}} x_{i + 1}^{a_{i}} \cdots $ . Since any finitely supported permutation is a product of adjacent transpositions, it follows that $j\hspace {-0.1mm}q^{\mathrm {comb}}_{\lambda /\mu }$ is symmetric in the $x_i$ variables.

The second part of Theorem 3.16 ensures that the map $\tau ^{\mathrm {ShBT}}$ restricts to an involution on the subset $\mathsf {ShBT}_P(\lambda /\mu )\subseteq \mathsf {ShBT}_Q(\lambda /\mu )$ consisting of those ShBT’s with no primed diagonal entries. Thus, $j\hspace {-0.1mm}p^{\mathrm {comb}}_{\lambda /\mu }$ is symmetric by the same argument.

Proof. The following argument is similar to the proof of [Reference Lam and Pylyavskyy11, Lem. 9.12]. Checking that $ \rho \vartriangleright \tau $ , $ \rho \vartriangleleft \tau $ and $ \rho \circ \tau $ are shifted skew shapes is straightforward. Suppose $T \in \mathsf {ShBT}_Q(\rho )$ and $U \in \mathsf {ShBT}_Q(\tau )$ . Let $T \vartriangleright U $ and $T \vartriangleleft U$ be the shifted bar tableaux of shapes $\rho \vartriangleright \tau $ and $\rho \vartriangleleft \tau $ given by translating the boxes of T and U in the obvious way.

If $T_{ij} < U_{kl}$ or $T_{ij} = U_{kl}\in \mathbb {Z}$ , then we have $T \vartriangleright U \in \mathsf {ShBT}_Q(\rho \vartriangleright \tau )$ , $|T \vartriangleright U| = |T|+|U|$ , and $x^{T \vartriangleright U} = x^T x^U$ . However, if $T_{ij}> U_{kl}$ or $T_{ij} = U_{kl}\in \mathbb {Z}'$ , then it holds that $T \vartriangleleft U \in \mathsf {ShBT}_Q(\rho \vartriangleleft \tau )$ , $|T \vartriangleleft U| = |T|+|U|$ and $x^{T \vartriangleleft U} = x^T x^U$ .

Merging the boxes of $\rho \vartriangleright \tau $ or $\rho \vartriangleleft \tau $ corresponding to $(i,j) \in \rho $ and $(k,l) \in \tau $ gives a bijection from the set of elements in $ \mathsf {ShBT}_Q(\rho \vartriangleright \tau )\sqcup \mathsf {ShBT}_Q(\rho \vartriangleleft \tau )$ that do not arise from one of these cases (namely, those in which the boxes that are adjacent but of different colors in Example 4.3 belong to the same bar) to $ \mathsf {ShBT}_Q(\rho \circ \tau )$ . The expansion for $j\hspace {-0.1mm}q^{\mathrm {comb}}_{\rho }j\hspace {-0.1mm}q^{\mathrm {comb}}_\tau $ then follows. Under these bijections, the diagonal boxes in $T \vartriangleleft U$ and $T \vartriangleright U$ are filled with the same entries as the diagonal boxes in T, so the formula for $j\hspace {-0.1mm}p^{\mathrm {comb}}_{\rho }j\hspace {-0.1mm}q^{\mathrm {comb}}_\tau $ also follows.