A Generalization of Schur’s P- and Q-Functions

# A Generalization of Schur’s P- and Q-Functions

Soichi Okada111 Graduate School of Mathematics, Nagoya University, Furo-cho, Chikusa-ku, Nagoya 464-8602, Japan, okada@math.nagoya-u.ac.jp 222 This work was partially supported by JSPS Grants-in-Aid for Scientific Research No. 24340003, No. 15K13425 and No. 18K03208.
Dedicated to Christian Krattenthaler on the occasion of his 60th birthday
###### Abstract

We introduce and study a generalization of Schur’s -/-functions associated to a polynomial sequence, which can be viewed as “Macdonald’s ninth variation” for -/-functions. This variation includes as special cases Schur’s -/-functions, Ivanov’s factorial -/-functions and the specialization of Hall–Littlewood functions associated to the classical root systems. We establish several identities and properties such as generalizations of Schur’s original definition of Schur’s -functions, Cauchy-type identity, Józefiak–Pragacz–Nimmo formula for skew -functions, and Pieri-type rule for multiplication.

## 1 Introduction

Schur (-)functions and Schur -/-functions are two important families of symmetric functions, and they appear in several parallel situations. For example, in the representation theory of the symmetric groups, Schur functions describe the characters of irreducible linear representations, while Schur -functions describe the characters of irreducible projective representations (see [21]). In the cohomology theory, Schur functions represent the Schubert classes of Grassmannians, while Schur -functions represent the Schubert classes of Lagrangian Grassmannians (see [19]). Also some identities for Schur functions have their counterparts for Schur -/-functions.

There are several generalizations, variations or deformations of Schur functions, such as Hall–Littlewood functions, Macdonald functions and factorial Schur functions. The generalization relevant to this paper is Macdonald’s ninth variation ([10], see also [15]) associated to a polynomial sequence, which is defined as follows.

Let be a sequence of polynomials , where is a ground field of characteristic , such that for . Given a partition of length , we define the generalized Schur function as the ratio of two alternants:

 sFλ(x1,…,xn)=det(fλj+n−j(xi))1≤i,j≤ndet(fn−j(xi))1≤i,j≤n, (1.1)

where . The original Schur functions are recovered by setting for . And the factorial Schur functions with factorial parameters are obtained by taking . Moreover, classical group characters are special cases of generalized Schur functions. For example, if the polynomial sequence is defined by

 fd(x+x−1)=xd+1−x−d−1x−x−1(d≥0),

then it is not difficult to see that the generalized Schur function equals to the irreducible character of the symplectic group with highest weight .

Generalized Schur functions share many of the same properties as the original Schur functions. For example, they satisfy the modified Jacobi–Trudi identity and the Giambelli identity:

 sFλ(x1,…,xn) =det(sF(λi−i+j)(xj,…,xn))1≤i,j≤l =det(sF(αi|βj)(x1,…,xn))1≤i,j≤r,

where is a partition of length and in the Frobenius notation.

The aim of this paper is to introduce and study the “ninth variation” of Schur -/-functions, which we call generalized -functions associated to polynomial sequences. We define generalized -functions in terms of Nimmo-type formula and derive Pfaffian identities and basic properties by following a linear algebraic approach similar to [18].

We use the following terminologies on polynomial sequences.

###### Definition 1.1.

Let be a sequence of polynomials . We say that is admissible if it satisfies the conditions

 f0(t)=1,degfd=d(d≥1). (1.2)

And an admissible sequence is called constant-term free if for any .

In this article, a partition of length is a weakly decreasing sequence of positive integers. We write and . A partition of length is called strict if . The empty sequence is the unique strict partition of length .

For a sequence of indeterminates, we put

 A(x)=(xj−xixj+xi)1≤i,j≤n,Δ(x)=∏1≤i

Now we give a definition of generalized Schur -functions associated to polynomial sequences in terms of Nimmo-type formula (see [17, (A13)]).

###### Definition 1.2.

For an admissible sequence of polynomials and a sequence of nonnegative integers, let be the matrix given by

 VFα(x)=(fαj(xi))1≤i≤n,1≤j≤r.

Given a strict partition of length , we define the corresponding generalized -function associated to by putting

 PFλ(x)=⎧⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎨⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎩1Δ(x)Pf(A(x)VFλ(x)−tVFλ(x)O)if n+l is even,1Δ(x)Pf(A(x)VFλ0(x)−tVFλ0(x)O)if% n+l is odd, (1.4)

where . We simply write and for and if there is no confusion, e.g., in the proofs.

Note that (see Proposition A.1)

 Δ(x)=⎧⎪⎨⎪⎩PfA(x)% if n is even,Pf(A(x)1−t10)if n is odd,

where is the all-one column vector of appropriate size. Hence our definition (1.4) can be regarded as a counterpart of the definition (1.1) of generalized Schur functions.

###### Example 1.3.
1. It follows from Nimmo’s formula [17, (A13)] that we recover the original Schur -function and Schur -function by setting and respectively.

2. It follows from Nimmo-type formula [6, Theorem 3.2] that Ivanov’s factorial -function and -function are obtained by taking and respectively.

3. As we will see in Section 7, our generalized -functions include the specializations of Hall–Littlewood functions associated to the root system of type , and .

Ikeda–Naruse [3] and Nakagawa–Naruse [16] introduced other generalizations of factorial - and -functions from the viewpoint of Schubert calculus.

The organization and main results of this paper are as follows. In Section 2, we relate our definition of generalized -functions (Definition 1.2) with generalizations of two other definitions of Schur -/-functions. Namely we prove that is also obtained by setting in the generalized Hall–Littlewood function associated to a polynomial sequence (see Theorem 2.3), and that is expressed as the Pfaffian of the skew-symmetric matrix with entries (see Theorem 2.6). In Section 3, we introduce the notion of generalized dual -functions and prove the Cauchy-type identity. In Section 4, we define generalized skew -functions in terms of Józefiak–Pragacz–Nimmo tpye Pfaffian and prove that appears as the coefficient of in the expansion of (see Theorem 4.2). In Section 5, we consider the modified Pieri coefficients in the expansion of the product and obtain a determinant formula for the generating function of modified Pieri coefficients (see Theorem 5.3). Section 6 focuses on Ivanov’s factorial -/-functions. We derive a determinant formula of the factorial skew -function in one variable (see Theorem 6.5), and an explicit product formula for the generating function of modified Pieri coefficients (see Theorem 6.6). In Section 7, we show that the Hall–Littlewood functions at associated to the classical root systems can be written as generalized -functions associated to certain polynomial sequences (see Theorem 7.2). Appendix A collects some Schur-type Pfaffian evaluations and useful formulas.

## 2 Several expressions of generalized P-functions

In this section, we give several expressions of generalized -functions associated to an admissible polynomial sequence, and study their basic properties.

### 2.1 Hall–Littlewood-type expression

In this subsection we prove that our generalized -functions are obtained as the specialization of Hall–Littlewood-type functions.

We begin with the following proposition.

###### Proposition 2.1.

Let be an admissible sequence of polynomials and . Then we have

1. For the empty partition , we have .

2. If is a strict partition of length , then we have .

• By using the definition of (1.4), we can derive (1) from the Pfaffian evaluations (A.3) and (A.4), and (2) from Proposition A.5. ∎

We define a generalization of Hall–Littlewood polynomials associated to an admissible polynomial sequence.

###### Definition 2.2.

Let be a positive integer and . Given a partition of length , we regard as a sequence of length , and define a polynomial by putting

 v(n)λ(t)=∏k≥0[mk]t!,

where and . For an admissible polynomial sequence and a partition of length , we define the generalized Hall–Littlewood function corresponding to by putting

 PFλ(x;t)=1v(n)λ(t)∑w∈Snw(n∏i=1fλi(xi)∏1≤i

where is the symmetric group acting on by permuting variables.

Setting for , we recover the original Hall–Littlewood polynomials. The following is the main theorem of this subsection.

###### Theorem 2.3.

For an admissible sequence and a strict partition of length , we have

 PFλ(x)=PFλ(x;−1). (2.2)

Note that Equation (2.2) with is the definition of Schur -function adopted in [11, III.8]. For the sake of completeness and the later use, we give a proof of this theorem, which follows the argument in [17, Appendix]. As a first step, we show the following lemma:

###### Lemma 2.4.

For a strict partition of length , we have

 PFλ(x;−1) =∑u∈Sn/Sn−lu⎛⎜ ⎜⎝l∏i=1fλi(xi)∏1≤i

where is the symmetric group on the last variables .

• Since and the product is invariant under , we have

 Pλ(x;t)=1v(n)λ(t)∑w′∈Sn/Sn−lw′⎛⎜ ⎜⎝l∏i=1fλi(xi)∏1≤i

By using (see [9, Theorem 2.8])

 ∑w′′∈Sn−lw′′⎛⎝∏l+1≤i

we have

 Pλ(x;t)=∑w′∈Sn/Sn−lw′⎛⎜ ⎜⎝l∏i=1fλi(xi)∏1≤i

By specializing , we obtain (2.3), from which (2.4) follows. ∎

• Since is alternating in , it follows from (2.3) that

 Pλ(x;−1)=(−1)(n2)+(n−l2)Δ(x)∑v∈Sn/Sn−lsgn(v)v⎛⎝l∏i=1fλi(xi)∏l+1≤i

Since is invariant under the symmetric group acting on the first variables , we have

 Pλ(x;−1) =(−1)(n2)+(n−l2)Δ(x)∑v′∈Sn/(Sl×Sn−l)sgn(v′)v′⎛⎝∑v′′∈Slsgn(v′′)v′′(l∏i=1fλi(xi))∏l+1≤i

We take as a complete set of coset representatives of . We note that the correspondence gives a bijection between the coset representatives and the set of all -element subsets of .

First we consider the case where is even. In this case, by using Schur’s Pfaffian evaluation (A.3), we have

 Pλ(x;−1)=(−1)(n2)+(n−l2)Δ(x)∑u∈Sn/Sl×Sn−lsgn(u)u(detVλ(x[l])PfA(x[n]∖[l])),

where , and for with . On the other hand, by applying Proposition A.5 (a Pfaffian version of the Laplace expansion) to the matrices and , we obtain

 Pf(A(x)Vλ(x)−tVλ(x)O)=∑I∈([n]n−l)(−1)Σ(I)−(n2)PfA(xI)detVλ(x[n]∖I),

where . Since is even, we can see that, if corresponds to , then the inversion number of is given by

 inv(u)=(n+12)−(l+12)−Σ(I)≡(n2)−(l2)+Σ(I)mod2.

Hence we have

 Pf(A(x)Vλ(x)−tVλ(x)O)=∑u∈Sn/(Sl×Sn−l)(−1)(l2)sgn(u)u(detVλ(x[l])PfA(x[n]∖[l])),

and

 Pλ(x;−1)=(−1)(n2)+(n−l2)+(l2)Δ(x)Pf(A(x)Vλ(x)−tVλ(x)O).

Now we can use the relation to complete the proof of (2.2) in the case where is even.

Next we consider the case where is odd. In this case, by using (A.4), we see that

 Pλ(x;−1)=(−1)(n2)+(n−l2)Δ(x)∑u∈Sn/(Sl×Sn−l)sgn(u)u(detVλ(x[l])Pf(A(x[n]∖[l])1−t10)),

where is the all-one column vector. On the other hand, by applying Proposition A.5 to the matrices

 Z=(A(x)1−t10),W=(Vλ(x)O),

we see that

 Pf⎛⎜⎝A(x)1Vλ(x)−t10O−tVλ(x)OO⎞⎟⎠=∑I(−1)Σ(I)−(n+12)PfZ(I)detW([n+1]∖I;[l]),

where runs over all -element subsets of . If , then we have

 detW([n+1]∖I;[l])=det(Vλ(x[n]∖I)O)=0.

Hence we have

 Pf⎛⎜⎝A(x)1Vλ(x)−t10O−tVλ(x)OO⎞⎟⎠=∑I∈([n]n−l)(−1)Σ(I∪{n+1})−(n+12)Pf(A(xI)1−t10)detVλ(x[n]∖I).

Since is odd, we see that, if corresponds to , then we have

 inv(u)=(n+12)−(l+12)−Σ(I)≡(n+1)2−(l2)−Σ(I∪{n+1})mod2.

Also, by permuting rows/columns we have

 Pf⎛⎜⎝A(x)1Vλ(x)−t10O−tVλ(x)OO⎞⎟⎠=(−1)lPf(A(x)Vλ0(x)−tVλ0(x)O).

Hence we have

 Pf(A(x)Vλ0(x)−tVλ0(x)O)=∑u∈Sn/(Sl×Sn−l)(−1)(l2)+lsgn(u)u(PfA(x[n]∖[l])detVλ(x[l])),

and

 Pλ(x;−1)=(−1)(n2)+(n−l2)+(l2)+lΔ(x)Pf(A(x)Vλ0(x)−tVλ0(x)O).

Now we can complete the proof in the case where is odd by using the congruence relation . ∎

By combining Theorem 2.3 and Lemma 2.4, we obtain

###### Corollary 2.5.

For a strict partition of length , we have

 PFλ(x) =∑u∈Sn/Sn−lu⎛⎜ ⎜⎝l∏i=1fλi(xi)∏1≤i

### 2.2 Schur-type Pfaffian formula

In this subsection we use the definition (1.4) and a Pfaffian version of Sylvester formula (Proposition A.4) to derive the Schur-type Pfaffian formula for , which generalizes (a part of) Schur’s original definition of Schur -functions [21, § 35] and a similar formula for factorial -functions [6, Theorem 9.1]. We use the following conventions:

 PF(0)(x)=1, (2.7) PF(s,r)(x)=−PF(r,s)(x),PF(r,0)(x)=−PF(0,r)(x)=PF(r)(x),PF(0,0)(x)=0, (2.8)

where and are positive integers.

###### Theorem 2.6.

Let be an admissible sequence. For a sequence of non-negative integers, let be the skew-symmetric matrix defined by

 SFα(x)=(PF(αi,αj)(x))1≤i,j≤r. (2.9)

Then, for a strict partition of length , we have

 PFλ(x)={PfSFλ(x)if l is even,PfSFλ0(x)if l is% odd, (2.10)

where and .

In order to prove this theorem, we can use the same argument as in [18, Theorem 4.1 (3) and Remark 4.3]. As we will see in Proposition 2.7, the generalized -functions do not have the stability property, so we cannot reduce the proof to the case where is even.

• By applying Proposition A.4 to the matrix given by

 X=⎧⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎨⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎩(A(x)Vλ(x)−tVλ(x)O)if n is even and l is even,(A(x)Vλ0(x)−tVλ0(x)O)if n is even % and l is odd,⎛⎜⎝A(x)1Vλ(x)−t10O−tVλ(x)OO⎞⎟⎠if n is odd and % l is even,⎛⎜ ⎜ ⎜ ⎜⎝A(x)1Vλ(x)0−t10O−1−tVλ(x)OO00100⎞⎟ ⎟ ⎟ ⎟⎠if n is odd and l is odd.

If is even, then we have

 PfXPfX([n])=Pλ(x),PfX([n]∪{n+i,n+j})PfX([n])=P(λi,λj)(x).

If is odd and is even, then by permuting rows/columns, we see that

 PfXPfX([n+1])=Pλ(x),PfX([n+1]∪{n+1+i,n+1+j})PfX([n+1])=P(λi,λj)(x).

If is odd and is odd, then by expanding the Pfaffians along the last row/column, we have

 PfXPfX([n+1])=Pλ(x),PfX([n+1]∪{n+1+i,n+1+(l+1)})PfX([n+1])=P(λi)(x),

and by permuting rows/columns we see that

 PfX([n+1]∪{n+1+i,n+1+j})PfX([n+1])=P(λi,λj)(x).

Now Theorem 2.6 follows immediately from Proposition A.4. ∎

### 2.3 Stability

The Schur -functions have the stability property (see [11, III, (2.5)])

 Pλ(x1,…,xn,0)=Pλ(x1,…,xn).

Our generalizations do not have the stability property in general. For example we can show that

 PF(r)(x1,…,xn,0)=PF(r)(x1,…,xn)+(−1)nfr(0)

for . The following “mod stability property” was given by [2, Proposition 8.1] for factorial -functions.

###### Proposition 2.7.

Let be an admissible sequence, and a strict partition.

1. In general, we have

 PFλ(x1,…,xn,0,0)=PFλ(x1,…,xn).
2. If is constant-term free, then we have

 PFλ(x1,…,xn,0)=PFλ(x1,…,xn).
• (1) Let and . It follows from the definition (1.4) that

 Pλ(x1,…,xn,xn+1,0)=1(−1)n+1Δ(˜x)Pf⎛⎜⎝A(˜x)−1Vλ∗(˜x)t10Vλ∗(0)−tVλ∗(˜x)−tVλ∗(0)O⎞⎟⎠,

where or according whether is even or odd, and is the row vector . Hence, if we put in the above formula, we have

 Pλ(x1,…,xn,0,0)=1(−1)n+1⋅(−1)nΔ(x)Pf⎛⎜ ⎜ ⎜ ⎜⎝A(x)−1−1Vλ∗(x)t10−1Vλ∗(0)t110Vλ∗(0)−tVλ∗(x)−tVλ∗(0)−tVλ∗(0)O⎞⎟ ⎟ ⎟ ⎟⎠.

By subtracting the st column/row from the nd column/row and then by expanding the resulting Pfaffian along the nd columns/row, we see that

 Pλ(x1,…,xn,0,0)=1(−1)n+1⋅(−1)nΔ(x)⋅(−1)Pf(A(x)Vλ∗(x)−tVλ∗(x)O)=Pλ(x).

(2) By the definition (1.4), we have

where or according whether is even or odd. If is even, by adding the st row/column to the last row/column and then by expanding the resulting Pfaffian along the last row/column, we see that . If is odd, then by permuting rows/columns, we obtain . ∎

### 2.4 Relation with generalized Schur functions

We conclude this section by proving a relation between generalized -functions and generalized Schur functions.

###### Proposition 2.8.

Let be an admissible sequence and . For a partition of length , let be the strict partition obtained from by removing s, where . Then we have

 PFμ+δn(x)=∏1≤i