# Approaches for enumerating permutations with a prescribed number of occurrences of patterns

###### Abstract

In recent work, Zeilberger and the author used a functional equations approach for enumerating permutations with occurrences of the pattern . In particular, the approach yielded a polynomial-time enumeration algorithm for any fixed . We extend that approach to patterns of the form by deriving analogous functional equations and using them to develop similar algorithms that enumerate permutations with occurrences of the pattern. We also generalize those techniques to handle patterns of the form and derive analogous functional equations and enumeration algorithms. Finally, we show how the functional equations and algorithms can be modified to track inversions as well as handle multiple patterns simultaneously. This paper is accompanied by Maple packages that implement the algorithms described.

## 1 Introduction

Let be a sequence of distinct positive integers. We define the reduction to be the length permutation that is order-isomorphic to (i.e., if and only if for every and ). Given a (permutation) pattern , we say that a permutation contains the pattern if there exists such that , in which case we call an occurrence of . We will define to be the number of occurrences of in . For example, if the pattern , the permutation avoids the pattern (so ), whereas the permutation contains two occurrences of (so ).

For a pattern and non-negative integer , we define the set

and also define . The corresponding generating function is defined as

Observe that the classical pattern avoidance problem corresponds to the case where and has been well studied. In this setting, is known to be enumerated by the Catalan numbers if [11]. However, much is still unknown even for length patterns. For example, permutations avoiding the pattern have been notoriously difficult to enumerate. Precise asymptotics are not even known, although Bóna recently gave an improved upper bound for the growth rate (in [3]) by modifying the approach used by Claesson, Jelínek, and Steingrímsson [8]. The survey paper by Kitaev and Mansour [10] provides an extensive overview of work in this area as well as related problems in permutation patterns.

While the more general problem (where ) has also been studied, the work has usually been restricted to small patterns (usually length three) and small . In [15], Noonan studied permutations containing exactly one occurrence of and proved that . Burstein recently gave a short combinatorial proof for the result [6]. In [16], Noonan and Zeilberger presented an approach using functional equations to enumerate for small and for the patterns , , and . Subsequent work has been done by Bóna [5, 4], Fulmek [9], Mansour and Vainshtein [13], Callan [7], and many others. Many of these focused on finding for and for small .

One difficulty arising from the initial Noonan-Zeilberger functional equation approach in [16] was that the approach became very complicated for even . In addition, there are many patterns that this approach does not readily extend to. One such pattern (explicitly mentioned in [16]) was . A modified approach was recently presented in [14] for the case of increasing patterns. Given a fixed , the resulting enumeration algorithm for computing was polynomial-time (in ). This, in a sense, tackled the first difficulty from [16] and allowed us to enumerate the sequence for even larger fixed .

In this paper, we extend the enumeration techniques in [14] to new families of patterns (including the pattern ) as well as multiple patterns. It should be noted that this general approach is different from the enumeration schemes approach pioneered by Zeilberger [21] and extended by Vatter [20], Pudwell [17, 2], and Baxter [1, 2]. The enumeration schemes approach is useful for enumerating pattern-avoiding permutations (the case) but does not appear to be readily adaptable to the generalized setting for permutations with occurrences of a pattern.

The paper is organized in the following manner. Section 2 extends the approach in [14] to the patterns , , , and so on. Section 3 generalizes the techniques used in [14] and applies them to the patterns , (which is equivalent to ), and so on. Section 4 extends this approach to handle multiple patterns simultaneously as well as refining by the inversion number^{1}^{1}1This is technically the same as tracking the number of patterns that occur.. Section 5 lists some possible future work as well as some conjectures. The enumeration algorithms developed in this paper are implemented in the Maple packages FINCR, FINCRT, F231, F2341, F123n132, F1234n1243, and FS3. They are all available from the author’s website.

## 2 Counting occurrences of the pattern

In this section, we adapt the approach in [14] (for increasing patterns) to the patterns . We first handle the case of in full detail and then outline how to generalize this approach to patterns , , and so on.

### 2.1 Permutations containing

Given a (fixed) pattern and non-negative integer , we define the polynomial

(1) |

Observe that the coefficient of in is exactly equal to . For a fixed pattern and fixed , our goal is to quickly compute . In the remainder of this section, we will assume that .

In addition to the variable , we introduce the catalytic variables and define the weight of a length permutation to be

In general, this will be written more simply as when the fixed pattern is clear from context (in this case ). For example, , , and . In essence, the weight of a permutation encodes the number of patterns as well as information on patterns (which may become the “” of a pattern if a term is inserted at the beginning of the permutation).

For each , we define the polynomial

Observe that is essentially a generalized multi-variate polynomial for and in particular, . We now get the following:

###### Lemma 1.

Let and suppose that . If , then

###### Proof.

We assume to be a fixed value and will compute from . We re-insert at the beginning of by shifting all the terms up by (i.e., for ). The new “” would create new patterns and would require an extra factor of for the weight. Also, observe that is equal to the number of occurrences of in plus the number of occurrences of in , where the term corresponding to the “” is larger than . Therefore, our shift now becomes for . ∎

This directly leads to the functional equation:

###### Theorem 1.

For the pattern ,

(FE132) |

Once is computed, the catalytic variables can all be set to to get . However, it is not necessary to compute in its entirety prior to setting the catalytic variables to . Observe that by (FE132), we have:

We get terms of the form in the summation, which can again be plugged into (FE132) to get:

Now, we must deal with terms of the form . We can continue this recursive process of plugging new terms into (FE132) to eventually compute . This is much faster than the direct weighted counting of all permutations, although it is still unfortunately an exponential-time (and memory) algorithm.

This algorithm has been implemented in the procedure F132full(n,t) (in the Maple package FINCRT). For example, the Maple call F132full(8,t); computes and outputs:

Suppose that for a small fixed , we wanted the first terms of the sequence . By this functional equation approach, one would compute and extract the coefficient of for each up to . This approach would expend quite a bit of computational effort in generating unnecessary information (namely, all the terms where ). This issue can mostly be circumvented, however, by a couple of observations. The first is the following lemma from [14]:

###### Lemma 2.

Let (where for each ) and suppose . Then, the coefficients of in

all vanish.

###### Proof.

The more general function is a multi-variate polynomial. ∎

This lemma allows us to collapse all the higher powers of into the coefficient and allows us to consider objects of the form regardless of how large is.

Let . Also, for any expression and positive integer , let denote . For example, is shorthand for . Now for any polynomial in the variable , let denote the polynomial of degree (at most) obtained by discarding all powers of larger than . Also, define the operator by .

An application of (FE132) and to becomes:

Due to the operator, many terms automatically disappear because of the power of in front. From a computational perspective, this observation eliminates many unnecessary terms and hence circumvents a lot of unnecessary computation. This has been automated in the Maple package FINCRT so that a computer can derive a “scheme” for any fixed (completely on its own) and use it to enumerate for as many terms as the user wants.^{2}^{2}2The “scheme” mentioned here is a liberal application of the word and differs from enumeration schemes.

For example, the Maple call F132rN(5,15); for the first terms of produces the sequence:

### 2.2 Extending to the pattern

First, we outline how to extend the previous approach to the pattern . In addition to the variable , we now introduce catalytic variables and . The weight of a length permutation will now be

For example, and . In essence, the weight encodes the number of patterns as well as information on and patterns (which may become the “” or “” of a when terms are inserted at the beginning of the permutation).

For each , we define the polynomial

We now observe the following:

###### Lemma 3.

Let and suppose that . If , then

where is the set of substitutions given by

###### Proof.

We assume to be a fixed value and will again compute from . First, we re-insert at the beginning of by shifting all the terms up by (i.e., and for ). The new “” would create new patterns and would require an extra factor of for the weight. Also, the re-insertion of would create new patterns. The number of such new patterns is exactly the number of patterns in the shifted , where the “” is greater than . Therefore, our shift now becomes for . Also, observe that is equal to the number of occurrences of in plus the number of occurrences of in , where the term corresponding to the “” is larger than . Therefore, our shift now becomes for . ∎

This directly leads to the new functional equation:

###### Theorem 2.

For the pattern ,

(FE1243) |

Again, our goal is to compute . We can apply the same computational methods as before. For example, we can apply (FE1243) directly to (and more generally, to objects of the form ) to compute . This again gives us an algorithm that is faster than the direct weighted counting of permutations but is still exponential-time (and memory).

This algorithm has been implemented in the procedure F1243full(n,t) (in Maple package FINCRT). For example, the Maple call F1243full(8,t); computes and outputs:

Additionally, both the obvious analog of Lemma 2 as well as the computational reduction using the operator still apply in this setting. This has also been automated in the Maple package FINCRT.

For example, the Maple call F1243rN(1,15); for the first terms of produces the sequence:

and the Maple call F1243rN(2,15); for the first terms of produces the sequence:

### 2.3 Extending to longer patterns

The approach for the patterns and can be extended analogously to longer patterns of the form . For example, if the pattern , we consider the variable and catalytic variables: and and . The weight of a length permutation will now be

where it is always assumed that .

An analogous functional equation is derived for the corresponding polynomial

and all the analogous computational methods work in this setting as well. The case has also been automated in the Maple package FINCRT.

For example, the Maple call F12354rN(0,14); for the first terms of produces the sequence:

and the Maple call F12354rN(1,15); for the first terms of produces the sequence:

## 3 Counting occurrences of the pattern

In this section, we extend the previous techniques to handle patterns of the form . Although for every and (by reversal), we will develop an approach for handling directly^{3}^{3}3As opposed to computing the equivalent pattern . and then show how this can be extended to longer patterns of the form . This new approach for handling will also be necessary in the next section for considering multiple patterns simultaneously.

### 3.1 Permutations containing

In this section, we will assume that our (fixed) pattern . We define the analogous polynomial

Recall that the coefficient of in will be exactly .

In addition to the variable , we introduce catalytic variables with and define the weight of a permutation to be

For example, and .

We will again define an analogous multi-variate polynomial on all the previously defined variables. However, for notational convenience, the variables will be written as a matrix of variables:

(2) |

where we will disregard the entries above the diagonal (i.e., the entries where ).

For each , we now define the polynomial

Recall that , where is the matrix of all ’s. We will derive a functional equation for this function, but first, we derive the following lemma:

###### Lemma 4.

Let and suppose that . If , then

where is the set of substitutions given by

###### Proof.

We assume to be a fixed value. Observe that is equal to the number of occurrences of in plus the number of occurrences of in , where the term corresponding to the “” is greater than and the term corresponding to the “” is less than . We make the following two observations. First, in , the exponents of and are equal for each (since ). Second, the number of patterns that include the first term is the sum of the exponents of for .

If we re-insert at the beginning of , we would shift all the terms up by . This (combined with the prior observations) would lead to the set of substitutions given by . Note that there is no case for since the variables are only defined for . Finally, the new “” would create new patterns and would require an extra factor of for the weight. ∎

Now, define the operator on an square matrix and to be:

(3) |

In essence, the operator deletes the -th row, merges the -th and -th columns via term-by-term multiplication, and multiplies this new column by a factor of . If , then is defined to be the matrix obtained by deleting the -th row and -th column from . It is important to note that while this operator is defined on any matrix, it will only be applied to our “matrix of variables” to get a smaller matrix.

The previous lemma now leads directly to the following:

###### Theorem 3.

For the pattern ,

(FE231) |

Note that while all entries in the matrix are changed for consistency, we will continue to disregard the entries above the diagonal.

Again, our goal is to compute , and the analogous computational techniques from previous sections will also apply in this setting. For example, we can apply (FE231) directly to as opposed to computing symbolically and substituting at the end. The following result, which is obvious from the definition of the operator , provides a substantial simplification:

###### Lemma 5.

Let be a square matrix where every row is identical (i.e., the -th row and the -th row are equal for every ). Then, will also be a square matrix with identical rows.

By Lemma 5, repeated applications of to the all ones matrix will still result in a matrix with identical rows. Therefore, it is sufficient to keep track of only one row as opposed to the entire matrix. Also observe that repeated applications of to the matrix will always result in a matrix whose entries are powers of . Let denote the polynomial , where is the matrix where every row is and every is a power of . This leads to a functional equation analogous to (FE231):

(FE231c) |

Note that is exactly our desired polynomial . However, this interpretation only forces us to deal with catalytic variables (the ’s) as opposed to catalytic variables (the ’s). Just as in prior sections, we can repeatedly apply our functional equation (FE231c) to compute .

When the sequence is desired for a fixed , the obvious analog of Lemma 2 and the computational reduction using the operator can again be used. This has been implemented in the Maple package F231.^{4}^{4}4Although all output would be equivalent to the case, the approach here will be necessary when considering multiple patterns.

### 3.2 Extending to the pattern

In this section, we outline how to extend the approach for to an analogous (but more complicated) approach for . In addition to the variable , we now introduce catalytic variables with and more catalytic variables with (a total of catalytic variables). Define the weight of a permutation to be

For example, .

The variables and the variables will be written as matrices of variables:

(4) |

where we will disregard the entries above the diagonal.

For each , we define the polynomial

and again is our desired polynomial. We now have the following result:

###### Lemma 6.

Let and suppose that . If , then

where is the set of substitutions given by