Composability of Permutation Classes

Composability of Permutation Classes

Mark Karpilovskij
Computer Science Institute of Charles University
Malostranské nám. 25, 118 00 Prague 1, Czech Republic
This work was supported by the Neuron Fund for Support of Science

We define the operation of composing two hereditary classes of permutations using the standard composition of permutations as functions and we explore properties and structure of permutation classes considering this operation. We mostly concern ourselves with the problem of whether permutation classes can be composed from their proper subclasses. We provide examples of classes which can be composed from two proper subclasses, classes which can be composed from three but not from two proper subclasses and classes which cannot be composed from any finite number of proper subclasses.

1 Introduction

Permutations of numbers or other finite sets are a very deeply and frequently studied combinatorial and algebraic object. There are two main structures on permutations investigated in modern mathematics: groups, closed under the composition operator, and hereditary pattern-avoiding classes, closed under the relation of containment. This paper is one of several texts exploring the relation between the two notions by applying the composition operator to permutation classes. That is, given two classes and , we denote by the class of all permutations which can be written as a composition of a permutation from and a permutation from .

The oldest results combining permutation classes and groups that we know of are due to Atkinson and Beals [3], who consider the permutation classes whose permutations of length form a subgroup of for every and completely characterise the types of groups which may occur this way. These results were recently refined and extended by Lehtonen and Pöschel [8, 9]. In an earlier version of their paper, Atkinson and Beals [5] also deal with composing permutation classes, showing that compositions of many pairs of finitely based classes are again finitely based.

Some permutation classes characterise permutations which can be sorted by some sorting machine such as a stack. In this view, a composition of two permutation classes can characterise permutations sortable by two corresponding sorting machines connected serially. For example, Atkinson and Stitt [4, Section 6.4] introduce the pop-stack, a sorting machine which sorts precisely the layered permutations (see Section 5 for a definition), and consider the class of permutations which can be sorted by two pop-stacks in series, i.e. which can be written as a composition of two layered permutations. Using their more general results they calculate its generating function and enumerate its basis.

Albert et al. [1] give more enumerative results on compositions of classes in terms of sorting machines.

In the present paper, we study a different question connected to compositions of classes; namely whether a permutation of a given class can always be written as a composition of two or more permutations from its subclasses, i.e. whether for some . If this is true, we say that the class is composable and we refer to this property of as composability.

The paper is organised as follows. In Section 2 we supply all the necessary definitions and facts about permutation classes. In Section 3 we introduce composability and give some basic results. In Section 4 we explore composability of the class . In Section 5 we explore composability of various classes of layered patterns. Finally in Section 6 we give several additional miscellaneous results.

2 Preliminaries

For a positive integer we let denote the set . A permutation of order is a bijective function . We denote the order of a permutation by . We may also interpret a permutation as a sequence of distinct elements of , or as a diagram in an square in the plane, consisting of points . For let denote the set of all permutations of order .

If and are two permutations of order we define their composition as for every .

We define two more permutation operators. The sum of permutations and is the permutation The skew sum is the permutation For example, and (see Figure 1).

Figure 1: An example of sums and skew sums

In addition, we will sometimes write as .

2.1 Permutation classes

Two sequences of numbers and are order-isomorphic if for any two indices it holds that if and only if .

We define the following partial ordering on the set of all permutations. We say that is contained in and write if has a subsequence of length order-isomorphic to . See the example of containment in Figure 2. On the other hand, if , we say that avoids .

Figure 2: The permutation 213 is contained in 143625.

A set of permutations is called a permutation class if for every and every we have . We say that avoids a permutation if , i.e. every avoids . Permutation classes are often described by the patterns they avoid. If is any set of permutations, we denote by the set of all permutations avoiding every element of . Observe that is a permutation class if and only if for some set . Indeed, if is a permutation class, then , and if , then avoids all permutations of and clearly avoids them too. If and is an anti-chain with respect to containment, we call the basis of . Also if is finite, we write just instead of . Finally, if for a single permutation , we say that is a principal class.

Let be finite sequences of numbers. We denote their concatenation by . If a sequence can be constructed by interleaving in some (not necessarily unique) way, we say that is a merge of or it is merged from .

We define resp. to be the class of all permutations merged from at most increasing resp. decreasing subsequences. Also let and , i.e. is the set of all increasing permutations and is the set of all decreasing permutations, and for convenience let .

The classes and are well-known examples of principal classes.

Fact 2.1 (Vatter [10]).

and for any positive integer . ∎

Next we recall a known and important property of infinite permutation classes which will become useful in the upcoming sections.

Fact 2.2 (Atkinson, Beals [3]).

Let be an infinite permutation class. Then either or . ∎

2.2 Splittability

In this section we shortly introduce another concept which has been recently used to derive enumerative results on permutation classes and which we will also utilize in our work.

A permutation is merged from permutations and if we can color the elements of with red and blue such that the red subsequence is order-isomorphic to and the blue sequence is order-isomorphic to . Given two permutation classes and we define their merge denoted by as the class of all permutations which can be merged from a (possibly empty) permutation from and a (possibly empty) permutation from . For example, it is easy to see that

We say that a class is splittable if it has two proper subclasses and such that . We refer the reader to the work of Jelínek and Valtr [6] for an exhaustive study of splittability.

3 The notion of composability

In the following sections we provide definitions of the key notions of this work as well as basic facts and observations.

3.1 Composing permutation classes

We define the composition of two permutation classes and as the set .

Lemma 3.1.

Let and be arbitrary permutation classes.

  1. is again a permutation class.

  2. Composing permutation classes is associative, i.e. .


Let , so that and . Then a permutation contained in at indices is composed of and such that is contained at indices in and is contained at indices in . Associativity follows from associativity of permutation composition. ∎

Having verified associativity of the composition operator we can now define the composition of more than two classes in a natural inductive way:

We will also sometimes use the power notation .

We continue by proving several simple lemmas about composing permutations merged from few increasing sequences.

Lemma 3.2.

for any integers .


Choose and , partition into increasing sequences and choose one of them at indices . Then and so is a subsequence of and therefore it can be partitioned into at most increasing sequences since that is the property of . This is true for the image of each of the increasing subsequences in and therefore can be partitioned into at most increasing subsequences. ∎

Since , the argument of the previous proof can be repeated to show that . We can generalise this even more.

Lemma 3.3.

Let be any non-negative integers. Then


Use the approach identical to that of Lemma 3.2. ∎

3.2 Composability

The main problem we are addressing in this work is whether permutations in a given permutation class can be constructed by composing permutations from two or more smaller classes. We formalise this as follows. A permutation class is said to be composable from classes if . A class is -composable, if it is composable from its proper subclasses . A class is composable, if it is -composable for some . Using this terminology, our goal is thus answering the question whether a given permutation class is composable.

Clearly, for every class we have . For an infinite class we have either , which implies , or , which implies and . Restricting ourselves to proper subclasses in the definition of a composable class is motivated by these trivial inclusions.

We begin the exploration of composability by proving the following result which implies that unlike splittability, -composability for does not imply -composability.

Theorem 3.4.

Let be an infinite permutation class such that . Then is not -composable for any positive integer .


Since does not contain , there is an integer such that avoids and therefore by Fact 2.1.

Now let be proper subclasses of and suppose that . Since all these classes are subsets of , Lemma 3.3 implies for every . Using Lemma 3.3 again we get that

therefore, according to our assumption, , which means that does not contain a decreasing permutation of length by Fact 2.1. But since is infinite and does not contain , it has to contain according to Fact 2.2, which is a contradiction. ∎

3.3 Properties of symmetries

In this section we explore how composability is preserved under some of the usual symmetrical maps.

For a permutation of length we define to be the reverse of , i.e. , and to be the complement of , i.e. . For a permutation class we define the inverse class , the reverse class , and the complementary class .

Figure 3: Symmetries of the permutation 14352

It is clear that all these class operators are involutory, i.e. , and . The following simple lemma describes how these operators relate to composition.

Lemma 3.5.

Let be permutation classes. Then

  1. ,

  2. and ,

  3. .


(a): If for every , then by the property of inverse elements in a group we have .

(b): Let and be permutations of order . By definition for every . Therefore and for every .

(c): Apparent from (b). ∎

Using this lemma we derive several composability criteria for symmetries of a given class, the first of which requires no further proof as it is an immediate consequence of Lemma 3.5.

Corollary 3.6.

Let be a permutation class. Then the following statements are equivalent:

  1. is composable,

  2. is composable,

  3. is composable.

The case of the reverse and complementary operators is more complicated and requires additional assumptions.

Lemma 3.7.

If is a -composable class and , then both and are -composable.


Let be composable from its proper subclasses . Then

It holds that , so we have

Clearly and since , we have , so the proper subclass criterion is met and is therefore -composable. Analogously we show that

4 On permutations avoiding a decreasing sequence

Recall that is the class of permutations merged from increasing sequences, or equivalently those avoiding a decreasing sequence of length . In this section, we prove that is 2-composable and show several examples of how can be composed from two proper subclasses.

4.1 Vertical and horizontal merge

Let be any permutation classes. We define the vertical merge of these classes as the class of permutations that can be written as a concatenation of (possibly empty) sequences such that is order-isomorphic to a permutation of . We write this class as . In addition, if , we let denote the class . Similarly we define the horizontal merge of these classes as the class of permutations that can be written as a merge of (possibly empty) sequences such that each is order-isomorphic to and every element of is smaller than every element of for . Note that this implies that each uses a set of consecutive integers. We let denote the horizontal merge of classes and if we write .

Alternatively, we can observe that resp. if and only if its plot in can be separated by vertical resp. horizontal lines into at most parts , -th of them containing a sequence order-isomorphic to a permutation in (see Figure 4), hence the names of the classes.

Figure 4: Examples of vertical and horizontal merges

In addition we define and for future convenience. We continue by observing an important connection between the horizontal and vertical merge.

Lemma 4.1.

Let be any permutation classes. Then


If , we have that such that is order-isomorphic to . For every , contains a set of consecutive integers on indices and the sequence at these indices is order-isomorphic to .

The opposite inclusion is equally straightforward. ∎

When composed with any other class , the classes , and can be viewed as a unary operator transforming in a specific way. We formalise this approach in the following lemma.

Lemma 4.2.

Let be an arbitrary permutation class. Then

  1. is precisely the class of permutations which can be obtained from a permutation of by dividing it into at most contiguous subsequences and interleaving them in any way,

  2. is precisely the class of permutations which can be obtained from a permutation of by dividing it into at most subsequences and concatenating them,

  3. is precisely the class of permutations which can be obtained from a permutation of by dividing it into at most subsequences and interleaving them in any way.


Let , , and .

(a): Consider the permutation . Then is merged from (possibly empty) sequences of consecutive integers . Now we define sequences such that and for every and . Every is a contiguous subsequence of and at the same time is merged from .

On the other hand, if a permutation is obtained from by dividing it into  contiguous subsequences and merging them in some way, we define  sequences such that is the sequence of indices of the elements of in . Then by definition for any suitable  and , and since we divided into contiguous subsequences, each is a sequence of consecutive integers. Now consider the permutation created by replacing the subsequence by the sequence in for every . Then is merged from , which are sequences of consecutive integers, therefore . At the same time, for any there are indices  and  such that , where the last equality holds because we replaced by when constructing from . Therefore .

(b): Consider the permutation . The permutation is formed by concatenating increasing sequences . Define sequences such that and . Each is a subsequence of and at the same time .

On the other hand, if a permutation is obtained from by dividing it into subsequences and then concatenating them, we define sequences such that is the sequence of indices of elements of in . Thus every is an increasing sequence and . Consider a permutation created by replacing the subsequence by the sequence in for every . Since is a concatenation of , we get that is a concatenation of and thus . Also, for any there are indices  and  such that , where the last equality holds because we replaced by when constructing from . Therefore .

(c): The proof is similar to the proofs of (a) and (b). ∎

4.2 Composability results

Using the machinery introduced in the previous section we now prove a key lemma which we will use to show several composability results.

Lemma 4.3.

Let be arbitrary permutation classes. Then


Consider a permutation and divide it into sequences such that is isomorphic to a permutation from . The permutation then lies in , which together with Lemma 4.2(a) implies . ∎

By reformulating the previous statement we immediately get the following.

Corollary 4.4.

Let , and be permutation classes such that . Then .

Using what has already been shown in this section it is now elementary to show that is 2-composable.

Theorem 4.5.

The class is 2-composable for every . In particular, .


Trivially and . Next we recall that

and use Lemma 4.3 for . ∎

We proceed by proving a result in some sense opposite to that of Lemma 3.2, namely we show that may be constructed from smaller using composition.

Theorem 4.6.

for all integers .


Consider a permutation , merged from two sequences and such that is merged from increasing sequences and is merged from increasing sequences . Let be the increasing sequence created by sorting the elements of . Consider a permutation created by merging the sequences and so that and form a single increasing sequence. Clearly and sequences are subsequences of , since they are increasing and therefore were not affected by sorting .

According to Lemma 4.2(c) the class contains all permutations we can create from by dividing it into subsequences and merging them in any way. It is therefore enough to find a way to divide into subsequences which can be merged into . A simple choice of such subsequences is . ∎

This theorem raises the question whether we could construct a bigger class from given and .


Given positive integers and , what is the largest integer such that ?

So far we have shown that (Lemma 3.2) and that
(Theorem 4.6). It is also not difficult to show the sharp inequality by constructing a permutation .

5 Classes of layered patterns

In this section we cover classes of permutations which can be written as a sum or as a skew sum of increasing or decreasing permutations. Among these classes we provide infinitely many examples of composable classes as well as several examples of classes which are not composable.

Let denote the increasing permutation of order and denote the decreasing permutation of order . A permutation is layered if it is a sum of decreasing permutations which are then called layers. We let denote the class of all layered permutations. We let denote the class of permutations which are sums of at most layers. The complement of a layered permutation is clearly a skew sum of increasing permutations and we call such a permutation co-layered. The class consists of precisely the co-layered permutations.

Figure 5: Examples of layered and co-layered patterns

We start by proving that is not composable using a counting argument. As it turns out, proper subclasses of are asymptotically too small to build the entire class using composition.

Theorem 5.1.

The class is not composable.


Suppose that such that for every . Each of these subclasses avoids at least one permutation of . In other words for every there is a  such that . Considering a sufficiently large so that for every we get that for every , in other words every permutation in these subclasses has one of its two layers shorter than . It follows that for a fixed integer there are at most permutations of order in any , therefore there are at most permutations in . But contains permutations of order for any , therefore we obtain a contradiction by choosing . ∎

The number of permutations of order in is linear in while any proper subclass contains only constantly many permutations of fixed order. We can use the same approach using the asymptotic jump from polynomial to exponential functions to show that a different class of permutations cannot be composable. Namely, let be the class of layered permutations with layers of size 1 or 2.

Theorem 5.2.

The class is not composable.


Suppose is composable from of its proper subclasses . We choose a permutation from for every and we select large enough so that every chosen permutation is contained in . Then if , we get that . Every permutation in contains fewer than layers of size 2, otherwise it would contain . Clearly there are at most permutations of that have order and exactly layers of size 2. Therefore contains at most permutations of order and the composition then contains at most permutations of order , which is a number polynomial in . As mentioned in [10, Chapter 4], the number of permutations of order of is counted by the Fibonacci numbers which grow exponentially, therefore there is large enough so that has more permutations of order than .

Note that this result also follows immediately from the theorem of Kaiser and Klazar ([7, 3.4]), which states that if the number of permutations of order in a permutation class is less than the -th Fibonacci number for at least one value of , then it is eventually polynomial in . This implies that every class counted by the Fibonacci numbers is uncomposable. ∎

The argument used in the proofs above cannot be used for , so we need a different approach to show that this class too is not composable. We will make use of the following property of .

Lemma 5.3.

is a subgroup of for every , i.e. it is closed under composition.


In this proof, we consider an additive group structure on the set with the neutral element and an operator defined as

First we prove that by itself is a subgroup of . Observe that contains exactly permutations such that there is a shifting number with for every . Indeed, if then for any we have and conversely if for every then . Now for two permutations with shifting numbers respectively we have for any , therefore since it has a shifting number .

It trivially holds that . Considering it remains to distinguish the following four cases:

  1. and , then by the discussion above,

  2. and , then ,

  3. and , then ,

  4. and , then . ∎

Theorem 5.4.

The class is not composable.


Suppose that such that for any . Using the same initial argumentation as in the proof of Theorem 5.1 we get that there is an such that , meaning that every permutation of can be composed from permutations having at least one of the three layers shorter than .

Let for and . We now claim that it is possible to remove at most elements from to obtain a two-layered or a two-co-layered permutation. We will prove this by induction on . The case is easy since avoids , so it has a layer of length shorter than whose removal creates a two-layered pattern.

For let and . Let all these permutations have the order . By the induction hypothesis, there are indices such that and restricted to these indices has the two-layer or the two-co-layer pattern. Also there are indices such that and restricted to these indices forms the two-layer or the two-co-layer pattern.

Let us now restrict the function to the set whose size is at least . Then both and are still two-layer or two-co-layer patterns, which implies the same for their composition according to Lemma 5.3. Therefore restricted to forms a two-layer or two-co-layer pattern and which completes the induction step.

Consequently, any permutation of order in contains a two-layered or a two-co-layered pattern of size at least . But choosing and considering the permutation we obtain a contradiction. ∎

If we allow more than three but still constantly many layers, we always get a composable class.

Theorem 5.5.

The class is -composable for every .


We will show that .

If of order has fewer than layers, then . Otherwise has at least 4 layers and has the form for some positive , , , . Since for every layered we have it is not hard to check that

The situation is represented in Figure 6. ∎

Figure 6:

This theorem raises the question whether could be 2-composable for . Our work from Section 3 quickly determines that this is not the case.

Proposition 5.6.

for is not 2-composable. In particular, it is not -composable for any even number .


Since is an infinite class which does not contain the statement directly follows from Theorem 3.4. ∎

We have now covered the classes for all . It remains to consider the class , which we show to be uncomposable. Before we proceed with the proof, we introduce an additional useful concept. We call a subsequence of a permutation block if is either an increasing or a decreasing contiguous subsequence of consecutive integers. We then call -block if it is a concatenation of at most blocks (see Figure 7).

Figure 7: An example of a 4-block
Lemma 5.7.

Let be a -block and let be an -block. Then is a -block.


Choose a block of at indices . Then the sequence

is a contiguous subsequence of either