Composability of Permutation Classes
Abstract
We define the operation of composing two hereditary classes of permutations using the standard composition of permutations as functions and we explore properties and structure of permutation classes considering this operation. We mostly concern ourselves with the problem of whether permutation classes can be composed from their proper subclasses. We provide examples of classes which can be composed from two proper subclasses, classes which can be composed from three but not from two proper subclasses and classes which cannot be composed from any finite number of proper subclasses.
1 Introduction
Permutations of numbers or other finite sets are a very deeply and frequently studied combinatorial and algebraic object. There are two main structures on permutations investigated in modern mathematics: groups, closed under the composition operator, and hereditary patternavoiding classes, closed under the relation of containment. This paper is one of several texts exploring the relation between the two notions by applying the composition operator to permutation classes. That is, given two classes and , we denote by the class of all permutations which can be written as a composition of a permutation from and a permutation from .
The oldest results combining permutation classes and groups that we know of are due to Atkinson and Beals [3], who consider the permutation classes whose permutations of length form a subgroup of for every and completely characterise the types of groups which may occur this way. These results were recently refined and extended by Lehtonen and Pöschel [8, 9]. In an earlier version of their paper, Atkinson and Beals [5] also deal with composing permutation classes, showing that compositions of many pairs of finitely based classes are again finitely based.
Some permutation classes characterise permutations which can be sorted by some sorting machine such as a stack. In this view, a composition of two permutation classes can characterise permutations sortable by two corresponding sorting machines connected serially. For example, Atkinson and Stitt [4, Section 6.4] introduce the popstack, a sorting machine which sorts precisely the layered permutations (see Section 5 for a definition), and consider the class of permutations which can be sorted by two popstacks in series, i.e. which can be written as a composition of two layered permutations. Using their more general results they calculate its generating function and enumerate its basis.
Albert et al. [1] give more enumerative results on compositions of classes in terms of sorting machines.
In the present paper, we study a different question connected to compositions of classes; namely whether a permutation of a given class can always be written as a composition of two or more permutations from its subclasses, i.e. whether for some . If this is true, we say that the class is composable and we refer to this property of as composability.
The paper is organised as follows. In Section 2 we supply all the necessary definitions and facts about permutation classes. In Section 3 we introduce composability and give some basic results. In Section 4 we explore composability of the class . In Section 5 we explore composability of various classes of layered patterns. Finally in Section 6 we give several additional miscellaneous results.
2 Preliminaries
For a positive integer we let denote the set . A permutation of order is a bijective function . We denote the order of a permutation by . We may also interpret a permutation as a sequence of distinct elements of , or as a diagram in an square in the plane, consisting of points . For let denote the set of all permutations of order .
If and are two permutations of order we define their composition as for every .
We define two more permutation operators. The sum of permutations and is the permutation The skew sum is the permutation For example, and (see Figure 1).
In addition, we will sometimes write as .
2.1 Permutation classes
Two sequences of numbers and are orderisomorphic if for any two indices it holds that if and only if .
We define the following partial ordering on the set of all permutations. We say that is contained in and write if has a subsequence of length orderisomorphic to . See the example of containment in Figure 2. On the other hand, if , we say that avoids .
A set of permutations is called a permutation class if for every and every we have . We say that avoids a permutation if , i.e. every avoids . Permutation classes are often described by the patterns they avoid. If is any set of permutations, we denote by the set of all permutations avoiding every element of . Observe that is a permutation class if and only if for some set . Indeed, if is a permutation class, then , and if , then avoids all permutations of and clearly avoids them too. If and is an antichain with respect to containment, we call the basis of . Also if is finite, we write just instead of . Finally, if for a single permutation , we say that is a principal class.
Let be finite sequences of numbers. We denote their concatenation by . If a sequence can be constructed by interleaving in some (not necessarily unique) way, we say that is a merge of or it is merged from .
We define resp. to be the class of all permutations merged from at most increasing resp. decreasing subsequences. Also let and , i.e. is the set of all increasing permutations and is the set of all decreasing permutations, and for convenience let .
The classes and are wellknown examples of principal classes.
Fact 2.1 (Vatter [10]).
and for any positive integer . ∎
Next we recall a known and important property of infinite permutation classes which will become useful in the upcoming sections.
Fact 2.2 (Atkinson, Beals [3]).
Let be an infinite permutation class. Then either or . ∎
2.2 Splittability
In this section we shortly introduce another concept which has been recently used to derive enumerative results on permutation classes and which we will also utilize in our work.
A permutation is merged from permutations and if we can color the elements of with red and blue such that the red subsequence is orderisomorphic to and the blue sequence is orderisomorphic to . Given two permutation classes and we define their merge denoted by as the class of all permutations which can be merged from a (possibly empty) permutation from and a (possibly empty) permutation from . For example, it is easy to see that
We say that a class is splittable if it has two proper subclasses and such that . We refer the reader to the work of Jelínek and Valtr [6] for an exhaustive study of splittability.
3 The notion of composability
In the following sections we provide definitions of the key notions of this work as well as basic facts and observations.
3.1 Composing permutation classes
We define the composition of two permutation classes and as the set .
Lemma 3.1.
Let and be arbitrary permutation classes.

is again a permutation class.

Composing permutation classes is associative, i.e. .
Proof.
Let , so that and . Then a permutation contained in at indices is composed of and such that is contained at indices in and is contained at indices in . Associativity follows from associativity of permutation composition. ∎
Having verified associativity of the composition operator we can now define the composition of more than two classes in a natural inductive way:
We will also sometimes use the power notation .
We continue by proving several simple lemmas about composing permutations merged from few increasing sequences.
Lemma 3.2.
for any integers .
Proof.
Choose and , partition into increasing sequences and choose one of them at indices . Then and so is a subsequence of and therefore it can be partitioned into at most increasing sequences since that is the property of . This is true for the image of each of the increasing subsequences in and therefore can be partitioned into at most increasing subsequences. ∎
Since , the argument of the previous proof can be repeated to show that . We can generalise this even more.
Lemma 3.3.
Let be any nonnegative integers. Then
Proof.
Use the approach identical to that of Lemma 3.2. ∎
3.2 Composability
The main problem we are addressing in this work is whether permutations in a given permutation class can be constructed by composing permutations from two or more smaller classes. We formalise this as follows. A permutation class is said to be composable from classes if . A class is composable, if it is composable from its proper subclasses . A class is composable, if it is composable for some . Using this terminology, our goal is thus answering the question whether a given permutation class is composable.
Clearly, for every class we have . For an infinite class we have either , which implies , or , which implies and . Restricting ourselves to proper subclasses in the definition of a composable class is motivated by these trivial inclusions.
We begin the exploration of composability by proving the following result which implies that unlike splittability, composability for does not imply composability.
Theorem 3.4.
Let be an infinite permutation class such that . Then is not composable for any positive integer .
3.3 Properties of symmetries
In this section we explore how composability is preserved under some of the usual symmetrical maps.
For a permutation of length we define to be the reverse of , i.e. , and to be the complement of , i.e. . For a permutation class we define the inverse class , the reverse class , and the complementary class .
It is clear that all these class operators are involutory, i.e. , and . The following simple lemma describes how these operators relate to composition.
Lemma 3.5.
Let be permutation classes. Then

,

and ,

.
Proof.
(a): If for every , then by the property of inverse elements in a group we have .
(b): Let and be permutations of order . By definition for every . Therefore and for every .
(c): Apparent from (b). ∎
Using this lemma we derive several composability criteria for symmetries of a given class, the first of which requires no further proof as it is an immediate consequence of Lemma 3.5.
Corollary 3.6.
Let be a permutation class. Then the following statements are equivalent:

is composable,

is composable,

is composable.
The case of the reverse and complementary operators is more complicated and requires additional assumptions.
Lemma 3.7.
If is a composable class and , then both and are composable.
Proof.
Let be composable from its proper subclasses . Then
It holds that , so we have
Clearly and since , we have , so the proper subclass criterion is met and is therefore composable. Analogously we show that
4 On permutations avoiding a decreasing sequence
Recall that is the class of permutations merged from increasing sequences, or equivalently those avoiding a decreasing sequence of length . In this section, we prove that is 2composable and show several examples of how can be composed from two proper subclasses.
4.1 Vertical and horizontal merge
Let be any permutation classes. We define the vertical merge of these classes as the class of permutations that can be written as a concatenation of (possibly empty) sequences such that is orderisomorphic to a permutation of . We write this class as . In addition, if , we let denote the class . Similarly we define the horizontal merge of these classes as the class of permutations that can be written as a merge of (possibly empty) sequences such that each is orderisomorphic to and every element of is smaller than every element of for . Note that this implies that each uses a set of consecutive integers. We let denote the horizontal merge of classes and if we write .
Alternatively, we can observe that resp. if and only if its plot in can be separated by vertical resp. horizontal lines into at most parts , th of them containing a sequence orderisomorphic to a permutation in (see Figure 4), hence the names of the classes.
In addition we define and for future convenience. We continue by observing an important connection between the horizontal and vertical merge.
Lemma 4.1.
Let be any permutation classes. Then
Proof.
If , we have that such that is orderisomorphic to . For every , contains a set of consecutive integers on indices and the sequence at these indices is orderisomorphic to .
The opposite inclusion is equally straightforward. ∎
When composed with any other class , the classes , and can be viewed as a unary operator transforming in a specific way. We formalise this approach in the following lemma.
Lemma 4.2.
Let be an arbitrary permutation class. Then

is precisely the class of permutations which can be obtained from a permutation of by dividing it into at most contiguous subsequences and interleaving them in any way,

is precisely the class of permutations which can be obtained from a permutation of by dividing it into at most subsequences and concatenating them,

is precisely the class of permutations which can be obtained from a permutation of by dividing it into at most subsequences and interleaving them in any way.
Proof.
Let , , and .
(a): Consider the permutation . Then is merged from (possibly empty) sequences of consecutive integers . Now we define sequences such that and for every and . Every is a contiguous subsequence of and at the same time is merged from .
On the other hand, if a permutation is obtained from by dividing it into contiguous subsequences and merging them in some way, we define sequences such that is the sequence of indices of the elements of in . Then by definition for any suitable and , and since we divided into contiguous subsequences, each is a sequence of consecutive integers. Now consider the permutation created by replacing the subsequence by the sequence in for every . Then is merged from , which are sequences of consecutive integers, therefore . At the same time, for any there are indices and such that , where the last equality holds because we replaced by when constructing from . Therefore .
(b): Consider the permutation . The permutation is formed by concatenating increasing sequences . Define sequences such that and . Each is a subsequence of and at the same time .
On the other hand, if a permutation is obtained from by dividing it into subsequences and then concatenating them, we define sequences such that is the sequence of indices of elements of in . Thus every is an increasing sequence and . Consider a permutation created by replacing the subsequence by the sequence in for every . Since is a concatenation of , we get that is a concatenation of and thus . Also, for any there are indices and such that , where the last equality holds because we replaced by when constructing from . Therefore .
(c): The proof is similar to the proofs of (a) and (b). ∎
4.2 Composability results
Using the machinery introduced in the previous section we now prove a key lemma which we will use to show several composability results.
Lemma 4.3.
Let be arbitrary permutation classes. Then
Proof.
Consider a permutation and divide it into sequences such that is isomorphic to a permutation from . The permutation then lies in , which together with Lemma 4.2(a) implies . ∎
By reformulating the previous statement we immediately get the following.
Corollary 4.4.
Let , and be permutation classes such that . Then .
Using what has already been shown in this section it is now elementary to show that is 2composable.
Theorem 4.5.
The class is 2composable for every . In particular, .
Proof.
We proceed by proving a result in some sense opposite to that of Lemma 3.2, namely we show that may be constructed from smaller using composition.
Theorem 4.6.
for all integers .
Proof.
Consider a permutation , merged from two sequences and such that is merged from increasing sequences and is merged from increasing sequences . Let be the increasing sequence created by sorting the elements of . Consider a permutation created by merging the sequences and so that and form a single increasing sequence. Clearly and sequences are subsequences of , since they are increasing and therefore were not affected by sorting .
According to Lemma 4.2(c) the class contains all permutations we can create from by dividing it into subsequences and merging them in any way. It is therefore enough to find a way to divide into subsequences which can be merged into . A simple choice of such subsequences is . ∎
This theorem raises the question whether we could construct a bigger class from given and .
Question.
Given positive integers and , what is the largest integer such that ?
5 Classes of layered patterns
In this section we cover classes of permutations which can be written as a sum or as a skew sum of increasing or decreasing permutations. Among these classes we provide infinitely many examples of composable classes as well as several examples of classes which are not composable.
Let denote the increasing permutation of order and denote the decreasing permutation of order . A permutation is layered if it is a sum of decreasing permutations which are then called layers. We let denote the class of all layered permutations. We let denote the class of permutations which are sums of at most layers. The complement of a layered permutation is clearly a skew sum of increasing permutations and we call such a permutation colayered. The class consists of precisely the colayered permutations.
We start by proving that is not composable using a counting argument. As it turns out, proper subclasses of are asymptotically too small to build the entire class using composition.
Theorem 5.1.
The class is not composable.
Proof.
Suppose that such that for every . Each of these subclasses avoids at least one permutation of . In other words for every there is a such that . Considering a sufficiently large so that for every we get that for every , in other words every permutation in these subclasses has one of its two layers shorter than . It follows that for a fixed integer there are at most permutations of order in any , therefore there are at most permutations in . But contains permutations of order for any , therefore we obtain a contradiction by choosing . ∎
The number of permutations of order in is linear in while any proper subclass contains only constantly many permutations of fixed order. We can use the same approach using the asymptotic jump from polynomial to exponential functions to show that a different class of permutations cannot be composable. Namely, let be the class of layered permutations with layers of size 1 or 2.
Theorem 5.2.
The class is not composable.
Proof.
Suppose is composable from of its proper subclasses . We choose a permutation from for every and we select large enough so that every chosen permutation is contained in . Then if , we get that . Every permutation in contains fewer than layers of size 2, otherwise it would contain . Clearly there are at most permutations of that have order and exactly layers of size 2. Therefore contains at most permutations of order and the composition then contains at most permutations of order , which is a number polynomial in . As mentioned in [10, Chapter 4], the number of permutations of order of is counted by the Fibonacci numbers which grow exponentially, therefore there is large enough so that has more permutations of order than .
Note that this result also follows immediately from the theorem of Kaiser and Klazar ([7, 3.4]), which states that if the number of permutations of order in a permutation class is less than the th Fibonacci number for at least one value of , then it is eventually polynomial in . This implies that every class counted by the Fibonacci numbers is uncomposable. ∎
The argument used in the proofs above cannot be used for , so we need a different approach to show that this class too is not composable. We will make use of the following property of .
Lemma 5.3.
is a subgroup of for every , i.e. it is closed under composition.
Proof.
In this proof, we consider an additive group structure on the set with the neutral element and an operator defined as
First we prove that by itself is a subgroup of . Observe that contains exactly permutations such that there is a shifting number with for every . Indeed, if then for any we have and conversely if for every then . Now for two permutations with shifting numbers respectively we have for any , therefore since it has a shifting number .
It trivially holds that . Considering it remains to distinguish the following four cases:

and , then by the discussion above,

and , then ,

and , then ,

and , then . ∎
Theorem 5.4.
The class is not composable.
Proof.
Suppose that such that for any . Using the same initial argumentation as in the proof of Theorem 5.1 we get that there is an such that , meaning that every permutation of can be composed from permutations having at least one of the three layers shorter than .
Let for and . We now claim that it is possible to remove at most elements from to obtain a twolayered or a twocolayered permutation. We will prove this by induction on . The case is easy since avoids , so it has a layer of length shorter than whose removal creates a twolayered pattern.
For let and . Let all these permutations have the order . By the induction hypothesis, there are indices such that and restricted to these indices has the twolayer or the twocolayer pattern. Also there are indices such that and restricted to these indices forms the twolayer or the twocolayer pattern.
Let us now restrict the function to the set whose size is at least . Then both and are still twolayer or twocolayer patterns, which implies the same for their composition according to Lemma 5.3. Therefore restricted to forms a twolayer or twocolayer pattern and which completes the induction step.
Consequently, any permutation of order in contains a twolayered or a twocolayered pattern of size at least . But choosing and considering the permutation we obtain a contradiction. ∎
If we allow more than three but still constantly many layers, we always get a composable class.
Theorem 5.5.
The class is composable for every .
Proof.
We will show that .
If of order has fewer than layers, then . Otherwise has at least 4 layers and has the form for some positive , , , . Since for every layered we have it is not hard to check that
The situation is represented in Figure 6. ∎
This theorem raises the question whether could be 2composable for . Our work from Section 3 quickly determines that this is not the case.
Proposition 5.6.
for is not 2composable. In particular, it is not composable for any even number .
Proof.
Since is an infinite class which does not contain the statement directly follows from Theorem 3.4. ∎
We have now covered the classes for all . It remains to consider the class , which we show to be uncomposable. Before we proceed with the proof, we introduce an additional useful concept. We call a subsequence of a permutation a block if is either an increasing or a decreasing contiguous subsequence of consecutive integers. We then call a block if it is a concatenation of at most blocks (see Figure 7).
Lemma 5.7.
Let be a block and let be an block. Then is a block.
Proof.
Choose a block of at indices . Then the sequence
is a contiguous subsequence of either