Permutation Testing
Fast property testing and metrics for permutations
Abstract
The goal of property testing is to quickly distinguish between objects which satisfy a property and objects that are far from satisfying the property. There are now several general results in this area which show that natural properties of combinatorial objects can be tested with “constant” query complexity, depending only on and the property, and not on the size of the object being tested. The upper bound on the query complexity coming from the proof techniques is often enormous and impractical. It remains a major open problem if better bounds hold.
Maybe surprisingly, for testing with respect to the rectangular distance, we prove there is a universal (not depending on the property), polynomial in query complexity bound for twosided testing hereditary properties of sufficiently large permutations. We further give a nearly linear bound with respect to a closely related metric which also depends on the smallest forbidden subpermutation for the property. Finally, we show that several different permutation metrics of interest are related to the rectangular distance, yielding similar results for testing with respect to these metrics.
AMS subject classification: 68W20, 68R05, 05D40, 05A05, 68R15
1 Introduction
Traditionally, algorithms that run in time polynomial in the input size were considered fast. However, as the desired input size has increased, this notion of fast is sometimes insufficient. Some examples include in algorithmic problems on networks like the internet or the brain, or in ranking websites for search algorithms, in which the structures being studied have billions of elements and are often not well understood. In order to handle such large structures, sublinear time algorithms are desired. One would not expect for such algorithms to be able to determine properties of the structures with certainty. This is where property testing comes in.
The goal of property testing is to quickly distinguish between objects which satisfy a property and objects that are far from satisfying the property. The study of this notion was initiated by Rubinfield and Sudan [28]. Subsequently, Goldreich, Goldwasser, and Ron [17] began the investigation of property testers for combinatorial objects. There are now several quite general results in this area which show that properties can be tested with “constant” query complexity, depending only on and the property, and not on the size of the object being tested. A property is onesided testable if there is a function and a randomized algorithm with query complexity which, on an input which has property , correctly outputs that the object has property , and on input that is far from satisfying , correctly outputs with probability at least that the object is far from satisfying . Property is twosided testable if it correctly outputs in either case with probability at least . Note that if an input neither satisfies nor is far from , it has no guarantee on the output. An exemplary result in this area, due to Alon and Shapira [4], states that every hereditary graph property is onesided testable. However the query complexity bound it gives is at least of wowzertype in , which is one level higher in the Ackermann hierarchy than the tower function, as it uses the strong regularity lemma. Enormous bounds like this on the query complexity are typical of many general results in this area and is a major drawback as the bounds are impractical. Besides regularity methods, which often give towertype or worse estimates, some of the other results in this area are established using compactness arguments and thus yield no bound. There are some examples of progress on quantitative bounds for property testing. See, for example, [1, 3, 5, 6, 13]. However, it remains a major open problem if better bounds hold for the various property testing results.
In this paper, we address this problem for permutations. To properly understand our results, it is important to try to first determine what is a good notion (or notions) of distance between combinatorial objects. This is because we need to understand which metric we are using when we say that two objects are far from each other in this metric. For graphs, the edit distance, which is the fraction of pairs which one needs to add or delete edges from in order to turn one graph into the other, is quite natural, and it is not surprising that it is the most studied with regards to graph property testing. For permutations, there are now several important metrics that naturally arise in ranking problems in statistics. See, for example, the book by Diaconis [8]. This makes it less clear for which metrics permutation property testing should be done with respect to.
An early paper of Cooper [7] develops an analogue of Szemerédi’s regularity lemma for permutations^{1}^{1}1This regularity lemma was subsequently improved upon by Hoppen, Kohayakawa, and Sampaio [19] and with much better quantitative estimates by Fox, Lovász, and Zhao [14]. and deduces a permutation removal lemma. Typically, removal lemmas are equivalent to saying that certain properties of combinatorial objects are onesided testable with respect to some metric. However, Cooper’s permutation removal lemma does not give such a metric and so does not actually translate to a result in property testing.
Since Cooper’s work, there are now two different notions of distance which have been studied for permutation property testing, the rectangular distance (or cut distance), and Kendall’s tau distance. The rectangular distance for permutations is an analogue of the Frieze–Kannan cut distance for graphs, which has played an important role in the development of the weak regularity lemma, graph limits, and approximation algorithms for graphs. See, for example, the book by Lovász [25] and the paper [14].
A permutation of length is a bijection from to itself. We can represent as points in the plane with the coordinates of the point being . The rectangular (cut) distance between two permutations of length is defined to be
where the maximum is over all subintervals of . Thus, the rectangular distance is the normalized maximum discrepancy in rectangles between the number of points of the form and the number of points of the form . While the rectangular distance is defined globally, through a counting lemma, it can be shown that two permutations have small rectangular distance if and only if they have roughly the same densities of all small subpermutations. This is an analogue of similar results for graphs; see [25] and [20] for details on these results for graphs and permutations, respectively.
A copy of a permutation of length in a permutation of length is a subsequence of that has the same order type as . That is, a copy of in is a sequence of integers such that if and only if . If contains a copy of , then we say that is a subpermutation of . If does not contain a copy of , then we say that avoids or is free. A permutation property is just a family of permutations. A permutation property is hereditary if it is closed under subpermutations, that is, if every subpermutation of a permutation in is also in . Hoppen, Kohayakawa, Moreira, and Sampaio [21] proved that every hereditary permutation property is onesided testable with respect to the rectangular distance. Their proof uses a compactness argument and does not give any bound on the query complexity. They also conjectured a stronger result that hereditary permutation properties are strongly testable, i.e., can be tested with respect to Kendall’s tau distance: for two permutations of length ,
Alternatively, Kendall’s tau distance between can also be defined as the minimum number of adjacent transpositions (i.e., swapping the coordinates of the points and ) required to turn into , and normalized by dividing by . This conjecture is stronger because the rectangular distance is small if Kendall’s tau distance is small, but the converse is not true. For example, for two random permutations of length almost surely have rectangular distance , but Kendall’s tau distance . The nice conjecture of Hoppen et al. was verified by Klimošová and Král’ [23]. However, even for the property of being free for some fixed permutation , the bound on the query complexity is enormous, of Ackermanntype in , and hence not primitive recursive.^{2}^{2}2In the conference version of [23], it is incorrectly stated that the proof gives a double exponential bound for testing freeness. In another work [15], we prove that there is a polynomial in bound for onesided testing freeness, where the exponent depends on . The result generalizes to show that hereditary properties are onesided testable with respect to Kendall’s tau distance, and for typical properties, it gives a polynomial bound.
Another important permutation metric is Spearman’s footrule distance. For two permutation , their Spearman’s footrule distance is
A fundamental result of Diaconis and Graham [10] states that
Thus Kendall’s tau distance and Spearman’s footrule distance are within a factor of two, and so testing with respect to Kendall’s tau distance is essentially equivalent to testing with respect to Spearman’s footrule distance.
Maybe surprisingly, for testing with respect to the rectangular distance, we prove that there is a universal (not depending on the property), polynomial in query complexity bound for twosided testing of hereditary properties of sufficiently large permutations. One drawback of the definition of the rectangular distance is that it is global, whereas with Kendall’s tau distance, we see that we can make sequential local moves in order to get from one permutation to the other. We study a new distance for permutations, whose general form is called earth mover’s distance in statistics. It turns out to be quite natural and defined based on local moves, yet we prove it is small if and only if the rectangular distance is small.
Definition 1.1 (Planar tau distance).
We say that is obtained from by a planar simple transposition, if there is an integer such that is the same as except , or . The planar tau distance between two permutations of length is defined as times the minimum number of planar simple transpositions needed to transform into .
The factor in the definition is the proper normalization in order to guarantee that this distance is always at most one. We can also define the planar tau distance in terms of Kendall’s tau distance, since the planar tau distance allows for adjacent transpositions in both the horizontal and vertical directions. Thus
It is also useful and interesting to define a planar analogue of Spearman’s footrule distance.
Definition 1.2 (Earth mover’s distance (for permutations)).
The earth mover’s distance between two permutations of length is defined as
where the minimum is over all bijections .
This is the sum of distances between a point in and the point in that it maps to under . We can treat as a permutation. Thus the earth mover’s distance is equivalent to
Taking to be the identity permutation id, we obtain and . Thus, the planar metrics are at most their classical analogues.
The earth mover’s distance is a special case of more general metrics that have been extensively studied before in other contexts. It is called the earth mover’s distance or the MongeKantorovich norm in computer science and was first introduced by Monge [27] in 1781 as a central concept in transportation. It is a natural way of measuring the similarity between two digital images (see, e.g., [29]). In the case of permutations, the digital image has a single one in each row and column. In analysis, it is known as the Wasserstein metric. It is also a special case of the minimum weighted matching problem (see, e.g., [30]).
By the definitions of the planar tau distance through Kendall’s tau distance and the earth mover’s distance through Spearman’s footrule distance, and by the DiaconisGraham inequality, we therefore get the following planar analogue of the DiaconisGraham inequality, which was pointed out by Diaconis [9].
Corollary 1.3.
The following result shows that the planar tau distance is small if and only if the rectangular distance is small. Together with the previous result, it shows that testing with respect to any of these metrics is the same up to a quadratic change in the testing parameter .
Theorem 1.4.
For any two permutations of length , we have
Thus the new planar metrics share many of the advantages of both the rectangular distance and Kendall’s tau distance. This result and other results relating permutation metrics are proved in the full version of this paper.
Definition 1.5 (Blowup of a permutation).
A permutation is a blowup of another permutation if and only if we can find positive integers with the following property. If satisfy that and with , then if and only if .
Intuitively, it means that each point blows up into a block The th block is of size . Notice that we did not specify the permutation within each block. Figure 1 is an example of a blowup of a permutation .
A blowup of is a blowup of where each block has size . We next define some important parameters for the property related to blowups of permutations. Since we work with a single property , we leave it out of the notation to make the notation simpler.
Definition 1.6 (Blowup parameter for ).
Given a permutation , let be the minimum positive integer (if it exists) such that no blowup of is in . If no such integer exists, i.e., for every there is a blowup of which is in , then we define . Given a positive integer , let be the maximum of over all permutations of length for which . If no such exists, i.e., for all permutations of length , then we define .
Note that if is hereditary and has a forbidden subpermutation of length at most , then it has a forbidden subpermutation of length , and is a blowup of itself and is not in , which implies that is finite in this case.
We give a nearly linear bound for the query complexity for testing which depends on the smallest forbidden subpermutation for the property. We first need an important definition. A matrix contains another matrix if there is a submatrix of of the same size as such that for every one entry of , the corresponding entry of is a one. For a permutation , the extremal number is the maximum number of one entries in a matrix with entries or which does not contain the permutation matrix of . Füredi and Hajnal [16] conjectured that for each permutation , the limit exists. Klazar [22] proved that the FürediHajnal conjecture implies the wellknown StanleyWilf conjecture. A celebrated result of Marcus and Tardos [26] verifies the FürediHajnal conjecture, and hence the StanleyWilf conjecture. It shows that . The first author [12] improved the bound to , and showed that for almost all permutations of a given order. The constant is known as the FürediHajnal constant of . The fact that is superadditive in implies that for all .
Theorem 1.7.
For each proper hereditary permutation property and , let , where is the FürediHajnal constant of a smallest forbidden subpermutation for . Let and . There is a twosided tester for with respect to the planar tau distance of query complexity for permutations of size at least .
The tester works as follows. Let be as specified in Theorem 1.7. For a permutation , we pick a subpermutation of of size uniformly at random; call it an sample. That is, we pick a subset of size uniformly at random, and is the subpermutation of induced on . If for all integers , there exists a blowup of , our algorithm outputs “ is in ”. If there exists an integer such that no blowup of is in , our algorithm outputs “ is not in ”. We remark that the constant dependence can sometimes be improved by using extremal properties of the family of forbidden subpermutations rather than just the smallest forbidden subpermutation.
The next theorem gives a universal quadratic bound (not depending on the property) on the query complexity for testing sufficiently large permutations.
Theorem 1.8.
For each hereditary permutation property and , let and . There is a twosided tester for with respect to the planar tau distance of query complexity for permutations of size at least .
Theorems 1.7 and 1.8 are both with respect to twosided testing. For onesided testing, we can still get reasonably good bounds, as stated in Theorem 1.9, by showing that very likely a permutation has the property that it is close to a blowup of a random subpermutation and a somewhat larger random subpermutation very likely contains a blowup of . These bounds are polynomial in as long as the blowup parameter for is bounded above by a polynomial in . In particular, for almost all permutations of length , we get a universal bound for testing freeness. This is because almost all permutations of length have no subinterval of length three whose image is an interval of length three, and hence is at most three for the property of being free.
Theorem 1.9.
Finally, we show that several different permutation metrics of interest are closely related to the cut metric, yielding similar results for testing with respect to these metrics.
We often consider the input permutation of length as a collection of points in the unit square . See Figure 2 for an example.
If we are testing for a hereditary permutation property , and we know that the following hold:

there is a permutation (or even if large blowups of are not in ),

there are rectangles whose horizontal intervals are disjoint and whose vertical intervals are disjoint,

each of the rectangles contains a significant fraction of the points corresponding to , and

if we pick one point from each of the rectangles, then we form a copy of ,
then a large sample of points from will likely have many points in each rectangle and thus contain a large blowup of and certify that . This simple idea is very important for our various property testing algorithms, and is demonstrated in Figure 3.
2 Equivalence between different metrics on permutations
We next define several different metrics between permutations of the same length. Each of the metrics we use here is normalized such that the maximum distance between two permutations cannot exceed 1. We then study properties of these metrics and the relationships between them. Each of these metrics has the property that if two permutations have small distance in one metric, they also have small distance in the other metrics. Furthermore, two permutations have small distance in any of these metrics is equivalent to having, for each small permutation , roughly the same density of as a subpermutation.
Recall that we sometimes view the permutation of length as the collection of points with or, normalized, as the collection of points . It should be clear from the context which is used. Viewing a permutation as a collection of points in , given any rectangle , let be the number of points of inside (including the boundary).
Definition 2.1.
Let be permutations of length .

Rectangular Distance, or Cut Distance. The rectangular (cut) distance between is defined as
where the maximum is over all closed intervals and .

Dyadic Distance. A closed interval is called a dyadic interval if there exist positive integers such that . The dyadic distance between is defined as
where the maximum is over all intervals and and are dyadic intervals.

Square Distance. The square distance between is defined as
where the maximum is over all intervals and and . Thus is a square in .

Dyadic Square Distance. The dyadic square distance between is defined as
where the maximum is over all dyadic squares , which means are dyadic intervals and .

Earth Mover’s Distance. The earth mover’s distance between is defined as
where the minimum is over all permutations of length . This is the sum of distances between a point in and the point in that it maps to under .

Planar tau Distance. A permutation can be obtained from a permutation of the same length by a planar simple transposition if there exists an integer such that is the same as except either , , or , . The planar tau distance between , denoted as , is the minimum number of planar simple transpositions required to transform into , and then normalized by . Restated, the planar tau distance between and is the normalized (so divided by ) minimum number of consecutive row or column swaps needed to obtain the permutation matrix of from the permutation matrix of . Recall that the permutation matrix of a permutation of length is a matrix with for each , and the remaining entries are .
We first make several remarks about these metrics. For two permutations of length , their planar tau distance is at most their Kendall’s tau distance as Kendall’s tau distance is defined in the same way but is more restrictive on the allowed moves (only allowing consecutive column swaps). Similarly, their earth mover’s distance is at most their Spearman’s tau distance as taking to be the identity permutation of length , we obtain Spearman’s footrule distance. Summarizing, the planar distances are at most their classical variants. We also recall Corollary 1.3, which shows that . Thus the planar tau distance and earth mover’s distance are within a factor two of each other.
Also, it is worth discussing the complexity of computing these metrics for two permutations of length . The rectangular distance is defined as the minimum over choices of pairs of intervals, while the Square distance is only over choices of pairs of intervals, and the Dyadic distance and the dyadic square distance is defined only over choices of pairs of intervals. Hence, the dyadic distances appear to be considerably faster to determine exactly.
If we only want to approximate these distances, we can do a much faster computation. Two squares whose horizontal and vertical intervals differ in endpoints by at most in each coordinate differ by at most in the fraction of points of the permutation in the rectangle for a given permutation. Hence, by considering only multiples of as possible endpoints, we can approximate the rectangular distance within using only rectangles. Similarly, we can approximate within the square distance by using at most squares, and the dyadic distance or the dyadic square distance within using only dyadic rectangles. Thus, these distances can be determined or approximated rather quickly, with the dyadic distances being the fastest to approximate.
On the other hand, while very natural, the earth mover’s distance is defined as the minimum over permutations, which requires a huge computation. Similarly, it is unclear if there is an efficient algorithm for computing the planar tau distance efficiently. However, it is possible to efficiently approximate these planar distances. By partitioning into boxes of side length about , just using the information about the fraction of points in each box, by considering roughly the fraction of points in each box that match up to points in other boxes, it is possible to show that one can compute the planar tau distance and the earth mover’s distance each within in time which is a function only of .
We next prove Theorem 1.4, which states that for any two permutations of length , we have
Proof of Theorem 1.4.
The earth mover’s distance between two permutations is the normalized minimum, over all permutations , which is the same as the normalized minimum over all matchings between points in and , of the sum over all matched pairs of the taxicab () distance between the two points that are matched.
We first show that if , then . Since , we can find a rectangle such that there are at least more points in this rectangle in than , or vice versa. Without loss of generality, we assume there are more points in than in . Therefore for any bijection that maps points in to the ones in , at least points in have to map to the points of that are outside the rectangle .
Assume . Let , and . Thus , and the difference between the two rectangles have margin at most . Figure 4 illustrates these two rectangles. Since is a permutation, there are at most points of that are inside the region . Therefore there are at least points of inside that have to map to points of outside . However for these points of , the distance between it and the point in it maps to is at least . Therefore . Thus we proved .
We now show that if , then . We may assume that as otherwise these distances are all . We will find a bijection such that . To see this, we partition into squares, each of side length , where with We will define which matches points of to points in recursively in rounds. In round , we match up as many points of to points in as possible that lie in the same dyadic square of side length . In round , we match up as many not yet matched points of to not yet matched points in as possible that lie in the same dyadic square of side length . For each pair of points matched in level , their distance is at most . In the last round , the remaining unmatched points in necessarily get matched to the unmatched points in as they all lie in the square of side length .
As the discrepancy in the number of points in and in in any square is at most , after round , the number of unmatched points is at most . Thus round matches at most pairs of points, each such pair of points has distance at most . Hence, the sum of the distances of the pairs matched at level is at most . Summing over all gives a total sum of distances in less than . Also, there are at most points in that get matched to a point in in the same dyadic square of side length so that there distance is at most . The sum of these distances is at most . Thus, the earth mover’s distance between and is at most
where the last inequality uses . Thus we have proved . ∎
The next lemma shows that, up to two logarithmic factors, the rectangular distance is the same as the dyadic distance, which is much faster to compute or approximate. It is easy to give a construction showing that the upper bound is sometimes tight, and reverse engineering the proof gives a construction showing that the lower bound is sometimes tight up to an absolute constant factor.
Lemma 2.2.
Proof.
It is clear from the definition that .
We thus need to show the other inequality. Given , we will show . Since , by definition, there exist intervals such that . We want to tile most of by dyadic rectangles; and thus one of them has large difference between the number of points in and . We do this by first covering most of the interval by dyadic intervals, shown in Figure 5.
We know must contain a dyadic interval such that and . Removing from , we are left with at most two other intervals and with coming before and each of length at most . In the next level, notice that since is a dyadic interval, each have one endpoint being dyadic (i.e., of the form for some integers ). Thus again we can find a dyadic interval with the same right endpoint as and ; again is another interval with one endpoint dyadic. We can similarly find a dyadic interval with the same left endpoint as with and such that is an interval with one endpoint having a dyadic coordinate. We know . Therefore removing from leaves us with at most two remaining intervals with each having a dyadic endpoint and their total length is at most . We repeat this process. In each step, we find at most two new dyadic intervals and removing them further from leaves us with two remaining intervals each with one dyadic endpoint and each of these remaining intervals has length at most half of the length of the intervals they came from in the previous step. Thus, we can partition into at most dyadic intervals and at most two intervals such that . This is because in the first step we used one dyadic interval , and in each further step, we picked out two dyadic intervals, and the total number of steps used is at most . Similarly, we can partition into at most dyadic intervals and at most two intervals such that .
Therefore we can cover most of the rectangle by at most dyadic rectangles except for four rectangles . This is illustrated in Figure 6.
However, notice that for any rectangle , we have simply because is a permutation, and the same applies to . Therefore . Similarly, . Thus . Let , and . Since and , we have that
However, we have shown that is covered by at most dyadic rectangles; therefore there must be a dyadic rectangle such that
This implies ∎
The next lemma relates the rectangular distance and the square distance. It is easy to see that the lower bound is sometimes tight, and reverse engineering the proof shows that the upper bound is sometimes tight up to an absolute constant factor.
Lemma 2.3.
Proof.
It is clear from the definition that .
We thus need to show the other inequality. Given , we will show .
As , there exist intervals such that . We want to find a square in in which there are many more points from than from , or vice versa.
Assuming , then we partition into intervals of length and a remaining interval with length . Thus the rectangle is partitioned into squares each of side length and a remaining rectangle of size . An example of this step can be seen as the three largest squares in Figure 7. And we do the same partitioning procedure for the remaining rectangle by covering the longer side (now it is the interval of length ) by as many intervals of length as possible. We repeat this process, until the remaining interval has length at most . In Figure 7, we repeat this step for four rounds (corresponding to squares of four different sizes), and stop at the smallest white rectangle since its shorter side has length at most .
We want to bound the number of squares we obtained. Notice that for each rectangle of size , assuming , after cutting it into as many squares of size as possible, the remaining rectangle has size , where satisfies for some positive integer . Since and , clearly we have . We then partition the rectangle into squares of size , and a remaining rectangle is of size where . By the same argument, again we have . We repeat the process, obtaining squares of side lengths until the side length of some rectangle is no more than . Say it stops after subdividing a rectangle of size into many squares, and the remaining rectangle is of size