Quicksort with median of medians is considered practical
The linear pivot selection algorithm, known as median-of-medians, makes the worst case complexity of quicksort be . Nevertheless, it has often been said that this algorithm is too expensive to use in quicksort. In this article, we show that we can make the quicksort with this kind of pivot selection approach be efficient.
Quicksort[2, 1, 3] is one of the most efficient and widely used sorting algorithms. The major drawback of quicksort is that worst case time complexity of naive implementations of quicksort is with input size . In order to avoid this worst case behavior, a variant of quicksort, so-called introsort 111 In many textbooks, it is written that this method is invented in ref.  in 1997. We found, however, that the same technique has already appeared in ref. , which is published in 1989. In this book, the author calls this method fail-safe quicksort. , is often used.
There exists another solution to keep the worst case complexity of quicksort . This is the use of median-of-medians or Blum-Floyd-Pratt-Rivest-Tarjan (BFPRT) algorithm — the pivot selection algorithm in the linear median finding algorithm. Despite the theoretical importance of this scheme, because of its somewhat large constant factor, this method have been considered impractical to use as a pivot selection algorithm in quicksort.
In this article, contrary to the above widespread view, we show that using BFPRT in quicksort is not inefficient, and moreover, considered practical. Indeed, we find that quicksort with median-of-medians shows comparable performance to the well-known optimized quicksort implementations[7, 8] for random sequence. We also show that the technique for improving performance is not only for the BFPRT but also available for other pivoting methods.
The main idea of this article is thinning out the input elements of each pivot selection. The outline of this idea is given in Algorithm 1. Here we thin out the input array and apply the BFPRT on only of . This BFPRT based pivot selection method guarantees the total time complexity of sorting to be , even in the case of . We can show this briefly as follows. BFPRT guarantees that of input elements are lesser than the output pivot, and other are greater. Thus, at least in with length are lesser and other are greater. This yields that one partition operation reduces the length of from to even in the worst case. The total time complexity of sorting is then asymptotically (see Ref. ).
The constant factor in the worst case of quicksort with the above method is estimated as follows. If the complexity follows the form for sufficiently large , the recurrence relation becomes
here and is the entropy function. The constant represents the cost of the pivot selection and the partitioning. If the bound of the complexity of BFPRT is , can be bound as because the partitioning requires comparisons222Because there are so many variants of BFPRT algorithms, we do not specify the value of in this article.. The upper bound of is then . If then , for example. The lower bound of can be estimated in the same way: .
2.2 Pseudo Median of
The idea of thinning out can be applicable for another pivot selection scheme, as long as it takes linear time. For example, median-of-three method and median-of-three-medians-of-three (pseudo-median-of-nine or Tukey’s ninther)[7, 11] are widely used pivot selection method. As the extreme case of these strategies, pseudo-median of () can be considered. Here we introduce another type of thinning method upon the pseudo-median of , as described in Algorithm 2.
The worst case complexity of Alg. 2 is estimated as the following. If and the length of the target array is , Alg. 2 guarantees that at least of the elements are lesser than the selected pivot and other are greater. Thus, for a large array with length , Alg. 2 guarantees that of the elements are greater than the pivot and are lesser. The depth of recursion for the worst case is then estimated as
Therefore, the worst case bound of quicksort with Alg. 2 is bound by . Because the averaged required comparison in a pivot selection for an array with length is , the lower bound of the complexity of quicksort with this method in the best case is estimated as .
We carried out a numerical experiment to estimate the coefficients of the quicksort with Alg. 1 (t-BFPRT) and Alg. 2 (t-PMed). We used the standard single pivot partitioning and the original BFPRT algorithm. For simplicity, we did not change to another sorting algorithm such as heapsort or insertion sort to sort small subarrays. We also measured quicksort with random pivot picking (Hoare’s original method) (Rand), median-of-three (Med3), and pseudo-median-of-nine (PMed9) for benchmarking our strategy. We used random sequences of distinct integers as target arrays. We executed 100 times of sorting by each scheme, obtained the mean and unbiased variance of required count of comparisons, and calculated their coefficients via least squares fittings.
Figure 1 and 2 show the number of comparison of each pivoting method. We can see that the performance of t-BFPRT with or t-PMed with is not good because of its expensive pivot selection procedure. On the other hand, t-BFPRT or t-PMed with exhibits excellent result; both method beat that of Med3. Table 1 shows the fitting result to the function . We can see that t-BFPRT and t-PMed with large clearly outperform Med3 and exhibit comparable result to PMed9.
Figure 3 shows the dependence of the coefficients on . In both methods, the larger is, the more improved the asymptotic behavior is. Because the larger makes the upper bound of the comparison count worse, within the range is probably the best choice in this configuration.
Note that the above discussion only considers the number of comparisons and the execution time is another question. As for t-PMed, there is no swapping in pivot selection phase. Therefore, unless the effect of the memory cache and cost of function calls are not negligible, the execution time shows the same tendency of the comparison count. On the other hand, t-BFPRT involves data modifications in the pivot selection. In our environment, it showed the similar tendency of the comparison count. However, further accurate investigations should be required anyway.
In this article, we show that the use of median-of-medians as a pivot selection algorithm in quicksort is not always inefficient; it works very well on random sequences, but still requires only comparisons in the worst case. We also show that the technique of thinning out can also be usable for other pivoting method, and can improve the worst case behavior with keeping good performance for random input.
There can be many other methods to thinning out the input of pivoting algorithm. It is easy to make a bad input for Alg. 1 (setting larger elements at the multiple of and last elements), but for another thinning method making a bad input may not be as easy as this. Also, the methods in this article perform well on neither increasing nor decreasing sequence because of its naive behavior for small subarrays. To fix this, equipping adaptive features (e.g., median-of-three for small input (typically for t-BFPRT and for t-PMed) and the main method for large input) is effective.
Because of its simple concept and efficiency, we believe that the presented approach can be useful in real world applications.
-  C. A. R. Hoare, “Algorithm 63: Partition”, Commun. ACM 4, 321 (1961).
-  C. A. R. Hoare, “Algorithm 64: Quicksort”, Commun. ACM 4, 321 (1961).
-  C. A. R. Hoare, “Quicksort”, Comput. J. 5, 10–16 (1962).
-  D. R. Musser, “Introspective Sorting and Selection Algorithms” , Softw: Pract. Exper. 27, 983–993 (1997).
-  K. Ishihata, Algorithms and data structures, Iwanami Software Science Series 3 (Iwanami, 1989).
-  M. Blum, R. W. Floyd, V. Pratt, R. L. Rivest, and R. E. Tarjan, “Time bounds for selection”, J. Comput. Syst. Sci. 7, 448–461 (1973).
-  J. L. Bentley, and M. D. McIlroy, “Engineering a Sort Function”, Softw. Pract. Exper. 23, 1249–1265 (1993).
-  M. Durand, “Asymptotic analysis of an optimized quicksort algorithm”, Inform. Process. Lett. 85, 73–77 (2003).
-  T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introduction to algorithms, third (The MIT Press, 2009).
-  R. C. Singleton, “Algorithm 347: An Efficient Algorithm for Sorting with Minimal Storage [M1]”, Commun. ACM 12, 185–186 (1969).
-  J. W. Tukey, “The ninther, a technique for low-effort robust (resistant) location in large samples”, in Contributions to survey sampling and applied statistics, edited by H. David, (Academic Press, 1978), pp. 251 –257.