Quicksort with median of medians is considered practical

Quicksort with median of medians is considered practical

Noriyuki Kurosawa kurosawa@vortex.c.u-tokyo.ac.jp
Abstract

The linear pivot selection algorithm, known as median-of-medians, makes the worst case complexity of quicksort be . Nevertheless, it has often been said that this algorithm is too expensive to use in quicksort. In this article, we show that we can make the quicksort with this kind of pivot selection approach be efficient.

1 Introduction

Quicksort[2, 1, 3] is one of the most efficient and widely used sorting algorithms. The major drawback of quicksort is that worst case time complexity of naive implementations of quicksort is with input size . In order to avoid this worst case behavior, a variant of quicksort, so-called introsort[4] 111 In many textbooks, it is written that this method is invented in ref. [4] in 1997. We found, however, that the same technique has already appeared in ref. [5], which is published in 1989. In this book, the author calls this method fail-safe quicksort. , is often used.

There exists another solution to keep the worst case complexity of quicksort . This is the use of median-of-medians or Blum-Floyd-Pratt-Rivest-Tarjan (BFPRT) algorithm — the pivot selection algorithm in the linear median finding algorithm[6]. Despite the theoretical importance of this scheme, because of its somewhat large constant factor, this method have been considered impractical to use as a pivot selection algorithm in quicksort.

In this article, contrary to the above widespread view, we show that using BFPRT in quicksort is not inefficient, and moreover, considered practical. Indeed, we find that quicksort with median-of-medians shows comparable performance to the well-known optimized quicksort implementations[7, 8] for random sequence. We also show that the technique for improving performance is not only for the BFPRT but also available for other pivoting methods.

2 Method

2.1 Idea

The main idea of this article is thinning out the input elements of each pivot selection. The outline of this idea is given in Algorithm 1. Here we thin out the input array and apply the BFPRT on only of . This BFPRT based pivot selection method guarantees the total time complexity of sorting to be , even in the case of . We can show this briefly as follows. BFPRT guarantees that of input elements are lesser than the output pivot, and other are greater[9]. Thus, at least in with length are lesser and other are greater. This yields that one partition operation reduces the length of from to even in the worst case. The total time complexity of sorting is then asymptotically (see Ref. [9]).

function select-pivot-with-thinning()
     Let be a parameter s.t. .
     .
     return (Apply BFPRT on ).
end function
Algorithm 1 Pivot selection by BFPRT with thining out

The constant factor in the worst case of quicksort with the above method is estimated as follows. If the complexity follows the form for sufficiently large , the recurrence relation becomes

(1)

here and is the entropy function. The constant represents the cost of the pivot selection and the partitioning. If the bound of the complexity of BFPRT is , can be bound as because the partitioning requires comparisons222Because there are so many variants of BFPRT algorithms, we do not specify the value of in this article.. The upper bound of is then . If then , for example. The lower bound of can be estimated in the same way: .

2.2 Pseudo Median of

The idea of thinning out can be applicable for another pivot selection scheme, as long as it takes linear time. For example, median-of-three[10] method and median-of-three-medians-of-three (pseudo-median-of-nine or Tukey’s ninther)[7, 11] are widely used pivot selection method. As the extreme case of these strategies, pseudo-median of () can be considered. Here we introduce another type of thinning method upon the pseudo-median of , as described in Algorithm 2.

function select-pivot-by-pseudo-median()
     Let be a parameter s.t. .
     .
     if  then
         return
     else
         
         
         
         return
     end if
end function
Algorithm 2 Pivot selection by Pseudo median of with thining out

The worst case complexity of Alg. 2 is estimated as the following. If and the length of the target array is , Alg. 2 guarantees that at least of the elements are lesser than the selected pivot and other are greater. Thus, for a large array with length , Alg. 2 guarantees that of the elements are greater than the pivot and are lesser. The depth of recursion for the worst case is then estimated as

(2)

Therefore, the worst case bound of quicksort with Alg. 2 is bound by . Because the averaged required comparison in a pivot selection for an array with length is , the lower bound of the complexity of quicksort with this method in the best case is estimated as .

3 Experiment

We carried out a numerical experiment to estimate the coefficients of the quicksort with Alg. 1 (t-BFPRT) and Alg. 2 (t-PMed). We used the standard single pivot partitioning[2] and the original BFPRT algorithm[6]. For simplicity, we did not change to another sorting algorithm such as heapsort or insertion sort to sort small subarrays. We also measured quicksort with random pivot picking (Hoare’s original method) (Rand), median-of-three (Med3), and pseudo-median-of-nine (PMed9) for benchmarking our strategy. We used random sequences of distinct integers as target arrays. We executed 100 times of sorting by each scheme, obtained the mean and unbiased variance of required count of comparisons, and calculated their coefficients via least squares fittings.

Figure 1: Comparison number of t-BFPRT. We do not plot the line of PMed9 because it almost overlaps that of t-BFPRT().
Figure 2: Comparison number of t-PMed. We do not plot the line of PMed9 because it almost overlaps that of t-PMed().
method
Rand 1
Med3 1
PMed9 1
t-BFPRT ()
t-BFPRT () 1
t-PMed () 1
t-PMed () 1
Table 1: Coefficients of each method for random sequences. (Fitting to .)
Figure 3: Dependence of the coefficients on . We plot the line as the theoretical limit.

Figure 1 and 2 show the number of comparison of each pivoting method. We can see that the performance of t-BFPRT with or t-PMed with is not good because of its expensive pivot selection procedure. On the other hand, t-BFPRT or t-PMed with exhibits excellent result; both method beat that of Med3. Table 1 shows the fitting result to the function . We can see that t-BFPRT and t-PMed with large clearly outperform Med3 and exhibit comparable result to PMed9.

Figure 3 shows the dependence of the coefficients on . In both methods, the larger is, the more improved the asymptotic behavior is. Because the larger makes the upper bound of the comparison count worse, within the range is probably the best choice in this configuration.

Note that the above discussion only considers the number of comparisons and the execution time is another question. As for t-PMed, there is no swapping in pivot selection phase. Therefore, unless the effect of the memory cache and cost of function calls are not negligible, the execution time shows the same tendency of the comparison count. On the other hand, t-BFPRT involves data modifications in the pivot selection. In our environment, it showed the similar tendency of the comparison count. However, further accurate investigations should be required anyway.

4 Conclusion

In this article, we show that the use of median-of-medians as a pivot selection algorithm in quicksort is not always inefficient; it works very well on random sequences, but still requires only comparisons in the worst case. We also show that the technique of thinning out can also be usable for other pivoting method, and can improve the worst case behavior with keeping good performance for random input.

There can be many other methods to thinning out the input of pivoting algorithm. It is easy to make a bad input for Alg. 1 (setting larger elements at the multiple of and last elements), but for another thinning method making a bad input may not be as easy as this. Also, the methods in this article perform well on neither increasing nor decreasing sequence because of its naive behavior for small subarrays. To fix this, equipping adaptive features (e.g., median-of-three for small input (typically for t-BFPRT and for t-PMed) and the main method for large input) is effective.

Because of its simple concept and efficiency, we believe that the presented approach can be useful in real world applications.

References

  • [1] C. A. R. Hoare, “Algorithm 63: Partition”, Commun. ACM 4, 321 (1961).
  • [2] C. A. R. Hoare, “Algorithm 64: Quicksort”, Commun. ACM 4, 321 (1961).
  • [3] C. A. R. Hoare, “Quicksort”, Comput. J. 5, 10–16 (1962).
  • [4] D. R. Musser, “Introspective Sorting and Selection Algorithms” , Softw: Pract. Exper. 27, 983–993 (1997).
  • [5] K. Ishihata, Algorithms and data structures, Iwanami Software Science Series 3 (Iwanami, 1989).
  • [6] M. Blum, R. W. Floyd, V. Pratt, R. L. Rivest, and R. E. Tarjan, “Time bounds for selection”, J. Comput. Syst. Sci. 7, 448–461 (1973).
  • [7] J. L. Bentley, and M. D. McIlroy, “Engineering a Sort Function”, Softw. Pract. Exper. 23, 1249–1265 (1993).
  • [8] M. Durand, “Asymptotic analysis of an optimized quicksort algorithm”, Inform. Process. Lett. 85, 73–77 (2003).
  • [9] T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introduction to algorithms, third (The MIT Press, 2009).
  • [10] R. C. Singleton, “Algorithm 347: An Efficient Algorithm for Sorting with Minimal Storage [M1]”, Commun. ACM 12, 185–186 (1969).
  • [11] J. W. Tukey, “The ninther, a technique for low-effort robust (resistant) location in large samples”, in Contributions to survey sampling and applied statistics, edited by H. David, (Academic Press, 1978), pp. 251 –257.
Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
""
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
   
Add comment
Cancel
Loading ...
316158
This is a comment super asjknd jkasnjk adsnkj
Upvote
Downvote
""
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters
Submit
Cancel

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test
Test description