Derandomized Balanced Allocation

Derandomized Balanced Allocation

Xue Chen
Computer Science Department
University of Texas at Austin
xchen@cs.utexas.edu
Supported by NSF Grant CCF-1526952 and a Simons Investigator Award (#409864, David Zuckerman), part of this work was done while the author was visiting the Simons Institute.
Abstract

In this paper, we study the maximum loads of explicit hash families in the -choice schemes when allocating sequentially balls into bins. We consider the Uniform-Greedy scheme [ABKU99], which provides independent bins for each ball and places the ball into the bin with the least load, and its non-uniform variant — the Always-Go-Left scheme introduced by Vöcking [Vöc03]. We construct a hash family with random bits based on the previous work of Celis et al. [CRSW13] and show the following results.

  1. This hash family has a maximum load of in the Uniform-Greedy scheme.

  2. It has a maximum load of in the Always-Go-Left scheme for a constant .

The maximum loads of our hash family match the maximum loads of a perfectly random hash function [ABKU99, Vöc03] in the Uniform-Greedy and Always-Go-Left scheme separately. Previously, the best known hash families that guarantee the same maximum loads as a perfectly random hash function in any of these schemes were -wise independent functions [Vöc03], which needs random bits.

1 Introduction

We investigate explicit constructions of hash functions for the classical problem of placing balls into bins. The basic model is to hash balls into bins independently and uniformly at random, which we call -choice scheme. A well-known and useful fact of the -choice scheme is that with high probability, each bin contains at most balls. For convenience, we always use logarithm of base 2 in this work. Here, by high probability, we mean probability for an arbitrary constant .

An alternative variant, which we call Uniform-Greedy, is to provide independent random choices for each ball and place the ball in the bin with the lowest load. In a seminal work, Azar et al. [ABKU99] showed that the Uniform-Greedy scheme with independent random choices guarantees a maximum load of only with high probability for balls. Later, Vöcking [Vöc03] introduced the Always-Go-Left scheme to further improve the maximum load to for choices where is the constant satisfying . For convenience, we always use -choice schemes to denote the Uniform-Greedy and Always-Go-Left scheme with choices.

Traditional analysis of load balancing assumes a perfectly random hash function. A large body of research is dedicated to the removal of this assumption by designing explicit hash families using fewer random bits. In the -choice scheme, it is well known that -wise independent functions guarantee a maximum load of with high probability, which reduces the number of random bits to . Recently, Celis et al. [CRSW13] designed a hash family with a description of random bits that achieves the same maximum load of as a perfectly random hash function.

In this work, we are interested in the explicit constructions of hash families that achieve the same maximum loads as a perfectly random hash function in the -choice schemes. More precisely, we study how to derandomize the perfectly random hash function whose maximum loads are in the Uniform-Greedy scheme[ABKU99, Vöc03] and in the Always-Go-Left scheme[Vöc03]. For these two schemes, -wise independent hash functions achieve the same maximum loads from Vöcking’s argument [Vöc03], which provides a hash family with random bits. Very recently, Reingold et al. [RRW14] showed that the hash family designed by Celis et al. [CRSW13] guarantees a maximum load of in the Uniform-Greedy scheme with random bits.

1.1 Our Contributions

For multiple-choice schemes, we strengthen the hash family of Celis et al. [CRSW13] for the 1-choice scheme — our hash family is -wise independent over bins and “almost” -wise independent over a fraction of bins. Then we prove that our hash family derandomizes Vöcking’s witness tree argument [Vöc03] such that random bits could guarantee the same maximum loads as a perfectly random hash function in the multiple-choice schemes.

We first show our hash family guarantees a maximum load of in the Uniform-Greedy scheme[ABKU99, Vöc03] with choices. We use to denote the pool of balls and consider placing balls into bins here. Without loss of generality, we always assume and is a constant at least in this work.

Theorem 1.1 (Informal version of Theorem 5.1)

For any , any constants and , there exists a hash family with random bits such that given any balls in , with probability at least , the max-load of the Uniform-Greedy scheme with independent choices of is .

Our hash family has an evaluation time in the RAM model based on the algorithm designed by Meka et al. [MRRR14] for the hash family of Celis et al. [CRSW13].

Then we show this hash family guarantees a load balancing of in the Always-Go-Left scheme [Vöc03] with choices. The Always-Go-Left scheme [Vöc03] is an asymmetric allocation scheme that partitions the bins into groups with equal size and uses an unfair tie-breaking mechanism. Its allocation process provides independent choices for each ball from the groups separately and always chooses the left-most bin with the least load for each ball. We defer the formal description of the Always-Go-Left scheme to Section 6. Notice that the constant in equation satisfies . Compared to the Uniform-Greedy scheme, the Always-Go-Left scheme [Vöc03] improves the maximum load exponentially with regard to . Even for , the Always-Go-Left scheme improves the maximum load from to .

Theorem 1.2 (Informal version of Theorem 6.3)

For any , any constants and , there exists a hash family with random bits such that given any balls in , with probability at least , the max-load of the Always-Go-Left scheme with independent choices of is .

At the same time, from the lower bound on the maximum load of any random -choice scheme shown by Vöcking [Vöc03], the maximum load of our hash family is optimal for -choice schemes up to the low order term of constants.

Finally, we show our hash family guarantees the same maximum load as a perfectly random hash function in the -choice scheme for balls. Given balls in , the maximum load of the -choice scheme becomes from the Chernoff bound. For convenience, we refer to this case of balls as a heavy load. In a recent breakthrough, Gopalan, Kane, and Meka [GKM15] designed a pseudorandom generator of seed length that fools the Chernoff bound within polynomial error. Hence the pseudorandom generator [GKM15] provides a hash function with random bits for the heavy load case. Compared to the hash function of [GKM15], we provide a simplified construction that achieves the same maximum load but only works for balls.

Theorem 1.3 (Informal version of Theorem 7.1)

For any constants and , there exist a hash function generated by random bits such that for any balls, with probability at least , the max-load of the bins in the 1-choice scheme with is .

1.2 Previous Work

The 1-choice scheme.

Natural explicit constructions of hash functions using a few random bits are -wise independent functions, small-biased spaces, and -wise small-biased spaces. For the -choice scheme with balls, Alon et al.[ADM99] showed the existence of a pairwise independent hash family that always has a maximum load of . On the other hand, it is well known that -wise independent functions achieve a maximum load of with high probability, which needs random bits. Using -wise small-biased spaces as milder restrictions, Celis et al.[CRSW13] designed a hash family with random bits achieving the same maximum load with high probability.

For the heavy load case in the -choice scheme, a perfectly random hash function guarantees a maximum load of from the Chernoff bound. Hence any pseudorandom generator fooling the Chernoff bound within polynomial small error is a hash family matching this maximum load. Schmidt et al. [SSS95] showed that -wise independence could derandomize the Chernoff bound, which provides a hash function with random bits. In a recent breakthrough [GKM15], Gopolan, Kane, and Meka designed a pseudorandom generator with seed length to fool halfspaces, the Chernoff bound, and many other classes, which provides a hash family of bits.

Multiple-choice schemes.

For balls in the -choice schemes, the original argument of [ABKU99] adopts an inductive proof that relies on the assumption of full randomness. It is folklore (e.g., [RRW14, DKRT16]) that -wise independent functions could derandomize Vöcking’s witness tree argument [Vöc03] to achieve a maximum load of in the Uniform-Greedy scheme, which takes random bits. Reingold et al. [RRW14] prove the hash family of [CRSW13] derandomizes the following property of random graphs: for a graph with vertices and random edges, every connected component has size with high probability. Besides applications in Cuckoo hashing, Reingold et al. [RRW14] show the hash family [CRSW13] guarantees a maximum load of in the Uniform-Greedy scheme for balls. However, for balls with choices, the upper bound of the maximum load proved by Reingold et al. [RRW14] becomes , while the maximum load of a perfectly random hash function is .

Vöcking [Vöc03] introduced Always-Go-Left scheme to further improve the maximum loads of -choice schemes to . In the same work, Vöcking showed a lower bound to illustrate that the load is optimal for random -choice schemes. However, much less is known about the derandomization of the Always-Go-Left scheme except -wise independent functions for Vöcking’s witness tree argument [Vöc03], which is pointed out in [RRW14, DKRT16].

Another long line of research on hash families focuses on studying functions with a constant evaluation time despite the expense of the number of random bits. For -wise independence, Siegel [Sie89] showed how to implement it in constant time. For multiple-choice schemes, Woelfel [Woe06] showed that the hash family of [DW03], which takes constant evaluation time and random bits, guarantees the same maximum loads as a perfectly random hash functions in the multiple-choices schemes. Pǎtraşcu and Thorup [PT12] introduced simple tabulation hashing, a function with constant evaluation time and random bits, that can replace the perfectly random hash functions in various applications. Very recently, Dahlgaard et al. [DKRT16] proved that the maximum load of the Uniform-Greedy scheme is with high probability in simple tabulation[PT12] using random bits. For the hash family in [CRSW13], Meka et al. [MRRR14] improved its evaluation time to .

scheme reference maximum load number of random bits
1-choice well known
1-choice [CRSW13]
Uniform-Greedy [ABKU99] full randomness
Uniform-Greedy [Vöc03]
Uniform-Greedy [RRW14]
Uniform-Greedy [DKRT16]
Uniform-Greedy this work
Always-Go-Left [Vöc03]
Always-Go-Left this work
Table 1: Summary of previous works about the maximum loads of placing balls in bins with choices

We summarize these results in Table 1. Finally, we refer surveys [MRS00, Mit01, Wie17] and the reference therein for various applications of multiple-choice schemes in computer science.

1.3 Discussion

In this work, we provide a hash family with random bits that matches the maximum loads of a perfectly random hash function in multiple-choice schemes. A natural question is to reduce the number of random bits to . A starting point would be to improve the hash families in the -choice scheme, where the best construction needs random bits from Celis et al.[CRSW13].

For the -choice scheme, the load of each bin is the summation of random indicator variables, which allows us to use the pseudorandom generators for concentration bounds [SSS95, GKM15] and space-bounded computation [Nis92, NZ96, GKM15]. One interesting direction is to investigate the application of the techniques in these two problems to the design of hash functions. At the same time, although Alon et al.[ADM99] proved lower bounds of -wise independent functions in the 1-choice scheme, it is still interesting to explore natural algebraic constructions of small-biased spaces in [AGHP90] such as the quadratic characters of modulo .

Our work is an application of the technique — milder restrictions [CRSW13, GMR12] in the design of pseudorandom generators. Even though -wise independence and small-biased spaces fool variants of classes, these two tools will not provide optimal pseudorandom generators for basic classes such as the 1-choice scheme [ADM99] or read-once CNFs [DETT10]. After Celis et al. [CRSW13] introduced milder restrictions, this technique has been successfully applied to construct almost optimal pseudorandom generators with random bits for several classes such as 1-choice scheme [CRSW13], read-once CNFs [GMR12], modulo functions and halfspaces [GKM15]. To the best of our knowledge, our work is the first application of milder restrictions in non read-once functions, while all previous applications of milder restrictions are in the read-once case. It would be of great interest to investigate this technique to the design of pseudorandom generators for broader classes such as circuits and space-bounded computation.

1.4 Organization

This paper is organized as follows. In Section 2, we introduce some notations and tools. We define witness trees and revisit Vöcking’s argument [Vöc03] in Section 3. We show the construction of our hash family in Section 4 and sketch our derandomization in Section 4.1. Next we prove Theorem 1.1 in Section 5 and Theorem 1.2 in Section 6, which provide upper bounds on the maximum loads of the Uniform-Greedy scheme and the Always-Go-Left scheme separately. Finally, we prove Theorem 1.3 in Section 7 which shows a bound of the heavy load case in the 1-choice scheme.

2 Preliminaries

We use to denote the pool of balls, to denote the numbers of balls in , and to denote the number of bins. We assume and is a power of in this work. We use to denote the indicator function of the event and to denote the Galois field of size for a prime power .

Definition 2.1

Given a prime power p, a distribution on is a -biased space if for any non-trivial character function in , .

A distribution on is a -wise -biased space if for any non-trivial character function in of support size at most , .

The seminal works [NN90, AGHP90] provide small-biased spaces with optimal seed length.

Lemma 2.2 ([Nn90, Aghp90])

For any prime power and integer , there exist explicit constructions of -biased spaces on with seed length and explicit constructions of -wise -biased spaces with seed length

Given two distributions and with the same support , we define the statistical distance to be . Vazirani [Vaz86] proved that small-biased spaces are close to the uniform distribution.

Lemma 2.3 ([Vaz86])

A -biased space on is close to the uniform distribution in statistical distance.

Given a subset of size in , a -wise -biased space on is close to the uniform distribution on in statistical distance.

Given a distribution on functions from to , is -wise independent if for any elements in , is a uniform distribution on . For small-biased spaces, we choose and the space to be in Lemma 2.2 and summarize the discussion above.

Lemma 2.4

Given and , a -wise -biased space from to is close to the uniform distribution from to on any balls, which needs random bits.

Remark 2.5

In this work, we always choose and in the small biased spaces such that the seed length is . At the same time, we only use -wise small-biased spaces rather than small biased spaces to improve the evaluation time from to .

We state the Chernoff bound in -wise independence by Schmidt et al. in [SSS95].

Lemma 2.6 (Theorem 5 (I) (b) in [Sss95])

If is the sum of -wise independent random variables, each of which is confined to the interval with , then for and ,

3 Witness Trees

We first provide several notations and definitions in this work. Then we review the witness tree argument of Vöcking [Vöc03] for the Uniform-Greedy scheme.

Definition 3.1 (Uniform-Greedy with choices)

The process inserts balls in any fixed order. Let be hash functions from to . The allocation process works as follows: for each ball , the algorithm considers bins and puts the ball into the bin with the least load among . When there are several bins with the least load, it picks an arbitrary one.

We define the height of a ball to be the height of it on the bin allocated in the above process.

Next we follow the notations of Vöcking [Vöc03] to define witness trees and pruned witness trees. Given the balls and hash functions in the allocation process, we construct a symmetric witness tree for each ball in this process.

Definition 3.2 (Symmetric witness trees)

A symmetric witness tree with height for a ball is a complete -ary tree of height . Every node in this tree corresponds to a ball ; and the root corresponds to the ball . A ball in has a ball as its th child iff when we allocate in the process, ball is the top ball in the bin . Hence and the bin is in the subset of when is the th child of .

Next we trim the repeated nodes in a witness trees such that there is no duplicate edge after the trimming.

Definition 3.3 (Pruned witness trees and collisions)

Given a witness tree where nodes in correspond to the same ball, let be the node among them in the most bottom level of . Consider the following process: first remove and their subtrees; then, redirect the edges of from their parents to and call these edges collisions. Given a symmetric witness tree , we call the new tree without repeated nodes after the above process as the pruned witness tree of .

We call different witness trees with the same structure but different balls a configuration. For example, the configuration of symmetric witness trees with distinct nodes is a full -ary tree without any collision.

Figure 1: A witness tree with distinct balls and a pruned witness tree with 3 collisions

Next we define the height and size of pruned witness trees.

Definition 3.4 (Height of witness trees)

Given any witness tree , let the height of be the length of the shortest path from the root of to its leaves. Because , the height of the pruned witness tree equals the height of the original witness tree. Given a ball of height and any , we always consider the pruned witness tree of with height whose leaves have height .

At the same time, let denote the number of vertices in for any witness tree and denote the number of nodes in a configuration .

Finally we review the argument of Vöcking [Vöc03] for balls. One difference between this proof and Vöcking’s [Vöc03] proof is an alternate argument for the case of witness trees with many collisions.

Lemma 3.5 ([Vöc03])

For any constants and , with probability at least , the max-load of the Always-Go-Left scheme with independent choices from perfectly random hash functions is .

Proof.  We fix a parameter for the height of witness trees. In this proof, we bound the probability that any symmetric witness tree of height with leaves of height at least 4 exists in perfectly random hash functions. From the definition of witness trees, this also bounds the probability of a ball with height in the -choice Uniform-Greedy scheme.

For symmetric witness trees of height , it is sufficient to bound the probability that their pruned counterparts appear in perfectly random hash functions. We separate all pruned witness trees into two cases according to the number of edge collisions: pruned witness trees with at most collisions and pruned witness trees with at least collisions.

Pruned witness trees with at most collisions.

Let us fix a configuration with at most collisions and consider the probability any pruned witness trees with configuration appears in perfectly random hash functions. Because each node of this configuration corresponds a distinct ball, there are at most possible ways to instantiate balls into .

Next, we fix one possible pruned witness tree and bound the probability of the appearance of in . We consider the probability of two events: every edge in the tree appears during the allocation process; and every leaf of has height at least . For the first event, an edge holds during the process only if the hash functions satisfy

(1)

Secondly, the probability that a fixed leaf ball has height at least is at most . A leaf ball of height 4 indicates that each bin in his choices has height at least . Because at most bins contain at least 3 balls at any moment, the probability that a random bin has height at least 3 is . Thus the probability that random bins have height is at most .

We apply a union bound on the probability that any witness tree with the configuration appears in perfectly random hash functions:

(2)

We lower bound the number of edges in by because is connected. Next we lower bound the number of leaves. Because is a -ary tree with at most collisions, the number of leaves is at least . At the same time, is trimmed from the -ary symmetric witness tree of height . Thus . From all discussion above, we bound (2) by

Finally, we apply a union bound on all possible configurations with at most collisions: the number of configurations is at most such that the probability of any witness tree with height and at most collisions existing is at most .

Pruned witness trees with at least collisions.

We use the extra collisions with equation (1) instead of the number of leaves in this case.

Given any configuration with at least collisions, we consider the first collisions in the BFS of . Let be the induced subgraph of that only contains nodes in and their ancestors in . At the same time, the size and the number of edges in is .

Figure 2: An example of extracting from given two collisions.

Because any pruned witness tree of exists only if its corresponding counterpart of exists in perfectly random hash functions, it is suffice to bound the probability of the latter event. There are at most instantiations of balls in . For each instantiation, we bound the probability that all edges survive by (1):

We bound the probability that any pruned witness tree of configuration survives in the perfectly random hash function by

Finally, we apply a union bound over all possible configurations : there are at most configurations of collisions.


Remark 3.6

Because the sizes of all witness trees are bounded by , -wise independent hash functions could adopt the above argument to prove a max-load of .

4 Hash functions

We construct our hash family and show its properties for the derandomization of -choice schemes in this section. We sketch the derandomization of Lemma 3.5 of Vocking’s argument in Section 4.1.

Let denote the concatenation operation and denote the bit-wise XOR operation.

Construction 4.1

Given , and two integers , let

  1. denote a function generated by an -wise -biased space for each ,

  2. denote a function generated by an -wise -biased space such that is a function by to ,

  3. denote a function from a -wise independent family from to .

We define a random function in our hash family with parameters and to be:

Hence the seed length of our hash family is . We always choose and such that the seed length is .

Remark 4.2

Our parameters of are stronger than the parameters in [CRSW13]. While the last function of [CRSW13] is still a -biased space, we use in to provide almost -wise independence on subsets of size for our calculations.

Properties of .

We state the properties of that will be used in the derandomization. Because of the -wise independence in and the operation, we have the same property for .

Property 4.3

is -wise independent.

Then we fix and discuss . For each , it is natural to think as a function from to , i.e., a hash function maps all balls into bins. Celis et al. [CRSW13] showed that for every , the number of balls in every bin of is close to its expectation in -biased spaces.

Lemma 4.4 ([Crsw13])

Given and , for any constant , there exists such that given balls, with probability at least , for all , every bin in contains at most balls under .

For completeness, we provide a proof of Lemma 4.4 in Appendix A. In this work, we use the following version that after fixing in the Construction 4.1, still allocates the balls evenly.

Corollary 4.5

For any constant , there exists such that given balls and any function , with probability at least over , for any bin , it contains at most balls in the hash function .

Next we discuss the last function generated from a -biased space on . For a subset , let denote the distribution of a random function on and denote the uniform distribution over all maps from . From Lemma 2.3 and the union bound, we have the following claim.

Claim 4.6

Given , for a fixed subset of size , is -close to the uniform distribution on , i.e., .

Then for subsets of size , we have

In another word, is close to the uniform distribution on subsets of size . However, (or ) is not close to -wise independence on balls.

Remark 4.7 (Evaluation time)

Our hash function has an evaluation time in the RAM model. Because we use -biased spaces in , we lose a factor of compared to the hash family of [CRSW13]. The reason is as follows.

can be evaluated by a degree polynomial in the Galois field of size , which takes time. The first hash functions use -biased spaces, which have total evaluation time in the RAM model from [MRRR14].

The last function in the RAM model is a -wise -biased space from to , which needs words in the RAM model. Thus the evaluation time becomes times the cost of a quadratic operation in the Galois field of size , which is .

4.1 Proof Overview

We sketch the derandomization of Lemma 3.5 in this section. Similar to the proof of Lemma 3.5, we bound the probability that any pruned witness tree of height exists in , where each . We use the property of to derandomize the case of pruned witness trees with at most collisions and the property of to derandomize the other case.

Pruned witness trees with at most collisions.

We show how to derandomize the union bound (2) for a fixed configuration with at most collisions. There are two probabilities in (2): the second term over all edges in and the last term over all leaves. We focus on the second term in this discussion, because it contributes a smaller probability. Since , it needs -wise independence over bins for every possible witness trees in (2), which is impossible to support with bits [Sti94].

We omit and focus on the other part in this case. Our strategy is to first fix the prefixes in the hash functions, , then recalculate (2) using the suffixes . Let be a possible witness tree in the configuration . For an edge in to satisfy (1), the prefixes of and must satisfy

(3)

After fixing the prefixes, let denote the subset of possible witness trees in the configuration that satisfy the prefix condition (3) for every edge. Because each bin of receives at most balls from every prefix function by Corollary 4.5, we could bound

instead of in the original argument.

Now we consider all possible witness trees in under the suffixes . We could treat as -wise independent functions for all possible witness trees in from Claim 4.6, because and . In the next step, we use -wise independence to rewrite (2) and finish the proof of this case.

Pruned witness trees with at least collisions.

In our alternate argument of this case in Lemma 3.5, the subconfiguration of has at most nodes and edges. Since , the number of edges in is . By choosing with a sufficiently large constant, with -wise independence supports the argument in Lemma 3.5.

5 The Uniform Greedy scheme

We prove our main result for the Uniform-Greedy scheme — Theorem 1.1 in this section.

Theorem 5.1

For any , any constant , and integer , there exists a hash family from Construction 4.1 with random bits that guarantees the max-load of the Uniform Greedy scheme with independent choices from is with probability at least for any balls in .

Proof.  We specify the parameters of as follows: , for a large constant , and such that Corollary 4.5 holds with probability at least . Let denote the independent hash functions from with the above parameters, where each

We use the notation to denote in the choices and to denote the group of hash functions in this proof.

We bound the probability that any symmetric witness tree of height with leaves of height at least exists in . Similar to the proof of Lemma 3.5, we bound the probability of pruned witness trees of height in . We separate all pruned witness trees into two cases according to the number of edge collisions: pruned witness trees with at most collisions and pruned witness trees with at least collisions.

Pruned witness trees with at least collisions.

We start with a configuration of pruned witness trees with height and at least collisions. Let be the first collisions in the BFS of . Let be the induced subgraph of that only contains nodes in these edges and their ancestors in . Therefore any pruned witness tree of configuration exists in only if the corresponding counterpart of with configuration exists in . The existence of in indicates that for every edge in , satsify

(4)

Notice that the number of edges in and is at most .

Because are -wise independent, We bound the probability that all edges of satisfy (4) in by

Now we apply a union bound over all choices of balls in . There are at most choices of balls in the nodes of . Therefore we bound the probability that any witness with at least collisions survives in -wise independent functions by

Next we apply a union bound over all configurations . Because there are at most configurations of collisions, with probability at least , there is no pruned witness trees with at least collision and height exists in .

Pruned witness trees with at most collisions.

We fix a configuration of pruned witness trees with height and less than collisions. Next we bound the probability that any pruned witness trees in this configuration with leaves of height at least exists in .

We extensively use the fact that after fixing and , at most elements in are mapped to any bin of from Corollary 4.5. Another property is the number of leaves in : because there are at most collisions in , has at least leaves. On the other hand, the number of leaves is at least .

For a pruned witness tree with configuration , exists in only if

(5)

We restate the above condition on the prefixes and suffixes of separately. Let