# Max-Throughput for (Conservative) -of- Testing

###### Abstract

We define a variant of -of- testing that we call conservative -of- testing. We present a polynomial-time, combinatorial algorithm for the problem of maximizing throughput of conservative -of- testing, in a parallel setting. This extends previous work of Kodialam and Condon et al., who presented combinatorial algorithms for parallel pipelined filter ordering, which is the special case where (or ) [8, 4, 5]. We also consider the problem of maximizing throughput for standard -of- testing, and show how to obtain a polynomial-time algorithm based on the ellipsoid method using previous techniques.

## 1 Introduction

In standard -of- testing, there are binary
tests, that can be applied to an “item” .
We use to denote the value of the test on ,
and treat as an
element of .
With probability ,
, and with probability ,
.
The tests are independent, and we are given
.
We need to determine whether at least of the tests on
have a value of 0, by applying the tests
sequentially to .
Once we have enough
information to determine whether this is the case, that is, once
we have observed tests with value 0, or tests with
value 1, we do not need to perform further tests.^{1}^{1}1In an alternative definition of -of- testing, the task is to determine whether at least of the tests have a value of 1. Symmetric results hold for this definition.

We define conservative -of- testing the same way, except that we continue performing tests until we have either observed tests with value 0, or have performed all tests. In particular, we do not stop testing when we have observed tests with value 1.

There are many applications where -of- testing problems arise, including quality testing, medical diagnosis, and database query optimization. In quality testing, an item manufactured by a factory is tested for defects. If it has at least defects, it is discarded. In medical diagnosis, the item is a patient; patients are diagnosed with a particular disease if they fail at least out of special medical tests. A database query may ask for all tuples satisfying at least of given predicates (typically or ).

For , standard and conservative -of- testing are the same. For , the conservative variant is relevant in a setting where, for items failing fewer than tests, we need to know which tests they failed. For example, in quality testing, we may want to know which tests were failed by items failing fewer than tests (i.e. those not discarded) in order to repair the associated defects.

Our focus is on the MaxThroughput problem for -of- testing. Here the objective is to maximize the throughput of a system for -of- testing in a parallel setting where each test is performed by a separate “processor”. In this problem, in addition to the probabilities , there is a rate limit associated with the processor that performs test , indicating that the processor can only perform tests on items per unit time.

MaxThroughput problems are closely related to MinCost problems [9, 6]. In the MinCost problem for -of- testing, in addition to the probabilities , there is a cost associated with performing the test. The goal is to find a testing strategy (i.e. decision tree) that minimizes the expected cost of testing an individual item. There are polynomial-time algorithms for solving the MinCost problem for standard -of- testing [10, 11, 1, 3].

Kodialam was the first to study the MaxThroughput -of- testing problem, for the special case where [8]. He gave a algorithm for the problem. The algorithm is combinatorial, but its correctness proof relies on polymatroid theory. Later, Condon et al. studied the problem, calling it “parallel pipelined filter ordering”. They gave two combinatorial algorithms, with direct correctness proofs [5].

#### Our Results.

In this paper, we extend the previous work by giving a polynomial-time combinatorial algorithm for the MaxThroughput problem for conservative -of- testing. Our algorithm can be implemented to run in time , matching the running time of the algorithms of Condon et al. for 1-of-n testing. More specifically, the running time is , where varies depending on the output representation used; the algorithm can be modified to produce different output representations. We discuss output representations below.

The MaxThroughput problem for standard -of- testing appears to be fundamentally different from its conservative variant. We leave as an open problem the task of developing a polynomial time combinatorial algorithm for this problem. We show that previous techniques can be used to obtain a polynomial-time algorithm based on the ellipsoid method. This approach could also be used to yield an algorithm, based on the ellipsoid method, for the conservative variant.

##### Output Representation

For the type of representation used by Condon et al. in achieving their bound, . A more explicit representation has size . We also describe a new, more compact output representation for which .

In giving running times, we follow Condon et al. and consider only the time taken by the algorithm to produce the output representation. We note, however, that different output representations may incur different post-processing costs when we want to use them them to implement the routings. For example, the compressed representation has , but it requires spending time in the worst case to extract any permutation of megaprocessors stored by the megaprocessor representation. We can reduce this complexity to using persistent search trees [13]. In contrast, the explicit representation gives direct access to the permutations. In practice, the choice of the best output representation can vary depending on the application and the setting.

For ease of presentation, in our pseudocode we use the megaprocessor representation, which is also used by Condon et al. [5] in their Equalizing Algorithm.

## 2 Related Work

Deshpande and Hellerstein studied the MaxThroughput problem for , when there are precedence constraints between tests [6]. They also showed a close relationship between the exact MinCost and MaxThroughput problems for -of- testing, when . Their results can be generalized to apply to testing of other functions.

Liu et al. [9] presented a generic, LP based method for converting an approximation algorithm for a MinCost problem, into an approximation algorithm for a MaxThroughput problem. Their results are not applicable to this paper, where we consider only exact algorithms.

Polynomial-time algorithms for the MinCost problem for standard -of- testing were given by Salloum, Breuer, Ben-Dov, and Chang et al. [10, 11, 1, 3, 12].

The problem of how to best order a sequence of tests, in a sequential setting, has been studied in many different contexts, and in many different models. See for example [9] and [5] for a discussion of related work on the filter-ordering problem (i.e. the MinCost problem for ) and its variants, and [14] for a general survey of sequential testing of functions.

## 3 Problem Definitions

A -of- testing strategy for tests is a binary decision tree that computes the -of- function, , where if and only if contains fewer than 0’s. Each node of is labeled by a variable . The left child of a node labeled with is associated with (i.e., failing test ), and the right child with (i.e., passing test ). Each corresponds to a root-to-leaf path in the usual way, and the label at the leaf is .

A -of- testing strategy is conservative if, for each root-to-leaf path leading to a leaf labeled 1, the path contains exactly non-leaf nodes, each labeled with a distinct variable .

Given a permutation of the tests, we define to be the conservative strategy described by the following procedure: Perform the tests in order of permutation until at least 0’s have been observed, or all tests have been performed, whichever comes first. Output in the first case, and in the second.

Similarly, we define to be the following standard -of- testing strategy: Perform the tests in order of permutation until at least 0’s have been observed, or until 1’s have been observed, whichever comes first. Output in the first case, and in the second.

Each test has an associated probability , where . Let denote the product distribution on defined by the ’s; that is, if is drawn from , then and the are independent. We use to denote a random drawn from . In what follows, when we use an expression of the form involving an item , we mean the probability with respect to .

### 3.1 The MinCost problem

In the MinCost problem for standard -of- testing, we are given probabilities and costs , for , associated with the tests. The goal is to find a -of- testing strategy that minimizes the expected cost of applying to a random item . The cost of applying a testing strategy to an item is the sum of the costs of the tests along the root-to-leaf path for in .

In the MinCost problem for conservative -of- testing, the goal is the same, except that we are restricted to finding a conservative testing strategy.

For example, consider the MinCost -of- problem with probabilities , and costs , . A standard testing strategy for this problem can be described procedurally as follows: Given item , begin by performing test . If , follow strategy , where . Else if , follow strategy , where .

Under the above strategy, which can be shown to be optimal, evaluating costs , and evaluating costs . The expected cost of applying this strategy to a random item is .

Because the MinCost testing strategy may be a tree of size exponential in the number of tests, algorithms for the MinCost problem may output a compact representation of the output strategy.

#### The Algorithm for the MinCost Problem.

In the literature, versions of the MinCost problem for -of- testing are studied under a variety of different names, including pipelined filter ordering, selection ordering, and satisficing search (cf. [5]).

The following is a well-known, simple algorithm for solving the MinCost problem for standard -of- testing (see e.g. [7]): First, sort the tests in increasing order of the ratio . Next, renumber the tests, so that . Finally, output the sorted list of tests, which is a compact representation of the strategy (which is the same as ).

The above algorithm can be applied to the MinCost problem
for conservative -of- testing, simply by treating as
a compact representation of the conservative strategy .
In fact, that strategy is optimal for conservative -of- testing: it has
minimum expected cost among
all conservative strategies. This follows immediately from a lemma of Boros et al. [2]^{2}^{2}2The lemma of Boros et al. actually
proves that the corresponding decision tree is 0-optimal.
A decision tree computing a function
is 0-optimal if it minimizes the expected cost of
testing an random , given that .
In conservative -of- testing, where is the
-of- function, the cost of testing is the same for all such that .
Thus the problem of finding a min-cost conservative strategy for -of- testing
is essentially equivalent to the problem of finding a 0-optimal decision tree
computing the -of- function. The lemma of Boros et al. also applies to a more
general class of functions that include the -of- functions.
.

### 3.2 The MaxThroughput problem

The MaxThroughput problem for -of- testing is a natural generalization of the MaxThroughput problem for -of- testing, first studied by Kodialam [8]. We give basic definitions and motivation here. For further information about this problem, including information relevant to its application in practical settings, see [8, 4, 5].

In the MaxThroughput problem for -of- testing, as in the MinCost problem, we are given the probabilities associated with the tests. Instead of costs for the tests, we are given rate limits . The MaxThroughput problem arises in the following context. There is an (effectively infinite) stream of items that need to be tested. Every item must be assigned a strategy that will determine which tests are performed on it. Different items may be assigned to different strategies. Each test is performed by a separate “processor”, and the processors operate in parallel. (Imagine a factory testing setting.) Item is sent from processor to processor for testing, according to its strategy . Each processor can only test one item at a time. We view the problem of assigning items to strategies as a flow-routing problem.

Processor performs test . It has rate limit (capacity) , indicating that it can only process items per unit time.

The goal is to determine how many items should be assigned to each strategy , per unit time, in order to maximize the number of items that can be processed per unit time, the throughput of the system. The solution must respect the rate limits of the processors, in that the expected number of items that need to be tested by processor per unit time must not exceed . We assume that tests behave according to expectation: if items are tested by processor per unit time, then of them will have the value 1, and will have the value 0.

Let denote the set of all -of- testing strategies and denote the set of all conservative -of- testing strategies. Formally, the MaxThroughput problem for standard -of- testing is defined by the linear program below. The linear program defining the MaxThroughput problem for conservative -of- testing is obtained by simply replacing the set of -of- testing strategies by the set of conservative -of- testing strategies .

We refer to a feasible assignment to the variables in the LP below as a routing. We call constraints of type (1) rate constraints. The value of is the throughput of the routing. We define as the probability that test will be performed on an item that is tested using strategy , when . For , if , we say that the routing saturates processor .

We will refer to the MaxThroughput problems for standard and conservative -of- testing as the “SMT problem” and the “CMT problem”, respectively.

As a simple example, consider the following CMT problem (equivalently, SMT problem) instance, where and : , , , . There are only two possible strategies, , where , and , where . Since all flow assigned to is tested by , ; this flow continues on to only if it passes test 1, which happens with probability , so . Similarly, while since . Consider the routing that assigns units of flow to strategy , and units to strategy . Then the amount of flow reaching is , and the amount of flow reaching is . Since and , this routing saturates both processors. By the results of Condon et al. [5], it is optimal.

MaxThroughput LP:

Given and , find an assignment to the variables , for all , that maximizes

subject to the constraints:

(1)
and

(2)

where
denotes the probability that test will be performed on an item that is tested using strategy , when .

## 4 The Algorithm for the Cmt problem

We begin with some useful lemmas. The algorithms of Condon et al. [5] for maximizing throughput of -of- testing rely crucially on the fact that saturation of all processors implies optimality. We show that the same holds for conservative -of- testing.

###### Lemma 1.

Let be a routing for an instance of the CMT problem. If saturates all processors, then it is optimal.

###### Proof.

Each processor can test at most items per unit time. Thus at processor , there are at most tests performed that have the value 0. Let denote the -of- function.

Suppose is a routing achieving throughput . Since items enter the system per unit time, items must also leave the system per unit time. An item such that does not leave the system until it fails tests. An item such that does not leave the system until it has had all tests performed on it. Thus, per unit time, in the entire system, the number of tests performed that have the value 0 must be , where .

Since at most tests with the value 0 can occur per unit time at processor , . Solving for , this gives an upper bound of on the maximum throughput. This bound is tight if all processors are saturated, and hence a routing saturating all processors achieves the maximum throughput. ∎

In the above proof, we rely on the fact that every routing with throughput results in the same number of 0 test values being generated in the system per unit time. Note that this is not the case for standard testing, where the number of 0 test values generated can depend on the routing itself, and not just on the throughput of that routing. We now give a simple counterexample showing that, in fact, saturation does not imply optimality for the SMT problem. Consider the MaxThroughput -of- testing instance where , and .

The following is a -of- testing strategy: Given item , peform test 1. If , follow strategy , where . Else if , follow strategy , where .

Assigning 2 units of flow to this strategy saturates the processors: is saturated since it receives the units entering the system, is saturated since it receives units from and items from . Similarly, is saturated since it receives units from and units from .

We show that the routing is not optimal by giving a different routing with higher throughput. The routing uses two strategies. The first is as follows: Given item , perform test 1. If , follow strategy , where . Else, if follow strategy , where . The second strategy used by the routing is , where . Assigning units to the first strategy uses units of the capacity of , units of the capacity of , and of the capacity of . This leaves and with residual capacity more than , and with residual capacity . We can then assign additional units to the second strategy without violating any of the rate constraints, for a routing with total throughput . (The resulting routing is not optimal, but illustrates our point.)

The routing produced by our algorithm for the CMT problem uses only strategies of the form , for some permutation of the tests (in terms of the LP, this means only if for some ). We call such a routing a permutation routing. We say that it has a saturated suffix if for some subset of the processors (1) saturates all processors in , and (2) for every strategy used by , the processors in (in some order) must form a suffix of .

With this definition, and the above lemma, we are now able to generalize a key lemma of Condon et al. to apply to conservative -of- testing. The proof is essentially the same as theirs; we present it below for completeness.

###### Lemma 2.

(Saturated Suffix Lemma) Let be a permutation routing for an instance of the CMT problem. If has a saturated suffix, then is optimal.

###### Proof.

If saturates all processors, then the previous lemma guarantees its optimality. If not, let denote the set of processors not saturated by . Imagine that we removed the rate constraints for each processor in . Let be an optimal routing for the resulting problem. We may assume that on any input , performs the tests in in some fixed arbitrary order (until and unless tests with value 0 are obtained), prior to performing any tests in . This assumption is without loss of generality, because if not, we could modify to first perform the tests in without violating feasibility, since the processors in have no rate constraints, and performing their tests first can only decrease the load on the other processors. Thus the throughput attained by is , where denotes the maximum throughput achievable just with the processors in , and is the probability that a random will have the value 0 for fewer than of the tests in (i.e. it will not be eliminated by the tests in ).

Routing also routes flow first through , and then through . Since it saturates the processors in , by the previous lemma, it achieves maximum possible throughput with those processors. It follows that achieves the same throughput as , and hence is optimal for the modified instance where processors in have no rate constraints. Since removing constraints can only increase the maximum possible throughput, it follows that is also optimal for the original instance. ∎

### 4.1 The Equal Rates Case

We begin by considering the CMT problem in the special case where the rate limits are equal to some constant value for all processors. Condon et al. presented a closed-form solution for this case when [5]. The solution is a permutation routing that uses strategies of the form . Each permutation is one of the left cyclic shifts of the permutation . More specifically, for , let , and let . The solution assigns units of flow to each (where is defined to be ). By simple algebra, Condon et al. verified that the solution saturates all processors. Hence it is optimal.

The solution of Condon et al. is based on the fact that for the -of- problem, assigning flow to each equalizes the load on the processors. Surprisingly, this same assignment equalizes the load for the -of- problem as well. Using this fact, we obtain a closed-form solution to the CMT problem.

###### Lemma 3.

Consider an instance of the CMT problem. For , let be as defined above. Let and let . Any routing that assigns a total of units of flow to the strategies , such that the fraction of the total that is assigned to each is , will cause each processor’s residual capacity to be reduced by units. If all processors have the same rate limit , then the routing that assigns units of flow to strategy saturates all processors.

###### Proof.

We begin by considering the routing in which units of flow are assigned to each . Consider the question of how much flow arrives per unit time at processor , under this routing. For simplicity, assume now that . Thus as soon as an item has failed 2 tests, it is discarded. Let .

Of the units assigned to strategy , all arrive at processor . Of the units assigned to strategy , all arrive at processor , since they can fail either 0 or 1 test (namely test ) beforehand.

Of the units assigned to strategy , the number reaching processor is , where is the probability that an item fails either 0 or 1 of tests and . Therefore, .

More generally, for , of the units assigned to , the number reaching processor is , where is the probability that a random item fails a total of 0 or 1 of tests . Thus, . It follows that the total flow arriving at processor is

Consider the second summation, . We claim that this summation is equal to , which is the probability that has at least two ’s that are 0. To see this, consider a process where we observe the value of , then the value of and so on down towards , stopping if and when we have observed exactly two 0’s. The probability that we will stop at some point, having observed two 0’s, is clearly equal to the probability that has at least two ’s that are set to 0. The condition is satisfied when exactly 1 of has the value 0. Thus is the probability that we observe exactly one in , and then we observe a second 0 at . That is, it is the probability that we stop after observing . Since the second summation takes the sum of over all between 1 and , the summation is precisely equal to the probability of stopping at some point in the above process, having seen two 0’s. This proves the claim.

An analogous argument shows that the first summation, , is equal to .

It follows that the amount of flow reaching processor is . This expression is symmetric in the processor numbers, so the amount of flow reaching every is equal to this value. Thus the above routing causes all processors to receive the same amount of flow.

Scaling each assignment in the above routing by a constant factor scales the amount of flow reaching each processor by the same factor. In the above routing, the fraction of total flow assigned to each is , so each unit of input flow sent along the results in each processor receiving units. Thus any routing that assigns a total of units of flow to the strategies , such that the fraction assigned to each is , will cause each processor to receive units.

Thus if all processors have the same rate limit , the routing that assigns units to each strategy will saturate all processors.

The above argument for can easily be extended to arbitrary . The corresponding proportional distribution of flow for arbitrary assigns a fraction of the total flow to strategy , and each unit of input flow sent along the according to these proportions results in units reaching each processor. The saturating routing for arbitrary , when all processors have rate limit , assigns units of flow to strategy . ∎

### 4.2 The Equalizing Algorithm of Condon et al.

Our algorithm for the CMT problem is an adaptation of one of the two MaxThroughput algorithms, for the special case where , given by Condon et al. [5]. We begin by reviewing that algorithm, which we will call the Equalizing Algorithm. Note that when , it only makes sense to consider strategies that are permutation routings, since an item can be discarded as soon as it fails a single test.

Consider the CMT problem for . View the problem as one of constructing a flow of items through the processors. The capacity of each processor is its rate limit, and the amount of flow sent along a permutation (i.e., assigned to strategy ) is equal to the number of items sent along that path per unit time. Sort the tests by their rate limits, and re-number them so that . Assume for the moment that all rate limits are distinct.

The Equalizing Algorithm constructs a flow incrementally as follows. Imagine pushing flow along the single permutation . Suppose we continuously increase the amount of flow being pushed, beginning from zero, while monitoring the “residual capacity” of each processor, i.e., the difference between its rate limit and the amount of flow it is already receiving. (For the moment, do not worry about exceeding the rate limit of a processor.)

Consider two adjacent processors, and . As we increase the amount of flow, the residual capacity of each decreases continuously. Initially, at zero flow, the residual capacity of is greater than the residual capacity of . It follows by continuity that the residual capacity of cannot become less than the residual capacity of without the two residual capacities first becoming equal. We now impose the following stopping condition: increase the flow sent along permutation until either (1) some processor becomes saturated, or (2) the residual capacities of at least two of the processors become equal. The second stopping condition ensures that when the flow increase is halted, permutation still orders the processors in decreasing order of their residual capacities. (Algorithmically, we do not increase the flow continuously, but instead directly calculate the amount of flow which triggers the stopping condition.)

If stopping condition (1) above holds when the flow increase is stopped, then the routing can be shown to have a saturated suffix, and hence it is optimal.

If stopping condition (2) holds, we keep the current flow, and then augment it by solving a new MaxThroughput problem in which we set the rate limits of the processors to be equal to their residual capacities under the current flow (their ’s remain the same).

We solve the new MaxThroughput problem as follows. We group the processors into equivalence classes according to their rate limits. We then replace each equivalence class with a single megaprocessor, with a rate limit equal to the residual capacities of the constituent processors, and probability equal to the product of their probabilities. We then essentially apply the procedure for the case of distinct rate limits to the megaprocessors. gen The one twist is the way in which we translate flow sent through a megaprocessor into flow sent through the constituent processors of that megaprocessor; we route the flow through the constituent processors so as to equalize their load. We accomplish this by dividing the flow proportionally between the cyclic shifts of a permutation of the processors, using the proportional allocation of Lemma 3. We thus ensure that the processors in each equivalence class continue to have equal residual capacity. Note that, under this scheme, the residual capacity of a processor in a megaprocessor may decrease more slowly than it would if all flow were sent directly to that processor (because some flow may first be filtered through other processors in the megaprocessor) and this needs to be taken into account in determining when the stopping condition is reached.

We illustrate the Equalizing Algorithm on the following CMT problem where and (since this is also an SMT problem, where and ). Suppose we have 3 processors, with rate limits , and , and probabilities and . When flow is sent along , after 6 units of flow is sent we achieve a stopping condition with and having the same residual capacity of ; the residual capacity of is .

Our algorithm then performs a recursive call where the processors and are combined into a megaprocessor with associated probability . Within megaprocessor , flow will be routed by sending of it along permuatation , and the remaining along permutation ; we observe that for one unit of flow sent through the amount of capacity used by each processor is . Using this internal routing for megaprocessor , the algorithm sends flow along ; after 12 units of flow, we reach a stopping condition when is saturated. Even though and are not saturated (they have residual capacity left) the flows constructed as described provide optimal throughput.

The Equalizing Algorithm, implemented in a straightforward way, outputs a representation of the resulting routing that consists of a sequence of pairs of the form , one for each recursive call. We call this a megaprocessor representation. The list represents the permutation of megaprocessors along which flow is sent during that call. Each is given by the subset of original processors contained in it, and is a real number that denotes the amount of flow to be sent along . Of course, flow coming into each megaprocessor should be routed so as to equalize the load on each of its constituent processors. The size of this representation is . Interpreted in a straightforward way, the representation corresponds to a routing that sends flow along an exponential number of different permutations of the original processors.

Condon et al. describe a combinatorial method to reduce the number of such permutations used to be [5]. After such a reduction, the output can be represented explicitly as a set of pairs of the form , one for each permutation that is used, indicating that amount of flow should be sent along permutation . We call such a representation a permutation representation. The size of this permutataion representation, given explicitly, is . (Hellerstein and Deshpande describe a linear algebraic method for reducing the number of permutations to be at most , yielding an explicit reprsentation of size , but at the cost of higher time complexity[6].)

We also describe a variant of the megaprocessor representation called the compressed representation, where the algorithm outputs only the first permutation explicitly, and the outputs the sequence of merges, yielding a representation of size .

### 4.3 An Equalizing Algorithm for the Cmt problem

In this section, we prove the following Theorem by presenting an algorithm. We will give an outline of the algorithm as well as its pseudocode. We will then describe how to achieve the running time stated in the Theorem.

###### Theorem 4.

There is a combinatorial algorithm for solving the CMT problem that can be implemented to run in time , where the value of depends on the output representation. For the megaprocessor representation, , for the permutation representation, , and for the compressed representation, .

#### Algorithm Outline

We extend the Equalizing Algorithm of Condon et al., to apply to arbitrary values of . Again, we will push flow along the permutation of the processors (where ) until one of the two stopping conditions is reached: (1) a processor is saturated, or (2) two processors have equal residual capacity. Here, however, we do not discard an item until it has failed tests, rather than discarding it as soon as it fails one test. To reflect this, we divide the flow into different types, numbered 0 through , depending on how many tests its component items have failed. Flow entering the system is all of type 0.

When units of flow of type enters a processor , units pass test , and units fail it. So, if , then of the incoming units of type , units will exit processor as type flow, and will exit as type flow. Both types will be passed on to the next processor in the permutation, if any. If , then units will exit as type flow and be passed on to the next processor, and the remaining will be discarded.

Algorithmically, we need to calculate the minimum amount of flow that triggers a stopping condition. This computation is only slightly more complicated for general than it is for . The key is to compute, for each processor , what fraction of the flow that is pushed into the permutation will actually reach processor (i.e. we need to compute the quantity in the LP.)

If stopping condition (2) holds, we keep the current flow, and augment it by solving a new MaxThroughput problem in which we set the rate limits of the processors to be equal to their residual capacities under the current flow (their ’s remain the same). To solve the new MaxThroughput problem, we again group the processors into equivalence classes according to their rate limits, and replace each equivalence class with a single megaprocessor, with a rate limit equal to the rate limit of the constituent processors, and probability equal to the product of their probabilities.

We then want to apply the procedure for the case of distinct rate limits to the megaprocessors. To do this, we need to translate flow sent into a megaprocessor into flow sent through the constituent processors of that megaprocessor, so as to equalize their load. We do this translation separately for each type of flow entering the megaprocessor. Note that flow of type must be discarded as soon as it fails an additional tests. We therefore send flow of type into the constituent processors of the megaprocessor according to the proportional allocation of Lemma 3 for -of- testing, where is the number of consituent processors of the megaprocessor. We also need to compute how much flow of each type ends up leaving the megaprocessor (some of the incoming flow of type entering the megaprocessor may, for example, become outgoing flow of type ), and how much its residual capacity is reduced by the incoming flow.

We give a more detailed description of the necessary computations in the pseudocode, which we discuss next. However, the pseudocode does not contain all the implementation details, and is not optimized for efficiency. It also gives the output using a megaprocessor representation. Following presentation of the pseudocode, we discuss how to implement it to achieve the running times stated in Theorem 4 for the different output representations.

#### Pseudocode

The main part of the pseudocode is presented below as Algorithm 1. The following information will be helpful in understanding it.

At each stage of the algorithm, the processors are partitioned into equivalence classes. The processors in each equivalence class constitute a megaprocessor. Each equivalence class consists of a contiguous subsequence of processors in the sorted sequence . We use to denote the number of megaprocessors (equivalence classes). The processors in each equivalence class all have the same residual capacity. In Step 1 of the algorithm, we partition the processors into equivalence classes according to their rate limits; two processors are in the same equivalence class if and only if they have the same rate limit. We use to denote both the equivalence class and the megaprocessor. In some our examples, we denote a megaprocessor containing processors by .

In Step 2, we compute the amount of flow that triggers one of the two stopping conditions. In order to do this, we need to know the rate at which the residual capacity of each processor within an equivalence class will be reduced when flow is sent down the megaprocessors in the order . We use to denote the amount by which the residual capacity of the processors in is reduced when one unit of flow is sent in that order.

The equation for follows from the preceding lemmas and discussion. We use to denote the amount of flow of type that would reach processor , if one unit of flow were sent down the permutation , where these are the original processors, not the megaprocessors. This is precisely equal to the probability that random item has exactly 0’s in tests . We compute the value of for all and in a separate initialization routine, given below. The key here is noticing that if you send one unit of flow down the megaprocessors , the amount of flow reaching megaprocessor is precisely , where is the highest index of a processor in ; the amount of flow reaching the megaprocessor depends only on how many 0’s have been encountered in test , and not on the order used to perform those tests.

The quantity is the amount of flow sent down that would cause saturation of the processors in . The quantity is the minimum amount of flow sent down that would cause the residual capacities of two megaprocessors to equalize. The stopping condition holds at the minimum of these two quantities.

#### Example

We illustrate our algorithm for the CMT problem on the following example. Let and . Suppose the probabilities are , and the rate limits are , .

Our algorithm first combines processors with same rate limits into megaprocessors; thus we combine and into megaprocessor with rate limit 12. It routes flow through this megaprocessor by sending a fraction of the flow in the order , and sending the other fraction in the order . Similarly, and have the same rate limit, so they are combined into a megaprocessor with rate limit 10, where a fraction of the flow is sent along , and the other fraction is sent along .

Our megaprocessor has a higher rate limit than , consequently our algorithm routes flow in the order . We now show that the stopping condition is reached after sending units of flow along this route.

The units of flow decreased the capacity of processors , and in by 6, since and thus flow cannot be discarded before it has been subject to at least two tests.

We now calculate the reduction of capacity in and caused by the units of flow sent through . Flow leaving has a probability of having failed both processors in and exiting the system; for flow that stays in the system to be tested by , it has a chance of having passed the test of both processors; it has a chance of having passed the test of one processor and having failed the test of the other processor. Thus, of the 6 units of flow sent into , units are passed on to as type 0 flow, and units of flow are passed on to as type 1 flow.

Of the units of type 0 flow, entering , all of it must undergo both test 3 and test 4, since flow is not discarded until it has failed two tests. Thus that flow reduces the capacity of both and by units.

Of the 3 units of type 1 flow entering , is tested first by , and then by only if it passes test (which it does with probability ). The remaining is tested first by , and then by only if it passes test (which is does with probability ). Thus of the 3 units of type 1 flow, units reach , and units reach . Hence the total units of flow entering reduce the capacities of both and by .

We have thus shown that the 6 units of flow sent first to and then to , cause the residual capacities of and to be , and the residual capacities of and to be . Thus the residual capacities of all processors equalize, as claimed.

At this point our algorithm constructs a new megaprocessor, by combining the processors in with the processors in . All the processors in the resulting megaprocessor, , have a residual capacity of . Using the proportional allocation of Lemma 3 to route flow sent into , we assign 1/7 of the flow into to permutation , 2/7 to permutation , 2/7 to permutation , and 2/7 to permutation . By sending a total of 7 units of flow through according to this allocation, we send 1, 2, 2, and 2 units respectively along the four permutations, achieving the saturating routing given in Lemma 3.

Our final routing achieves a throughput of which is optimal.

#### Achieving the running time.

Let us first consider the running time of the algorithm excluding the computation of and the time it takes to construct the output representation . It is easy to see that the algorithm makes at most recursive calls, because megaprocessors can only be merged a total of times. Excluding the computation of , the time spent in each recursive call is clearly . However, we can implement the algorithm so as to ensure this time is , as follows. First, the maintenance of the equivalence classes can be handled in time per merge by simply taking a union of the sets of adjacent processors in each megaprocessor, instead of recomputing these sets from scratch.

Second, we do not need to compute the residual capacity of each megaprocessor at every recursive call. In fact, for all megaprocessors except the first one, we only need enough information about its residual capacity to allow us to compute . This suggests that for each megaprocessor where , we keep the quantity , where instead of . The megaprocessors can be stored in a priority queue, according to their values.

Consider any where or are not involved in a merge. Then

Thus following the merge, is decreased by the same amount for all such . Therefore, instead of updating the for these in the priority queue, we can keep their current values, and maintain the sum of the values computed so far; this can be subtracted from if its updated value is needed. We do need to remove the two merged megaprocessors from the priority queue, insert the information about the resulting new megaprocessor, and update the values for megaprocessors such that or were involved in a merge. Note that we need to change the values for such megaprocessors due to the change in the value of the newly formed megaprocessor. The above operations can be performed in time time per merge, using the priority queue.

Therefore, the running time of the algorithm excluding the computation of is