Novel Architectures and Algorithms for Delay Reduction in Backpressure Scheduling and Routing
Abstract
The backpressure algorithm is a wellknown throughputoptimal algorithm. However, its delay performance may be quite poor even when the traffic load is not close to network capacity due to the following two reasons. First, each node has to maintain a separate queue for each commodity in the network, and only one queue is served at a time. Second, the backpressure routing algorithm may route some packets along very long routes. In this paper, we present solutions to address both of the above issues, and hence, improve the delay performance of the backpressure algorithm. One of the suggested solutions also decreases the complexity of the queueing data structures to be maintained at each node.
1 Introduction
Resource allocation in wireless networks is complicated due to the shared nature of wireless medium. One particular allocation algorithm called the backpressure algorithm which encompasses several layers of the protocol stack from MAC to routing was proposed by Tassiulas and Ephremides, in their seminal paper [1]. The backpressure algorithm was shown to be throughputoptimal, i.e., it can support any arrival rate vector which is supportable by any other resource allocation algorithm. Recently, it was shown that the backpressure algorithm can be combined with congestion control to fairly allocate resources among competing users in a wireless network [2, 3, 4, 5, 6, 7], thus providing a complete resource allocation solution from the transport layer to the MAC layer. While such a combined algorithm can be used to perform a large variety of resource allocation tasks, in this paper, we will concentrate on its application to scheduling and routing.
Even though the backpressure algorithm delivers maximum throughput by adapting itself to network conditions, there are several issues that have to be addressed before it can be widely deployed in practice. As stated in the original paper [1], the backpressure algorithm requires centralized information and computation, and its computational complexity is too prohibitive for practice. Much progress has been made recently in easing the computational complexity and deriving decentralized heuristics. We refer the interested reader to [8, 9] and references within for some recent results along these lines. We do not consider complexity or decentralization issues in this paper; our proposed solutions can be approximated well by the solutions suggested in the above papers.
Besides complexity and decentralization issues which have received much attention recently, the backpressure algorithm can also have poor delay performance. To understand that, we consider two different network scenarios: one in which the backpressure algorithm is used to adaptively select a route for each packet, and the other in which a flow’s route is chosen upon arrival by some standard multihop wireless network routing algorithm such as DSR or AODV and the backpressure algorithm is simply used to schedule packets. We refer to the first case as adaptiverouting and the second case as fixedrouting, respectively.
We first discuss networks with fixed routing. The backpressure algorithm assigns a weight to each flow on each link. The weight is equal to the flow’s queue backlog at the transmitter of the link minus the flow’s queue backlog at the receiver. The weight of a link is equal to the maximum weight of any flow that uses the link. The backpressure algorithm then selects a schedule which maximizes the sum of the weights of the links included in the schedule. Under such an algorithm, for a link to be scheduled, its weight should be slightly larger than zero. Now, let us consider a flow that traverses links, and use an informal argument to show why it is very intuitive that the flow’s total queue accumulation along its route should grow quadratically with the route length. The queue length at the destination for this flow is equal to zero. The queue length at the first upstream node from the destination will be some positive number, say, The queue length at the second upstream node from the destination will be even larger and for the purposes of obtaining insight, let us say that it is Continuing this reasoning further, the total queue length for the flow will be Thus, the total backlog on a path is intuitively expected to grow quadratically in the number of hops. On the other hand, suppose a fixed service rate is allocated to each flow on each link on its path, then the queue length at each hop will be roughly depending on the utilization at that link. With such a fixed service rate allocation, the total endtoend backlog should then grow linearly in the number of hops. However, such an allocation is possible only if the packet arrival rate generated by each flow is known to the network a priori. One of the contributions of this paper is to use counters called shadow queues introduced in [10] to allocate service rates to each flow on each link in an adaptive fashion without knowing the set of packet arrival rates.
We will also show that the concept of shadow queues can reduce the number of real queues maintained at each node significantly. In particular, we will show that it is sufficient to maintain perneighbor queues at each node, instead of perflow queues required by the backpressure algorithm in the case of fixed routing. In large networks, the number of flows is typically much larger compared to the number of neighbors of each node, thus using perneighbor queues can result in significant reduction in implementation complexity. A different idea to reduce the number of queues at each node has been proposed in [11], but the implementation using shadow queues has the additional benefit of delay reduction.
Next, we discuss the case of adaptive routing. The backpressure algorithm tends to explore many routes to find sufficient capacity in the network to accommodate the offered traffic. Since the goal of the algorithm is to maximize throughput, without considering Quality of Service (QoS), backpressure based adaptive routing can result in very long paths leading to unnecessarily excessive delays. In this paper, we propose a modification to the backpressure algorithm which forces it to first explore short paths and only use long paths when they are really needed to accommodate the offered traffic. Thus, under our proposed modification, the backpressure algorithm continues to be throughputoptimal, but it pays attention to the delay performance of the network. We also refer the reader to a related work in [12] where the authors use the same cost function as us, but their formulation is different and hence their solution is also different.
2 System Model
Let us consider a network modeled by a graph, where is the set of nodes and is the set of links. We assume that time is slotted, with a typical time slot denoted by If a link is in then it is possible to transmit packets from node to node subject to the interference constraints which will be described shortly.
We let be the set of flows that share the network resources. Packets of each flow enter the network at one node, travel along multiple hops (which may or may not predetermined), and then exit the network at another node. For each flow let denote the begin (entering) node, and denote the end (exiting) node of flow
We define a valid schedule to be a set of link rates (measured in terms of number of packets) that can be simultaneously supported. Note that due to the interference between links, for each some could be zero. Moreover, we make a natural and nonrestrictive assumption that if is a valid schedule, then if we replace any subset of its components by zeros, the modified schedule is valid as well. We also assume that is upperbounded by some for any and Let be the set of all possible valid schedules, and denote the convex hull of
If the routes of flows are not predetermined, i.e., when adaptive routing is used, then the capacity region of the network is defined as the set of all flow rates which are supportable by the network. Tassiulas and Ephremides [1] have shown that if

there exists a

for any link there exists some allocation such that and

for any node for all flows
The term can be interpreted as the longterm average rate that link allocates to serve flow Note that the equation in the third bullet above is simply the law of flow conservation.
Now, if the routes of flows are predetermined, i.e., when fixed routing is used, then for each let denote the set of links forming the route of The capacity region of the network is defined as the set of all flow rates which are supportable given a set of flows and their corresponding routes. In the case of fixed routing, if there exists a such that
The traffic in the network can be elastic or inelastic. If the traffic is inelastic, i.e., the flows’ rates are fixed (and within the capacity region), then the goal is to route/schedule the traffic through the network while ensuring that the queues in the network are stable. If the traffic is elastic, then the goal is to allocate the network’s resources to all flows in some fair manner. More precisely, suppose that each flow has a utility function associated with it. The utility function of flow denoted by is defined as a function of the data rate sent by flow and assumed to be concave and nondecreasing. The goal, in the case of elastic traffic, is to determine the optimal solution to the following resource allocation problem:
(1)  
s.t. 
where is the capacity region described above.
It has been shown that, for inelastic traffic, the backpressure algorithm is throughputoptimal. Furthermore, for elastic traffic, a joint congestion control and backpressure routing/scheduling algorithm has been shown to be able to solve the resource allocation problem (1). However, as we mentioned in Section 1, the delay performance of such algorithms can be quite poor. In the subsequent sections, we describe our architectures and algorithms in detail.
3 The Shadow Algorithm
In this section, we consider networks with fixed routing, and propose an architecture to reduce delays and reduce the number of queues maintained at each node. The main idea is to use a fictitious queueing system called the shadow queueing system to perform flow control and resource allocation in the network while using only a single physical FIFO queue for each outgoing link (also known as perneighbor queueing) at each node. The idea of shadow queues was introduced in [10], but the main goal there was to extend the network utility maximization framework for wireless networks to include multicast flows. However, one of the main points of this work is to show that shadow queues can be useful even in networks with unicast flows only for the purpose of delay reduction. Further, the idea of using perneighbor queueing and establishing its stability is new here.
3.1 Description
The traditional backpressure algorithm requires the queue length of every flow that passes through a node to perform resource allocation. The main idea of the shadow algorithm is to decouple the storage of this information from the queueing data structure required to store packets at each node. The details of the shadow algorithm are described as follows.
Queues and Counters: At each node, instead of keeping a separate queue for each flow as in the backpressure algorithm, a FIFO (firstcome firstserved) queue is maintained for each outgoing link. This FIFO queue stores packets for all flows going through the corresponding link. When a node receives a packet, it looks at the packet’s header: if the node is not the final destination of that packet, it will send the packet to the FIFO queue of the nexthop link; otherwise, it will deliver the packet to the upper layer. We let denote the length of the queue maintained at link and at the beginning of time slot
Each node maintains a separate shadow queue (i.e., a counter) for each flow going through it. Let be the length of the shadow queue (i.e., the value of the counter) of flow at node at the beginning of time slot The shadow queues and real queues are updated according to the scheduling algorithm described next. Note that each node still needs to keep a separate shadow queue for every flow going through it, but these are just counters, not actual physical queues. A counter is much easier to implement than a physical queue.
Backpressure scheduling using the shadow queue lengths: At time slot

Each link looks at the maximum shadow differential backlog of all flows going through that link:
(2) 
Backpressure scheduling:
(3) 
A schedule is interpreted by the network as follows: link transmits shadow packets from the shadow queue of the flow whose differential backlog achieves the maximum in (2) (if the shadow queue has fewer than packets, then it is emptied); link also transmits as many real packets as shadow packets from its real FIFO queue. Again, if the number of real packets in the queue is less than the number of transmitted shadow packets, then all the real packets are transmitted.
We recall that shadow queues are just counters. The action of “transmitting shadow packets” is simply the action of updating the counters’ values. In other words, “transmitting” shadow packets from to means that we subtract from and add to From the above description, it should be clear that the shadow packets can be interpreted as permits which allow a link to transmit. Unlike the traditional backpressure algorithm, the permits are associated with just a link rather than with a link and a flow.
Congestion control at the source: At time slot the source of flow computes the rate at which it injects packets into the ingress shadow queue as follows:
(4) 
where is an upperbound of the arrival rates, and is a positive parameter. The source also generates real traffic at rate where is a positive number less than If and are not integers, the actual number of shadow and real packets generated can be random variables with these expected values. Since the shadow packets are permits that allow realpacket transmission, from basic queueing theory, it follows that the actual packet arrival rate must be slightly smaller than the shadow packet arrival rate to ensure the stability of real queues. The parameter is chosen to be less than for this purpose. As we will see later in simulations, the queue backlog in the network would be smaller for smaller values of
The above description of the shadow algorithm applies to elastic traffic. For inelastic traffic, the same shadow algorithm can be used without congestion control. To ensure stability of the real queues, if the real arrival rate of an inelastic flow is the shadow arrival rate for this flow must be larger than For example, if we wish to make the shadow arrival rate larger than the real arrival rate by a factor of it can be accomplished as follows: for every real packet arrival, generate a shadow packet. Generate an additional shadow packet for each real packet with probability This procedure ensures that the shadow arrival rate will be times the real arrival rate. For the algorithm to be stable, the set of arrival rates must lie in the interior of capacity region.
Alternatively, the shadow algorithm for inelastic traffic can be implemented slightly differently if we are willing to tolerate packet loss: fix the shadow arrival rate for each flow and regulate the arrival rate of real packets to be a fraction of the shadow arrival rate. For example, if the rate of shadow arrivals in a time slot is then one can inject real packets according to a Poisson distribution of mean The real packets could be stored in a queue at its ingress node, and drained at this rate to inject into the network. If the mean real arrival rate is larger than times the mean shadow arrival rate, then the real packet buffer at the edge of the network will overflow leading to packet loss. Packet loss is unavoidable for inelastic flows unless the arrival rate is less than the capacity that the network is willing to allocate to the flow. The shadow arrival rate in this case should be thought of as the networkdesignated capacity for a flow.
We note that the concept of shadow queues here is different from the notion of virtual queues used in [13] for the Internet and in [5] for wireless networks. In networks with virtual queueing systems, the arrival rates to both the real and virtual queues are the same, but the virtual queue is drained at a slower rate than the real queue. Instead, here the arrival rates to the real queues are slightly smaller than the arrival rates to the corresponding shadow queues. This subtle difference is important in that it allows us to use perneighbor FIFO queues and prove stability in a multihop wireless network in the next section.
3.2 Stability of the shadow algorithm
In this subsection, we establish the optimality and stability of the real and shadow queues. First, we note that the optimality of the resource allocation and the stability of shadow queues follow from previous results in the literature. In particular, we have the following theorem.
Theorem 1
The shadowqueuebased congestion control and scheduling algorithms described in Section 3.1 above asymptotically achieve the optimal rate allocation, i.e.,
(5) 
where is the optimal solution to (1). Furthermore, the shadow queues are stable in the sense that the Markov chain of shadow queues is positive recurrent and the steadystate expected values of the shadow queue lengths are bounded as follows:
The remaining goal is to prove the stability of the real queues. Note that the sources are sending real traffic with smaller rates than shadow traffic, and we know that the shadow queues are stable. However, it does not automatically mean that the real queues are stable as well, since each of them is an aggregated FIFO queue storing packets for all flows going through its corresponding link. Fortunately, we can apply results from the stochastic networks literature to establish the following result.
Theorem 2
The process describing the joint evolution of both shadow and real queues,
is an irreducible, aperiodic, positive recurrent Markov chain. Therefore, the real FIFO queues are also stable.
The proof is based on the fluid limit approach and a result by Bramson [14]. In his paper, Bramson proved that fluid models of Kellytype FIFO queueing networks are stable as long as the nominal load on each server is strictly less than its capacity. Thus, the basic idea of the proof is as follows. The random process describing the behavior of shadow queues, under the joint congestion control and scheduling algorithm (running on the shadow system), is positive recurrent (as specified in Theorem 1). Therefore, the average service rate on each network link that the shadow algorithm yields is strictly greater than the nominal load of the link due to the thinning of actual traffic; moreover, the (random) cumulative amount of service provided on each link up to time satisfies the functional strong law of large numbers, as goes to infinity. As a result, if we take the fluid limit of the process describing real FIFO queues, it has exactly same form as if each network link would have constant, nontimevarying capacity (equal to the average rate provided by the shadow algorithm). Then, this fluid limit is stable by the results of [14], which implies stability of the process of real queues. The proof’s details are presented in Appendix 7 just for the purpose of completeness.
Note that the real traffic throughput will always be slightly smaller than the optimal solution to (1), but this difference from the optimal solution can be made arbitrarily small by adjusting the parameter
4 Performance Comparison: BackPressure Algorithm versus the Shadow Algorithm
In this section, we compare and contrast the performances of the traditional backpressure algorithm and the shadow algorithm for networks with fixed routing.
4.1 Elastic traffic under the traditional backpressure algorithm
We present simple calculations in this section to get some feel for the performance of the traditional backpressure algorithm when it is used with congestion control. Deriving expressions for the queue backlog for generaltopology networks seems quite hard, so we confine our discussions to the case of a linear network with links as in Figure 1.
There are flows sharing this network: one flow (indexed ) goes through all links, and other flows (indexed to ) where each of them goes through each link. The capacity of each link is and for simplicity, we assume that there is no interference between links.
Let and denote the rate and the utility function flow respectively. The network utility maximization problem we would like to solve is as follows:
s.t.  
where is the resource that link allocates to serve flow
If the utility is logarithmic (proportional fairness), i.e., then one can easily compute the optimal rates and optimal queue lengths (which are the Lagrange multipliers) for the above optimization problem as follows:
(6) 
where and are the optimal queue lengths maintained at node for flow and flow , respectively. Then, the endtoend total queue length for flow is
For a more general class of utility functions which model a large class of fairness concepts [15],
we still have similar results:
(7) 
which again lead to As mentioned in the Introduction section, if a fixed rate (larger than its arrival rate) is allocated to each flow, then the total queue length in this network is expected to increase as the order of instead of
4.2 Inelastic traffic under the traditional backpressure algorithm
In the previous subsection, we showed that the combined backpressure and congestion control algorithm for elastic traffic can lead to quadratic endtoend queueing delay in terms of the number of hops. It is interesting to see whether such a behavior can also be observed in the case of inelastic traffic, i.e., the flows’ rates are fixed, and the traditional backpressure algorithm is used. The following theorem establishes an upperbound on the endtoend queue backlog for any flow.
Theorem 3
Consider a general topology network accessed by a set of flows with fixed routes. Let be the maximum number of hops in the route of any flow, i.e., Suppose the arrival rate vector is such that, for some , lies in the interior of the capacity region of the network. Assume that the arrival processes of the flows are independent of each other, independent from time slot to time slot, and have finite second moments. Then, the expected value of the sum of queue lengths (in steadystate) along the route of any flow is bounded as follows:
where constant depends only on .
The proof is presented in Appendix 8.
While the above result is only an upper bound, it suggests the quadratic growth of the total flow queue length on the flow route length. The simulation results shown next validate such quadratic growth.
4.3 Simulation results for inelastic traffic
To illustrate the queue length behavior under backpressure algorithm in the case of inelastic traffic, we simulate the linear network in Figure 1. We choose i.e., the network has nodes and links, with no interference between links. Each link has capacity i.e., it can transmit up to packets per time slot. Let be the fixed rate of flow and be the fixed rate of flows We know that the backpressure algorithm will stabilize the network as long as We let the sources send shadow traffic at fixed rates and send real traffic at a slightly smaller rate with .
Figure 2 shows the mean queue lengths of all queues maintained at each node when and The value of here is We see that the shadow queue lengths of flow increase nearly linearly when going from the end node to the begin node, which leads to a quadratic growth (in terms of the number of hops) of the endtoend queue backlog. Moreover, we also see that the real FIFO queue lengths are significantly reduced, even with a small amount thinning of traffic ().
4.4 Simulation results for elastic traffic
In this subsection, we investigate the performance of the shadow algorithm with elastic traffic in a network with a more complicated topology than a line.
In particular, we consider a grid network as shown in Figure 3. We assume that all flows have a logarithmic utility function, i.e., for all The network has nodes (represented by circles) and links (represented by dash lines). We assume a simple onehop interference model under which a matching in the graph represents a valid schedule. Each link has a capacity of i.e., it can transmit up to packets in one time slot if scheduled. There are flows (represented by arrows) sharing this network.
We implement the shadow algorithm as described in Section 3.1 with the parameter In Figure 4, we plot the evolution of total shadow queue length and total real queue length for several values of parameter (the total queue length is the sum of all queue lengths in the network). Note that the shadow queue length is also the queue length of traditional backpressure scheduling without the shadow algorithm. The figure indicates that the total real queue length with the shadow algorithm decreases dramatically compared to the traditional backpressure algorithm. Since the shadow queue length is very large compared to the real queue lengths, it is hard to see the actual values of the real queue lengths in the figure, so we present some numbers from the simulations here: after a half million time slots, the total shadow queue length is around while the total real queue lengths are only about and when is and respectively. Thus, significant gains in performance can be realized at the expense of a small loss in throughput (represented by the parameter ). Note that the traditional backpressure algorithm can perform poorly due to many reasons: (i) As in Section 4.1, if the number of hops for a flow is large, then the queue backlog can increase quadratically. (ii) The choice of the parameter in the congestion control algorithm (see Equation (4)) can lead to queue backlogs of the order of (see the upper bound in Theorem 1 and simulation results in [5]). (iii) A separate queue is maintained for each destination. The shadow algorithm solves all of these problems at once by “reserving” capacity between each sourcedestination pair, i.e., for each flow.
5 MinResource Routing using BackPressure Algorithm
In this section, we consider wireless networks where each flow’s route is not predetermined, but is adaptively chosen by the backpressure algorithm for each packet. As mentioned in the Introduction section, the backpressure algorithm explores all paths in the network and as a result may choose paths which are unnecessarily long and may even contain loops, thus leading to poor performance. We address this problem by introducing a cost function which measures the total amount of resources used by all the flows in the network. Specifically, we add up traffic load on all links in the network and use this as our cost function. In the case of inelastic flows, the goal then is to minimize this cost subject to network capacity constraints. Due to the nature of the cost function, in a network with links of equal capacity, shorter hop paths will be preferred over longer hop paths.
In the case of elastic flows, one can maximize the sum of flow utilities minus a weighted function of the cost described above, where the weight provides a tradeoff between network utility and resource usage. Since the solutions to both problems are similar, we only present the inelastic case here.
5.1 Description
Given a set of packet arrival rates that lie within the capacity region, our goal is to find the routes for flows such that as few network resources as possible are used. Thus, we formulate the following optimization problem:
(8)  
where is the rate that link allocates to serve flow i.e., and is the fixed rate of flow An algorithm that asymptotically solves the minresource routing problem (8) is as follows. (It is a special case of the algorithm in [3], where the scaling parameter is called .)
Minresource routing by backpressure: At time slot

Each node maintains a separate queue of packets for each destination ; its length is denoted . Each link is assigned a weight
(9) where is a parameter (having the same meaning as earlier in this paper).

Scheduling/routing rule:
(10) 
If the schedule says, for example, to send packets over link then link transmits up to packets from the queue to for the destination achieving the maximum in (9).
Note that the above algorithm does not change if we replace the weights in (9) by the following, rescaled ones:
(11) 
Therefore, compared with the traditional backpressure scheduling/routing, the only difference is that each link weight is equal to the maximum differential backlog minus parameter . ( reverts the algorithm to traditional.)
The performance of the stationary process which is “produced” by the algorithm with fixed parameter is within of the optimal (analogously to (5)):
where is an optimal solution to (8). However, larger means larger queues and slower convergence to the (nearly optimal) stationary regime. On the other hand, “too small” results in a stationary regime being “too far” from optimal, and queues being large for that reason. Therefore, a good value for for a practical use should be neither too large nor too small. Our simulations confirm these intuitions.
5.2 Simulation results
We ran a simulation for a network with nodes, links, and flows as in Figure 5.
We assume the nodeexclusive spectrum sharing interference model, i.e., each node can only communicate with at most one other node in any time slot. Each link has a capacity of when activated. Flow enters the network at node and exits at node while flow enters at node and exits at node Note that the flows’ routes have not been prespecified, and the described above algorithm with parameter is used.
We fix each flow’s rate at value It is easy to see that under the nodeexclusive spectrum sharing interference model, the backpressure algorithm can stabilize the network as long as The arrival processes are Poisson, i.e., the number of arrivals for each flow at each time slot is a Poisson random variable with mean Each simulation run was million timeslots long and such runs were performed. The results reported are averaged over these runs.
Table 1
Link  Rate for  Rate for  Rate for  Rate for  Rate for  Rate for 

flow  flow  flow  flow  flow  flow  
1.9492  1.9671  0.0000  0.0000  0.0000  0.0000  
1.9759  1.5622  0.0000  0.0000  0.0000  0.0000  
1.9055  2.0058  0.0000  0.0000  0.0000  0.0000  
0.0563  2.2417  0.0000  4.9998  0.0000  5.0001  
1.4913  2.4595  0.0000  0.0000  0.0000  0.0000  
2.2504  0.0466  4.9993  0.0000  4.9996  0.0000  
1.5551  1.9881  0.0000  0.0000  0.0000  0.0000  
1.2590  2.0853  0.0000  0.0000  0.0000  0.0000  
2.3592  1.5055  0.0000  0.0000  0.0000  0.0000  
2.0094  1.2535  0.0000  0.0000  0.0000  0.0000 
shows the rate allocation of each link to each flow when the value of is fixed at and for and Note that corresponds to the traditional backpressure algorithm. We see that the traditional backpressure algorithm uses all links in the network, while our modified backpressure algorithm (with or ) essentially uses only link for flow and link for flow (which are the minresource routes for these flows).
We then turn our attention to the queue backlog (the sum of all queue lengths) in the network. Figure 6
shows the queue backlog in the network corresponding to various values of We see that the queue backlog of our modified backpressure algorithm with is significantly smaller than that of the traditional backpressure algorithm. However, when is increased to the delay performance gets slightly worse. This result confirms our observation about the tradeoff in choosing the value of which is discussed at the end of Section 5.1.
6 Conclusions
In this paper, we have proposed a new shadow architecture to improve the delay performance of backpressure scheduling algorithm. The shadow queueing system allows each node to maintain a single FIFO queues for each of its outgoing links, instead of keeping a separate queue for each flow in the network. This architecture not only reduces the queue backlog (or, equivalently, delay by Little’s law) but also reduces the number of actual physical queues that each node has to maintain. Next, we proposed an algorithm that forces the backpressure algorithm to use the minimum amount of network resources while still maintaining throughput optimality. This results in better delay performance compared to the traditional backpressure algorithm.
We presented the shadow algorithm for the case of fixed routing, i.e., the route for each flow is fixed. The shadow algorithm can also be used in the case of adaptive routing, but a node cannot use just one FIFO queue for each neighbor. If one still maintains a separate queue for each destination at each node, then the extension of the shadow algorithm to the case of adaptive routing is straightforward. On the other hand, it would be interesting to study if a single perneighbor FIFO queue can be maintained even in the case of adaptive routing. This is an interesting topic for future research.
7 Proof of Theorem 2
In this appendix, we provide details of the proof of Theorem 2. First, recall the result from Theorem 1 that
(12) 
where is within boundary of the optimal solution and can be made arbitrarily small by increasing To simplify the notations, from now on, we will drop in In other words, we will use the notation for the approximate optimal solution.
From the above result, the following convergence results can be established.
Lemma 1
For every flow
i.e., the time average of real packet arrivals also converges to the optimal solution.
Consider any flow We have the sequence of flow rates and the sequence of generated shadow packets Note that given the sequence of flow rates ’s are independent Poisson random variables with means ’s. For simplicity, we drop the subscript in the notations within the scope of this proof.
Let and We have that
Now, for
i.e., and are uncorrelated. The last equality is due to the fact that and are independent given the sequence Hence,
Furthermore, Therefore,
Since is finite, the first term in the righthand side will vanish as goes to infinity. The second term also vanishes as goes to infinity due to (12). Therefore,
Lemma 2
Recall that is the outcome of the scheduling algorithm (3) at every time step Then, for every link
for some such that
In other words, the outcome of the scheduling algorithm
converges to a set of link rates that can support the given set
of flow arrival rates.
Since the Markov chain of shadow queues is positive recurrent, the proof follows the ergodic theorem and the fact that is upperbounded.
To be consistent with [14], we introduce the concept of packet class. Each flow consists of packet classes; each class going through one link in the route of We let denote the set of all packet classes. In other words, there is a bijection mapping a pair to a packet class Clearly,
For each flow let be the set of packet classes belonging to For each link let be the set of packet classes going through Conversely, for each packet class let be the corresponding flow (i.e., ), and be the corresponding link.
Let denote the constituency matrix with size :
Also, let be the routing matrix with size :
Next, let denote the total external arrivals of packet class up to time Thus,
Also, we define the arrival rates corresponding to packet classes:
We then extend the definition of for continuous time as follows: for each time Hence, is right continuous having left limits in time.
Recall that is the outcome of the scheduling algorithm at time slot Now, for each we let denote the total amount of service (in terms of number of packets that can be transmitted) of link up to time Now, for each let us define and define
(13) 
Similarly, let us define as the total arrivals, and as the total departures, of packet class up to time Thus,
(14) 
Let be the number of packets of packet class which are waiting to be served. Then,
(15) 
Recall that is the length of FIFO queue at link at time Thus,
(16) 
Now, we define

as the amount of time that the server at link has spent serving packet class in

as the amount idle time of the server at link during

as the immediate workload at the server of link measured in units of time.
Then we have the following equations:
(17) 
(18) 
and the fact that can only increase when i.e., if then for some
We can rewrite the above equations (14)(18) in vector form to get the following set of equations which describes the evolution of the system:
(19)  
(20)  
(21)  
(22)  
(23)  
(24) 
where is defined in (13) and Additionally, we have that
(25)  
(26) 
where Equation (25) comes from the fact that each class has at most one packet being served at any time, and Equation (26) comes from the FIFO property of the real queues.
Note that and are right continuous having left limits, while and are continuous in time. We also assume that
Let us define
where is the shadow queue of class Then is a Markov process. Furthermore, and are themselves Markov processes. By Theorem 1, we know that is positive recurrent.
We now describe the fluid model of the system. The set of fluid model equations is as follows:
(27)  
(28)  
(29)  
(30)  
(31)  
(32)  
(33)  
(34) 
where and Recall that is defined in Lemma 2 as the set of supporting link rates. Equation (32) means that for each whenever there exists such that i.e., is constant in
7.1 Preliminaries
Theorem 4 (see [16])
If random variables and satisfy that where the notation denotes the convergence in distribution (weak convergence), and if the are uniformly integrable, i.e.,
then is integrable and
Consider the sequence of scaled processes
then we have the following theorem:
Theorem 5 ([17, 18])
Suppose that, for any sequence of scaled processes satisfying there exist a subsequence and a constant such that
Then the queueing system is stable, in the sense that the Markov process is positive recurrent.
Corollary 1
Suppose that there exists a deterministic function
such that the following conditions hold:
 [(i)]

For any sequence there exists a subsequence such that as

For any satisfying there exists a such that

is uniformly integrable for all
then the original process is positive recurrent.
7.2 Proof’s details
For an integer let be the set of functions that are right continuous on having the left limits on For we use to denote By convention,
Let us endow the function space with the Skorohod