Novel Architectures and Algorithms for Delay Reduction in Back-pressure Scheduling and Routing

# Novel Architectures and Algorithms for Delay Reduction in Back-pressure Scheduling and Routing

## Abstract

The back-pressure algorithm is a well-known throughput-optimal algorithm. However, its delay performance may be quite poor even when the traffic load is not close to network capacity due to the following two reasons. First, each node has to maintain a separate queue for each commodity in the network, and only one queue is served at a time. Second, the back-pressure routing algorithm may route some packets along very long routes. In this paper, we present solutions to address both of the above issues, and hence, improve the delay performance of the back-pressure algorithm. One of the suggested solutions also decreases the complexity of the queueing data structures to be maintained at each node.

## 1 Introduction

Resource allocation in wireless networks is complicated due to the shared nature of wireless medium. One particular allocation algorithm called the back-pressure algorithm which encompasses several layers of the protocol stack from MAC to routing was proposed by Tassiulas and Ephremides, in their seminal paper [1]. The back-pressure algorithm was shown to be throughput-optimal, i.e., it can support any arrival rate vector which is supportable by any other resource allocation algorithm. Recently, it was shown that the back-pressure algorithm can be combined with congestion control to fairly allocate resources among competing users in a wireless network [2, 3, 4, 5, 6, 7], thus providing a complete resource allocation solution from the transport layer to the MAC layer. While such a combined algorithm can be used to perform a large variety of resource allocation tasks, in this paper, we will concentrate on its application to scheduling and routing.

Even though the back-pressure algorithm delivers maximum throughput by adapting itself to network conditions, there are several issues that have to be addressed before it can be widely deployed in practice. As stated in the original paper [1], the back-pressure algorithm requires centralized information and computation, and its computational complexity is too prohibitive for practice. Much progress has been made recently in easing the computational complexity and deriving decentralized heuristics. We refer the interested reader to [8, 9] and references within for some recent results along these lines. We do not consider complexity or decentralization issues in this paper; our proposed solutions can be approximated well by the solutions suggested in the above papers.

Besides complexity and decentralization issues which have received much attention recently, the back-pressure algorithm can also have poor delay performance. To understand that, we consider two different network scenarios: one in which the back-pressure algorithm is used to adaptively select a route for each packet, and the other in which a flow’s route is chosen upon arrival by some standard multi-hop wireless network routing algorithm such as DSR or AODV and the back-pressure algorithm is simply used to schedule packets. We refer to the first case as adaptive-routing and the second case as fixed-routing, respectively.

We will also show that the concept of shadow queues can reduce the number of real queues maintained at each node significantly. In particular, we will show that it is sufficient to maintain per-neighbor queues at each node, instead of per-flow queues required by the back-pressure algorithm in the case of fixed routing. In large networks, the number of flows is typically much larger compared to the number of neighbors of each node, thus using per-neighbor queues can result in significant reduction in implementation complexity. A different idea to reduce the number of queues at each node has been proposed in [11], but the implementation using shadow queues has the additional benefit of delay reduction.

Next, we discuss the case of adaptive routing. The back-pressure algorithm tends to explore many routes to find sufficient capacity in the network to accommodate the offered traffic. Since the goal of the algorithm is to maximize throughput, without considering Quality of Service (QoS), back-pressure based adaptive routing can result in very long paths leading to unnecessarily excessive delays. In this paper, we propose a modification to the back-pressure algorithm which forces it to first explore short paths and only use long paths when they are really needed to accommodate the offered traffic. Thus, under our proposed modification, the back-pressure algorithm continues to be throughput-optimal, but it pays attention to the delay performance of the network. We also refer the reader to a related work in [12] where the authors use the same cost function as us, but their formulation is different and hence their solution is also different.

## 2 System Model

Let us consider a network modeled by a graph, where is the set of nodes and is the set of links. We assume that time is slotted, with a typical time slot denoted by If a link is in then it is possible to transmit packets from node to node subject to the interference constraints which will be described shortly.

We let be the set of flows that share the network resources. Packets of each flow enter the network at one node, travel along multiple hops (which may or may not pre-determined), and then exit the network at another node. For each flow let denote the begin (entering) node, and denote the end (exiting) node of flow

We define a valid schedule to be a set of link rates (measured in terms of number of packets) that can be simultaneously supported. Note that due to the interference between links, for each some could be zero. Moreover, we make a natural and nonrestrictive assumption that if is a valid schedule, then if we replace any subset of its components by zeros, the modified schedule is valid as well. We also assume that is upper-bounded by some for any and Let be the set of all possible valid schedules, and denote the convex hull of

If the routes of flows are not predetermined, i.e., when adaptive routing is used, then the capacity region of the network is defined as the set of all flow rates which are supportable by the network. Tassiulas and Ephremides [1] have shown that if

• there exists a

• for any link there exists some allocation such that and

• for any node for all flows

 λfI{n=b(f)}+∑(k,n)μfkn=∑(n,m)μfnm.

The term can be interpreted as the long-term average rate that link allocates to serve flow Note that the equation in the third bullet above is simply the law of flow conservation.

Now, if the routes of flows are predetermined, i.e., when fixed routing is used, then for each let denote the set of links forming the route of The capacity region of the network is defined as the set of all flow rates which are supportable given a set of flows and their corresponding routes. In the case of fixed routing, if there exists a such that

 ∑f:l∈L(f)λf≤μl,∀l∈L.

The traffic in the network can be elastic or inelastic. If the traffic is inelastic, i.e., the flows’ rates are fixed (and within the capacity region), then the goal is to route/schedule the traffic through the network while ensuring that the queues in the network are stable. If the traffic is elastic, then the goal is to allocate the network’s resources to all flows in some fair manner. More precisely, suppose that each flow has a utility function associated with it. The utility function of flow denoted by is defined as a function of the data rate sent by flow and assumed to be concave and nondecreasing. The goal, in the case of elastic traffic, is to determine the optimal solution to the following resource allocation problem:

 max ∑f∈FUf(xf) (1) s.t. x∈Λ,

where is the capacity region described above.

It has been shown that, for inelastic traffic, the back-pressure algorithm is throughput-optimal. Furthermore, for elastic traffic, a joint congestion control and back-pressure routing/scheduling algorithm has been shown to be able to solve the resource allocation problem (1). However, as we mentioned in Section 1, the delay performance of such algorithms can be quite poor. In the subsequent sections, we describe our architectures and algorithms in detail.

In this section, we consider networks with fixed routing, and propose an architecture to reduce delays and reduce the number of queues maintained at each node. The main idea is to use a fictitious queueing system called the shadow queueing system to perform flow control and resource allocation in the network while using only a single physical FIFO queue for each outgoing link (also known as per-neighbor queueing) at each node. The idea of shadow queues was introduced in [10], but the main goal there was to extend the network utility maximization framework for wireless networks to include multicast flows. However, one of the main points of this work is to show that shadow queues can be useful even in networks with unicast flows only for the purpose of delay reduction. Further, the idea of using per-neighbor queueing and establishing its stability is new here.

### 3.1 Description

The traditional back-pressure algorithm requires the queue length of every flow that passes through a node to perform resource allocation. The main idea of the shadow algorithm is to decouple the storage of this information from the queueing data structure required to store packets at each node. The details of the shadow algorithm are described as follows.

Queues and Counters: At each node, instead of keeping a separate queue for each flow as in the back-pressure algorithm, a FIFO (first-come first-served) queue is maintained for each outgoing link. This FIFO queue stores packets for all flows going through the corresponding link. When a node receives a packet, it looks at the packet’s header: if the node is not the final destination of that packet, it will send the packet to the FIFO queue of the next-hop link; otherwise, it will deliver the packet to the upper layer. We let denote the length of the queue maintained at link and at the beginning of time slot

Each node maintains a separate shadow queue (i.e., a counter) for each flow going through it. Let be the length of the shadow queue (i.e., the value of the counter) of flow at node at the beginning of time slot The shadow queues and real queues are updated according to the scheduling algorithm described next. Note that each node still needs to keep a separate shadow queue for every flow going through it, but these are just counters, not actual physical queues. A counter is much easier to implement than a physical queue.

Back-pressure scheduling using the shadow queue lengths: At time slot

• Each link looks at the maximum shadow differential backlog of all flows going through that link:

 wnm[t]=maxf:(n,m)∈L(f)(~Qfn[t]−~Qfm[t]). (2)
• Back-pressure scheduling:

 π∗[t]=maxπ∈Γ∑(n,m)cπnmwnm[t]. (3)
• A schedule is interpreted by the network as follows: link transmits shadow packets from the shadow queue of the flow whose differential backlog achieves the maximum in (2) (if the shadow queue has fewer than packets, then it is emptied); link also transmits as many real packets as shadow packets from its real FIFO queue. Again, if the number of real packets in the queue is less than the number of transmitted shadow packets, then all the real packets are transmitted.

We recall that shadow queues are just counters. The action of “transmitting shadow packets” is simply the action of updating the counters’ values. In other words, “transmitting” shadow packets from to means that we subtract from and add to From the above description, it should be clear that the shadow packets can be interpreted as permits which allow a link to transmit. Unlike the traditional back-pressure algorithm, the permits are associated with just a link rather than with a link and a flow.

Congestion control at the source: At time slot the source of flow computes the rate at which it injects packets into the ingress shadow queue as follows:

 xf[t]=min⎧⎪⎨⎪⎩U′−1f⎛⎜⎝~Qfb(f)[t]M⎞⎟⎠,xmax⎫⎪⎬⎪⎭, (4)

where is an upper-bound of the arrival rates, and is a positive parameter. The source also generates real traffic at rate where is a positive number less than If and are not integers, the actual number of shadow and real packets generated can be random variables with these expected values. Since the shadow packets are permits that allow real-packet transmission, from basic queueing theory, it follows that the actual packet arrival rate must be slightly smaller than the shadow packet arrival rate to ensure the stability of real queues. The parameter is chosen to be less than for this purpose. As we will see later in simulations, the queue backlog in the network would be smaller for smaller values of

The above description of the shadow algorithm applies to elastic traffic. For inelastic traffic, the same shadow algorithm can be used without congestion control. To ensure stability of the real queues, if the real arrival rate of an inelastic flow is the shadow arrival rate for this flow must be larger than For example, if we wish to make the shadow arrival rate larger than the real arrival rate by a factor of it can be accomplished as follows: for every real packet arrival, generate a shadow packet. Generate an additional shadow packet for each real packet with probability This procedure ensures that the shadow arrival rate will be times the real arrival rate. For the algorithm to be stable, the set of arrival rates must lie in the interior of capacity region.

Alternatively, the shadow algorithm for inelastic traffic can be implemented slightly differently if we are willing to tolerate packet loss: fix the shadow arrival rate for each flow and regulate the arrival rate of real packets to be a fraction of the shadow arrival rate. For example, if the rate of shadow arrivals in a time slot is then one can inject real packets according to a Poisson distribution of mean The real packets could be stored in a queue at its ingress node, and drained at this rate to inject into the network. If the mean real arrival rate is larger than times the mean shadow arrival rate, then the real packet buffer at the edge of the network will overflow leading to packet loss. Packet loss is unavoidable for inelastic flows unless the arrival rate is less than the capacity that the network is willing to allocate to the flow. The shadow arrival rate in this case should be thought of as the network-designated capacity for a flow.

We note that the concept of shadow queues here is different from the notion of virtual queues used in [13] for the Internet and in [5] for wireless networks. In networks with virtual queueing systems, the arrival rates to both the real and virtual queues are the same, but the virtual queue is drained at a slower rate than the real queue. Instead, here the arrival rates to the real queues are slightly smaller than the arrival rates to the corresponding shadow queues. This subtle difference is important in that it allows us to use per-neighbor FIFO queues and prove stability in a multihop wireless network in the next section.

### 3.2 Stability of the shadow algorithm

In this subsection, we establish the optimality and stability of the real and shadow queues. First, we note that the optimality of the resource allocation and the stability of shadow queues follow from previous results in the literature. In particular, we have the following theorem.

###### Theorem 1

The shadow-queue-based congestion control and scheduling algorithms described in Section 3.1 above asymptotically achieve the optimal rate allocation, i.e.,

 limT→∞1TT−1∑t=0E[x[t]]=x∗+O(1/M), (5)

where is the optimal solution to (1). Furthermore, the shadow queues are stable in the sense that the Markov chain of shadow queues is positive recurrent and the steady-state expected values of the shadow queue lengths are bounded as follows:

 ∑n,fE(~Qfn[∞])=O(M).
{IEEEproof}

The proof of this theorem was presented in [6, 3, 7], and hence, is omitted here.

The remaining goal is to prove the stability of the real queues. Note that the sources are sending real traffic with smaller rates than shadow traffic, and we know that the shadow queues are stable. However, it does not automatically mean that the real queues are stable as well, since each of them is an aggregated FIFO queue storing packets for all flows going through its corresponding link. Fortunately, we can apply results from the stochastic networks literature to establish the following result.

###### Theorem 2

The process describing the joint evolution of both shadow and real queues,

 ((~Qfn[t])f∈F,n∈N;(Pnm[t])(n,m)∈L),

is an irreducible, aperiodic, positive recurrent Markov chain. Therefore, the real FIFO queues are also stable.

The proof is based on the fluid limit approach and a result by Bramson [14]. In his paper, Bramson proved that fluid models of Kelly-type FIFO queueing networks are stable as long as the nominal load on each server is strictly less than its capacity. Thus, the basic idea of the proof is as follows. The random process describing the behavior of shadow queues, under the joint congestion control and scheduling algorithm (running on the shadow system), is positive recurrent (as specified in Theorem 1). Therefore, the average service rate on each network link that the shadow algorithm yields is strictly greater than the nominal load of the link due to the thinning of actual traffic; moreover, the (random) cumulative amount of service provided on each link up to time satisfies the functional strong law of large numbers, as goes to infinity. As a result, if we take the fluid limit of the process describing real FIFO queues, it has exactly same form as if each network link would have constant, non-time-varying capacity (equal to the average rate provided by the shadow algorithm). Then, this fluid limit is stable by the results of [14], which implies stability of the process of real queues. The proof’s details are presented in Appendix 7 just for the purpose of completeness.

Note that the real traffic throughput will always be slightly smaller than the optimal solution to (1), but this difference from the optimal solution can be made arbitrarily small by adjusting the parameter

## 4 Performance Comparison: Back-Pressure Algorithm versus the Shadow Algorithm

In this section, we compare and contrast the performances of the traditional back-pressure algorithm and the shadow algorithm for networks with fixed routing.

### 4.1 Elastic traffic under the traditional back-pressure algorithm

We present simple calculations in this section to get some feel for the performance of the traditional back-pressure algorithm when it is used with congestion control. Deriving expressions for the queue backlog for general-topology networks seems quite hard, so we confine our discussions to the case of a linear network with links as in Figure 1.

There are flows sharing this network: one flow (indexed ) goes through all links, and other flows (indexed to ) where each of them goes through each link. The capacity of each link is and for simplicity, we assume that there is no interference between links.

Let and denote the rate and the utility function flow respectively. The network utility maximization problem we would like to solve is as follows:

 max N∑i=0Ui(xi) s.t. x0 ≤ μ0,1, μ0,i ≤ μ0,i+1, i=1,…,N−1, xi+μ0,i ≤ c, i=1,…,N,

where is the resource that link allocates to serve flow

If the utility is logarithmic (proportional fairness), i.e., then one can easily compute the optimal rates and optimal queue lengths (which are the Lagrange multipliers) for the above optimization problem as follows:

 x∗0=μ∗0,1=…=μ∗0,N=cN+1, x∗1=…=x∗N=NcN+1, q∗i=q∗0,i−q∗0,i+1=N+1Nc, i=1,…,N, (6)

where and are the optimal queue lengths maintained at node for flow and flow , respectively. Then, the end-to-end total queue length for flow is

 N∑i=1q∗0,i=N+1NcN∑i=1i=(N+1)22c=Θ(N2).

For a more general class of utility functions which model a large class of fairness concepts [15],

 Ui(x)=x1−α1−α,α>0,

we still have similar results:

 x∗0=μ∗0,1=…=μ∗0,N=Θ(N−1/α), x∗1=…=x∗N=Θ(1), q∗i=q∗0,i−q∗0,i+1=Θ(1), i=1,…,N, (7)

which again lead to As mentioned in the Introduction section, if a fixed rate (larger than its arrival rate) is allocated to each flow, then the total queue length in this network is expected to increase as the order of instead of

### 4.2 Inelastic traffic under the traditional back-pressure algorithm

In the previous subsection, we showed that the combined back-pressure and congestion control algorithm for elastic traffic can lead to quadratic end-to-end queueing delay in terms of the number of hops. It is interesting to see whether such a behavior can also be observed in the case of inelastic traffic, i.e., the flows’ rates are fixed, and the traditional back-pressure algorithm is used. The following theorem establishes an upper-bound on the end-to-end queue backlog for any flow.

###### Theorem 3

Consider a general topology network accessed by a set of flows with fixed routes. Let be the maximum number of hops in the route of any flow, i.e., Suppose the arrival rate vector is such that, for some , lies in the interior of the capacity region of the network. Assume that the arrival processes of the flows are independent of each other, independent from time slot to time slot, and have finite second moments. Then, the expected value of the sum of queue lengths (in steady-state) along the route of any flow is bounded as follows:

 E⎡⎣∑n∈R(f)Qfn[∞]⎤⎦ ≤ 1+ϵϵbλf|F|K2max , ∀f∈F,

where constant depends only on .

{IEEEproof}

The proof is presented in Appendix 8.

While the above result is only an upper bound, it suggests the quadratic growth of the total flow queue length on the flow route length. The simulation results shown next validate such quadratic growth.

### 4.3 Simulation results for inelastic traffic

To illustrate the queue length behavior under back-pressure algorithm in the case of inelastic traffic, we simulate the linear network in Figure 1. We choose i.e., the network has nodes and links, with no interference between links. Each link has capacity i.e., it can transmit up to packets per time slot. Let be the fixed rate of flow and be the fixed rate of flows We know that the back-pressure algorithm will stabilize the network as long as We let the sources send shadow traffic at fixed rates and send real traffic at a slightly smaller rate with .

Figure 2 shows the mean queue lengths of all queues maintained at each node when and The value of here is We see that the shadow queue lengths of flow increase nearly linearly when going from the end node to the begin node, which leads to a quadratic growth (in terms of the number of hops) of the end-to-end queue backlog. Moreover, we also see that the real FIFO queue lengths are significantly reduced, even with a small amount thinning of traffic ().

### 4.4 Simulation results for elastic traffic

In this subsection, we investigate the performance of the shadow algorithm with elastic traffic in a network with a more complicated topology than a line.

In particular, we consider a grid network as shown in Figure 3. We assume that all flows have a logarithmic utility function, i.e., for all The network has nodes (represented by circles) and links (represented by dash lines). We assume a simple one-hop interference model under which a matching in the graph represents a valid schedule. Each link has a capacity of i.e., it can transmit up to packets in one time slot if scheduled. There are flows (represented by arrows) sharing this network.

## 5 Min-Resource Routing using Back-Pressure Algorithm

In this section, we consider wireless networks where each flow’s route is not pre-determined, but is adaptively chosen by the back-pressure algorithm for each packet. As mentioned in the Introduction section, the back-pressure algorithm explores all paths in the network and as a result may choose paths which are unnecessarily long and may even contain loops, thus leading to poor performance. We address this problem by introducing a cost function which measures the total amount of resources used by all the flows in the network. Specifically, we add up traffic load on all links in the network and use this as our cost function. In the case of inelastic flows, the goal then is to minimize this cost subject to network capacity constraints. Due to the nature of the cost function, in a network with links of equal capacity, shorter hop paths will be preferred over longer hop paths.

In the case of elastic flows, one can maximize the sum of flow utilities minus a weighted function of the cost described above, where the weight provides a tradeoff between network utility and resource usage. Since the solutions to both problems are similar, we only present the inelastic case here.

### 5.1 Description

Given a set of packet arrival rates that lie within the capacity region, our goal is to find the routes for flows such that as few network resources as possible are used. Thus, we formulate the following optimization problem:

 min ∑(n,m)μnm (8) s.t. λfI{n=b(f)}+∑(k,n)μfkn ≤ ∑(n,m)μfnm,∀f∈F,n∈N, {μnm}(n,m)∈L ∈ co(Γ),

where is the rate that link allocates to serve flow i.e., and is the fixed rate of flow An algorithm that asymptotically solves the min-resource routing problem (8) is as follows. (It is a special case of the algorithm in [3], where the scaling parameter is called .)

Min-resource routing by back-pressure: At time slot

• Each node maintains a separate queue of packets for each destination ; its length is denoted . Each link is assigned a weight

 wnm[t]=maxd∈D(1MQdn[t]−1MQdm[t]−1), (9)

where is a parameter (having the same meaning as earlier in this paper).

• Scheduling/routing rule:

 π∗[t]=maxπ∈Γ∑(n,m)πnmwnm[t]. (10)
• If the schedule says, for example, to send packets over link then link transmits up to packets from the queue to for the destination achieving the maximum in (9).

Note that the above algorithm does not change if we replace the weights in (9) by the following, re-scaled ones:

 wnm[t]=maxd∈D(Qdn[t]−Qdm[t]−M). (11)

Therefore, compared with the traditional back-pressure scheduling/routing, the only difference is that each link weight is equal to the maximum differential backlog minus parameter . ( reverts the algorithm to traditional.)

The performance of the stationary process which is “produced” by the algorithm with fixed parameter is within of the optimal (analogously to (5)):

 ∣∣ ∣∣ E⎡⎣∑(n,m)π∗nm[t]⎤⎦−∑(n,m)μ∗nm ∣∣ ∣∣=O(1/M),

where is an optimal solution to (8). However, larger means larger queues and slower convergence to the (nearly optimal) stationary regime. On the other hand, “too small” results in a stationary regime being “too far” from optimal, and queues being large for that reason. Therefore, a good value for for a practical use should be neither too large nor too small. Our simulations confirm these intuitions.

### 5.2 Simulation results

We ran a simulation for a network with nodes, links, and flows as in Figure 5.

We assume the node-exclusive spectrum sharing interference model, i.e., each node can only communicate with at most one other node in any time slot. Each link has a capacity of when activated. Flow enters the network at node and exits at node while flow enters at node and exits at node Note that the flows’ routes have not been prespecified, and the described above algorithm with parameter is used.

We fix each flow’s rate at value It is easy to see that under the node-exclusive spectrum sharing interference model, the back-pressure algorithm can stabilize the network as long as The arrival processes are Poisson, i.e., the number of arrivals for each flow at each time slot is a Poisson random variable with mean Each simulation run was million time-slots long and such runs were performed. The results reported are averaged over these runs.

Table 1

shows the rate allocation of each link to each flow when the value of is fixed at and for and Note that corresponds to the traditional back-pressure algorithm. We see that the traditional back-pressure algorithm uses all links in the network, while our modified back-pressure algorithm (with or ) essentially uses only link for flow and link for flow (which are the min-resource routes for these flows).

We then turn our attention to the queue backlog (the sum of all queue lengths) in the network. Figure 6

shows the queue backlog in the network corresponding to various values of We see that the queue backlog of our modified back-pressure algorithm with is significantly smaller than that of the traditional back-pressure algorithm. However, when is increased to the delay performance gets slightly worse. This result confirms our observation about the trade-off in choosing the value of which is discussed at the end of Section 5.1.

## 6 Conclusions

In this paper, we have proposed a new shadow architecture to improve the delay performance of back-pressure scheduling algorithm. The shadow queueing system allows each node to maintain a single FIFO queues for each of its outgoing links, instead of keeping a separate queue for each flow in the network. This architecture not only reduces the queue backlog (or, equivalently, delay by Little’s law) but also reduces the number of actual physical queues that each node has to maintain. Next, we proposed an algorithm that forces the back-pressure algorithm to use the minimum amount of network resources while still maintaining throughput optimality. This results in better delay performance compared to the traditional back-pressure algorithm.

We presented the shadow algorithm for the case of fixed routing, i.e., the route for each flow is fixed. The shadow algorithm can also be used in the case of adaptive routing, but a node cannot use just one FIFO queue for each neighbor. If one still maintains a separate queue for each destination at each node, then the extension of the shadow algorithm to the case of adaptive routing is straightforward. On the other hand, it would be interesting to study if a single per-neighbor FIFO queue can be maintained even in the case of adaptive routing. This is an interesting topic for future research.

\appendices

## 7 Proof of Theorem 2

In this appendix, we provide details of the proof of Theorem 2. First, recall the result from Theorem 1 that

 limT→∞1TT−1∑t=0E[x[t]]=x∗(ϵ), (12)

where is within -boundary of the optimal solution and can be made arbitrarily small by increasing To simplify the notations, from now on, we will drop in In other words, we will use the notation for the -approximate optimal solution.

From the above result, the following convergence results can be established.

###### Lemma 1

For every flow

 1TT−1∑t=0af[t] \lx@stackrelT→∞⟶ βx∗fm.s.,

i.e., the time average of real packet arrivals also converges to the optimal solution.

{IEEEproof}

Consider any flow We have the sequence of flow rates and the sequence of generated shadow packets Note that given the sequence of flow rates ’s are independent Poisson random variables with means ’s. For simplicity, we drop the subscript in the notations within the scope of this proof.

Let and We have that

 E⎡⎣(1TT∑i=1a[i]−βx∗)2⎤⎦ = 1T2E⎡⎣(T∑i=1(a[i]−βx∗))2⎤⎦ = 1T2T∑i=1E[(a[i]−βx∗)2] + 1T2T∑i=1T∑j=1,j≠iE[(a[i]−βx∗)(a[j]−βx∗)].

Now, for

 Cov(a[i],a[j]) = E[(a[i]−¯a[i])(a[j]−¯a[j])] = E[E[(a[i]−¯a[i])(a[j]−¯a[j])|{x[k]}∞k=0]] = 0,

i.e., and are uncorrelated. The last equality is due to the fact that and are independent given the sequence Hence,

 E[(a[i]−βx∗)(a[j]−βx∗)] = E[(a[i]−¯a[i])(a[j]−¯a[j])]+(¯a[i]−βx∗)(¯a[j]−βx∗) = (¯a[i]−βx∗)(¯a[j]−βx∗).

Furthermore, Therefore,

 E⎡⎣(1TT∑i=1a[i]−βx∗)2⎤⎦ = 1T2T∑i=1Var(a[i])+1T2T∑i=1T∑j=1(¯a[i]−βx∗)(¯a[j]−βx∗) = 1T2T∑i=1Var(a[i])+[1TT∑i=1(¯a[i]−βx∗)]2.

Since is finite, the first term in the right-hand side will vanish as goes to infinity. The second term also vanishes as goes to infinity due to (12). Therefore,

 limT→∞E⎡⎣(1TT∑i=1a[i]−βx∗)2⎤⎦ = 0
###### Lemma 2

Recall that is the outcome of the scheduling algorithm (3) at every time step Then, for every link

 1TT−1∑t=0π∗l[t] \lx@stackrelT→∞⟶ μ∗la.s.

for some such that

In other words, the outcome of the scheduling algorithm converges to a set of link rates that can support the given set of flow arrival rates.

{IEEEproof}

Since the Markov chain of shadow queues is positive recurrent, the proof follows the ergodic theorem and the fact that is upper-bounded.

To be consistent with [14], we introduce the concept of packet class. Each flow consists of packet classes; each class going through one link in the route of We let denote the set of all packet classes. In other words, there is a bijection mapping a pair to a packet class Clearly,

For each flow let be the set of packet classes belonging to For each link let be the set of packet classes going through Conversely, for each packet class let be the corresponding flow (i.e., ), and be the corresponding link.

Let denote the constituency matrix with size :

 Hl,s = {1if s∈C(l),0otherwise.

Also, let be the routing matrix with size :

 Rs,u = {1if f(s)=f(u) and u is the % next hop of s in the route of f,0otherwise.

Next, let denote the total external arrivals of packet class up to time Thus,

 Es(t) = ⎧⎪ ⎪⎨⎪ ⎪⎩t−1∑k=0af[k]if s is the first hop of f(s),0otherwise.

Also, we define the arrival rates corresponding to packet classes:

 λs = {x∗fif s is% the first hop of f(s),0otherwise.

We then extend the definition of for continuous time as follows: for each time Hence, is right continuous having left limits in time.

Recall that is the outcome of the scheduling algorithm at time slot Now, for each we let denote the total amount of service (in terms of number of packets that can be transmitted) of link up to time Now, for each let us define and define

 M(t):=diag(⌊t⌋m1(t),⌊t⌋m2(t),…,⌊t⌋m|S|(t)). (13)

Similarly, let us define as the total arrivals, and as the total departures, of packet class up to time Thus,

 As(t)=Es(t)+∑u∈SDu(t)Ru,s. (14)

Let be the number of packets of packet class which are waiting to be served. Then,

 Qs(t)=Qs(0)+As(t)−Ds(t). (15)

Recall that is the length of FIFO queue at link at time Thus,

 Pl(t)=∑s∈C(l)Qs(t)=∑sHl,sQs(t). (16)

Now, we define

• as the amount of time that the server at link has spent serving packet class in

• as the amount idle time of the server at link during

• as the immediate workload at the server of link measured in units of time.

Then we have the following equations:

 ∑sHl,sTs(t)+Il(t)=t (17)
 Wl(t)=⌊t⌋Ml(t)∑sHl,s(As(t)+Qs(0))−∑sHl,sTs(t), (18)

and the fact that can only increase when i.e., if then for some

We can rewrite the above equations (14)-(18) in vector form to get the following set of equations which describes the evolution of the system:

 A(t)=E(t)+RTD(t) (19) Q(t)=Q(0)+A(t)−D(t) (20) P(t)=HQ(t) (21) HT(t)+I(t)=et (22) W(t)=HM(t)[A(t)+Q(0)]−HT(t) (23) Il(t) can only increase when Wl(t)=0, l∈L, (24)

where is defined in (13) and Additionally, we have that

 M(t)D(t)≤T(t)≤M(t)(D(t)+e) (25) Ds(t+Wl(s)(t))=Qs(0)+As(t) (26)

where Equation (25) comes from the fact that each class has at most one packet being served at any time, and Equation (26) comes from the FIFO property of the real queues.

Note that and are right continuous having left limits, while and are continuous in time. We also assume that

Let us define

 X(t) := (A(t),D(t),Q(t),W(t),T(t),I(t),~Q(t)),

where is the shadow queue of class Then is a Markov process. Furthermore, and are themselves Markov processes. By Theorem 1, we know that is positive recurrent.

We now describe the fluid model of the system. The set of fluid model equations is as follows:

 A(t)=βλt+RTD(t) (27) Q(t)=Q(0)+A(t)−D(t) (28) P(t)=HQ(t) (29) HT(t)+I(t)=et (30) W(t)=HM[A(t)+Q(0)]−HT(t) (31) Il(t) can only increase when Wl(t)=0, l∈L (32) T(t)=MD(t) (33) Ds(t+Wl(s)(t))=Qs(0)+As(t), (34)

where and Recall that is defined in Lemma 2 as the set of supporting link rates. Equation (32) means that for each whenever there exists such that i.e., is constant in

### 7.1 Preliminaries

###### Theorem 4 (see [16])

If random variables and satisfy that where the notation denotes the convergence in distribution (weak convergence), and if the are uniformly integrable, i.e.,

 limα→∞supn∫|Zn|>α|Zn|dP=0,

then is integrable and

 limnE[Zn]=E[Z].

Consider the sequence of scaled processes

 Xr(t)=1rX(rt), t≥0, r=1,2,…,

then we have the following theorem:

###### Theorem 5 ([17, 18])

Suppose that, for any sequence of scaled processes satisfying there exist a subsequence and a constant such that

 limrk→∞E∥(Qrk(t),~Qrk(t))∥=0,∀ t≥T.

Then the queueing system is stable, in the sense that the Markov process is positive recurrent.

###### Corollary 1

Suppose that there exists a deterministic function

 ¯X(t)=(¯A(t),¯D(t),¯Q(t),¯W(t),¯T(t),¯I(t),¯~Q(t))

such that the following conditions hold:

[(i)]
1. For any sequence there exists a subsequence such that as

2. For any satisfying there exists a such that

3. is uniformly integrable for all

then the original process is positive recurrent.

{IEEEproof}

From conditions (i) and (ii), we have that for all as Along with condition (iii), Theorem 4 yields that We then apply Theorem 5 to get the result.

### 7.2 Proof’s details

For an integer let be the set of functions that are right continuous on having the left limits on For we use to denote By convention,

Let us endow the function space with the Skorohod