DelayBased BackPressure Scheduling in Multihop Wireless Networks
Abstract
Scheduling is a critical and challenging resource allocation mechanism for multihop wireless networks. It is well known that scheduling schemes that favor links with larger queue length can achieve high throughput performance. However, these queuelengthbased schemes could potentially suffer from large (even infinite) packet delays due to the wellknown last packet problem, whereby packets belonging to some flows may be excessively delayed due to lack of subsequent packet arrivals. Delaybased schemes have the potential to resolve this last packet problem by scheduling the link based on the delay the packet has encountered. However, characterizing throughputoptimality of these delaybased schemes has largely been an open problem in multihop wireless networks (except in limited cases where the traffic is singlehop.) In this paper, we investigate delaybased scheduling schemes for multihop traffic scenarios with fixed routes. We develop a scheduling scheme based on a new delay metric, and show that the proposed scheme achieves optimal throughput performance. Further, we conduct simulations to support our analytical results, and show that the delaybased scheduler successfully removes excessive packet delays, while it achieves the same throughput region as the queuelengthbased scheme.
I Introduction
Link scheduling is a critical resource allocation component in multihop wireless networks, and also perhaps the most challenging. The seminal work of [1] introduces a joint adaptive routing and scheduling algorithm, called Queuelengthbased BackPressure (QBP), that has been shown to be throughputoptimal, i.e., it can stabilize the network under any feasible load. This paper focuses on the settings with fixed routes, where the QBP algorithm becomes a scheduling algorithm. Since the development of QBP, there have been numerous extensions that have integrated it in an overall optimal crosslayer framework. Further, easiertoimplement queuelengthbased scheduling schemes have been developed and shown to be throughputefficient (see [2] and references therein). Some recent attempts [3, 4, 5] focus on designing realworld wireless protocols using the ideas behind these algorithms.
While these queuelengthbased schedulers have been shown to achieve excellent throughput performance, they are usually evaluated under the assumption that flows have an infinite amount of data and keep injecting packets into the network. However, in practice, when accounting for multiple time scales [6, 7, 8], there also exist other types of flows that have a finite number of packets to transmit, which can result in the wellknown last packet problem: consider a queue that holds the last packet of a flow, then the packet does not see any subsequent packet arrivals, and thus the queue length remains very small and the link may be starved for a long time, since the queuelengthbased schemes give a higher priority to links with a larger queue length. In such a scenario with flowlevel dynamics, it has also been shown in [6] that the queuelengthbased schemes may not even be throughputoptimal.
Recent works in [9, 10, 11, 12, 13, 14] have studied the performance of delaybased scheduling algorithms that use HeadofLine (HOL) delays instead of queue lengths as link weights. One desirable property of the delaybased approach is that they provide an intuitive way around the last packet problem. The schedulers give a higher priority to the links with a larger weight as before, but now the weight (i.e., the HOL delay) of a link increases with time until the link is scheduled. Hence, if the link with the last packet is not scheduled at this moment, it is more likely to be scheduled in the next time. However, the throughput of the delaybased scheduling schemes is not fully understood, and has only been established for limited cases with singlehop traffic.
The delaybased approach was introduced in [9] for scheduling in InputQueued switches. The results have been extended to wireless networks for singlehop traffic, providing throughputoptimal delaybased MaxWeight scheduling algorithms [11, 12, 15]. It has also been shown that delaybased schemes with appropriately chosen weight parameters provide good Quality of Service (QoS) [10], and can be used as an important component in a crosslayer protocol design [14]. The performance of the delaybased MaxWeight scheduler has been further investigated in a singlehop network with flowlevel dynamics [13]. The results show that, when flows arrive at the base station carrying a finite amount of data, the delaybased MaxWeight scheduler achieves optimal throughput performance while its queuelengthbased counterpart does not.
It should be noted that even for the multihop wireless networks with fixed routes, the scheduling problem is both important and challenging. There are many existing works focusing on such scenarios with fixed routes (see [16, 17, 18] for examples). However, in multihop wireless networks, the throughput performance of these delaybased schemes has largely been an open problem. To the best of our knowledge, even with the assumption of fixed routes, there are no prior works that employ delaybased algorithms to address the important issue of throughputoptimal scheduling in multihop wireless networks. Indeed, the problem becomes much more challenging in the multihop scenario. In [12], the key idea in showing throughputoptimality of the delaybased MaxWeight scheduler is to exploit the following property: after a finite time, there exists a linear relation between queue lengths and HOL delays in the fluid limits (which we formally define in Section IIIA), where the ratio is the mean arrival rate. Hence, the delaybased MaxWeight scheme is basically equivalent to its queuelengthbased counterpart, and thus achieves the optimal throughput. This property holds for the singlehop traffic. Since given that the exogenous arrival processes follow the Strong Law of Large Numbers (SLLN) and the fluid limits exist, the arrival processes are deterministic with constant rates in the fluid limits. However, such a linear relation does not necessarily hold for the multihop traffic, since at a nonsource (or relay) node, the arrival process may not satisfy SLLN and the packet arrival rate may not even be a constant, depending on the underlying scheduler s dynamics. To this end, we investigate delaybased scheduling schemes that achieve optimal throughput performance in multihop wireless networks.
Unlike previous delaybased schemes, we view the packet delay as a sojourn time in the network, and redesign the delay metric of the queue as the sojourntime difference between the queue’s HOL packet and the HOL packet of its previous hop (see Eq. (36) for the formal definition). Using this new metric, we can establish a linear relation between queue lengths and delays in the fluid limits. The linear relation then plays the key role in showing that the proposed Delaybased BackPressure (DBP) scheduling scheme is throughputoptimal in multihop networks.
In summary, the main contributions of our paper are as follows:

We devise a new delay metric for multihop wireless networks and develop the DBP algorithm, under which a linear relation between queue lengths and delays in the fluid limits can be established. From this linear relation, we can show that DBP achieves optimal throughput performance. To do this, we first revisit throughputoptimality of QBP using fluid limit techniques. Further, we develop a simpler greedy approximation of DBP for practical implementation.

We provide extensive simulation results to evaluate the performance of the delaybased schedulers, including DBP. Through simulations, i) we observe that the last packet problem can cause excessive delays for certain flows under QBP, while the problem is eliminated under DBP. ii) We show that DBP also achieves better fairness and prevents the flows that lack subsequent packet arrivals from starving. iii) Finally, we simulate the simpler greedy approximation algorithms of QBP and DBP, and show that the delaybased approximation empirically achieves a throughput region that is no smaller than that of its queuebased counterpart.
The paper is organized as follows. In Section II, we present a detailed description of our system model. In Section III, we show throughputoptimality of QBP using fluid limit techniques, and extend the analysis to DBP in Section IV. The discussions are further extended to the greedy algorithms in Section V. We evaluate the performance of delaybased schedulers through simulations in Section VI, and conclude our paper in Section VII.
Ii System Model
We consider a multihop wireless network described by a directed graph , where denotes the set of nodes and denotes the set of links. Nodes are wireless transmitters/receivers and links are wireless channels between two nodes if they can directly communicate with each other. During a single time slot, multiple links that do not interfere with each other can be active at the same time, and each active link transmits one packet during the time slot if its queue is not empty. Let denote the set of flows in the network. We assume that each flow has a single, fixed, and loopfree route. The route of flow has an hop length from the source to the destination, where each th hop link is denoted by . Let denote the length of the longest route over all flows. Note that the assumption of single route and unit link capacity is only for ease of exposition, and one can readily extend the results to more general scenarios with multiple fixed routes and heterogeneous link rates, applying the techniques used in this paper. To specify wireless interference, we consider the th hop of each flow or linkflowpair . Let denote the set of all linkflowpairs, i.e.,
The set of linkflowpairs that interfere with can be described as
(1) 
Note that the interference model we adopt is very general, and includes the class of the hop interference model^{1}^{1}1Under the hop interference model, two links within a hop “distance” interfere with each other and cannot be activated at the same time [19]. When , it is also called the primary or nodeexclusive interference model. The 1hop interference model has been known as a good representation for Bluetooth or FHCDMA networks [20, 21, 22, 23]. When , it is often used to model the ubiquitous IEEE 802.11 DCF (Distributed Coordination Function) wireless networks [24, 25, 26, 22].. A schedule is a set of (active or inactive) linkflowpairs, and can be represented by a vector , where denotes the cardinality of a set. Each element is set to 1 if linkflowpair is active, and 0 if linkflowpair is inactive. Slightly abusing the notation, we also use to denote the set of active linkflowpairs of , i.e., . A schedule is said to be feasible if no two linkflowpairs of interfere with each other, i.e., for all , with and . Let denote the set of all feasible schedules in , and let denote its convex hull.
Let denote the number of packet arrivals at the source node of flow at time slot . We assume that packets are of unit length. Similar to [12], we assume that each arrival process is a stationary and ergodic Markov chain with countable state space, and satisfies the Strong Law of Large Numbers (SLLN): That is, with probability one,
(2) 
for each flow , where denotes the mean arrival rate of flow . We let denote the arrival rate vector.
Let denote the number of packets at the queue of at the beginning of time slot . For notational ease, we also use to denote the queue itself. We let denote the queue length vector at time slot , and use to denote the norm of a vector, e.g., . Let denote the service of at time slot , which takes a value of either 1 if linkflowpair is active, or 0 otherwise, in our settings. We let denote the actual number of packets transmitted from at time slot . Clearly, we have for all time slots . Let denote the cumulative queue lengths up to the th hop for flow . By convention, we set , and then we have . The queue length evolves according to the following equations:
(3) 
where we set .
Let be the total number of packets that arrive at the source node of flow until time slot , including those present at time slot 0, and let be the total number of packets that are served at until time slot . By convention, we set for all linkflowpairs . We let denote the sojourn time of the th packet of in the network at time slot , where the time is measured from the time when the packet arrives in the network (i.e., when the packet arrives at the source node), and let denote the sojourn time of the HOL packet of in the network at time slot . We set for all . Further, if , we set . Letting denote the time when the HOL packet of arrives in the network, we have that
(4) 
As in [27], a discretetime queueing system is said to be stable, if the underlying Markov chain is positive Harris recurrent. When the state space is countable and all states communicate (as in the system that we consider in this paper), this is equivalent to the Markov chain being positive recurrent. The throughput region of a scheduling policy is defined as the set of arrival rate vectors for which the network remains stable under this policy. Further, the optimal throughput region (or stability region) is defined as the union of the throughput regions of all possible scheduling policies. We let denote the optimal throughput region, which can be represented as
(5) 
An arrival rate vector is strictly inside , if the inequalities above are all strict.
We summarize the notations in Appendix A for quick reference.
Iii Queuelengthbased BackPressure Algorithm
It has been shown in [1] that QBP stabilizes the network for any feasible arrival rate vector using stochastic Lyapunov techniques. Specifically, we can use a quadratic Lyapunov function to show that the function has a negative drift under QBP when queue lengths are large enough. In this section, we revisit throughputoptimality of QBP using fluid limit techniques. The analysis will be extended later to prove throughputoptimality of the delaybased backpressure algorithm.
To begin with, we define the queue differential as
(6) 
and specify the backpressure algorithm based on queue lengths as follows.
Queuelengthbased BackPressure (QBP) algorithm:
(7) 
The algorithm needs to solve a MaxWeight problem with weights as queue differentials, and ties can be broken arbitrarily if there is more than one schedule that has the largest weight sum.
We establish the fluid limits of the system in the following subsection.
Iiia Fluid Limits
We define the process describing the behavior of the underlying system as , where
We define the norm of as
(8) 
Clearly, under QBP, the evolution of forms a discretetime Markov chain with countable state space. Let denote a process with an initial configuration such that
(9) 
The following Lemma was derived in [28] for continuoustime countable Markov chains, and it follows from more general results in [29] for discretetime countable Markov chains.
Lemma 1 (Theorem 4 of [12])
Suppose there exist an and a finite integer such that for any sequence of processes , we have
(10) 
Then, the Markov process is positive recurrent.
A stability criteria of (10) leads to a fluid limit approach [30, 31] to the stability problem of queueing systems. Hence, we start our analysis by establishing the fluid limit model as in [30, 12]. We define the process , and it is clear that a sample path of uniquely defines the sample path of . Then we extend the definition of and to continuous time domain as for each continuous time .
As in [12], we extend the definition of to the negative interval by assuming that the packets present in the initial state arrived in the past at some of the time instants , according to their delays in the state . By this convention, we have for all and , and for all .
Then, applying the techniques used in the proof for Theorem 4.1 of [30] or Lemma 1 of [12], we can show that with probability one, for any sequence of processes , where is a sequence of positive integers with , there exists a subsequence with as such that the following convergences hold uniformly over compact (u.o.c.) intervals:
(11)  
(12)  
(13)  
(14)  
(15)  
(16)  
(17) 
Similarly, the following convergences (which are denoted by “”) hold at every continuous point of the limit function:
(18)  
(19) 
The above convergence properties follow directly from the ArzelaAscoli Theorem and the structure of the model: that the arrival process satisfies the SLLN and that the sequence of the (scaled) departure process is uniformly bounded and uniformly equicontinuous.
Any set of limiting functions is called a fluid limit. The family of these fluid limits is associated with our original stochastic network. The scaled sequences and their limits are referred to as a fluid limit model [27]. Since some of the limiting functions, namely are Lipschitz continuous in , they are absolutely continuous. Hence, at almost all points , the derivatives of these limiting functions exist. We call such points regular time.
We then present the fluid model equations of the system as follows.
(20)  
(21)  
(22)  
(23)  
(24)  
(25)  
(26)  
(27) 
where , and we set . Fluid model equations can be thought of as belonging to a fluid network which is the deterministic equivalence of the original stochastic network. Any set of functions satisfying the fluid model equations is called a fluid model solution of the system. It is easy to check that any fluid limit is a fluid model solution.
It is clear from (7) that QBP will not schedule linkflowpair if . Hence, if linkflowpair is scheduled, it must satisfy that . Moreover, the length of queue can decrease by at most one within one time slot, and the length of queue can increase by at most one within one time slot, due to the assumption of unit link capacity (a similar argument also holds with nonunit link rates). This implies that, if
(28) 
initially holds for all at time slot 0, then the inequality holds for every time slot . This further implies that
(29) 
for all (scaled) time , from the convergence of (14). We assume that at time slot 0, all queues on the route of each flow are empty except for the first queue, then it follows that (28) holds for all (scaled) time , and thus, holds for all time .
Remark: Note that we make the assumption of empty queues for ease of analysis. Even without this assumption, we can show that there exists a finite time such that for all time , (29) holds for all . This can be proved by induction. The detailed proof can be found in Appendix C, but the basic idea is as follows: Consider a flow . We want to show that there exists a finite time such that for all time , (29) holds for all with .

First, we show that there exists a finite time such that for all time , (29) holds for linkflowpair . Suppose that (29) does not hold for . Then QBP does not schedule , i.e., does not decrease and does not increase. On the other hand, due to the exogenous arrivals at the source node of flow , must increase with time. Hence, there must exist a finite time such that (29) holds for at time . We can further show that (29) holds for all under QBP. This can be proved by contradiction.

Then, we discuss the induction step: Consider . Suppose that for all time , (29) holds for and for all , we show that there exists a finite time such that for all time , (29) holds for and for all . For simplicity, we consider the case for which , and the general induction step follows similarly. Now, suppose that (29) does not hold for , and we prove it by contradiction. Clearly, QBP will schedule only linkflowpairs for which (29) holds (i.e., linkflowpair in this case). Hence, the fluid limit model of the subsystem that consists of linkflowpairs for which (29) holds must be stable, from the throughputoptimality of QBP (see Proposition 2). This, in particular, implies that is stable, which further implies that must increase with time, because QBP keeps forwarding packets from to while not serving . Hence, there must exist a finite time such that for all time , (29) holds for .
Hence, letting , we have that for all time , (29) holds for all with . Since the above arguments can be applied to any flow , we can complete the proof by setting .
IiiB ThroughputOptimality of QBP
Proposition 2
QBP can support any traffic with arrival rate vector that is strictly inside .
Before giving the proof of Proposition 2, in the following lemma, we present a linear relation between cumulative queue length and waiting time , which is used for proving Proposition 2.
Lemma 3
For any fixed , the two conditions and are equivalent for every linkflowpair . Further, if the conditions hold, we have
(30) 
for all , with probability one.
Fig. 1 describes the relations between the variables.
Since the first part, i.e., that the two conditions are equivalent, is straightforward from the definition of fluid limits and (4), we focus on the second part, i.e., if , then (30) follow.
Suppose that . Then, by the definition of , we have , for all . From (22), (23) and (24), we obtain that
[Proof of Proposition 2] We prove stability using standard Lyapunov techniques Let denote the Lyapunov function defined as
(31) 
From the results of Lemmas 1 and 3, to show positive recurrence, we only need to prove that for any , there exists a finite time such that for any fluid limit with , we have
(32) 
for all time . To show the above, it is sufficient to show that for any , there exists such that implies for any regular time , where .
Suppose is strictly inside , then there exists a vector such that , i.e., for all . Since is differentiable, then for any regular time , we can obtain the derivative of as
(33) 
Note that , for any . Hence, we have . Let us choose , then implies . Since and for all , then in the final result of (33), we can conclude that the first term is bounded as follows:
and that the second term becomes nonpositive due to the following. Since QBP chooses schedules that maximize the queue differential weight sum (7), then we have that
which implies that
for all . Therefore, this shows that implies . Then, it immediately follows that for any , there exists a finite time such that for any fluid limit with , we have for any time . Also, we have
(34) 
for all . Let us choose large enough, then it follows from (20), (22) and (34) that
for all and for any time . Hence, we have (30) from Lemma 3, and thus, we have
where (a) and (b) are from (30) and (34), respectively. We can make arbitrarily small by choosing small enough .
Now, consider any fixed sequence of processes (for simplicity also denoted by ). Hence, for any fixed , we can always choose a large enough integer such that for any subsequence of , there exists a further (sub)subsequence such that
almost surely. This in turn implies (for small enough ) that
(35) 
almost surely. This is because there must exist a subsequence of that converges to the same limit as .
One can readily show that the sequence is uniformly integrable using standard techniques by invoking the Dominated Convergence Theorem and so the details are omitted here. Then, the almost sure convergence in (35) along with uniform integrability implies the following convergence in the mean:
Iv Delaybased BackPressure Algorithm
Iva Algorithm Description
In this section, we develop the Delaybased BackPressure (DBP) policy, and in Section IVB, we prove that it is throughput optimal. A similar delaybased approach has appeared first in [12] for singlehop networks. However, as mentioned earlier, when packets travel multiple hops before leaving the system, the analytical approach in [12] (i.e., using HOL delay in the queue as the metric) cannot capture queueing dynamics of multihop traffic and the resultant solutions cannot guarantee the linear relation. We will carefully design link weights using a new delay metric, and reestablish the linear relation between queue lengths and delays in the fluid limits for multihop traffic.
Recall that denotes the sojourn time of the HOL packet of queue in the network, where the time is measured from the time when the packet arrives in the network. We define the delay metric as
(36) 
and also define delay differential as
(37) 
The relations between these delay metrics are illustrated in Fig. 2. We specify the backpressure algorithm with the new delay metric as follows.
Delaybased BackPressure (DBP) algorithm:
(38) 
DBP computes the weight of as the delay differential and solves the MaxWeight problem, i.e., finds a set of noninterfering linkflowpairs that maximizes weight sum. Ties can be broken arbitrarily if there is more than one schedule that has the largest weight sum. An intuitive interpretation of the new delay metric is as follows. Note that the queue length is roughly the number of packets arriving at the source node of flow during the time slots between , and from the SLLN, is on the order of when is large. Hence, a large implies a large queue length , and similarly, a large delay differential implies a large queue length differential . Therefore, being favorable to the delay weight sum in (38) is in some sense “equivalent” to being favorable to the queue length weight sum in (7) as QBP. We later formally establish the linear relation between the fluid limits of queue lengths and delays in Section IVB.
We highlight here that the last packet problem can be solved by the DBP scheme using our proposed delay metric. Let us focus on the source nodes first. Suppose that at the source node of flow , there are a finite number of packets waiting to be transmitted and there are no further packet arrivals. From the definition of (36) and the fact that , we have . If some of the packets are stuck at the source node, the delay metric keeps increasing with time. On the other hand, is equal to the interarrival time between two packets and does not increase with time, in particular because some packets at the source node are not served. Hence, the delay differential also increases with time. This implies that under DBP, the increasing delay will eventually “push” all the packets that are waiting at the source node to the secondhop link. After all the packets leave the source node, we can observe similar procedure at the transmitting node of the secondhop link: since and , we have . Repeating the same argument, we can conclude that all the packets will ultimately be “pushed” to the destination node of flow .
Recall that denotes the time when the HOL packet of arrives in the network (or the source node, rather than the current node). We let denote the time when the packet that arrives (in the network or the source node) immediately after the HOL packet of arrives in the network. Let denote the interarrival time between the HOL packet of and the packet that arrives immediately after it. Clearly, DBP will not schedule linkflowpair if
Hence, if linkflowpair is scheduled, it must satisfy . Moreover, the delay can decrease by at most within one time slot, and the delay can increase by at most within one time slot, due to the assumption of unit link capacity (a similar argument also holds with nonunit link rates). Therefore, if inequality
(39) 
initially holds for all at time slot 0, then the inequality holds for all time slot . This further leads to
(40) 
for all (scaled) time , in the fluid limits, from the convergence of (18) and that , as (otherwise we will arrive a contradiction with the assumption on the arrival process, i.e., it satisfies the Strong Law of Large Numbers). Recall that we assume that all queues on each route are empty at time slot 0, except for the first queue, then (39) and (40) follow.
IvB ThroughputOptimality
The following lemma provides the linear relation between queue lengths and delays in the fluid limits.
Lemma 4
For any fixed , if for every linkflowpair , then we have
(41) 
for all , with probability one.
It follows immediately from Lemma 3.
We emphasize the importance of (41). Lemma 4 implies that after a finite time (i.e., ), the queue lengths are times delays in the fluid limit model. Then the schedules of DBP are very similar to those of QBP, which implies that DBP achieves the optimal throughput region . In the following, we show that the condition of Lemma 4 indeed holds, i.e., such a finite time exists.
Lemma 5
Consider a system under the DBP policy. Then for strictly inside , there exists a finite time such that the fluid limits satisfy the following property with probability one,
(42) 
for all linkflowpairs .
We can prove Lemma 5 by induction following the techniques described in Lemma 7 of [12]. The formal proof is provided in Appendix B. We next outline an informal discussion, which highlights the main idea of the proof. First, we consider the base case. DBP chooses one of the feasible schedules in (we omit the term “feasible” in the following, whenever there is no confusion) at each time slot. Each schedule receives a fraction of the total time and there must exist a schedule that receives at least fraction of the total time. Thus, after a large enough time , there must exist a schedule that is chosen for at least amount of time. The number of initial packets of is bounded from (20), thus, for a large enough , all initial “fluid” of at least one linkflowpair of must be completely served, i.e., , for at least one with .
Next, we consider the inductive step. Suppose there exists a , such that for at least one subset of cardinality , we have
(43) 
for all . Then there exists such that
(44) 
holds for all linkflowpairs within at least one subset of cardinality . Since flows travel hopbyhop, packets that have been served by one link must have been served by the link at the previous hop (of the flow that the packets belong to). Hence, if , we must have . Repeating the argument, if , we have for . Let