Delay-Based Back-Pressure Scheduling in Multihop Wireless Networks

# Delay-Based Back-Pressure Scheduling in Multihop Wireless Networks

Bo Ji,  Changhee Joo,  and Ness B. Shroff,  B. Ji is with Department of ECE at the Ohio State University. C. Joo is with School of ECE at UNIST, Korea. N. B. Shroff is with Departments of ECE and CSE at the Ohio State University.Emails: ji@ece.osu.edu, cjoo@unist.ac.kr, shroff@ece.osu.edu.This work was supported in part by ARO MURI Award W911NF-08-1-0238, and NSF Awards 1012700-CNS, 0721236-CNS, 0721434-CNS, and 1065136-CNS, and in part by the Basic Science Research Program through the National Research Foundation of Korea (NRF), funded by the Ministry of Education, Science, and Technology (No. 2012-0003227).A preliminary version of this work was presented at IEEE INFOCOM 2011.
###### Abstract

Scheduling is a critical and challenging resource allocation mechanism for multihop wireless networks. It is well known that scheduling schemes that favor links with larger queue length can achieve high throughput performance. However, these queue-length-based schemes could potentially suffer from large (even infinite) packet delays due to the well-known last packet problem, whereby packets belonging to some flows may be excessively delayed due to lack of subsequent packet arrivals. Delay-based schemes have the potential to resolve this last packet problem by scheduling the link based on the delay the packet has encountered. However, characterizing throughput-optimality of these delay-based schemes has largely been an open problem in multihop wireless networks (except in limited cases where the traffic is single-hop.) In this paper, we investigate delay-based scheduling schemes for multihop traffic scenarios with fixed routes. We develop a scheduling scheme based on a new delay metric, and show that the proposed scheme achieves optimal throughput performance. Further, we conduct simulations to support our analytical results, and show that the delay-based scheduler successfully removes excessive packet delays, while it achieves the same throughput region as the queue-length-based scheme.

## I Introduction

Link scheduling is a critical resource allocation component in multihop wireless networks, and also perhaps the most challenging. The seminal work of [1] introduces a joint adaptive routing and scheduling algorithm, called Queue-length-based Back-Pressure (Q-BP), that has been shown to be throughput-optimal, i.e., it can stabilize the network under any feasible load. This paper focuses on the settings with fixed routes, where the Q-BP algorithm becomes a scheduling algorithm. Since the development of Q-BP, there have been numerous extensions that have integrated it in an overall optimal cross-layer framework. Further, easier-to-implement queue-length-based scheduling schemes have been developed and shown to be throughput-efficient (see [2] and references therein). Some recent attempts [3, 4, 5] focus on designing real-world wireless protocols using the ideas behind these algorithms.

While these queue-length-based schedulers have been shown to achieve excellent throughput performance, they are usually evaluated under the assumption that flows have an infinite amount of data and keep injecting packets into the network. However, in practice, when accounting for multiple time scales [6, 7, 8], there also exist other types of flows that have a finite number of packets to transmit, which can result in the well-known last packet problem: consider a queue that holds the last packet of a flow, then the packet does not see any subsequent packet arrivals, and thus the queue length remains very small and the link may be starved for a long time, since the queue-length-based schemes give a higher priority to links with a larger queue length. In such a scenario with flow-level dynamics, it has also been shown in [6] that the queue-length-based schemes may not even be throughput-optimal.

Recent works in [9, 10, 11, 12, 13, 14] have studied the performance of delay-based scheduling algorithms that use Head-of-Line (HOL) delays instead of queue lengths as link weights. One desirable property of the delay-based approach is that they provide an intuitive way around the last packet problem. The schedulers give a higher priority to the links with a larger weight as before, but now the weight (i.e., the HOL delay) of a link increases with time until the link is scheduled. Hence, if the link with the last packet is not scheduled at this moment, it is more likely to be scheduled in the next time. However, the throughput of the delay-based scheduling schemes is not fully understood, and has only been established for limited cases with single-hop traffic.

The delay-based approach was introduced in [9] for scheduling in Input-Queued switches. The results have been extended to wireless networks for single-hop traffic, providing throughput-optimal delay-based MaxWeight scheduling algorithms [11, 12, 15]. It has also been shown that delay-based schemes with appropriately chosen weight parameters provide good Quality of Service (QoS) [10], and can be used as an important component in a cross-layer protocol design [14]. The performance of the delay-based MaxWeight scheduler has been further investigated in a single-hop network with flow-level dynamics [13]. The results show that, when flows arrive at the base station carrying a finite amount of data, the delay-based MaxWeight scheduler achieves optimal throughput performance while its queue-length-based counterpart does not.

It should be noted that even for the multihop wireless networks with fixed routes, the scheduling problem is both important and challenging. There are many existing works focusing on such scenarios with fixed routes (see [16, 17, 18] for examples). However, in multihop wireless networks, the throughput performance of these delay-based schemes has largely been an open problem. To the best of our knowledge, even with the assumption of fixed routes, there are no prior works that employ delay-based algorithms to address the important issue of throughput-optimal scheduling in multihop wireless networks. Indeed, the problem becomes much more challenging in the multihop scenario. In [12], the key idea in showing throughput-optimality of the delay-based MaxWeight scheduler is to exploit the following property: after a finite time, there exists a linear relation between queue lengths and HOL delays in the fluid limits (which we formally define in Section III-A), where the ratio is the mean arrival rate. Hence, the delay-based MaxWeight scheme is basically equivalent to its queue-length-based counterpart, and thus achieves the optimal throughput. This property holds for the single-hop traffic. Since given that the exogenous arrival processes follow the Strong Law of Large Numbers (SLLN) and the fluid limits exist, the arrival processes are deterministic with constant rates in the fluid limits. However, such a linear relation does not necessarily hold for the multihop traffic, since at a non-source (or relay) node, the arrival process may not satisfy SLLN and the packet arrival rate may not even be a constant, depending on the underlying scheduler s dynamics. To this end, we investigate delay-based scheduling schemes that achieve optimal throughput performance in multihop wireless networks.

Unlike previous delay-based schemes, we view the packet delay as a sojourn time in the network, and re-design the delay metric of the queue as the sojourn-time difference between the queue’s HOL packet and the HOL packet of its previous hop (see Eq. (36) for the formal definition). Using this new metric, we can establish a linear relation between queue lengths and delays in the fluid limits. The linear relation then plays the key role in showing that the proposed Delay-based Back-Pressure (D-BP) scheduling scheme is throughput-optimal in multihop networks.

In summary, the main contributions of our paper are as follows:

• We devise a new delay metric for multihop wireless networks and develop the D-BP algorithm, under which a linear relation between queue lengths and delays in the fluid limits can be established. From this linear relation, we can show that D-BP achieves optimal throughput performance. To do this, we first re-visit throughput-optimality of Q-BP using fluid limit techniques. Further, we develop a simpler greedy approximation of D-BP for practical implementation.

• We provide extensive simulation results to evaluate the performance of the delay-based schedulers, including D-BP. Through simulations, i) we observe that the last packet problem can cause excessive delays for certain flows under Q-BP, while the problem is eliminated under D-BP. ii) We show that D-BP also achieves better fairness and prevents the flows that lack subsequent packet arrivals from starving. iii) Finally, we simulate the simpler greedy approximation algorithms of Q-BP and D-BP, and show that the delay-based approximation empirically achieves a throughput region that is no smaller than that of its queue-based counterpart.

The paper is organized as follows. In Section II, we present a detailed description of our system model. In Section III, we show throughput-optimality of Q-BP using fluid limit techniques, and extend the analysis to D-BP in Section IV. The discussions are further extended to the greedy algorithms in Section V. We evaluate the performance of delay-based schedulers through simulations in Section VI, and conclude our paper in Section VII.

## Ii System Model

 P≜{(s,k) | s∈S, 1≤k≤H(s)}.

The set of link-flow-pairs that interfere with can be described as

 I(s,k)≜{(r,j)∈P |(r,j) interferes with (s,k),or (r,j)=(s,k)}. (1)

Note that the interference model we adopt is very general, and includes the class of the -hop interference model111Under the -hop interference model, two links within a -hop “distance” interfere with each other and cannot be activated at the same time [19]. When , it is also called the primary or node-exclusive interference model. The 1-hop interference model has been known as a good representation for Bluetooth or FH-CDMA networks [20, 21, 22, 23]. When , it is often used to model the ubiquitous IEEE 802.11 DCF (Distributed Coordination Function) wireless networks [24, 25, 26, 22].. A schedule is a set of (active or inactive) link-flow-pairs, and can be represented by a vector , where denotes the cardinality of a set. Each element is set to 1 if link-flow-pair is active, and 0 if link-flow-pair is inactive. Slightly abusing the notation, we also use to denote the set of active link-flow-pairs of , i.e., . A schedule is said to be feasible if no two link-flow-pairs of interfere with each other, i.e., for all , with and . Let denote the set of all feasible schedules in , and let denote its convex hull.

Let denote the number of packet arrivals at the source node of flow at time slot . We assume that packets are of unit length. Similar to [12], we assume that each arrival process is a stationary and ergodic Markov chain with countable state space, and satisfies the Strong Law of Large Numbers (SLLN): That is, with probability one,

 limt→∞∑t−1τ=0As(τ)t=λs, (2)

for each flow , where denotes the mean arrival rate of flow . We let denote the arrival rate vector.

Let denote the number of packets at the queue of at the beginning of time slot . For notational ease, we also use to denote the queue itself. We let denote the queue length vector at time slot , and use to denote the -norm of a vector, e.g., . Let denote the service of at time slot , which takes a value of either 1 if link-flow-pair is active, or 0 otherwise, in our settings. We let denote the actual number of packets transmitted from at time slot . Clearly, we have for all time slots . Let denote the cumulative queue lengths up to the -th hop for flow . By convention, we set , and then we have . The queue length evolves according to the following equations:

 Qs,k(t+1)=Qs,k(t)+Ψs,k−1(t)−Ψs,k(t), (3)

where we set .

Let be the total number of packets that arrive at the source node of flow until time slot , including those present at time slot 0, and let be the total number of packets that are served at until time slot . By convention, we set for all link-flow-pairs . We let denote the sojourn time of the -th packet of in the network at time slot , where the time is measured from the time when the packet arrives in the network (i.e., when the packet arrives at the source node), and let denote the sojourn time of the HOL packet of in the network at time slot . We set for all . Further, if , we set . Letting denote the time when the HOL packet of arrives in the network, we have that

 Us,k(t)=inf{τ≤t | Fs(τ)>^Fs,k(t)}, % for all t≥0. (4)

As in [27], a discrete-time queueing system is said to be stable, if the underlying Markov chain is positive Harris recurrent. When the state space is countable and all states communicate (as in the system that we consider in this paper), this is equivalent to the Markov chain being positive recurrent. The throughput region of a scheduling policy is defined as the set of arrival rate vectors for which the network remains stable under this policy. Further, the optimal throughput region (or stability region) is defined as the union of the throughput regions of all possible scheduling policies. We let denote the optimal throughput region, which can be represented as

 Λ∗≜{→λ | ∃→ϕ∈Co(MP) s.t. λs≤ϕs,k,∀(s,k)∈P}. (5)

An arrival rate vector is strictly inside , if the inequalities above are all strict.

We summarize the notations in Appendix A for quick reference.

## Iii Queue-length-based Back-Pressure Algorithm

It has been shown in [1] that Q-BP stabilizes the network for any feasible arrival rate vector using stochastic Lyapunov techniques. Specifically, we can use a quadratic Lyapunov function to show that the function has a negative drift under Q-BP when queue lengths are large enough. In this section, we re-visit throughput-optimality of Q-BP using fluid limit techniques. The analysis will be extended later to prove throughput-optimality of the delay-based back-pressure algorithm.

To begin with, we define the queue differential as

 ΔQs,k(t)≜Qs,k(t)−Qs,k+1(t), (6)

and specify the back-pressure algorithm based on queue lengths as follows.

Queue-length-based Back-Pressure (Q-BP) algorithm:

 →M∗∈\operatornamewithlimitsargmax→M∈MP∑(s,k)∈PΔQs,k(t)⋅Ms,k. (7)

The algorithm needs to solve a MaxWeight problem with weights as queue differentials, and ties can be broken arbitrarily if there is more than one schedule that has the largest weight sum.

We establish the fluid limits of the system in the following subsection.

### Iii-a Fluid Limits

We define the process describing the behavior of the underlying system as , where

 X(t)≜((Zs,k,1(t),⋯,Zs,k,Qs,k(t)(t)),(s,k)∈P).

We define the norm of as

 ∥X(t)∥≜∥→Q(t)∥+∥→W(t)∥. (8)

Clearly, under Q-BP, the evolution of forms a discrete-time Markov chain with countable state space. Let denote a process with an initial configuration such that

 ∥X(x)(0)∥=x. (9)

The following Lemma was derived in [28] for continuous-time countable Markov chains, and it follows from more general results in [29] for discrete-time countable Markov chains.

###### Lemma 1 (Theorem 4 of [12])

Suppose there exist an and a finite integer such that for any sequence of processes , we have

 limsupx→∞E[1x∥X(x)(xT)∥]≤1−ϵ. (10)

Then, the Markov process is positive recurrent.

A stability criteria of (10) leads to a fluid limit approach [30, 31] to the stability problem of queueing systems. Hence, we start our analysis by establishing the fluid limit model as in [30, 12]. We define the process , and it is clear that a sample path of uniquely defines the sample path of . Then we extend the definition of and to continuous time domain as for each continuous time .

As in [12], we extend the definition of to the negative interval by assuming that the packets present in the initial state arrived in the past at some of the time instants , according to their delays in the state . By this convention, we have for all and , and for all .

Then, applying the techniques used in the proof for Theorem 4.1 of [30] or Lemma 1 of [12], we can show that with probability one, for any sequence of processes , where is a sequence of positive integers with , there exists a subsequence with as such that the following convergences hold uniformly over compact (u.o.c.) intervals:

 1xnj∫xnjt0A(xnj)s(τ)dτ→λst, (11) 1xnjF(xnj)s(xnjt)→fs(t), (12) 1xnj^F(xnj)s,k(xnjt)→^fs,k(t), (13) 1xnjQ(xnj)s,k(xnjt)→qs,k(t), (14) 1xnjP(xnj)s,k(xnjt)→ps,k(t), (15) 1xnj∫xnjt0Π(xnj)s,k(τ)dτ→∫t0πs,k(τ)dτ, (16) 1xnj∫xnjt0Ψ(xnj)s,k(τ)dτ→∫t0ψs,k(τ)dτ. (17)

Similarly, the following convergences (which are denoted by “”) hold at every continuous point of the limit function:

 1xnjW(xnj)s,k(xnjt)⇒ws,k(t), (18) 1xnjU(xnj)s,k(xnjt)⇒us,k(t). (19)

The above convergence properties follow directly from the Arzela-Ascoli Theorem and the structure of the model: that the arrival process satisfies the SLLN and that the sequence of the (scaled) departure process is uniformly bounded and uniformly equicontinuous.

Any set of limiting functions is called a fluid limit. The family of these fluid limits is associated with our original stochastic network. The scaled sequences and their limits are referred to as a fluid limit model [27]. Since some of the limiting functions, namely are Lipschitz continuous in , they are absolutely continuous. Hence, at almost all points , the derivatives of these limiting functions exist. We call such points regular time.

We then present the fluid model equations of the system as follows.

 ∑s∈Sfs(0)≤1, (20) ps,k(t)=∑ki=1qs,i(t), (21) ps,k(t)=fs(t)−^fs,k(t), (22) fs(t)=fs(0)+λst, (23) us,k(t)=t−ws,k(t), (24) ψs,k(t)≤πs,k(t), (25) Δqs,k(t)=qs,k(t)−qs,k+1(t), (26) ddtqs,k(t)={ψs,k−1(t)−πs,k(t),if qs,k(t)>0,(ψs,k−1(t)−πs,k(t))+,otherwise, (27)

where , and we set . Fluid model equations can be thought of as belonging to a fluid network which is the deterministic equivalence of the original stochastic network. Any set of functions satisfying the fluid model equations is called a fluid model solution of the system. It is easy to check that any fluid limit is a fluid model solution.

It is clear from (7) that Q-BP will not schedule link-flow-pair if . Hence, if link-flow-pair is scheduled, it must satisfy that . Moreover, the length of queue can decrease by at most one within one time slot, and the length of queue can increase by at most one within one time slot, due to the assumption of unit link capacity (a similar argument also holds with non-unit link rates). This implies that, if

 Qs,k(t)≥Qs,k+1(t)−2 (28)

initially holds for all at time slot 0, then the inequality holds for every time slot . This further implies that

 qs,k(t)≥qs,k+1(t), i.e., Δqs,k(t)≥0, (29)

for all (scaled) time , from the convergence of (14). We assume that at time slot 0, all queues on the route of each flow are empty except for the first queue, then it follows that (28) holds for all (scaled) time , and thus, holds for all time .

Remark: Note that we make the assumption of empty queues for ease of analysis. Even without this assumption, we can show that there exists a finite time such that for all time , (29) holds for all . This can be proved by induction. The detailed proof can be found in Appendix C, but the basic idea is as follows: Consider a flow . We want to show that there exists a finite time such that for all time , (29) holds for all with .

1. First, we show that there exists a finite time such that for all time , (29) holds for link-flow-pair . Suppose that (29) does not hold for . Then Q-BP does not schedule , i.e., does not decrease and does not increase. On the other hand, due to the exogenous arrivals at the source node of flow , must increase with time. Hence, there must exist a finite time such that (29) holds for at time . We can further show that (29) holds for all under Q-BP. This can be proved by contradiction.

2. Then, we discuss the induction step: Consider . Suppose that for all time , (29) holds for and for all , we show that there exists a finite time such that for all time , (29) holds for and for all . For simplicity, we consider the case for which , and the general induction step follows similarly. Now, suppose that (29) does not hold for , and we prove it by contradiction. Clearly, Q-BP will schedule only link-flow-pairs for which (29) holds (i.e., link-flow-pair in this case). Hence, the fluid limit model of the subsystem that consists of link-flow-pairs for which (29) holds must be stable, from the throughput-optimality of Q-BP (see Proposition 2). This, in particular, implies that is stable, which further implies that must increase with time, because Q-BP keeps forwarding packets from to while not serving . Hence, there must exist a finite time such that for all time , (29) holds for .

Hence, letting , we have that for all time , (29) holds for all with . Since the above arguments can be applied to any flow , we can complete the proof by setting .

### Iii-B Throughput-Optimality of Q-BP

###### Proposition 2

Q-BP can support any traffic with arrival rate vector that is strictly inside .

Before giving the proof of Proposition 2, in the following lemma, we present a linear relation between cumulative queue length and waiting time , which is used for proving Proposition 2.

###### Lemma 3

For any fixed , the two conditions and are equivalent for every link-flow-pair . Further, if the conditions hold, we have

 ps,k(t)=λsws,k(t), (30)

for all , with probability one.

Fig. 1 describes the relations between the variables.

{proof}

Since the first part, i.e., that the two conditions are equivalent, is straightforward from the definition of fluid limits and (4), we focus on the second part, i.e., if , then (30) follow.

Suppose that . Then, by the definition of , we have , for all . From (22), (23) and (24), we obtain that

 ps,k(t)=fs(t)−^fs,k(t)=(fs(0)+λst)−(fs(0)+λsus,k(t))=λs⋅(t−us,k(t))=λsws,k(t).
{proof}

[Proof of Proposition 2] We prove stability using standard Lyapunov techniques Let denote the Lyapunov function defined as

 (31)

From the results of Lemmas 1 and 3, to show positive recurrence, we only need to prove that for any , there exists a finite time such that for any fluid limit with , we have

 ∥→q(t)∥≤ζ, (32)

for all time . To show the above, it is sufficient to show that for any , there exists such that implies for any regular time , where .

Suppose is strictly inside , then there exists a vector such that , i.e., for all . Since is differentiable, then for any regular time , we can obtain the derivative of as

 D+dt+V(→q(t))(a)=∑(s,k)∈Pqs,k(t)⋅(ψs,k−1(t)−πs,k(t))(b)≤∑(s,k)∈Pqs,k(t)⋅(πs,k−1(t)−πs,k(t))=∑(s,k)∈PΔqs,k(t)⋅λs−∑(s,k)∈PΔqs,k(t)⋅πs,k(t)=∑(s,k)∈PΔqs,k(t)⋅(λs−ϕs,k)+∑(s,k)∈PΔqs,k(t)⋅(ϕs,k−πs,k(t)), (33)

where (a) and (b) are from (27) and (25), respectively.

Note that , for any . Hence, we have . Let us choose , then implies . Since and for all , then in the final result of (33), we can conclude that the first term is bounded as follows:

 ∑(s,k)∈PΔqs,k(t)⋅(λs−ϕs,k)≤−ζ3mins,k(ϕs,k−λs)≜−ζ2<0,

and that the second term becomes non-positive due to the following. Since Q-BP chooses schedules that maximize the queue differential weight sum (7), then we have that

 →π(t)∈\operatornamewithlimitsargmax→ϕ∈Co(MP)∑(s,k)∈PΔqs,k(t)⋅ϕs,k,

which implies that

 ∑(s,k)∈PΔqs,k(t)⋅ϕs,k≤∑(s,k)∈PΔqs,k(t)⋅πs,k(t),

for all . Therefore, this shows that implies . Then, it immediately follows that for any , there exists a finite time such that for any fluid limit with , we have for any time . Also, we have

 ps,k(t)≤∥→q(t)∥≤ζ, (34)

for all . Let us choose large enough, then it follows from (20), (22) and (34) that

 ^fs,k(T1)=fs(T1)−ps,k(T1)>fs(0),

for all and for any time . Hence, we have (30) from Lemma 3, and thus, we have

 ∥→q(t)∥+∥→w∥(a)≤∥→q(t)∥+1minsλs∥→p(t)∥(b)≤(1+|S|Hmaxminsλs)ζ≜ϵ1,

where (a) and (b) are from (30) and (34), respectively. We can make arbitrarily small by choosing small enough .

Now, consider any fixed sequence of processes (for simplicity also denoted by ). Hence, for any fixed , we can always choose a large enough integer such that for any subsequence of , there exists a further (sub)subsequence such that

 limj→∞1xnj∥X(xnj)(xnjT)∥=∥→q(T)∥+∥→w(T)∥≤ϵ1

almost surely. This in turn implies (for small enough ) that

 limsupx→∞1x∥X(x)(xT)∥≤ϵ1≜1−ϵ<1 (35)

almost surely. This is because there must exist a subsequence of that converges to the same limit as .

One can readily show that the sequence is uniformly integrable using standard techniques by invoking the Dominated Convergence Theorem and so the details are omitted here. Then, the almost sure convergence in (35) along with uniform integrability implies the following convergence in the mean:

 limsupx→∞E[1x∥X(x)(xT)∥]≤1−ϵ.

Since the above convergence holds for any sequence of processes , the condition of (10) in Lemma 1 is satisfied. This completes the proof.

## Iv Delay-based Back-Pressure Algorithm

### Iv-a Algorithm Description

In this section, we develop the Delay-based Back-Pressure (D-BP) policy, and in Section IV-B, we prove that it is throughput optimal. A similar delay-based approach has appeared first in [12] for single-hop networks. However, as mentioned earlier, when packets travel multiple hops before leaving the system, the analytical approach in [12] (i.e., using HOL delay in the queue as the metric) cannot capture queueing dynamics of multihop traffic and the resultant solutions cannot guarantee the linear relation. We will carefully design link weights using a new delay metric, and re-establish the linear relation between queue lengths and delays in the fluid limits for multihop traffic.

Recall that denotes the sojourn time of the HOL packet of queue in the network, where the time is measured from the time when the packet arrives in the network. We define the delay metric as

 ^Ws,k(t)≜Ws,k(t)−Ws,k−1(t), (36)

and also define delay differential as

 Δ^Ws,k(t)≜^Ws,k(t)−^Ws,k+1(t). (37)

The relations between these delay metrics are illustrated in Fig. 2. We specify the back-pressure algorithm with the new delay metric as follows.

Delay-based Back-Pressure (D-BP) algorithm:

 →M∗∈\operatornamewithlimitsargmax→M∈MP∑(s,k)∈PΔ^Ws,k(t)⋅Ms,k. (38)

D-BP computes the weight of as the delay differential and solves the MaxWeight problem, i.e., finds a set of non-interfering link-flow-pairs that maximizes weight sum. Ties can be broken arbitrarily if there is more than one schedule that has the largest weight sum. An intuitive interpretation of the new delay metric is as follows. Note that the queue length is roughly the number of packets arriving at the source node of flow during the time slots between , and from the SLLN, is on the order of when is large. Hence, a large implies a large queue length , and similarly, a large delay differential implies a large queue length differential . Therefore, being favorable to the delay weight sum in (38) is in some sense “equivalent” to being favorable to the queue length weight sum in (7) as Q-BP. We later formally establish the linear relation between the fluid limits of queue lengths and delays in Section IV-B.

We highlight here that the last packet problem can be solved by the D-BP scheme using our proposed delay metric. Let us focus on the source nodes first. Suppose that at the source node of flow , there are a finite number of packets waiting to be transmitted and there are no further packet arrivals. From the definition of (36) and the fact that , we have . If some of the packets are stuck at the source node, the delay metric keeps increasing with time. On the other hand, is equal to the inter-arrival time between two packets and does not increase with time, in particular because some packets at the source node are not served. Hence, the delay differential also increases with time. This implies that under DBP, the increasing delay will eventually “push” all the packets that are waiting at the source node to the second-hop link. After all the packets leave the source node, we can observe similar procedure at the transmitting node of the second-hop link: since and , we have . Repeating the same argument, we can conclude that all the packets will ultimately be “pushed” to the destination node of flow .

Recall that denotes the time when the HOL packet of arrives in the network (or the source node, rather than the current node). We let denote the time when the packet that arrives (in the network or the source node) immediately after the HOL packet of arrives in the network. Let denote the inter-arrival time between the HOL packet of and the packet that arrives immediately after it. Clearly, D-BP will not schedule link-flow-pair if

 ^Ws,k(t)−^Ws,k+1(t)<0.

Hence, if link-flow-pair is scheduled, it must satisfy . Moreover, the delay can decrease by at most within one time slot, and the delay can increase by at most within one time slot, due to the assumption of unit link capacity (a similar argument also holds with non-unit link rates). Therefore, if inequality

 ^Ws,k(t)≥^Ws,k+1(t)−2Bs,k(t) (39)

initially holds for all at time slot 0, then the inequality holds for all time slot . This further leads to

 ^ws,k(t)≥^ws,k+1(t), i.e., Δ^ws,k(t)≥0, (40)

for all (scaled) time , in the fluid limits, from the convergence of (18) and that , as (otherwise we will arrive a contradiction with the assumption on the arrival process, i.e., it satisfies the Strong Law of Large Numbers). Recall that we assume that all queues on each route are empty at time slot 0, except for the first queue, then (39) and (40) follow.

### Iv-B Throughput-Optimality

The following lemma provides the linear relation between queue lengths and delays in the fluid limits.

###### Lemma 4

For any fixed , if for every link-flow-pair , then we have

 qs,k(t)=λs^ws,k(t), (41)

for all , with probability one.

{proof}

It follows immediately from Lemma 3.

We emphasize the importance of (41). Lemma 4 implies that after a finite time (i.e., ), the queue lengths are times delays in the fluid limit model. Then the schedules of D-BP are very similar to those of Q-BP, which implies that D-BP achieves the optimal throughput region . In the following, we show that the condition of Lemma 4 indeed holds, i.e., such a finite time exists.

###### Lemma 5

Consider a system under the D-BP policy. Then for strictly inside , there exists a finite time such that the fluid limits satisfy the following property with probability one,

 ^fs,k(T)>fs(0), (42)

for all link-flow-pairs .

We can prove Lemma 5 by induction following the techniques described in Lemma 7 of [12]. The formal proof is provided in Appendix B. We next outline an informal discussion, which highlights the main idea of the proof. First, we consider the base case. D-BP chooses one of the feasible schedules in (we omit the term “feasible” in the following, whenever there is no confusion) at each time slot. Each schedule receives a fraction of the total time and there must exist a schedule that receives at least fraction of the total time. Thus, after a large enough time , there must exist a schedule that is chosen for at least amount of time. The number of initial packets of is bounded from (20), thus, for a large enough , all initial “fluid” of at least one link-flow-pair of must be completely served, i.e., , for at least one with .

Next, we consider the inductive step. Suppose there exists a , such that for at least one subset of cardinality , we have

 ^fs,k(Tl)>fs(0), (43)

for all . Then there exists such that

 ^fs,k(Tl+1)>fs(0), (44)

holds for all link-flow-pairs within at least one subset of cardinality . Since flows travel hop-by-hop, packets that have been served by one link must have been served by the link at the previous hop (of the flow that the packets belong to). Hence, if , we must have . Repeating the argument, if , we have for . Let

 S∗l≜{(r,j) |(r,j)∉S