ThroughputOptimal Opportunistic Scheduling in the Presence of FlowLevel Dynamics
Abstract
We consider multiuser scheduling in wireless networks with channel variations and flowlevel dynamics. Recently, it has been shown that the MaxWeight algorithm, which is throughputoptimal in networks with a fixed number of users, fails to achieve the maximum throughput in the presence of flowlevel dynamics. In this paper, we propose a new algorithm, called workloadbased scheduling with learning, which is provably throughputoptimal, requires no prior knowledge of channels and user demands, and performs significantly better than previously suggested algorithms.
1 Introduction
Multiuser scheduling is one of the core challenges in wireless communications. Due to channel fading and wireless interference, scheduling algorithms need to dynamically allocate resources based on both the demands of the users and the channel states to maximize network throughput. The celebrated MaxWeight algorithm developed in [3] for exploiting channel variations works as follows. Consider a network with a single base station and users, and further assume that the base station can transmit to only one user in each time slot. The MaxWeight algorithm computes the product of the queue length and current channel rate for each user, and chooses to transmit to that user which has the largest product; ties can be broken arbitrarily. The throughputoptimality property of the MaxWeight algorithm was first established in [3], and the results were later extended to more general channel and arrival models in [4, 5, 6]. The MaxWeight algorithm should be contrasted with other opportunistic scheduling such as [7, 8] which exploit channel variations to allocate resources fairly assuming continuously backlogged users, but which are not throughputoptimal when the users are not continuously backlogged.
While the results in [3, 4, 5] demonstrate the power of MaxWeightbased algorithms, they were obtained under the assumptions that the number of users in the network is fixed and the traffic flow generated by each user is longlived, i.e., each user continually injects new bits into the network. However, practical networks have flowlevel dynamics: users arrive to transmit data and leave the network after the data are fully transmitted. In a recent paper [1], the authors show that the MaxWeight algorithm is in fact not throughput optimal in networks with flowlevel dynamics by providing a clever example showing the instability of the MaxWeight scheduling. The intuition is as follows: if a longlived flow does not receive enough service, its backlog builds up, which forces the MaxWeight scheduler to allocate more service to the flow. This interaction between user backlogs and scheduling guarantees the correctness of the resource allocation. However, if a flow has only a finite number of bits, its backlog does not build up over time and it is possible for the MaxWeight to stop serving such a flow and thus, the flow may stay in the network forever. Thus, in a network where finitesize flows continue to arrive, the number of flows in the network could increase to infinity. One may wonder why flowlevel instability is important since, in real networks, base stations limit the number of simultaneously active flows in the network by rejecting new flows when the number of existing flows reaches a threshold. The reason is that, if a network model without such upper limits is unstable in the sense that the number of flows grows unbounded, then the corresponding real network with an upper limit on the number of flows will experience high flow blocking rates. This fact is demonstrated in our simulations later.
In [1], the authors address this instability issue of MaxWeightbased algorithms, and establish necessary and sufficient conditions for the stability of networks with flowlevel dynamics. The authors also propose throughputoptimal scheduling algorithms. However, as the authors mention in [1], the proposed algorithms require prior knowledge of channel distribution and traffic distribution, which is difficult and sometimes impossible to obtain in practical systems, and further, the performance of the proposed algorithms is also not ideal. A delaydriven MaxWeight scheduler has also been proposed to stabilize the network under flowlevel dynamics [2]. The algorithm however works only when the maximum achievable rates of the flows are identical.
Since flow arrivals and departures are common in reality, we are interested in developing practical scheduling algorithms that are throughputoptimal under flowlevel dynamics. We consider a wireless system with a single base station and multiple users (flows). The network contains both longlived flows, which keep injecting bits into the network, and shortlived flows, which have a finite number of bits to transmit. The main contributions of this paper include the following:

We obtain the necessary conditions for flowlevel stability of networks with both longlived flows and shortlived flows. This generalizes the result in [1], where only shortlived flows are considered.

We propose a simple algorithm for networks with shortlived flows only. Under this algorithm, each flow keeps track of the best channel condition that it has seen so far. Each flow whose current channel condition is equal to the best channel condition that it has seen during its lifetime is eligible for transmission. It is shown that an algorithm which uniformly and randomly chooses a flow from this set of eligible flows for transmission is throughputoptimal. Note that the algorithm is a purely opportunistic algorithm in that it selects users for transmission when they are in the best channel state that they have seen so far, without considering their backlogs.

Based on an optimization framework, we propose to use the estimated workload, the number of time slots required to transmit the remainder of a flow based on the best channel condition seen by the flow so far, to measure the backlog of shortlived flows. By comparing this shortlived flow backlog to the queue lengths and channel conditions of the longlived flows, we develop a new algorithm, named workloadbased scheduling with learning, which is throughputoptimal under flowlevel dynamics. The term ”learning” refers to the fact that the algorithm learns the best channel condition for each shortlived flow and attempts to transmit when the channel condition is the best.

We use simulations to evaluate the performance of the proposed scheduling algorithm, and observe that the workloadbased scheduling with learning performs significantly better than the MaxWeight scheduling in various settings.
The terminology of longlived and shortlived flows above has to be interpreted carefully in practical situations. In practice, each flow has a finite size and thus, all flows eventually will leave the system if they receive sufficient service. Thus, all flows are shortlived flows in reality. Our results suggest that transmitting to users who are individually in their best estimated channel state so far is thus, throughput optimal. On the other hand, it is also well known that real network traffic consists of many flows with only a few packets and a few flows with a huge number of packets. If one considers the time scales required to serve the smallsized flows, the largesized flows will appear to be longlived (i.e., persistent forever) in the terminology above. Thus, if one is interested in performance over short timescales, an algorithm which considers flows with a very large number of packets as being longlived may lead to better performance and hence, we consider the more general model which consists of both shortlived flows and longlived flows. Our simulations later confirm the fact that the algorithm which treats some flows are being longlived leads to better performance although throughputoptimality does not require such a model. In addition, longlived flows partially capture the scenario where all bits from a flow do not arrive at the base station all at once. This fact is also exploited in our simulation experiments.
2 Basic Model
Network Model: We consider a discretetime wireless downlink network with a single base station and many flows, each flow associates with a distinct mobile user. The base station can serve only one flow at a time.
Traffic Model: The network consists of the following two types of flows:

Longlived flows: Longlived flows are traffic streams that are always in the network and continually generate bits to be transmitted.

Shortlived flows: Shortlived flows are flows that have a finite number of bits to transmit. A shortlived flow enters the network at a certain time, and leaves the system after all bits are transmitted.
We assume that the set of longlived flows is fixed, and shortlived flows arrive and depart. We let be the index for longlived flows, be the set of longlived flows, and be the number of longlived flows, i.e., Furthermore, we let be the number of new bits injected by longlived flow in time slot where is a discrete random variable with finite support, and independently and identically distributed (i.i.d.) across time slots. We also assume and for all and
Similarly, we let be the index for shortlived flows, be the set of shortlived flows in the network at time and be the number of shortlived flows at time i.e., We denote by the size (total number of bits) of shortlived flow and assume for all
It is important to note that we allow different shortlived flows to have different maximum link rates. A careful consideration of our proofs will show the reader that the learning algorithm is not necessary if all users have the same maximum rate and that one can simply transmit to the user with the best channel state if it is assumed that all users have the same maximum rate. However, we do not believe that this is a very realistic scenario since SNR variations will dictate different maximum rates for different users.
Residual Size and Queue Length: For a shortlived flow let which we call the residual size, denote the number of bits still remaining in the system at time . For a longlived flow let denote the number of bits stored at the queue at the base station.
Channel Model: There is a wireless link between each user and the base station. Denote by the state of the link between shortlived flow and the base station at time (i.e., the maximum rate at which the base station can transmit to shortlived flow at time ), and the state of the link between longlived flow and the base station at time We assume that and are discrete random variables with finite support. Define and to be the largest values that these random variables can take, i.e., for each Choose and such that
The states of wireless links are assumed to be independent across flows and time slots (but not necessarily identically distributed across flows). The independence assumption across time slots can be relaxed easily but at the cost of more complicated proofs.
3 Workloadbased Scheduling with Learning
In this section, we introduce a new scheduling algorithm called Workloadbased Scheduling with Learning (WSL).
Workloadbased Scheduling with Learning: For a shortlived flow we define
where is the time shortlived flow joins the network and is called the learning period. A key component of this algorithm is to use to evaluate the workload of shortlived flows (the reason will be explained in a detail in Section 5). However, is in general unknown, so the scheduling algorithm uses as an estimate of
During each time slot, the base station first checks the following inequality:
(1) 
where

If inequality (1) holds, then the base station serves a shortlived flow as follows: if at least one shortlived flow (say flow ) satisfies or then the base station selects such a flow for transmission (ties are broken according to a good tiebreaking rule, which is defined at the end of this algorithm); otherwise, the base station picks an arbitrary shortlived flow to serve.

If inequality (1) does not hold, then the base station serves a longlived flow such that
(ties are broken arbitrarily).
“Good” tiebreaking rule: Assume that the tiebreaking rule is applied to pick a shortlived flow every time slot (but the flow is served only if ). We define to be the event that the tiebreaking rule selects a shortlived flow with Define
which is he total workload of the system at time A tiebreaking rule is said to be good if the following condition holds: Consider the WSL with the given tiebreaking rule and learning period Given any there exist and such that
if and
Remark 1: While all WSL scheduling algorithms with good tiebreaking rules are throughput optimal, their performances in terms of other metrics could be different depending upon the tiebreaking rules. We consider two tiebreaking rules in this paper:

Uniform Tiebreaking: Among all shortlived flows satisfying or the basestation uniformly and randomly selects one to serve.

Oldestfirst Tiebreaking: Let denote the number of time slots a shortlived flow has been in the network. The base station keeps track for every shortlived flow, where is some fixed positive integer. Among all shortlived flows satisfying or the tiebreaking rule selects the one with the largest and the ties are broken uniformly and randomly.
^{3}
The “goodness” of these two tiebreaking rules are proved in Appendix C and D, and the impact of the tiebreaking rules on performance is studied in Section 6 using simulations.
Remark 2: The in inequality (1) is a parameter balancing the performance of longlived flows and shortlived flows. A large will lead to a small number of shortlived flows but large queuelengths of longlived flows, and vice versa.
Remark 3: In Theorem 3, we will prove that WSL is throughput optimal when is sufficiently large. From purely throughputoptimality considerations, it is then natural to choose However, in practical systems, if we choose too large, such as then it is possible that a flow may stay in the system for a very long time if its best channel condition occurs extremely rarely. Thus, it is perhaps best to choose a finite to tradeoff between performance and throughput.
Remark 4: If all flows are shortlived, then the algorithm simplifies as follows: If at least one shortlived flow (say flow ) satisfies or then the base station selects such a flow for transmission according to a “good” tiebreaking rule; otherwise, the base station picks an arbitrary shortlived flow to serve. Simply stated, the algorithm serves one of the flows which can be completely transmitted or sees its best channel state, where the best channel state is an estimate based on past observations. If no such flow exists, any flow can be served. We do not separately prove the throughput optimality of this scenario since it is a special case of the scenario considered here. But it is useful to note that, in the case of shortlived flows only, the algorithm does not consider backlogs at all in making scheduling decisions.
We will prove that WSL (with any ) is throughputoptimal in the following sections, i.e., the scheduling policy can support any set of traffic flows that are supportable by any other algorithm. In the next section, we first present the necessary conditions for the stability, which also define the network throughput region.
4 Necessary Conditions for Stability
In this section, we establish the necessary conditions for the stability of networks with flowlevel dynamics. To get the necessary condition, we need to classify the shortlived flows into different classes.

A shortlived flow class is defined by a pair of random variables . Class is associated with random variables and
^{4} A shortlived flow belongs to class if has the same distribution as and the size of flow () has the same distribution as We let denote the number of class flows joining the network at time where are i.i.d. across time slots and independent but not necessarily identical across classes, and Denote by the set of distinct classes. We assume that is finite, and for all and 
Let denote an dimensional vector describing the state of the channels of the longlived flows. In state is the service rate that longlived flow can receive if it is scheduled. We denote by the set of all possible states.

Let denote the state of the longlived flows at time and denote the probability that is in state

Let be the probability that the base station serves flow when the network is in state Clearly, for any we have
Note that the sum could be less than if the base station schedules a shortlived flow in this state.

Let be the probability that the base station serves a shortlived flow when the network is in state

Let denote the number of shortlived flows that belong to class and have residual size Note that can only take on a finite number of values.
Theorem 1
Consider traffic parameters and and suppose that there exists a scheduling policy guaranteeing
Then there exist and such that the following inequalities hold:
(2)  
(3)  
(4) 
Inequality (2) and (3) state that the service allocated should be no less than the user requests if the flows are supportable. Inequality (4) states that the overall time used to serve longlived and shortlived flows should be no more than the time available. To prove this theorem, it can be shown that for any traffic for which we cannot find and satisfying the three inequalities in the theorem, a Lyapunov function can be constructed such that the expected drift of the Lyapunov function is larger than some positive constant under any scheduling algorithm, which implies the instability of the network. The complete proof is based on the Strict Separation Theorem and is along the lines of a similar proof in [5], and is omitted in this paper.
5 Throughput Optimality of WSL
First, we provide some intuition into how one can derive the WSL algorithm from optimization decomposition considerations. Then, we will present our main throughput optimality results. Given traffic parameters and the necessary conditions for the supportability of the traffic is equivalent to the feasibility of the following constraints:
(5)  
For convenience, we view the feasibility problem as an optimization problem with the objective where is some constant. While we have not explicitly stated that the ’s and ’s are nonnegative, this is assumed throughout.
Partially augmenting the objective using Lagrange multipliers, we get
For the moment, let us assume Lagrange multipliers and are given. Then the maximization problem above can be decomposed into a collection of optimization problems, one for each
It is easy to verify that one optimal solution to the optimization problem above is:

if then and

otherwise, and for some and for other
The complementary slackness conditions give
Since is the mean arrival rate of longlived flow and is the mean service rate, the condition on says that if the mean arrival rate is less than the mean service rate, is equal to zero. Along with the nonnegativity condition on this suggests that perhaps behaves likes a queue with these arrival and service rates. Indeed, it turns out that the mean of the queue lengths are proportional to Lagrange multipliers (see the surveys in [9, 10, 11]). For longlived flow we can treat the queuelength as a timevarying estimate of Lagrange multiplier Similarly can be associated with a queue whose arrival rate is which is the mean rate at which workload arrives where workload is measured by the number of slots needed to serve a shortlived flow if it is served when its channel condition is the best. The service rate is which is the rate at which the workload can potentially decrease when a shortlived flow is picked for scheduling by the base station. Thus, the workload in the system can serve as a dynamic estimate of
Letting () be an estimate of the observations above suggest the following workloadbased scheduling algorithm if are known.
Workloadbased Scheduling (WS): During each time slot, the base station checks the following inequality:
(6) 

If inequality (6) holds, then the base station serves a shortlived flow as follows: if at least one shortlived flow (say flow ) satisfies or then such a flow is selected for transmission (ties are broken arbitrarily); otherwise, the base station picks an arbitrary shortlived flow to serve.

If inequality (6) does not hold, then the base station serves a longlived flow such that (ties are broken arbitrarily).

The factor can be obtained from the optimization formulation by multiplying constraint (5) by on both sides
However, this algorithm which was directly derived from dual decomposition considerations is not implementable since ’s are unknown. So WSL uses to approximate Note that an inaccurate estimate of not only affects the base station’s decision on whether but also on its computation of However, it is not difficult to see that the error in the estimate of the total workload is a small fraction of the total workload when the total workload is large: when the workload is very large, the total number of shortlived flows is large since their file sizes are bounded. Since the arrival rate of shortlived flows is also bounded, this further implies that the majority of shortlived flows must have arrived a long time ago which means that with high probability, their estimate of their best channel condition must be correct.
Next we will prove that both WS and WSL can stabilize any traffic and such that and are supportable, i.e., satisfying the conditions presented in Theorem 1. In other words, the number of shortlived flows in the network and the queues for longlived flows are all bounded. Even though WS is not practical, we study it first since the proof of its throughput optimality is easier and provides insight into the proof of throughputoptimality of WSL.
Let
Since the base station makes decisions on and under WS. It is easy to verify that is a finitedimensional Markov chain under WS. Assume that , and are such that the Markov chain is irreducible and aperiodic.
Theorem 2
Given any traffic and such that and are supportable, the Markov chain is positiverecurrent under WS, and
We consider the following Lyapunov function:
(7) 
and prove that
for some , , and a finite set Positive recurrence of then follows from Foster’s Criterion for Markov chains [12], and the boundedness of the first moment follows from [13]. The detailed proof is presented in Appendix A.
We next study WSL, where is estimated from the history. We define to be the number of shortlived flows that belong to class have a residual size of and have Furthermore, we define
from some It is easy to see that is a finitedimensional Markov chain under WSL.
Theorem 3
Consider traffic and such that and are supportable. Given WSL with a good tiebreaking rule, there exists such that the Markov chain is positiverecurrent under the WSL with learning period and the given tiebreaking rule. Further,
The proof of this theorem is built upon the following two facts:

When the number of shortlived flows is large, the majority of shortlived flows must have been in the network for a long time and have obtained the correct estimate of the best channel condition, which implies that

When the number of shortlived flows is large, the shortlived flow selected by the base station (say flow ) has a high probability to satisfy or
From these two facts, we can prove that with a high probability, the scheduling decisions of WSL are the same as those of WS, which leads to the throughput optimality of WSL. The detailed proof is presented in Appendix B.
6 Simulations
In this section, we use simulations to evaluate the performance of different variants of WSL and compare it to other scheduling policies. There are three types of flows used in the simulations:

Sflow: An Sflow has a finite size, generated from a truncated exponential distribution with mean value and maximum value Noninteger values are rounded to integers.

Mflow: An Mflow keeps injecting bits into the network for time slots and stops. The number of bits generated at each time slot follows a Poisson distribution with mean value

Lflow: An Lflow keeps injecting bits into the network and never leaves the network. The number of bits generated at each time slot follows a truncated Poisson distribution with mean value and maximum value .
Here Sflows represent shortlived flows that have finite sizes and whose bits arrive all at once; Lflows represent longlived flows that continuously inject bits and never leave the network; and Mflows represent flows of finite size but whose arrival rate is controlled at their sources so that they do not arrive instantaneously into the network. Our simulation will demonstrate the importance of modeling very large, but finitesized flows as longlived flows.
We assume that the channel between each user and the base station is distributed according to one of the following three distributions:

Glink: A Glink has five possible link rates and each of the states happens with probability

Plink: A Plink has five possible link rates and each of the states happens with probability

Rlink: An Rlink has five possible link rates and the probabilities associated with these link states are
The G, P and R stand for Good, Poor and Rare, respectively. We include these three different distributions to model the SNR variations among the users, where Glinks represent links with high SNR (e.g., those users close to the base station), Plinks represent links with low SNR (e.g., those users far away from the base station), and Rlinks represent links whose best state happens rarely. The Rlinks will be used to study the impact of learning period on the network performance.
We name the WSL with the uniform tiebreaking rule WSLU, and the WSL with the oldestfirst tiebreaking rule WSLO. In the following simulations, we will first demonstrate that the WSLU performs significantly better than previously suggested algorithms, and then show that the performance can be further improved by choosing a good tiebreaking policy (e.g., WSLO). We set to be in all the following simulations.
Simulation I: Shortlived Flow or Longlived Flow?
We first use the simulation to demonstrate the importance of considering a flow with a large number of packets as being longlived. We consider a network consisting of multiple Sflows and three Mflows, where the arrival of Sflows follows a truncated Poisson process with maximum value and mean value All the links are assumed to be Glinks. We evaluate the following two schemes:

Scheme1: Both Sflows and Mflows are considered to be shortlived flows.

Scheme2: An Mflow is considered to be longlived before its last packet arrives, and to be shortlived after that.
The performance of these two schemes are shown in Figure 1, where WS with Uniform Tiebreaking Rule is used as the scheduling algorithm. We can see that the performances are substantially different (note that the network is stable under both schemes). The number of queued bits of Mflows under Scheme1 is larger than that under Scheme2 by two orders of magnitude. This is because even an Mflow contains a huge number of bits ( on average), it can be served only when the link rate is under Scheme1. This simulation suggests that when the performance we are interested is at a small scale (e.g. acceptable queuelength being less than or equal to ) compared with the size of the flow (e.g., in this simulation), the flow should be viewed as a longlived flow for performance purpose.
Simulation II: The Impact of Learning Period
In this simulation, we investigate the impact of on the performance of WSLU. Recall that it is nature to choose for purely throughputoptimality considerations, but the disadvantage is that a flow may stay in the network for a very long time if the best link state occurs very rarely. We consider a network consisting of Sflows, which arrive according to a truncated Poisson process with maximum value and mean and three Lflows. All links are assumed to be Rlinks. Figure 2 depicts the mean and standard deviation of the filetransfer delays with and when the traffic load is light or medium. As we expected, the standard deviation under WSLU with is significantly larger than that under WSLU with when is large. This occurs because the best link rate occurs with a probability This simulation confirms that in practical systems, we may want to choose a finite to get desired performance.
Further we would like to comment that while the WSLU algorithm with a small has a better performance in light or medium traffic regimes, throughput optimality is only guaranteed when is sufficiently large. Figure 3 illustrates the average number of Sflows and average filetransfer delay for and in heavy traffic regime. We can observe that in the heavy traffic regime, the WSLU with still stabilizes the network but the algorithm with does not. So there is a clear tradeoff in choosing : A small reduces the filetransfer delay in light or medium traffic regimes, but a large guarantees stability in heavy traffic regime.
Simulation III: Performance comparison of various algorithms
In the following simulations, we choose In the introduction, we have pointed out that the MaxWeight is not throughput optimal under flowlevel dynamics because the backlog of a shortlived queue does not build up even when it has not been served for a while. To overcome this, one could try to use the delay of the headofline packet, instead of queuelength, as the weight because the headofline delay will keep increasing if no service is received. In the case of longlived flows only, this algorithm is known to be throughputoptimal [5]. We will show that this Delaybased scheduling does not solve the instability problem when there are shortlived flows.
Delaybased Scheduling: At each time slot, the base station selects a flow such that where is the delay experienced so far by the headofline packet of flow
We first consider the case where all flows are Sflows, which arrive according to a truncated Poisson process with maximum value and mean An Sflow is assigned with a Glink or a Plink equally likely.
Figure 4 shows the average filetransfer delay and average number of Sflows under different values of We can see that WSLU performs significantly better than the MaxWeight and Delaybased algorithms. Specifically, under MaxWeight and Delaybased algorithms, both the number of Sflows and filetransfer delay explode when WSLU, on the other hand, performs well even when
Next, we consider the same scenario with three Lflows in the network. Two of the Lflows have Glinks and one has a Plink. Figure 5 shows the average number of shortlived flows and average filetransfer delay under different values of We can see that the MaxWeight becomes unstable even when the arrival rate of Sflows is very small. This is because the MaxWeight stops serving Sflows when the backlogs of Lflows are large, so Sflows stay in the network forever. The delaybased scheduling performs better than the MaxWeight, but significantly worse than WSLU.
Simulation IV: Blocking probability of various algorithms
While our theory assumes that the number of flows in the network can be infinite, in reality, base stations limit the number of simultaneously active flows, and reject new flows when the number of existing flows above some threshold. In this simulation, we assume that the base station can support at most Sflows. A new Sflow will be blocked if Sflows are already in the network. In this setting, the number of flows in the network is finite, so we compute the blocking probability, i.e., the fraction of Sflows rejected by the base station.
We consider the case where no longlived flow is in the network and the case where both shortlived and longlived flows are present in the network. The flows and channels are selected as in Simulation III. The results are shown in Figure 7 and 7. We can see that the blocking probability under WSLU is substantially smaller than that under the MaxWeight or the delaybased scheduling. Thus, this simulation demonstrates that instability under the assumption when the number of flows is allowed to unbounded implies high blocking probabilities for the practical scenario when the base station limits the number of flows in the network.
Simulation V: WSLU versus WSLO
In this simulation, we study the impact of tiebreaking rules on performance. We compare the performance of the WSLU and WSLO. We first study the case where the base station does not limit the number of simultaneously active flows and there is no longlived flow in the network. The simulation setting is the same as that in Simulation III. Figure 8 shows the average filetransfer delay and average number of Sflows under different values of We can see that the WSLO reduces the filetransfer delay and number of Sflows by nearly when which indicates the importance of selecting a good tiebreaking rule for improving the network performance.
Next, we study the case where the base station does not limit the number of simultaneously active flows and there are three Lflows in the network. Figure 9 shows the average number of shortlived flows and average filetransfer delay under different values of We can see again that the WSLO algorithm has a much better performance than the WSLU, especially when is large.
Finally we consider the situation where the base station can support at most Sflows. A new Sflow will be blocked if Sflows are already in the network. The simulation setting is the same as that in Simulation IV. We calculate the blocking probabilities, and the results are shown in Figure 11 and 11. We can see that the blocking probability under the WSLO is much smaller than that under the WSLU policy when is large.
7 Conclusions and Discussions
In this paper, we studied multiuser scheduling in networks with flowlevel dynamics. We first obtained necessary conditions for flowlevel stability of networks with both longlived flows and shortlived flows. Then based on an optimization framework, we proposed the workloadbased scheduling with learning that is throughputoptimal under flowlevel dynamics and requires no prior knowledge about channels and traffic. In the simulations, we evaluated the performance of the proposed scheduling algorithms, and demonstrated that the proposed algorithm performs significantly better than the MaxWeight algorithm and the Delaybased algorithm in various settings. Next we discuss the limitations of our model and possible extensions.
7.1 The choice of
According to Theorem 3, the learning period should be sufficiently large to guarantee throughputoptimality. Our simulation results on the other hand suggested that a small may result in better performance. Therefore, there is clear tradeoff in choosing The study of the choice for is one potential future work.
7.2 Unbounded file arrivals and file sizes
One limitation of our model is that the random variables associated with the number of file arrivals and file sizes are assumed to be upper bounded. One interesting future research problem is to extend the results to unbounded number of file arrivals and file sizes.
Appendix A: Proof of Theorem 2
Recall that We define to be the largest achievable link rate of class shortlived flows, and which is the amount of new workload (from shortlived flows) injected in the network at time and to be the decrease of the workload at time i.e., if the workload of shortlived flows is reduced by one and otherwise. Based on the notations above, the evolution of shortlived flows can be described as:
Further, the evolution of can be described as
where is the decrease of due to the service longlived flow receives at time and is the unused service due to the lack of data in the queue.
We consider the following Lyapunov function
(8) 
We will prove that the drift of the Lyapunov function satisfies
for some and a finite set (the values of these parameters will be defined in the following analysis). Positive recurrence of then follows from Foster’s Criterion for Markov chains [12].
First, since the number of arrivals, the sizes of shortlived flows and channel rates are all bounded, it can be verified that there exists independent of such that
Recall that we assume that and satisfy the supportability conditions of Theorem 1. By adding and subtracting corresponding and we obtain that
where
Next we assume and analyze the following quantity
(9) 
We have the following facts:

Fact 1: Assume that there exists a shortlived flow such that or If a shortlived flow is selected to be served, then the workload of the selected flow is reduced by one and If longlived flow is selected, the rate flow receives is Thus, we have that
where the last inequality holds because Therefore, we have in this case.

Fact 2: Assume that there does not exist a shortlived flow such that or In this case, we have
Now we define a set such that
where is a positive integer satisfying that
(10)  
(11) 
and is a positive integer satisfying
(12) 
We next compute the drift of the Lyapunov function according to the value of

Case I: Assume According to the definition of we have

Case II: Assume Since the size of a shortlived flow is upper bounded by implies that at least shortlived flows are in the network at time Define to be the following event: no shortlived flow satisfies or .
Recall that
Given at least shortlived flows are in the network, we have that

Case III: Assume that and for some In this case, if a longlived flow is selected for a given we have
Otherwise, if a shortlived flow is selected, it means for the given we have and
Therefore, we can conclude that in this case,
(15) (16) where the last inequality yields from the definition of (12).
From the analysis above, we can conclude that
where and is a set with a finite number of elements. Since for all the Lyapunov function is always lower bounded. Further the drift of the Lyapunov is upper bounded when belongs to a finite set and is negative otherwise. So invoking Foster’s criterion, the Markov chain is positive recurrent and the boundedness of the first moment follows from [13].
Appendix B: Proof of Theorem 3
Consider the network that is operated under WSL, and define to be
Now given we define the following notations:

Define if flow is selected by WSL, and otherwise.

Define if flow is selected by WSL and the workload of flow can be reduced by one, and otherwise.

Define if flow is selected by WS, and otherwise.

Define if flow is selected by WS and the workload of flow can be reduced by one, and otherwise.
We remark that is the action selected by the base station at time under WSL and is the action selected by the base station at time under WS, assuming the same history
We define the Lyapunov function to be
(17) 
This Lyapunov function is similar to the one used in the proof of Theorem 2, and we will show that this is a valid Lyapunov function for the workloadbased scheduling with learning. Then, it is easy to verify that there exists independent of such that
Dividing the time into two segments and we obtain