Network Utility Maximization with Heterogeneous Traffic Flows
We consider the Network Utility Maximization (NUM) problem for wireless networks in the presence of arbitrary types of flows, including unicast, broadcast, multicast, and anycast traffic. Building upon the recent framework of a universal control policy (UMW), we design a utility optimal cross-layer admission control, routing and scheduling policy, called UMW+. The UMW+ policy takes packet level actions based on a precedence-relaxed virtual network. Using Lyapunov optimization techniques, we show that UMW+ maximizes network utility, while simultaneously keeping the physical queues in the network stable. Extensive simulation results validate the performance of UMW+; demonstrating both optimal utility performance and bounded average queue occupancy. Moreover, we establish a precise one-to-one correspondence between the dynamics of the virtual queues under the UMW+ policy, and the dynamics of the dual variables of an associated offline NUM program, under a subgradient algorithm. This correspondence sheds further insight into our understanding of UMW+.
Increasingly, networks need to support a heterogeneous mix of traffic that includes unicast, multicast and broadcast traffic flows. Such traffic may be comprised of video streaming, file downloads, distributed computation and storage, as well as a host of other applications. The increase in both volume, as well as QoS requirements, of these emerging applications can significantly strain network resources, especially in bandwidth limited wireless networks. This calls for efficient resource allocation schemes for wireless networks with heterogeneous type of traffic demands. Unfortunately, however, existing resource allocation schemes, both in theory and practice, are limited to dealing with point-to-point unicast traffic. While there has been some work on resource allocation for multicast and broadcast flows, until very recently there has been no mechanism for allocating resources in networks with arbitrary traffic flows that include concurrent unicast, multicast and broadcast traffic. In , the authors developed the first such algorithm for joint routing and scheduling in wireless networks with general traffic flows. The algorithm of , called Universal Max Weight is guaranteed to stabilize the network for any arrivals that are within the network capacity (stability) region. However, it provides no service guarantees for traffic that is outside the network capacity region.
When traffic arrivals are outside of the network’s capacity region, an admission control mechanism is needed to limit the admitted traffic to being within the network’s capacity region. In particular, a resource allocation scheme is needed that can optimize user-level performance across the network. Network Utility Maximization (NUM) is an approach for allocating resources across the network in a manner that maximizes overall “utility,” where user’s level of satisfaction is measured using a concave function of the allocated data rate to that user (i.e., the utility function).
Starting with the work of Kelly nearly two decades ago, , there has been a tremendous amount of work on the NUM problem (see, for example, , ,  and references therein). These approaches use convex optimization and Lagrangian duality to optimize overall network utility in networks with static traffic demand. In the context of stochastic traffic, the works of [6, 7, 8] used Lyapunov optimization techniques to develop utility optimal flow control and resource allocation for wireless networks with unicast traffic demands. Unfortunately, however, all of these works deal with unicast (point-to-point) traffic demands.
In this paper, we develop a jointly optimal admission control, routing, and scheduling algorithm for networks with generalized traffic flows. Our approach builds upon the Universal Max Weight (UMW) algorithm of  that solves the optimal routing and scheduling problem for networks with general traffic flows, thus we call our algorithm UMW+. To the best of our knowledge, UMW+ is the first efficient solution to the NUM problem for networks with generalized traffic flows, including, but not limited to concurrent unicast, broadcast, multicast, and anycast traffic. We derive UMW+ by using the concept of precedence-relaxed virtual queueing network . Moreover, by formulating the NUM problem as a static convex optimization problem, we show that, under the dual subgradient algorithm, the evolution of the dual variables precisely corresponds to the evolution of the virtual queues in UMW+. This sheds a new light on the relationship between Max-Weight and sub-gradient algorithms . We make the following key contributions in this paper:
We design the first efficient policy for the NUM problem for arbitrary types of traffic, including, but not limited to, concurrent unicast, broadcast, multicast, and anycast sessions.
We formulate a static version of the NUM problem by decomposing the flows route-wise. This is to be contrasted with earlier works on this problem (e.g., , ), which use node-based flow conservation, applicable only to the unicast flows. We show that, under the dual subgradient algorithm with unit step size, the evolution of the dual variables exactly corresponds to the evolution of the virtual queues under the proposed UMW+ policy.
We explicitly characterize the dual objective function of the above optimization problem in terms of cost of the shortest route for each source and the Fenchel conjugates of the utility function.
Finally, we validate our theoretical results through extensive numerical simulations.
The rest of the paper is organized as follows: In Section II, we describe the network and traffic model. In Section III, we detail the virtual network framework and derive the UMW+ control policy. In Section IV, we prove the stability of the physical queues under the UMW+ policy. In Section V, we establish the equivalence between the UMW+ policy and the dual subgradient algorithm. In Section VI, we provide simulation results and conclude the paper in Section VII.
Ii System Model and Problem Formulation
Ii-A System Model
Let the topology of a wireless network be given by the graph - where is the set of nodes and is the set of links. Time is slotted, and at every slot, each wireless link could be in either the ON (1) or, OFF (0) state. The random link state process is assumed to be evolving according to a stationary ergodic stochastic process. If at a slot , link is ON and is activated, it can transmit packets in that slot 111A link does not transmit any packet if it is OFF..
Due to the wireless inter-channel interference, not all links can be activated at a slot simultaneously. The set of all interference-free links, which may be activated together in a slot, is given by . As an example, in the case of primary interference constraints, the set consists of the set of all Matchings in the graph. On the other hand, in an interference-free wired network, the set consists of the set of all subsets of links.
Traffic and Utility Model
External packets from different traffic classes are admitted to the network by an admission controller . The set of all possible classes of traffic is denoted by . A traffic class is associated with the following two attributes - (1) type of the traffic (e.g., unicast, broadcast, multicast or anycast), and (2) a monotone increasing strictly concave Utility function . Let denote the set of all possible routes for routing packets from class in the graph . As an example, for a unicast class with source node and destination node , the set consists of the set of all paths in the graph. Similarly, for a broadcast class, the set of all routes is given by the set of all spanning trees in the graph. We consider an infinitely backlogged traffic model, where the admission controller has potentially an unlimited number of packets from each class available for admission.
Ii-B Admissible Policies
An admissible policy consists of the following three modules: (1) an admission controller , (2) a routing module , and (3) a link scheduler module . The admission controller determines the number of external packets to be admitted to the network from each class in each slot. We assume that, due to physical constraints (power, capacity limitations etc.), at most number of packets may be feasibly admitted from any class per slot 222Taking yields the same optimal utility obtained without this constraint.. The routing module routes the admitted packets according to the type of flow they belong to, and the link scheduler activates a subset of interference-free links from the set in every slot. The set of all admissible policies is denoted by .
Ii-C The Network Utility Maximization (NUM) Problem
The Network Utility Maximization problem seeks to find an admissible policy , which maximizes the sum utility of all classes, while keeping the queues in the network stable 333Throughout the paper, by stability, we will mean almost sure rate stability, defined in Eqn. (3). . Formally, let the random variable denote the number of packets received in common by the destination(s) of class up to time under the action of an admissible policy . Also, denote the random queue length of packets waiting to cross the edge under the policy by . Then the NUM problem seeks to find a policy , which solves the following problem:
where the expectation and the almost sure limits are taken over the randomness of the network configurations and possible randomness in the policy. In the following, we will drop the superscript from the random variables when the driving policy is clear from the context.
Iii The Virtual Network Framework
In this section, we describe a virtual network framework, obtained by relaxing the natural precedence constraints associated with a multi-hop network. Our goal in this section is to design a utility-optimal stabilizing control policy for the simpler virtual queueing system. Section IV shows that when the same policy is used in the actual physical network, the physical queues are also stable. The virtual network methodology in a similar context was first introduced in .
Consider an admissible policy , which admits packets from class , and activates the links at slot . Taking the random wireless link states into account, the service rate for the virtual queue is packets per slot. Since the virtual network is precedence-relaxed, all admitted packets immediately enter all virtual queues on the selected route, i.e., unlike the physical network, arrivals to a virtual queue need not wait to cross the intermediate links . Thus, the virtual queues evolve as
where is the total number of (controlled) arrival of packets to the virtual queue at slot . Clearly, depends on the routes selected by the routing module , i.e.,
In order to design a utility-optimal stabilizing control policy for the virtual queues, we use the drift-plus-penalty framework of . Consider the following Lyapunov function, which is quadratic in the virtual queue lengths (as opposed to the usual physical queue lengths )
The one-slot conditional drift of under the action of a control policy is given as follows
where the inequality (a) is obtained by using the virtual queue dynamics in Eqn. (4), the equality (b) is obtained by interchanging the order of summation in the first term, is a finite constant, upper bounded by , where is the maximum number of external admissions per slot per class (defined in Section II-B) and .
Moreover, admission of packets from class in slot yields a “one-slot utility” of for class . Following the “Drift-Plus-Penalty” framework of , we consider a cross-layer admission control, routing and link scheduling policy, which is obtained by minimizing the objective function given at the bottom of this page, over all admissible controls per slot. For the objective , we have substituted the upper bound of the drift from Eqn. (7) (without the constant ), and is taken to be a fixed positive constant. This yields the following joint routing, admission control, and link scheduling policy , which we call Universal Max-Weight Plus (UMW+):
Denote the cost of the weighted shortest route corresponding to class obtained above by , i.e.,
In the case of differentiable utility functions , the number of packets admitted may be obtained in closed form as follows:
where denotes projection onto the interval .
In the special case of primary interference constraints, where the set consists of the set of all Matchings of the graph, the problem (11) corresponds to the Maximum-Weighted Matching problem, which can be solved efficiently even in a distributed fashion .
Our main result in this section is Theorem 1, which claims that under the UMW+ control policy described above, the virtual queues are stable and the average expected utility obtained is arbitrarily close to the optimal utility.
See the Appendix (Section VIII). ∎
An alternative treatment of Utility Optimality and Stability of the Virtual Queues under the UMW+ policy will be given in Section V, where we relate the virtual queue evolution to the subgradient descents of an appropriately defined dual optimization problem. This sheds new and fundamental insights in the construction and operation of the virtual queues.
Iv Control of the Physical Network
In the physical network, the same admission control, routing and link scheduling policy are used as in the virtual network of Section III. Hence, the same number of packets are admitted at each slot from each class to the virtual and the physical network. As a result, the utility achieved in these two networks are the same, however, their queueing evolutions are different. In this Section, we establish the stability of the physical queues, which, unlike the virtual queues, are subjected to the usual precedence constraint of a multi-hop network.
Iv-A Packet arrivals to the Physical Queues
To connect the arrivals of packets in the virtual and the physical network, observe that since the routes are fixed at the sources, the total number of physical packets admitted to the physical network in any time interval , that will cross edge in future, is the same as the number of virtual packets arrivals to the corresponding virtual queue . Let the total service allocated to serve the virtual queue (resp. physical queue) in the time interval be denoted by (resp. ). Using the Skorokhod map  for the virtual queue iterations (4), we have
However, as noted above, we have
Iv-B Stability of The Physical Queues
Note that, Theorem 1 establishes the stability of the virtual queues under the action of the UMW+ policy. To prove the optimality of this policy in the NUM setting of Eqn. (1), we need to establish the stability of the physical queues . This is a non-trivial task because, unlike the virtual queues, the dynamics of the physical queues is subject to the precedence constraints, and regulated by the packet scheduling policy employed in the physical network. A packet scheduling policy is a rule which resolves the contention when multiple packets want to cross the same edge at the same slot. The most common examples of packet scheduling policies include First In First Out (FIFO), Last In First Out (LIFO) disciplines etc. Following our earlier work , we consider a simple packet scheduling policy called the Extended Nearest to Origin (ENTO) policy:
Definition 1 (Extended Nearest to Origin policy).
The ENTO policy prioritizes packets according to the decreasing order of their current distances (measured in hop-lengths) from their respective sources.
For example, if there are two packets and , which have traversed, say, and hops respectively from their sources, and wish to cross the same active edge (with a unit capacity per slot) in the same slot, the ENTO policy prioritizes the packet over to cross the edge .
Using ideas from adversarial queueing theory , it is shown in  (Theorem 3) that, under the ENTO packet scheduling policy, the almost sure packet arrival bound (14) also implies the rate stability of the physical queues. Combining this result with Theorem 1, we conclude the following:
Theorem 2 (Stability of the Physical Queues).
Under the action of the UMW+ control policy, the physical queues are rate stable, i.e.,
The proof of Theorem 2 follows directly from the proof of Theorem 3 of  and uses the bound (14) as a starting point. Combining Theorem 1 with Theorem 2, we conclude that the proposed UMW+ policy is a utility optimal stabilizing control policy, which solves the NUM problem efficiently. This concludes the first half of the paper.
V A Dual Perspective on the Virtual Queue Dynamics
In this Section, we consider the dual of an offline version of the NUM problem in a static wired network setting (no interference constraints). We give an alternative derivation of the UMW+ policy from an optimization theory perspective, which sheds further insight into the structure of the optimal policy. Our motivation in this section is similar to , , which give similar development for the Backpressure policy . The dual problem is also of sufficient theoretical and practical interest, as the powerful machinery of convex optimization may be used to derive alternative efficient algorithms for solving the dual problem, which may then be translated to other dynamic policies (apart from the UMW+) for solving the NUM problem.
As in the previous section, the topological structure of a route depends on the type of flow - e.g., a route is a path for unicast and anycast flows, a spanning tree for broadcast flows, a Steiner tree for multicast flows etc.
Fix a strictly positive constant . Our goal is to solve the following utility maximization problem :
The objective (15) denotes the total utility, scaled by . The constraint (16) is obtained by decomposing the total incoming flow to a class into all available routes, where the variable corresponds to the amount of flow carried by the route . The constraint (17) corresponds to the capacity of edge , and the constraint (18) corresponds to the non-negativity property of the flow variables.
In the special case of the NUM problem dealing exclusively with the unicast flows, the flow decomposition constraint (16) is usually replaced with the flow conservation constraints at the nodes, which leads to the Backpressure policy . In contrast, we formulate the NUM problem with the flow decomposition constraint, since flow, in general, is not conserved at the nodes due to packet replications (as in broadcast and multicast flows).
V-A The Dual Problem
The problem is a concave maximization problem with linear constraints. To obtain its dual problem, we relax the capacity constraints (17) by associating a non-negative dual variable with the constraint corresponding to the edge . We choose not to relax the flow decomposition constraints (16) and the non-negativity constraints (18)444Relaxation of these constraints yields a different dual problem.. This yields the following partial Lagrangian:
leading to the following dual objective function :
By strong duality , the problem is equivalent to the following dual problem:
We next establish a simple lemma which will be useful for our subsequent development.
For any fixed , an optimal solution to the problem (20) is obtained by routing the entire flow from each class along a weighted shortest route , weighted by the corresponding dual variables .
Proof: Exchanging the order of summation in the last term of the Lagrangian in Eqn. (19), we have
Define to be the cost of the route where each edge is weighted by the dual variable , . Hence, from the last term of the above expression (23) and the flow decomposition constraint (21), it immediately follows that the objective (20) is maximized by routing the entire incoming flow along a weighted shortest route corresponding to class . In other words, for any class ,
where (ties are broken arbitrarily).
Define to be the cost of the shortest route corresponding to class , i.e., Since, is defined to be the point wise minimum of several linear functions, it is a concave function of . For several important traffic classes (e.g., unicast, broadcast, anycast) there are standard combinatorial algorithms for efficiently computing (e.g. Weighted Shortest Path, Minimum Weight Spanning Tree etc.).
With an optimal setting of the flow variables resolved by Lemma (1), Eqn. (23) implies that the computation of the dual objective function (20) reduces to optimizing the traffic admission rates for each class as follows:
Due to strict concavity of the utility functions , the optimal solution to the problem (20) is obtained by setting the derivative of the objective with respect to the variable to zero, which yields
where we project on the set of non-negative real numbers due to non-negativity constraints of the rates. Substituting Eqn. (25) into Eqn. (20), we obtain an implicit expression of the dual objective function . In the following, we derive an explicit expression of the dual function in terms of the Fenchel conjugate of the utility functions .
V-B Derivation of the Dual Objective Function
Substituting the value of into the Lagrangian (19) and noting that only along the corresponding shortest route and is zero otherwise, we have
The above expression may also be written in terms of the Fenchel’s conjugate  of the utility functions. For this, we recall the definition of the Fenchel conjugate of (an extended real-valued) function :
Now, let the function denote the Fenchel conjugate of the function , which, by our assumption, is a strictly convex function. Thus, from Eqn. (26) we can write
In the following, we use Eqn. (28) to derive explicit functional forms of the dual objective functions for two important classes of network utility functions.
1. Logarithmic Utility Functions: Consider the class of Logarithmic utility functions defined as follows:
where is a positive constant. Among its many attractive properties, the Logarithmic utility functions ensures proportionally fair rate allocations among all participating classes.
The Fenchel’s conjugate of may be computed as
Hence, the dual function is given as
2. -fair Utility Functions: Next, we consider the -fair utility functions defined as follows:
where and are positive parameters.
The Fenchel’s conjugate of may be computed as
Hence, the dual function is given as
V-C Subgradient Method and its Equivalence with UMW+
Since the dual objective , as given in Eqn. (20), is a point wise maximum of linear functions, it is convex . Moreover, the objective , as seen from Eqn. (28), is non-differentiable, as the shortest path cost is not a differentiable function of , in general. Hence, we use a first order method suitable for non-smooth objectives, known as the Subgradient Descent , to solve the dual problem (22).
A subgradient of the dual function may be computed directly from the capacity constraint of the primal problem (Eqn. (17)) as follows:
Finally, we solve the dual problem by the dual subgradient method with constant step-size equal to . This yields the following iteration for the dual variables :
For the step-size , the above iteration corresponds exactly to the virtual queue dynamics. More precisely, Eqn. (8) corresponds to Lemma 1, which is concerned with routing along the shortest route in the weighted graph; Eqn. (10) corresponds to Eqn. (25), which corresponds to the packet admission control, and the virtual queue dynamics in Eqn. (4) corresponds to the dual subgradient update of Eqn. (32). Moreover, for any constant step-size , at any step , the virtual queues under UMW+ are exactly equal to the -scaled versions of the corresponding dual variables under the subgradient descent iterations.
Convergence of the Subgradient Descent
To establish the convergence of the subgradient iterations, it is useful to add an additional constraint in the primal problem , without changing the optimal solution. For this, observe that, for any feasible solution to , from the capacity constraint (17), we have
With this additional constraint, we conveniently restrict the range of each of the admission control variables , in the sub-problems (24), to . This upper-bounds the rate of flows crossing the edge to . Hence, the norms of the subgradients (from Eqn. (31)) are uniformly bounded by
We have the following convergence result:
Theorem 3 (Convergence within a Neighborhood).
Under the subgradient iterations (32), we have
Vi Numerical Simulations
In this Section, we provide simulation results to explore the performance of the UMW+ policy in diverse network and traffic settings.
Vi-A Unicast Traffic in a Wired Network
To begin with, we consider the same wired network topology as in , with two unicast sources and destinations , shown in Figure 1 (a). The capacity of each directed link is taken to be one packet per slot. From the network topology, it can be easily seen that there are two paths (e.g., and and one path (e.g., ), which are mutually edge disjoint. Moreover, we also have and . This implies that the optimal solution to the NUM problem is attained at for any non-decreasing utility function.
Next, we run the proposed UMW+ policy with logarithmic utility functions . The theoretical optimal utility value, in this case, is easily computed to be Figure 2 (a) shows the achieved utility by the UMW+ policy with the variation of the parameter. Figure 2 (b) shows the variation of the total queue length as a function of . These figures clearly demonstrate the the utility-optimality of the UMW+ policy.
Figure 3 shows the temporal dynamics of the subgradient algorithm for different parameters, and two different step-sizes, and . Note that, any solution to the dual problem gives an upper bound of the primal objective (weak duality). The optimal solution of the dual minimization problem has zero duality gap with the optimal primal solution. It is evident from the plots in Figure 3 that the speed of convergence increases and the optimality gap decreases with the increase of the parameter . This observation is consistent with the statement of Theorem 3, which states that the solution obtained by the dual algorithm lies within an additive gap of from the optimal utility .
Broadcast Traffic in a Time-varying Wireless Network
Next, we simulate the UMW+ policy for a time-varying wireless network, shown in Figure 1 (b). Each wireless link is ON i.i.d. at every slot with probability . Link activations are limited by primary interference constraints. We use the same logarithmic utility function as before. Under the proposed UMW+ policy, variation of the average utility and average queue lengths with the parameter is shown in Figure 4 (a) and 4 (b), for three different values of . As expected, with better average channel conditions (i.e., higher values of ) higher utility is achieved with smaller in-network queue lengths.
In this paper, we have proposed the first network control policy, called UMW+, to solve the Utility Maximization Problem with multiple types of concurrent traffic, including unicast, broadcast, multicast, and anycast. The proposed policy effectively exploits the novel idea of precedence-relaxation of a multi-hop network. We relate the UMW+ policy to the subgradient iterations of an associated dual problem. The dual objective function of the associated static NUM problem has been characterized in terms of the Fenchel conjugates of the associated utility function. Finally, illustrative simulation results have been provided for both wired and wireless networks under different input traffic settings.
-  A. Sinha and E. Modiano, “Optimal control for generalized network-flow problems,” IEEE/ACM Transactions on Networking, vol. 26, no. 1, pp. 506–519, Feb 2018.
-  F. P. Kelly, A. K. Maulloo, and D. K. Tan, “Rate control for communication networks: shadow prices, proportional fairness and stability,” Journal of the Operational Research society, vol. 49, no. 3, pp. 237–252, 1998.
-  J. W. Lee, R. R. Mazumdar, and N. B. Shroff, “Downlink power allocation for multi-class cdma wireless networks,” in INFOCOM 2002. Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies. Proceedings. IEEE, vol. 3. IEEE, 2002, pp. 1480–1489.
-  S. H. Low, “A duality model of tcp and queue management algorithms,” IEEE/ACM Transactions On Networking, vol. 11, no. 4, pp. 525–536, 2003.
-  M. Chiang, S. H. Low, A. R. Calderbank, and J. C. Doyle, “Layering as optimization decomposition: A mathematical theory of network architectures,” Proceedings of the IEEE, vol. 95, no. 1, pp. 255–312, 2007.
-  M. J. Neely, E. Modiano, and C.-P. Li, “Fairness and optimal stochastic control for heterogeneous networks,” IEEE/ACM Transactions on Networking (TON), vol. 16, no. 2, pp. 396–409, 2008.
-  X. Lin, N. B. Shroff, and R. Srikant, “A tutorial on cross-layer optimization in wireless networks,” IEEE Journal on Selected areas in Communications, vol. 24, no. 8, pp. 1452–1463, 2006.
-  A. Eryilmaz and R. Srikant, “Fair resource allocation in wireless networks using queue-length-based scheduling and congestion control,” IEEE/ACM Transactions on Networking (TON), vol. 15, no. 6, pp. 1333–1344, 2007.
-  V. Valls and D. J. Leith, “On the relationship between queues and multipliers,” in Communication, Control, and Computing (Allerton), 2014 52nd Annual Allerton Conference on. IEEE, 2014, pp. 250–255.
-  A. Sinha and E. Modiano, “Throughput-optimal broadcast in wireless networks with point-to-multipoint transmissions,” in Proceedings of the 18th ACM International Symposium on Mobile Ad Hoc Networking and Computing, ser. Mobihoc ’17. New York, NY, USA: ACM, 2017, pp. 3:1–3:10.
-  J. Zhang, A. Sinha, J. Llorca, A. Tulino, and E. Modiano, “Optimal Control of Distributed Computing Networks with Mixed-Cast Traffic Flows,” in Computer Communications (INFOCOM), 2017 IEEE Conference on (to appear). IEEE, 2018. [Online]. Available: http://bit.ly/2BlmaoA
-  M. J. Neely, “Stochastic network optimization with application to communication and queueing systems,” Synthesis Lectures on Communication Networks, vol. 3, no. 1, pp. 1–211, 2010.
-  A. Sinha, P. Mani, J. Liu, A. Flavel, and D. A. Maltz, “Distributed load management algorithms in anycast-based cdns,” Comput. Netw., vol. 115, no. C, pp. 1–15, Mar. 2017. [Online]. Available: https://doi.org/10.1016/j.comnet.2017.01.016
-  J.-H. Hoepman, “Simple distributed weighted matchings,” arXiv preprint cs/0410047, 2004.
-  L. Kleinrock, “Queueing systems, vol. 1,” 1975.
-  D. Gamarnik, “Stability of adaptive and non-adaptive packet routing policies in adversarial queueing networks,” in Proceedings of the thirty-first annual ACM symposium on Theory of computing. ACM, 1999.
-  L. Tassiulas and A. Ephremides, “Stability properties of constrained queueing systems and scheduling policies for maximum throughput in multihop radio networks,” Automatic Control, IEEE Transactions on, vol. 37, no. 12, pp. 1936–1948, 1992.
-  D. P. Bertsekas, A. Nedich, A. E. Ozdaglar et al., “Convex analysis and optimization,” 2003.
-  D. P. Bertsekas and A. Scientific, Convex optimization algorithms. Athena Scientific Belmont, 2015.
-  R. Durrett, Probability: theory and examples. Cambridge university press, 2010.
Appendix A Appendix
A-A Proof of Optimality of UMW+
It is well-known that (Theorem 4.5 of ), for arbitrarily small , there exists a stationary randomized policy RAND, which takes randomized actions (packet admission, routing and link scheduling) based only on the current realization of the network state 555This class of policies have been referred to as the -only policies in ., and achieves near-optimal performance:
where the expectations are taken over the randomness of the network configuration process and the randomized actions of the policy RAND.
Combining Eqns. (7) with the objective , under the action of the UMW+ policy, we have
where the inequality (a) follows from the construction of the UMW+ policy in Section III, which is defined to be minimizing the upper-bound of the conditional drift plus penalty, given in the objective . Rearranging the RHS of the above equation, and recalling that the actions of the policy are independent of the virtual queue lengths , we have
Taking expectations of the inequality (A-A) throughout w.r.t. the random network state process , we have
Since the above equation holds for arbitrarily small , taking , we have the bound
Taking expectation of the above inequality w.r.t. the Virtual Queue length process , we have
Adding up the inequalities in Eqn. (36) corresponding to the time , telescoping the LHS, and dividing both sides by , we obtain the following performance guarantee under the action of the UMW+ policy:
Since, is the optimal utility achievable by any feasible policy , we have
Since, , the above equation implies
By definition of , this implies that there exists a finite time , such that, for all , we have
i.e., . Applying Jensen’s inequality , we conclude
Using the bounded admission assumption in Section II-B, the total arrival to each virtual queue is upper bounded by . Hence, we have the following trivial deterministic upper bound on the virtual queue sizes at time :
Thus, the sequence of non-negative random variables are uniformly bounded.
Using the Bounded Convergence Theorem  666 We are implicitly assuming that the limiting random variable is well-defined almost surely., we conclude that
where the equality (b) follows from the bound in Eqn. (38). Since the random variables