Network Utility Maximization with Heterogeneous Traffic Flows
Abstract
We consider the Network Utility Maximization (NUM) problem for wireless networks in the presence of arbitrary types of flows, including unicast, broadcast, multicast, and anycast traffic. Building upon the recent framework of a universal control policy (UMW), we design a utility optimal crosslayer admission control, routing and scheduling policy, called UMW+. The UMW+ policy takes packet level actions based on a precedencerelaxed virtual network. Using Lyapunov optimization techniques, we show that UMW+ maximizes network utility, while simultaneously keeping the physical queues in the network stable. Extensive simulation results validate the performance of UMW+; demonstrating both optimal utility performance and bounded average queue occupancy. Moreover, we establish a precise onetoone correspondence between the dynamics of the virtual queues under the UMW+ policy, and the dynamics of the dual variables of an associated offline NUM program, under a subgradient algorithm. This correspondence sheds further insight into our understanding of UMW+.
I Introduction
Increasingly, networks need to support a heterogeneous mix of traffic that includes unicast, multicast and broadcast traffic flows. Such traffic may be comprised of video streaming, file downloads, distributed computation and storage, as well as a host of other applications. The increase in both volume, as well as QoS requirements, of these emerging applications can significantly strain network resources, especially in bandwidth limited wireless networks. This calls for efficient resource allocation schemes for wireless networks with heterogeneous type of traffic demands. Unfortunately, however, existing resource allocation schemes, both in theory and practice, are limited to dealing with pointtopoint unicast traffic. While there has been some work on resource allocation for multicast and broadcast flows, until very recently there has been no mechanism for allocating resources in networks with arbitrary traffic flows that include concurrent unicast, multicast and broadcast traffic. In [1], the authors developed the first such algorithm for joint routing and scheduling in wireless networks with general traffic flows. The algorithm of [1], called Universal Max Weight is guaranteed to stabilize the network for any arrivals that are within the network capacity (stability) region. However, it provides no service guarantees for traffic that is outside the network capacity region.
When traffic arrivals are outside of the network’s capacity region, an admission control mechanism is needed to limit the admitted traffic to being within the network’s capacity region. In particular, a resource allocation scheme is needed that can optimize userlevel performance across the network. Network Utility Maximization (NUM) is an approach for allocating resources across the network in a manner that maximizes overall “utility,” where user’s level of satisfaction is measured using a concave function of the allocated data rate to that user (i.e., the utility function).
Starting with the work of Kelly nearly two decades ago, [2], there has been a tremendous amount of work on the NUM problem (see, for example, [3], [4], [5] and references therein). These approaches use convex optimization and Lagrangian duality to optimize overall network utility in networks with static traffic demand. In the context of stochastic traffic, the works of [6, 7, 8] used Lyapunov optimization techniques to develop utility optimal flow control and resource allocation for wireless networks with unicast traffic demands. Unfortunately, however, all of these works deal with unicast (pointtopoint) traffic demands.
In this paper, we develop a jointly optimal admission control, routing, and scheduling algorithm for networks with generalized traffic flows. Our approach builds upon the Universal Max Weight (UMW) algorithm of [1] that solves the optimal routing and scheduling problem for networks with general traffic flows, thus we call our algorithm UMW+. To the best of our knowledge, UMW+ is the first efficient solution to the NUM problem for networks with generalized traffic flows, including, but not limited to concurrent unicast, broadcast, multicast, and anycast traffic. We derive UMW+ by using the concept of precedencerelaxed virtual queueing network [1]. Moreover, by formulating the NUM problem as a static convex optimization problem, we show that, under the dual subgradient algorithm, the evolution of the dual variables precisely corresponds to the evolution of the virtual queues in UMW+. This sheds a new light on the relationship between MaxWeight and subgradient algorithms [9]. We make the following key contributions in this paper:

We design the first efficient policy for the NUM problem for arbitrary types of traffic, including, but not limited to, concurrent unicast, broadcast, multicast, and anycast sessions.

We formulate a static version of the NUM problem by decomposing the flows routewise. This is to be contrasted with earlier works on this problem (e.g., [4], [5]), which use nodebased flow conservation, applicable only to the unicast flows. We show that, under the dual subgradient algorithm with unit step size, the evolution of the dual variables exactly corresponds to the evolution of the virtual queues under the proposed UMW+ policy.

We explicitly characterize the dual objective function of the above optimization problem in terms of cost of the shortest route for each source and the Fenchel conjugates of the utility function.

Finally, we validate our theoretical results through extensive numerical simulations.
The rest of the paper is organized as follows: In Section II, we describe the network and traffic model. In Section III, we detail the virtual network framework and derive the UMW+ control policy. In Section IV, we prove the stability of the physical queues under the UMW+ policy. In Section V, we establish the equivalence between the UMW+ policy and the dual subgradient algorithm. In Section VI, we provide simulation results and conclude the paper in Section VII.
Ii System Model and Problem Formulation
IiA System Model
Network Model
Let the topology of a wireless network be given by the graph  where is the set of nodes and is the set of links. Time is slotted, and at every slot, each wireless link could be in either the ON (1) or, OFF (0) state. The random link state process is assumed to be evolving according to a stationary ergodic stochastic process. If at a slot , link is ON and is activated, it can transmit packets in that slot ^{1}^{1}1A link does not transmit any packet if it is OFF..
Interference Model
Due to the wireless interchannel interference, not all links can be activated at a slot simultaneously. The set of all interferencefree links, which may be activated together in a slot, is given by . As an example, in the case of primary interference constraints, the set consists of the set of all Matchings in the graph. On the other hand, in an interferencefree wired network, the set consists of the set of all subsets of links.
Traffic and Utility Model
External packets from different traffic classes are admitted to the network by an admission controller . The set of all possible classes of traffic is denoted by . A traffic class is associated with the following two attributes  (1) type of the traffic (e.g., unicast, broadcast, multicast or anycast), and (2) a monotone increasing strictly concave Utility function . Let denote the set of all possible routes for routing packets from class in the graph . As an example, for a unicast class with source node and destination node , the set consists of the set of all paths in the graph. Similarly, for a broadcast class, the set of all routes is given by the set of all spanning trees in the graph. We consider an infinitely backlogged traffic model, where the admission controller has potentially an unlimited number of packets from each class available for admission.
IiB Admissible Policies
An admissible policy consists of the following three modules: (1) an admission controller , (2) a routing module , and (3) a link scheduler module . The admission controller determines the number of external packets to be admitted to the network from each class in each slot. We assume that, due to physical constraints (power, capacity limitations etc.), at most number of packets may be feasibly admitted from any class per slot ^{2}^{2}2Taking yields the same optimal utility obtained without this constraint.. The routing module routes the admitted packets according to the type of flow they belong to, and the link scheduler activates a subset of interferencefree links from the set in every slot. The set of all admissible policies is denoted by .
IiC The Network Utility Maximization (NUM) Problem
The Network Utility Maximization problem seeks to find an admissible policy , which maximizes the sum utility of all classes, while keeping the queues in the network stable ^{3}^{3}3Throughout the paper, by stability, we will mean almost sure rate stability, defined in Eqn. (3). . Formally, let the random variable denote the number of packets received in common by the destination(s) of class up to time under the action of an admissible policy . Also, denote the random queue length of packets waiting to cross the edge under the policy by . Then the NUM problem seeks to find a policy , which solves the following problem:
(1) 
Subject to,
(2)  
(3) 
where the expectation and the almost sure limits are taken over the randomness of the network configurations and possible randomness in the policy. In the following, we will drop the superscript from the random variables when the driving policy is clear from the context.
Iii The Virtual Network Framework
In this section, we describe a virtual network framework, obtained by relaxing the natural precedence constraints associated with a multihop network. Our goal in this section is to design a utilityoptimal stabilizing control policy for the simpler virtual queueing system. Section IV shows that when the same policy is used in the actual physical network, the physical queues are also stable. The virtual network methodology in a similar context was first introduced in [1].
Consider an admissible policy , which admits packets from class , and activates the links at slot . Taking the random wireless link states into account, the service rate for the virtual queue is packets per slot. Since the virtual network is precedencerelaxed, all admitted packets immediately enter all virtual queues on the selected route, i.e., unlike the physical network, arrivals to a virtual queue need not wait to cross the intermediate links [1]. Thus, the virtual queues evolve as
(4) 
where is the total number of (controlled) arrival of packets to the virtual queue at slot . Clearly, depends on the routes selected by the routing module , i.e.,
(5) 
In order to design a utilityoptimal stabilizing control policy for the virtual queues, we use the driftpluspenalty framework of [12]. Consider the following Lyapunov function, which is quadratic in the virtual queue lengths (as opposed to the usual physical queue lengths [12])
(6) 
The oneslot conditional drift of under the action of a control policy is given as follows
(7) 
where the inequality (a) is obtained by using the virtual queue dynamics in Eqn. (4), the equality (b) is obtained by interchanging the order of summation in the first term, is a finite constant, upper bounded by , where is the maximum number of external admissions per slot per class (defined in Section IIB) and .
Moreover, admission of packets from class in slot yields a “oneslot utility” of for class . Following the “DriftPlusPenalty” framework of [12], we consider a crosslayer admission control, routing and link scheduling policy, which is obtained by minimizing the objective function given at the bottom of this page, over all admissible controls per slot.
For the objective , we have substituted the upper bound of the drift from Eqn. (7) (without the constant ), and is taken to be a fixed positive constant. This yields the following joint routing, admission control, and link scheduling policy , which we call Universal MaxWeight Plus (UMW+):
1. Routing (): The routing policy follows by minimizing the term (a) of the objective . Consider a weighted graph , where each edge is weighted by the corresponding virtual queue length . Under the policy , all admitted packets from class (refer to in Part (2) below) are assigned a route corresponding to the shortest route in . In particular,

For a unicast flow, is the weighted shortest path in .

For a broadcast flow originating from the node , is the Minimum Weight Spanning Tree (MST) in the weighted graph .

For a multicast flow, is the corresponding Steiner tree in the weighted graph .

For an anycast flow [13], is the weighted shortest path from the source to any of the given destinations.
Denote the cost of the weighted shortest route corresponding to class obtained above by , i.e.,
(8) 
In the case of differentiable utility functions , the number of packets admitted may be obtained in closed form as follows:
(10) 
where denotes projection onto the interval .
3. Link Scheduling (): The driftminimizing link scheduling policy is obtained by minimizing the term (b) of the perslot objective function . Consider the weighted graph , where the link of the graph is given a weight of , if it is ON in slot (i.e. ), and , otherwise. Then the link scheduler activates the set of ON links which maximizes the total weight among the set of all interferencefree links , i.e.,
(11) 
In the special case of primary interference constraints, where the set consists of the set of all Matchings of the graph, the problem (11) corresponds to the MaximumWeighted Matching problem, which can be solved efficiently even in a distributed fashion [14].
Our main result in this section is Theorem 1, which claims that under the UMW+ control policy described above, the virtual queues are stable and the average expected utility obtained is arbitrarily close to the optimal utility.
Theorem 1.
Proof.
See the Appendix (Section VIII). ∎
An alternative treatment of Utility Optimality and Stability of the Virtual Queues under the UMW+ policy will be given in Section V, where we relate the virtual queue evolution to the subgradient descents of an appropriately defined dual optimization problem. This sheds new and fundamental insights in the construction and operation of the virtual queues.
Iv Control of the Physical Network
In the physical network, the same admission control, routing and link scheduling policy are used as in the virtual network of Section III. Hence, the same number of packets are admitted at each slot from each class to the virtual and the physical network. As a result, the utility achieved in these two networks are the same, however, their queueing evolutions are different. In this Section, we establish the stability of the physical queues, which, unlike the virtual queues, are subjected to the usual precedence constraint of a multihop network.
IvA Packet arrivals to the Physical Queues
To connect the arrivals of packets in the virtual and the physical network, observe that since the routes are fixed at the sources, the total number of physical packets admitted to the physical network in any time interval , that will cross edge in future, is the same as the number of virtual packets arrivals to the corresponding virtual queue . Let the total service allocated to serve the virtual queue (resp. physical queue) in the time interval be denoted by (resp. ). Using the Skorokhod map [15] for the virtual queue iterations (4), we have
(12) 
However, as noted above, we have
(13) 
Hence, using Theorem (1) for the stability of the virtual queues and combining it with Eqns (12) and (13), we have
(14) 
where .
IvB Stability of The Physical Queues
Note that, Theorem 1 establishes the stability of the virtual queues under the action of the UMW+ policy. To prove the optimality of this policy in the NUM setting of Eqn. (1), we need to establish the stability of the physical queues . This is a nontrivial task because, unlike the virtual queues, the dynamics of the physical queues is subject to the precedence constraints, and regulated by the packet scheduling policy employed in the physical network. A packet scheduling policy is a rule which resolves the contention when multiple packets want to cross the same edge at the same slot. The most common examples of packet scheduling policies include First In First Out (FIFO), Last In First Out (LIFO) disciplines etc. Following our earlier work [1], we consider a simple packet scheduling policy called the Extended Nearest to Origin (ENTO) policy:
Definition 1 (Extended Nearest to Origin policy).
The ENTO policy prioritizes packets according to the decreasing order of their current distances (measured in hoplengths) from their respective sources.
For example, if there are two packets and , which have traversed, say, and hops respectively from their sources, and wish to cross the same active edge (with a unit capacity per slot) in the same slot, the ENTO policy prioritizes the packet over to cross the edge .
Using ideas from adversarial queueing theory [16], it is shown in [1] (Theorem 3) that, under the ENTO packet scheduling policy, the almost sure packet arrival bound (14) also implies the rate stability of the physical queues. Combining this result with Theorem 1, we conclude the following:
Theorem 2 (Stability of the Physical Queues).
Under the action of the UMW+ control policy, the physical queues are rate stable, i.e.,
The proof of Theorem 2 follows directly from the proof of Theorem 3 of [1] and uses the bound (14) as a starting point. Combining Theorem 1 with Theorem 2, we conclude that the proposed UMW+ policy is a utility optimal stabilizing control policy, which solves the NUM problem efficiently. This concludes the first half of the paper.
V A Dual Perspective on the Virtual Queue Dynamics
In this Section, we consider the dual of an offline version of the NUM problem in a static wired network setting (no interference constraints). We give an alternative derivation of the UMW+ policy from an optimization theory perspective, which sheds further insight into the structure of the optimal policy. Our motivation in this section is similar to [5], [7], which give similar development for the Backpressure policy [17]. The dual problem is also of sufficient theoretical and practical interest, as the powerful machinery of convex optimization may be used to derive alternative efficient algorithms for solving the dual problem, which may then be translated to other dynamic policies (apart from the UMW+) for solving the NUM problem.
As in the previous section, the topological structure of a route depends on the type of flow  e.g., a route is a path for unicast and anycast flows, a spanning tree for broadcast flows, a Steiner tree for multicast flows etc.
Fix a strictly positive constant . Our goal is to solve the following utility maximization problem :
(15) 
Subject to,
(16)  
(17)  
(18) 
The objective (15) denotes the total utility, scaled by . The constraint (16) is obtained by decomposing the total incoming flow to a class into all available routes, where the variable corresponds to the amount of flow carried by the route . The constraint (17) corresponds to the capacity of edge , and the constraint (18) corresponds to the nonnegativity property of the flow variables.
In the special case of the NUM problem dealing exclusively with the unicast flows, the flow decomposition constraint (16) is usually replaced with the flow conservation constraints at the nodes, which leads to the Backpressure policy [5]. In contrast, we formulate the NUM problem with the flow decomposition constraint, since flow, in general, is not conserved at the nodes due to packet replications (as in broadcast and multicast flows).
VA The Dual Problem
The problem is a concave maximization problem with linear constraints. To obtain its dual problem, we relax the capacity constraints (17) by associating a nonnegative dual variable with the constraint corresponding to the edge . We choose not to relax the flow decomposition constraints (16) and the nonnegativity constraints (18)^{4}^{4}4Relaxation of these constraints yields a different dual problem.. This yields the following partial Lagrangian:
(19) 
leading to the following dual objective function :
(20) 
Subject to,
(21) 
By strong duality [18], the problem is equivalent to the following dual problem:
(22) 
We next establish a simple lemma which will be useful for our subsequent development.
Lemma 1.
For any fixed , an optimal solution to the problem (20) is obtained by routing the entire flow from each class along a weighted shortest route , weighted by the corresponding dual variables .
Proof: Exchanging the order of summation in the last term of the Lagrangian in Eqn. (19), we have
(23) 
Define to be the cost of the route where each edge is weighted by the dual variable , . Hence, from the last term of the above expression (23) and the flow decomposition constraint (21), it immediately follows that the objective (20) is maximized by routing the entire incoming flow along a weighted shortest route corresponding to class . In other words, for any class ,
where (ties are broken arbitrarily).
Define to be the cost of the shortest route corresponding to class , i.e.,
Since, is defined to be the point wise minimum of several linear functions, it is a concave function of [18]. For several important traffic classes (e.g., unicast, broadcast, anycast) there are standard combinatorial algorithms for efficiently computing (e.g. Weighted Shortest Path, Minimum Weight Spanning Tree etc.).
With an optimal setting of the flow variables resolved by Lemma (1), Eqn. (23) implies that the computation of the dual objective function (20) reduces to optimizing the traffic admission rates for each class as follows:
(24) 
Due to strict concavity of the utility functions , the optimal solution to the problem (20) is obtained by setting the derivative of the objective with respect to the variable to zero, which yields
(25) 
where we project on the set of nonnegative real numbers due to nonnegativity constraints of the rates. Substituting Eqn. (25) into Eqn. (20), we obtain an implicit expression of the dual objective function . In the following, we derive an explicit expression of the dual function in terms of the Fenchel conjugate of the utility functions [18].
VB Derivation of the Dual Objective Function
Substituting the value of into the Lagrangian (19) and noting that only along the corresponding shortest route and is zero otherwise, we have
(26) 
The above expression may also be written in terms of the Fenchel’s conjugate [18] of the utility functions. For this, we recall the definition of the Fenchel conjugate of (an extended realvalued) function :
(27) 
Now, let the function denote the Fenchel conjugate of the function , which, by our assumption, is a strictly convex function. Thus, from Eqn. (26) we can write
(28) 
In the following, we use Eqn. (28) to derive explicit functional forms of the dual objective functions for two important classes of network utility functions.
1. Logarithmic Utility Functions:
Consider the class of Logarithmic utility functions defined as follows:
(29) 
where is a positive constant. Among its many attractive properties, the Logarithmic utility functions ensures proportionally fair rate allocations among all participating classes.
The Fenchel’s conjugate of may be computed as
Hence, the dual function is given as
2. fair Utility Functions: Next, we consider the fair utility functions defined as follows:
(30) 
where and are positive parameters.
The Fenchel’s conjugate of may be computed as
Hence, the dual function is given as
VC Subgradient Method and its Equivalence with UMW+
Since the dual objective , as given in Eqn. (20), is a point wise maximum of linear functions, it is convex [18]. Moreover, the objective , as seen from Eqn. (28), is nondifferentiable, as the shortest path cost is not a differentiable function of , in general. Hence, we use a first order method suitable for nonsmooth objectives, known as the Subgradient Descent [18], to solve the dual problem (22).
A subgradient of the dual function may be computed directly from the capacity constraint of the primal problem (Eqn. (17)) as follows:
(31) 
where .
Finally, we solve the dual problem by the dual subgradient method with constant stepsize equal to . This yields the following iteration for the dual variables :
(32) 
For the stepsize , the above iteration corresponds exactly to the virtual queue dynamics. More precisely, Eqn. (8) corresponds to Lemma 1, which is concerned with routing along the shortest route in the weighted graph; Eqn. (10) corresponds to Eqn. (25), which corresponds to the packet admission control, and the virtual queue dynamics in Eqn. (4) corresponds to the dual subgradient update of Eqn. (32). Moreover, for any constant stepsize , at any step , the virtual queues under UMW+ are exactly equal to the scaled versions of the corresponding dual variables under the subgradient descent iterations.
Convergence of the Subgradient Descent
To establish the convergence of the subgradient iterations, it is useful to add an additional constraint in the primal problem , without changing the optimal solution. For this, observe that, for any feasible solution to , from the capacity constraint (17), we have
With this additional constraint, we conveniently restrict the range of each of the admission control variables , in the subproblems (24), to . This upperbounds the rate of flows crossing the edge to . Hence, the norms of the subgradients (from Eqn. (31)) are uniformly bounded by
We have the following convergence result:
Theorem 3 (Convergence within a Neighborhood).
Vi Numerical Simulations
In this Section, we provide simulation results to explore the performance of the UMW+ policy in diverse network and traffic settings.
ViA Unicast Traffic in a Wired Network
To begin with, we consider the same wired network topology as in [1], with two unicast sources and destinations , shown in Figure 1 (a). The capacity of each directed link is taken to be one packet per slot. From the network topology, it can be easily seen that there are two paths (e.g., and and one path (e.g., ), which are mutually edge disjoint. Moreover, we also have and . This implies that the optimal solution to the NUM problem is attained at for any nondecreasing utility function.
Next, we run the proposed UMW+ policy with logarithmic utility functions . The theoretical optimal utility value, in this case, is easily computed to be Figure 2 (a) shows the achieved utility by the UMW+ policy with the variation of the parameter. Figure 2 (b) shows the variation of the total queue length as a function of . These figures clearly demonstrate the the utilityoptimality of the UMW+ policy.
Figure 3 shows the temporal dynamics of the subgradient algorithm for different parameters, and two different stepsizes, and . Note that, any solution to the dual problem gives an upper bound of the primal objective (weak duality). The optimal solution of the dual minimization problem has zero duality gap with the optimal primal solution. It is evident from the plots in Figure 3 that the speed of convergence increases and the optimality gap decreases with the increase of the parameter . This observation is consistent with the statement of Theorem 3, which states that the solution obtained by the dual algorithm lies within an additive gap of from the optimal utility .
Broadcast Traffic in a Timevarying Wireless Network
Next, we simulate the UMW+ policy for a timevarying wireless network, shown in Figure 1 (b). Each wireless link is ON i.i.d. at every slot with probability . Link activations are limited by primary interference constraints. We use the same logarithmic utility function as before. Under the proposed UMW+ policy, variation of the average utility and average queue lengths with the parameter is shown in Figure 4 (a) and 4 (b), for three different values of . As expected, with better average channel conditions (i.e., higher values of ) higher utility is achieved with smaller innetwork queue lengths.
Vii Conclusion
In this paper, we have proposed the first network control policy, called UMW+, to solve the Utility Maximization Problem with multiple types of concurrent traffic, including unicast, broadcast, multicast, and anycast. The proposed policy effectively exploits the novel idea of precedencerelaxation of a multihop network. We relate the UMW+ policy to the subgradient iterations of an associated dual problem. The dual objective function of the associated static NUM problem has been characterized in terms of the Fenchel conjugates of the associated utility function. Finally, illustrative simulation results have been provided for both wired and wireless networks under different input traffic settings.
References
 [1] A. Sinha and E. Modiano, “Optimal control for generalized networkflow problems,” IEEE/ACM Transactions on Networking, vol. 26, no. 1, pp. 506–519, Feb 2018.
 [2] F. P. Kelly, A. K. Maulloo, and D. K. Tan, “Rate control for communication networks: shadow prices, proportional fairness and stability,” Journal of the Operational Research society, vol. 49, no. 3, pp. 237–252, 1998.
 [3] J. W. Lee, R. R. Mazumdar, and N. B. Shroff, “Downlink power allocation for multiclass cdma wireless networks,” in INFOCOM 2002. TwentyFirst Annual Joint Conference of the IEEE Computer and Communications Societies. Proceedings. IEEE, vol. 3. IEEE, 2002, pp. 1480–1489.
 [4] S. H. Low, “A duality model of tcp and queue management algorithms,” IEEE/ACM Transactions On Networking, vol. 11, no. 4, pp. 525–536, 2003.
 [5] M. Chiang, S. H. Low, A. R. Calderbank, and J. C. Doyle, “Layering as optimization decomposition: A mathematical theory of network architectures,” Proceedings of the IEEE, vol. 95, no. 1, pp. 255–312, 2007.
 [6] M. J. Neely, E. Modiano, and C.P. Li, “Fairness and optimal stochastic control for heterogeneous networks,” IEEE/ACM Transactions on Networking (TON), vol. 16, no. 2, pp. 396–409, 2008.
 [7] X. Lin, N. B. Shroff, and R. Srikant, “A tutorial on crosslayer optimization in wireless networks,” IEEE Journal on Selected areas in Communications, vol. 24, no. 8, pp. 1452–1463, 2006.
 [8] A. Eryilmaz and R. Srikant, “Fair resource allocation in wireless networks using queuelengthbased scheduling and congestion control,” IEEE/ACM Transactions on Networking (TON), vol. 15, no. 6, pp. 1333–1344, 2007.
 [9] V. Valls and D. J. Leith, “On the relationship between queues and multipliers,” in Communication, Control, and Computing (Allerton), 2014 52nd Annual Allerton Conference on. IEEE, 2014, pp. 250–255.
 [10] A. Sinha and E. Modiano, “Throughputoptimal broadcast in wireless networks with pointtomultipoint transmissions,” in Proceedings of the 18th ACM International Symposium on Mobile Ad Hoc Networking and Computing, ser. Mobihoc ’17. New York, NY, USA: ACM, 2017, pp. 3:1–3:10.
 [11] J. Zhang, A. Sinha, J. Llorca, A. Tulino, and E. Modiano, “Optimal Control of Distributed Computing Networks with MixedCast Traffic Flows,” in Computer Communications (INFOCOM), 2017 IEEE Conference on (to appear). IEEE, 2018. [Online]. Available: http://bit.ly/2BlmaoA
 [12] M. J. Neely, “Stochastic network optimization with application to communication and queueing systems,” Synthesis Lectures on Communication Networks, vol. 3, no. 1, pp. 1–211, 2010.
 [13] A. Sinha, P. Mani, J. Liu, A. Flavel, and D. A. Maltz, “Distributed load management algorithms in anycastbased cdns,” Comput. Netw., vol. 115, no. C, pp. 1–15, Mar. 2017. [Online]. Available: https://doi.org/10.1016/j.comnet.2017.01.016
 [14] J.H. Hoepman, “Simple distributed weighted matchings,” arXiv preprint cs/0410047, 2004.
 [15] L. Kleinrock, “Queueing systems, vol. 1,” 1975.
 [16] D. Gamarnik, “Stability of adaptive and nonadaptive packet routing policies in adversarial queueing networks,” in Proceedings of the thirtyfirst annual ACM symposium on Theory of computing. ACM, 1999.
 [17] L. Tassiulas and A. Ephremides, “Stability properties of constrained queueing systems and scheduling policies for maximum throughput in multihop radio networks,” Automatic Control, IEEE Transactions on, vol. 37, no. 12, pp. 1936–1948, 1992.
 [18] D. P. Bertsekas, A. Nedich, A. E. Ozdaglar et al., “Convex analysis and optimization,” 2003.
 [19] D. P. Bertsekas and A. Scientific, Convex optimization algorithms. Athena Scientific Belmont, 2015.
 [20] R. Durrett, Probability: theory and examples. Cambridge university press, 2010.
Appendix A Appendix
AA Proof of Optimality of UMW+
Proof.
It is wellknown that (Theorem 4.5 of [12]), for arbitrarily small , there exists a stationary randomized policy RAND, which takes randomized actions (packet admission, routing and link scheduling) based only on the current realization of the network state ^{5}^{5}5This class of policies have been referred to as the only policies in [12]., and achieves nearoptimal performance:
(33)  
(34) 
where the expectations are taken over the randomness of the network configuration process and the randomized actions of the policy RAND.
Combining Eqns. (7) with the objective , under the action of the UMW+ policy, we have
where the inequality (a) follows from the construction of the UMW+ policy in Section III, which is defined to be minimizing the upperbound of the conditional drift plus penalty, given in the objective . Rearranging the RHS of the above equation, and recalling that the actions of the policy are independent of the virtual queue lengths , we have
(35) 
Taking expectations of the inequality (AA) throughout w.r.t. the random network state process , we have
Now using the properties of the policy from Eqns. (33) and (34), we have
Since the above equation holds for arbitrarily small , taking , we have the bound
Taking expectation of the above inequality w.r.t. the Virtual Queue length process , we have
(36)  
Adding up the inequalities in Eqn. (36) corresponding to the time , telescoping the LHS, and dividing both sides by , we obtain the following performance guarantee under the action of the UMW+ policy:
(37) 
Since, is the optimal utility achievable by any feasible policy , we have
where the inequality (a) follows by Jensen’s inequality applied to the concave functions .
Taking limits of both sides of the inequality (AA) and using (AA), we conclude that
Since, , the above equation implies
By definition of , this implies that there exists a finite time , such that, for all , we have
i.e., . Applying Jensen’s inequality [20], we conclude
(38) 
Using the bounded admission assumption in Section IIB, the total arrival to each virtual queue is upper bounded by . Hence, we have the following trivial deterministic upper bound on the virtual queue sizes at time :
(39) 
Thus, the sequence of nonnegative random variables are uniformly bounded.
Using the Bounded Convergence Theorem [20] ^{6}^{6}6 We are implicitly assuming that the limiting random variable is welldefined almost surely., we conclude that
(40) 
where the equality (b) follows from the bound in Eqn. (38). Since the random variables