Ultra Dense Small Cell Networks: Turning Density into Energy Efficiency

Ultra Dense Small Cell Networks: Turning Density into Energy Efficiency

Sumudu Samarakoon,  Mehdi Bennis,  Walid Saad,  Mérouane Debbah,  and Matti Latva-aho,  This work is supported by the TEKES grant 2364/31/2014, the Academy of Finland (289611), the 5Gto10G project, the SHARING project under the Finland grant 128010, the U.S. National Science Foundation (NSF) under Grants CNS-1460333, CNS-1460316, and CNS-1513697, and the ERC Starting Grant 305123 MORE (Advanced Mathematical Tools for Complex Network Engineering). S. Samarakoon, M. Bennis and M. Latva-aho are with the Department of Communications Engineering, University of Oulu, Finland (e-mail: {sumudu,bennis,matti.latva-aho}@ee.oulu.fi). W. Saad is with Wireless@VT, Bradley Department of Electrical and Computer Engineering, Virginia Tech, Blacksburg, VA (email: walids@vt.edu). M. Debbah is with Mathematical and Algorithmic Sciences Lab, Huawei France R&D, Paris, France, (email: merouane.debbah@huawei.fr) and with the Large Systems and Networks Group (LANEAS), CentraleSupélec, Université Paris-Saclay, 3 rue Joliot-Curie, 91192 Gif-sur-Yvette, France.
Abstract

In this paper, a novel approach for joint power control and user scheduling is proposed for optimizing energy efficiency (EE), in terms of bits per unit energy, in ultra dense small cell networks (UDNs). Due to severe coupling in interference, this problem is formulated as a dynamic stochastic game (DSG) between small cell base stations (SBSs). This game enables to capture the dynamics of both the queues and channel states of the system. To solve this game, assuming a large homogeneous UDN deployment, the problem is cast as a mean-field game (MFG) in which the MFG equilibrium is analyzed with the aid of low-complexity tractable partial differential equations. Exploiting the stochastic nature of the problem, user scheduling is formulated as a stochastic optimization problem and solved using the drift plus penalty (DPP) approach in the framework of Lyapunov optimization. Remarkably, it is shown that by weaving notions from Lyapunov optimization and mean-field theory, the proposed solution yields an equilibrium control policy per SBS which maximizes the network utility while ensuring users’ quality-of-service. Simulation results show that the proposed approach achieves up to gains in EE and reductions in the network’s outage probabilities compared to a baseline model which focuses on improving EE while attempting to satisfy the users’ instantaneous quality-of-service requirements.

{keywords}

Dynamic stochastic game, Lyapunov optimization, mean field games, ultra dense network, 5G

I Introduction

Wireless network densification is viewed as a promising approach to enable a 1000x improvement in wireless cellular network capacity [1, 2, 3, 4]. Indeed, 5G systems are expected to be ultra-dense in terms of transmitters and receivers rendering network optimization highly complex [2] and [5]. In such ultra dense network (UDN) environments, resource management problems, such as power control and user equipment (UE) scheduling, become significantly more challenging due to the scale of the network [2, 6]. In addition, the spatio-temporal traffic demand fluctuations in the network, the dynamics in channel conditions, and the increasing overhead due to the need for coordination will further exacerbate the challenges of resource management in UDNs. For instance, the uncertainties in terms of queue state information (QSI) and channel state information (CSI) as well as their evolution over time will play a pivotal role in resource optimization.

Due to the involvement of large number of devices and the severe coupling of their control parameters with one another, the current state-of-the-art approaches used in classical, small scale network deployments are inadequate to study the resource optimization in UDNs. Thus, power control, UE scheduling, interference mitigation, and base station (BS) deployment strategies for energy-efficient UDN have been recently investigated in a number of works such as [1] and [5, 7, 8, 6, 9, 10, 11]. The work in [6] investigates the potential gains and limitations of energy efficiency (EE) for outdoor UDNs considering small cells idle mode capabilities and BS/UE densification. In [7], an analytical framework was proposed for designing UDNs that maximizes EE by modeling the network using stochastic geometry and obtaining new lower bound on the average spectral efficiency. The authors in [8] investigate how different BS and antenna deployment strategies, for both indoor and outdoor UDNs, can impact EE. In [9], a power control and UE scheduling mechanism is proposed for UDN of clustered small cells which have access to a database consisting signal-to-noise ratio values of each BS-UE link. The work in [10] proposes an optimization framework which allows flexible UE association among densely deployed BSs by minimizing the signaling overhead. The authors in [11] study how to improve EE of an UDN by jointly optimizing the resource utilization and the temporal variances in the traffic load. These studies provide valuable insights of both performance gains and limitations of UDNs. However, most of these works [7, 8, 6, 9, 10], ignore uncertainties in QSI, do not properly model network dynamics, and typically focus on the problems of UE scheduling and resource allocation in isolation. Instead, in practice, there is a need to treat those problems jointly while accounting for the QSI dynamics and uncertainty. Moreover, none of these existing works study the behavior of the ultra dense system setting in which the number of BSs and UEs grows infinitely large.

Recently, mean-field games (MFGs) received significant attention in the context of cellular networks with large number of players [12, 13, 14, 15, 16]. In MFGs, a large population consists of small interacting individual players and their decision making strategies are studied [12, 13]. Due to the size of the population, the impacts of individual states and decisions are negligible while the abstract behavior of the population is modeled by a mean-field (MF). As a result, the MF regime allows to cast the multi-player problem into a more tractable single player problem which depends on the state distribution of the population. In [14], concurrent packet transmissions among large number of transmitter-receiver pairs with the objective of minimizing transmit power under the uncertainties of QSI and CSI are investigated using MFGs. The authors in [15] study an energy efficient power control problem in multiple access channels whereas transmitters strategies depend on their receivers, battery levels and the strategies of others. In [16], a competition of wireless resource sharing among large number of agents is investigated as an MFG. Despite their interesting analytical perspective, these existing works remain limited to traditional macrocellular networks and do not address the challenges of small cell networks. Moreover, for simplicity, these works consider models with a single UE per BS or they address multi-user scenarios with conventional UE scheduling schemes (proportional-fair, round-robin, first-in-first-out). Thus, the opportunity to improve the proposed solutions for multi-user cases by smart UE scheduling has not been treated yet.

When considering multiple UEs associated with SBS, UE scheduling directly affects the performance of each SBS. Due to the dynamic nature of channels and arbitrary arrivals, the UE scheduling process becomes a stochastic optimization problem [17, 18, 19, 20]. To solve such stochastic optimization problems while ensuring queue stability, the dual-plus-penalty (DPP) theorem from the Lyapunov optimization framework can be used [17, 18, 19, 21, 20, 22]. Lyapunov DPP approach simplifies the stochastic optimization problem into multiple subproblems which can be solved at each time instance. Remarkably, this simplification ensures the convergence to the optimal solution of the original stochastic optimization problem [17, 18, 19, 20]. The work in [19] presents a utility optimal scheduling algorithm for processing networks based on Lyapunov DPP approach. In [21], Lyapunov DPP method is used to solve stochastic optimization problem with non-convex objective functions. In [22], a Lyapunov DPP framework is used to dynamically select service nodes for multiple video streaming UEs in a wireless network. It can be noted that the Lyapunov DPP approach is a powerful tool for solving stochastic optimization problems. However, to implement the above technique in decentralized control mechanisms, i) the objective function should be chosen such that it can be decoupled among the control nodes as in [19] or ii) necessary bounds need to be identified for the coupling term as the authors in [22] evaluate a lower bound for the capacity in order to mitigate coupling due to interference. Therefore, a suitable method to make an accurate evaluation on the coupled term (interference in wireless UDNs) needs to be investigated.

The main contribution of this paper is to propose a novel joint power control and user scheduling mechanism for ultra-dense small cell networks with large number of small cell base stations (SBSs). In the studied model, the objective of SBSs is to maximize their own time average energy efficiency (EE) in terms of the amount of bits transmitted per joule of energy consumption. Due to the ultra-dense deployment, severe mutual interference is experienced by all SBSs and thus, must be properly managed. To this end, SBSs have to compete with one another in order to maximize their EEs while ensuring UEs’ quality-of-service (QoS). This competition is cast as a dynamic stochastic game (DSG) in which players are the SBSs and their actions are their control vectors which determine the transmit power and user scheduling policy. Due to the non-tractability of the DSG, we study the problem in the mean-field regime which enables us to capture a very dense small cell deployment. Thus, employing a homogeneous control policy among all the SBSs in the network, the solution of the DSG is obtained by solving a set of coupled partial differential equations (PDEs) known as Hamilton-Jacobi-Bellman (HJB) and Fokker-Planck-Kolmogorov (FPK) equations [12]. The resulting equilibrium is known as the mean-field equilibrium (MFE). We analyze the sufficient conditions for the existence of an MFE in our problem and we prove that all SBSs converge to this MFE. Exploiting the stochastic nature of the objective with respect to QSI and CSI, the UE scheduling procedure is modeled as a stochastic optimization problem. To ensure the queue stability and thus, UE’s QoS, the UE scheduling problem is solved within a DPP-based Lyapunov optimization framework [22, 17]. An algorithm is proposed in which each SBS schedules its UEs as a function of CSI, QSI and the mean-field of interferers. Remarkably, it is shown that combining the power allocation policy obtained from the MFG and the DPP-based scheduling policy enables SBSs to autonomously determine their optimal transmission parameters without coordination with other neighboring cells. To the best of our knowledge, this is the first work combining MFG and Lyapunov frameworks within the scope of UDNs. Simulation results show that the proposed approach achieves up to gains in EE and reductions in the network’s outage probabilities compared to a baseline model which improves EE while satisfying users’ instantaneous quality-of-service requirements.

The rest of this paper is organized as follows. Section II presents the system model, formulates the DSG with finite number of players and discusses the sufficient conditions to obtain an equilibrium for the DSG. In Section III, using the assumptions of large number of players the DSG is cast as a MFG and solved. The solution for UE scheduling at each SBS based on DPP framework is examined in Section IV. The results are discussed in Section V and finally, conclusions are drawn in Section VI.

Ii System Model and Problem Definition

Consider a downlink of an ultra dense wireless small cell network that consists of a set of SBSs using a common spectrum with bandwidth . These SBSs serve a set of UEs. Here, where is the set of UEs served by SBS . For UE scheduling, we use a scheduling vector for SBS where indicates that UE is served by SBS at time and otherwise. The channel gain between UE and SBS at time is denoted by and an additive white Gaussian noise with zero mean and variance is assumed. The instantaneous data rate of UE is given by:

(1)

where is the transmission power of SBS , is the channel gain between SBS and UE , is the interference term.

We assume that SBS sends bits to UE . Thus, the time evolution of the -th SBS queue, is given by:

(2)

where and are the vectors of arrivals and transmission rates at SBS and is the vector of channel gains. Moreover, a vector of control variables is defined such that is the SBS local control vector and is the control vector of interfering SBSs. The evolution of the channels are assumed to vary according to the following known stochastic model [23]:

(3)

where the deterministic part considers path loss and shadowing while the random part with positive constant includes fast fading and channel uncertainties. The evolution of the entire system can be described by the QSI and the CSI as per (2) and (3), respectively. Thus, we define as the state of the system at time with over the state space . The feasibility set of SBS ’s control at state is defined as . Note that we will frequently use instead for notational simplicity. As the system evolves, UEs need to be scheduled at each time slot based on QSI and CSI. The service quality of UE is ensured such that the average queue of packets intended for UE at its serving SBS is finite, i.e. .

The objective of this work is to determine the control policy per SBS which maximizes a utility function while ensuring UEs’ quality of service (QoS). The utility of a SBS at time is its EE which is defined as . Here, is the fixed circuit power consumption at an SBS [24]. Let be the limiting time average expectation of the control variables . Formally, the utility maximization problem for SBS can be written as:

(4a)
subject to (4d)

Furthermore, we assume that SBSs serve their scheduled UEs for a time period of . Therefore, we use the notion of time scale separation between transmit power allocation and UE scheduling processes, hereinafter. For SBS , the transmit power allocation is determined for each transmission and thus, is a fast process. However, UE scheduling is fixed for a duration of to ensure a stable transmission. Therefore, UE scheduling is a slower process than power allocation.

Ii-a Resource management as a dynamic stochastic game

We focus on finding a control policy which solves (4) over a time period for a given set of scheduled UEs considering the state transitions . Therefore, we define the following time-and-state-based utility for SBS :

(5)

The goal of each SBS is to maximize this utility over for a given UE scheduling subject to the system state dynamics , where:

and .

As the network state evolves as a function of QSI and CSI, the strategies of the SBSs must adapt accordingly. Thus, maximizing the utility for under the evolution of the network states can be modeled as a dynamic stochastic game (DSG) as follows:

Definition 1

For a given UE scheduling mechanism, the power control problem can be formulated as a dynamic stochastic game where:

  • is the set of players which are the SBSs.

  • is the set of actions of player which encompass the choices of transmit power for given scheduled UEs .

  • is the state space of player consisting QSI and CSI .

  • is the average utility of player that depends on the state transition as follows:

    (6)

One suitable solution for the defined DSG is the closed-loop Nash equilibrium (CLNE) defined as follows:

Definition 2

The control variables constitute a closed-loop Nash equilibrium if, for ,

(7)

is satisfied , and .

To satisfy (7), each SBS must have the knowledge of its own state and a feedback on the optimal strategy of the rest of SBSs and thus, the achieved equilibrium is known as a CLNE. We denote by the trajectories of the utilities induced by the NE. The existence of the NE is ensured by the existence of a joint solution to the following coupled HJB equations [12]:

(8)

defined for each SBS and tr() is the matrix trace operation.

Proposition 1

(Existence of a closed-loop NE in ) A sufficient condition for the existence of a closed-loop NE in is that for all , we must have where for all and their scheduled UEs .

{proof}

Proving the existence of a solution to the HJB equation given in (8) each SBS provides a sufficient condition for the existence of a NE for the DSG [12]. Thus, there exists a solution if the function

(9)

is smooth [25]. According to (6), the average utility that depends on the state transition is independent of the current choice of controls . Therefore, the terms , and become constant with respect to and thus, are neglected since they do not impact on . Considering the first derivative, the maximizer needs to satisfy which yields,

(10)

where is the transmit power of the respective SBS and . Note that we have omitted the dependencies over time and states for notational simplicity.

In order to evaluate the smoothness of , we use the implicit function theorem [26]. Thus, we define a function and check whether is held. If is held, there exists such that and . Since , the smoothness is ensured by

(11)

Note that we consider and thus, and is bounded above, i.e. . A bounded ensures the solution to (10) remains finite (finite transmit power) while guarantees for all feasible transmit powers. Thus, the above sufficient condition (11) is held for all the particular choices of and .

Solving mutually coupled HJB equations is complex when . Furthermore, it requires gathering QSI and CSI from all the SBSs throughout the network which incurs a tremendous amount of information exchange. Hence, it is impractical for UDNs with large . However, as grows extremely large, it can be observed that the set of all players transform into a continuum and their individuality becomes indistinguishable. In such a scenario, we can solve the dynamic stochastic game using the tools of mean-field theory, as explained next.

Iii Optimal Power Control: A Mean-Field Approach

As the number of SBSs becomes large (), we assume that the interference tends to be bounded in order to have non-zero rates as observed in [14, 15] and [27]. Moreover, each SBS implements a transmission policy based on the knowledge of its own state. SBSs in such an environment are indistinguishable from one another thus, resulting in a continuum of players. This is a reasonable assumption in UDNs because the impact of an individual SBS’s decision on the system is negligible from the macroscopic view unless such a decision is adopted by a considerable large portion of the population. As an example, the choice of a SBS to increase its energy consumption by 1 Watt in a system consisting of thousands of SBSs consuming couple of megawatts is insignificant. However, it is a significant increment of energy consumption if the same decision is made by of the population together. Remarkably, increasing the number of SBSs infinitely large and allow them to become a continuum allows us to simplify the solution of all HJB equations by reducing them to two equations as discussed below.

At a given time and state and for a scheduled UE , the impact of other SBSs on the choice of a given SBS appears in the interference term:

As the number of SBSs grows large, we consider that the interference is bounded as shown in [27, 28, 29] and [30] since the path loss is modeled according to the inverse-square law. Thus, the interference term can be rewritten by using a normalization factor for the channels [27]. Let be the normalization factor where is SBS density and thus, the channel gain becomes with . Thus, the interference can be rewritten as (12)

(12)

 

with a SBS state distribution where

(13)

Here, is the Dirac delta function and .

Now, we assume that each SBS has only the knowledge of its own state and, thus, it will implement a homogeneous transmission policy, i.e. for all , based the knowledge that it has available. Furthermore, the transmission policy satisfies . With the above assumptions, the SBS states for all become exchangeable under the transmission policy  [31, 32], i.e.,

(14)

where is any joint rule of the SBS states and produces any permutation over . Due to this exchangeability, the game now consists of a set of generic players including a generic SBS with the state at time . Thus, the interference under the homogeneous transmission policy with respect to the state distribution will be given by:

(15)

As , the set of players becomes a continuum. Since the transmission policy preserves the indistinguishable property (exchangeability in SBS states), there exists a limiting distribution such that [33, 15],

(16)

This limiting distribution of the states is defined as the MF.

Fig. 1: Mean-field interference from the perspective of an SBS. As the number of interfering SBSs grows large, the interference observed at a generic UE becomes independent of individual states and transmission policies of SBSs and only depends on the time and the limiting distribution .

Furthermore, when , the channel gain is a finite quantity for any given time , i.e. . Since the homogeneous transmission policy satisfies and is bounded by a maximum transmit power, we have . Therefore,

Applying this result on (15), as , the interference

(17)

converges to a finite value which depends on . With the convergence of the interference along the indistinguishable transmission policy and SBS states, the DSG converges to an MFG [33]. A simplified illustration of the system is shown in Fig. 1.

Thus, the utility maximization problem and the evolution of the states are reformulated for a generic SBS as follows:

(18a)
subject to (18c)

for all where and a diagonal matrix . Here, . Similarly, the utility of a generic SBS follows (6) with the necessary modifications. The formal definition of the MF equilibrium is as follows:

Definition 3

The control vector constitutes a mean-field equilibrium if, for all with the MF distribution , it holds that,

In the MF framework, the MF equilibrium given by the solution of (18) is equivalent to the NE of the -players DSG [12]. Moreover, the optimal trajectory is found by applying backward induction to a single HJB equation and the MF (limiting distribution) is obtained by forward solving the FPK equation as (19),

(19)

 

respectively. The optimal transmit power strategy is given by,

(20)

Solving (19)-(20) yields the behavior of a generic SBS in terms of transmission power, utility and state distribution.

The main motivation behind this mean-field analysis is to reduce the complexity of the -player DSG. To solve the original DSG with SBSs/players presented in Section II-A, one must solve HJB PDEs given by (8). However, the solution of the MFG with large can be found by simply solving the two coupled HJB-FPK PDEs rather solving coupled HJB PDEs as in the original problem. Clearly, this reduces complexity considerably and can scale well with the number of players. The relation between the solution of -player DSG () and the solution of MFG () is where is the -th order correction coefficient [12]. It should be noted that the solution of the MFG is equivalent to the solution of the -player DSG only for large . Therefore, it could be challenging to solve the HJB-FPK coupled PDEs for small number of SBSs and thus, the proposed method is not applicable for highly sparse networks. As the number of SBSs increases, the solution of the DSG tends towards the solution of the MFG. Therefore, for dense to ultra dense networks, the complexity of the proposed method is much lower compared to solving the original -player DSG. In fact, the complexity of the proposed method remains constant for all large . Fig. 2 illustrate the complexity of the proposed method by the number of iterations needed to solve the HJB-FPK coupled PDEs for different number of SBSs.

Fig. 2: Number of iterations to solve the HJB-FPK PDEs for different number of SBSs in the system. The plot consists of a set of results for different number of SBSs over a fixed area of

Here, it can be observe that for systems with large number of SBSs, the MFE is achieved by a small number of iterations and thus, the complexity remains fixed.

Iv UE Scheduling Via Lyapunov Framework

Fig. 3: Inter-relation between the MFG and the Lyapunov framework.

By using time scale separation, the scheduling variables are decoupled from the mean-field game and thus, can be optimized separately. Some of the existing baseline approaches for UE scheduling are proportional fair (PF) scheduling in terms of rates, best-CSI based UE scheduling, and scheduling based on highest QSI [19, 18, 34] and [35]. PF scheduling ensures the fairness among UEs in terms of their average rates (history) while the latter methods exploit the instantaneous CSI or QSI. Using such conventional schedulers for solving (4) within a UDN context will not properly account for the inherent CSI and QSI dynamics over space and time, thus yielding poor performance. Therefore, we solve the original utility maximization problem of SBS with respect to the scheduling variables as follows;

(21a)
subject to (21c)

Here, the feasible set consists of all the vectors with and . Note that the scheduling variables are relaxed from integers to real numbers for the ease of analysis. Furthermore, we consider UE is expected to be served by SBS with a data rate of .

Proposition 2

Given that the transmit power profile and the state distribution , the expected data rate of UE at time is calculated as follows:

(22)

assuming that UE is scheduled by SBS .

{proof}

As discussed in Section III, as , all SBSs will follow the optimal transmission policy given in (20). Moreover, solving (19) yields the MF and thus, the expected interference becomes . For any given time , since all the SBSs aware of their own states, the knowledge of and is available at SBS . Therefore, as UE is scheduled by SBS , the data rate given in (1) becomes (22).

In order to solve the stochastic optimization problem (21) per SBS , the drift plus penalty (DPP) approach in Lyapunov optimization framework can be applied [17, 20]. The Lyapunov DPP approach decomposes the stochastic optimization problem into sub-policies that can be implemented in a distributed way. Therefore, copies of problem (21) are locally solved at each SBS and, thus, the proposed solution can apply for a large number of SBSs i.e., as .

First, a vector of auxiliary variables is defined to satisfy the constraints (21c). These additional variables are chosen from a set independent from both time and state. Thus, (21) is transformed as follows;

(23a)
subject to (23d)

To satisfy the constraint in (23d), we introduce a set of virtual queues for each associated UE . The evolution of virtual queues will follow [17];

(24)

Consider the combined queue and its quadratic Lyapunov function . Modifying (2) considering a chunk of time, the evolution of the queue of UE can be reformulated as . Thus, one-slot drift of Lyapunov function is given by,

Neglecting the indices and for simplicity and using,

the one-slot drift can be simplified as follows:

(25)

where is a uniform bound on the term . The conditional expected Lyapunov drift at time is defined as . Let be a parameter which control the tradeoff between queue length and the accuracy of the optimal solution of (23) and be the current running time averages of scheduling variables. Introducing a penalty term to the expected drift and minimizing the upper bound of the drift DPP,

yields the control policy of SBS . Thus, the objective of SBS is to minimize the below expression given by,

(26)

at each time . The terms and are neglected since they do not depend on and . Note that terms and have decoupled the scheduling variables and the auxiliary variables, respectively. Thus, the subproblems of finding auxiliary variables and scheduling variables can be presented as below.

Iv-a Evaluation of auxiliary variables

Since the auxiliary variables are decoupled from scheduling variables as given in (26), the formal representation of auxiliary variables evaluation at SBS for time is as follows:

(27a)
subject to (27b)

It can be noted that the feasible set is a convex hull. Due to the affine nature of the objective function, we can conclude that the optimal solution for (27) should lie on a vertex of the convex hull [36]. Thus, the optimal solution is given by,

(28)

Iv-B Determining the scheduling variables

1:  Input: and for and SBS .
2:  while  true  do
3:     Observation: queues and , and running averages .
4:     Auxiliary variables: .
5:     Scheduling: .
6:     Update: , and .
7:     
8:  end while
Algorithm 1 UE Scheduling Algorithm Per SBS

Optimal scheduling of SBS at time is found from solving a subproblem based on term in (26) as follows:

(29a)
s.t. (29b)

Since the feasible set is a convex hull and the objective is an affine function of , the optimal solution of (29) lies on a vertex of the convex hull . Thus, the optimal scheduling at time is given by,

(30)

where .

Remark 1

Even though we relax the boolean scheduling variables to continuous variables in the set , the optimal solution given in (30) results a boolean scheduling vector. Thus, we claim that relaxing (21) does not change the optimality of the scheduling.

The Lyapunov DPP method ensures that the gap between the time average penalty and the optimal solution is bounded by the term  [20, 18], and [22]. Thus, the optimality of the solution is ensured by the choice of a sufficiently large . The interrelation between the MFG and the Lyapunov optimization is illustrated in Fig. 3 and UE scheduling algorithm which solves (23) is given in Algorithm 1.

V Numerical Results

For simulations, the dimensions of the problem must be simplified in order to solve the coupled PDEs using a finite element method. We used the MATLAB PDEPE solver for this purpose. The utility of an SBS at time is its EE and the ultimate goal is to maximize the average expected energy efficiency defined as . Here, is the duration of the entire simulation and is the fixed circuit power consumption at an SBS [24]. We assume that channels are not time-varying and thus, the state is solely defined by the QSI111 Note that the assumption of fixed channels for simulations does not impact the theoretical contribution. Furthermore, even under non-fading channels, the provided results can model the ergodic behavior of the system. . The initial limiting distribution, , are assumed to follow a Gaussian distributions with means and variance . The choice of the final utility, the boundary condition, is to encourage the scheduled UE to obtain an almost empty queue by the end of its scheduled period . The arrival rate for a UE is modeled as a Poisson process with a mean of . For simulations, the QSI is normalized by the queuing capacity of an SBS which is assumed to be kb. Moreover, for simulations we assume that the transmit power is , the circuit power consumption is , and the variance of Gaussian noise is . The system parameters and channel models are based on [37].

The proposed method is compared to a baseline method which uses a proportional fair UE scheduler and an adaptive transmission policy. Similar to the proposed method, SBSs in the baseline method seek to maximize EE by optimizing the transmit powers. However, in the baseline, SBSs are not able to optimize their control parameters over the states due to the fact that they are oblivious to the states of the rest of the network. Thus, the SBSs in the baseline method adopt a myopic approach in which they seek to maximize the instantaneous EE by optimizing for each subject to UEs’ instantaneous QoS requirement. Moreover, each SBS needs to estimate the interference in order to solve its optimization problem. Here, we assume that SBSs use the time average interference from their past experience and consider it as a valid estimation for the interference. The SBS density of the system is defined by the average inter-site-distance (ISD) normalized by half of the minimum ISD, i.e. 1 unit of ISD implies an average distance of . The average load per SBS is while during the discussion on results, we use low load for UEs per SBS and high load for UEs per SBS. Once a UE is scheduled, 100 transmissions will take place within the time period of .

V-a Mean-field equilibrium of the proposed model

(a) MF distribution at the equlibrium as a function of time and QSI.
(b) Evolution of the limiting distribution for a given QSI.
Fig. 4: Evolution of limiting distribution at the MF equilibrium.

Fig. 4 shows the MF distribution , which represents the evolution of the QSI distribution of scheduled UEs over time at the MF equilibrium. During the period of , SBSs transmit to their scheduled UEs and expect to achieve a QSI close to zero by as shown in Fig. (a)a. Noted that, by the end of the transmission phase, the number of scheduled UEs with high QSI decreases thus, allowing the SBSs to schedule a new set of UEs by the next UE scheduling phase. Here, the queues for all UEs will buildup whether they are scheduled or not due to the continuous arrivals with mean at their respective serving SBSs. As the queue grows beyond an SBS’s capacity (i.e. ), the arrivals are dropped and thus, the associated UE will suffers from an outage. Therefore, achieving a QSI almost close to zero for a scheduled UE provides the opportunity to build up its queue over a large period thus contributing to a reduction in the outage probability. In Fig. (b)b, we can see that the fraction of queues with vanishes before the transmission duration ends. As time evolves, the queues get empty based on the rates prior to new arrivals and thus, an oscillation is observed for the queue fractions with while exhibits a monotonic increase.

(a) Transmit power at the MF equilibrium as a function of time and QSI.
(b) Evolution of the transmit power for a given QSI.
Fig. 5: Evolution of transmit power at the MF equilibrium.

The transmit power policy at the MF equilibrium is shown in Fig. 5. It can be observed that a higher transmit power is needed when the QSI is high and this power can be reduced at low QSI, showing the overall EE of the proposed approach. At , a moderate transmit power is used even for UEs with high QSI. Thus, SBSs prevent unnecessary interference within the system. As the scheduling period arrives to an end, we recall that the choice of forces SBSs to obtain smaller QSI at . For UEs with low QSI, SBSs use moderate transmit power to maximize EE. However, for UEs with high QSI, SBSs use higher transmit power to provide high data rates in order to empty their respective queues as well as preventing outages. Thus, as time evolves, SBSs increase their transmit power for UEs with high QSI as illustrated in Fig. (b)b thereby improving the final utility.

V-B Energy efficiency and outage comparisons

(a) Comparison of EE in terms of transmit bits per unit energy for low and high loads .
(b) Comparison of outage probabilities for low and high loads .
Fig. 6: Comparison of the behavior of EE and outage probabilities for different SBS densities.

In Fig. (a)a, we show the EE of the system in terms of the number of bits transmitted per joule of energy consumed as a function of ISD. Here, the load represents the number of UEs served by an SBS. For a dense network (), the proposed method will improve EE of about compared to the baseline model with . However, as the load increases to , this EE improvement will reach up to . These improved EE for the dense scenario shown in the proposed approach is due to its ability to adapt to the dynamics of the network, as enabled by the stochastic game formulation and its MF approximation. Naturally, as the network becomes less dense, the advantages of the proposed approach will be smaller. For instance, for a non-dense network with an and UEs per SBS, performance advantages of the MF approach decreases to as shown in Fig. (a)a. For a very lightly loaded and non-dense network, such as when and UEs per SBS, the use of the baseline method will be slightly more advantageous when compared to the proposed approach. This advantage is of about of improvement in the EE.

Fig. (b)b illustrates the outage probability as a function of ISD. Here, the outage probability is defined as the fraction of unsatisfied UEs whose arrivals are dropped due to limitations of the queue capacity. It can be noted that the proposed method uses smart UE scheduling and transmit power policy based on QSI which ensures a higher UE QoS. Although the baseline model optimizes the transmit power with the goal of improving EE, it is unable to track the QSI dynamics and adapt the UE scheduling accordingly. Thus, higher number of UEs suffer from outages. As the network becomes highly loaded and more dense (i.e., as the number of SBSs and UEs increase), the number of UEs in outage will naturally increase for both approaches. This is due to the increased interference and the high average waiting time due to increased number of UEs. From Fig. (b)b, we observe that the proposed method yields and reductions in outages compared to the baseline model for both low and high loads, respectively, in UDNs. For a sparse network, the outage reductions are and for low and high loads, respectively.

(a) Comparison of EE for sparse and dense networks .
(b) Comparison o