Distributed Virtual Resource Allocation in Small Cell Networks with Full Duplex Selfbackhauls and Virtualization
Abstract
Wireless network virtualization has attracted great attentions from both academia and industry. Another emerging technology for next generation wireless networks is inband full duplex (FD) communications. Due to its promising performance, FD communication has been considered as an effective way to achieve selfbackhauls for small cells. In this paper, we introduce wireless virtualization into small cell networks, and propose a virtualized small cell network architecture with FD selfbackhauls. We formulate the virtual resource allocation problem in virtualized small cell networks with FD selfbackhauls as an optimization problem. Since the formulated problem is a mixed combinatorial and nonconvex optimization problem, its computational complexity is high. Moreover, the centralized scheme may suffer from signaling overhead, outdated dynamics information, and scalability issues. To solve it efficiently, we divide the original problem into two subproblems. For the first subproblem, we transfer it to a convex optimization problem, and then solve it by an efficient alternating direction method of multipliers (ADMM)based distributed algorithm. The second subproblem is a convex problem, which can be solved by each infrastructure provider. Extensive simulations are conducted with different system configurations to show the effectiveness of the proposed scheme.
I Introduction
By abstracting and sharing resources among different parties, virtualization can significantly reduce the cost of equipment and management in networks [1]. With the tremendous growth in wireless traffic and service, it is inevitable to extend virtualization to wireless networks [2, 3]. In wireless virtualization, physical wireless network infrastructure and physical radio resources are abstracted and sliced into virtual wireless resources, which can be shared by multiple parties. After virtualization, the wireless network infrastructure owned by an infrastructure provider (InP) can be decoupled from the services that it provides. At the same time, mobile virtual network operators (MVNOs) provide services to users. Since the physical resources are abstracted and sliced into virtual resources, it is possible that different MVNOs coexist on the same InP to share the infrastructure and radio spectrum resources, which enables the reducing of capital expenses (CapEx) and operation expenses (OpEx)[2].
The authors of [4] discussed the control and management frameworks of wireless network virtualization. Virtual resource sharing mechanisms were investigated in [5], where the dynamic interactions among MVNOs and InPs are modeled as a stochastic game. In addition, there are some other works focusing on the virtualization of certain specific wireless networks. For example, in the context of cellular networks, Zaki et al. [6] proposed a virtualization framework for longterm evolution (LTE) systems, in which a supervisor is used to virtualize the eNB and manage the physical resources. For wireless local area networks (WLANs) virtualization, a SplitAP architecture was proposed in [7], and a resource sharing algorithm based on control theory was designed in [8]. Moreover, virtualization techniques for WiMAX networks [9] and wirelessoptical heterogeneous networks [10] were also studied.
Although some excellent researches have been done for wireless virtualization, most existing works do not consider small cell networks with selfbackhauls. Recently, small cell networks have been regarded as one of the key components of next generation cellular networks to improve spectrum efficiency and energy efficiency [11, 12]. Traditionally, there are two kinds of backhauls in small cell networks: wired backhaul (e.g., optical [13] or DSL [14]) and wireless backhaul (e.g., microwave [15] or millimeter waves [16]). Since these traditional backhauls are very expensive for infrastructure deployments, selfbackhauled small cells have attracted great attentions from both academia and industry [17, 18]. Wireless selfbackhauling can improve reachability and coverage by easing connectivity between nodes [18]. In [17], a selfbackhauling scheme was proposed, in which the selfbackhaul link uses the same spectrum with the small cell downlink, but on different time slots.
Another emerging technology for next generation wireless networks is inband full duplex (FD) communications. With the recent advances of selfinterference cancellation techniques [19, 20], it is possible for radios to transmit and receive simultaneously in the same frequency band. Due to its promising performance, FD communication has been considered as an effective way to achieve selfbackhauls for small cells. The authors of [21] and [20] made fine attempts in this direction. Nevertheless, no detail has been reported.
Despite the potential vision of small cell networks with FD selfbackhauls and virtualization, many research challenges remain to be addressed. One of the main research challenges is resource allocation, which plays an important role in traditional wireless networks [22, 23, 24]. When wireless virtualization and FD selfbackhauls are jointly considered, the problem of resource allocation becomes even more challenging, since the backhaul and access links are coupled, which depend on virtual resource allocation and selfinterference cancellation performance. To the best of our knowledge, the problem of virtual resource allocation in small cell networks with FD selfbackhauls and virtualization has not been studied in previous works. The distinct features of this paper are summarized as follows.

We introduce wireless virtualization into small cell networks, and propose a virtualized small cell network architecture, where multiple InPs and multiple MVNOs coexist. In the proposed virtualized small cell networks, in addition to radio spectrum, both macro base stations (MBSs) and small cell base Stations (SBSs) from different InPs are virtualized as virtual resources, which can be dynamically shared by users from different MVNOs.

We formulate the virtual resource allocation problem in virtualized small cell networks with FD selfbackhauls as an optimization problem, which maximizes the total utility of all MVNOs, considering not only the revenue earned by serving users but also the cost paid to InPs for consuming power, spectrum, and backhaul resources. In addition, we take the residual selfinterference of FD communications into account in the formulated problem.

Since the formulated problem is a mixed combinatorial and nonconvex optimization problem, its computational complexity is high. Moreover, the centralized scheme may suffer from signaling overhead, outdated dynamics information, and scalability issues. To solve it efficiently, we divide the original problem into two subproblems. For the first subproblem, we transfer it to a convex optimization problem, and then solve it by an efficient alternating direction method of multipliers[25] (ADMM)based distributed algorithm, in which the InPs and virtual resource manager (VRM) only need to solve their own problems without exchange of channel state information with fast convergence rate. The second subproblem is a convex problem, which can be solved by each InP.

Extensive simulations are conducted with different system configurations to verify the effectiveness of the proposed scheme. It’s shown that we can take the advantages of both wireless network virtualization and FD selfbackhauls with the proposed distributed virtual resource allocation algorithm. InPs, MVNOs, and users can benefit from the proposed resource allocation scheme.
The rest of paper is organized as follows. The proposed virtualized small cell networks architecture and the FD selfbackhaul mechanism are described in II. The resource allocation problem is formulated in III. Then we divide the formulated problem into two subproblems and the solution details are described in IV. Simulation results are discussed in V. Finally, we conclude this study in Section VI.
Ii System Model
In this section, we first describe the virtualized small cell network architecture. Then we present the FD selfbackhauling mechanism where the SBSs can transmit and receive data on the same spectrum simultaneously.
Iia Virtualized Small Cell Network Architecture
We present a virtualized small cell network architecture with multiple InPs and multiple MVNOs, as shown in Fig. 1. There are InPs offering wireless access services in a certain geographical area. Each InP deploys and manages a cellular network with one MBS and several selfbackhauled SBSs. There are MVNOs, which provide various services to their subscribers through the same substrate networks. Following the general frameworks of wireless network virtualization [5, 26], the virtualized small cell network architecture consists of three layers: the physical resource layer (PRL), the control and management layer (CML), and the MVNO layer. The PRL, including base stations (BSs), spectrum, power and backhauls from different InPs, is responsible for providing available physical resources. Moreover, the PRL also provides CML with the interfaces needed to control resources. The CML virtualizes the physical resources from different InPs and enables the sharing for MVNOs. Then, the CML manages and allocates the virtual resources to different MVNO users. The resource management functions in CML are realized by a virtual network controller and a virtual resource manager (VRM). The virtual network controller of MVNOs is responsible to collect the resource consumption prices negotiated with InPs, and the users’ information (e.g., payment information and QoS requirements) from MVNOs, then feedback the resource allocation results to MVNOs for the purpose of finishing the settlement between MVNOs and InPs. To maximize the total utility of all MVNOs, the VRM is responsible to dynamically allocate the virtual resources from multiple InPs to different MVNO users. Through the virtualization architecture above, each MVNO can have a virtual network composed of the substrate networks from multiple InPs. Hence each user can get services via different access points (either MBSs or SBSs) from different InPs.
We assume that the spectrum bandwidth of the th InP is . The transmit power of the MBS and the transmit power of the SBSs, are and , respectively. is used to represent the set of SBSs that belong to the th InP. Let be the th SBS in . For ease of presentation, we use to represent in this paper. The set of users of MVNO is denoted as . Then, let be the set of all the users and be the total number of users. For the users, there are two access choices: MBS or selfbackhauled SBS. Those users who are associated to the th MBS is denoted by . Similarly, those users who access to the th SBS in the th InP is denoted by . To facilitate the formulation of the virtual resource allocation problem, the selfbackhauling mechanism via inband FD communications will be introduced in the next subsection.
IiB Small Cell Selfbackhauling Mechanism Based on Full Duplex Communications
As shown in Fig. 2(a), SBSs are equipped with FD hardware, which enables them to backhaul data for themselves. In the downlink (DL), a SBS can receive data from the MBS, while simultaneously transmitting to its users on the same frequency band. In the uplink (UL), a SBS can receive data from the users, while simultaneously transmitting data to the MBS on the same frequency band. In this mechanism, the SBS can effectively backhaul itself, eliminating the need for a separate backhaul solution and a separate frequency band. Therefore, selfbackhauling can significantly reduce the cost and complexity of rolling out small cell networks. In order to distinguish DL from UL in access and backhaul transmissions, we call the relevant links as access UL, access DL, backhaul UL, and backhaul DL, respectively. Due to the limitation of selfinterference cancellation technologies, the backhaul DL and access UL will suffer some selfinterference from access DL and backhaul UL, respectively. Different from the FD relay mechanism in [20], the spectrum can be reused by different SBSs and the SBSs can allocate resource to their users flexibly in our selfbackhaul scheme. Compared to DL, UL usually has less traffic. If the transmission of DL is satisfied, the transmission of UL will also be satisfied. As a result, we focus on the transmission of DL in this paper.
In this paper, the orthogonal spectrum reuse pattern is adopted, where the spectrum is divided for the MBS and SBSs to avoid the interlayer interference [27]. As shown in Fig. 2(a), we divide the spectrum of InP into two parts: as for the MBS and as for the SBSs. In DL, the MBS transmits data to not only macro users on but also SBSs on . Similarly, the MBS receives the data from macro users on and SBSs on in UL. At the same time, SBSs transmit and receive data to their users on . Obviously, the spectrum indicator vector , which decides the throughput of backhaul and access link, plays an important role in achieving small cell selfbackhauls.
For user , the VRM will decide its association BS and allocate some resources to it from the BS. We denote by and the user’s association indicator and allocated resources ratio (Note that this resources mean time slot resources), respectively. When a user is associated with one BS, , otherwise . Further, means the user is associated to the th MBS, means the user is associated to . Similarly, denotes the resource ratio that the user gets from the th MBS, and denotes the resource ratio that the user gets from . Obviously, is related to . Only when , will be meaningful.
For the access DL, we denote by the achievable link rate of one user. Generally, and are logarithmic functions of signal to interference and noise ratio (SINR). For macro users, they do not suffer interference from other BSs because of the different spectrum bands among InPs and the OSR pattern between macro and SBSs, so the achievable link rate can be expressed as
(1) 
For small cell users, they suffer cochannel interference from other SBSs in the same InP, so the achievable link rate can be expressed as
(2) 
In our virtualized small cell networks, each user from any MVNO can access to either a MBS or a SBS in any InP. Hence, for a given user , we define as its overall longterm rate, which can be expressed as follows.
(3) 
where and are the overall longterm rates of user getting from the th MBS and , respectively. For the backhaul DL, the MBS transmits data to SBSs on , and the SBSs buffer these data to transmit to small cell users. We define the backhaul link rate of by . When the SBS receives data from the MBS, it transmits data to its users at the same time, which results in the selfinterference (SI). The value of SI is determined by selfinterference cancellation technologies, and is proportional to the transmission power of SBS DL, which could be expressed as . represents the residual selfinterference gain. Different selfinterference cancellation technologies may result in different . Any selfinterference cancellation technology can be applied at the SBSs, and the analysis in this paper is a general case. In addition, a SBS also suffers cochannel interference from other SBSs because they use the same spectrum , so the achievable link rate can be expressed as
(4) 
In each InP, SBSs share the spectrum , but the spectrum resource must be divided in time domain or frequency domain (TDD or FDD) to avoid interference. If FDD is applied, the SBS will have less spectrum receiving data than that transmitting data, which increases the difficulty to investigate SI. As a result, TDD model is adopted in this paper, as shown in Fig. 2(b). We use to denote the time slot ratio occupied by SBS . If we denote by the overall longterm rate of backhaul DL from the th MBS to , it can be expressed as .
In (1), (2), and (4), denotes the noise power level, denotes the channel gain between one user and macro (small) cell BS and denotes the channel gain between MBS and SBS. In general, the channel gain includes path loss, shadowing and antenna gain. The association is assumed to be carried out in a large time scale compared to the dynamics of channels. The SINR for association is averaged over the association time, and thus it is a constant regardless of the dynamics of channels (i.e., fast fading is averaged out). As for resource allocation, we assume that resource allocation is carried out well during the channel coherence time, and thus the channel can be regarded as static during each resource allocation period. This model is applicable for low mobility environment.
The notations used in this paper are presented in Table I.
Notation  Definition 

set of users  
set of SBSs  
spectrum allocation indicator vector  
the number of InPs  
the number of MVNOs  
index of MBSs  
index of MVNOs  
index of SBSs  
transmit power spectrum density of macro BS of InP  
transmit power spectrum density of SBSs of InP  
the bandwidth of the th InP’ spectrum  
channel gain between to the th macro BS  
channel gain between user to the th SBS in InP  
achievable link rate between user to BSs of InP  
achievable backhaul link rate of the th SBS in InP  
user cell association indicator,  
user resource allocation indicator  
SBS resource allocation indicator  
rate of user getting from BSs of th InP  
sum rate of user getting from all BSs of th InP  
rate of the backhaul link of the th SBS in InP 
Iii Problem formulation
In this section, we formulate the virtual resource allocation problem in virtualized small cell networks with FD selfbackhauls. Firstly, we present the utility functions for users, MVNOs, and InPs. Then, the problem of virtual resource allocation is formulated to maximize the total utility of all MVNOs in a centralized manner at the VRM.
Iiia Utility Function Definition
In our virtualized small cell networks, users pay to their MVNO for getting services. Hence, for a given user , the utility function can be defined as the difference between her/his service rate and the money she/he paid, which is expressed as
(5) 
where means the profit of users for per bit rate. For simplicity, we assume MVNOs purchase resources (including spectrum,time slot, power and backhaul) from InPs to provide services to their users. The responsibility of the VRM is to efficiently allocate the virtual resources to maximize the total utility of all MVNOs. In this paper, we consider a longterm rateaware utility function for MVNOs, which is defined as the difference between the total income of all MVNOs earned by serving the users and the total resource consumption cost. For the th MVNO, its utility can be expressed as:
(6) 
The first term represents the income of the MVNO from providing services to users, and represents the money user has paid to its subscribed MVNO. The second term defines the cost of MVNO for using the spectrum and power resource, and represents the deliberated price between the th InP and MVNOs. can be expressed as
(7) 
where bandwidthpower product is used to quantify the consumed wireless resources of the access cell [28], coefficient specifies the weight of small cell resources with respect to the MBS resources. As is well known, the load unbalance problem is a serious problem in small cell networks, which leads to low utilization of SBSs[29],[22]. As a result, we choose to attract more users to SBSs. This method was also used in [30]. The last term represents the cost of MVNO for using the backhaul resource of InPs, which depends on the amount of backhaul data of MVNO and the type of backhaul technique. It’s obvious that the amount of backhaul data for the th MVNO in can be expressed as . We define the price of different backhaul techniques for backhaul one bit by the price function , where represents the backhaul technique type of the th InP (e.g., optical, micowave, DSL, cell communication backhaul). Thus, the cost of MVNO for using backhaul resources can be expressed as
(8) 
Actually, the FD selfbackhaul just utilizes part of MBS power on SBS access spectrum to achieve backhaul rather than additional spectrum or infrastructure, which is cheaper than traditional backhaul methods. Mathematically, of our proposed FD selfbackhaul is expressed as .
As mentioned earlier, the responsibility of the VRM is to efficiently allocate the virtual resources for the purpose of maximizing the total utility of all MVNOs. In order to guarantee the fairness during the resource allocation process, we change the utility of MVNO as follows by adopting the method in [29],
(9) 
where function is defined as
(10) 
Then the utility function of VRM will be changed as
(11) 
For an InP, it not only leases its spectrum and power resource to MVNOs and earn money from them, but also rents or deploys infrastructure to backhaul data. So the utility function of the th InP can be expressed as
(12) 
where means the cost of renting or deploying backhaul infrastructure. For simplicity, we assume that all the backhaul income from MVNOs is used to pay for the rent or deployment of backhaul infrastructure in nonselfbackhauled small cell networks, which means .
IiiB Virtual Resource Allocation Problem Formulation
In each resource allocation cycle, the VRM needs to dynamically allocate the virtual resources, including MBSs, SBSs, and spectrum, to MVNOs. The objective of this paper is to develop a slotbyslot virtual resource allocation algorithm that maximizes the total utility in (IIIA) under the following constraints to determine , , and .
Firstly, in our proposed small cell selfbackhaul mechanism, the DL data of small cell users are transmitted from MBS to SBSs firstly, and then the SBSs store and forward it to their users. In this process, the throughput of backhaul DL decides the throughput of access DL. Hence, we have the throughput constraint of SBSs
(13) 
Secondly, for simplicity, each user can only be served either by one MBS or by one SBS. Therefore, the cell association indicator should also satisfy the following constraints.
(14)  
(15) 
Thirdly, one BS schedules its associated users on time dimension and represents the schedule ratio of user , the SBSs in the same InP share the frequency on time dimension to access and backhaul. defines the schedule ratio of SBS to backhaul. So, the following condition must be satisfied.
(16)  
(17)  
(18)  
(19) 
Furthermore, the spectrum allocation indicator will take a value in the interval of . Mathematically, it can be expressed as follows.
(20) 
Then, based on the above constraints, the problem of resource allocation can be formulated as follows
(21)  
Iv Distributed Virtual Resource Allocation Algorithms
It can be observed that the considered problem is combinatorial and nonconvexity. The combinatorial nature comes from the integer constraint . The nonconvexity is caused by the objective function and constraint . As a result, a brute force approach can be used to obtain the optimal virtual resource allocation policy. However, such a method is computationally infeasible for a large system and does not provide useful system design insight. To reduce the computational complexity, firstly, we assume the spectrum allocation indicator vector is fixed, and then we convert the original problem into a convex problem by variable transformation and perspective function theory to solve , and . Secondly, based on the obtained results, we can prove that the problem is convex problem about , then it’s easy to get the optimized by some convex optimization algorithms. Furthermore, we come back to first step with the result of to get the new values of , , and . By iterations like this for a number of times, the values of , , and will converge, which can be seen as the solution of original problem . In Subsection IVA, we solve , and by assuming is given. Subsection IVB will introduce how to solve when , and are fixed. In Subsection IVC, we describe the whole process of solution and the convergence of the proposed algorithms.
Iva Solving , and under Fixed Spectrum Allocation Indicators
For any given spectrum allocation indicator vector , the original problem reduces to the following problem:
(22)  
Theorem 1: If the problem is feasible, the objective function achieves the optimal solution only when the constraint is tight, which means must hold when the objective function gets the maximum value.
Proof: We will prove Theorem 1 by a simple contradiction statement. Assume that the optimal resource allocation scheme (e.g., , , ) for problem is obtained when . Now all SBSs can achieve backhaul data well because the backhaul rate is higher than the access rate. However, there must exist a such that . According to the definition of , it’s obvious that . That’s to say, a larger utility for the VRM can be obtained at , which also satisfies other constraints. Therefore, there is a contradiction between this conclusion and our assumption. In other words, the optimal cell association and resource allocation scheme is not obtained when . As a result, Theorem 1 is proved.
Based on Theorem 1, we can get . Namely, we can replace in objective function by and , which means that the original problem will be transformed from a three variables problem into a two variables problem as following:
(23)  
Although problem is simplified based on Theorem 1, the above problem is still difficult to solve based on the following observations:

The feasible set of is nonconvex as a result of the binary variables .

The objective function is not convex due to the product relationship between and convex function of .
As a result, we first transfer this problem into a convex problem and solve it via a distributed ADMMbased algorithm.
IvA1 Transferring into convex problem
As is well known, a mixed discrete and nonconvex optimization problem is expected to be very challenging to find its global optimum. Thus, we have to simplify problem . Following the approach in [29], we relax ( is included) in C2 and C3 to be real value variables such that . The relaxed can be interpreted as the time sharing factor that represents the ratio of time when user associates to the th MBS or SBS . However, even after relaxing the variables, the problem is still nonconvex due to the nonconvex objective function. Thus, to make the problem tractable and solvable, a second step is necessary. Next, we give a proposition of the equivalent problem of .
Proposition: If we define and for , there exists an equivalent formulation of as shown in (IVA1).
(24)  
s.t.  
The relaxed problem can be recovered by substitution of variable into problem except . Due to the loss of definition when , it is not a onetoone mapping. However, certainly holds because of the optimality. Obviously, BS does not allocate any resource to any user if the user does not associate with the BS. Thus, it becomes a onetoone mapping when the completed mapping between and is defined as
(24) 
Holding the Proposition and wellknown perspective function in convex optimization theory [31], we have the following theory that gives the convexity of .
Theorem 2: If problem is feasible, it is jointly convex with respect to all optimization variables and .
Proof: Since the constraints are linear, they are obviously convex. Due to the fact that objective function is a linear sum of , and , we just need to prove the convexity of , and , based on the following principle: the linear sum of convex functions is still convex[31].
The convexity proof of is similar to [32], which will be described briefly as follows. Firstly, we prove the continuity of function at the point of . Let , then . Function is the wellknown perspective operation of logarithmic function, and the perspective function of a convex function is also a convex function based on [31]. Since is th perspective function of , and the function is convex about , is convex. What’s more, function is a weighted sum of series of convex function , so it is also a convex function. Function is linear function of , and the convexity of it is obvious. Since the function is a sum of quadratic function about , we can easily know it is a concave function by solving the second derivative of . With the theory that a negative concave problem is a convex problem, we can conclude that the form of the objective function in is a linear sum of convex problems. With the addition that all the constraints are convex, the convexity of the is proved.
Since problem is a convex problem, a lot of methods (e.g., interior point method) can be used to solve it. However, with the increase of the number of BSs and users, the size of problem will be very large. Practically, even with a powerful computing center, the overhead of deliver enough local information (e.g., channel status information (CSI)) to the global center is extremely inefficient. Therefore, for the purpose of implementation, a distributed algorithm running on each InP should be adopted.
IvA2 ADMMbased solution algorithm of
Due to constraint in , the problem is not separable with respect to different InPs. To apply ADMM to the resource allocation problem , this coupling must be handled appropriately. Therefore, we introduce local copy of the related global cell association variable for the th InP. Roughly speaking, each local variable can be interpreted as the InP’s opinion about the corresponding global variable. Naturally, variables is also local variables for InP , since the InP operates without any limits from other InP. For the sake of brevity, let us define vector to represent the local variables of the th InP. Inspired by [31], to deal with the constraints, we introduce an indicator function such that when ; otherwise, , where represents the feasible set of problem . With these notations above, problem of maximizing on set is equivalent to
(25)  
s.t. 
where
(26) 
It can be seen that the objective function is separable across InPs in the virtualized small networks. However, the global cell association variables involved in the consensus constraints couple the problem with respect to the InPs. Therefore, the basic idea to solve problem is that each InP only determines its local variables based on the local information, and VRM is responsible for achieving consensus between the local variables and the global variable according to the consensus constraint.
We apply ADMM [33] for solving the problem in in a distributed way. The machinery of the ADMM applied to is initiated by forming an augmented Lagrangian with respect to the consensus constraints. The augmented Lagrangian not only includes a set of consensus constraints weighted by Lagrange multipliers (conventional Lagrangian), but it also involves an additional regularized quadratic term: a squared norm with respect to the consensus constraints.
Let be the Lagrange multipliers set associated with consensus constraints of . Thus, the augmented Lagrangian for can be written as
(27) 
where is a positive constant parameter for adjusting the convergence speed of the ADMM. Note that due to the structured interconnections in the virtualized small cell networks, the augmented Lagrangian is separable with respect to the InPs.
Fundamentally, the augmentation can be seen as a penalty term added to the primal objective function [25]. The regularization term facilitates the algorithm to drive the local and the related global variables into consensus. It’s worth emphasizing that the solution of the original optimization problem is not affected by adding the quadratic regularization term to the Lagrangian since it vanishes for any set of primal feasible variables. The ADMM method consists of sequential optimization phases over the primal variables followed by the method of multipliers update for the dual variables [25]. By applying the ADMM to the problem in , we first minimize the augmented Lagrangian in (IVA2) over the local variables, then over the global variables, and finally, perform the dual variable update. Thus, the ADMM method consists of the following steps:
(28) 
(29) 
(30) 
where stands for the iteration index of the ADMM algorithm. The basic idea behind the ADMM iterations is that we first minimize the augmented Lagrangian with respect to local variables in update, and then the VRM calculates the global variable based on all local variables, finally update the Lagrange multipliers. In the following, we will deal with how to update , and in each iteration. Moreover, the distributed implementation of each update will be also discussed.
a) Update: Interestingly, as mentioned earlier, it can be found that update is separable across each InP. Therefore, update can be decomposed into subproblems, which can be solved locally at each InP. After dropping off the constant terms in (IVA2) which do not affect the solution, the subproblem to be solved at InP can be given as
(31)  
Obviously, problem (IVA2) is a convex problem because of the convexity of problem . For the purpose of fast convergence, steepest descent method [31] will be adopted at each InP to realize update.
Recall that we have relaxed the cell association indicator to be a real value between zero and one instead of a Boolean at the beginning of this subsection, hence we have to recover it to a Boolean after we get the optimal solution of subproblem (IVA2). Assume , (including ) and are the optimal solution of (IVA2) obtained directly from the steepest descent method. Inspired by [34], we first compute the marginal benefit for each as follows. , where is the objective function of problem (IVA2). Then, the indicator can be recovered to zero or one by
(32) 
After recovering the indicator , we resolve the problem (IVA2) to get the optimal solution of according to the known recovered