Green Wireless Sensor Networks with Wireless Power Transfer
Abstract
An energy cooperation policy for energy harvesting wireless sensor networks (WSNs) with wireless power transfer is proposed in this paper to balance the energy at each sensor node and increase the total energy utilization ratio of the whole WSNs. Considering the unbalanced spatiotemporal properties of the energy supply across the deployment terrain of energy harvesting WSNs and the dynamic traffic load at each sensor node, the energy cooperation problem among sensor nodes is decomposed into two steps: the local energy storage at each sensor node based on its traffic load to meet its own needs; within the energy storage procedure sensor nodes with excess energy transmit a part of their energy to nodes with energy shortage through the energy trading. Inventory theory and game theory are respectively applied to solving the local energy storage problem at each sensor node and the energy trading problem among multiple sensor nodes. Numerical results show that compared with the static energy cooperation method without energy trading, the Stackelberg Model based Game we design in this paper can significantly improve the trading volume of energy thereby increasing the utilization ratio of the harvested energy which is unevenly distributed in the WSNs.
1 Introduction
Recently, energy efficiency has become a hot research topic in wireless networks [1, 2, 3, 4, 5, 6, 7]. Particularly, energy management is always important for wireless sensor networks (WSNs) due to the limited battery capacity of senor node. A viable approach for sustainably powering the WSNs is to harvest energy from the environment, such as solar, vibrations, thermal, etc. [8] [9]. For the energy harvesting WSNs, the difference between the available environment power and the power consumption through the network is the key challenge. To use the harvested energy efficiently, the energy aware task allotment, (i.e., the task distribution among nodes is adapted to the detailed characteristics of environmental energy availability) has been researched extensively in recent years [10] [11]. However, with the emerging research hot spot of wireless power transfer (WPT) [12] [13] and simultaneously wireless information and power transfer (SWIPT) [14][18], the energy management problem of energy harvesting WSNs should be rethought from the perspective of energy cooperation.
Energy cooperation method among wireless network nodes powered by renewable energy is studied in [19], where the authors determine energy management policies that maximize the system throughput within a given duration using a Lagrangian formulation and the resulting KarushKuhnTucker (KKT) optimality conditions. In [20], base stations in coordinated multipoint enabled cellular networks are equipped with energy harvesting devices to provide renewable energy and employ smart meters and aggregator to enable both twoway information and energy flows with the smart grid. The renewable energy cooperation among base stations is formulated as a convex optimization problem. Different from the energy cooperation problem in renewable energy powered cellular networks and wireless local area networks in which the aim of energy cooperation is usually optimize the network throughput or increase the performance of the total network and the energy cooperation is usually a optimization problem, the energy cooperation in WSNs is a equilibrium problem because information from each node is equally important for us to surveillance environment. Considering the spatiotemporal properties of the energy supply across the deployment terrain of energy harvesting WSNs and the dynamic traffic load at each sensor node, the aim of energy cooperation among multiple sensor nodes is to balance the energy and traffic at each sensor node and to improve the overall energy utilization ratio of the WSNs.
In this paper we propose the energy cooperation policy for energy harvesting WSNs with WPT, through the local energy storage at each node and energy trade among multiple nodes. A combination of inventory theory and game theory is applied to energy cooperation among sensor nodes for the first time to guarantee the energy supply at each sensor node and simultaneously increase the total energy utilization ratio. The structure of this paper is as follows. Section 2 presents the system model which includes the network model, and the energy supply and demand model at each sensor node. Section 3 formulates the local storage of energy at each sensor node as an inventory problem, and formulates the energy cooperation problem among nodes as a game model. Section 4 solves the inventory problem using inventory theory and designs the Cournot Model based Game and Stackelberg Model based Game to solve the energy trading problem among multiple sensor nodes. Section 5 describes the algorithm flow of the proposed games in detail through a case study and discusses the numerical simulation results. Finally, section 6 concludes the works of this paper.
2 System model
In this section, we present the system model which includes network model, and the energy supply and demand model of each sensor node.
2.1 Network Model
Each sensor node equipped with energy harvesting devices collects ambient energy, such as solar power, microbial fuel cells, vibrations and acoustic noise, converses the energy into electrical form, and stores the energy into a rechargeable battery. As shown in Figure 1, energy cooperation among geographically distributed nodes is implemented through the wireless power transfer from one node to the other. Sensor node in the network is either a renewable energy supplier or an energy demander. For simplicity, mobility [22] and handover [21, 23] are not considered in this paper. Each sensor node helps point to point packet delivery through routing packets from its neighbor nodes[24][26]. The radio channel which characterizes the propagation of radio signals among the nodes is modeled as the bandlimited Gaussian channel with power constraint [27, 29].
2.2 Energy Supply: Cooperation based Local Storage
Energy stored at node (denoted by ) and time is, . is in general a stochastic process, consisting of its own harvesting energy denoted by , and cooperative energy transferred from other node , , denoted by . The value of can be positive, representing an energy deficit at , or be negative, representing a surplus of energy at .
For , the amount of and is realtimely monitored and recorded.
2.3 Energy Demand: Data Packets Arrival and Energy Consumption Model
The number of data packets arriving at each sensor node has been dynamically changing not only in the time domain but also in the spatial domain. Data packets, which we call traffic load interchangeably in this paper, arriving at one node for a long period of time has great uncertainty, however, during a certain period of time, the arrival process of traffic load can be regarded as Poisson process. Assuming that the arrival rate of data packet at during time period is ( ), the number of data packet arriving during is a random variable that subjects to Poisson distribution with parameter which means the expected value of traffic load which we call traffic quantity. The probability distribution function (p.d.f.) of is expressed as:
The energy demand of node at time : , is also Poisson process, since the energy consumed at each node is primarily used to transmit data packets. For node , when the number of data packet arriving during is , the energy demand, is a random variable also obeying Poisson distribution. Here, represents the average energy consumption of per unit data packet. Denote the probability of as then we get,
(1) 
When the arriving rate of data packet changes along with time, the p.d.f. of and changes along with accordingly.
For each node in the WSN, the energy consumed for data packets transmission is determined by two factors which include data rate and link quality. Consider a bandlimited Gaussian channel model with power constraint. The achievable data rate at is as follows
(2) 
where denotes the power spectral density of the additive white Gaussian noise, and denote the transmit power and bandwidth from one sensor node to node respectively. To reach a threshold level of data rate at a sensor node with a specific link quality, the transmit power needed is,
(3) 
The terms energy and power are interchangeably used in this paper, since the the transmitting power at each sensor node is the main part of energy consumption.
3 Problem Formulation
Through analyzing the distinctive attribute of the energy cooperation among nodes in WSNs, we formulate the local energy storage procedure as an inventory problem and formulate the energy cooperation process among nodes as a game theory problem.
3.1 Traffic Aware Energy Inventory
Inventory theory is a branch of operation research [28]. A main reason to develop inventory theory is that, for the supply side it is rarely possible to predict the demand exactly. Inventory serves as a buffer against the uncertain and fluctuation of the demand and keeps a supply of items available in case the item are needed by its customers. An inventory problem consists of four basic elements: demand, supply, inventory strategy and inventory cost. The inventory cost generally consists of four components: setup cost , holding cost , shortage cost and purchasing cost , i.e., .
The goal of inventory control is determining the optimal inventory amount at an appropriate time point to satisfy the demand by using a specific inventory strategy at the lowest inventory cost . There will be two different results of inventory amount for an inventory strategy:

: The storage amount is less than demand. The shortage cost need to pay is . Here, represents unit shortage cost.

: The storage amount is more than demand. The holding cost is . Here, represents unit holding cost.
At each sensor node, traffic load leads to the demand for energy and the harvested energy acts as the supply. Each node at WSNs respectively determines the optimal amount of the harvested energy reserved for its own traffic load. Inventory strategy for the harvested energy at each node is a bridge that connects traffic load with energy supply. After briefly introducing the inventory theory, we obtain the optimal distributed management of the harvested energy unbalancedly distributed in WSNs through adopting an appropriate inventory strategy to find the optimal inventory amount of the item that is needed, i.e., the harvested energy. As to inventory cost, holding cost and shortage cost is the most important part we consider, since the setup cost and purchasing cost of the ambient energy are negligible. In this paper, holding cost represents the limited capacity of battery that the higher capacity means the higher cost and shortage cost represents expenses due to the deficient storage of energy, such as the loss of packets. When demand is a stochastic variable and the p.d.f. of is known, denoted as , in the case of only considering holding cost and shortage cost, the expected value of inventory cost can be expressed as:
(4) 
The optimal inventory amount of renewable is a value of , to make the inventory cost minimum.
3.2 Energy Cooperation: Energy Trading Method between Energy Supplier and Demander
After calculating the optimal amount of the harvested energy that should be stored in advance through the inventory process at each node, all the sensor nodes are divided into two categories: energy suppliers, i.e., the energy stored in battery is more than demand, and energy demanders, i.e. the energy stored in battery cannot meet the demand. Each energy demander applies for a certain amount of energy from energy suppliers, and the suppliers want to acquire an income from selling their extra energy. The supplier also needs to consider its selling cost resulting from that the energy sold to the demander will not available to the supplier although the supplier need more energy later. Hence, confronting an energy demander with a certain amount of energy demand, each supplier should adopt proper strategies to determine the appropriate amount of energy to sell, and all suppliers get into a relationship of checks and balances to determine the optimal amount to give and the optimal price to take. It forms the game among all energy suppliers.
When it comes to game theory, games are characterized by a number of players or decision makers who interact, possibly threaten each other and form coalitions, take actions under uncertain conditions, and finally receive some benefit or reward or possibly some punishment or monetary loss [30]. The strategic form of a game is typically defined by these three objects:

the set, , of players,

the sequence, , of strategy sets of the players, and

the sequence, , of realvalued payoff functions of the players.
The game is denoted as . Each energy supplier node , , acts as a player, and all energy supplier nodes construct a game. Energy supplier choose a strategy based on its own situation and other suppliers’ decision, to maximize its own payoff function . Relative to supplier , other suppliers respectively choose their best strategies, we call the game reach the Nash Equilibrium, if the payoff function satisfies the following equation:
(5) 
In other words, each player in the game cannot achieve a better payoff only through changing the strategy itself. In this paper, game among all energy suppliers is one kind of duopoly game with complete information (i.e. for each energy supplier, the history decisions of other suppliers are known). Through finding the Nash Equilibrium solution, the ultimate aim of the game theoretic approach of energy cooperation among all sensor nodes is to improve the selling volume of the harvested energy thereby to improve the utilization ratio of the harvested energy distributed in the whole WSN.
4 Problem Solution
In this section, we firstly present the inventory control policy where and are levels of inventory quantity, and deduce the solution method of the two parameters. After the inventory calculation, we design the game theoretic energy cooperation mode based on the Cournot Model of Duopoly and Stackelberg Model of Duopoly.
4.1 Energy Inventory
Assume the existing storage amount of the harvested energy of node at time is . In the inventory control policy , an order is placed to increase the item’s inventory amount to the level as soon as this inventory amount reaches or drops below the level [31]. In this paper we name inventory bottom line and optimal Inventory amount. Inventory process proceeds from one time period to the next, and this cycle repeats. When demand is a stochastic variable changing along with time, parameters and also change along with time, denoted as . Each node at WSN respectively implements policy then obtains the optimal amount of energy reserved for its total traffic load.
4.1.1 Optimal Inventory Amount
As analyzing in section II, energy demand during time period , is a random variable obeying Poisson distribution that the p.d.f is expressed in Equation(1). The quantity of data packet arriving during takes the discrete value, so that energy demand also takes discrete value. For simplicity, the value of parameters and range in . When equals , denote as , . There are three cases of the relationship between and :

: The amount of energy reserved in the battery is deficient. The energy cannot satisfy the demand and sensor node has so limited energy to route packets that results in packets loss and other network problems, which we call energy shortage cost.

: It is the optimal inventory amount. Energy reserved in the battery exactly satisfies the demand.

: Energy reserved in the battery exceeds demand and the redundant energy needs additional battery capacity which leads to holding cost.
At each inventory period, on the basis of Equation(4) we deduce the expected value of inventory cost for the inventory amount :
(6) 
where is expressed in Equation(1) and is the unit purchase cost. Take Equation(1) into Equation(6), we get the expected inventory cost for every specific value of .
(7) 
When , we obtain the following derivation,
(8) 
Denote as , and as . Generally, has a value between 0 and 1 . Now Equation(8) is rewritten as:
(9) 
where , and monotonically increases along with . The plus or minus characteristic of is the same with that of so that is also monotonically increasing.
The p.d.f. of analyzed in section II shows that the probability of one data packet or data packets arriving during time period , i.e., and , are extremely low. Set , due to the very small value of , then take . Similarly, take , i.e., . Now we get these following equations:
We have known that is monotonically increasing along with , and now we conclude that the value of increases from a minus value to a plus one. Accordingly, the value of firstly increase and then decrease. Therefore, a value is subsistent to make minimum. Here, is a value in in . The value definitely makes these following relational expressions simultaneously valid:

, according to Equation(10): , i.e.,.

, according to Equation(10): , i.e.,.
The optimal inventory amount that makes the expected value of inventory cost minimum can be obtained from the following equation:
(10) 
Two extreme situations:

, i.e., . In this case, for an arbitrary value of the relational expression: is always established, that is . Additionally, the relational expression: demonstrates that the value of is very large. In other words, the probability of a very less demand for energy is very high, and this corresponds to the situation of very low traffic load at a sensor node.

, i.e., . In this case, for an arbitrary value of the relational expression: is always established, that is . Additionally, the relational expressions and demonstrate that the value of is very large. That is to say, the probability of a great demand for energy is very high, and this corresponds to the situation of very high traffic load at a sensor node.
4.1.2 Inventory Bottom Line
The optimal inventory amount has been derived in previous subsection. When the existing storage amount reaches the level , the expected value of cost resulting from not increasing inventory should be less than that of increasing inventory to , expressed by the relational expression:
i.e.,
(11) 
Equation(11) is obviously valid, when , so that there must be at least one value of to make Equation(11) established. Since the inventory item for each sensor node at WSN is the harvested energy which is harvested freely at the sensor node itself, the setup cost and unit purchasing cost are negligible. Let and in Equation(11) equal zero, we find that in the scenario described in this paper, the order point is exactly the optimal inventory amount . That is to say, for each sensor node, an order is placed to increase the inventory amount of energy to the level if the inventory amount drops below the level at each set time period.
4.2 Energy Trading Method: A Game Theoretical Approach
We first discuss the existence of the Nash Equilibrium solution in more general games, more details can be found in literature [30] and its references.
Theorem 1.
(Nash 1950): In the nplayer normalform game , if is finite and is finite for every then there exists at least one Nash equilibrium, possibly involving mixed strategies.
A more universal method to determine the existence of Nash equilibrium solution [30] is to verify whether the game process meets the following conditions:

the number of player is finite;

the strategy set (i.e., action set) is bounded closed and convex set.

the payoff function in the action set is continuous and quasi concave.
The first two conditions are easy to be satisfied. To achieve the Nash equilibrium solution, the key point is designing appropriate payoff function for players. In what follows, strictly according to the requirements stated above and the network feature of WSN, we design the payoff function for both energy supplier and demander node at the energy harvesting WSN. The games proposed in this paper is based on the classical Cournot Model of Duopoly which is Static game with complete information, and the classical Stackelberg Model of Duopoly which is Dynamic Games with Complete Information.
When a sensor node (which is an energy demander) requests for a certain amount of energy, the energy supplier nodes execute the game theory process and provide an appropriate amount of energy. If the amount of energy each supplier determines to sell stays the same for three times in succession of the games, it indicates achieving the Nash Equilibrium solution. An energy cooperation process consists of the following steps:

Sensor node which is energy demander makes a request for buying energy and broadcast this request to energy suppliers.

Energy supplier nodes receive the request and each energy supplier get into the decision making state to determine the amount of energy to sell through a game theoretic approach.

Stop the game process when the amount and price of energy provided by energy suppliers reach the steady value, i.e., achieve the Nash Equilibrium solution.

Each energy supplier transmit the energy with the amount determined by step 2 to the energy demander node.

The energy demander node receive the energy transmitted from energy suppliers.
4.2.1 Cournot Model based Game
Cournot Model of Duopoly is a kind of static game with complete information. The form of Cournot Model of Duopoly is as follows: first the players simultaneously choose actions; then the players receive payoffs that depend on the combination of actions just chosen. At each move in the game the player with the move knows the full history of the play of the game thus far, and each player’s payoff function is common knowledge among all the players. Considering the particular attributes of the energy harvesting WSN, we design the game among energy suppliers based on the Cournot Model. The payoff function of energy supplier node is,
(12) 
where is the amount of energy sold from energy supplier node , is the price of selling the energy, and is the cost function of selling amount energy.
The price of energy is determined by the total amount of energy sold by all the suppliers. The precise relationship between the total sellthrough and the price of energy can be deduced from the payoff function of the energy demander. Referring to literature [32], we design the payoff function,
(13) 
where is the energy efficiency of demander, defining as the ratio of the data rate of the energy demander node to the power used for transmitting these packets. The energy efficiency can be deduced from Equation (3):
(14) 
Equation (13) is a quadratic and quasi concave function, and there must be a maximum value. Taking the derivative of the amount of power in Equation (13), we achieve the maximum value of payoff of the demander and the relationship between energy price and amount.
(15) 
We get the price of energy,
(16) 
Take Equation (14) into (16), we achieve the relationship between energy price and the sales,
(17) 
Then we propose the selling cost function of the energy supplier. If a supplier decides to sell a certain amount of energy to a demander, the sold energy can not been taken back, although the supplier faces serious shortages of energy later. In other words, suppliers take risks to sell their energy. Based on the impact of the quality of service of the supplier node resulting from selling energy, we design the selling cost function .
(18) 
where is the traffic quantity of data packets, is the amount of energy needed, is the amount of harvested energy stored at , is the energy utilization ratio of energy supplier node, , is the weight of cost function. The higher value of means a longer distance between supplier and demander, and that means a greater cost causing by power transmit loss.
Take Equation (17) and (18) into (12), we achieve the payoff function of the energy supplier node, as follows.
(19) 
The game process: Each energy supplier sells energy with the same quality and price. The price fluctuates with the demand. The action of each supplier is selfish an uncooperative, and the game among suppliers is the determination of selling volume of energy. According to other suppliers history data of selling volume, each supplier determines the most suitable selling volume. The suppliers simultaneously choose actions to determine their own selling volume. Then the suppliers receive payoffs that depend on the combination of actions just chosen. After several times games, all energy suppliers get into the balanced state and each supplier achieves the stable equilibrium value of the selling volume of energy. In section 5, through a case study the game algorithm will be introduced in detail.
4.2.2 Stackelberg Model based Game
Stackelberg Model of Duopoly is a kind of dynamic games with complete information. The payoff function of energy supplier and demander, and wireless channel model of the Stackelberg Model are the same with that of Cournot Model based Game we analyzed above. The difference is the game process: In the Stackelberg Model based Game, a certain number of suppliers move first and another part of suppliers move second. The detailed algorithm will be presented through a case study in section 5.
5 case study and simulation
In this section, we first set the parameters values for our system model and state the game model we proposed in section 4.2 in detail. Then we provide numerical results for evaluating the performance of the energy cooperation policy we proposed based on the game theoretic framework.
5.1 Case Study
At sensor node , data packet arrives during time period at rate . The expected value of traffic load that we call traffic quantity is respectively set to 5, 10, 20. One data packet corresponds to one unit energy 1. Neglect setup cost and purchasing cost, for that the harvested energy is free. Two cases are taken into consideration. In the case which the holding cost is lower than shortage cost, we set the following values: unit holding cost ; unit shortage cost ; the existing storage amount units of energy. In the case which the holding cost is higher than shortage cost, we set the following values: unit holding cost ; unit shortage cost ; the existing storage amount units of energy.
The Crossbow Berkeley motes are one of the most versatile wireless sensor network devices on the market for prototyping purposes. In this paper, we take the correlated parameters of Berkeley motes as standard to set the parameters values of our model. The operating frequency of Berkeley motes are in ISM (Industrial Scientific Medical) band, either 916.5 MHz or 433 MHz, with a data rate of 40 kilobits per seconds, and having a range of 30 feet to 100 feet. Set the threshold value of data rate of each sensor node as 40k bits, bits, the bandwidth as 10 MHz, Hz, and the the power spectral density of the additive white Gaussian noise as 50 dBm, dBm. Set the weight of the selling cost function as , i.e., the transmit loss of the wireless power transfer is percent. , , . Take these parameter values into the Cournot Model based Game and the Stackelberg Model based Game, and the algorithm flow of game processes are represented in Algorithm 1 and Algorithm 2 respectively. The completely static energy cooperation method in which all the suppliers provide the same amount of energy is presented in Algorithm 3 as a comparison.
5.2 Numerical Simulation and Analysis
5.2.1 Optimal inventory amount of energy at each sensor node
Figure 2 shows the expected value of inventory cost with a higher holding cost of the harvested energy. The optimal inventory quantity of energy makes the expected value of inventory cost at a minimum level. For sensor nodes with different traffic quantity during a time period, their optimal inventory amount of energy and the minimum inventory cost are correspondingly different. As shown in the figure, when the traffic quantity is , and units, the optimal inventory quantity of energy is , and units respectively, with the minimum inventory cost , and .
When the holding cost is lower, the optimal inventory amount of harvested energy is shown in Figure 3. It is observed that the optimal inventory quantity of energy is , and units respectively, with the minimum inventory cost , and , when the traffic quantity is , and units. Compared with Figure 2, we can see that, with an equal quantity of traffic, lower holding cost means that the optimal inventory amount of harvested energy is higher, and meanwhile the inventory cost is lower. It illustrates that, for each sensor node, the cooperation based inventory amount of harvested energy depends on the storage capacity and cost, and the traffic quantity.
5.2.2 Game theoretical approach to energy cooperation among sensor nodes
The process of achieving the Nash equilibrium solution of the Cournot Model based Game is shown in Figure 4. Four energy suppliers are in this game, and the history selling volume of energy of each supplier is set as , , and . As shown in the figure, after times game, the equilibrium is achieved at the sixth time. The process of obtaining the Nash Equilibrium solution of the Stackelberg Model based Game is shown in Figure 5. Six energy suppliers are in the game with three first moving suppliers and three second moving suppliers. The history decision is set as , , , , and respectively. After 5 times game, the three first moving suppliers come to an equilibrium value and the second moving suppliers come to the other equilibrium value.
The total selling volume through different cooperation method is shown in Figure 6. The Stackelberg Model based Game 1 is the case: , i.e., there are one supplier first moving to choose action and the number of the second moving supplier increases from to . The Stackelberg Model based Game 2 is the case: , i.e., the number of the first moving supplier increases from to , and there is one supplier second moving to choose action. As we see from Figure 6, with the same number of energy suppliers to sell their excess energy, the total selling volume of energy through the Static Energy Cooperation Method is far less than that though the game theoretic approach. It illustrates that, energy cooperation through the game theoretic approach can highly improve the utilization ratio of the harvested energy distributed in the energy harvesting WSN. In addition, the more sensor nodes in the game to supply energy the more surplus harvested energy will be sold to the energy insufficient node. The total selling volume of the Stackelberg Model based Game and Cournot Model based Game is similar, however the performance is still different. Zooming up Figure 6 and freely choosing a part, take , suppliers and , suppliers as examples shown in Figure 7. The Stackelberg Model based Game sells more energy than the Cournot Model based Game. It means that in the aspect of increasing the utilization ratio of the harvested energy, the dynamic game we propose is better than the static game. More specialized, the Stackelberg Model based Game 2 sells more energy than the Stackelberg Model based Game 1. This reflects the first moving advantage of the Stackelberg Model based Game.
The price of the selling energy is shown in Figure 8. With the increase of the number of energy suppliers, the price of selling energy through the game theoretic approach is much lower than that through Static Energy Cooperation Method. It is expected, since the more energy is in the market to be sold the price of the energy will be lower. We have verify in Figure 6 that energy supplier node will provide more energy to the demander node through the game theoretic approach. The price of energy sold through the Stackelberg Model based Game is lower than that through the Cournot Model based Game. The first moving advantage of the Stackelberg Model based Game is also reflected in the price. The lowest price of energy appears in the Stackelberg Model based Game 2.
6 Conclusions
In this paper, we proposed a game theoretic framework for energy cooperation in wireless sensor networks with energy harvesting and wireless power transfer. Based on the optimal inventory amount of energy at each sensor node, sensor nodes with excess energy sold part of their energy to nodes with energy shortage through the Stackelberg Model based Game and Cournot Model based Game we designed to balance the energy at each sensor node and increase the total energy utilization ratio. The numerical results showed that compared with the static energy cooperation method, energy cooperation through the game theoretic approach can highly improve the utilization ratio of the harvested energy distributed in the energy harvesting WSNs by a higher selling volume of energy with a lower price. The Stackelberg Model based Game sold more energy than the Cournot Model based Game, i.e., the dynamic game was better than the static game. More specialized, the Stackelberg Model based Game 2 sold more energy than the Stackelberg Model based Game 1. This reflected the first moving advantage of the Stackelberg Model based Game.
7 Acknowledgments
The author would like to thank the editor and reviewers for their detailed reviews and constructive comments, which have helped to improve the quality of this paper. This work has been supported by the National Natural Science Foundation of China (61571059).
References
 [1] R. Xie, F. R. Yu, H. Ji, and Y. Li, “Energyefficient resource allocation for heterogeneous cognitive radio networks with femtocells,” IEEE Trans. Wireless Commun., vol. 11, pp. 3910 –3920, Nov. 2012.
 [2] S. Bu, F. R. Yu, Y. Cai, and P. Liu, “When the smart grid meets energyefficient communications: Green wireless cellular networks powered by the smart grid,” IEEE Trans. Wireless Commun., vol. 11, pp. 3014–3024, Aug. 2012.
 [3] R. Xie, F. R. Yu, and H. Ji, “Dynamic resource allocation for heterogeneous services in cognitive radio networks with imperfect channel sensing,” IEEE Trans. Veh. Tech., vol. 61, pp. 770–780, Feb. 2012.
 [4] F. R. Yu, P. Zhang, W. Xiao, and P. Choudhury, “Communication systems for grid integration of renewable energy resources,” IEEE Network, vol. 25, pp. 22 –29, Sept. 2011.
 [5] S. Bu and F. R. Yu, “Green cognitive mobile networks with small cells for multimedia communications in the smart grid environment,” IEEE Trans. Veh. Tech., vol. 63, pp. 2115–2126, June 2014.
 [6] S. Bu, F. R. Yu, and H. Yanikomeroglu, “Interferenceaware energyefficient resource allocation for heterogeneous networks with incomplete channel state information,” IEEE Trans. Veh. Tech., vol. 64, pp. 1036–1050, Mar. 2015.
 [7] Y. Wei, F. R. Yu, and M. Song, “Distributed optimal relay selection in wireless cooperative networks with finitestate Markov channels,” IEEE Trans. Veh. Tech., vol. 59, pp. 2149 –2158, June 2010.
 [8] A. Kansal and M. B. Srivastava, “An environmental energy harvesting framework for sensor networks,” Low Power Electronics and Design, 2003. ISLPED ’03. Proceedings of the 2003 International Symposium on, pp. 481486, Aug. 2003.
 [9] K. Kaushik, Deepak Mishra, Swades De, Kaushik Roy Chowdhury and Wendi Heinzelman, “Lowcost Wakeup Receiver for RF Energy Harvesting Wireless Sensor Networks,” IEEE Sensors Journal, vol. 16, no. 16, pp. 62706278, Aug. 2016.
 [10] HweePink Tan, Pius W. Q. Lee, Winston K. G. Seah and Zhi Ang Eu, “Impact of Power Control in Wireless Sensor Networks Powered by Ambient Energy Harvesting (WSNHEAP) for Railroad Health Monitoring,” Advanced Information Networking and Applications Workshops, 2009. WAINA ’09. International Conference on, pp. 804809, May. 2009.
 [11] M. Yousof Naderi, Prusayon Nintanavongsa and Kaushik R. Chowdhury, “ RFMAC: A Medium Access Control Protocol for ReChargeable Sensor Networks Powered by Wireless Energy Harvesting,” IEEE Transactions on Wireless Communications, vol. 13, no. 7, pp. 39263937, Jul. 2014.
 [12] Alanson P. Sample, David A. Meyer and Joshua R. Smith, “Analysis, Experimental Results, and Range Adaptation of Magnetically Coupled Resonators for Wireless Power Transfer,” IEEE Transactions on Industrial Electronics, vol. 58, no. 2, pp. 544554, Feb. 2011.
 [13] Nan Zhao, F. Richard Yu and Victor C.M. Leung, “Opportunistic Communications in Interference Alignment Networks with Wireless Power Transfer,” IEEE Wireless Communications, vol. 22, no. 1, pp. 8895, Feb. 2015.
 [14] Lav R. Varshney, “Transporting Information and Energy Simultaneously,” 2008 IEEE International Symposium on Information Theory, pp. 16121616, Jul. 2008.
 [15] Pulkit Grover and Anant Sahai, “Shannon meets Tesla: Wireless information and power transfer,” 2010 IEEE International Symposium on Information Theory, pp. 23632367, Jun. 2010.
 [16] Kaibin Huang and Erik Larsson, “ Simultaneous Information and Power Transfer for Broadband Wireless Systems,” IEEE Transactions on Signal Processing, vol. 61, no. 23, pp. 59725986, Dec. 2013.
 [17] Sixing Yin, Zhaowei Qu and Linlin Zhang, “Wireless Information and Power Transfer in Cooperative Communications with Power Splitting,” 2015 IEEE Global Communications Conference (GLOBECOM), pp. 16, Dec. 2015.
 [18] Nan Zhao, F. Richard Yu and Victor C.M. Leung, “ Wireless Energy Harvesting in Interference Alignment Networks,” IEEE Communications Magazine, vol. 53, no. 6, pp. 7278, Jun. 2015.
 [19] Berk Gurakan, Omur Ozel, Jing Yang and Sennur Ulukus, “Energy Cooperation in Energy Harvesting Communications,” IEEE Transactions on Communications, vol. 61, no. 12, pp. 48844898, Dec. 2013.
 [20] Jie Xu and Rui Zhang, “CoMP Meets Smart Grid: A New Communication and Energy Cooperation Paradigm,” IEEE Transactions on Vehicular Technology, vol. 64, no. 6, pp. 24762488, Jun. 2015.
 [21] L. Ma, F. Yu, V. C. M. Leung, and T. Randhawa, “A new method to support UMTS/WLAN vertical handover using SCTP,” IEEE Wireless Commun., vol. 11, pp. 44–51, Aug. 2004.
 [22] F. Yu and V. C. M. Leung, “Mobilitybased predictive call admission control and bandwidth reservation in wireless cellular networks,” in Proc. IEEE INFOCOM’01, (Anchorage, AK), Apr. 2001.
 [23] F. Yu and V. Krishnamurthy, “Optimal joint session admission control in integrated WLAN and CDMA cellular networks with vertical handoff,” IEEE Trans. Mobile Computing, vol. 6, pp. 126–139, Jan. 2007.
 [24] I. F. Akyildiz, Weilian Su, Y. Sankarasubramaniam and E. Cayirci, “A survey on sensor networks,”IEEE Communications Magazine, vol. 40, no. 8, pp. 102114, Aug. 2002.
 [25] Shengdong Xie and Yuxiang Wang, “Construction of Tree Network with Limited Delivery Latency in Homogeneous Wireless Sensor Networks,” Wireless Personal Communications, vol. 78, no. 1, pp. 231246, Sep. 2014.
 [26] Jian Shen, Haowen Tan, Jin Wang, Jinwei Wang, and Sungyoung Lee, “A Novel Routing Protocol Providing Good Transmission Reliability in Underwater Sensor Networks,” Journal of Internet Technology, vol. 16, no. 1, pp. 171178, Jan. 2015.
 [27] Z. Li, F. R. Yu, and M. Huang, “A distributed consensusbased cooperative spectrumsensing scheme in cognitive radios,” IEEE Trans. Veh. Tech., vol. 59, no. 1, pp. 383–393, 2010.
 [28] David R. Anderson, Dennis J. Sweeney, Thomas A. Williams, Jeffrey D. Camm and R. Kipp Martin, An Introduction to Management Science: Quantitative Approaches to Decision Making, 13th Edition, SouthWestern College Published, 2011.
 [29] F. R. Yu, M. Huang, and H. Tang, “Biologically inspired consensusbased spectrum sensing in mobile ad hoc networks with cognitive radios,” IEEE Network, vol. 24, pp. 26 –30, May 2010.
 [30] Robert Gibbons, A Primer in Game Theory, Financial Times Prentice Hall Published, 1992.
 [31] YuSheng Zheng and A. Federgruen, “Finding Optimal (s, S) Policies is about as Simple as Evaluating a Single Policy ,” Operations Research, vol. 39, no. 4, pp. 654665, Jul.Aug. 1991.
 [32] Nirvikar Singh and Xavier Vives, “Price and quantity competition in a differentiated duopoly,” The RAND Journal of Economics, vol. 15, no. 4, pp. 546554, Winter 1984.