Autonomous Demand Side Management based on Energy Consumption Scheduling and Instantaneous Load Billing: An Aggregative Game Approach This work of He (Henry) Chen was supported by International Postgraduate Research Scholarship (IPRS), Australian Postgraduate Award (APA), and Norman I Price Supplementary scholarship. H. Chen, Y. Li, and B. Vucetic are with School of Electrical and Information Engineering, The University of Sydney, Sydney, NSW 2006, Australia (email: he.chen@sydney.edu.au, yonghui.li@sydney.edu.au, branka.vucetic@sydney.edu.au). R. Louie is with the Electronic and Computer Engineering, Hong Kong University of Science and Technology, Clear Water Bay, Hong Kong (email:eeraylouie@ust.hk).

# Autonomous Demand Side Management based on Energy Consumption Scheduling and Instantaneous Load Billing: An Aggregative Game Approach ††thanks: This work of He (Henry) Chen was supported by International Postgraduate Research Scholarship (IPRS), Australian Postgraduate Award (APA), and Norman I Price Supplementary scholarship.††thanks: H. Chen, Y. Li, and B. Vucetic are with School of Electrical and Information Engineering, The University of Sydney, Sydney, NSW 2006, Australia (email: he.chen@sydney.edu.au, yonghui.li@sydney.edu.au, branka.vucetic@sydney.edu.au).††thanks: R. Louie is with the Electronic and Computer Engineering, Hong Kong University of Science and Technology, Clear Water Bay, Hong Kong (email:eeraylouie@ust.hk).

He (Henry) Chen,  Yonghui Li,
Raymond H. Y. Louie,  and Branka Vucetic,
###### Abstract

In this paper, we investigate a practical demand side management scenario where the selfish consumers compete to minimize their individual energy cost through scheduling their future energy consumption profiles. We adopt an instantaneous load billing scheme to effectively convince the consumers to shift their peak-time consumption and to fairly charge the consumers for their energy consumption. For the considered DSM scenario, an aggregative game is first formulated to model the strategic behaviors of the selfish consumers. By resorting to the variational inequality theory, we analyze the conditions for the existence and uniqueness of the Nash equilibrium (NE) of the formulated game. Subsequently, for the scenario where there is a central unit calculating and sending the real-time aggregated load to all consumers, we develop a one timescale distributed iterative proximal-point algorithm with provable convergence to achieve the NE of the formulated game. Finally, considering the alternative situation where the central unit does not exist, but the consumers are connected and they would like to share their estimated information with others, we present a distributed synchronous agreement-based algorithm and a distributed asynchronous gossip-based algorithm, by which the consumers can achieve the NE of the formulated game through exchanging information with their immediate neighbors.

Smart grid, demand side management, aggregative game, Nash equilibrium, distributed iterative proximal-point method, distributed agreement (consensus) method, distributed gossip-based algorithm.

## I Introduction

Recently, demand side management (DSM) has emerged as one of the key techniques to transform today’s aging power grid into a more efficiently and more reliably operated smart grid [1, 2]. Thanks to the two-way communication capabilities of smart grid, real-time pricing [3] has been regarded as a promising technique to implement DSM due to its ability to effectively convince consumers to shift their peak-time energy consumption to non-peak time. In real-time pricing schemes, the energy price for a certain operation period is normally designed to be proportional to the aggregated load of all consumers during the considered period [3, 4, 5, 6]. As a result, the consumers would prefer to consume more energy during non-peak times rather than peak times in order to decrease their energy cost. This can improve the operation efficiency of the whole grid since its demand is flattened.

In a real-time pricing based DSM framework, the billing mechanism (i.e., how to charge the consumers for their energy usage) is of great importance since it may significantly affect the consumers’ motivation to participate in the DSM program. However, there has only been limited work investigating this important billing issue. [4] proposed a simple billing approach, where the consumers were charged in proportional to their total energy consumption for the next operation period. This total load billing method can minimize the whole grid energy cost. However, the consumers are charged the same amount if they consume the same total amount of electricity, regardless in peak or off-peak times, which leads to unfair charging for the consumers who use less electricity in peak times [5]. To address this problem, [5, 6] proposed a new billing approach, where each consumer is charged based on his/her instantaneous load in each time slot during the next operation period. As a result, the consumers will be charged more if they consume more during peak times and this can effectively improve the fairness of charging between different consumers [5]. In this paper, the billing approach proposed in [5, 6] is termed as instantaneous load billing, in contrast to the total load billing in [4]. Based on the proposed billing approach, [6] also developed a classical non-cooperative game for the DSM scenario where the traditional consumers as well as consumers owing distributed energy sources and/or energy storage compete to reduce their energy bills. However, the main analysis and results in [5, 6] are only valid when the energy price is a linear function of the total load of all consumers in each time slot. Very recently, [7] extended [6] to the scenario with a general energy price function. Based on the proximal decomposition method [8], synchronous and asynchronous algorithms were respectively developed in [6] and [7] for the consumers to achieve their optimal strategies in a distributed manner.

In this paper, we develop three novel distributed algorithms for autonomous DSM scenario, which enable the selfish consumers to optimize their own energy payment through scheduling their future energy consumption. The key contributions of this paper, with a particular emphasis on the differences with [6, 7], are summarized as follows:

(1) Inspired by [5, 6, 7], we adopt the instantaneous load billing scheme to effectively convince the consumers to shift their peak-time energy consumption and fairly charge the consumers. In this paper, we are interested in a practical polynomial energy price model instead of the general energy price model considered in [7], since the polynomial model has been widely adopted in power systems (e.g., spot market price model [9, 10]). By exploring the aggregative property of the instantaneous load billing scheme that the energy cost of each consumer only depends on its own and all consumers’ aggregated energy consumption profiles [6, 7], we develop a novel aggregative game111An aggregative game is a special kind of the non-cooperative game where each player’s payoff is parameterized by its own action and the aggregative of the actions taken by all players [11, 12][13, Ch. 4]. to model the strategic behaviors of the selfish consumers. Additionally, we perform new theoretical analysis for the Nash equilibrium (NE) of the formulated game. This analysis will be facilitated by using advanced variational inequality theory [14]. As shown in this paper, the formulation of the aggregative game can facilitate the game analysis, the algorithm design and the convergence proof for the proposed algorithms. In our previous work [15], a distributed and parallel gradient projection algorithm was proposed for the considered DSM framework.

(2) For the algorithm design, we first consider the same setup as in [6, 7] where a central unit exists and broadcasts the real-time aggregated energy consumption profile to all consumers. In this case, the synchronous and asynchronous proximal decomposition algorithms proposed in [6, 7] can be directly applied to compute the NE of the formulated game. It should be noted that the algorithms in [6, 7] are two timescale, which is due to the nature of the problem (i.e., the mapping function associated with distributed generation and storage is monotone) in [6, 7]. However, as shown later, the formulated problem in this paper can be guaranteed to possess strictly monotone mapping. Thus, we may not need to apply the two timescale algorithms, which are generally harder to implement in online settings than the one timescale algorithms [16]. Motivated by this, we develop a distributed iterative proximal-point algorithm to achieve the NE of the formulated game. This new algorithm is a parallel and one timescale algorithm and the choice of algorithm parameters does not depend on the system arguments.

(3) Considering the alternative situation without a central unit but where the consumers are connected and they exchange their estimated information with others, we develop a distributed agreement (consensus)-based algorithm, by which the consumers can achieve the NE of the formulated game through exchanging information with their immediate neighbors. Although information exchanges are required between the consumers in this algorithm, no private information (e.g., the exact energy consumption profile of each consumer) is shared between the consumers, thus effectively protecting the consumers’ privacy. Moreover, the parameters of this algorithm can also be chosen without knowing the system’s arguments a priori.

(4) Although the central unit is not necessary for the aforementioned agreement-based algorithm, synchronization between the consumers and coordination in terms of algorithm step sizes are still required, which are challenging in very large networks. Motivated by this, we develop a distributed asynchronous gossip-based algorithm for computing the NE of the formulated game without the need of a central unit. In this developed algorithm, synchronization is not required between the consumers. Besides the asynchronous updates, the consumers are allowed to use uncoordinated step sizes that are based on the frequency of the consumer update. Note that although the distributed consensus and gossip algorithms are well-known techniques, their application to achieve the NE of the formulated game is not straightforward at all and is not feasible without the formulation of the aggregative game in this paper.

Notations: All the vectors, except as specially stated, are column vectors. and denote the transpose and Euclidean norm of a vector , respectively. is the cartesian product of sets and . denotes the operation of concatenating all vectors into a single column vector, i.e., . To emphasize the -th element within , we sometimes write instead of with . We use to denote the Euclidean projection operator onto a set . and respectively denote the gradient vector and Hessian matrix of a scalar function , while denotes the Jacobian matrix of a vector function .

The rest of this paper is organized as follows. The system model and the instantaneous load billing scheme are described in Section II. Section III formulates the aggregated game and analyzes the existence and uniqueness for the NE of the formulated game. The three new distributed algorithms are proposed in Section IV-VI, respectively. In section VII, numerical results are presented to illustrate and validate the theoretical analysis. Finally, Section VIII concludes this paper.

## Ii System Model

We consider an electricity network comprised of consumers, which are served by a common energy provider. We denote the set of these consumers as . Each consumer is equipped with an energy management controller unit, which has full responsibility for scheduling the consumer’s energy consumption. In addition, there exists a two-way communications network connecting each consumer to the energy provider. Similar to [4, 17], we assume that the energy requirement of each consumer is determined in advance for future time slots. Each time slot can represent different timing horizons, e.g., one hour of a day.

### Ii-a Energy Consumption Model

We consider an energy consumption model as in [17], where the th () consumer’s energy consumption profile can be formulated as

 qn=(q1n,…,qHn)T, (1)

where is the energy consumption of consumer in the th time slot and it is subject to the following constraints:

 qh,minn≤qhn≤qh,maxnand∑Hh=1qhn=En, (2)

where and denote consumer ’s minimum and maximum energy levels222Note that the minimum and maximum energy levels can be estimated in practice by sophisticated predictive techniques, such as machine learning and stochastic signal processing [17]. Moreover, the approaches presented in this work can be easily extended to the appliance-level energy consumption model [4]. in time slot , respectively, and is the total energy requirement of consumer over all time slots. Therefore, the individual feasible energy consumption set of consumer can be expressed as

 Qn={qn:∑Hh=1qhn=En,andqh,minn≤qhn≤qh,maxn,∀h∈H}, (3)

where is the set of all future time slots. The feasible energy consumption set of all consumers can thus be expressed as

 Q=Q1×…×QN. (4)

To effectively convince the consumers to shift their peak-time energy consumption and fairly charge the consumers for their energy consumption, we adopt the instantaneous load billing scheme [5, 6, 7], where the energy price (the cost of one unit energy) of a certain time slot is set as an increasing and smooth function of the total demand in that time slot, and the consumers are charged based on the instantaneous energy price as well as the energy amount they consume in each time slot. Instead of the general price model[7], we focus on a practical and specific polynomial energy price model in this paper, which has been widely adopted in power systems (e.g., the spot market price model [9, 10]). Specifically, the energy price of the th time slot is given by:

 ph(Lh)=ah(Lh)bh+ch, (5)

where are time slot-specific parameters with , , , and is the total energy consumed by all consumers in time slot . It is should note that the price function in (5) can readily account for the important characteristics of energy prices that are needed for DSM in smart grid. For example, the increasing and convex price function ensures that the energy price will grow more rapidly as the aggregated load increases. This can effectively convince the consumers to shift their peak-time consumption to non-peak hours, thereby flattening the overall demand curve and reducing the need for carbon-intensive and expensive peaking power plants. Therefore, the considered energy price model can improve the efficiency of the energy provider, and motivate and engage the energy provider to enforce such price model.

Follow the adopted energy price model, the total energy cost for consumer over all future time slots can thus be given by:

 Bn(qn,q−n)=∑Hh=1[ph(∑Nm=1qhm)qhn], (6)

where denotes the vector of all consumers’ energy consumption profiles, except the th one. This is in contrast to the total load billing method in [4], where the energy payment of the th consumer is calculated by

 BTLBn(qn,q−n)=En∑Nm=1EmH∑h=1[ph(N∑m=1qhm)N∑m=1qhm]. (7)

It has been shown in [5] that the adopted billing method in (6) is fairer than the total load billing method given in (7). This will also be validated by the simulation results in this paper.

Note that (6) can be further rewritten as

 Bn(qn,qΣ)=∑Hh=1[ph(qhΣ)qhn], (8)

where denotes the aggregated energy consumption profile of all consumers over future time slots and is the th element of . From (8), we can see that the calculation of the total energy cost of each consumer only requires the knowledge of the aggregated energy consumption profile of all consumers (), and that the individual consumption profile of each consumer () is not required any more.

## Iii Game Formulation and Analysis

In this section, we formulate an aggregative game for the considered DSM scenario. By employing variational inequality theory, we then analyze the existence and uniqueness of the NE for the formulated aggregative game.

### Iii-a Aggregative Game Formulation

We consider the scenario where all consumers are selfish. In particular, each consumer aims to minimize his/her total cost through energy consumption scheduling. Mathematically, this will involve the th consumer () solving the following optimization problem:

 (9)

We can observe from (9) that the consumers solve optimization problems which are coupled with the aggregated energy consumption of all consumers. Hence, this energy consumption control scenario can be modeled by the following aggregative game [11, 12][13, Ch. 4]:

• Players: The consumers.

• Actions: Each consumer selects its energy consumption to minimize his/her total energy cost.

• Payoffs: The total energy cost defined in (8).

For convenience, we denote this Nash equilibrium (NE) problem as . In the following subsection, we will employ variational inequality theory [14] to analyze the formulated game.

### Iii-B NE Analysis

Before proceeding, it is convenient to first present the following lemma regarding the properties of the formulated game’s action sets and payoff functions:

###### Lemma 1

For each , the set is convex and compact, and each function is continuously differentiable in . For each and each fixed tuple , the function is convex in over the set .

{proof}

See Appendix -A.

Under Lemma 1 and according to [14, Prop. 1.4.2], we have the following lemma:

###### Lemma 2

The NE of the formulated game is equivalent to the solution of the variational inequality (VI) problem333Given a subset of the Euclidean -dimensional space and a mapping : , the variational inequality problem, denoted VI, is to find a vector such that . denoted by VI where and

 F(q)=(Fn(qn,qΣ))Nn=1, (10)

with

 Fn(qn,qΣ)=∇qnBn(qn,qΣ). (11)

By investigating the monotonicity property of the mapping , we can derive the following proposition:

###### Proposition 1

If the price parameter satisfies for any , then the formulated aggregative game admits a unique NE.

{proof}

See Appendix -B.

###### Remark 1

As can be seen from Proposition 1, only a specific relationship between the exponential factor of the polynomial price function and the number of consumers is required to guarantee the uniqueness of the NE. Specifically, the exponential factor of the price function is subject to an upper bound, which is inversely proportional to the number of consumers .

One could consider to solve the aforementioned game in a centralized manner, where a central unit adopts the algorithms proposed in [14, Ch. 12] to solve the associated VI problem. However, such an approach requires each consumer to release detailed information about their energy consumption feasible set, which may lead to consumers’ privacy and security concerns. To overcome this issue, in the following sections, we will develop three different distributed algorithms to achieve the NE of the formulated aggregative game for the scenarios with and without a central unit, which calculates the aggregated load and broadcasts it to consumers in each iteration of the algorithm.

## Iv Distributed Iterative Proximal-point Algorithm with A Central Unit

In this section, we consider the same setting as in [6, 7] where there is a central unit, which can provide the consumers with the latest information of the aggregated energy consumption profile after all consumers update their individual ones. In this case, we develop a distributed iterative Proximal-point algorithm to achieve the NE of the formulated aggregative game.

Before presenting our algorithm, it is worth mentioning that the formulated game can also be solved by the synchronous and asynchronous proximal decomposition algorithms proposed in [6] and [7], which were guaranteed to converge under some conditions on the algorithm parameters. The distributed algorithms in [6, 7] were proposed based on the proximal decomposition method [8] and solved a sequence of regulated versions of the original problem, each of which may need a distributed iterative process in itself. This is actually a two timescale approach (i.e., the proximal method updates at a slower timescale while solutions of the regularized problems change at a faster timescale) and is generally harder to implement in online settings [16]. Additionally, the regulation parameter of such kind of algorithms has to be chosen centrally since it is normally dependent on the system arguments. It should be noted that the two-timescale property of the algorithms in[6, 7] is due to the nature of the problem (i.e., the mapping function associated with distributed generation and storage is monotone). However, as shown in Section III, the formulated problem in this paper can be guaranteed to possess strictly monotone mapping. Thus, we may not need to apply the two timescale algorithms. Motivated by this, we present a single timescale distributed algorithm based on the iterative regulation technique [16, 18], which requires only one projection step in each iteration. This algorithm is formally described in Algorithm 1.

The convergence property of Algorithm 1 is summarized in the following proposition:

###### Proposition 2

Assume that the condition in Proposition 1 holds. Then, the sequence of the energy consumption profile generated by Algorithm 1 converges to the unique NE of the game if the step-size satisfies the following:

 ∑∞t=1γ(t)=∞and∑∞t=1γ2(t)<∞. (12)
{proof}

See Appendix -C.

As shown above, Algorithm 1 can converge to the NE of the formulated game when there is a central unit that calculates the aggregated energy consumption profile and broadcasts it to all consumers in each iteration. However, the developed Algorithm 1 and the algorithms in [6, 7] cannot be directly implemented for situations where the central unit does not exist, in which case the consumers thus do not have ready access to the aggregated energy consumption profile. Motivated by this issue, we will develop a distributed synchronous agreement-based algorithm and a distributed asynchronous gossip-based algorithm to achieve the NE of the formulated game in the following sections.

## V Distributed Synchronous Agreement-based Algorithm without A Central Unit

In this section, we consider an alternative scenario where the central unit does not exist, but the consumers are connected in some manner and they are willing to share their estimated information through local communication. For this setting, we develop a distributed agrement-based algorithm, through which the consumers can achieve the NE of the game via exchanging information with their immediate neighbors. In the developed algorithm, the connection topology of the consumers is modeled as an undirected (not necessarily complete) static graph. In practice, such a connection can be established through either wired or wireless communication techniques. Specifically, the connection can be implemented by employing the power line communication technique or using the resources of cellular networks to establish a virtual private network. As these techniques are widely deployed, the connection of a large number of consumers in large areas is feasible. Since only immediate connected consumers exchange information, the amount of data to be exchanged at each iteration of the developed algorithm is proportional to the numbers of connections between the consumers.

Recall that, in the formulated aggregative game, each consumer’s payoff is only determined by his/her own energy consumption profile and the aggregated energy consumption profile of all consumers. Hence, the unique NE of the formulated game is achieved when the consumers reach an agreement (consensus) on the aggregated energy consumption profile. Following this equivalence and inspired by [13, Ch. 4], we develop a distributed agreement-based algorithm to achieve the unique NE of the considered aggregative game. In each iteration of this algorithm, each consumer executes the following three steps:

• Step 1: Estimate the average energy consumption of all consumers through a weighted combination of his/her own estimation and the estimation of the immediate neighbors in the last iteration. We use to denote the average energy consumption of all consumers estimated by the consumer in the th iteration. Then, the aggregated load of the whole network estimated by consumer is .

• Step 2: Update his/her energy consumption profile based on the estimated aggregated load through executing a Euclidean projection operation.

• Step 3: Update his/her own estimation of the average energy consumption.

To proceed, it is convenient to first model the connection topology between consumers. For simplicity but without loss of generality, we model the connection topology of the consumers as an undirected static graph with being the set of all consumers and being the set of undirected edges among the consumers. The notation means that consumer and consumer are immediate neighbors, and denotes the set of consumer ’s neighbors, i.e., . Now we are ready to present the distributed agreement-based algorithm, which is formally described in Algorithm 2, where the notation denotes the nonnegative weight that consumer assigns to the estimate of consumer , which is set to zero if and .

In terms of the convergence of Algorithm 2, we have the following proposition:

###### Proposition 3

Assume that the undirected graph is connected, the step-size is monotonically decreasing with and satisfies the following:

 ∑∞t=0α(t)=∞,and∑∞t=0[α(t)]2<∞, (13)

and the weights adopted by the consumers meets the following444Note that the summations in the following equations are actually equivalent to that over the set . This is because that if consumer is not a neighbor of consumer .:

 ∑Nk=1wn,k=1,∀n,and∑Nn=1wn,k=1,∀k, (14)

and the condition in Proposition 1 holds. Then the sequence generated by Algorithm 2 converges to the unique NE of the formulated game .

{proof}

See Appendix -D.

###### Remark 2

In this paper, we use the following formula for the weights [13, Ch. 4]:

 wn,k={τ/τ[maxn|Dn|][maxn|Dn|]ifn≠k1−|Dn|τ/τ[maxn|Dn|][maxn|Dn|]ifn=k, (15)

where denotes the cardinality of the set , and is used to measure the relative proportion of the neighbors’ estimates in each consumer’s estimation of the average energy consumption. It is straightforward to validate that the weights in (15) satisfy the conditions in (14). Other choices of the weights can be found in [19].

Although information exchanges are required between the consumers in Algorithm 2, the consumers only need to share their estimations of the average energy consumption of all consumers instead of their exact energy consumption profiles with their immediate neighbors. Thus, the developed algorithm can avoid the consumers’ security and privacy concerns.

Note that although the central unit is not necessary for the developed Algorithm 2, synchronization between the consumers and coordination in terms of algorithm step sizes are still required, which are challenging in very large networks. Motivated by this, we will develop a distributed asynchronous algorithm in next section.

## Vi Distributed Asynchronous Gossip-based Algorithm without A Central Unit

In this section, we develop a distributed asynchronous gossip-based algorithm for computing the NE of the formulated game without the need of a central unit. The consumers perform their estimations and updates in the same way as in the Algorithm 2, but the updates occur asynchronously instead of synchronously. The developed algorithm allows the consumers to use uncoordinated step size values. More specifically, the consumers can choose the step size based on their own information-update frequency. The graph model for the connection topology of the consumers in Section V is also applicable in this section.

To allow for asynchronous updates, we adopt the gossip protocol [20] to model the consumers’ exchange of their estimations for the average energy consumption of all consumers. In this protocol, each consumer is assumed to have a clock which ticks according to a Poisson process with rate 1. At a tick of his/her clock, consumer contacts a randomly selected555Here, we consider that each neighbor has an equal chance of being selected. neighbor to exchange information. With reference to [20], the consumers’ clocks processes can be equivalently modeled as a single virtual clock that ticks according to a Poisson process with rate . We assume that only one consumer communicates with its neighbor at each tick of the virtual clock and we use to denote th tick of the virtual Poisson process. We discretize time so that the instant corresponds to the time-slot . At each time , every consumer has his/her consumption profile and estimation of the average energy consumption of all consumers . Let denote the consumer whose local clock ticked at time . Note that is uniformly distributed in the set since the Poisson clocks at each consumer are independent. Moreover, the memoryless property of the Poisson arrival process ensures that the process is independent and identically distributed. We use to denote the consumer randomly contacted by the consumer , where is a neighbor of the consumer , i.e., . Then, these two consumers will exchange their estimations of average energy consumption and update their own energy profiles. The developed asynchronous gossip-based algorithm is formally described in Algorithm 3. As can be seen from Algorithm 3, the consumers perform the same updates as in the synchronous Algorithm 2, but only two randomly selected consumers update their estimations of average energy consumption and their own energy profiles at each iteration, while the other consumers do not update.

For the convergence of Algorithm 3, we have the following proposition and the proof follows from Appendixes -B--D, the adopted step sizes and [13, Ch. 4, Prop. 12]:

###### Proposition 4

Assume that the condition in Proposition 1 holds and the undirected graph is connected. Then, the sequence of the energy consumption profile generated by Algorithm 3 converges to the unique NE of the game almost surely.

###### Remark 3

In this developed algorithm, synchronization is not required between the consumers. Besides the asynchronous updates, the consumers are allowed to use uncoordinated step sizes that are based on the frequency of the consumers’ updates. Specifically, consumer uses the step size

 αn(t)=1ϵn(t), n∈{It,Jt}, (16)

at the th iteration, where denotes the numbers of updates that consumer has performed up to time inclusively. In addition, analogous to Algorithm 2, no private information is required to exchange between the consumers in Algorithm 3.

It is worth mentioning that the pairwise gossip protocol (i.e, only a random pair of consumers is chosen to update at each iteration) is adopted for simplicity in this paper. The developed algorithm can be extended to the general setup where a random subset of consumers (more than one pair) exchange their estimations and update their energy profiles at each iteration. This will be considered in our future work.

## Vii Numerical Results

In this section, we present some numerical results to validate the above theoretical analysis and illustrate the performance of the developed algorithms.

In the following simulation results, we consider the residential scenario consisting of consumers, where the consumers determine their energy consumption for the following whole day, which starts from 8 AM. Each time slot is set as one hour, i.e., and the first time slot corresponds to the hour between 8 AM and 9 AM. In Fig. 1, we provide a typical energy consumption interval of a residential consumer [21, Figs. 2.5-2.7],[22]. Considering that different consumers may have different energy consumption interval in practice, the ‘Low limit’ and ‘Upper limit’ of each consumer in each time slot are formed by respectively adding a random real number to the corresponding value in Fig. 1. Then, the initial energy consumption of a certain consumer in each time slot, , is uniformly chosen between his/her corresponding ‘Low limit’ and ‘Upper limit’. The numerical results show that the selected consumption parameters yield the total energy consumption of every consumer in the order of 10 kWh to 30 kWh, which is representative of a residential consumer [22].

According to Fig. 1, we classify the whole time horizon into three segments: off-peak hours (12 AM to 7 AM), mid-peak hours (7 AM to 4 PM and 10 PM to 12 AM), and on-peak hours (4 PM to 10 PM). We also set equal to 0.003, 0.004 and 0.005 for the off-peak, mid-peak and on-peak hours, respectively, and parameters and are set equal to 1.2 and 0 for , respectively. In the considered DSM scenario, the value for the th consumer is set to his/her ‘Low limit’ of the th time slot. In addition, the values of for mid-peak and on-peak hours are set to his/her maximum value of the ‘Upper limit’, while the values of for the off-peak hours are uniformly chosen from the interval . The values of are chosen to be equal to the sum of the consumers’ initial energy consumption profiles before applying the DSM program. Moreover, the parameters for the algorithms are chosen666We refer the readers to [18] for more discussion on the choice of algorithm parameters. as follows: and for Algorithm 1, and and for Algorithm 2. Finally, a randomly generated connection structure of the consumers for Algorithm 2 and 3 are given in Fig. 2, where two consumers are directly linked means that they are immediate neighbors, who can exchange information in the iterations of the algorithms.

Fig. 3 plots the total energy cost for three different consumers versus the number of iterations of the developed algorithms. It can be observed from Fig. 3 (a) that both Algorithm 1 and Algorithm 2 converge to the NE of the formulated game very quickly. Specifically, the energy cost of each consumer has already achieved a relatively stable state after the first 10 iterations, which verifies the validness of both Proposition 2 and Proposition 3, as well as displaying the high efficiency of the developed algorithms. Fig. 3 (b) is plotted to illustrate the convergence performance of the Algorithm 3. As can be observed from Fig. 3 (b) that the total energy cost of different consumers approach to coincide with that obtained by Algorithm 1 and 2 after iterations. This validates the results given in Proposition 4. Note that due to space limitations, we only show results in Fig. 3 for three randomly selected consumers, although it can be shown that similar results also hold for the other consumers and a wide range of settings with different parameters.

In Fig 4, we compare the aggregated energy consumption profiles of all consumers corresponding to the situations before and after DSM program. We clearly observe from Fig. 4 that the proposed DSM scheme effectively encourages the consumers to shift their energy consumption from peak to non-peak hours. We also investigate the peak-to-average ratio (PAR) of the aggregated load defined as

 PAR=HmaxhqhΣ∑Hh=1qhΣ. (17)

The simulation results show that the PAR decreases from to (i.e., % less) before and after the DSM program. This will result in a generally flattened demand profile, which will not only reduce the consumers’ energy cost but also benefit the efficiency of the whole power grid.

To show that the adopted billing method can fairly charge the consumers, we plot the energy consumption profiles of consumer and consumer after applying the proposed DSM program in Fig. 5. Their total daily energy requirements are  (kWh) and  (kWh), respectively. If the total load billing method [4] was used, consumer would be charged more than consumer since . However, as can be observed from Fig. 5, the on-peak energy usage of consumer is larger than that of consumer . This can also be reflected by the PAR values of these two consumers, i.e., and . Thus, it may be not fair to charge consumer more than consumer simply because he/she consumes more energy totally. In contrast, our numerical results show that consumer and consumer will finally be charged and (i.e., ) after the proposed DSM program. This result is understandable since the adopted billing method considers not only how much the consumers consume the energy totally but also when the consumers use the energy. By this example, we show that the adopted billing approach can charge the consumers more fairly, thereby motivating the consumers to participate in the DSM program.

Fig. 6 compares the total energy cost of all consumers for three different cases with different number of consumers. As expected, it can be observed from Fig. 6 that the total energy cost is significantly reduced after the proposed DSM program. We also compare the performance of the proposed game-theoretical DSM program with the optimal one obtained by solving the following social welfare optimization problem:

 min{q1,…,qN}∑Nn=1Bn(qn,q−n)s.t.qn∈Qn,∀n∈N. (18)

From Fig. 6, we can observe that the total energy cost achieved by the proposed DSM program is almost the same with the optimal one777The theoretical analysis of this observation (i.e., the price of anarchy analysis for the formulated game) is out of the scope of this paper and will be considered in future work.. Thus, we can claim that the proposed DSM framework qualifies as a practically appealing candidate for the DSM of future smart grid.

## Viii Conclusions

In this paper, we formulated an aggregative game for the demand side management program based on energy consumption scheduling and instantaneous load billing, where the consumers are selfish and compete to minimize their individual energy cost. The sufficient condition for the existence and uniqueness of Nash equilibrium (NE) of the formulated game was subsequently given and proved. Based on the formulation of the aggregative game, we developed three distributed algorithms to achieve the NE of the formulated game, corresponding to the two scenarios where the consumers can or cannot access the real-time information of the aggregated load. In these algorithms, the choice for the algorithm parameters do not depend on the arguments of the system and no private information is required to exchange between consumers. Numerical results showed that the developed algorithms can quickly converge to the NE of the formulated game and efficiently convince the consumers to shift their on-peak consumption, which are beneficial to both the consumers and the whole grid.

### -a Proof of Lemma 1

It is evident that the statements in the first part of Lemma 1 holds. Hence, we only need to prove the convexity of in for every fixed . This can be achieved by proving that the Hessian of is positive semidefinite [23]. After some algebraic manipulation, we have

 ∇2qnBn(qn,qΣ)=diag{[qhnph′′(qhΣ)+2ph′(qhΣ)]Hh=1}. (19)

Since (19) is a diagonal matrix with all diagonal elements being positive, the Hessian matrix of is positive semidefinite. This completes the proof.

### -B Proof of Proposition 1

Based on Lemma 2, the proof of this proposition follows if we can show that the formulated VI in Lemma 2 only possesses one solution. With reference to [8, Thm. 4.1][24, Thm. 2.3], the VI admits a unique solution if the mapping is strictly monotone888A mapping : is said to be strictly monotone on if . over since the feasible set is compact and convex.

To prove the strict monotonicity of the mapping , it suffices to show that

 (20)

for any .

Let and , then we can be re-write (20) as

 ∑Hh=1[(lh−jh)T(∇lhBhn(lh)−∇jhBhn(jh))]>0, (21)

where , and .

We observe that a sufficient condition for (21) to hold is if

 (lh−jh)T[gh(lh)−gh(jh)]>0,∀h∈H, (22)

where , which is defined for the sake of notation.

Recall the definition of a strictly monotone mapping, we can obtain that (22) holds if the mapping is strictly monotone. With reference to [8, Eq. (4.8)], the condition in (22) can be shown to be equivalent to proving the Jacobian matrix of is positive definite. Since the transpose operation does not change the definite property of a given matrix, what we only to prove is that the transpose of the Jacobian matrix of , denoted by , is positive definite.

To proceed, we have the th entry of after some algebraic manipulation given by

 [Gh(lh)]n,m={σh[2qhΣ+(bh−1)qhn],ifn=mσh[qhΣ+(bh−1)qhn],ifn≠m (23)

where .

Since the matrix may not be symmetric, we can prove its positive definiteness by showing that the symmetric matrix

 Gh(lh)+Gh(lh)T=σh⎛⎜ ⎜⎝zh1T+1(zh)TTb+2qhΣI⎞⎟ ⎟⎠, (24)

is positive definite [25], where , is a vector with every element of 1. This is equivalent to showing that the smallest eigenvalue of this matrix is positive.

After some appropriate calculations [25], the two non-zero eigenvalues of the matrix in (24) are given by

 η1Tb=(N+1+bh)qhΣ+√N(zh)Tzh−2qhΣ,η2Tb=(N+1+bh)qhΣ−√N(zh)Tzh−2qhΣ. (25)

Note that is implicit here.

Since , the smallest eigenvalue of the matrix can be expressed as

 ηmin=σhmin((N+1+bh)qhΣ−√N(zh)Tzh,2qhΣ), (26)

where the second term in the function arises because there are zero eigenvalues in the matrix .

To further simplify (26), we have

 (zh)Tzh=∑Nn=1(qhΣ+(bh−1)qhn)2≤(N−1+(bh)2)(qhΣ)2, (27)

Substituting (27) into (26), we obtain

 ηmin≥σhmin(κhqhΣ,2qhΣ), (28)

where .

Since , we observe from the right hand side of (28) that if , or equivalently , which we can re-written as . Thus, a sufficient condition for is

 bh<3+4/(N−1). (29)

This completes the proof.

### -C Proof of Proposition 2

The proof for the convergence of a general iterative proximal-point algorithm and the corresponding necessary conditions were presented in [18, Sec. 3]. Therefore, we only need to prove that the formulated NEP meets all the required conditions listed in [18, Assumption (A3)].

Firstly, it is evident that the set is compact, and that and are both bounded for . Secondly, as we have proved in Appendix -B, the mapping is strictly monotone on when the price parameter satisfies for . Therefore, we only need to prove that is Lipschitz continuous over , i.e., show that there exists a real constant such that, for all ,

 ∥F(q)−F(s)∥2≤clip1∥q−s∥2. (30)

According to the definition of Euclidean norm, (30) can be shown to hold if each element of the function , denoted by , is Lipschitz continuous in , i.e., for any and any , there exists a real constant such that, for all ,

 ∣∣fhn(q)−fhn(s)∣∣≤clipn,h∥q−s∥2. (31)

We now proceed to prove (31) holds. After some algebraic manipulation, we have

 fhn(q)=ph′(qhΣ)qhn+ph(qhΣ)=fhn(qhn,qhΣ). (32)

Substituting (32) into the left-hand side of (31), we have

 (33)

where and the inequality follows according to the triangular inequality.

Now by recalling that , we can rewrite the function as

 fhn(qhn,qhΣ)=ah(qhΣ)bh−1(bhqhn+qhΣ)+ch. (34)

With reference to [26, Ch. 12], it is straightforward to deduce that the function in (34) is Lipschitz continuous in for a fixed and is Lipschitz continuous in for a fixed . That is there exists two real constant such that for any and ,

 ∣∣fhn(qhn,qhΣ)−fhn(shn,qhΣ)∣∣≤clipn,h,1∣∣qhn−shn∣∣                =clipn,h,1√(qhn−shn)2≤clipn,h,1∥q−s∥2, (35)

and for any and ,

 ∣∣fhn(shn,qhΣ)−fhn(shn,shΣ)∣∣≤clipn,h,2∣∣qhΣ−shΣ∣∣≤clipn,h,2√∑Nn=1(qhn−shn)2≤clipn,h,2∥q−s∥2. (36)

By substituting (35) and (36) into (33), we deduce that it can always find a real constant such that (