On the Efficiency of Nash Equilibria in Charging Games

# On the Efficiency of Nash Equilibria in Charging Games

## Abstract

We consider charging problems, where the goal is to coordinate the charge of a fleet of electric vehicles by means of a distributed algorithm. Several works have recently modelled the problem as a game, and thus proposed algorithms to coordinate the vehicles towards a Nash equilibrium. However, Nash equilibria have been shown to posses desirable system-level properties only in simplified cases, and it is unclear to what extent these results generalize. In this work, we use the game theoretic concept of price of anarchy to analyze the inefficiency of Nash equilibria when compared to the centralized social optimum solution. More precisely, we show that i) for linear price functions depending on all the charging instants, the price of anarchy always converges to one as the population of vehicles grows and ii) for price functions that depend only on the instantaneous demand, the price of anarchy converges to one if the price function takes the form of a positive pure monomial. For finite populations, iii) we provide a bound on the price of anarchy as a function of the number vehicles in the system. We support the theoretical findings by means of numerical simulations.

\IEEEoverridecommandlockouts\overrideIEEEmargins

## 1 Introduction

In the last decade we have witnessed a profound change in the way energy systems are operated. Traditional principles such as scheduling controllable power plants according to the exogenous demand are being turned upside down. A new paradigm called demand response is emerging, according to which the energy requirements of a population of users are tuned, by means of incentives, to account for the operational needs of the power grid [1]. Previous works [2, 3, 4] have suggested modeling these demand response methods as a game. Therein, each user represents a player that needs to optimize his energy consumption over a given period of time, with the objective of minimizing his electricity bill. What couples the users, and thus makes the charging problem a game, is the assumption that the energy price depends at every instant of time on the sum of the energy demand of the whole population. The seminal paper [2] shows that the (unique) Nash equilibrium of such game has desirable properties from the standpoint of the grid operator, in the case of very large and homogeneous populations. This formulation requires all users to have equal energy demand and the same charging window (homogeneity). Under these assumptions, [2] shows that the equilibrium is socially optimum in the sense that it minimizes the collective electricity bill (including the cost of flexible and inflexible demand) and thus fills the overnight demand valley. Consequently, a rich body of literature has focused on devising distributed and decentralized schemes that can be used by the grid operator to coordinate the strategies of the agents to a Nash or Wardrop equilibrium [3, 4, 5, 6, 7]. These works consider games with more complex costs and constraints than what originally presented in [2]. What has not been done so far, is to study under which conditions the optimality statement made in [2] is still valid in the presence of more complex cost functions, agents heterogeneity and realistic charging constraints (e.g., upper bounds on the instantaneous charging, different charging windows, ramping constraints, etc). Nonetheless, this is a fundamental prerequisite for the applicability of the coordination schemes described above. In fact, if the Nash equilibria do not posses desirable system-level properties, the grid operator has no incentive in coordinating the agents to such a strategy profile in the first place.
A first attempt in this direction was made in [8] where the same model studied in [2] is analyzed for finite populations, and in [9] where the result of [2] is generalized, in a probabilistic sense, to heterogeneous populations. Both these works however assume that the price function is linear. Additionally, the analysis of [2, 8, 9, 10] is limited to simplex constraints and assume that the price at time depends only on the consumption at the very same time instant. The aim of this paper is to generalize these results to general convex constraints, non linear price functions and finite populations of vehicles. In order to do so, we model a charging problem as an aggregative game [11], and study the efficiency of the equilibrium allocations using the notion of price of anarchy (PoA). The PoA is a measure introduced in game theory to quantify how much selfish behavior degrades the performance of a given system [12]. The PoA is by definition greater or equal to and the closer to the better the overall performance of the system. The result in [2] can equivalently be restated as the fact that for homogeneous populations with simplified constraints, the PoA converges to  as the population size grows.

Our main contributions are as follows:

1. We show that the PoA for charging games with linear price function (that might however depend on all charging instants) and generic convex constraints always converges to , thus extending [2, 8, 9, 10];

2. For charging games with generic convex constraints and a nonnegative price function that depends only on the instantaneous demand, we show that the PoA converges to if the price function is a positive pure monomial (i.e., for some ). On the contrary, if the price function does not have this structure, it is possible to construct a sequence of games whose PoA does not converge to . This result greatly extends the applicability of [2, 8, 9, 10].

3. In both the previous cases we provide an explicit bound connecting the efficiency of the equilibria with the (finite) number of vehicles in the game. To the best of our knowledge, this is the first result providing a bound on the price of anarchy as a function of the population size, for charging games with general convex constraints and linear / non linear price functions.

#### Organization

Section 2 includes the game formulation and some preliminary notions. Section 3 characterizes the aggregate behavior at the Wardrop equilibria and social optimizers. Section 4 presents the main results for linear and nonlinear price functions. Section 5 focus on the application of charging a fleet of electric vehicles.

#### Notation

and denote the elements of whose components are non negative and positive, respectively. Given not necessarily symmetric, . Given a function we define the matrix component-wise as . An operator is called strongly monotone if there exists such that for all ; represents the uniform distribution on the real interval .

## 2 Problem formulation

Let us consider a population of agents, each choosing an action . Agent is charged a cost that depends on his own action and on the average action of the population, typical of aggregative games [11]. We assume that the cost function of agent is

 Ji(xi,σ(x))\coloneqqp(σ(x)+d)⊤xi, (1)

with and . The cost in (1) can be used to describe applications where denotes the usage level of a certain commodity, whose per-unit cost depends on the average usage level of the other players plus some inflexible normalized usage level  [2, 6]. We denote with , and identify such game with the tuple

 G\coloneqq{M,{Xi}Mi=1,p}. (2)

### 2.1 Nash, Wardrop equilibrium and social optimizer

We consider two notions of equilibrium for the game .

###### Definition 1 (Nash equilibrium [13]).

A set of actions is a Nash equilibrium of the game if and for all and all

 Ji(xi\textupN,σ(x\textupN))≤Ji(xi,1Mxi+1M∑j≠ixj\textupN). (3)

Observe that on the right-hand side of (3) the variable appears in both arguments of . As the population size grows, the contribution of an agent to the average decreases. This motivates the definition of Wardrop equilibrium.

###### Definition 2 (Wardrop equilibrium [14, 15]).

A set of actions is a Wardrop equilibrium of the game if , and for all , and all ,

 Ji(xi\textupW,σ(x\textupW))≤Ji(xi,σ(x\textupW)). (4)

Note that in this latter definition the average is fixed to on both sides of (4). Consequently, a feasible set of actions is a Wardrop equilibrium if no agent can improve his cost, assuming that the average action is fixed.

###### Definition 3 (Social optimizer).

A set of actions is a social optimizer of if and it minimizes the cost

 J\textupS(σ(x))\coloneqqp(σ(x)+d)⊤(σ(x)+d).

Note that the cost is the sum of all the players costs, divided by , and the additional term . The reason why the latter term is included is that we want to compute the total cost of buying the commodity both for the flexible () and inflexible () users. This cost was first introduced in [2] and then used in [8, 9, 10]. The inflexible usage level is sometimes modeled in the literature [9] as an additional player with constraint set represented by . We do not follow such approach here because we are interested in large populations and this set is unbounded as . Throughout the manuscript, we denote with

 Σ\coloneqq{z∈Rn | z=1MM∑j=1xj, xj∈Xj, ∀ j=1,…,M},

and make the following regularity assumptions on costs and constraints.

###### Standing Assumption.

For , the constraint set is closed, convex, non empty. For , the function is continuously differentiable and strongly monotone while is strongly convex.

In the following we denote with and the Lipschitz constant and monotonicity constant of , respectively. Further we denote with the Lipschitz constant of .

## 3 Characterization of the average

This section characterizes the average players’ action at the Wardrop equilibrium and at the social optimizer of using the theory of variational inequalities. These preliminaries are required to derive the results of Section 4.

###### Definition 4 (Variational inequality [16]).

Given and . A point is a solution of the variational inequality if

 F(¯x)⊤(x−¯x)≥0.
###### Lemma 1 (Equivalent characterizations).
1. Given a Wardrop equilibrium, its average solves , with ,

 F\textupW(z)\coloneqqp(z+d). (5)

The admits a unique solution . Let us define . Then any vector of strategies is a Wardrop equilibrium.

2. Given a social optimizer, its average solves , with ,

 F\textupS(z)\coloneqqp(z+d)+[∇zp(z+d)](z+d). (6)

The admits a unique solution . Define . Then any vector of strategies is a social optimizer.

The proof is reported in the Appendix.

## 4 Price of Anarchy for finite and large populations

In this section we study the efficiency of equilibria as a function of the population size . To do so, we consider a sequence of games of increasing population size. For fixed , the game is played amongst agents and is defined as in (2) with arbitrary sets . The function is instead the same for every game of the sequence.

###### Assumption 1.

There exists a convex, compact set such that for each game in the sequence . Moreover, is convex in for all fixed , for all . In the following, we let .

Given a game in the sequence, we quantify the efficiency of the equilibrium allocations using the notion of price of anarchy [12]

 \textupPoAM\coloneqqmaxxN∈\textupNEMJ\textupS(σ(xN))J\textupS(σ(x\textupS)), (7)

where defines the set of Nash equilibria of and a social optimizer of . The price of anarchy captures the ratio between the cost at the worst Nash equilibrium and the optimal cost; by definition . We divide the analysis of in two parts depending on whether the function is linear or not.

### 4.1 Linear price function

Throughout this subsection we consider linear price functions , as detailed in the following.

###### Assumption 2.

The price function is linear, that is , with , .

Note that Assumption 2 implies the strong monotonicity of and the strong convexity of , therefore Assumption 2 is consistent with the standing assumption. It is easy to verify that is convex in , consistently with Assumption 1. Nevertheless, is not required to be diagonal as it was instead required for the previous results of [8, 9] to hold.

###### Theorem 1 (\textupPoAM bound and convergence to 1).
• Under Assumption 2, for any fixed game in the sequence, every Wardrop equilibrium is a social optimizer i.e. .

• With the further Assumption 1, for any fixed game in the sequence it holds that

 J\textupS(σ(x\textupS))≤J\textupS(σ(x\textupN))≤J\textupS(σ(x\textupS))+c√M, (8)

with constant, social optimizer.
Thus, if there exists s.t. for every game in the sequence , one has

 1≤\textupPoAM≤1+c^J√M   and  limM→∞\textupPoAM=1.

The proof is reported in the Appendix.

###### Remark 1.

The previous theorem extends the results of [2, 8, 9, 10] simultaneously allowing for arbitrary convex constraints, finite populations, and non diagonal price function. Observe that the condition is merely technical and is required to properly define the price of anarchy. This condition is trivially satisfied in the applications when, e.g., every agent requests an amount of charge bounded away from zero. Even if the latter condition is not satisfied, the cost at any Nash equilibrium converges to the minimum cost for , see Equation (8).

### 4.2 Non linear price

In this section we consider to be a nonlinear function, and further assume that its -th component depends only on the -th component for all . This models, for example, the fact that the unit cost of electricity at every instant of time depends only on the total consumption at that very same instant.

###### Assumption 3.

The price function takes the form

 p(z+d)=⎡⎢ ⎢⎣f(z1+d1)⋮f(zn+dn)⎤⎥ ⎥⎦for somef(y):R>0→R>0.

Further and  .

If is not linear, a simple check shows that, in general, when . Consequently, the game is not potential, see [16, Theorem 1.3.1]. Hence methods to bound the price of anarchy based on the existence of an underlying cost function, such as those in [8, 9], can not be used here.

###### Theorem 2 (\textupPoAM convergence and counterexample).

Suppose that Assumptions 1, 3 hold. Further assume that for some , for every game in .

• If with and , it holds

 1≤\textupPoAM≤1+c^J√M   and  limM→∞\textupPoAM=1,

with constant.

• For , if satisfies the assumptions, but does not take the form for some and , it is possible to construct a sequence of games for which .

The proof is reported in the Appendix. Therein, the counterexample relative to b) is constructed using . In other words our impossibility result holds also for the case of homogeneous populations. This is not in contrast with the result in [2] or [10], because therein the sets were assumed to be simplexes. Here we claim that there exists a set (not a simplex) such that does not converge to .

###### Remark 2.

The previous theorem is of fundamental importance from the standpoint of the system operator, in that it suggests the use of monomial price functions to guarantee the highest achievable efficiency (all Nash equilibria become social optimizers for large ). If different price functions are chosen, it is always possible to construct a problem instance such that all the Nash equilibria are not social optimizers.

## 5 Application to charging of electric vehicles

We consider the problem of charging a fleet of electric vehicles (EVs) in a demand response market [2, 3, 4]. Specifically, we consider a population of electric vehicles. The state of charge of vehicle at time is described by the variable and the evolution of is specified by the discrete-time system , where is the charging control and is the charging efficiency. We assume that the charging control cannot take negative values and that it cannot exceed at time . The final state of charge is constrained to , where is the desired state of charge of agent . Denoting , the individual constraint of agent can be expressed as

 xi∈Xi\vcentcolon=⎧⎪ ⎪⎨⎪ ⎪⎩xi∈Rn∣∣ ∣ ∣∣0≤xit≤~xit,     ∀t=1,…,n∑nt=1xit≥θi|xit+1−xit|≤ri,∀t=1,…,n−1⎫⎪ ⎪⎬⎪ ⎪⎭, (9)

where , with the state of charge at the beginning of the time horizon. Note that the vehicles are thus heterogeneous in the total amount of energy required as well as the time-varying upper bounds (that can be used to model deadlines, availability for charging, etc.), and the ramping constraints . Such constraints satisfy the standing assumption. In the following, we also assume that there exists such that for each and for each , so that Assumption 1 is satisfied. Note that this is without loss of generality in any practical scenario. The cost function of each vehicle represents its electricity bill, which we model as

 Ji(xi,σ(x))=∑nt=1pt(dt+σt(x)κt)xit:=p(σ(x))⊤xi, (10)

where we assumed that the energy price for each time interval depends on the ratio between total consumption and total capacity , where and are the non-EV and EV demand at time divided by and is the total production capacity divided by as in [2, eq. (6)]. To sum up, we define the game as in (2), with and as in (9) and (10) respectively. Let be the vector of charging schedules for the whole population. The social cost of the game is , that is, the overall electricity bill for the sum of non-EV and EV demand. The simulations are conducted on a heterogeneous population and . The vehicles differ in , , drawn according to and , respectively. The upper bound is chosen so that the charge is allowed in a connected interval, with left and right endpoints uniformly randomly distributed. The demand is taken as the typical (non-EV) summer base demand in the United States [2, Figure 1].

Figure 1 shows the PoA computed numerically as a function of the population size. For any fixed , we have randomly chosen games . The worst PoA amongst the realization is plotted in Figure 1 as a function of . We selected as in [2, eq.(25)].1 With this choice, Assumption 3 is met and consequently by Theorem 2 we can guarantee . The simulation results are consistent with it. Observe that, to plot the price of anarchy, we computed the ratio between one Nash equilibrium of the game and the social optimum (instead of considering the worst Nash equilibrium). This choice was imposed by the fact that computing all the Nash equilibria of the game is in general a hard problem (due to the non-monotonicity of the Nash problem, see the footnote). This however does not affect our asymptotic analysis, as for large the Nash equilibrium is unique [15].

## 6 Conclusions

We considered the problem of charging a fleet of heterogeneous electric vehicles as formulated using game theoretic tools. More precisely, we studied the efficiency of the resulting equilibrium allocations, measured by the concept of price of anarchy. We showed that for linear price functions depending on all charging instants, the price of anarchy converges to one as the population of vehicles grow. For games with a generic price function that depends only on the instantaneous demand, we showed that the price of anarchy tends to one, when the price function is a positive pure monomial. In both these cases we provided bounds on the PoA as a function of the population size. Our theoretical findings are corroborated by means of numerical simulations. Proof of Lemma 1
1) The sets are convex and closed by standing assumption; further, for fixed , the functions are linear and thus convex in for all . It follows that (see [15]) a Wardrop equilibrium satisfies for all and for all

 ⎡⎢ ⎢ ⎢⎣p(σ(x\textupW)+d)⋮p(σ(x\textupW)+d)⎤⎥ ⎥ ⎥⎦⊤⎡⎢ ⎢ ⎢⎣x1−x1\textupW⋮xM−xM\textupW⎤⎥ ⎥ ⎥⎦≥0. (11)

Rearranging and dividing by we get that

 p(σ(x\textupW)+d)⊤(1M∑Mj=1xi−1M∑Mj=1xi\textupW)≥0, (12)

or equivalently that is, solves . By standing assumption is strongly monotone and is closed and convex since are so, hence by [16] has a unique solution .
By definition of variational inequality, for any it holds . By definition of , we have . It follows that for any . By definition of , we conclude that (11) must hold for all for all . Consequently, is a Wardrop equilibrium (see [15]).
2) By standing assumption, the set is convex and closed and the function is convex. Hence, a social optimizer satisfies for all

 Unknown environment '% (13)

Note that for all . Consequently, (13) is equivalent to

 [p(σ(x\textupS)+d)+[∇zp(σ(x\textupS)+d)](σ(x\textupS)+d)]⊤(σ(x)−σ(x\textupS))≥0,

that is solves . The remaining claims are proven in a similar fashion to what shown in 1).

Proof of Theorem 1
a) Let be a Wardrop equilibrium. By Lemma 1 part 1, solves . Because of Assumption 2, . Since the two operators and are parallel for each , it follows from the definition of variational inequality that must solve too. Using Lemma 1 part 2 we conclude that must be a social optimizer.
b) By definition and so . Further, assumption 1 and the strong monotonicity of (standing assumption) allow us to use the convergence result of [15, Theorem 1]. That is, for any Nash equilibrium and Wardrop equilibrium of the game , It follows that Since every Wardrop equilibrium is socially optimum (previous point of this proof), one has and thus . The final result regarding the price of anarchy follows from the latter inequality upon dividing both sides by :

 \textupPoAM=J\textupS(σ(x\textupN))J\textupS(σ(x\textupS))=1+c√M−1J\textupS(σ(x\textupS))≤1+c^J√M.

Proof of Theorem 2
a) We first show that any Wardrop equilibrium is a social optimizer. To do so, observe that the function satisfies all the assumptions required by Lemma 1 (see Lemma 3 in the Appendix). Let be a Wardrop equilibrium of . By Lemma 1, solves . Thanks to Assumption 3 and the choice of ,

 F\textupS(z)=(k+1)⎡⎢ ⎢⎣α(z1+d1)k⋮α(zn+dn)k⎤⎥ ⎥⎦=(k+1)F\textupW(z).

Hence solves too. Using Lemma 1 we conclude that must be a social optimizer. The proof is now identical to the proof of Theorem 1, part b), and is not included due to space limitation.
b) If does not take the form for some and , by Lemma 2 there exists a point for which and are not aligned, i.e. for which for all . We intend to construct a sequence of games so that for every in the sequence the unique average at the Wardrop equilibrium is exactly , that is solves , but does not solve . This fact indeed proves, by Lemma 1, that for any game the Wardrop equilibria of are not social minimizers. Since as [15, Theorem 1], one easily concludes that PoA cannot converge to .

In the following we construct a sequence of games with the above mentioned properties. To this end let us define , so that with

 ¯X\coloneqq{¯z+αv1+βv2  α,β∈[0 1]}∩Rn≥0,

where , and , ; see Figure 2. The intuition is that is the component of that lives in the same plane as and and is orthogonal to , so that .

Observe that is the intersection of a bounded and convex set with the positive orthant and thus satisfies the standing assumption and Assumptions 1, 3. It is easy to verify that and that for all , so that solves . Let us pick . Note that since , for small enough belongs to as well and thus to . Then . The inequality is strict because , are neither parallel nor zero (Lemma 2). We conclude that does not solve .

###### Lemma 2.

For , if satisfies the standing assumption, Assumptions 1 and 3, but does not take the form for some and , then there exists such that for all . Moreover, it holds that , .

###### Proof.

Let us consider the first statement. By contradiction, assume there exists such that for all . This implies

 f′(zt+dt)(zt+dt)=(β(z1,…,zn)−1)f(zt+dt), (14)

for all and for all , . By Assumption 3, . Hence one can divide (14) for without loss of generality, and conclude that with for all . For the last condition implies constant. Equation (14) reads as , whose continuously differentiable solutions are all and only . Note that if or , the standing assumptions are not satisfied, while if and we contradicted the assumption that did not take the form for some and . Setting in the previous claim gives . Since , it follows that . ∎

###### Lemma 3.

Suppose that the price function is as in Assumption 3 with , . Then satisfies the standing assumption and Assumption 1.

###### Proof.

Note that is a diagonal matrix with entry in position . Since for all and since is positive by assumption for all , we get that is continuously differentiable and that i.e. that is strongly monotone. Similarly, the Hessian of is a diagonal matrix with entry in position . Note that for all and by assumption for all . Consequently, the Hessian of