An asynchronous, forward-backward, distributed generalized Nash equilibrium seeking algorithm.

# An asynchronous, forward-backward, distributed generalized Nash equilibrium seeking algorithm.

Carlo Cenedese    Giuseppe Belgioioso    Sergio Grammatico    Ming Cao Engineering and Technology Institute Groningen (ENTEG), Faculty of Science and Engineering, University of Groningen, The Netherlands c.cenedese@rug.nl and m.cao@rug.nl. The work of Cenedese, and Cao was supported in part by the European Research Council (ERC-CoG-771687) and the Netherlands Organization for Scientific Research (NWO-vidi-14134). Control System group, TU Eindhoven, 5600 MB Eindhoven, The Netherlands g.belgioioso@tue.nl. Delft Center for Systems and Control, TU Delft, The Netherlands s.grammatico@tudelft.nl. The work of Grammatico was partially supported by NWO, under research projects OMEGA (TOP 613.001.702) and P2P-TALES (ESI-BIDA 647.003.003) and by the ERC under research project COSMOS (ERC-StG 802348).
###### Abstract

In this paper, we propose an asynchronous distributed algorithm for the computation of generalized Nash equilibria in noncooperative games, where the players interact via an undirected communication graph. Specifically, we extend the paper “Asynchronous distributed algorithm for seeking generalized Nash equilibria” by Yi and Pavel: we redesign the asynchronous update rule using auxiliary variables over the nodes rather than over the edges. This key modification renders the algorithm scalable for highly interconnected games. The derived asynchronous algorithm is robust against delays in the communication and it eliminates the idle times between computations, hence modeling a more realistic interaction between players with different update frequencies. We address the problem from an operator-theoretic perspective and design the algorithm via a preconditioned forward-backward splitting. Finally, we numerically simulate the algorithm for the Cournot competition in networked markets.

## I Introduction

### I-a Motivation and literature overview

Noncooperative generalized games over networks is currently a very active research field, due to the spreading of multi-agent network systems in modern society. Such type of games emerge in several application domains, such as smart grids [1, 2], social networks [3] and robotics [4]. In a game setup the players, or agents, have a private and local objective function that depends on the decisions of some other players, which shall be minimized while satisfying both local and global, coupling, constraints. Typically each agent defines its decision, or strategy, based on some local information exchanged with a subset of other agents, called neighbors. One popular notion of solution for these games is a collective equilibrium where no player benefits from changing its strategy, e.g. a generalized Nash equilibrium (GNE). Various authors proposed solutions to this problem [5, 3, 6]. These works propose only synchronous solutions for solving noncooperative games. So, all the agents shall wait until the slowest one in the network completes its update, before starting a new operation. This can slow down the convergence drammatically, especially in large scale and heterogeneous systems. On the other hand, adopting an asynchronous update reduces the idle times, increasing efficiency. In addition, it can also speed up the convergence, facilitate the insertion of new agents in the network and even increse robusteness w.r.t. communication faults [7]. The pioneering work of Bertsekas and Tsitsiklis [8] can be considered the starting point of the literature on parallel asynchronous optimization. During the past years, several asynchronous algorithms for distributed convex optimization were proposed [9, 10, 11, 12], converging under different assumptions. The novel work in [13], provides a simple framework (ARock) to develop a wide range of iterative fixed point algorithms based on nonexpansive operators and it is already adopted in [14] to seek variational GNE seeking under equality constraints and using edge variables.

In this paper, we propose an extension of the work in [14]. Specifically, we consider inequality coupling constraints and use a restricted set of auxiliary variables, namely, associated with the nodes rather than with the edges. Especially this latter upgrade is non-trivial and presents technical challenges in the asynchronous implementation of the algorithm, which we overcome by analyzing the influence of the delayed information on the update of the auxiliary variables. The use of node variables only, rather than edge variables, preserves the scalability of the algorithm, with respect to the number of nodes.

### I-B Structure of the paper

The paper is organized as follows: Section III formalizes the problem setup and introduces the concept of variational v-GNE. In Section IV the iterative algorithm for v-GNE seeking is derived for the synchronous case. The asynchronous counterpart of the algorithm is presented in Section V. Section VI is dedicated to the simulation results for the problem of Cournot competition in networked markets. Section VII ends the paper presenting the conclusions and the outlooks of this work.

## Ii Notation

### Ii-a Basic notation

The set of real, positive, and non-negative numbers are denoted by , , , respectively; . The set of natural numbers is . For a square matrix , its transpose is , is the -th row of the matrix and represents the elements in the row and column . () stands for positive definite (semidefinite) matrix, instead () describes element wise inequality. is the Kronecker product of the matrices and . The identity matrix is denoted by . () is the vector/matrix with only () elements. For , the collective vector is denoted as . describes a block-diagonal matrix with the matrices on the main diagonal. The null space of a matrix is . The Cartesian product of the sets , is .

### Ii-B Operator-theoretic notation

The identity operator is by . The indicator function of is defined as if ; otherwise. The set valued mapping denotes the normal cone to the set , that is if and otherwise. The graph of a set valued mapping is . For a function , define and its subdifferential set-valued mapping, , . The projection operator over a closed set is and it is defined as . A set valued mapping is -Lipschitz continuous with , if for all ; is (strictly) monotone if holds true, and maximally monotone if it does not exist a monotone operator with a graph that strictly contains . Moreover, it is -strongly monotone if it holds . The operator is -averaged (-AVG) with if for all ; is -cocoercive if is -averaged, i.e. firmly nonexpansive (FNE). The resolvent of an operator is .

## Iii Problem Formulation

### Iii-a Mathematical formulation

We consider a set of agents (players), involved in a noncooperative game subject to coupling constraints. Each player has a local decision variable (strategy) that belongs to its private decision set , the vector of all the strategies played is where , and are the decision variables of all the players other than . The aim of each agent is to minimize its local cost function , where , that leads to a coupling between players, due to the dependency on both and the strategy of the other agents in the game. In this work we assume the presence of affine constraints between the agent strategies. These shape the collective feasible decision set

 X:=Ω∩{x∈Rn|Ax≤b}, (1)

where and . Then, the feasible set of each agent reads as

 Xi(x−i):={y∈Ωi|Aiy−bi≤∑j∈N∖{i}bj−Ajxj},

where , and . We note that both the local decision set and how the player is involved in the coupling constraints, i.e. and , are private information, hence will not be accessible to other agents. Assuming affine constraints is common in the literature on noncooperative games [15, 5]. In the following, we introduce some other common assumptions over the aforementioned sets and cost function.

###### Standing Assumption 1 (Convex constraint sets)

For each player , the set is convex, nonempty and compact. The feasible local set satisfies Slater’s constraint qualification. \QEDopen

###### Standing Assumption 2 (Convex and diff. cost functions)

For all , the cost function is continuous, -Lipschitz continuous, continuously differentiable and convex in its first argument. \QEDopen

In compact form, the game between players reads as

 xi∈argminy∈Rnfi(y,x−i)\textups.t.y∈Xi(x−i). (2)

In this paper, we are interested in the generalized Nash equilibia (GNE) of the game in (2).

###### Definition 1 (Generalized Nash equilibrium)

A collective strategy is a GNE if, for each player , it holds

 x∗i∈argminy∈Rnfi(y,x∗−i)\textups.t.y∈Xi(x∗−i). (3)
\QEDopen

### Iii-B Variational GNE

Let us introduce an interesting subset of GNE, the set of so called variational GNE (v-GNE), or normalized equilibrium point, of the game in (2) referring to the fact that all players share a common penalty in order to meet the constraints. This is a refinement of the concept of GNE that has attracted a growing interest in recent years - see [16] and references therein. This set can be rephrased as solutions of a variational inequality (VI), as in [6].

First, we define the pseudo-gradient mapping of the game (2) as

 F(x)=col({∇xifi(xi,x−i)}i∈N), (4)

that gathers all the subdifferentials of the local cost functions of the agents. The following are some standard technical assumptions on , see [17, 18].

###### Standing Assumption 3

The pseudo-gradient in (4) is -Lipschitz continuous and -strongly monotone, for some . \QEDopen

Standing Assumption 2 implies that is a single valued mapping, hence one can define VI() as the problem:

 \textupfindx∗∈X,\textups.t.⟨F(x∗),x−x∗⟩≥0,∀x∈X. (5)

Next, let us define the KKT conditions associated to the game in (2). Due to the convexity assumption, if is a solution of (2), then there exist dual variables , , such that the following inclusion is satisfied:

 (6)

While in general the dual variables can be different, here we focus on the subclass of equilibria sharing a common dual variable, i.e., .

In this case, the KKT conditions for the VI() in (5) (see [6, 19]) read as

 (7)

By (6) and (7), we deduce that every solution of VI() is also a GNE of the game in (2), [6, Th. 3.1(i)]. In addition, if the pair satisfies the KKT conditions in (7), then and the vectors satisfy the KKT conditions for the GNE, i.e. (6) [6, Th. 3.1(ii)].

Note that under Standing Assumptions 13 the set of v-GNE is guaranteed to be a singleton [19, Cor. 2.2.5; Th. 2.3.3].

## Iv Synchronous Distributed GNE Seeking

In this section, we describe the Synchronous Distributed GNE Seeking Algorithm with Node variables (SD-GENO). First, we outline the communication graph supporting the communication between agents, then we derive the algorithm via an operator splitting methodology.

### Iv-a Communication network

The communication between agents is described by an undirected and connected graph where is the set of players and is the set of edges. We define , and . If an agent shares information with , then , then we say that belongs to the neighbours of , i.e., where is the neighbourhood of . Let us label the edges , for . We denote by the incidence matrix, where is equal to (respectively ) if () and otherwise. By construction, . Then, we define (respectively ) as the set of all the indexes of the edges that start from (end in) node , moreover . The node Laplacian of an undirected graph is a symmetric matrix and can be expressed as , [20, Lem. 8.3.2]. In the remainder of the paper, we exploit the fact that the Laplacian matrix is such that and .

### Iv-B Algorithm design

Now, we present a distributed algorithm with convergence guarantees to the unique v-GNE of the game in (2). The KKT system in (6), can be cast in compact form as

 0∈F(x)+Λ⊤λ+NΩ(x)0∈¯b−Λx+NRmN≥0(λ), (8)

where , and .

As highlighted before, for an agent , a solution of its local optimality conditions is given by the strategy and the dual variable . To enforce consensus among the dual variables, hence obtain a v-GNE, we introduce the auxiliary variables , one for every edge of the graph. Defining and using , we augment the inclusion in (8) as

 0∈F(x)+Λ⊤λ+NΩ(x)0∈¯b−Λx+NRmN≥0(λ)+E⊤σ0∈−Eλ. (9)

The variables are used to simplify the analysis, but we will show how we decrease their number to one for each node, increasing the scalability of the algorithm, especially for dense networks.

From an operator theoretic perspective, a solution to (9) can be interpreted as a zero of the sum of two operators, and , defined as

 (10)

In fact, if and only if satisfies (9).

Next, we show that the zeros of are actually the v-GNE of the initial game.

###### Proposition 1

Let and be as in (10). Then the following hold:

1. ,

2. if then satisfies the KKT conditions in (7), hence is the v-GNE for the game in (2). \QEDopen

The proof exploits the property of the incidence matrix of having the same null space of , i.e. , and the assumption that the graph is connected. It can be obtained via an argument analogue to the one used in [5, Th. 4.5], hence we omit it.

The problem of finding the zeros of the sum of two monotone operators is widely studied in literature and a plethora of different splitting method can be used to iteratively solve the problem [21], [22, Ch. 26]. A necessary first step is to prove the monotonicity of the defined operators.

###### Lemma 1

The mappings and in (10) are maximally monotone. Moreover, is -cocoercive. \QEDopen

The splitting method chosen here to find is the preconditioned forward-backward splitting (PFB), which can be applied thanks to the properties stated in Lemma 1.

###### Remark 1

The choice is driven by two main features simplicity and implementability. In fact, the PFB requires only one round of communication between agents at each iteration, minimizing in this way the most demanding operation in multi-agent algorithms, i.e., information sharing.

The iteration of the algorithm takes the form of the so called Krasnosel’skiĭ iteration, namely

 ~ϖk=Tϖkϖk+1=ϖk+η(~ϖk−ϖk) (11)

where , and is the PFB splitting operator

 T=JγΦ−1A∘(Id−γΦ−1B), (12)

where is a step size. The so-called preconditioning matrix is defined as

 Φ:=⎡⎢⎣τ−10−Λ⊤0δ−1ImME−ΛE⊤ε−1⎤⎥⎦ (13)

where , with and is defined in a similar way.

From (12), we note that , indeed , [22, Th. 26.14]. Thus, the zero-finding problem is translated into the fixed point problem for the mapping in (12).

At this point, we calculate from (11) the explicit update rules of the variables. We first focus on the first part of the update, i.e., . It can be rewritten as and finally

 0∈A~ϖk+Bϖk+Φ(~ϖk−ϖk), (14)

here . For ease of notation, we drop the time superscript . By solving the first row block of (14), i.e. , we obtain

 ~x=JNΩ∘(x−τ(F(x)+Λ⊤λ)). (15)

 ~λ=JNRmN≥0∘(λ−ε(Λ(2~x−x)−¯b−E⊤(2~σ−σ))). (16)

The second row block of (14) defines the simple update . We note that in the update (16) of , only is used, hence an agent needs only an aggregated information over the edge variables , to update its state and the dual variables. We exploit this property by replacing the edge variables with . In this way, the auxiliary variables are one for each agent, instead of being one for each edge. Using the property , we cast the update rule of these new auxiliary variables as

 ~zk=zk+δLλkzk+1=zk+η(~zk−zk). (17)

By introducing in (16), we then have

 ~λ=JNRmN≥0∘(λ+ε(Λ(2~x−x)−¯b−2~z+z)). (18)

The next theorem shows that an equilibrium of the new mapping is a v-GNE.

###### Theorem 1

If is a solution to the equations (15), (17) and (18), with , then is a v-GNE. \QEDopen

###### Remark 2

The change of auxiliary variables, from to , is particularly useful in large non-so-sparse networks and it is in general convenient when the number of edges higher than the number of nodes. In fact, for dense networks, we have one auxiliary variable for each player, hence the scalability of the algorithm is preserved.

### Iv-C Synchronous, distributed algorithm with node variables (SD-GENO)

We are now ready to state the update rules defining the synchronous version of the proposed algorithm. The update rule is obtained by gathering (15), (17), (18) and by modifying the second part of (11) via the auxiliary variables :

 ~xk=projΩ(xk−τ(F(xk)+Λ⊤λk))~zk=zk+δLλk~λk=projRmN≥0(λk+ε(Λ(2~xk−xk)−¯b−2~zk+zk))xk+1=xk+η(~xk−xk)zk+1=zk+η(~zk−zk)λk+1=λk+η(~λk−λk), (19)

The convergence of Algorithm 1 to the v-GNE of the game in (2) is guaranteed by the following theorem.

###### Theorem 2

Let , , , such that and . Then, Algorithm 1 converges to the v-GNE of the game in (2). \QEDopen

## V Asynchronous Distributed Algorithm

In this section, we present the main contribution of the paper, the Asynchronous Distributed GNE Seeking Algorithm with Node variables (AD-GENO), namely, the asynchronous counterpart of Algorithm 1. As in the previous section, we first define a preliminary version of the algorithm using the edge auxiliary variables , and then we derive the final formulation via the variable . To achieve an asynchronous update of the agent variables, we adopt the “ARock” framework [13].

### V-a Algorithm design

We modify the update rule in (11) to describe the asynchronism, in the local update of the agent , as follows

 ϖk+1=ϖk+ηΥi(Tϖk−ϖk), (20)

where is a real diagonal matrix of dimension , where the element is if the -th element of is an element of and otherwise.We assume that the choice of which agent performs the update at iteration is ruled by an i.i.d. random variable , that takes values in . Given a discrete probability distribution , let , . Therefore, the formulation in (20) becomes

 ϖk+1=ϖk+ηζk(Tϖk−ϖk). (21)

We also consider the possibility of delayed information, namely the update (21) can be performed with outdated values of . We refer to [13, Sec. 1] for a more complete overview on the topic. Due to the structure of the , the update of , and are performed at the same moment, hence they share the same delay at .

We denote the vector of possibly delayed information at time as , hence the reformulation of (21) reads as

 ϖk+1=ϖk+ηζk(T−Id)^ϖk. (22)

Now, we impose that the maximum delay is uniformly bounded.

###### Standing Assumption 4 (Bounded maximum delay)

The delays are uniformly upper bounded, i.e. there exists such that . \QEDopen

From the computational perspective, we assume that each player has a public and a private memory. The first stores the information obtained by the neighbours . The private is instead used during the update of at time and it is an unchangeable copy of the public memory at iteration . The local update rules in Algorithm 2 are obtained similarly to Sec. IV-B for SD-GENO, hence by using the definition of . The obtained algorithm resembles ADAGNES in [14, Alg. 1], therefore we name it E-ADAGNES.

The convergence of the update (22) is proven relying on the theoretical results provided in [13] for the Krasnosel’skiĭ asynchronous iteration.

###### Theorem 3

Let , where and . Then, the sequence defined by Algorithm 2 converges to the v-GNE of the game in (2) almost surely. \QEDopen

### V-B Asynchronous, distributed algorithm with node variables (AD-GENO)

We complete the technical part of the paper by performing the change from auxiliary variables over the edges to variables over the nodes, attaining in this way the final formulation of our proposed algorithm. With Algorithm 2 as starting point, we show that this change does not affect the dynamics of the pair , thus preserving the convergence.

However, in this case, we need to introduce an extra variable for each node , i.e., . This is an aggregate information that groups all the changes of the neighbours dual variables from the previous update of to the present iteration. We highlight that these variables are updated during the writing phase of the neighbours, therefore they do not require extra communications between the agents.

###### Remark 3

The need for arises from the different update frequency between and . Therefore, we cannot characterize the dynamics of , if we define only.

Algorithm 3 presents AD-GENO, where are rigorously defined.

The convergence of AD-GENO is proven by the following theorem. Essentially, we show that introducing does not change the dynamics of .

###### Theorem 4

Let with and . Then, the sequence defined by Algorithm 3 converges to the v-GNE of the game in (2) almost surely. \QEDopen

## Vi Simulation

This section presents the implementation of AD-GENO to solve a network Cournot game, that models the interaction of companies competing over markets. The problem is widely studied and we adopt a set-up similar to the one in [23, 5]. We chose companies, each operating strategies, i.e., , . It ranges in , where ans its elements are randomly drawn from . The markets are , named , , and . In Figure 1, an edge between a company and a market is drawn, if at least one of that player’s strategies is applied to that market. Two companies are neighbors if they share a market.

The constraint matrix is and the columns of have a nonzero element in position if the -th strategy of player is applied to market . The nonzero values are randomly chosen from . The elements of are the markets’ maximal capacities and are randomly chosen from . The arising inequality coupling constraint is . The local cost function is , where is the cost of playing a certain strategy and the price obtained by the market. We define the markets price as a linear function , where and is a diagonal matrix, the values of their elements are randomly chosen respectively from and . The cost function is quadratic , where the elements of the diagonal matrix and the vector are randomly drawn respectively from and .

We propose two setups, the case of communication over a ring graph with alphabetic order and the case of random communication (in Figure 2, respectively the blue and red trajectories). In the latter, we only ensure that every iterations all the agents performed a similar number of updates. The edges of the graph are arbitrarily oriented. We assume that the agents update with uniform probability, i.e., . The step sizes in AD-GENO are randomly chosen, the first from and the others from , in order to ensure and . The maximum delay is assumed , therefore in (22) is where each is randomly chosen from .

The results of the simulations are shown in Figure 2. In particular, Figure (a)a presents the convergence of the collective strategy to the v-GNE . Furthermore, Figure (b)b highlights the convergence of the Lagrangian multipliers to consensus. We noticed that a simple update sequence, as the alphabetically ordered one, leads to a faster convergence than a random one. In general, the more the agents’ updates are well mixed the faster the algorithm converge.

## Vii Conclusion

This work propose a variant of the forward-backward splitting algorithm to solve generalized Nash equilibrium problems via asynchronous and distributed information exchange, that is robust to communication delays. A change of variables based on the node Laplacian matrix of the information-exchange graph allows one to preserve the scalability of the solution algorithm in the number of nodes (as opposed to the number of edges). Full theoretical and numerical comparison between the proposed solution algorithm and that in [14] is left as future work. Another interesting topic is the adaptation of the algorithm to the case of changing graph topology, in fact the independence from the edge variables makes this approach more suitable to address this problem.

### Proof of Lemma 1

It follows as in [5, Proof of Th. 4.5]. \QED

### Proof of Theorem 1

Given the equilibrium point , (17) evaluated in the equilibrium reduces to