An asynchronous, forward-backward, distributed generalized Nash equilibrium seeking algorithm.
In this paper, we propose an asynchronous distributed algorithm for the computation of generalized Nash equilibria in noncooperative games, where the players interact via an undirected communication graph. Specifically, we extend the paper “Asynchronous distributed algorithm for seeking generalized Nash equilibria” by Yi and Pavel: we redesign the asynchronous update rule using auxiliary variables over the nodes rather than over the edges. This key modification renders the algorithm scalable for highly interconnected games. The derived asynchronous algorithm is robust against delays in the communication and it eliminates the idle times between computations, hence modeling a more realistic interaction between players with different update frequencies. We address the problem from an operator-theoretic perspective and design the algorithm via a preconditioned forward-backward splitting. Finally, we numerically simulate the algorithm for the Cournot competition in networked markets.
I-a Motivation and literature overview
Noncooperative generalized games over networks is currently a very active research field, due to the spreading of multi-agent network systems in modern society. Such type of games emerge in several application domains, such as smart grids [1, 2], social networks  and robotics . In a game setup the players, or agents, have a private and local objective function that depends on the decisions of some other players, which shall be minimized while satisfying both local and global, coupling, constraints. Typically each agent defines its decision, or strategy, based on some local information exchanged with a subset of other agents, called neighbors. One popular notion of solution for these games is a collective equilibrium where no player benefits from changing its strategy, e.g. a generalized Nash equilibrium (GNE). Various authors proposed solutions to this problem [5, 3, 6]. These works propose only synchronous solutions for solving noncooperative games. So, all the agents shall wait until the slowest one in the network completes its update, before starting a new operation. This can slow down the convergence drammatically, especially in large scale and heterogeneous systems. On the other hand, adopting an asynchronous update reduces the idle times, increasing efficiency. In addition, it can also speed up the convergence, facilitate the insertion of new agents in the network and even increse robusteness w.r.t. communication faults . The pioneering work of Bertsekas and Tsitsiklis  can be considered the starting point of the literature on parallel asynchronous optimization. During the past years, several asynchronous algorithms for distributed convex optimization were proposed [9, 10, 11, 12], converging under different assumptions. The novel work in , provides a simple framework (ARock) to develop a wide range of iterative fixed point algorithms based on nonexpansive operators and it is already adopted in  to seek variational GNE seeking under equality constraints and using edge variables.
In this paper, we propose an extension of the work in . Specifically, we consider inequality coupling constraints and use a restricted set of auxiliary variables, namely, associated with the nodes rather than with the edges. Especially this latter upgrade is non-trivial and presents technical challenges in the asynchronous implementation of the algorithm, which we overcome by analyzing the influence of the delayed information on the update of the auxiliary variables. The use of node variables only, rather than edge variables, preserves the scalability of the algorithm, with respect to the number of nodes.
I-B Structure of the paper
The paper is organized as follows: Section III formalizes the problem setup and introduces the concept of variational v-GNE. In Section IV the iterative algorithm for v-GNE seeking is derived for the synchronous case. The asynchronous counterpart of the algorithm is presented in Section V. Section VI is dedicated to the simulation results for the problem of Cournot competition in networked markets. Section VII ends the paper presenting the conclusions and the outlooks of this work.
Ii-a Basic notation
The set of real, positive, and non-negative numbers are denoted by , , , respectively; . The set of natural numbers is . For a square matrix , its transpose is , is the -th row of the matrix and represents the elements in the row and column . () stands for positive definite (semidefinite) matrix, instead () describes element wise inequality. is the Kronecker product of the matrices and . The identity matrix is denoted by . () is the vector/matrix with only () elements. For , the collective vector is denoted as . describes a block-diagonal matrix with the matrices on the main diagonal. The null space of a matrix is . The Cartesian product of the sets , is .
Ii-B Operator-theoretic notation
The identity operator is by . The indicator function of is defined as if ; otherwise. The set valued mapping denotes the normal cone to the set , that is if and otherwise. The graph of a set valued mapping is . For a function , define and its subdifferential set-valued mapping, , . The projection operator over a closed set is and it is defined as . A set valued mapping is -Lipschitz continuous with , if for all ; is (strictly) monotone if holds true, and maximally monotone if it does not exist a monotone operator with a graph that strictly contains . Moreover, it is -strongly monotone if it holds . The operator is -averaged (-AVG) with if for all ; is -cocoercive if is -averaged, i.e. firmly nonexpansive (FNE). The resolvent of an operator is .
Iii Problem Formulation
Iii-a Mathematical formulation
We consider a set of agents (players), involved in a noncooperative game subject to coupling constraints. Each player has a local decision variable (strategy) that belongs to its private decision set , the vector of all the strategies played is where , and are the decision variables of all the players other than . The aim of each agent is to minimize its local cost function , where , that leads to a coupling between players, due to the dependency on both and the strategy of the other agents in the game. In this work we assume the presence of affine constraints between the agent strategies. These shape the collective feasible decision set
where and . Then, the feasible set of each agent reads as
where , and . We note that both the local decision set and how the player is involved in the coupling constraints, i.e. and , are private information, hence will not be accessible to other agents. Assuming affine constraints is common in the literature on noncooperative games [15, 5]. In the following, we introduce some other common assumptions over the aforementioned sets and cost function.
Standing Assumption 1 (Convex constraint sets)
For each player , the set is convex, nonempty and compact. The feasible local set satisfies Slater’s constraint qualification. \QEDopen
Standing Assumption 2 (Convex and diff. cost functions)
For all , the cost function is continuous, -Lipschitz continuous, continuously differentiable and convex in its first argument. \QEDopen
In compact form, the game between players reads as
In this paper, we are interested in the generalized Nash equilibia (GNE) of the game in (2).
Definition 1 (Generalized Nash equilibrium)
A collective strategy is a GNE if, for each player , it holds
Iii-B Variational GNE
Let us introduce an interesting subset of GNE, the set of so called variational GNE (v-GNE), or normalized equilibrium point, of the game in (2) referring to the fact that all players share a common penalty in order to meet the constraints. This is a refinement of the concept of GNE that has attracted a growing interest in recent years - see  and references therein. This set can be rephrased as solutions of a variational inequality (VI), as in .
First, we define the pseudo-gradient mapping of the game (2) as
Standing Assumption 3
The pseudo-gradient in (4) is -Lipschitz continuous and -strongly monotone, for some . \QEDopen
Standing Assumption 2 implies that is a single valued mapping, hence one can define VI() as the problem:
Next, let us define the KKT conditions associated to the game in (2). Due to the convexity assumption, if is a solution of (2), then there exist dual variables , , such that the following inclusion is satisfied:
While in general the dual variables can be different, here we focus on the subclass of equilibria sharing a common dual variable, i.e., .
By (6) and (7), we deduce that every solution of VI() is also a GNE of the game in (2), [6, Th. 3.1(i)]. In addition, if the pair satisfies the KKT conditions in (7), then and the vectors satisfy the KKT conditions for the GNE, i.e. (6) [6, Th. 3.1(ii)].
Iv Synchronous Distributed GNE Seeking
In this section, we describe the Synchronous Distributed GNE Seeking Algorithm with Node variables (SD-GENO). First, we outline the communication graph supporting the communication between agents, then we derive the algorithm via an operator splitting methodology.
Iv-a Communication network
The communication between agents is described by an undirected and connected graph where is the set of players and is the set of edges. We define , and . If an agent shares information with , then , then we say that belongs to the neighbours of , i.e., where is the neighbourhood of . Let us label the edges , for . We denote by the incidence matrix, where is equal to (respectively ) if () and otherwise. By construction, . Then, we define (respectively ) as the set of all the indexes of the edges that start from (end in) node , moreover . The node Laplacian of an undirected graph is a symmetric matrix and can be expressed as , [20, Lem. 8.3.2]. In the remainder of the paper, we exploit the fact that the Laplacian matrix is such that and .
Iv-B Algorithm design
where , and .
As highlighted before, for an agent , a solution of its local optimality conditions is given by the strategy and the dual variable . To enforce consensus among the dual variables, hence obtain a v-GNE, we introduce the auxiliary variables , one for every edge of the graph. Defining and using , we augment the inclusion in (8) as
The variables are used to simplify the analysis, but we will show how we decrease their number to one for each node, increasing the scalability of the algorithm, especially for dense networks.
From an operator theoretic perspective, a solution to (9) can be interpreted as a zero of the sum of two operators, and , defined as
In fact, if and only if satisfies (9).
Next, we show that the zeros of are actually the v-GNE of the initial game.
The proof exploits the property of the incidence matrix of having the same null space of , i.e. , and the assumption that the graph is connected. It can be obtained via an argument analogue to the one used in [5, Th. 4.5], hence we omit it.
The problem of finding the zeros of the sum of two monotone operators is widely studied in literature and a plethora of different splitting method can be used to iteratively solve the problem , [22, Ch. 26]. A necessary first step is to prove the monotonicity of the defined operators.
The mappings and in (10) are maximally monotone. Moreover, is -cocoercive. \QEDopen
The splitting method chosen here to find is the preconditioned forward-backward splitting (PFB), which can be applied thanks to the properties stated in Lemma 1.
The choice is driven by two main features simplicity and implementability. In fact, the PFB requires only one round of communication between agents at each iteration, minimizing in this way the most demanding operation in multi-agent algorithms, i.e., information sharing.
The iteration of the algorithm takes the form of the so called Krasnosel’skiĭ iteration, namely
where , and is the PFB splitting operator
where is a step size. The so-called preconditioning matrix is defined as
where , with and is defined in a similar way.
At this point, we calculate from (11) the explicit update rules of the variables. We first focus on the first part of the update, i.e., . It can be rewritten as and finally
here . For ease of notation, we drop the time superscript . By solving the first row block of (14), i.e. , we obtain
The third row block of (14) instead reads as that leads to
The second row block of (14) defines the simple update . We note that in the update (16) of , only is used, hence an agent needs only an aggregated information over the edge variables , to update its state and the dual variables. We exploit this property by replacing the edge variables with . In this way, the auxiliary variables are one for each agent, instead of being one for each edge. Using the property , we cast the update rule of these new auxiliary variables as
By introducing in (16), we then have
The next theorem shows that an equilibrium of the new mapping is a v-GNE.
The change of auxiliary variables, from to , is particularly useful in large non-so-sparse networks and it is in general convenient when the number of edges higher than the number of nodes. In fact, for dense networks, we have one auxiliary variable for each player, hence the scalability of the algorithm is preserved.
Iv-C Synchronous, distributed algorithm with node variables (SD-GENO)
We are now ready to state the update rules defining the synchronous version of the proposed algorithm. The update rule is obtained by gathering (15), (17), (18) and by modifying the second part of (11) via the auxiliary variables :
See also Algorithm 1, for the local updates.
V Asynchronous Distributed Algorithm
In this section, we present the main contribution of the paper, the Asynchronous Distributed GNE Seeking Algorithm with Node variables (AD-GENO), namely, the asynchronous counterpart of Algorithm 1. As in the previous section, we first define a preliminary version of the algorithm using the edge auxiliary variables , and then we derive the final formulation via the variable . To achieve an asynchronous update of the agent variables, we adopt the “ARock” framework .
V-a Algorithm design
We modify the update rule in (11) to describe the asynchronism, in the local update of the agent , as follows
where is a real diagonal matrix of dimension , where the element is if the -th element of is an element of and otherwise.We assume that the choice of which agent performs the update at iteration is ruled by an i.i.d. random variable , that takes values in . Given a discrete probability distribution , let , . Therefore, the formulation in (20) becomes
We also consider the possibility of delayed information, namely the update (21) can be performed with outdated values of . We refer to [13, Sec. 1] for a more complete overview on the topic. Due to the structure of the , the update of , and are performed at the same moment, hence they share the same delay at .
We denote the vector of possibly delayed information at time as , hence the reformulation of (21) reads as
Now, we impose that the maximum delay is uniformly bounded.
Standing Assumption 4 (Bounded maximum delay)
The delays are uniformly upper bounded, i.e. there exists such that . \QEDopen
From the computational perspective, we assume that each player has a public and a private memory. The first stores the information obtained by the neighbours . The private is instead used during the update of at time and it is an unchangeable copy of the public memory at iteration . The local update rules in Algorithm 2 are obtained similarly to Sec. IV-B for SD-GENO, hence by using the definition of . The obtained algorithm resembles ADAGNES in [14, Alg. 1], therefore we name it E-ADAGNES.
V-B Asynchronous, distributed algorithm with node variables (AD-GENO)
We complete the technical part of the paper by performing the change from auxiliary variables over the edges to variables over the nodes, attaining in this way the final formulation of our proposed algorithm. With Algorithm 2 as starting point, we show that this change does not affect the dynamics of the pair , thus preserving the convergence.
However, in this case, we need to introduce an extra variable for each node , i.e., . This is an aggregate information that groups all the changes of the neighbours dual variables from the previous update of to the present iteration. We highlight that these variables are updated during the writing phase of the neighbours, therefore they do not require extra communications between the agents.
The need for arises from the different update frequency between and . Therefore, we cannot characterize the dynamics of , if we define only.
Algorithm 3 presents AD-GENO, where are rigorously defined.
The convergence of AD-GENO is proven by the following theorem. Essentially, we show that introducing does not change the dynamics of .
This section presents the implementation of AD-GENO to solve a network Cournot game, that models the interaction of companies competing over markets. The problem is widely studied and we adopt a set-up similar to the one in [23, 5]. We chose companies, each operating strategies, i.e., , . It ranges in , where ans its elements are randomly drawn from . The markets are , named , , and . In Figure 1, an edge between a company and a market is drawn, if at least one of that player’s strategies is applied to that market. Two companies are neighbors if they share a market.
The constraint matrix is and the columns of have a nonzero element in position if the -th strategy of player is applied to market . The nonzero values are randomly chosen from . The elements of are the markets’ maximal capacities and are randomly chosen from . The arising inequality coupling constraint is . The local cost function is , where is the cost of playing a certain strategy and the price obtained by the market. We define the markets price as a linear function , where and is a diagonal matrix, the values of their elements are randomly chosen respectively from and . The cost function is quadratic , where the elements of the diagonal matrix and the vector are randomly drawn respectively from and .
We propose two setups, the case of communication over a ring graph with alphabetic order and the case of random communication (in Figure 2, respectively the blue and red trajectories). In the latter, we only ensure that every iterations all the agents performed a similar number of updates. The edges of the graph are arbitrarily oriented. We assume that the agents update with uniform probability, i.e., . The step sizes in AD-GENO are randomly chosen, the first from and the others from , in order to ensure and . The maximum delay is assumed , therefore in (22) is where each is randomly chosen from .
The results of the simulations are shown in Figure 2. In particular, Figure (a)a presents the convergence of the collective strategy to the v-GNE . Furthermore, Figure (b)b highlights the convergence of the Lagrangian multipliers to consensus. We noticed that a simple update sequence, as the alphabetically ordered one, leads to a faster convergence than a random one. In general, the more the agents’ updates are well mixed the faster the algorithm converge.
This work propose a variant of the forward-backward splitting algorithm to solve generalized Nash equilibrium problems via asynchronous and distributed information exchange, that is robust to communication delays. A change of variables based on the node Laplacian matrix of the information-exchange graph allows one to preserve the scalability of the solution algorithm in the number of nodes (as opposed to the number of edges). Full theoretical and numerical comparison between the proposed solution algorithm and that in  is left as future work. Another interesting topic is the adaptation of the algorithm to the case of changing graph topology, in fact the independence from the edge variables makes this approach more suitable to address this problem.
Proof of Lemma 1
It follows as in [5, Proof of Th. 4.5]. \QED
Proof of Theorem 1
Given the equilibrium point , (17) evaluated in the equilibrium reduces to