# Dynamic Coalitional TU Games: Distributed Bargaining among Players’ Neighbors

###### Abstract

We consider a sequence of transferable utility (TU) games where, at each time, the characteristic function is a random vector with realizations restricted to some set of values. The game differs from other ones in the literature on dynamic, stochastic or interval valued TU games as it combines dynamics of the game with an allocation protocol for the players that dynamically interact with each other. The protocol is an iterative and decentralized algorithm that offers a paradigmatic mathematical description of negotiation and bargaining processes. The first part of the paper contributes to the definition of a robust (coalitional) TU game and the development of a distributed bargaining protocol. We prove the convergence with probability 1 of the bargaining process to a random allocation that lies in the core of the robust game under some mild conditions on the underlying communication graphs. The second part of the paper addresses the more general case where the robust game may have empty core. In this case, with the dynamic game we associate a dynamic average game by averaging over time the sequence of characteristic functions. Then, we consider an accordingly modified bargaining protocol. Assuming that the sequence of characteristic functions is ergodic and the core of the average game has a nonempty relative interior, we show that the modified bargaining protocol converges with probability 1 to a random allocation that lies in the core of the average game.

## 1 Introduction

Coalitional games with transferable utilities (TU) have been introduced by von Neumann and Morgenstern [VM]. A coalitional TU game constitutes of a set of players, who can form coalitions, and a characteristic function that provides a value for each coalition. The value of a coalition can be thought of as a monetary value that can be divided among the members of the coalition according to some appropriate fairness allocation rule.

Coalitional TU games have been used to model cooperation in supply chain or inventory management applications [Drechsel10, HDS00], as well as in network flow applications to coordinate flows of goods, materials, or resources between different production/distribution sites [BBP10]. Recently, coalitional games have also sparked much interest in communication networks. We refer an interested reader to tutorial [SHDHB09] for an in-depth discussion on potential applications in this field.

In this paper, we consider a sequence of coalitional TU games for a finite set of players. We assume that the game is played repeatedly over time thus generating a sequence of time varying characteristic functions. We refer to such a repeated game as dynamic coalitional TU game. In addition, we assume that a player need not necessarily observe the allocations of all the other players, but he can only observe the allocations of his neighbors, where the neighbors may change in time.

In this setting, we consider robust and average TU games, and for each of these games we propose an allocation process to reach a solution of the dynamic game. More specifically, we formulate a dynamic bargaining process as a dynamically changing constrained problem. Iterations are simply bargaining steps free of any concrete reward allocation. We assume that the reward allocation occurs only once at the end of the bargaining process, when the players have agreed on how to allocate rewards among themselves. Our main objective is to explore distributed agreement on solutions in the core of the game, where the players interact only with their neighbors.

In particular, we consider bargaining protocols assuming that each player obeys rationality and efficiency by deciding on an allocation vector which satisfies the value constraints of all the coalitions that include player . This set is termed bounding set of player . At every iteration, a player observes the allocations of some of his neighbors. This is modeled using a directed graph with the set of players as the vertex set and a time-varying edge set composed of directed links whenever player observes the allocation vector proposed by player at time . We refer to this directed graph as players’ neighbor-graph. Given a player’s neighbor-graph, each player negotiates allocations by adjusting the allocations he received from his neighbors through weight assignments. As the balanced allocation may violate his rationality constraints (it lies outside player ’s bounding set), the player selects a new allocation by projecting the balanced allocation on his bounding set. We propose such bargaining protocols for solving both the robust and the average TU game. For each of these games, we use some mild assumptions on the connectivity of the players’ neighbor-graph and the weights that the players use when balancing their own allocations with the neighbors’ allocations. Assuming that the core of the robust game is nonempty, we show that our bargaining protocol converges with probability 1 to a common (random) allocation in the core. In the case when the core of the robust game is empty, we consider an average game that can provide a meaningful solution under some conditions on the sequence of the characteristic functions. Specifically, in this case, we consider a dynamic average game by averaging over time the sequence of characteristic functions. We then modify accordingly the bounding set and the associated bargaining process by using the dynamic average game. This means that the value constraints are defined by using the time-averaged sequence rather than the sequence of random characteristic functions. Under the assumptions that the time-averaged sequence is ergodic and that the core of the average game has a nonempty relative interior, we show that the players’ allocations generated by our bargaining protocol converge with probability 1 to a common (random) allocation in the core of the average game.

The main contributions of this paper are in the two-fold dynamic aspect of the games, and in the development and the analysis of distributed bargaining process among the players. The novelty in the dynamic aspect is in the consideration of games with time-varying characteristic functions and the use of time-varying local players interactions.

The work in this paper is related to stochastic cooperative games [G77, SB99, TBT03]. However, we deviate from this stochastic framework in a number of aspects among which are the existence of a local players interactions captured by players’ neighbor-graph and the presence of multiple iterations in the bargaining process, and the consideration of robust game. Bringing dynamical aspects into the framework of coalitional TU games is an element in common with papers [FP00, H75]. However, unlike [FP00, H75], the values of the coalitions in this paper are realized exogenously and no relation is assumed between consecutive samples. Dynamic robust TU games have been considered in [BR10] and [BT09] but for a continuous time setting in the former work and for a centralized allocation process in the latter one. Convergence of allocation processes is a main topic in [C98, L02]. There, rewards are allocated by a game designer repeatedly in a centralized manner and based on the current excess rewards of the coalitions (accumulated reward up to current time minus the value of the coalition). Our approach, however, differs from that of [C98, L02] as we resort to a decentralized scheme where the allocation process is the result of a bargaining process among the players with local interactions. Convergence of bargaining processes has also been explored under dynamic coalition formation [AS02] for a different dynamic model, where players decide both on which coalition to address and what payoff to announce. Coalitions form when announced payoffs are all met for all players in the coalition. Our bargaining procedure is different as it produces an agreement resulting in the formation of grand coalition, which is stable with respect to any sub-coalitions.

The work in this paper is also related to the literature on agreement among multiple agents, where an underlying communication graph for the agents and balancing weights have been used with some variations [Tsitsiklis84, NOOT09] to reach an agreement on common decision variable, as well as in [Nedic09, NOP10, Sundhar08c, Sundhar09] for distributed multi-agent optimization.

This paper is organized as follows. In Section 2, we introduce the dynamic TU game, the robust game and the bargaining protocol for this game. We then motivate the game and give some preliminary results. In Section 3, we prove the convergence of the bargaining protocol to a point in the core of the robust game with probability 1. In Section 4, we introduce the dynamic average game and the modified bargaining protocol. We also prove the convergence of this protocol to an allocation in the core of the average game with probability 1. In Section 5, we report some numerical simulations to illustrate our theoretical study, and we conclude in Section 6.

Notation. We view vectors as columns. For a vector , we use or to denote its th coordinate component. For two vectors and , we use () to denote () for all coordinate indices . We let denote the transpose of a vector , and denote its Euclidean norm. An matrix is row-stochastic if the matrix has nonnegative entries and for all . For a matrix , we use or to denote its th entry. A matrix is doubly stochastic if both and its transpose are row-stochastic. Given two sets and , we write to denote that is a proper subset of . We use for the cardinality of a given finite set .

We write to denote the projection of a vector on a set , and we write for the distance from to , i.e., and , respectively. Given a set and a scalar , the set is defined by . Given two sets , the set sum is defined by . Given a set of players and a function defined for each nonempty coalition , we write to denote the transferable utility (TU) game with the players’ set and the characteristic function . We let be the value of the characteristic function associated with a nonempty coalition . Given a TU game , we use to denote the core of the game,

## 2 Dynamic TU Game and Robust Game

In this section, we discuss the main components of a dynamic TU game and a robust game. We state some basic assumptions on these games and introduce a bargaining process conducted by the players to reach an agreement on their allocations. We also provide some examples of the TU games motivating our development and establish some basic results that we use later on in the convergence analysis of the bargaining process.

### 2.1 Problem Formulation and Bargaining Process

Consider a set of players and the set of all (nonempty) coalitions arising among these players. Let be the number of possible coalitions. We assume that the time is discrete and use to index the time slots.

We consider a dynamic TU game, denoted , where is a sequence of characteristic functions. Thus, in the dynamic TU game , the players are involved in a sequence of instantaneous TU games whereby, at each time , the instantaneous TU game is with for all . Further, we let denote the value assigned to a nonempty coalition in the instantaneous game . In what follows, we deal with dynamic TU games where each characteristic function is a random vector with realizations restricted to some set of values. We will consider the robust TU game in this section and Section 3, while in Section 4 we deal with the average TU game.

In our development of robust TU game, we impose restrictions on each realization of for every . Specifically, we assume that the grand coalition value is deterministic for every , while the values of the other coalitions have a common upper bound. These conditions are formally stated in the following assumption.

###### Assumption 1

There exists such that for all ,

We refer to the game as robust game. In the first part of this paper, we rely on the assumption that the robust game has a nonempty core.

###### Assumption 2

The core of the robust game is not empty.

An immediate consequence of Assumptions 1 and 2 is that the core of the instantaneous game is always not empty. This follows from the fact that for any satisfying and for , and the assumption that the core is not empty.

Throughout the paper, we assume that each player is rational and efficient. This translates to each player choosing his allocation vector within the set of allocations satisfying value constraints of all the coalitions that include player . This set is referred to as the bounding set of player and, for a generic game , it is given by

Note that each is polyhedral.

In what follows, we find it suitable to represent the bounding sets and the core in alternative equivalent forms. For each nonempty coalition , let be the incidence vector for the coalition , i.e., the vector with the coordinates given by

Then, the bounding sets and the core can be represented as follows:

(1) |

(2) |

Furthermore, observe that the core of the game coincides with the intersection of the bounding sets of all players , i.e.,

(3) |

We now discuss the bargaining protocol where repeatedly over time each player submits an allocation vector that the player would agree on. The allocation vector proposed by player at time is denoted by , where the th component represents the amount that player would assign to player . To simplify the notation in the dynamic game , we let denote the bounding set of player for the instantaneous game , i.e., for all and ,

(4) |

We assume that each player may observe the allocations of a subset of the other players at any time, which are termed as the neighbors of the player. The players and their neighbors at time can be represented by a directed graph , with the vertex set and the set of directed links. A link exists if player is a neighbor of player at time . We always assume that for all , which is natural since every player can always access its own allocation vector. We refer to graph as a neighbor-graph at time . In the graph , a player is a neighbor of player (i.e., ) only if player can observe the allocation vector of player at time . Figure 1 illustrates how the players’ neighbor-graph may look at two time instances.

Given the players’ neighbor-graph , each player negotiates allocations by averaging his allocation and the allocations he received from his neighbors. More precisely, at time , the bargaining process for each player involves the player’s individual bounding set , its own allocation and the observed allocations of some of his neighbors . Formally, we let be the set of neighbors of player at time (including himself), i.e.,

With this notation, the bargaining process is given by:

(5) |

where is a scalar weight that player assigns to the proposed allocation of player and is the projection onto the player bounding set . The initial allocations are selected randomly and independently of .

The bargaining process in (5) can be written more compactly by introducing zero weights for players whose allocations are not available to player at time . Specifically by defining for all and all , we have the following equivalent representation of the bargaining process:

(6) |

Here, for all while for .

We now discuss the specific assumptions on the weights and the players’ neighbor-graph that we will rely on. We let be the weight matrix with entries . We will use the following assumption for the weight matrices.

###### Assumption 3

Each matrix is doubly stochastic with positive diagonal. Furthermore, there exists a scalar such that

In view of the construction of matrices , we see that for and perhaps for some players that are neighbors of player . The requirement that the positive weights are uniformly bounded away from zero is imposed to ensure that the information from each player diffuses to his neighbors in the network persistently in time. The requirement on the doubly stochasticity of the weights is used to ensure that in a long run each player has equal influence on the limiting allocation vector.

It is natural to expect that the connectivity of the players’ neighbor-graphs impacts the bargaining process. At any time, the instantaneous graph need not be connected. However, for the proper behavior of the bargaining process, the union of the graphs over a period of time is assumed to be connected, as given in the following assumption.

###### Assumption 4

There is an integer such that the graph is strongly connected for every .

Assumptions 3 and 4 together guarantee that the players communicate sufficiently often to ensure that the information of each player is persistently diffused over the network in time to reach every other player. Under these assumptions, we will study the dynamic bargaining process in (6). We want to provide conditions under which the process converges to an allocation in the core of the robust game. Before this, we provide some motivating examples in the following section.

### 2.2 Motivations

Dynamic coalitional games capture coordination in a number of network flow applications. Network flows model flow of goods, materials, or other resources between different production/distribution sites [BBP10]. We next provide two examples. The first one is more practically oriented and describes a supply chain application [Drechsel10]. The second one is more theoretical and re-casts the problem at hand in terms of controlled flows over hyper-graphs.

#### 2.2.1 Supply chain

A single warehouse serves a number of retailers , , each one facing a demand unknown but bounded by pre assigned values and at any time period . After demand has been realized, retailer must choose to either fulfill the demand or not. The retailers do not hold any private inventory and, therefore, if they wish to fulfill their demands, they must reorder goods from the central warehouse. Retailers benefit from joint reorders as they may share the total transportation cost (this cost could also be time and/or players dependent). In particular, if retailer “plays” individually, the cost of reordering coincides with the full transportation cost . Actually, when necessary a single truck will serve only him and get back to the warehouse. This is illustrated by the dashed path in the network of Figure 2(a). The cost of not reordering is the cost of the unfulfilled demand .

If two or more retailers “play” in a coalition, they agree on a joint decision (“everyone reorders” or “no one reorders”). The cost of reordering for the coalition also equals the total transportation cost that must be shared among the retailers. In this case, when necessary a single truck will serve all retailers in the coalition and get back to the warehouse. This is illustrated, with reference to coalition by the dashed path in Figure 2(b). A similar comment applies to the coalition and the path in Figure 2(c). The cost of not reordering is the sum of the unfulfilled demands of all retailers. How the players will share the cost is a part of the solution generated by the bargaining process.

The cost scheme can be captured by a game with the set of players where the cost of a nonempty coalition is given by

Note that the bounds on the demand reflect into the bounds on the cost as follows: for all nonempty and ,

(7) |

To complete the derivation of the coalitions’ values we need to compute the cost savings of a coalition as the difference between the sum of the costs of the coalitions of the individual players in and the cost of the coalition itself, namely,

Given the bound for in (7), the value is also bounded, as given: for any and ,

Thus, the cost savings (value) of each coalition is bounded uniformly by a maximum value.

#### 2.2.2 Network controlled flows

Here, we provide a more theoretical example where the bargaining process provides a way for the nodes of a hyper-graph to agree on the edge controlled flows [BBP10]. Consider the hyper-graph with the vertex set and the edge-set The vertex set has one vertex per each coalition, while the edge-set has one edge per each player. An edge is incident to a vertex if the player is in the coalition associated with . For a 3-player coalitional game, the vertex-coalition correspondence is shown in Table 1. The corresponding hyper-graph is shown in Figure 3.

The incidence relations are described by a matrix whose rows are the characteristic vectors of all coalitions . The flow control reformulation arises naturally by viewing an allocation value of player as the flow on the edge associated with player . The demand at a vertex is the value of the coalition associated with the vertex . In view of this, an allocation in the core translates into satisfying in excess the demand at the vertices, or formally:

In this case, the efficiency condition corresponds to the equality constraint . When is a constant, say for an , the resulting constraint can be interpreted as a total flow constraint, stating that the total flow in the network has to be equal to the given value .

In this framework, the bargaining process is an algorithmic and distributed mechanism that ensures the players reach an agreement on controlled flows satisfying uncertain demand at the nodes.

### 2.3 Preliminary Results

In this section we derive some preliminary results pertinent to the core of the robust game and some error bounds for polyhedral sets applicable to the players’ bounding sets . We later use these results to establish the convergence of the bargaining process in (6).

In our analysis we often use the following relation that is valid for the projection operation on a closed convex set : for any and any ,

(8) |

This property of the projection operation is known as a strictly non-expansive projection property (see [Facchinei2003], volume II, 12.1.13 Lemma on page 1120).

We next prove a result that relates the distance between a point and the core with the distances between and the bounding sets . This result will be crucial in our later development. The result relies on the polyhedrality of the bounding sets and the core , as given in (1) and (2) respectively, and a special relation for polyhedral sets. This special relation states that for a nonempty polyhedral set there exists a scalar such that

(9) |

where and the scalar depends on the vectors only. Relation (9) has been established by Hoffman [Hoffman1952] and is known as Hoffman bound.

Aside from the Hoffman bound, in establishing the forthcoming Lemma 1, we also use the fact that the square distance from a point to a closed convex set contained in an affine set is given by

(10) |

which is illustrated in Figure 4.

Now, we are ready to present the result relating the values and .

###### Lemma 1

Let be a TU game with a nonempty core . Then, there is a constant such that

where depends on the collection of vectors , where each is the projection of on the hyperplane .

Proof. Since the hyperplane contains the core (see (2)), by relation (10) we have

(11) |

The point and the core lie in the -dimensional affine set . By applying the Hoffman bound relative to the affine set (cf. Eq. (9)), we obtain

where the summation is over nonempty subsets , , while the constant depends on the collection of projections of vectors on the hyperplane for . Thus, it follows

(12) | |||||

(13) |

where is the number of nonempty subsets of and the last inequality follows by , which is valid for any finite collection of scalars with . Combining Eq. (12) with equality (11), we obtain for any ,

where . Since the set is affine, in view of relation (10) we have , implying that for any ,

From the preceding relation, it follows for any

(14) |

where denotes the cardinality of the coalition . Note that

(15) | ||||

(16) |

We also note that for each nonempty and , which follows by the definition of and relation (1). Furthermore, since for any and for any two closed convex sets such that , it follows that for all ,

(17) |

By combining relations (14)–(17), we obtain

where is the number of coalitions that contain player , which is the same number for every player ( does not depend on ). The desired relation follows by letting , and by recalling that and that depends on the projections of vectors , on the hyperplane .

Note that the scalar in Lemma 1 does not depend on the coalitions’ values for . It depends only on the vectors , , and the grand coalition value .

As a direct consequence of Lemma 1, we have the following result for the instantaneous game under the assumptions of Section 2.1.

###### Lemma 2

## 3 Convergence to Core of Robust Game

In this section, we prove convergence of the bargaining process in (6) to a random allocation that lies in the core of the robust game with probability 1. We find it convenient to re-write the bargaining protocol (6) by isolating a linear and a non-linear term. The linear term is the vector defined as:

(18) |

Note that is linear in players’ allocations . The non-linear term is the error

(19) |

Now, using (18) and (19), we can rewrite protocol (6) as follows:

(20) |

Recall that the weights are nonnegative and such that for all . Also, recall that is the matrix with entries , which governs the construction of the vectors in (18).

The main result of this section shows that, with probability 1, the bargaining protocol (18)–(20) converges to the core of the robust game , provided that happens infinitely often in time with probability 1. To establish this we use some auxiliary results, which we develop in the next two lemmas.

The following lemma provides a result on the sequences and shows that the errors are diminishing.

###### Lemma 3

Proof. By and by strictly non-expansive property of the Euclidean projection on a closed convex set (see (8)), we have for any , and ,

(21) |

Under Assumptions 1 and 2, the core is contained in the core for all , implying that for all . Furthermore, since , it follows that for all and . Therefore, relation (21) holds for all . Thus, by summing the relations in (21) over , we obtain for all and ,

(22) |

By the definition of in (18), using the stochasticity of and the convexity of the squared norm, we obtain

By the doubly stochasticity of , we have for every , implying . By substituting this relation in (22), we arrive at

(23) |

The preceding relation shows that the scalar sequence is non-increasing for any given . Therefore, the sequence must be convergent since it is nonnegative. Moreover, by summing the relations in (23) over and taking the limit as , we obtain

which implies that for all .

In our next result, we will use the instantaneous average of players allocations, defined as follows:

The result shows that the difference between the bargaining payoff vector for any player and the average of these payoffs converges to 0 as time goes to infinity. The proof essentially uses the line of analysis that has been employed in [NOP10], where the sets are static in time, i.e., for all . In addition, we also use the rate result for doubly stochastic matrices that has been established in [NOOT09].

###### Lemma 4

Proof. For any , define matrices

with . Using the matrices and the expression for in (20), we can relate the vectors with the vectors at a time for , as follows:

(24) |

Under the doubly stochasticity of the matrices , using and relation (24), we obtain

(25) |

By our assumption, we have for all . Thus, for any , there is an integer such that for all and all . Using relations (24) and (25) with , we obtain for all and ,

Since for all and all , it follows that