A Process Algebraic Form to Represent Extensive Games 111published in control and cybernetics journal, no. 1, vol. 44, 2015.
In this paper, we introduce an agent-based representation of games, in order to propose a compact representation for multi-party games in game theory. Our method is inspired by concepts in process theory and process algebra. In addition, we introduce an algorithm whose input is a game in the form of process algebra (proposed in this paper) and as an output, finds the Nash equilibrium of the game in linear space complexity.
extensive games, Nash equilibrium, process theory, process algebra
Extensive representation form of a game is a directed graph whose nodes are players, and edges are actions. Assuming that the game has agents, and each agent has two actions available, the game can be represented by a graph of size .
However, by explaining the behavior of each agent individually using an adequate process (called process-game), and obtaining the whole game through parallel composition these process-games, it is possible to represent the same game in space. We take advantage of process algebra to define process-game and the appropriate notion of parallel composition for them.
The word “process” refers to the behavior of a system which is a set of actions that are performed in the system and the order of their executions. The process theory makes it possible to model the behavior of a system with enormous complexity through modeling the behavior of its components [Middelburg and Reniers, 2005]. By taking advantages of process algebra–automating calculations and running algorithms using parallel computing techniques–we can code the process theory terms and definitions [Fokkink, 2007], [Tadjouddine, 2008]. On the other hand, a game is a system and its behavior is established by the behavior of all of its players (components). Hence, the game could be studied and formally modeled as interactive process [Van Benthem, 2002].
Moving along this path in formal methods, in order to reduce the representation of games with lots of players, we modify the process theory in an appropriate manner to provide a model called “process-game” that encompasses both process theory and game theory notions. This proposed process algebraic model makes it possible to have a compact representation for extensive games–specially in social extensive games which have local interaction–via appropriate parallel composition (section 3).
Also, there are other efforts to reduce the representation of games with very high number of players. In comparison with graph-based representation that is proposed to reach the same goal [Kearns, Littman, and Singh, 2001], our proposed model, facilitates the reduction of representation of games in the formal method.
Eventually, to manipulate process-game model efficiently, we propose an algorithm to find the equilibrium path of games in linear space complexity by using a revision of depth first search and backward induction (section 4).
In this section, we review some preliminary concepts in game and process theory.
2.1 Game-Theoretic Concepts
In this part we briefly review definitions and concepts of strategic and extensive games with perfect information which appeared in the literature using the same notations as in [Osborne and Rubinstein, 1994], Page 89.
A strategic game is a model of decision-making such that decision-makers choose their plan simultaneously from their possible actions, once and for all.
Definition 1 (Strategic Game).
A strategic game consists of:
A finite set of players
for each player a nonempty set of actions
for each player a payoff function .
If for each player ’s action set is finite, the the game is finite.
Example 1 (prisoner’s dilemma).
Two members of a criminal gang are arrested and imprisoned. Each prisoner is in solitary confinement with no means of speaking to or exchanging messages with the other. The police admit they don’t have enough evidence to convict the pair on the principal charge. They plan to sentence both to a year in prison on a lesser charge. Simultaneously, the police offer each prisoner a Faustian bargain. Each prisoner is given the opportunity either to betray () the other, by testifying that the other committed the crime, or to cooperate () with the other by remaining silent. Here’s how it goes:
If A and B both betray the other, each of them serves 2 years in prison
If A betrays B but B remains silent, A will be set free and B will serve 3 years in prison (and vice versa)
If A and B both remain silent, both of them will only serve 1 year in prison (on the lesser charge)
The above situation is shown as a strategic game in Figure 1.
The set of players is . Possible actions for each player come from the set of . Each cell in the above table corresponds to a strategy profile and shows the resulting payoffs. A strategy profile is a set of strategies for all players which fully specifies all actions in a game. A strategy profile must include one and only one strategy for every player. In the cell, the first value is the player 1’s payoff and the player 2’s payoff is the second one.
One of the most common solution concepts in game theory is Nash equilibrium. This notion captures a steady state which no player wants to deviate from the current state if action of the other players are fixed (therefore, all player choose their action in a rational manner).
For each strategy profile , shows the action of all players, except player .
Definition 2 (Nash Equilibrium).
A Nash equilibrium of a strategic game is a strategy profile with the property that for every player we have
In Example 1, strategy profile is a Nash equilibrium. Because, If player 1 deviates from action to , his payoff decreases from to in strategy profile . Therefore, he has not any motivation to deviate from the state . Also, because of symmetry, we can say the same reason for player 2 to have not any motivation to deviate from the current state. Therefore, strategy profile is a Nash equilibrium in this game.
Another form of games, which is the pivot of discussion in this paper, is extensive game. In this form, decision-makers act sequentially (unlike strategic game).
Definition 3 (Extensive Game with Perfect Information).
An extensive game with perfect information is defined by a four-tuple which has the following properties:
A set of players
A set of sequences (finite or infinite) that satisfies the following three properties:
The empty sequence (the empty history representing the start of the game) is a member of .
If (where K may be infinite) and positive integer then .
If an infinite sequence satisfies for every positive integer then .
Each member of H is a history and each term of a history is an action which is taken by a player. A history is terminal if it is infinite or there is no such that . The set of all terminal histories is denoted by .
A function (the player function) that assigns to each nonterminal history (each member of ) a member of , returns the player who takes an action after the history ().
A function (the payoff function) that assigns to each terminal history (each member of ) a member of (, is player ’s payoff in terminal history ).
An extensive game with perfect information is finite if and only if the set of possible histories is finite. Throughout this paper, whenever we use the term extensive games, we mean extensive games with perfect information. In an extensive game, chooses an action after any nonterminal history from the set where means a history followed by an action which is one of the actions available to the player who moves after .
Definition 4 (Strategy).
A strategy of player in an extensive game is a function that assigns an action from to each (nonterminal history) for which .
The outcome of a strategy profile is a terminal history which is constructed by , and is denoted by .
Two people (“husband” and “wife”) are buying items for a dinner party. The husband buys either fish (F) or meat (M) for the meal; the wife buys either red wine (R) or white wine (W). Both people prefer red wine with meat and white wine with fish, rather than either of the opposite combinations. However, the husband prefers meat over fish, while the wife prefers fish over meat. Assume that the husband buys the meal and tells his wife what was bought; his wife then buys some wine. If we want to consider this problem as an extensive game with perfect information we can determine its component like
Possible actions for husband are a member of the set and wife’s actions come from . So in this example, is a set of sequences which are started by the action or and terminated by or . All possible histories and terminal histories are shown below:
For each , is as follows:
We can represent the preferences as utility-based payoffs:
Nash equilibrium is a common solution concept for extensive games. This concept is defined in the following definition.
The strategy profile in an extensive game with perfect information is a Nash equilibrium if for each player we have:
for every strategy of player .
The notion of a Nash equilibrium ignores the sequential structure of an extensive form game and treats strategies as if they were choices made once and for all. To refine the Nash equilibrium definition is defined a new notion of subgame perfect equilibrium (see [Narahari, 2014], Chapter 3).
Let be an extensive game with perfect information. The subgame of that follows the history (a nonterminal history) is the extensive game where all sequences of actions for which are in the set and vice versa, is the same as function but its domain come from the set , and for each (the set consists of all the terminal histories in the set ) is defined that if and only if (note that ).
In a moment, we can define the notion of subgame perfect equilibrium as mentioned before, using the notion of subgame.
The strategy profile in an extensive game with perfect information is a subgame perfect equilibrium if for every player and every nonterminal history for which we have:
for every strategy of player
in the subgame .
Most often, an extensive game is shown by a tree and marked the subgame perfect Nash equilibria (SPNE) of the game on the tree as in [Narahari, 2014], Chapter 3 (for a better understanding see Example 3 and Figure 2).
The definition of extensive game with perfect information (Definition 3) can be generalized by allowing simultaneous moves of the players. This type of games is called extensive game with imperfect information, in this paper. An extensive game with imperfect information is determined by a quintuple . Relative to the definition of an extensive game with perfect information, the new element is the collection of information partitions. For each player , is a partition of with the property that whenever and are in the same member of the partition. For , we denote by the set and by , the player for any ( is the information partition of player ; a set is an information set of player ) [Osborne and Rubinstein, 1994].
We define a variant of the battle of sexes (BoS) game (Example 3) as an example of extensive games with imperfect information.
Assume in Example 3, that wife decides to hold a dinner party or not. If she decides not to hold the dinner party, the game ends and nothing happens. On the other hand, there are games similar to Example 3 where players move simultaneously instead of moving sequentially. As shown in Figure 3, simultaneous moving is specified by dashed line and means the wife does not know that she is in which history.
Formally, we have , , , and .
A pure strategy of player in an extensive game with imperfect information is a function that assigns an action in to each information set .
2.2 Process Theory
The notion of a transition system can be considered to be the fundamental notion for the description of process behavior [Middelburg and Reniers, 2005]. In this section we state some abstract formal definitions regarding transition systems and specify the notion of authentication using these definitions.
A transition system is a quintuple where
is a set of states,
is a set of actions containing an internal action ,
is a set of transitions,
is a set of successfully terminating states,
is the initial state.
The set shows generalized transitions of ( is a set of all possible chains of actions from ). A state is called reachable state of if there is such that . The set of all reachable states of a transition system is denoted by . We define . In the sequel, we assume that every transition system is connected, i.e., , and . If and are finite, is called a finite transition system.
We refer to by .
We define . If there exists such that , where is the length of the sequence , then is called a finite-depth transition system. If for every , is finite, then is called a finite-branching transition system. By the notation , we refer to the silent action.
If is both a finite-depth and a finite-branching transition system, then it is a finite transition system.
It is straightforward. ∎
Let be a transition system. Then is deterministic if the following condition holds: Whenever and , then .
Let be a set of actions. A communication function on is a partial function such that for any : is not defined, and if is defined then is defined and . The image of is shown by . We define . Assume that if is defined then both .
Definition 12 (Parallel Composition).
Let and be two transition systems, and a communication function on a set of actions that includes . The parallel composition of and under , written , is the transition system where
is the smallest subset of such that:
if and , then ,
if and , then ,
if , , and is defined, then ,
Definition 13 (Encapsulation).
Let be a transition system. Let be a set of actions. The encapsulation of with respect to , written as , is the transition system where
, , , and
Assume and are two processes , and execute them in parallel. Then for , the encapsulation of the process makes the processes to communicate. It means the difference between and is that in the second process only communication actions exist.
If and are two transition systems, and is finite-depth, then is finite-depth.
It is straightforward. ∎
Assume two transition systems and , and for , let , and for every , . We say are communication finite-branching with respect to communication function , if for any the set is finite.
If two transition systems and are communication finite-branching with respect to a communication function , then is finite-branching.
It is straightforward.∎
3 The Processes of Games
In this section, we introduce a process-algebraic representation for extensive games with perfect information in order to reduce the social large games into logarithmic (or polylog) size proportional to the size of extensive representation of the same games.
To analyse a game, we need to illustrate it with the game tree. If the number of agents is too large then we may need to use a machinery to provide the game tree illustration. We propose a machinery for this aim and call it process-game. The process game is a combination of process theory and game theory for solving games with large number of agents. We model the operation of each agent using a process-game. Then, we run all of these process-games in parallel to obtain the game tree.
Let be a set of actions. The language , is the smallest superset of such that
Let be a transition system. For , the history of , denoted by , is the trace such that . For a state and a formula , we define the satisfaction , as follows:
for , iff ,
iff or ,
Let and be two sets of actions. The set of conditional actions over with conditions in , denoted by is defined as follows:
for each , and , is a conditional action, i.e., .
There is an injective mapping from to which maps each to .
Definition 17 (Encapsulation of Conditional Actions).
Let and be two sets of actions and be a transition system over conditional actions . The encapsulation of conditional actions of , written , is the transition system where
, , , and
for and , iff for some , and .
Now we give an example to illustrate the above definitions.
Let . The transition system is a transition systems obtained from by cutting some of its transitions in the following way:
Assume we are given a which is an extensive game with perfect information. Now we model it using process theory as a process-game by mapping each player (in ) to a process, each terminal state to a member of , and payoff function () to profit value for each member of .
Definition 19 (Process-game).
Let be an extensive game with perfect information. A process-game model for is a tuple where each is a transition systems and each is a profit function. are conditioned by () so that
if then ,
and the game is mapped to process-game so that
is a set of states where at each state player , decides to perform one of his/her possible actions which is determined by for each node on game tree,
is a set of actions containing an internal action that represents possible actions of player ,
is a set of transitions that represents what happens when a player chooses one of his actions to do (using ),
is a set of successfully terminating states that represents terminal states on game tree which can be defined by terminal histories (),
is a payoff function that represents payoff () for each action of the player in each subprocess (like a subgame) which is started by .
is set of initial states which player can choose to start his game from.
Now we can construct the process tree of the game using .
The sizes of and (set of players or agents) in and are denoted by , , and respectively. Let which is called the branching factor of the process/game tree.
Suppose a process-game with is given. The size of the equivalent extensive representation of the which is denoted by is .
The size of the extensive representation is equal to the size of the tree of the game. As players act sequentially, the maximum number nodes in the first level of the tree is , in the second level would be and so on. Therefore, we have
The size of a given process-game with is .
The size of is equal to the sum of and for all players. The size of is . Therefore,
Assume that each action of players are in the form of or (without condition or just with one condition) , therefore the size of for each player is . According to Theorem 4, it is easy to see that the size of the process-game representation would be . Based on the assumption, the size of the process-game can be logarithmic proportional to the size of the extensive representation (as is the maximum number of plsyers’ actions, it is a constant). The following equation shows this fact.
We can extend the definition of process-game for extensive games with imperfect information in the following manner.
Let and be two disjoint sets. For , we define
Using the above notation, we modify definition 11 over disjoint sets of actions in the following.
Let be disjoint sets of actions. is a communication function which is defined for all .
We may extend to the set of conditional actions in the manner as in the following definition:
Let be three disjoint sets of actions. is a communication function over conditional actions and for which is defined as follows:
This communication function helps to model simultaneous players’ moves in extensive games with imperfect information (See Example 4).
Now, we can define the extended process-game by a tuple for a given extensive game with imperfect information . Relative to the definition of a process-game, the new element is the as a communication function over which are conditioned by . Definition of is based on .
In the extended version, the process/game tree is constructed with the same method as in process-game (). Also, if the size of is denoted by , we have
Similar theorems for extended process-game can be proved by replacing equation 3 in those theorems.
One of the application of process-game in representing the real social extensive games in log or polylog size could be in modeling of social systems where agents have local interaction and local competition (these two terms might be used interchangeably, conform to this article). A significant approach which works on the assumption of local interaction is generative social science.
Generative social science [Epstein, 2006] is an approach to study social systems via agent-based computational models. Its aim is to answer that how the regularity of the society emerges from local interaction of heterogeneous autonomous agents. One of the main assumptions of generative social science is called “local interaction” (see [Epstein, 2006], Page 6). Base on this assumption, agents interact with their neighbors and compete in the social network, in which they are involved.
On the other hand, social extensive games–each player is viewed as being involved in a local competition with the players in geographically neighboring regions–can be modeled as a graph [Kearns, Littman, and Singh, 2001]. In the graph-based model, each player is represented by a vertex in . The interpretation of edges is that each player is in a game only with the respective neighbors in .
In the above ven, it can be proposed that lots of social systems are based on local interactions and competitions. The observation forwarded below stipulates that in considering local interaction in the form of extensive games, the size of the process-game can be logarithmic in comparison with the extensive representation.
Suppose there is an extensive game with perfect information and interaction graph-based model . The maximum degree of is denoted by . Actions of each player are limited just to the player’s neighbors in (the size of in the equivalent process-game would be ). Hence, the size of the process-game representation would be . As a consequence, we have
As is a common situation in social extensive games, the size of the process-game can be logarithmic (or polylog, depending on and ) with respect to the size of the extensive representation. For instance, considering local competition property, we have , then . Therefore, .
Note that local competition is kind of local interaction. Recalling the assumption of local interaction in generative social science, we can come to the conclusion that it is obvious that in lots of social systems .
There have been various representations of extensive games aiming to represent the large games compactly, such as the graphical model [Kearns, Littman, and Singh, 2001] and MAIDs [Koller and Milch, 2003]. However, our proposal (originating from the process theory) of producing a compact representation, is quite different. Also, we bring the advantages of process theory (in execution and management) to the game theory environment.
Our model can be applied in management of complex systems too, but how? Each process has its own profit and each profit is a function of some inputs. Suppose a process game is defined with some initial values and may deliver some equilibria path as a result (using the algorithm of the following section). However, for a manager, it would be critical to control the path as he wants it to be. Therefore he can try changing the inputs (so the profits will be changed, leading also to some new equilibria path) up to getting the best one.
4 The Algorithm
The notion of process-game has been explained completely in the previous section. As strategies and agents in the process-game are the same as in its equivalent extensive game, therefore the subgame perfect equilibrium (or equilibrium path) is a solution concept for the process-game, too. Now, in this section, we propose an algorithm to find the equilibrium path in a process-game. Algorithm 1 illustrates the finding of equilibria path under the assumption that there exists only one equilibrium path for each subprocess (like a subgame).
Equilibrium path in a process-game is a sequence of actions from (suppose player starts the entirely of the game) to (suppose the last action is for player ).
In line 1 of Algorithm 1, expansion takes place by virtue of conditional actions. As in Algorithm 1, each player’s payoff value is calculated bottom-up, it is sufficient to save players’ payoff in the subprocess equilibria at each level. To reuse space and keep the space required by the algorithm linear, we delete all process nodes which are expanded in that subprocess previously in line 11 of Algorithm 1. We present below an example, meant to clarify how this algorithm works.
Two people select a policy that affects them both by alternately vetoing the available (voted) policies until only one remains. First person 1 vetoes a policy. If more than one policy remains, person 2 then vetoes a policy. If more than one policy still remains, person 1 then vetoes another policy. The process continues until only one policy has not been vetoed. Suppose there are three possible policies, , , and , person 1 prefers to to , and person 2 prefers to to [Osborne, 2004]. Now, we want to represent this situation through the process-game, as a compact representation proportional to the extensive model. To define a process-game , we should first determine the transition system . Players’ process model is specified below, with and being the processes of person 1 and person 2, respectively. and can be obtained by decoding theses process models.
Definitions of and are based on the players’ preferences. The payoff function of each player on each subprocess is equal to the player’s payoff over the complete path from the root to the leaf, passing through the subprocess. This path for each subprocess is denoted by . Therefore,
Actually, we define the above functions compactly. Thus, their space complexity is much lower than when defining the function for each path separately. Now, let us compute the Nash equilibrium. We know that the process tree is constructed completely by . However, using Algorithm 1, the process tree is expanded step by step to save the space. The steps of the DFS expansion of the process tree are sketched in Figure 6.
As it is not necessary existing any pure equilibrium path for an extensive game with imperfect information, this algorithm is not working for the extended process-game. However, we can detect the situation in the bottom-up calculation and report “no pure equilibrium path”.
Time complexity of Algorithm 1 in worst case would be in the NP-complete complexity class, like for backward induction [Nisan, Roughgarden, Tardos, and Vazirani, 2007] (because we want to find pure Nash equilibria). Its space complexity is linear in the size of the game which is given as an input i.e., linear in the depth of the process-game, maximum number of actions which are possible to do by a player, and the size of the payoff function.
In the extended version, if the number of simultaneous moves grows, the extensive game will be transforming to the pure strategic form, so that in the worst case, its space complexity would be exponential.
The algorithm is like a depth-first search and the space complexity of depth-first search is , here is the height of the tree and <