Constrained existence problem for weak subgame perfect equilibria with -regular Boolean objectives (full version)††thanks: This article is based upon work from COST Action GAMENET CA 16228 supported by COST (European Cooperation in Science and Technology).
We study multiplayer turn-based games played on a finite directed graph such that each player aims at satisfying an -regular Boolean objective. Instead of the well-known notions of Nash equilibrium (NE) and subgame perfect equilibrium (SPE), we focus on the recent notion of weak subgame perfect equilibrium (weak SPE), a refinement of SPE. In this setting, players who deviate can only use the subclass of strategies that differ from the original one on a finite number of histories. We are interested in the constrained existence problem for weak SPEs. We provide a complete characterization of the computational complexity of this problem: it is P-complete for Explicit Muller objectives, NP-complete for Co-Büchi, Parity, Muller, Rabin, and Streett objectives, and PSPACE-complete for Reachability and Safety objectives (we only prove NP-membership for Büchi objectives). We also show that the constrained existence problem is fixed parameter tractable and is polynomial when the number of players is fixed. All these results are based on a fine analysis of a fixpoint algorithm that computes the set of possible payoff profiles underlying weak SPEs.
Two-player zero-sum graph games with -regular objectives are the classical mathematical model to formalize the reactive synthesis problem [PnueliR89, Thomas95]. More recently, generalization from zero-sum to non zero-sum, and from two players to players have been considered in the literature, see e.g. [berwanger07, BrenguierCHPRRS16, BRS-concur15, BRS14, BMR14, KHJ06, FismanKL10, KupfermanPV14, Ummels06] and the surveys [Bruyere17, GU08]. Those extensions are motivated by two main limitations of the classical setting. First, zero-sum games assume a fully antagonistic environment while this is often not the case in practice: the environment usually has its own goal. While the fully antagonistic assumption is simple and sound (a winning strategy against an antagonistic environment is winning against any environment that pursues its own objective), it may fail to find a winning strategy even if solutions exist when the objective of the environment is accounted. Second, modern reactive systems are often composed of several modules, and each module has its own specification and should be considered as a player on its own right. This is why we need to consider -player graph games.
For -player graph games, solution concepts like Nash equilibria (NEs) [nash50] are natural to consider. A strategy profile is an NE if no player has an incentive to deviate unilaterally from his strategy, i.e. no player can strictly improve on the outcome of the strategy profile by changing his strategy only. In the context of sequential games (such as games played on graphs), NEs allow for non-credible threats that rational players should not carry out. To avoid non-credible threats, refinements such as subgame perfect equilibria (SPEs) [osbornebook] have been advocated. A strategy profile is an SPE if it is an NE in all the subgames of the original game. So players need to play rationally in all subgames, and this ensures that non-credible threats cannot exist. For applications of this concept to -player graph games, we refer the reader to [BrihayeBDG12, KHJ06, Ummels06].
In [BrihayeBMR15], the notion of weak subgame perfect equilibrium (weak SPE) is introduced, and it is shown how it can be used to study the existence SPEs (possibly with contraints) in quantitative reachability games. While an SPE must be resistant to any unilateral deviation of one player, a weak SPE must be resistant to deviations restricted to deviating strategies that differ from the original one on a finite number of histories only. In [Bruyere0PR17] the authors study general conditions on the structure of the game graph and on the preference relations of the players that guarantee the existence of a weak SPE for quantitative games. Weak SPEs retain most of the important properties of SPEs and they coincide with them when the payoff function of each player is continuous (see e.g. [fudenberg1991game]). Weak SPEs are also easier to characterize and to manipulate algorithmically. We refer the interested reader to [BrihayeBMR15, Bruyere0PR17] for further justifications of their interest, as well as for related work on NEs and SPEs.
In this paper, we concentrate on graph games with -regular Boolean objectives. While SPEs, and thus weak SPEs, are always guaranteed to exist in such games, we here study the computational complexity of the constrained existence problem for weak SPEs, i.e. equilibria in which some designated players have to win and some other ones have to loose. More precisely, our main results are as follows:
We study the constrained existence problem for games with Reachability, Safety, Büchi, Co-Büchi, Parity, Explicit Muller, Muller, Rabin, and Streett objectives. We provide a complete characterization of the computational complexity of this problem for all the classes of objectives with one exception: Büchi objectives. The problem is P-complete for Explicit Muller objectives, it is NP-complete for Co-Büchi, Parity, Muller, Rabin, and Streett objectives, and it is PSPACE-complete for Reachability and Safety objectives. In case of Büchi objectives, we show membership to NP but we fail to prove hardness.
Our complexity results rely on the identification of a symbolic witness for the constrained existence of a weak SPE, the size of which allows us to prove NP/PSPACE-membership. As the constrained existence problem is PSPACE-complete for Reachability and Safety objectives, symbolic witnesses as compact as those for the other objectives cannot exist unless NP PSPACE. The identification of symbolic witnesses is obtained thanks to a fixpoint algorithm that computes the set of all possible payoff profiles underlying weak SPEs.
When the number of players is fixed, we show that the constrained existence problem can be solved in polynomial time for all -regular objectives. We also prove that it is fixed parameter tractable where the parameter is the number of players, for Reachability, Safety, Büchi, Co-Büchi, and Parity objectives. For Rabin, Streett, and Muller objectives, we still establish fixed parameter tractability but we need to consider some additional parameters depending on the objectives. These tractability results are obtained by a fine analysis of the complexity of the fixpoint algorithm mentioned previously.
Related work and additional contributions
In [GU08, Ummels06], a tree automata-based algorithm is given to decide the constrained existence problem for SPEs on graph games with -regular objectives defined by parity conditions. A complexity gap is left open: this algorithm executes in EXPTIME and NP-hardness of the decision problem is proved. In this paper, we focus on weak SPEs for which we provide precise complexity results for the constrained existence problem. We also observe that our results on Reachabilty and Safety objectives transfer from weak SPEs to SPEs: the constrained existence problem for SPEs is PSPACE-complete for those objectives. Quantitative Reachability objectives are investigated in [BrihayeBMR15] where it is proved that the constrained existence problem for weak SPEs and SPEs is decidable, but its exact complexity is left open.
In [BrihayeBMR15, Bruyere0PR17, FleschKMSSV10], the existence of (weak) SPEs in graph games is established using a construction based on a fixpoint. Our fixpoint algorithm is mainly inspired by the fixpoint technique of [Bruyere0PR17]. However, we provide complexity results based on this fixpoint while transfinite induction is used in [Bruyere0PR17]. Furthermore, we have modified the technique of [Bruyere0PR17] in a way to get a fixpoint that contains exactly all the possible payoff profiles of weak SPEs. This is necessary to get a decision algorithm for the constrained existence problem.
Profiles of strategies with finite-memory are more appealing from a practical point of view. It is shown in [Ummels06] that when there exists an SPE in a graph game with -regular objectives, then there exists one that uses finite-memory strategies and has the same payoff profile. Thanks to the symbolic witnesses, we have refined those results for weak SPEs.
Structure of the paper
In Section 2, we recall the notions of -player graph games and of (weak) SPE, and we state the studied constrained existence problem. In Section 3, we provide a fixpoint algorithm that computes all the possible payoff profiles for weak SPEs on a given graph game. From this fixpoint, we derive symbolic witnesses of weak SPEs. In Section 4, we study the complexity classes of the constrained existence problem for all objectives except Explicit Muller objectives. In Section 5, we prove the fixed parameter tractability of the constrained existence problem and we show that is in polynomial time when the number of players is fixed. We also show that this problem it is P-complete for Explicit Muller objectives. In Section 6, we give a conclusion and propose future work.
In this section, we introduce multiplayer graph games in which each player aims to achieve his Boolean objective. We focus on classical -regular objectives, like Reachability, Büchi, aso. We recall two classical concepts of equilibria: Nash equilibrium and subgame perfect equilibrium (see [GU08]). We also recall weak variants of these equilibria as proposed in [BrihayeBMR15, Bruyere0PR17]. We conclude the section by the constrained existence problem that is studied in this paper.
2.1 Multiplayer Boolean games
Definition 1 (Boolean game).
A multiplayer Boolean game is a tuple where
is a finite set of players;
is a finite directed graph and for all there exists such that ;
is a partition of between the players;
is a tuple of functions that assigns a Boolean value to each infinite path of for player .
A play in is an infinite sequence of vertices such that for all , . A history is a finite sequence () defined similarly. We denote the set of plays by and the set of histories by . Moreover, the set is the set of histories such that the last vertex is a vertex of player , i.e. . The length of is the number of its edges. A play is called a lasso if it is of the form with . Notice that is not necessary a simple cycle. The length of a lasso is the length of . For all , we denote by the first vertex of . We use notation when a history is prefix of a play (or a history) . Given a play , the set is the set of vertices visited by , and is the set of vertices infinitely often visited by . Given a vertex , is the set of successors of , and is the set of vertices reachable from in .
When an initial vertex is fixed, we call an initialized game. A play (resp. a history) of is a play (resp. a history) of starting in . The set of such plays (resp. histories) is denoted by (resp. ). We also use notation when these histories end in a vertex .
The goal of each player is to achieve his objective, i.e., to maximize his gain.
Definition 2 (Objective).
For each player , let be his objective. In the setting of multiplayer Boolean game, the gain function is defined such that (resp. ) if and only if (resp. ).
An objective (or the related gain function ) is prefix-independent if for all and , we have if and only if . In this paper, we focus on classical -regular objectives: Reachability, Safety, Büchi, Co-Büchi, Parity, Explicit Muller, Muller, Rabin, and Streett and we suppose that each player has the same type of objective. For instance, we say that is a Boolean game with Büchi objectives to express that all players have a Büchi objective.
Definition 3 (Classical -regular objective).
The set is a Reachability, Safety, Büchi, Co-Büchi, Parity, Explicit Muller, Muller, Rabin, or Streett objective for player if and only if is composed of the plays satisfying:
Reachability: given , ;
Safety: given , ;
Büchi: given , ;
Co-Büchi: given , ;
Parity: given a coloring function , ;
Explicit Muller: given , ;
Muller: given a coloring function , and , ;
Rabin: given a family of pair of sets ,
there exists such that and ;
Streett: given a family of pair of sets ,
for all , or .
All these objectives are prefix-independent except Reachability and Safety objectives.
A strategy of a player is a function . This function assigns to each history with , a vertex such that . In an initialized game , needs only to be defined for histories starting in . A play is consistent with if for all we have that . A strategy is positional if it only depends on the last vertex of the history, i.e., for all . It is finite-memory if it can be encoded by a deterministic Moore machine where is a finite set of states (the memory of the strategy), is the initial memory state, is the update function, and is the next-action function. The Moore machine defines a strategy such that for all histories , where extends to histories as expected. The size of the strategy is the size of its machine . Note that is positional when .
A strategy profile is a tuple of strategies, one for each player. It is called positional (resp. finite-memory) if for all , is positional (resp. finite-memory). Given an initialized game and a strategy profile , there exists an unique play from consistent with each strategy . We call this play the outcome of and it is denoted by . Let , we say that is a strategy profile with payoff or that has payoff if for all .
2.2 Solution concepts
In the multiplayer game setting, the solution concepts usually studied are equilibria (see [GU08]). We here recall the concepts of Nash equilibrium and subgame perfect equilibrium, as well as some variants. We begin with the notion of deviating strategy.
Let be a strategy profile in an initialized Boolean game . Given , a strategy is a deviating strategy of player , and denotes the strategy profile where replaces . Such a strategy is a profitable deviation for player if . We say that is finitely deviating from if and only differ on a finite number of histories, and that is one-shot deviating from if and only differ on [BrihayeBMR15, Bruyere0PR17].
The notion of Nash equilibrium (NE) is classical: a strategy profile in an initialized game is a Nash equilibrium if no player has an incentive to deviate unilaterally from his strategy since he has no profitable deviation, i.e., for each and each deviating strategy of player from , the following inequality holds: . In this paper we focus on two variants of NE: weak/very weak NE [BrihayeBMR15, Bruyere0PR17].
Definition 4 (Weak/very weak Nash equilibrium).
A strategy profile is a weak NE (resp. very weak NE) in if, for each player , for each finitely deviating (resp. one-shot) strategy of player from , we have .
Figure 1 illustrates an initialized Boolean game with Büchi objectives in which there exists a weak NE that is not an NE. In this game, player 1 (resp. player 2) owns round (resp. square) vertices and wants to visits (resp. or ) infinitely often. The positional strategy profile is depicted by dashed arrows, its outcome is equal to , and has payoff . Notice that player 1 has an incentive to deviate from his strategy with a strategy that goes to for all histories ending in . This is indeed a profitable deviation for him since . So, is not an NE. Nevertheless, it is a weak NE because is the only profitable deviation and it is not finitely deviating (it differs from on all histories of the form for ).
When considering games played on graphs, a well-known refinement of NE is the concept of subgame perfect equilibrium (SPE) which a strategy profile being an NE in each subgame. Variants of weak/very weak SPE can also be studied as done with NEs. Formally, given an initialized Boolean game and a history , the initialized game is called a subgame222Notice that is subgame of itself. of such that and for all and . Moreover if is a strategy for player in , then denotes the strategy in such that for all histories , . Similarly, from a strategy profile in , we derive the strategy profile in . The play is called a subgame outcome of .
Definition 5 (Subgame perfect equilibrium and weak/very weak subgame perfect equilibrium).
A strategy profile is a (resp. weak, very weak) subgame perfect equilibrium in if for all , is a (resp. weak, very weak) NE in .
When one needs to show that a strategy profile is a weak SPE, the next proposition is very useful because it states that it is enough to consider one-shot deviating strategies.
Proposition 1 ([BrihayeBMR15]).
A strategy profile is a weak SPE if and only if is a very weak SPE.
In Example 1 is given a weak NE in the game depicted in Figure 1. This strategy profile is also a very weak SPE (and thus a weak SPE by Proposition 1). For instance, in the subgame with and , the only one-shot deviating strategy is such that coincides with except that . This is not a profitable deviation for player in . Notice that is not an SPE since it is not an NE as explained in Example 1.
In general, the notions of SPE and weak SPE are not equivalent (see Example 2). Nevertheless they coincide for the class of Boolean games with Reachability objectives.
Let be a strategy profile in an initialized Boolean game with Reachability objectives. Then is an SPE if and only if is a weak SPE.
Each player has a Reachability objective, let be the set of vertices he aims to visit.
This implication is a consequence of the definitions of SPE and weak SPE.
Let be a weak SPE in . Assume that is not an SPE, i.e., there exists such that is not an NE in . Then some player has a profitable deviation in the subgame . As takes its values in , this means that
with and . We consider the first occurrence of a vertex of along (which appears in and not in as ): let of mininal length such that and ends in some . Let us define a strategy that is finitely deviating from and profitable for player in . This will be in contradiction with our hypothesis. For all , let
By definition of , we have that and is finitely deviating from since is finite. ∎
2.3 Constraint problem
It is proved in [Bruyere0PR17] that there always exists a weak SPE in Boolean games. In this paper, we are interested in solving the following constraint problem:
Definition 6 (Constraint problem).
Given an initialized Boolean game and thresholds , decide whether there exists a weak SPE in with payoff such that .333The order is the componentwise order, that is, , for all .
In the next sections, we solve the constraint problem for the classical -regular objectives. The complexity classes that we obtain are shown in Table 1. They are detailed in Section 4 with the case of Explicit Muller objectives postponed to Section 5.3. The case of Büchi objectives remains open, since we only propose a non-deterministic algorithm in polynomial time but no matching lower bound. In Section 5, we prove that the constraint problem for weak SPEs is fixed parameter tractable and becomes polynomial when the number of players is fixed. All these results are based on a characterization of the set of possible payoffs of a weak SPE, that is described in Section 3.
In this section our aim is twofold: first, we characterize the set of possible payoffs of weak SPEs and second, we show how it is possible to build a weak SPE given a set of lassoes with some “good properties”. Those characterizations work for Boolean games with prefix-independent gain functions. We make this hypothesis all along Section 3.
3.1 Remove-Adjust procedure
Let be an initialized Boolean game with prefix-independent gain functions. The computation of the set of all the payoffs of weak SPEs in is inspired by a fixpoint procedure explained in [Bruyere0PR17]. Each vertex is labeled by a set of payoffs . Initially, these payoffs are those for which there exists a play in with payoff . Then step by step, some payoffs are removed for the labeling of as soon as we are sure they cannot be the payoff of in a subgame for some weak SPE .444The value of is not important since the gain functions are prefix independent. This is why we only focus on and not on . When a fixpoint is reached, the labeling of the initial vertex exactly contains all the payoffs of weak SPEs in . Hence, at each step of this procedure, the payoffs labeling a vertex are payoffs of potential subgame outcomes of a weak SPE. Their number decreases until reaching a fixpoint.
We formally proceed as follows. For all , we define the initial labeling of as:
Then for each step , we compute the set by alternating between two operations: Remove and Adjust. To this end, we need to introduce the notion of -labeled play. Let be a payoff and be a step, a play is -labeled if for all we have , that is, visits only vertices that are labeled by at step . We first give the definition of the Remove-Adjust procedure and then give some intuition about it.
Definition 7 (Remove-Adjust procedure).
If is odd, process the Remove operation:
If for some there exists and such that for all , then and for all , .
If such a vertex does not exist, then for all .
If is even, process the Adjust operation:
If some payoff was removed from (that is, ), then
For all such that , check whether there still exists a -labeled play with payoff from . If it is the case, then , otherwise .
For all such that :
Otherwise for all .
Let us explain the Remove operation. Let that labels vertex . This means that it is the payoff of a potential subgame outcome of a weak SPE that starts in . Suppose that is a vertex of player and has a successor such that for all labeling . Then cannot be the payoff of in the subgame for some weak SPE and some history , otherwise player would have a profitable (one-shot) deviation by moving from to in this subgame.
Now it may happen that for another vertex having in its labeling, all potential subgame outcomes of a weak SPE from with payoff necessarily visit vertex . As has been removed from the labeling of , these potential plays do no longer survive and is also removed from the labeling of by the Adjust operation.
We can state the existence of a fixpoint of the sequences , , in the following meaning:
Proposition 3 (Existence of a fixpoint).
There exists an even natural number such that for all , .
For all , the sequence is nonincreasing because the Remove and Ajdust operations never add a new payoff. As each is finite (it contains at most payoffs), there exists a natural odd number such that for all , during the Remove operation, and thus for all , during the Adjust operation. ∎
We illustrate the different steps of the Remove-Adjust procedure on the example depicted in Figure 1, and we display the result of this computation in Table 2. Initially, the sets , , contains all payoffs such that there exists a play with . At step , we apply a Remove operation to (this is the only possible ): is a vertex of player and has a successor such that . Therefore is removed from to get . By definition of the Remove operation, the other sets are not modified and are thus equal to . At step , we apply an Adjust operation. The only way to obtain the payoff from is by visiting with the play . As there does not exist a -labeled play with payoff anymore, we have to remove from . The other sets remain unchanged. At step , the Remove operation removes payoff from due to the unique payoff in . At step , the Adjust operation leaves all sets unchanged. Finally at step , the Remove operation also leaves all sets unchanged, and the fixpoint is reached.
3.2 Characterization and good symbolic witness
The fixpoint , , provides a characterization of the payoffs of all weak SPEs as described in the following theorem. This result is in the spirit of the classical Folk Theorem which characterizes the payoffs of all NEs in infinitely repeated games (see for instance [fudenberg1991game, Chapter 8]).
Theorem 1 (Characterization).
Let be an initialized Boolean game with prefix-independent gain functions. Then there exists a weak SPE with payoff in if and only if for all and .555We use notation to highlight that this is the payoff of from vertex . It should not be confused with any component , , of a payoff .
In this theorem, only sets with are considered. Indeed subgames of deals with histories , that is, with vertices reachable from . The rest of this section is devoted to the proof of Theorem 1.
We begin with a lemma that states that if a payoff survives at step in the labeling of , this means that there exists a play with payoff from that only visits vertices also labeled by .
For all even and in particular for , belongs to if and only if there exists a -labeled play such that .
Suppose that there exists a -labeled play such that . By definition of a -labeled play, we have for all , and so in particular for .
Let us prove that if belongs to , then there exists a -labeled play such that . We proceed by induction on . For , the assertion is satisfied by definition of and because is prefix-independent for all .
Suppose that the assertion is true for an even and let us prove that it remains true for . Let . As , we have and there exists a -labeled play such that by induction hypothesis. In other words for all .
We suppose that there exists such that (the fixpoint is not reached), otherwise for all and is also a -labeled play. Therefore the Remove operation has removed some payoff from one and the Adjust operation has possibly removed from some other . If , then clearly still belongs to each and is again a -labeled play. If , then since by hypothesis. Moreover, by the Adjust operation, this means that there exists a -labeled play with payoff from which never visits . Let us show that is also a -labeled play, that is, for all . Each suffix of is a -labeled play with payoff thanks to prefix-independence of . By the Adjust operation, it follows that for all . This concludes the proof. ∎
The proof of Theorem 1 uses the concept of (good) symbolic witness defined hereafter. Some intuition about it is given after the definitions.
Definition 8 (Symbolic witness).
Let be an initialized Boolean game with prefix-independent gain functions. Let be the set
A symbolic witness is a set such that each is a lasso of with and with length bounded by .
A symbolic witness has thus at most lassoes (by definition of ) with polynomial length.
Definition 9 (Good symbolic witness).
A symbolic witness is good if for all , for all vertices such that and , we have .
Let us now give some intuition. A strategy profile in induces an infinite number of subgame outcomes , . A symbolic witness is a compact representation of . It is a finite set of lassoes that represent some subgame outcomes of : the lasso of represents the outcome , and its other lassoes represents the subgame outcome for some particular histories . The index records that player can move from (the last vertex of ) to (with the convention that for the outcome ). When is a weak SPE, the related symbolic witness is good, that is, its lassoes avoid profitable one-shot deviations between them.
We come back to our running example. The weak SPE of Example 2 depicted in Figure 1 has payoff . A symbolic witness of is given in Table 3 which is here composed of all the subgame outcomes of . One can check that is a good symbolic witness. For instance, consider its lassoes and , the vertex of and the edge . We have . Indeed in the subgame , player has no profitable one-shot deviation by using the edge .
In Proposition 4 below, we are going to prove that there exists a weak SPE if and only if there exists a good symbolic witness, and that the existence of this witness is equivalent to the non-emptiness of the fixpoint , . In this way, we will prove Theorem 1. We will see that the lassoes of a good symbolic witness can be constructed from )-labeled plays for well-chosen payoffs .
Let be an initialized Boolean game with prefix-independent gain functions. The following assertions are equivalent:
There exists a weak SPE with payoff in ;
for all and
There exists a good symbolic witness that contains a lasso with payoff ;
There exists a finite-memory weak SPE with payoff in such that the size of each strategy is in .
We prove that .
Suppose that there exists a weak SPE with payoff in . To show that for all , let us prove by induction on that
For case , this is true by definition of . Suppose that this assertion is satisfied for an even . Let us prove that it remains true for by showing that payoff can be removed neither from at step , nor from at step .
Payoff cannot be removed from by the Remove operation at step . Otherwise, if , this means that there exists such that
By induction hypothesis,
To get a contradiction, we prove that in the subgame there exists a one-shot deviating strategy from that is a profitable deviation for player . We define that only differs from on : . Therefore we get . It follows by (2), (3), and prefix-independence of that . This is impossible since is a weak SPE.
Payoff cannot be removed from by the Adjust operation at step . Otherwise, this means that there exists such that (by the Remove operation at step ) and there is no -labeled play with payoff from . However by Lemma 1, as , there exists a -labeled play with payoff from . This means that visit . Let such that . Then we get a contradiction with being a weak SPE if we repeat the argument done before in the previous item for and the subgame (instead of and ).
Now that we know that for all , it remains to prove that