The game semantics of game theory

The game semantics of game theory

Jules Hedges
Abstract

We use a reformulation of compositional game theory to reunite game theory with game semantics, by viewing an open game as the System and its choice of contexts as the Environment. Specifically, the system is jointly controlled by noncooperative players, each independently optimising a real-valued payoff. The goal of the system is to play a Nash equilibrium, and the goal of the environment is to prevent it. The key to this is the realisation that lenses (from functional programming) form a dialectica category, which have an existing game-semantic interpretation.

In the second half of this paper, we apply these ideas to build a compact closed category of ‘computable open games’ by replacing the underlying dialectica category with a wave-style geometry of interaction category, specifically the Int-construction applied to the cartesian monoidal category of directed-complete partial orders.

1 Introduction

Although the mathematics of games shares a common ancestor in Zermelo’s work on backward induction, it split early on into two subjects that are essentially disjoint: game theory and game semantics. Game theory is the applied study of modelling real-world interacting agents, for example in economics or artificial intelligence. Game semantics, by contrast, uses agents to model – this time in the sense of semantics rather than mathematical modelling – situations in which a system interacts with an environment, but neither would usually be thought of as agents in a philosophical sense. On a technical level, game semantics not only restricts to the two-player zero-sum case, but moreover promotes one of the players to be the Player, and demotes the other to mere Opponent. This induces a deep logical duality that pervades game semantics, apparently destroying any hope of bridging the gap to game theory, which typically involves players treated symmetrically.

Compositional game theory [Hed16, GHWZ18], as its name suggests, is an attempt to introduce the principle of compositionality into game theory, motivated by practical concerns about modelling large (for example economic) systems. It is loosely inspired by game semantics, as well as categorical quantum mechanics [AC04, CK17] and much recent work in applied category theory (e.g. [Fon16]). On a technical level, game semantics involves (typically monoidal) categories in which games are the objects and strategies (with various conditions) are the morphisms, whereas open games form the morphisms of a monoidal category. This means that open games can be denoted by string diagrams, which is invaluable for working with them in practice. As with other categories of open systems, ordinary “closed” games are recovered as scalars, or endomorphisms of the monoidal unit (see [Abr05]), and depicted as string diagrams with trivial boundary.

Central to understanding open games is the concept of a context, which is a compressed representation of a game-theoretic situation in which an open game can be played. Whereas an ordinary game has a set of strategy profiles and a subset of those which are Nash equilibria, in an open game the equilibria depend on the context. This is the key to reuniting game theory and game semantics: we ignore the linguistic coincidence of the term player, and instead view an open game as the System and the choice of contexts as the Environment.

The essence of this idea is already contained in the following quote from the introduction of [Abr97]: “If Tom, Tim and Tony converse in a room, then from Tom’s point of view, he is the System, and Tim and Tony form the Environment; while from Tim’s point of view, he is the System, and Tom and Tony form the Environment.” The view of open games presented in this paper makes this precise when Tom, Tim and Tony are players in a noncooperative game.

In [Hed18] open games were first reformulated in terms of lenses from functional programming [PGW17]. This was extremely useful as a technical trick, but lenses are usually used as destructive update operators on data structures and it is unclear what they have to do with game theory, if anything. The key was a comment by Dusko Pavlovic to the author that the category of lenses is a dialectica category [dP91]; combined with a game-semantic view of dialectica categories [Bla91] we can see open games in their true form: as an interleaving of game theory and game semantics.

Specifically we find that an open game is a dialogue of a particular sort played between a system and its environment. The system is jointly controlled by noncooperative players, each independently optimising a real-valued payoff. The winning condition turns out to be Nash equilibrium: the goal of the system is to play an equilibrium, and the goal of the environment is to prevent it. Specifically, an open game consists of three pieces of data: a set of strategy profiles, a labelling function , and a winning (for ) relation .

In the second half of this paper, we apply these ideas to build a compact closed category of ‘computable open games’ by replacing the underlying dialectica category with a wave-style geometry of interaction category, specifically the Int-construction applied to the cartesian monoidal category of directed-complete partial orders. Ultimately we rely on the following transport of structure result:

Proposition 1.

Let be a compact closed category, a symmetric monoidal category and a strict monoidal functor that is bijective on objects. Then can be given a compact closed structure, with duals given by , units by and counits by .

Proof.

The assumption that is bijective on objects means that every object of is uniquely assigned a dual, unit and counit. It is simple to check the yanking equations [KL80]:

and similarly for the other equation. ∎

The hypotheses of this theorem are already satisfied by a particular functor that identifies with the subcategory of zero-player open games. Thus it suffices to replace the source category with one that is compact closed, while preserving the hypotheses (and the game-theoretic interpretation).

2 Dialogues

While the name ‘dialectica’ should bring to mind dialogues in the tradition of philosophical logic (for example via Hegel’s dialectics), this is apparently a coincidence. The dialectica interpretation is named after the journal Dialectica, who published Gödel’s paper in their Paul Bernays festschrift. But the dialectica interpretation does have a very dialectical feeling to it.

The game semantic viewpoint on Gödel’s dialectica interpretation [AF98] and de Paiva’s dialectica categories [dP91] was described in Blass’ paper that first introduced game semantics [Bla91]. It has not been considered much since, and is now largely folklore. In this section we recall this viewpoint in detail.

We first introduce a category of dialogues and strategies, which is the dialectica category over an inconsistent (1-valued) logic.

An object of is a 2-stage dialogue in which first the System chooses , and then the Environment chooses . This breaks a common requirement in game semantics that Environment moves first. We denote the dialogue by .

Notice that the set of -strategies for is , and the set of -strategies is .

We introduce a monoidal product operator given by synchronous parallel play. Specifically, the parallel play of and is the 4-stage dialogue . This peculiar ordering of moves, with the right-hand dialogue being played in the middle of the left-hand dialogue, is characteristic of dialectica. This 4-stage dialogue is strategically equivalent to the 2-stage dialogue , so we set .

Next, given objects and , we consider the same 4-stage dialogue but with the players interchanged in the former. That is, we consider the dialogue . We consider this to be played relative to , and denote it by .

The set of -strategies for is , or isomorphically . The set of -strategies is . We denote the set of -strategies for by . As the notation suggests, these are the morphisms of .

Given an object , there is a copycat -strategy for . As an element of it is the pair consisting of the identity and the projection. This is the identity morphism for . Following [Abr97] we denote this strategy by a string diagram:

(This syntax is strikingly similar to grammatical reductions in pregroups [CSC10].)

Now suppose we are given -strategies and . There is a way to combine them to produce a -strategy . Namely, simulates playing the two together with playing a copycat strategy for the middle moves. That is, she simulates the 8-stage dialogue

with the assumption that uses a copycat strategy for the moves and . By then hiding the and moves we get a -strategy for the required 4-stage dialogue.

We denote this strategy by the following string diagram:

Whereas the cap denotes a copycat -strategy, the cup denotes a copycat -strategy.

A little calculation shows that if is given by and , and is given by and , then the composite is given by

and

It is routine to check that this is associative, with identities given by copycat. Thus is indeed a category.

Given this category structure we can also make into a genuine symmetric monoidal product. Given -strategies and , we can combine them to produce a winning strategy .

Finally, we notice that all of the above can be generalised to any base category with finite products, replacing sets and functions, yielding a category whose morphisms are strategies internal to . Specifically, we set

(By writing it this way, we do not need to assume that is cartesian closed.) The category we have been considering so far is .

Proposition 2.

For any category with finite products, is a symmetric monoidal category.

There is a much less obvious generalisation of when is only a monoidal category [Ril18], but we will not need it in this paper.

3 Negation and -strategies

To talk about open games, we need to talk explicitly about -strategies in a dialogue. However, the categorical structure of is built on -strategies. In turns out, however, that we can use -strategies to talk about -strategies, in a way that respects composition.

Recall that the monoidal unit of is the trivial game . The dialogue is , which is strategically equivalent to . Thus the set of -strategies for is .

If we fix a -strategy for and another -strategy , we can compose them to yield a -strategy for , by

Succinctly, there is a functor taking every object to its set of -strategies, namely the covariant functor represented by . Explicitly, and .

On the other hand, the dialogue is , which is equivalent to . This is not an object, but is with players interchanged. Thus the set of -strategies for is equal to the set of -strategies for , namely .

Given an -strategy for and a -strategy , we obtain an -strategy for by

In this, ‘hijacks’ ’s strategy to produce an element of , since is a -strategy for a dialogue in which plays the role of in .

Succinctly, there is a functor taking every object to its set of -strategies, namely the contravariant functor represented by .

An -strategy for consists of a -strategy for and an -strategy for . This defines a functor , namely

(where is intended to stand for ‘cohomomorphism’, not ‘cohomology’!) On objects, it is concretely given by , or more generally over a category with finite products, .

Given an -strategy for , a -strategy and a -strategy , we obtain an -strategy for . This is the -strategy for the dialogue

with appropriately hidden copycat moves, as given by the string diagram

4 Open games

We can now give an equivalent definition of open games [Hed16, GHWZ18] in terms of dialogues.

An open game is in one dimension a dialogue played between a System and an Environment, and in another dimension it is a non-cooperative game in the sense of economics, in which several players jointly control the System while independently optimising payoffs.

An open game is defined by three pieces of data:

  • A set of strategy profiles

  • A labelling function , by which every element labels a -strategy for the 4-stage dialogue

  • A winning condition, which is a relation between and the set of -strategies of , namely .

We write for . We say that is a winning strategy profile if for all .

We interpret as an equilibrium condition. That is, from the dialogue perspective the goal of the System is to reach equilibrium and the goal of the Environment is to prevent equilibrium. In real examples there is rarely a winning strategy profile, and so we focus on as a binary relation, or ask about winning strategy profiles for the System against a fixed -strategy.

From the dialogue perspective, the order of play in an open game is:

  1. The Environment chooses an initial state of the game from

  2. The System chooses the final state of the game from

  3. The Environment chooses payoffs for the System from

  4. The System chooses payoffs for the Environment from

An -strategy is a pair where and . The history determines the initial state of the game. The continuation determines the payoffs for System given the final state. The pair completely determines the strategic context in which the players that make up System make their choices, reducing the open game to an ordinary normal-form game. For this reason, we also call an -strategy a context for the open game.

We only need two families of examples of open games to generate a large family of examples, corresponding roughly to extensive-form games, using the sequential and parallel play operators we will define in the next section. These two generating families are the zero-player and the one-player open games.

The zero-player open games are in bijection with the -strategies , and correspond to the situation in which the System has no strategic choices but always follows the strategy like an automaton. Specifically, the zero-player open game is defined by:

  • The set of strategy profiles is the singleton , where is a token representing the -strategy

  • The labelling function is

  • is a winning strategy profile, that is, for all -strategies

Perhaps the only surprising part of this definition is that is a winning strategy profile. The reason for this ultimately comes down to agreeing with Nash equilibrium on real examples. Nash equilibrium is a negative definition: a strategy profile should fail to be a Nash equilibrium if some particular player has positive incentive to deviate from it. Since there are no players in , is declared a Nash equilibrium by default.

The second family of examples are the one-player open games. There is one such open game for every nonempty set and . In this game:

  1. The Environment chooses an initial state from

  2. The (now unique) Player chooses a final state from

  3. The Environment chooses a payoff from

The winning condition of this game is intensional by being a property of the strategies of both Player and Environment, and cannot be written in terms of the play alone. This is because optimality in game theory is a counterfactual: if the System had made a different choice then the resulting payoff would be lower.

Observe that a -strategy for this game is a function , while an -strategy is a pair where and . By definition, the Player wins this game iff , that is to say, if for all .

This is a small shift in perspective that is quite natural from the perspective of game semantics. In game theory there is no concept of winning, only optimality and equilibrium. Declaring a player to have won if they make an optimal choice may not be meaningful as game theory, but it is appropriate terminology when combining game theory with game semantics.

Writing this out:

  • The set of strategy profiles is

  • The labelling function takes to itself considered as a -strategy , via the bijection

  • The winning condition is iff

5 Composing open games

We can make open games into the morphisms of a symmetric monoidal category. The two composition operators, categorical composition and tensor product, correspond to sequential play and simultaneous play.

Suppose we have given open games and . The sequential composition has set of strategy profiles . The idea is that and are each associated with sets of decisions. Each decision has an associated set of strategies, and the set of strategy profiles in each case should be thought of as the set of tuples of strategies, one for each decision: and . The set of decisions made in a composite game is the disjoint union of the decisions made in the components, and so .

The labelling function for a sequential composition can be defined using the underlying composition in : .

In order to define the winning condition of , we must modify a context for into contexts for and . We can do this using the fact that is a functor, together with the fact that we have strategy profiles for and available. A strategy profile for is winning (that is to say, a Nash equilibrium) against the -strategy iff is winning in against the -strategy , and is winning in against the -strategy . That is to say,

This makes open games into the morphisms of a category. (More properly it is a bicategory, and only a category after quotienting out compatible bijections of strategy profiles. This is a purely technical complication that we will ignore in this paper.)

Next we consider simultaneous play. Given open games and , we combine them to form an open game

As before the strategy profiles of are pairs, , for the same reason as before: we take the disjoint union of the set of decisions. The strategy profile labels the synchronous parallel play of and , that is, .

In order to define the winning condition for we need to do some more work.

Given strategy profiles and , and an -strategy for , we need to ‘project’ to and ’s view of it, as -strategies for and .

We can indeed do this. To produce an -strategy for , consider the dialogue

with the strategy

We call this -strategy . When we write . Concretely, the new continuation is .

Similarly, we can produce an -strategy for by considering the same dialogue with the strategy

When we write , where .

With this, we can finally define the winning condition for : The strategy profile is winning against in iff is winning against in and is winning against in , that is to say,

Proposition 3.

There is a symmetric monoidal (bi)category whose objects are pairs of sets and morphisms are open games.

What cannot be stated easily is that this categorical structure moreover does the right thing in game theory. We can draw string diagrams in this monoidal category that reflect the information flow in a game, and interpret them in the category , and find that the equilibrium condition is the same as the one given by a standard game-theoretic analysis. The reader is referred to [GHWZ18] for details, with proofs in [Hed16] and a formal analysis of the relevant string diagrams in [Hed17].

6 Dialogues and wave-style geometry of interaction

In order to obtain a connection between the dialectica and constructions, we need to apply the construction to categories that are traced cartesian monoidal. This is wave-style geometry of interaction, so-called because every point in our string diagrams is consistently assigned a value [Abr96]. (It is contrasted with particle-style GoI, which applies to monoidal categories built on a coproduct and in which we imagine a token moving around the diagram.)

Game-semantic interpretations of wave-style GoI have not been widely considered. In this section we suggest such an interpretation that will be suitable for our purposes.

The -construction can be defined over any traced monoidal category , but we restrict to traced cartesian categories. These are equivalent to Conway cartesian categories, or cartesian categories with a natural family of fixpoint operators [Has99]. A canonical example is the category of directed-complete partial orders and continuous maps.

By definition, an object of the category is a pair of objects of , and a morphism in is a morphism in . Since is cartesian monoidal, a morphism is equivalently a pair of morphisms and .

The identity on in is the identity on in . The composition of and in is given by

in , using the string diagram language for traced monoidal categories [Sel11, section 5.7].

The monoidal product of is defined on objects by , with the obvious definition on morphisms. As is well known, can be equipped with the structure of a compact closed category, which satisfies the universal property of being the free compact closed category on the traced monoidal category . Note that there are two different conventions in use: we follow [JSV96], which defines with a twist in the contravariant place, rather than [Abr96] which does not.

The idea of interpreting objects and morphisms of as dialogues is to view them as repeated play of the corresponding dialogues for , starting from and converging to a fixpoint, after which the play terminates and all moves except the final ones are hidden.

We view the object as a dialogue

We do not allow arbitrary strategies, but restrict the allowed -strategies to -morphisms , and the allowed -strategies to the -morphisms . Given such a pair of strategies , the play that results is by definition

When is or another suitable category, this play stabilises after finitely many stages to the that is the least fixpoint of the recursion , . By hiding the approximating moves, we consider the play resulting from to be .

We can succinctly represent this repetition and hiding by extending our string diagram language for strategies with a trace.

Given objects and , the dialogue is

We restrict the allowed -strategies to -morphisms and the allowed -strategies to -morphisms . Given a -strategy and an -strategy , the resulting play is

This stabilises after finitely many stages to which is the least fixpoint of , . Again we hide the approximating moves so that is the visible play.

7 From dialectica to geometry of interaction

The previous section suggests that every -strategy in can also be viewed as a -strategy in . We could also discover this fact simply by inspecting the definitions, without thinking in terms of dialogues.

Incidentally, [HH17] refers to the Int-construction as “bidirectional computation”, a technical term that usually refers to lenses and related constructions (e.g. [GS16]).

In this section we use string diagrams in the underlying category . This is the language of traced symmetric monoidal categories [Sel11, section 5.7] which are cartesian monoidal [Sel11, section 6.1]. Implicitly, string diagrams for cartesian monoidal categories use the fact that a monoidal product is cartesian iff every object can be compatibly equipped with a commutative comonoid structure making every morphism into a comonoid homomorphism [Fox76].

Proposition 4.

Let be a traced cartesian category. Then there is a strict monoidal functor , which is identity on objects and takes the strategy to

Proof.

The identity morphism of is sent to

which is equal to the identity on since the black structure is a comonoid. This is the identity morphism of .

Next, consider morphisms and in . If we compose them in we obtain the morphism with string diagram

Using the fact that is a comonoid homomorphism, followed by coassociativity and symmetry of the black structure, we can transform this to

On the other hand, if we compose in , we obtain with string diagram

By inspection, we see that these string diagrams are equivalent. Equality of the two morphisms then follows from the coherence theorem for traced symmetric monoidal categories [Sel11, theorem 5.22].

Finally, it can be seen by inspection that the functor is strict monoidal, since and have the same objects and the monoidal product is defined in the same way. ∎

The author conjectures that when is only a monoidal category there is still an identity-on-objects strict monoidal category , where is now taken to be the more general category of optics [Ril18].

We also note that the functor takes the counit lens to the morphism that is the counit of the compact closed structure of .

8 Abstracting open games

Inspecting the definition of open games, it appears that we can define open games over any symmetric monoidal category with a chosen functor . This does indeed give us a category of open games, but defining a monoidal product of open games requires an additional piece of structure, namely the ability to project individual -strategies out of a -strategy for a composite. This is axiomatised by the following definition.

Definition 1.

A context for a symmetric monoidal category is a symmetric monoidal functor together with a natural family of functions

The naturality condition required is that for all morphisms , and , the diagram

in commutes.

Using the symmetry, we can derive from this a natural family of functions

and vice versa.

The structures we defined earlier do indeed give a context on , namely

There are trivial examples of contexts that carry no game-theoretic information, which we will ignore. For example, we can always take to be a constant functor. We give a second family of nontrivial examples, which we will use later.

Proposition 5.

Every traced monoidal category can be equipped with the context , with defined by

Proof.

Let , , and . We chase the context around the commuting diagram in definition 1. By the upper route we obtain

and by the lower route we obtain