Dynamic landscape models of coevolutionary games
Players of coevolutionary games may update not only their strategies but also their networks of interaction. Based on interpreting the payoff of players as fitness, dynamic landscape models are proposed. The modeling procedure is carried out for Prisoner’s Dilemma (PD) and Snowdrift (SD) games that both use either birth–death (BD) or death–birth (DB) strategy updating. The main focus is on using dynamic fitness landscapes as a mathematical model of coevolutionary game dynamics. Hence, an alternative tool for analyzing coevolutionary games becomes available, and landscape measures such as modality, ruggedness and information content can be computed and analyzed. In addition, fixation properties of the games and quantifiers characterizing the interaction networks are calculated numerically. Relations are established between landscape properties expressed by landscape measures and quantifiers of coevolutionary game dynamics such as fixation probabilities, fixation times and network properties.
For describing evolutionary dynamics the framework of fitness landscapes has been introduced, [Kauffman and Johnsen, 1991, Richter and Engelbrecht, 2014, Stadler and Stephens, 2003]. A fitness landscape formulates relationships between genetic specifications, individual instantiations, and their fitness. Together with postulating differences in fitness over all possible genetic specifications and a moving bias towards higher fitness, the setup suggests the picture of an evolving population that is moving directedly on the landscape. On a conceptual level, this picture is based on the notion of evolutionary paths that are created by the topological features of the fitness landscape. Evolutionary paths are a succession of moves on the landscape with ascending fitness values. The existence and abundance of such evolutionary paths gives rise to estimates about how likely a transition from low–fitness regions to high–fitness regions in the landscape is.
Apart from fitness landscapes, another approach for specifying evolutionary dynamics is evolutionary games, [Szabo and Fath, 2007, Nowak and Sigmund, 2004, Nowak, 2006, Maynard Smith, 1991]. Evolutionary games are mathematical models of dynamic interactions between individuals in a population and explain how their behavioral strategies (for instance cooperation or competition) spread in a population. The main question is how adoption of the strategies contributes to payoff allocation and consequently to the fitness characterizing the success of each individual. An evolutionary game becomes dynamic if it is played iteratively over several rounds and the individuals are allowed to change their strategies and/or to recast the network describing with whom they are interacting. Such an iterated evolutionary game comprises of an evolving population of individuals acting as players and can be seen as an expression of evolutionary dynamics.
Given the fact that there are two frameworks for addressing evolutionary dynamics, it is natural to ask about their relationships. Unfortunately, both frameworks are not immediately compatible. Although it is acknowledged that evolutionary games cast fitness landscapes, it has become clear that such game landscapes change with an evolving population of players, [Nowak and Sigmund, 2004]. This is attributed to frequency–dependent selection. In other words, game landscapes are dynamic. Based on some earlier results on dynamic fitness landscapes, e.g. [Foster et al., 2013, Richter, 2008, Richter, 2014b], there are some first attempts at applying these ideas to games, for instance, [Richter, 2015]. In this paper dynamic landscapes are employed for analyzing coevolutionary games by using and extending a framework introduced recently, [Richter, 2016]. Games are considered where the players may update their strategies (evolutionary games), see e.g. [Allen and Nowak, 2014, Greenwood and Ashlock, 2012, Szabo and Fath, 2007], but also games where the players may additionally change their interaction network (coevolutionary games), see e.g. [Perc and Szolnoki, 2010, Szolnoki and Perc, 2009a, Tanimoto, 2007]. In particular, it is shown that the proposed method makes it possible to model and analyze evolutionary games and coevolutionary games within the same framework.
The paper is structured as follows. In Sec. 2, some basic definitions are given, and evolutionary and coevolutionary games are briefly recalled. Sec. 3 reviews game dynamics, particularly the processes to update strategies and interaction networks. Dynamic landscape models of coevolutionary games are introduced and discussed in Sec. 4. It is shown that evolutionary games can be modeled by either player landscapes or strategy landscapes. As a player landscape is dynamic due to frequency–dependence, it is difficult to extend such a model to coevolutionary games. This paper introduces a method by which player–wise strategy landscapes can be aggregated to obtain a game landscape, which is static for evolutionary games and dynamic for coevolutionary games. The modeling procedure is demonstrated for Prisoner’s Dilemma (PD) and Snowdrift (SD) games that both use either birth–death (BD) or death–birth (DB) strategy updating. It is further shown that BD and DB updating yield landscapes with symmetry properties, and that replacement restrictions entail symmetry breaking. Moreover, the local topological features of absorbing configurations of the games are interpreted as absorption structure. It is described how landscape properties may be linked to fixation via the absorption structure. Sec. 5 reports numerical experiments on landscape measures such as modality, ruggedness and information content. Fixation probabilities and fixation times are calculated as well as network measures characterizing the interaction networks of the coevolutionary games. It is shown and discussed how the landscape measures relate to both fixation properties and network measures. Sec. 6 closes the paper with a summary and conclusions.
2 Definitions and game description
The coevolutionary dynamics of the games considered in this paper stems from three levels of activity: (i) game playing, (ii) updating the strategy, and (iii) updating the interaction network. The game playing is done by a finite population of players that use one of two strategies during each round . A player , , can either cooperate () or defect (). A pairwise interaction between two players and (which can be seen as player and coplayer) yields rewards in form of payoff as given by the payoff matrix
Here, stands for temptation to defect, is reward for mutual cooperation, is punishment for mutual defection, and the sucker payoff for cooperating with a defector. Depending on the numerical values of and their order, particular examples of the game are obtained, which have become metaphors for studying social dilemmas and discussing strategy selection along with the effect on short– and long–term success in accumulating payoff, [Maynard Smith, 1991, Nowak, 2006]. Most prominently, there are Prisoner’s Dilemma (PD) games, where , and Snowdrift (SD) games, where .
The payoff of a player in round depends not only on the player’s strategy and the values of the payoff matrix (1), but also on who its coplayer is (or more precisely as to what the coplayer’s strategy is) and how many coplayers there are. The question of who–plays–whom in a given round of the game is addressed by the interaction network. A convenient way of expressing and visualizing the interaction network is by using elements from evolutionary graph theory, [Allen and Nowak, 2014, Lieberman et al., 2005, Ohtsuki et al., 2007, Shakarian et al., 2012]. Evolutionary graph theory places each player of the population on a vertex of an (undirected) graph. This graph describes the interaction network and consequently it can be called an interaction graph. As there are no empty vertices and a vertex can only be occupied by one player, the number of vertices of the graph equals the number of players . For each player, its coplayers are indicated by edges that connect the vertex of the player with the vertices of the coplayers. Such an edge defines the connected players to be adjacent. Each vertex can have up to edges (self–play is not allowed). As the degree is the number of edges incident with a vertex, the degrees of the interaction graph equal the number of coplayers that are engaged with each player in a single round. A graph is called regular if the degree is the same for all vertices. Hence, a regular interaction graph means that all players have the same number of coplayers.
The interaction graph can be described algebraically by its (interaction) adjacency matrix , which is called an interaction matrix. The matrix is a symmetric matrix with entries indicating an edge between vertex and and showing that players and are not adjacent. The diagonal elements because there is no self–play. The symmetry of reflects the fact that two players and mutually engage in the game. From the perspective of player , the player may be the coplayer. If so, then from the perspective of player , the player is the coplayer. Consequently, in the adjacency matrix . If all (except ), the graph is complete, meaning that every player has all other players as coplayers and the evolutionary game is understood to be well–mixed, [Szabo and Fath, 2007]. To summarize, for describing completely and deterministically the game and the allocation of payoff, apart from the payoff matrix (1) only two other entities are necessary: the strategy vector with and the adjacency matrix . This setting deterministically fixes the payoff for each player. Hence, the distribution of payoff amongst the players remains the same if the players were to engage in the game with the same entities for a second time in round . Put another way for these entities being constant, the game can be seen as static. Consequently, making the evolutionary game dynamic requires updating either the players’ strategies or the interaction network, or both.
3 Coevolutionary game dynamics
3.1 Updating strategies
There is a huge amount of work devoted to the modes of updating the player strategies in evolutionary games, [Allen and Nowak, 2014, Ohtsuki et al., 2007, Pattni et al., 2015]. Most models use versions of stochastic strategy updating based on a Moran process, but there are also works emphasizing limiting the effect of randomness and including the self–interest of players, e.g. [Greenwood and Avery, 2014]. According to a Moran process, in each round a player (or more precisely its strategy) is replaced by (the strategy of) a player . The players and are selected at random, but the probabilities of the selection may not be uniform, for instance depending on the players’ fitness, which may vary. Versions of stochastic updating rules differ in several respects. Differences are, for example, the actual probabilities that given players and are selected or whether or not there is an order between selecting the player providing the strategy (the source) and selecting the player receiving the strategy (the target). Finally, there may be general restrictions as to which players are allowed to be a possible source and/or target of another player. Such predetermined restrictions imply a replacement structure, [Ohtsuki et al., 2007]. Conceptually similar to interaction, the question of who–may–replace–whom can be described by a network of replacement. This network is expressible by a replacement graph and consequently by a (replacement) adjacency matrix , which is called a replacement matrix. The matrix has entries , and indicates that player may provide its strategy for player to receive. The values of contribute to the probabilities that player is source and player is target. If all for a constant , every player may be the source to every target player with equal probability. Consequently, if there are no restrictions, the replacement graph is fully connected with evenly weighted edges.
Amongst strategy updating, the following replacement rules are frequently studied: birth–death (BD), death–birth (DB), imitation (IM), and pair–wise comparison (PC), [Allen and Nowak, 2014, Pattni et al., 2015, Shakarian et al., 2012]. All these Moran–based updating rules share that they depend only on random (and possibly on players’ fitness and replacement restrictions), but not on details of the interaction (for instance who the source or target are actually interacting with and what those strategies are). Therefore, they do not account for self–interested players, [Greenwood and Avery, 2014]. This property makes it possible to disentangle player and strategy in the sense that it makes no difference from which source the target receives its strategy updating. In other words, for all these updating rules it is possible to specify probabilities that the strategy of a source replaces the strategy of a target depending only on replacement matrix and fitness, [Pattni et al., 2015].
3.2 Updating interaction networks
If, in addition to the strategies, also the interaction network can be updated in evolutionary games, the game is called coevolutionary. However, the players of the coevolutionary game are functionally alike and can hence be thought as belonging to the same species. Therefore, coevolution takes place within a single population of players and is between different features of the players’ function. The focus of this paper is on coevolving of game strategies and interaction networks. There are alternative settings, such as coevolution between game strategies and other features or parameters of the players, for instance their ability to promote their strategies, which is known as teaching, [Szolnoki and Perc, 2008, Szolnoki and Perc, 2009b], or their temptation to defect, which affects their perception of the social dilemma and leads to multigames, [Szolnoki and Perc, 2014]. Apart from the network structure, coevolution can also act on network interdependence, [Wang et al., 2014]. All these coevolutionary settings are methodologically different from an alternative understanding of coevolution, which is between different ecological functions (and hence different species), for instance between predator and prey, or between host and parasite, see e.g. [Thompson, 1995].
According to the focus of this paper, coevolution in evolutionary games is in essence considering the interaction network as dynamic, from which follows that the interaction matrix must be time–dependent. There is a substantial variety of schemes and rules for coevolution, [Pacheco et al., 2006, Perc and Szolnoki, 2010, Szolnoki et al., 2009, Szolnoki and Perc, 2009a, Tanimoto, 2007]. Unfortunately, the topic of dynamic network updating has not yet matured as far as to express for a given coevolutionary rule the transitions from one interaction network to another as a probabilistic function. Whereas for strategy updating, there are replacement probabilities for different updating rules, [Pattni et al., 2015], the same is not known for network updating. However, it might be reasonable to assume that network updating involves creating an interaction matrix at point in time from a matrix at the previous point , for an integer time variable .
Such a succession of interaction networks can be modeled by instances of an Erdös–Rényi graph. In this paper, the discussion is restricted to the case where the number of coplayers is the same for all players and constant for all updating instances. Employing such a model precludes situations where a more highly connected player possesses high fitness due to its connectedness, but not necessarily due to the effectiveness of its strategy. For coplayers, such an interaction graph has degree and belongs to a special class of Erdös–Rényi graphs, namely random –regular graphs. Modeling the interaction network by random –regular graphs makes it possible to systematically carry out numerical experiments because recently efficient algorithms for generating such graphs became available, [Bayati et al., 2010, Blitzstein and Diaconis, 2011, Kim and Vu, 2003]. Moreover, for random –regular graphs, some analytic results about the number of different graphs are known. This, in turn, corresponds to the number of possible player–coplayer combinations. As a –regular graph with vertices has edges, the number needs to be even. Thus, employing such an interaction network model implies that we cannot have an odd number of players with an odd number of coplayers.
For a small number of edges ( coplayers) , the number of different –regular graphs on vertices ( players) can be found by enumeration, see for instance the entries in the Sloane encyclopedia of integer sequences, [Sloane, 2016]. Thus, and , while , , and , and , . Note that for all , which means that a complete interaction network representing a well–mixed population holds only one instance of the matrix . Thus, for a complete network graph the game cannot be coevolutionary. It is always static with respect to interaction because no dynamic changes can be cast out of a single instance of . Further note that grows rapidly. For interaction networks with coplayers, the number of possible player–coplayer combinations can be calculated exactly, as there is a recursive formula for the number of –regular graphs, [Bollobás, 2001], p.56:
valid for , with , and . For , no formula is known to compute exactly the total number of –regular graphs on vertices, but asymptotic expressions have been found, [Wormald, 1999]. Asymptotically, and for and even, the number is
Based on a collection of random –regular graphs the effect of different networks of interaction on payoff collecting and fitness can be analyzed, for which a landscape approach is proposed in the next section.
4 Landscape models of game dynamics
4.1 Static and dynamic fitness landscapes
A general definition of a (static) fitness landscape is the triple , where is a configuration space, is a neighborhood structure that assigns to every a set of direct neighbors, and is a fitness function that provides every with a proprietary quantity to be interpreted as a ’quality’ information or fitness, [Richter and Engelbrecht, 2014, Stadler and Stephens, 2003]. In this definition, the configuration space together with the neighborhood structure expresses a (multi–dimensional) ’location’, while fitness is a property of this location. The configuration space itself can be seen as an unordered (finite or infinite) list of configurations that genetic specifications of biological systems can have. The neighborhood structure defines a (possibly multi–dimensional) order of this list by establishing what is directly next to each element of the configuration space. As direct neighbors of an element have a neighborhood structure themselves, this naturally establishes distant neighbors of the element as well.
The definition of a (static) landscape has the consequence of each configuration possessing a constant fitness value. For several reasons this might not realistically reflect the evolutionary scenario to be described and may generally restrict the descriptive power and versatility of the landscape model. Hence, assuming that fitness may change over time, while configuration space and neighborhood structure remain constant, the definition above can be extended to a dynamic fitness landscape, which can be expressed as the quintuple , [Richter, 2014a]. In addition to the elements of the static landscape, there is the time set , the set of all fitness functions in time , and the transition map defining how the fitness function changes over time. For a discrete time set , for instance for the integer numbers , the notion of a dynamic landscape coincides with the notion of a series of static landscapes. Hence, two static landscapes and can be reformulated as one dynamic landscape with and describing how changes into . Such a dynamic landscape model implies the time variable to act as an integer counting and ordering scale for dynamic instances of a static landscape. Hence, is numerically tantamount to yet conceptually different from counting the rounds of an coevolutionary game by .
Applying a landscape approach for describing evolutionary dynamics requires addressing what may constitute a configuration and its neighborhood , and also what defines fitness . For the coevolutionary games described in the previous sections, there are several modeling options, which are discussed in the following. The actual modeling choice of , and and their interdependencies may either result in a static landscape or entail a landscape that is dynamic and additionally requires , and to be specified.
Evolutionary as well as coevolutionary games allocate payoff to players according to the payoff matrix (1). For making the payoff interpretable as reproduction rate or survival probability (and lastly as fitness ), it has been suggested to rescale by a positive, increasing, differentiable function, [Allen and Nowak, 2014, Shakarian et al., 2012]. In the following the linear function is used with the intensity of selection .
4.2 Player landscapes
The simplest landscape model of an evolutionary game arises from equating configurations with players , which for players leads to a player configuration space with elements. The neighborhood structure follows from the coplayers that each player has, which can be . Thus, the neighborhood of a player consists of all the other players with which it is mutually engaged in a game according to the interaction matrix . Hence, assuming that the players can be attributed with a fitness , such a player landscape could be specified by . A popular form of such player landscapes is to place the players on a two–dimensional square lattice and define the coplayers to be Von Neumann (or Moore) neighborhoods, which consists of the lattices cells orthogonally (or additionally diagonally–adjacent) surrounding a central cell, [Nowak and May, 1993]. Admittedly, such an arrangement fixes the number of direct neighbors to (or , but yields a convenient way of visualizing the quality information over the resulting two–dimensional structure, which might be one reason for the popularity of these neighborhoods. The most obvious choice of the quality information is payoff or quantities directly derived from it such as the linear fitness introduced earlier. This has led to label such landscapes as payoff landscapes, [Brede, 2011].
There are, however, several problems with such a player landscape model. The main problem is that the configuration is the player, not its strategy, nor the strategies of its coplayers. Hence, with the player’s and coplayers’ strategies, two quantities decisive for the amount of payoff are not directly attached to the configuration. Strategies can be seen as ambiguous and polyvalent properties of the configuration of players. This means that the payoff attributable to a configuration depends on both the player’s strategy and also on the strategies of its neighboring coplayers. This aspect is known as frequency–dependence, as the payoff can be seen as to depend on how frequent the strategy that the player adopts also occurs in the coplayers. Consequently, frequency–dependent fitness refutes the assumption that each player can be attributed with a unique and static fitness. In short, fitness derived from payoff can be seen as dynamic so that the real player landscape cannot be static, but should be dynamic: . However, the dynamics of is caused not only by frequency–dependence, but also by strategy updating for which the player landscape model does not directly account and both these causes can hardly be separated from each other. Hence, the transition map describing how the fitness relates to is not straightforwardly definable. In addition, modeling configurations of a landscape by players means that the neighborhood structure is expressed by the adjacency matrix . A variable interaction network, as in coevolutionary games, therefore implies a changing neighborhood structure. To conclude a player landscape of a coevolutionary game would involve changing neighborhood structure as well as dynamic fitness. This may make analyzing such a landscape rather complicated.
There is another reason for the difficulties to deduce meaningful conclusions from payoff–based fitness over a player landscape. Topological features of the landscape can be used as a starting point for estimating how likely transitions from low–fitness configurations to high–fitness configuration are and also which configurations are most likely to be a steady state of evolutionary dynamics. However, which player in a symmetric game as specified by the payoff matrix (1) exactly is a likely high–fitness outcome of an evolutionary process is not very relevant. A much more important question is what fraction of the players in the long run settles stably to one of the possible strategies. In pursuing this question, there are several works that define the quality information of the landscape to be the strategy to which a player temporarily or finally settles. This means the ’fitness’ is expressed by the strategy vector . The results have been visualized by coloring the players according to their strategy, [Nowak and May, 1993, Nowak and Sigmund, 2004]. Such a model has the advantage that the spatial and temporal distribution of the strategy preferences can be visualized with respect to the player–coplayer structure. However, payoff–based fitness as the main drive of evolutionary game dynamics is not an explicit component of such a landscape and the number of coplayers is defined by the restrictions of the adjacency of the lattice grid.
4.3 Strategy landscapes
An alternative landscape model arises from equating configurations with all possible combinations of strategies that each player and its coplayers can have. An element of the strategy configuration space is comprised of the strategies of any player , , and its up to coplayers: . The strategy configuration space generalizes the time–dependent strategy vector towards all of its possible instances. Hence, for players with two possible strategies, contains elements. If we binary code the strategies cooperation and defection (for instance , ), an element appears as binary string of length . Note that for this case the bit–count of , , provides a simple way of expressing the fraction of cooperators . It is assumed that only one player or coplayer can change its strategy at a given point in time. This implies a neighborhood structure where each element has direct neighbors which are distanced by Hamming distance of , which is denoted by . For instance, With such a model we obtain for payoff–based fitness a unique and static landscape for each player and each interaction network. As the game specified by the payoff matrix (1) is symmetric, the strategy landscapes are topologically alike for all players . The landscapes can be transformed into each other by shifting and reshuffling. For a landscape interpretation this topological likeness implies that landscape quantifiers such as the number of maxima, or correlation structure, or information content do not vary over the .
For , the landscapes can be visualized in two dimensions, see Fig. 1. It shows , , for the payoff matrix and the adjacency matrix specifying a PD game with a complete interaction network and coplayers for each player .
We find that and hence the game is static with respect to updating the interaction network. Observe that for each player there is only one maximum fitness value (the player is defecting, while all coplayers cooperate) and one minimum fitness value (the player cooperates, while all coplayers defect). Apart from the single maximum and the single minimum, there are several configurations that have the same fitness value. Interestingly, these configurations do not form a neutral network, [Richter and Engelbrecht, 2014], as they have Hamming distance of and hence are not direct neighbors. From the strategy landscape it can be concluded which strategy for the player (with respect to the strategies of the coplayers) yields the highest fitness and is therefore most desirable from the perspective of . Nonetheless, the evolutionary path from a given initial configuration may depend on, and be influenced by, the strategies provided to and/or received from the coplayers. In addition, from the perspective of another player, another strategy configuration is best. Best configurations for respective players, however, are mutually exclusive, which is a defining feature of social dilemma games such as the PD. Consequently, each strategy landscape can be seen as a building block that constructs a strategy landscape of the game. Such a game landscape would allow conclusions as to what strategy configurations are adopted if all players and their interactions are taken into account. In other words, a game landscape may model the dynamics caused by the strategy updating processes discussed in Sec. 3.1.
4.4 Game landscapes
Reconsider the game with players, for which Fig. 1 depicts the player–wise strategy landscapes . At every point in time , the game can be seen as occupying one of its configurations. Put another way, the actual strategy vector specifies an actual configuration on the landscapes . For each player , its landscape gives its actual fitness . The strategy updating process means that one player provides its strategy for another player to receive. The receiving player changes its strategy. According to the landscape view this process corresponds with changing the actual configuration to a neighboring configuration . As the change of configuration affects all players (and consequently all player–wise strategy landscapes), it entails that all players may experience a change of fitness as well. No player can hold onto its configuration as long as the strategy updating process is underway unless one of the two absorbing configurations are reached, namely or .
In the following, the strategy updating processes birth–death (BD) and death–birth (DB) are discussed. For these processes transition probabilities can be derived, [Pattni et al., 2015], which can now be employed to define game landscapes. Therefore, it will be convenient to rewrite the landscape as its decomposition , , where each contains the fitness and preserves the neighborhood of configuration . Assume that the game is in configuration , which means that player is defecting, while the three other players are cooperating. According to the PD game, the fitness of is highest, the three other players have the same (albeit lower) fitness. To start with, let us consider a BD strategy updating, which selects source before target, [Pattni et al., 2015, Shakarian et al., 2012]. A player’s strategy is chosen at random with a probability proportional to fitness to be a source (hence birth). The birth probability of a configuration of player therefore scales to
where the are the decompositions of the landscape containing the fitness. The player with the highest fitness is most likely to be a source, which is presumably with strategy . Which one of the three players is the target to receive the strategy (hence death) is due to chance but influenced by possible restrictions regarding the replacement. Hence, the death probability of a player is
where the are the elements of the replacement matrix possibly restricting replacements of strategies as discussed in Sec. 3.1. Note that the death probability is independent of fitness and hence the same for all configurations of each player. A BD (and also a DB) updating does not envisage self–replacement and hence the replacement matrix must have diagonal elements . If, on the other hand, there are no replacement restrictions, then the death probability is invariant over players: for all players using a BD updating. Assume that all players can be a target and is chosen. Hence, the strategy configuration after the strategy updating is . The players and have leveled their fitnesses, while the fitness of both the other players is fallen even more.
Now consider a DB strategy updating, which selects target before source, [Pattni et al., 2015, Shakarian et al., 2012]. Here, a player’s strategy is chosen at random with a probability proportional to the inverse of its fitness to be a target (hence death). Therefore, the death probability of a configuration of player can be expressed as scaling to
Still assume that the game is in configuration and as the players , , and have the same (low) fitness values, one of them is most likely to be the target. Suppose is chosen. Which one of the three players provides its strategy as a source (hence birth), depends on chance and possible replacement restrictions. We get the birth probability
which is the same as the death probability (5) in BD, but the target and the source are switched in the elements of the replacement matrix. Note that only if the player is chosen, a change in configuration takes place, that is the strategy configuration after the strategy updating is . Put differently, the outcome of both a BD and a DB updating may be the same, but the probabilities to reach it may be different.
For a sufficiently large number of strategy updating events (and therefore changes of configuration), there may be some configurations that the game occupies more often than others. These may, for instance, be the absorbing configurations with a bit count and . Analyzing whether or not these absorbing configurations are reached and how long this takes, gives rise to fixation probabilities and fixation times, which will be discussed in Sec. 4.5. Before, however, we focus on the question of how frequent any configuration is over strategy updating events. The frequency of reaching a configuration scales to the probabilities of birth and death discussed so far. Hence, for a BD updating the game landscape
can be defined, while for a DB updating, we set
both with and being a sensitivity weight. Both game landscapes retain the configuration space and the neighborhood structure of the player–wise strategy landscapes , hence using them as building blocks. The fitness of each configuration summarizes via a Fermi function the probabilities to reach the configuration according to the birth and death events of the strategy updating process. The fitness of the game landscape therefore depends on the fitness of each player–wise landscape and possible replacement restrictions. Moreover, different updating processes cast different game landscapes and out of the same strategy landscapes of the players . Given that the are topological alike, and hence might be seen as possessing symmetry properties, different strategy updating rules break the symmetry of the player–wise strategy landscapes. At the same time, the BD and DB updating processes themselves possess symmetry properties via the birth and death probabilities (4) and (6). Consequently (and in the absence of replacement restrictions), the game landscapes and also retain symmetry properties. Replacement restrictions induced by different , however, yield another symmetry breaking. These symmetries (and broken symmetries) are reflected by the landscape properties discussed in the next section.
The discussion so far has been for a constant interaction network, that is for a specific matrix . As pointed out in Sec. 3.2, network updating can be described by a series of adjacency matrices . Hence, as the genetic description of the coevolutionary game comprises of the strategy vector and the interaction network, the strategy configurations made up by the space could be augmented by interaction configurations built by all possible networks of interaction. Consequently, the different interaction graphs enumerated approximately by Eq. (3) could be seen as configurations according to the landscape definitions discussed above. However, in view of the rather large number of possible graphs for given and (a rough estimate of Eq. (3) for yields ), an alternative model is to understand different interaction graphs as dynamic instances of the strategy landscape. Put differently, the dynamics of the strategy landscape is the results of its variability with respect to the interaction network. A consequence of such a modeling is that the timely order of varying interaction networks could be interpreted as temporal neighborhoods according to the neighborhood structure inherent in landscapes. With network updating expressed as dynamic instances of player–wise strategy landscapes, we get a dynamic landscape for describing a coevolutionary game. Apart from the strategy configuration with neighborhood and the integer time set , the quantity gives payoff–based fitness for each configuration, each player , and the –th interaction network. The matrix pair of subsequent adjacency matrices specifies how the fitness relates to , thus constructing the transition map .
Taking up the example of with the same values of the payoff matrix as in Sec. 4.3, but coplayers, we get and hence a game that is dynamic with respect to updating the interaction network. The three dynamic instances are shown in Fig. 2, where Fig. 2a is for the adjacency matrix , Fig. 2b is for , and Fig. 2c is for . It can be seen that the three different networks produce three different player–wise strategy landscapes for each player, which means that we indeed obtain dynamic changes over the three instances of –regular graphs on vertices.
Comparing these strategy landscapes with those for the complete interaction network (see Fig. 1) reveals differences. A first is that for each player, there are now two maxima and two minima. Each player retains a maximum (minimum) if this player itself defects (cooperates), while its two coplayers cooperate (defect). The two maxima (minima) come about as it makes no difference for the player’s payoff whether the fourth player (who is no coplayer as ) cooperates or defects. A second difference is that two neighboring configurations may build a block of equal fitness in connection with every configuration belonging to such a same–fitness block. Consequently, there is neutrality in these fitness landscapes. Moreover, these results suggest that the number of maxima and the degree of neutrality scales to the number of coplayers, which can be confirmed by numerical experiments for landscapes with more than players.
Within the given modeling framework of coevolutionary games, the timely order of the adjacency matrices is not associated with a particular updating process of the interaction network. The main reason is the general lack of established algebraic descriptions of network updating. The dynamic landscapes proposed may offer such an algebraic description as the transition map can be formulated uniquely for regular graphs, for instance for the transient between and of the example considered above as . For dynamic player–wise strategy landscapes , game landscapes for BD and DB updating can be defined according to the probabilities of birth/death and expressed as dynamic counterparts of Eqn. (8) and (9). As the fitness of each player now depends on the time variable specifying dynamic instances of the adjancency matrix, the death and birth probabilities , , , are also dynamic. Consequently, the static games landscapes (8) and (9) become dynamic game landscapes: and . These dynamic BD and DB landscapes are the main topic of the numerical experiments reported in Sec. 5.
4.5 Landscapes and fixation
The game specified by the payoff matrix (1) and the updating processes described in Sec. 3 instantiate evolutionary dynamics describable by the game landscapes (8) and (9) introduced above. As updating processes such as BD and DB depend on random processes, the resulting game dynamics can also be seen as a stochastic process. Consequently, stochastic properties such as fixation probability and fixation time have been suggested for evaluating and comparing the long–term outcome of evolutionary game dynamics, and studied widely in theory and numerical experiment, see, for instance, [Lieberman et al., 2005, Nowak, 2006, Pattni et al., 2015, Shakarian et al., 2012]. These fixation properties particularly account for whether or not the game dynamics settles on a steady state, and if so, how long this takes on average, and how frequent it is.
The fixation probability quantifies how likely it is that one of the two strategies that a player can use (cooperate or defect) spreads to the whole population of players, given that only one player started using this strategy. According to the landscape interpretation, this corresponds to reaching one of the two absorbing configurations with bit count and , given that the initial configuration had bit count and , respectively. For each absorbing configuration, there are different configurations that can possibly serve as an initial configuration. Hence, as tends to zeros for getting larger, initial configurations getting scarce in the overall topological structure of the game landscape for a sufficiently large number of players. The same also applies to absorbing configurations. This is in agreement with the observation that fixation probabilities generally decrease while increases (see also the results of numerical experiments in Sec. 5.4). As there are two absorbing configurations, two distinct fixation probabilities can be defined, one for complete cooperation and another for complete defection. The probability to reach the configuration where all player cooperate, , is denoted by , while the probability of all players defecting, , is named .
Fixation probabilities can be analytically calculated for Moran processes based on properties of Markov chains for well–mixed populations, [Nowak, 2006, Hindersin and Traulsen, 2015] and numerically for games on graphs, [Hindersin et al., 2016]. For games on graphs with replacement restrictions, estimates of the fixation probabilities using diffusion theory have been reported, [Ohtsuki et al., 2007]. For coevolutionary games considering dynamic networks of interaction of varying degree, numerical experiments can be carried out. Following previous experimental works, the fixation probabilities are approximated by the relative frequency of fixation. The fixation time quantifies how many changes in configuration it takes on average to reach an absorbing configuration . This corresponds with the average amount of time needed to achieve fixation. The notion of fixation time can be refined by distinguishing which one of the two absorbing configuration is reached, which gives rise to conditional fixation times, [Traulsen and Hauert, 2009]. The fixation times for the cooperative and defective absorbing configurations are denoted by and , respectively.
As fixation probability and fixation time are the most important quantities in stochastic game dynamics, these quantities are discussed next with respect to the landscape interpretation proposed in the previous sections. The fitness of the landscapes (8) and (9) derives from payoffs of each player and summarizes the probabilities that a particular configuration is occupied by the game. Hence, possible differences in fitness across neighborhoods generate topological features of the landscape. These topological features, in turn, create evolutionary paths on the landscape, which any evolutionary dynamics has to observe. However, the evolutionary dynamics is governed by a move bias towards higher fitness, which is not a move imperative. In other words, the landscape view implies that there are probabilities that the maxima are reached, but no certainties. Moreover, these probabilities depend on what exactly the topological features of the landscape are, for instance, on the number of maxima, their distribution and their accessibility. For evaluating the effect of landscape features, just focusing on the maxima is not sufficient. Therefore, different types of landscape measures have been proposed which aim at reflecting, in a more general sense, the impact that landscape features have on evolutionary dynamics, see also the discussion in Sec. 5.1.
Fixation occurs if a succession of changes in configuration leads from prescribed initial configurations to the absorbing configurations. Fixation probabilities , require evolutionary paths connecting initial configurations with respective absorbing configurations. The values of , scale to how easy or how difficult it is that these evolutionary paths can be accessed and maintained. The fixation time, on the other hand, varies with the length of the evolutionary path. Consequently, by analyzing the topological structure of the game landscape, it may be feasible to infer fixation properties. This kind of analysis, however, is impeded by the fact that absorbing configurations in game landscapes are, topologically interpreted, non–passable points in the landscape. However, non–passable points are not a standard concept of landscapes. Perhaps most similar are steady states of a landscape, but there is the difference that the evolutionary dynamics can, under certain conditions, leave a steady state and there is the even more fundamental difference that steady states are by definition maxima of the landscape. Absorbing configurations may or may not be maxima of the game landscape. In the same way, the initial configurations marking the origin of the evolutionary path may or may not be minima of the landscape. The numerical experiments discussed in Sec. 5.4 confirm such a characteristics for the game landscapes (8) and (9).
This line of reasoning suggests that a landscape analysis should take into account that fixation properties are likely to be related to landscapes via the local (and possibly also the global) topological features of absorbing and initial configurations. In analogy to the landscape structure, which describes globally the topological features of the entire landscape, these topological features and their interdependences with fixation we may call absorption structure. The numerical experiments reported next section address not only topological features of the landscapes, but also the absorption structure, where the focus is on the local structures.
5 Numerical experiments
5.1 Landscape measures
Game landscapes can only be visualized as two–dimensional projections up to players. For analyzing landscapes with more players, we need to resort to landscape measures. A first measure we look at is modality expressed by the number of local maxima . Local maxima are potential steady states on the evolutionary path. Hence, the number of local maxima relates to the variety of possible evolutionary paths and consequently to the complexity of the evolutionary dynamics displayed. If there is just one (smoothly accessible) maximum, then all evolutionary paths converge to it and the evolutionary dynamics displayed is rather simple. If, on the other hand, there is a large number of maxima, then the possible evolutionary paths may differ from each other massively resulting in more complex evolutionary dynamics. For a landscape a configuration is a local maximum , if , the fitness of this strategy configuration satisfies . Moreover, if this condition holds , then is a global maximum.
For evaluating the local absorption structure, we need to consider three further local topological features: minimum, neutrality, and saddles. A local minimum has at least one neighbor that has a smaller fitness and no neighbors that have larger fitness than itself. A neutral configuration has only neighbors with the same fitness, which means that a landscape area containing and its neighbors is flat. Lastly, a saddle has some neighbors that are larger and some other neighbors that are smaller. In numerical experiments, the number of local maxima can be computed for the game landscapes (8) and (9). The same applies to whether the absorbing configuration and its initial configurations are maxima, minima, neutral or saddles. Consequently, for the dynamic instances of these landscapes, a time series containing the numbers of local maxima is obtained. The same applies to all other measures of dynamic landscapes.
There are two immediate problems with analyzing landscapes by modality expressed by the number of local maxima . First, on a practical level, we find that can only be calculated by enumeration, which entails the proverbial curse of dimensionality. Second, on a conceptual level, there is the fact that the pure number of local maxima is a decisive (and arguable the most important) factor defining evolutionary paths, but the distribution of the maxima and their accessibility is also profoundly influential. To overcome these issues, other types of measures have been proposed for quantifying smoothness, ruggedness, or neutrality of the landscape. Two of them are studied here, correlation length and information content .
The correlation length evaluates across the landscape how strongly the fitness of any configuration relates to the fitness of neighboring configurations and hence is a measure of the landscape’s ruggedness, [Stadler, 1996, Richter and Engelbrecht, 2014]. For calculating the correlation length , a random walk on the landscape of length is used. The fitness values for each step on the walk are recorded by the sequence
and thus a series of neighboring fitness relations is obtained. Assuming that the landscape is isotropic, these neighboring fitness relations account for general changes in fitness across the landscape. Hence, the autocorrelation of sequence (10) with time lag defines the landscape’s random walk correlation function , where , and . The function measures the correlation between different regions of the fitness landscape and expresses a measure of how smooth or rugged the landscape is. The correlation length
The information content , [Muñoz et al., 2015, Vassilev et al., 2000], is an entropic landscape measure, which also uses the fitness sequence (10) generated by a random walk on the landscape. It can be interpreted as a measure of the amount of information required to reconstruct the landscape structure. From the time series (10), differences in fitness between two consecutive walking steps are coded by symbols , , taken from the set . These symbols are obtained by
for a fixed , where is the maximum difference between two fitness values. The obtained symbols are concatenated to a string
The parameter defines the sensitivity by which the string accounts for differences in the fitness values. For example, if , the string contains the symbol zero only for the random walk reaching a strictly flat area. Hence, discriminates very sensitively between increasing and decreasing fitness values. By contrast, for , the string only contains the symbol zero, which makes evaluating the structure of the landscape pointless. A fixed value of with defines a level of detail with respect to the information gained about the landscape structure.
For defining the information content of the landscape, the distribution of subblocks of length two, , , within the string (13) is analyzed. These subblocks stand for local patterns in the landscape. The probabilities of the occurrence of the pattern with and are denoted by . For numerical calculation, these probabilities are approximated by the relative frequency of the patterns within the string (13). As the set consists of three elements, we find six different kinds of subblock with within the string . From their probabilities and a given sensitivity level the entropic measure
is calculated, which is called information content of the fitness landscape, [Muñoz et al., 2015, Vassilev et al., 2000]. Note that by taking the logarithm in Eq. (14) with the base , the information content is scaled to the interval .
5.2 Graph–theoretical properties of interaction networks
Networks of interaction may be described by instances of a random –regular graph, as set out in Sec. 3.2. Based on this description, varying interaction networks have been interpreted in this paper as to cast dynamic instances of a landscape characterizing the coevolutionary games. Instances of interaction networks specify who–plays–whom in the game, which means that even if for each player the number of coplayers is constant, who in fact the coplayers are is not. As different coplayers may imply different strategies and hence different allocations of payoff, different networks of interaction may result in topologically different game landscapes. Put another way, if properties of game landscapes vary over dynamic instances, the variations should be reflected by properties of interaction networks, that is graph–theoretical quantifiers of –regular graphs. Spectral graph theory defines several such quantifiers, which take advantage of connections between the algebraic description of a graph and its structural properties; see for instance [Biggs, 1994, Brouwer and Haemers, 2012, Cvetkovic et al., 2009, Li et al., 2012, Spielman, 2009], upon which the remainder of this section about quantification of graph–theoretical properties of interaction networks relies. The main propose of this quantification is to map structural differences of the interaction graph to different values of graph–spectral invariants, which, in turn, are interpretable as (graph–theoretical) network measures. For definitions of these invariants, also see [Biggs, 1994, Brouwer and Haemers, 2012, Cvetkovic et al., 2009, Li et al., 2012, Spielman, 2009].
The quantities considered are based on algebraic properties of the adjacency matrix , or the Laplacian matrix , which is derived from to include the degree explicitly. For the matrices and , the spectra of eigenvalues, and , are starting points for further consideration. For connected –regular graphs, we find for spectra of the adjacency matrix that all eigenvalues are real and , while eigenvalues of the Laplacian are also all real, and non–negative as well as sortable according to .
A first quantity is the (normalized) energy of a graph:
which can be interpreted as the spectral distance between a given graph and an empty graph, and can hence be seen as scaling to the degree of difference between graphs. A second graph–theoretical network measure based on the eigenvalues is the independence number
which is an approximation of the size of the largest independent set of vertices in a graph. An independent set is a set of vertices in a graph such that no two vertices of the set are connected by a edge.
A network measure based on the Laplacian derives from the smallest eigenvalue of larger than zeros, , is termed (normalized) algebraic connectivity
and scales to how well a graph is connected. Connectivity denotes the structural property of a graph that removal of vertices or edges disconnects the graph. The Laplacian eigenvalue if the graph is not connected, and if the graph is complete (that is fully connected). Larger values of indicate graphs with a rounder shape, and high connectivity and girth, while for smaller values of the graph is more path–like with low connectivity and girth. Also calculated from the Laplacian is the expander index
which is a measure for the –regular graph possessing expander properties. The expander index has small values for all eigenvalues being close to , and larger values for the opposite. Expander graphs are marked by all small sets of vertices usually having a larger number of neighbors. Thus, they can be seen as their neighborhood expanding.
As far as possible and needed, the graph–theoretical quantifiers are normalized with respect to the order of the graph. Hence, they can be compared over a varying number of vertices and hence players. In Sec. 5.4 results are given that analyze the correlation between the network measures (15)–(18) and the landscape measures (11) and (14).
5.3 Experimental setup
The numerical experiments with the dynamic fitness landscapes of coevolutionary games consider a PD game and a SD game with and , respectively, which is a parametrization as suggested by Axelrod’s seminal work, [Axelrod, 1980]. The dynamics of the landscape is addressed by examining the effect of varying networks of interaction. Algorithms are employed that numerically generate adjacency matrices specifying random regular graphs with given order and degree, [Bayati et al., 2010, Blitzstein and Diaconis, 2011, Kim and Vu, 2003]. It is checked to see if the graphs are connected. If a graph fails the check, there are isolated vertices that may bias controlling the interaction network via the graph’s degree and hence such graphs are discarded. For the experiments, different sets of graphs with prescribed and are generated and used. The absorption structure was analyzed with a set of graphs. Some experiments studying landscape measures and fixation properties were done with a set of graphs. These experiments have shown that for a considerable number of different networks, rather similar results are obtained. For this reason and also to facilitate the numerical experiments, unless stated differently the results presented in the figures are based on a set of different interaction networks. In all cases for , the complete set of possible networks of interaction is taken. The experiments are conducted for even to guarantee the existence of –regular graphs for all .
Different replacement structures are analyzed. The experimental setup follows previous works, [Ohtsuki et al., 2007], and defines the replacement matrix as random regular graph with given degree and guaranteed connectivity. Additionally, the elements are filled with realizations of a random variable uniformly distributed on the interval . The degree of is set to match the degree of . All these experiments are carried out for BD and DB landscapes. Other updating schemes such as PC or IM can be treated likewise. For these processes transition probabilities are known, [Pattni et al., 2015], and hence landscapes similarly to (8) and (9) can be computed. With the conventional PC–based computational resources that were available, it was possible to experiment within a reasonable time–frame with up to players. All experiments employ a linear relationship between payoff and fitness with . The game landscapes are computed with a sensitivity weight . For calculating the correlation length and the information content , a random walk of length was used, and the results are averaged over independent walks. Numerical experiments have shown that the results obtained are statistically equivalent over different initial configurations that the walks starts with. Hence, it appears reasonable to assume for the tested cases that the game landscapes are isotropic. The information content (14) is computed with a sensitivity level that scales to the number of players by .
The numerical experiments calculating fixation properties are based on repetitions. This is a rather small amount compared to other recent experimental works, e.g. [Hindersin and Traulsen, 2015, Zukewich et al., 2013]. Some auxiliary experiments with a larger amount of repetition, however, have shown that the values of the fixation properties analyzed converge well so that the numerical setup used appears sufficient for up to players.
5.4 Experimental results
Fig. 3 shows the landscape measures number of local maxima , correlations length and information content over and . The red lines indicate a BD updating, the green lines a DB updating. In addition to the quantities averaged over the up to different interaction networks tested (horizontal lines), the vertical spikes indicate the range between the least and the largest value of , or over these networks described by the adjacency matrices . This presentation and color code is kept for all landscape measures and fixation properties.
For the SD game, the number of local maxima are given as semi–logarithmic plots, see Fig. 3b. The results show that for the PD landscape (Fig. 3a) there is only one maximum for all tested and , and both BD and DB updating. It hence is unimodal, while for the SD game the number of maxima increases with and is hence multimodal. Moreover, for the SD game, we find for DB updating that decreases for a given and getting larger. Also, for the SD game accounting for does not reflect the symmetry between BD and DB landscapes, which is not the case for the PD game. Another observation is that for the SD landscape the number of local maxima sometimes shows vertical spikes indicating the amount to be not constant for a given and and varying networks. In other words, there is a certain variety in the number of local maxima over instances of interaction networks expressed as –regular graphs. Some interaction graphs yield game landscapes with a larger (or smaller) number of local maxima than average. Further note that for getting larger. All these results support previous findings about evolutionary games, for instance that PD games and BD updating do not provide an advantage for cooperators, [Ohtsuki et al., 2007, Zukewich et al., 2013]. Thus, for PD the small number of maxima of the player–wise landscapes (compare to Fig. 1) corresponds with the small number of maxima in the game landscape. By contrast, for the SD game and DB updating not only configurations where the defecting player earns the largest payoff are maxima of the game landscapes. Consequently, the number is larger. For the landscape measures and in Fig. 3c–f, we find almost identical results for BD and DB landscapes, yet the different games can be distinguished, albeit not as clearly as for . Hence, correlation length and information content largely reflect the symmetry of the game landscapes. It can also be seen that the variety over different networks of interaction is slightly stronger for than for . Also, it is interesting to see that the landscape measures are similar for PD and SD, while modality is different. A possible explanation is that for the PD game the landscape is, apart from being unimodal, structurally similar to holey landscapes, [Gavrilets, 2004] where neutral ridges are mixed with holes of lower fitness.
We next analyze the effect of replacement restrictions imposed by the replacement graph not being fully connected with evenly weighted edges, and focus on the differences between replacement restriction being set or not, see Fig. 4. A main observation is that replacement restrictions modify the game landscapes, which is also shown by the landscape measures. For instance, the number of local maxima for the PD game and DB updating is no longer strictly unimodal (compare Figs. 3a and 4a). Interestingly, for BD updating, even replacement restrictions do not alter unimodality. These is still just one maximum for all tested and , see the red lines in Fig. 4a. For the SD game, the inverse proportionality between and ceases, and generally the number of local maxima does vary more strongly for different networks of interaction. These characteristics can also be seen for the landscape measures correlation length (see Fig. 4c,d) and information content (see Fig. 4e,f). Here, the main difference is that the measures are no longer the same (or almost the same) for BD and DB updating. This reflects the broken symmetry that replacement imposes on BD and DB game landscapes. Generally, replacement restrictions imply landscapes that vary more substantially over different networks of interaction. Furthermore, as there is an inverse relationship between ruggedness and correlation length , it can be noted that ruggedness decreases as the number of player gets larger. This effect is amplified by replacement restrictions.
In Fig. 5 fixation properties of the cooperative absorbing configuration with are given over and . The fixation probability is zero for the PD game and BD updating, which corresponds to previous results showing that cooperation is never favored or beneficial under such an updating, [Ohtsuki et al., 2007, Zukewich et al., 2013]. Apart from this result, falls with the number of players getting larger, but for a given , the fixation probability is the same for a varying number of coplayers . These results are in line with finding that regular evolutionary graphs are generally isothermal, [Lieberman et al., 2005]. Moreover, except for very small numbers of players ( and partly ), the fixation probability for the well–mixed case () is the same as for a smaller number of coplayers. This, however, is only the case for averages over interaction networks. There are for a constant and interaction networks with fixation probabilities larger or smaller than average indicated by the vertical spikes (for there cannot be a spike as only one instance of exists). Hence, these results suggest that the graph structure of the interaction network does matter for . Moreover, which causes the largest or smallest varies over and . Regarding the fixation times , similar observations can be made. Note that for the PD game no fixation time for BD updating are given as the fixation probability is zero. For the SD game the fixation times for BD updating are much larger than for DB. Fixation properties of the defective absorbing configuration with are given in Fig. 6. All game settings produce fixation probabilities . Apart from this, the results are similar to the general trends for the cooperative absorption, namely fixation probability differs for BD and DB updating, falls with an increasing number of players and is isothermal for given and a varying number of coplayers . The fixation times in Fig. 6c,d also show some similarity, but also differences. The main observation is that for the SD game the maximal fixation times are not substantially larger than for the PD game.
We next consider the local absorption structure of the game landscapes, which is based on up to different interaction matrices . The results for the PD and SD game with BD and DB landscapes are given in Tab. 1. The final absorption structure (F) indicates the local topological features of the absorbing configurations , while the initial absorption structure (I) denotes the features of the initial configurations from where potential paths to the absorbing configuration may start. There are four different local topological features (maximum, minimum, neutral, and saddle), which are given for both the cooperative absorbing configuration with and the defective absorbing configuration with .
Cooperative absorbing Defective absorbing
|BD||F: mn||F: nt||BD||F: mx||F: mx|
|I: sd||I: sd||I: sd||I: mn|
|DB||F: mx||F: mx,mn,nt||DB||F: mn||F: mn|
|I: sd||I: sd||I: sd||I: mx, sd|
Not global for
The results in Tab. 1 show some general properties of the local absorption structure for the game settings considered, which in turn can be interpreted as specific for the game landscape proposed by Eqn. (8) and (9). A first general property is that for both games and both absorbing configurations, the absorption structure of BD updating generally differs from DB updating. This may answer the question of why game landscapes that are symmetric with respect to BD and DB for no replacement restrictions yield fixation properties that do differ from BD to DB. A possible explanation is that while the game landscapes are topologically the same as shown by the landscape measures and (see Figs. 3c,d and 3e,f), their absorption structure is not. This suggests the absorption structure to be a determining factor in the relationships between game landscapes and fixation. A second general property is that the absorption structure is inverse complementary for BD and DB, the meaning of which is that if there is a maximum for BD, then DB has a minimum, and vice versa, while a saddle remains a saddle. This follows from the symmetry properties via the birth and death probabilities (4) and (6) as discussed in Sec. 4.4. There is an exception with the cooperative absorption of the SD game discussed later.
The local absorption structure also shows differences between the social dilemma games. A property specific for the PD game is that there is no variety over the number of players and coplayers. In other words, within a given game setting changing the number of players and coplayers does not alter the absorption structure. This also applies to a general absence of variety over different interaction matrices . At least for the interaction matrices tested, the absorption structure is invariant over interaction networks. Also, the maxima/minima of the absorption structure are global. For the SD game the results are slightly different. For this game setting, we find that the cooperative final absorbing configuration is neutral for BD updating, and a combination of maximal, minimal and neutral for DB updating. Further analysis shows that these topological features vary over , and . For most and the absorbing configuration is neutral, while for a few and it is either a maximum or a minimum. In these cases, the topological features are fixed for varying interaction networks. In addition, there are and for which varying interaction networks give either a mixture of maxima and neutral, or a mixture of minima and neutral. For the defective absorbing structure, the initial configurations vary over , where for most and we have saddles, while for some other and there is a mix of saddles and maxima. Also, for the cooperative absorption structure and DB, the maxima/minima are local, not global. Numerical evaluation affirms that such a variety of the topological features of the absorption structure over interaction networks can be observed for other game settings as well, particularly those on the line . The SD game with is exactly on this line for the parametrization used (, , ).
Based on the absorption structure of the game landscapes, the next set of numerical results deals with correlations between landscape measures and fixation properties. As already discussed in Sec. 4.5, relationships between topological structures of the game landscapes and fixation events are likely to be shaped and typified by the absorption structure. The results for the correlations between landscape measures and fixation properties are shown in Fig. 7 for the cooperative absorbing configuration and Fig. 8 for the defective absorbing configuration. The results are for all game settings considered with red markers indicating a PD game with BD updating, green PD game and DB, blue is SD game with BD and yellow SD and DB. The lines connecting the markers are depicted to ease following trends. The Pearson product–moment correlation coefficient is calculated to aggregate over interaction networks and the number of players for each number of coplayers . The advantages of such a mode of calculation are that the database for analyzing correlations becomes sufficiently large (it comprised of data for the instance of interaction matrices times the number of players ) and that trends over a varying number of players are captured. However, there should be an awareness that the calculation implicitly assumes that fixation properties and landscape measures scale on in ways compatible with the dependence on .
Comparing the results in Fig. 7 tells us that the correlations between and either or are volatile and hardly evaluable, while for there are clear trends. The same can be said about the defective fixation, see Fig. 8 for and . Further analysis (not shown in a figure) confirms this to remain if the calculation is done for each and . Hence, it appears to be justified to conclude that the information content correlates more clearly to fixation properties than the correlation length . Another results (also not shown in a figure for brevity reasons) shows that the number of local maxima also correlates poorly to fixation properties. Apart from the fact that there is no correlation for the PD game, BD updating and the fixation of cooperation as , there are further results to note. For , the correlations are always zero. This is why: with the experimental setup employed (see Sec. 5.3) only for , there can be , with the additionally meaning that such a game is well–mixed with just one instance of the interaction matrix . As correlation cannot be based on a single data pair, the correlation must be zero. Furthermore, it can be seen that there is a negative correlation between and (and ), while the correlation between and (and ) is positive. This appears reasonable as the fixation probabilities fall with increasing number of players , yet the fixation times grow. Also, it can be observed that generally the correlations are strongest for small number of players and weaken before they reach zero for .
Regarding the shaping and typifying effect of absorption, the following can be observed comparing the local absorption structure in Tab. 1 with the correlations in Figs. 7 and 8. From a topological point of view the correlation should be particularly strong if the absorbing configuration is a maximum and the initial configurations are all minima. For absorbing and initial configurations being minima and maxima, the opposite should apply. Only for one example the final absorbing configuration is a maximum and the initial configurations are minima for all , and : the defective absorption of the SD game with BD. For this case, the correlation between and is indeed slightly stronger than for the other cases. However, for the correlation between and the opposite is true. In general, it can be noted that the correlations for all settings (except PD–BD for the cooperative absorption) are clearly visible and rather similar. It can be concluded that while there are some hints in the local absorption structure to explain the correlations between landscape measures and fixation properties, the explanatory framework should not be overstretched. Analogous to a landscape analysis that only focuses on selected points in the landscape (for instance the maxima/minima), the local absorption structure only captures a subset of the topological structures that shape coevolutionary game dynamics. Therefore, extensions toward a global absorption structure seem desirable.
A last set of experiments reports correlations between landscape measures and network measures of the interaction matrices . Figs. 9 and 10 give the correlations between information content and either graph energy, Eq. (15), independence number, Eq. (16), algebraic connectivity, Eq. (17), or expander index, Eq. (18). Again, the Pearson coefficient is used aggregating over interaction networks and the number of players for each number of coplayers . The same color code for the game settings as for the correlations with fixation properties is used. It can be seen that the information content correlates well to all networks measures, see Fig. 9. It is conspicuous that the results are indistinguishable for BD and DB landscapes, which fully reflects the symmetry properties of these landscapes. The symmetry is broken by replacements restrictions. The correlations between the network measures and reported in Fig. 10 confirm this as there are differences between BD and DB for replacement restrictions. The correlations, however, are less smooth over as compared to the results in Fig. 9, which is most likely due to the additional stochastic nature of replacement restrictions. Lastly, two more observations can be noted. A first is that the correlations between and the networks measures are negative for graphs energy, independence number and expander index, while they are positive for algebraic connectivity. The main reason is that amongst the network measures studied only algebraic connectivity increases continuously for getting larger in both mean and variance over instances of the interaction matrices . A second is that for the graph energy and the expander index the correlations are weak for both small numbers of coplayers () before they get stronger to weaken again for larger number of players (). For the other two network measures the weakening is only for , which is similar to the correlations with fixation properties.
The experimental results given above set out relationships between landscape measures and both fixation properties and network properties, and argue that dynamic landscape models of coevolutionary games are viable. In this section, features and implications of such a modeling approach are discussed, the experimental findings of Sec. 5.4 are put into context, and some concluding observations are offered.
An essential part of the numerical experiments is the study of correlations between landscape measures and both fixation properties and quantifiers of the networks of interaction. A main results is that information content scales well to fixation properties such as fixation probability and fixation time, see Figs. 7 and 8. Particularly ruggedness as measured by the correlation length relates less clearly and much weaker to fixation properties than information content, which is understood to account not specifically for ruggedness, but more for the interplay between smooth, rugged and flat landscape areas. Hence, a conclusion may be that also for game landscapes ruggedness alone is not a good predictor for evolutionary dynamics, as also reported for other types of fitness landscapes, [Malan and Engelbrecht, 2013]. There are additional entropic landscape measures based on the information content, for instance partial information content, information stability or density–basin information, [Muñoz et al., 2015, Vassilev et al., 2000]. Thus, it might be interesting to study whether these measures also scale well for game landscapes and may offer further insight into game dynamics. Regarding the correlations between landscape measures and quantifiers of interaction networks, the results are more consistent, see Figs. 9 and 10. There are clear correlations for all four of the network measures considered, with algebraic connectivity and independence number scaling slightly better than graph energy and expander index. However, it should be noted that the correlations established between the landscape measures and network measures are based on the Laplacian or adjacency spectra of the adjacency matrix . As these spectra do not uniquely determine the interaction graph, there might be correlations between the graph structure of the interaction network and the game landscape that are not captured. The discussion might be extendable by considering alternatives, for instance generalized graph distance measures as reported by [Gu et al., 2016].
The experiments studying the effect of different networks of interaction with given and on fixation properties and landscape measures only report mean, maximum, and minimum values and their interdependencies. Further statistical analysis, for instance considering variances or higher–order moments, is deliberately omitted. The main argument is that we should beware of drawing conclusions based on such a statistical analysis as it is not clear if it really produces generalizable results. To illustrate the point, let us consider coplayers, for which the number of different graphs can be enumerated exactly by Eq. (2). The experiments presented are for up to players. Accordingly, at the upper limit of the experimental setup, we find . There is no alternative but to conclude that any number of numerically testable networks of interaction represents just a tiny subset of all interaction networks. At the same time it is far from being clear how well the finite number of graphs generated by the numerical procedures represents all possible different graphs for given and . Thus, it might be possible that some trends are biased by the algorithmic process of numerically generating interaction networks . Further work is needed to clarify these interdependencies. Such a work should go along with categorizing interaction matrices into clusters. These clusters could quantify, on the one hand, similar fixation properties of the coevolutionary games, and on the other, similar spectral properties or similar generalized graph distance measures of the interaction matrices as mentioned above. Hence, it might be possible to study the effect of network properties of the interaction matrix on coevolutionary game dynamics for a larger number of networks.
The experimental results showing landscape measures and their correlations to fixation properties as well as to graph–theoretical quantities of the interaction networks are for the specific algebraic description of the game landscapes (8) and (9). The algebraic form of how to cumulate death and birth probabilities from the player–wise strategy landscapes is definatory and was employed as it fitted well to fixation properties and previous results known about social dilemma game dynamics. It is an open question whether an alternative algebraic form of (8) and (9) can achieve similar or even better results. Similarly, the results obtained here are specific for the linear relation . Hence, it might be interesting to analyze how different payoff–to–fitness relations modify the results, for instance the exponential relations or , as suggested by [Allen and Nowak, 2014, Shakarian et al., 2012]. This may go along with experimental studies of different levels of the intensity of selection , which is also opened up by the game landscape approach proposed in this paper. In the case of weak selection, that is for , the player–wise strategy landscapes lose their distinct topological features, which yields (in the absence of replacement restrictions) a game landscape that is neutral. Consequently, the game dynamics on this landscape would be random drift. For larger or large values of the topological features of the player–wise strategy landscapes become more prominent, and the game landscape is more rugged. From this line of argument it can be conjectured that there is a direct relation between the intensity of selection and the ruggedness of the game landscape, which might be verifiable by future studies.
The results of dynamic game landscapes presented are for up to players. Naturally, it would be desirable to extend the experimental setup to a larger population size. However, if the trends identified in these results remain valid for a larger number of players needs to be studied in further work, as the maximal number of players used was the upper limit that could be realized within a reasonable time–frame and the computational resources available in this study. A general feature surely is that the number of configurations to be analyzed in the landscapes increases exponentially with the number of players, hence setting bounds as to how far such experiments might be extendable. Therefore, with the computational resources currently available the modeling framework is likely to be confined to a moderate number of players. However, for an increased number of players there is the framework of replicator dynamics which sufficiently describes game dynamics for populations becoming large, [Traulsen et al., 2005].
Some primary experiments have shown that for replacement restrictions, the correlation between landscape measures on the one hand, and fixations and network properties on the other, cease. Apparently, the replacement restrictions seriously modify the structure of the game landscapes. It is another open question if these relationships can be reestablished by taking into account properties of the replacement process, for instance the absorption structure of the restricted network. This may be extended by foregoing the setting that the degree of the replacement matrix matches the degree of the adjacency matrix .
6 Summary and conclusions
Coevolutionary games cast players that update their strategies as well as their networks of interaction. In this study, a reinterpretation of coevolutionary games as dynamic fitness landscapes is proposed. The dynamic landscapes are based on three major components: (i) a description of strategy updating as a Moran process with definable probabilities of strategy transitions, (ii) a formulation of updating the interaction network as instances of random regular graphs, and (iii) a linear relation between payoff and fitness. Using these components, payoff–related fitness landscapes can be defined for each player. It is further shown that coevolutionary game dynamics can be expressed by a game landscape derived from these player–wise landscapes by including the strategy updating process. Moreover, different strategy updating processes, such as death–birth (DB) or birth–death (BD) produce different game landscapes, which can be seen as strategy updating breaking the symmetry of the play–wise landscapes. In numerical experiments it has been demonstrated that landscape measures such as modality, ruggedness and information content allow to differentiate between different game landscapes. Fixation probabilities and fixation times have been calculated as well as network measures characterizing the networks of interaction of the coevolutionary games. By correlation analysis it has been shown how the landscape measures relate to both fixation properties and network measures.
The approach presented is a technique for analyzing coevolutionary games by landscapes. Moreover, the approach is not restricted to Moran processes as long as strategy transition probabilities can be derived, at least approximately. Finally, network updating is currently modeled as a given sequence of random regular graphs, but should be understood as a transition process, for instance by using reproducing graphs, [Southwell and Cannings, 2010] as a tool to refine the description of transitions between adjacency matrices.
Different settings of the game represented by the numeric values of the payoff matrix and different rules of the strategy updating result into a large variety of coevolutionary game dynamics. A considerable number of works have analyzed and discussed this game dynamics with respect to fixation properties such as fixation probability and fixation time from both a theoretical as well as an experimental point of view, [Allen and Nowak, 2014, Lieberman et al., 2005, Nowak, 2006, Pattni et al., 2015, Shakarian et al., 2012]. The results reported here contribute to this discussion by offering a fitness landscape view as an alternative explanatory framework. In other words, by the approach presented coevolutionary games may become amenable to be analyzed by dynamic landscapes.
- [Allen and Nowak, 2014] Allen, B., Nowak, M. A., 2014. Games on graphs. EMS Surv. Math. Sci. 1, 113–151.
- [Axelrod, 1980] Axelrod, R., 1980. Effective choice in the prisoner’s dilemma. Jour. Conflict Resolut. 24, 3–25.
- [Bayati et al., 2010] Bayati, M., Kim, J. H., Saberi, A., 2010. A sequential algorithm for generating random graphs. Algorithmica 58, 860–910.
- [Biggs, 1994] Biggs, N., 1994. Algebraic Graph Theory, 2nd edition. Cambridge University Press, Cambridge.
- [Blitzstein and Diaconis, 2011] Blitzstein, J., Diaconis, P., 2011. A sequential importance sampling algorithm for generating random graphs with prescribed degrees. Internet Mathematics 6, 489–522.
- [Bollobás, 2001] Bollobás, B., 2001. Random Graphs. 2nd edition. Cambridge University Press, Cambridge.
- [Brede, 2011] Brede, M., 2011. The evolution of cooperation on correlated payoff landscapes. Artificial Life 17, 365–373.
- [Brouwer and Haemers, 2012] Brouwer, A. E., Haemers, W. H., 2012. Spectra of Graphs. Springer-Verlag, New York.
- [Cvetkovic et al., 2009] Cvetkovic, D., Rowlinson, P., Simic, S., 2009. An Introduction to the Theory of Graph Spectra. Cambridge University Press, Cambridge.
- [Foster et al., 2013] Foster D. V., Rorick M. M., Gesell T., Feeney L. M., Foster J. G., 2013. Dynamic landscapes: A model of context and contingency in evolution. J. Theor. Biol. 334, 162–172.
- [Gavrilets, 2004] Gavrilets, S., 2004. Fitness Landscapes and the Origin of Species. Princeton University Press, Princeton, NJ.
- [Greenwood and Ashlock, 2012] Greenwood, G. W., Ashlock, D, 2012. Evolutionary games and the study of cooperation: Why has so little progress been made? In: Abbass, H., Essam, D., Sarker, R. (Eds.), Proc. IEEE Congress on Evolutionary Computation, IEEE CEC 2012, IEEE Press, Piscataway, NJ, pp. 1–8.
- [Greenwood and Avery, 2014] Greenwood, G. W., Avery, P., 2014. Does the Moran process hinder our understanding of cooperation in human populations? In: Rudolph, G., Preuss, M. (Eds.), Proc. IEEE Conference on Computational Intelligence and Games, IEEE CIG 2014, IEEE Press, Piscataway, NJ, pp. 1–6.
- [Gu et al., 2016] Gu, J., Jost, J., Liu, S., Stadler, P. F., 2016. Spectral classes of regular, random, and empirical graphs. Linear Algebra Appl. 489, 30–49.
- [Hindersin and Traulsen, 2015] Hindersin, L., Traulsen, A., 2015. Most undirected random graphs are amplifiers of selection for birth-death dynamics, but suppressors of selection for death-birth dynamics. PLoS Comput Biol 11(11): e1004437. doi:10.1371/journal.pcbi.1004437.
- [Hindersin et al., 2016] Hindersin, L., Möller, M., Traulsen, A., Bauer, B., 2016. Exact numerical calculation of fixation probability and time on graphs. BioSystems 150, 87–91.
- [Kauffman and Johnsen, 1991] Kauffman, S.A., Johnsen, S., 1991. Coevolution to the edge of chaos: Coupled fitness landscapes, poised states, and coevolutionary avalanches. J. Theor. Biol. 149, 467–505.
- [Kim and Vu, 2003] Kim, J. H., Vu, V. H., 2003. Generating random regular graphs. In: Larmore, L. L., Goemans, M. X. (Eds.), Proc. ACM Symposium on Theory of Computing, STOC 2003, ACM, New York, pp. 213–222.
- [Li et al., 2012] Li, X., Shi, Y., Gutman, I., 2012. Graph Energy. Springer Science & Business Media, New York.
- [Lieberman et al., 2005] Lieberman, E., Hauert, C., Nowak, M. A., 2005. Evolutionary dynamics on graphs. Nature 433, 312–316.
- [Malan and Engelbrecht, 2013] Malan, K. M., Engelbrecht, A. P., 2013. A survey of techniques for characterising fitness landscapes and some possible ways forward. Information Sciences 241, 148–163.
- [Maynard Smith, 1991] Maynard Smith, J., 1991. Evolution and the Theory of Games. Cambridge University Press, Cambridge.
- [Muñoz et al., 2015] Muñoz, M. A., Kirley, M., Halgamuge, S. K., 2015. Exploratory landscape analysis of continuous space optimization problems using information content. IEEE Trans. Evolut. Comp. 19, 74–87.
- [Nowak, 2006] Nowak, M. A., 2006. Evolutionary Dynamics: Exploring the Equations of Life. Harvard University Press, Cambridge, MA.
- [Nowak and May, 1993] Nowak, M. A., May, R. M., 1993. The spatial dilemmas of evolution. Int. J. Bifurc. Chaos 3, 35–78, 1993.
- [Nowak and Sigmund, 2004] Nowak, M. A., Sigmund, K., 2004. Evolutionary dynamics of biological games. Nature 303, 793–799.
- [Ohtsuki et al., 2007] Ohtsuki H., Pacheco, J. M., Nowak, M. A., 2007. Evolutionary graph theory: Breaking the symmetry between interaction and replacement. J. Theor. Biol. 246, 681–694.
- [Pacheco et al., 2006] Pacheco J. M., Traulsen, A., Nowak, M. A., 2006. Coevolution of strategy and structure in complex networks with dynamical linking. Phys. Rev. Lett. 97, 258103.
- [Pattni et al., 2015] Pattni, K., Broom, M., Silvers, L., Rychtar, J., 2015. Evolutionary graph theory revisited: When is an evolutionary process equivalent to the Moran process? Proc. Roy. Soc. A471, 20150334.
- [Perc and Szolnoki, 2010] Perc, M., Szolnoki, A., 2010. Coevolutionary games–A mini review. BioSystems 99, 109–125.
- [Richter, 2008] Richter, H., 2008. Coupled map lattices as spatio–temporal fitness functions: Landscape measures and evolutionary optimization. Physica D237, 167–186.
- [Richter, 2014a] Richter, H., 2014a. Fitness landscapes that depend on time. In: Richter, H., Engelbrecht, A. P. (Eds.), Recent Advances in the Theory and Application of Fitness Landscapes, Springer–Verlag, Berlin Heidelberg New York, pp. 265–299.
- [Richter, 2014b] Richter, H., 2014b. Codynamic fitness landscapes of coevolutionary minimal substrates. In: C. A. Coello Coello (Ed.) Proc. IEEE Congress on Evolutionary Computation, IEEE CEC 2014, IEEE Press, Piscataway, NJ, pp. 2692–2699.
- [Richter, 2015] Richter, H., 2015. Coevolutionary intransitivity in games: A landscape analysis. In: Mora, A. M., Squillero, G. (Eds.), Applications of Evolutionary Computation - EvoApplications 2015, Springer-Verlag, Berlin, pp. 869–881.
- [Richter, 2016] Richter, H., 2016. Analyzing coevolutionary games with dynamic fitness landscapes. In: Ong, Y. S. (Ed.), Proc. IEEE Congress on Evolutionary Computation, IEEE CEC 2016, IEEE Press, Piscataway, NJ, pp. 610–616.
- [Richter and Engelbrecht, 2014] Richter, H., Engelbrecht, A. P. (Eds.), 2014. Recent Advances in the Theory and Application of Fitness Landscapes. Springer-Verlag, Berlin.
- [Shakarian et al., 2012] Shakarian, P., Roos, P., Johnson, A., 2012. A review of evolutionary graph theory with applications to game theory. BioSystems 107, 66–80.
- [Sloane, 2016] Sloane, N., The on–line encyclopedia of integer sequence. https://oeis.org (retrieved August, 20, 2016).
- [Southwell and Cannings, 2010] Southwell, R., Cannings, C., 2010. Some models of reproducing graphs. I. Pure reproduction. Appl. Math. 1, 137–145.
- [Spielman, 2009] Spielman, D., 2009. Spectral graph theory. In: Naumann, U., Schenk, O. (Eds.), Combinatorial Scientific Computing. CRC Press, Boca Raton, FL, pp. 495–524.
- [Stadler, 1996] Stadler, P. F., 1996. Landscapes and their correlation functions. J. Math. Chem. 20, 1–45.
- [Stadler and Stephens, 2003] Stadler, P. F., Stephens, C. R., 2003. Landscapes and effective fitness. Comm. Theor. Biol. 8, 389–431.
- [Szabo and Fath, 2007] Szabo, G., Fath, G., 2007. Evolutionary games on graphs. Phys. Rep. 446, 97-216.
- [Szolnoki et al., 2009] Szolnoki, A., Perc, M., Szabo, G., Stark, H. U., 2009. Impact of aging on the evolution of cooperation in the spatial prisoner’s dilemma game. Phys. Rev. E80, 021901.
- [Szolnoki and Perc, 2008] Szolnoki, A., Perc, M., 2008. Coevolution of teaching activity promotes cooperation. New J. Phys. 10, 043036.
- [Szolnoki and Perc, 2009a] Szolnoki, A., Perc, M., 2009a. Resolving social dilemmas on evolving random networks. Europhys. Lett. 86, 30007.
- [Szolnoki and Perc, 2009b] Szolnoki, A., Perc, M., 2009b. Promoting cooperation in social dilemmas via simple coevolutionary rules. Eur. Phys. J. B67, 337–344.
- [Szolnoki and Perc, 2014] Szolnoki, A., Perc, M., 2014. Coevolutionary success–driven multigames. Europhys. Lett. 108, 28004.
- [Tanimoto, 2007] Tanimoto, J., 2007. Dilemma solving by the coevolution of networks and strategy in a 2 2 game. Phys. Rev. E76, 021126-1-7.
- [Thompson, 1995] Thompson, J. N., 1995. The Coevolutionary Process. University of Chicago Press, Chicago, 1995.
- [Traulsen et al., 2005] Traulsen, A., Claussen, J. C., Hauert. C. 2005. Coevolutionary dynamics: from finite to infinite populations. Phys. Rev. Lett. 95.23, 238701.
- [Traulsen and Hauert, 2009] Traulsen, A., Hauert, C., 2009. Stochastic evolutionary game dynamics. In: Schuster, H. G. (Ed.), Reviews of Nonlinear Dynamics and Complexity Vol. II, Wiley-VCH, Weinheim, pp. 25–61.
- [Vassilev et al., 2000] Vassilev, V. K., Fogarty, T. C., Miller, J. F., 2000. Information characteristics and the structure of landscapes. Evolut. Comput. 8, 31–60.
- [Wang et al., 2014] Wang, Z., Szolnoki, A., Perc, M., 2014. Self–organization towards optimally interdependent networks by means of coevolution. New J. Phys. 16, 033041.
- [Wormald, 1999] Wormald, N. C. 1999. Models of random regular graphs. In: Lamb, J.D., Preece, D.A. (Eds.), Surveys in Combinatorics, London Mathematical Society Lecture Note Series, vol. 267, Cambridge University Press, Cambridge, pp. 239–298.
- [Zukewich et al., 2013] Zukewich, J., Kurella, V., Doebeli, M., Hauert, C. 2013. Consolidating birth-death and death-birth processes in structured populations. PLoS One 8, e54639.