# Evolutionary dynamics of cooperation in neutral populations

## Abstract

Cooperation is a difficult proposition in the face of Darwinian selection. Those that defect have an evolutionary advantage over cooperators who should therefore die out. However, spatial structure enables cooperators to survive through the formation of homogeneous clusters, which is the hallmark of network reciprocity. Here we go beyond this traditional setup and study the spatiotemporal dynamics of cooperation in a population of populations. We use the prisoner’s dilemma game as the mathematical model and show that considering several populations simultaneously give rise to fascinating spatiotemporal dynamics and pattern formation. Even the simplest assumption that strategies between different populations are payoff-neutral with one another results in the spontaneous emergence of cyclic dominance, where defectors of one population become prey of cooperators in the other population, and vice versa. Moreover, if social interactions within different populations are characterized by significantly different temptations to defect, we observe that defectors in the population with the largest temptation counterintuitively vanish the fastest, while cooperators that hang on eventually take over the whole available space. Our results reveal that considering the simultaneous presence of different populations significantly expands the complexity of evolutionary dynamics in structured populations, and it allow us to understand the stability of cooperation under adverse conditions that could never be bridged by network reciprocity alone.

## 1 Introduction

Methods of statistical physics, in particular Monte Carlo simulations and the theory of phase transitions [1, 2, 3], have been successfully applied to a rich plethora of challenging problems in the social sciences [4, 5, 6, 7]. The evolution of cooperation in social dilemmas — situations where what is best for the society is at odds with what is best for an individual — is a vibrant example of this development. Many reviews [8, 9, 10, 11] and research papers that reveal key mechanisms for socially preferable evolutionary outcomes have been published in recent years [12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26]. Since cooperative behaviour is central to the survival of many animal species, and since it is also at the heart of the remarkable evolutionary success story of humans [27, 28], it is one of the great challenges of the 21st century that we succeed in understanding how best to sustain and promote cooperation [29].

It has been shown that phase transitions leading to cooperation depend sensitively on the structure of the interaction network and the type of interactions [30, 31, 32, 33, 34], as well as on the number and type of competing strategies [8, 35, 36, 37]. An important impetus for the application of statistical physics to evolutionary social dilemmas and cooperation has been the seminal discovery of Nowak and May [38], who showed that spatial structure can promote the evolution of cooperation through the mechanism that is now widely referred to as network reciprocity [39, 40]. A good decade latter Santos and Pacheco have shown just how important the structure of the interaction network can be [13], which paved the way further towards a flourishing development of this field of research.

But while research concerning the evolutionary dynamics of cooperation in structured populations has come a long way, models where different populations do not interact directly but compete for space at the level of individuals have not been considered before. Motivated by this, we consider a system where two or more populations are distributed randomly on a common physical space. Between the members of a particular population the interactions are described by the prisoner’s dilemma game. But there are no such interactions between players belonging to different populations, and hence players are unable to collect payoffs from neighbors belonging to a different population. The populations on the same physics space, for example on a square lattice, are thus neutral. Nevertheless, all players compete for space regardless to which population they belong, so that a player with a higher fitness is likely to invade a neighboring player with a lower fitness.

As we will show, such a conglomerate of otherwise neutral populations gives rise to fascinating spatiotemporal dynamics and pattern formation that is rooted in the spontaneous emergence of cyclic dominance. Within a very simple model, we observe the survival of cooperators under extremely adverse conditions where traditional network reciprocity would long fail, and we observe the dominance of the weakest due to the greediness of the strongest when considering different temptations to defect in different populations.

In what follows, we first present the studied prisoner’s dilemma game and the details of the mathematical model. We then proceed with the presentation of the main results and a discussions of their wider implications.

## 2 Prisoner’s dilemma in neutral populations

As the backbone of our mathematical model, we use a simplified version of the prisoner’s dilemma game, where the key aspects of this social dilemma are preserved while its strength is determined by a single parameter [38]. In particular, mutual cooperation yields the reward , mutual defection leads to punishment , while the mixed choice gives the cooperator the sucker’s payoff and the defector the temptation . We note that the selection of this widely used and representative parameterisation gives results that remain valid in a broad range of pairwise social dilemmas, including the snowdrift and the stag-hunt game.

All players occupy the nodes of a square lattice with four neighbours each. To introduce the simultaneous presence of different populations, the players are assigned to different populations uniformly at random. All populations contain an equal fraction of cooperators and defectors, who upon pairwise interactions receive payoffs in agreement with the above-described prisoner’s dilemma game. Importantly, between different populations players are payoff-neutral with one another, which means that when meets or , its payoff does not change, and vice versa. In the next subsection, we first consider the model where all populations have the same temptation to defect ( for all ), and then we relax this condition to allow different temptations to defect in different populations.

We use the Monte Carlo simulation method to determine the spatiotemporal dynamics of the mathematical model, which comprises the following elementary steps. First, a randomly selected player acquires its payoff by playing the game potentially with all its four neighbours. Next, player randomly chooses one neighbour , who then also acquires its payoff in the same way as previously player . Finally, player imitates the strategy of player with the probability , where we use as the inverse of the temperature of selection to obtain results comparable with existing research [8]. Naturally, when neighbouring players compete for space then the above describe microscopic dynamics involves not only the adoption of more successful strategy but also the imitation of the involved population tag.

In agreement with the random sequential simulation procedure, during a full Monte Carlo step (MCS) each player obtains a chance once on average to imitate a neighbor. The average fractions of all microscopic states on the square lattice are determined in the stationary state after a sufficiently long relaxation time. Depending on the proximity to phase transition points and the typical size of emerging spatial patterns, the linear system size was varied from to , and the relaxation time was varied from to MCS to ensure that the statistical error is comparable with the size of the symbols in the figures.

## 3 Results

Naively, one might assume that introducing several populations simultaneously which bear the same serious conflict of competing strategies might not bring about any changes in the evolutionary outcome. As is well known, the Nash equilibrium of the prisoner’s dilemma game is mutual defection [41], and since this applies to all populations, the overall outcome should be mutual defection too. This reasoning is actually completely correct in well-mixed populations, where the consideration of different, otherwise neutral populations really does not change the result: cooperators die out in all populations as soon as . But as we will show next, this naive expectation is completely wrong in structured populations, where excitingly different evolutionary outcomes can be observe due to the simultaneous presence of different populations.

As far as cooperation promotion is concerned, and before elucidating the responsible microscopic mechanism for such favourable evolutionary outcomes, we show in Fig. 1 how the fraction of cooperators changes in dependence on the temptation to defect for different numbers of populations that form the global system. For comparison, we also show the baseline case, which corresponds to the traditional version of the weak prisoner’s dilemma game on the square lattice, and where cooperators benefit from network reciprocity to survive up to [43]. It can be observed that, as we increase , the fraction of cooperators increases dramatically. In fact, the higher the value of , the higher the stationary fraction of cooperators in the whole system.

The spatiotemporal dynamics behind this promotion of cooperation in a complex system consisting of two populations can be seen in the animation provided in [42], while a representative snapshot of the stationary state is shown in Fig. 2. In both cases cooperators are depicted blue while defectors are depicted red, and different shades of these two colours denote adherence to the two different populations. In Fig. 2, we have circled two crucial details that explain how the patterns evolve over time. The white circle marked ‘I’ highlights that dark red defectors can easily invade dark blue cooperators. However, the invaded space is quickly lost to light blue cooperators belonging to the other population. The latter, on the other hand, are successfully invaded by light red defectors from their own population, who are in turn again invaded by dark blue cooperators. In this way the loop is closed, revealing the spontaneous emergence of cyclic dominance in the form , which determines the stationary distribution of strategies in our system. As is well-known, the cyclic dominance is crucial for the maintenance of biodiversity [36], which in our case translates to the survival of all four competing strategies, and thus to the sustenance of cooperation even at very high temptation values.

This cyclic dominance can be observed directly if we launch the evolution from a prepared initial state, such that homogeneous domains of the competing strategies are separated by straight interfaces, as in the animation provided in [44] (in this animation a higher temptation to defect was used to yield clearer propagating fronts). It can be observed that conceptually similar propagating fronts emerge as were observed before in rock-paper-scissors-like systems [36].

Turning back to Fig. 2, the white ellipse marked ‘II’ highlights another important aspect of the spatiotemporal dynamics, namely the smooth interface separating the two cooperative strategies in the absence of defectors. This may be surprising at first because these strategies are payoff-neutral, and thus a voter-model-like coarsening with highly fluctuating interfaces would be expected [45]. Indeed, while a cooperator does not benefit from the vicinity of a cooperator, other cooperators close by of course increase each other’s payoffs (and vice versa for cooperators). As a consequence of this the payoffs of and cooperators along the interface differ, so that one will likely invade the other. This process always aims to straighten the interfaces. If an interface cannot be straightened, for example around a small island, the latter will shrink due to an effective surface tension.

Lastly in terms of the results presented in Fig. 1, it remains to explain why the larger the number of populations forming the global system the higher the level of cooperation in the stationary state, and this regardless of the temptation to defect. To that effect we provide in [46] an animation showing the spatiotemporal dynamics when , and in Fig. 3 a representative snapshot of the distribution of strategies on the square lattice in the stationary state. These results reveal that the increasing positive effect is due to the fact that the addition of one new population always yields one additional prey to the cooperators in other populations. At the same time, no new predators to them are introduced, i.e., defectors who act as the prey to cooperators in the other populations are predators only to cooperators, but the latter find their prey in defectors from other populations too. The snapshot in Fig. 3 features two white ellipses, where it is highlighted that the plain red defectors are dominated by both (dark blue) and (light blue) cooperators (see also the animation in [46]).

Thus far, we have only considered cases where the temptation to defect was the same in all populations. By relaxing this restriction, the number of free parameters increases significantly, yet it is still possible to determine general properties of the spatiotemporal dynamics that governs the evolutionary outcomes in a presented system.

We begin by presenting results for the generalized two-population setup where . As we have shown above, the emergence of cyclic dynamics between the four competing microscopic states in general dictates a stable coexistence. By increasing the temptation to defect in one population practically increases the rate in the corresponding invasion. The consequences of this fact, based on the fundamental principles of cyclic dominance [36], actually completely explain the evolutionary outcomes in Fig. 4. The first potentially surprising observation is that increasing the temptation to defect between defectors and cooperators will not only lower the stationary fraction of and increase the stationary fraction of , but also elevate the fraction of cooperators. This is because defectors are prey to cooperators, and it is well-known that a species entailed in cyclic dominance is promoted not by weakening its predator, but rather by making its prey stronger. This paradox is a frequently observed trademark of systems that are governed by cyclic dominance [47]. However, despite the described boost to the growth of cooperators, the overall fraction of all cooperators in the whole system decreases slightly as we increase towards very large values, as illustrated in the inset of Fig. 4.

For a better demonstration of the acceleration of the invasion and the resulting boost to cooperators (dark blue), we provide an animation in [48], where an extreme high was used at linear system size. As the animation shows, although cooperators (light blue) are invaded very efficiently by defectors (light red), the abundance of defectors (dark red) always offers an evolutionary escape hatch out of extinction of cooperators. In agreement with the above described cyclic dominance, defectors are fast invaded by cooperators. Interestingly, defectors would also beat cooperators because is above the cooperation survival threshold of a single population, yet the propagating front always comes to the rescue, bringing with it defectors as prey.

In comparison to the results obtained when the temptation to defect is the same in all populations (see Fig. 1), it may come as a surprise that cooperators die out if , and this despite the fact that qualitatively the same cyclic dominance emerges there. The explanation of this difference illustrated in Fig. 4 is that in the symmetrical case the and invasion rates change simultaneously as we vary . However, it is precisely this simultaneous change of invasion rates that may jeopardize the stable coexistence in models of cyclic dominance. As shown previously for a symmetric strategy Lotka-Volterra system, the coexistence disappears if the difference between the invasion rates exceeds a threshold value [49]. For an illustration, the effective food-web of the four competing strategies in a two-population model is shown in left panel of Fig. 5.

Naturally, if we allow different temptation values in different populations the behaviour becomes even more complex, as we show next using still a relatively simple three-population system as an example. The effective food-web is shown in the right panel of Fig. 5. If we just vary , while the temptation to defect in the other two populations remains fixed at , the invasion rate will influence invasions in several other cycles in the effective food-web. Examples include the cycle, or the cycle, or the cycle, all of which contain the elementary invasion that is directly affected by . This is why it is almost impossible to predict the response of a system comprised of several neutral populations, even if only a single temptation to defect is varied.

For the above case, the results showing how different values affect the evolutionary outcome are presented in Fig. 6. It can be observed that upon increasing the value of , the stationary fraction of and cooperators is not affected, even though they are the predators of who should in principle be promoted by large values. On the other hand, the overall fraction of all defectors in the system remains very low. But the most exotic reaction is that of the fraction of cooperators, which is of course the direct prey of defectors. While initially their fraction in the stationary state decreases to a shallow minimum across the intermediate range of values, it ultimately increases to complete dominance above a threshold value. In other words, while defectors survive when all values in the system are equal to , they die out if we increase one of them sufficiently, as it happens in Fig. 6 when the value is sufficiently large. Due to the symmetry of the model the same results are of course obtained if either the value of or would be enlarged instead of the value of .

To better understand and illustrate the seemingly paradoxical effect the increasing value has on the evolutionary outcome, we provide an animation from a prepared initial state in [50]. Here the square lattice is horizontally divided into two parts, where in the top half cooperators (dark blue) are framed by defectors (dark red), while in the bottom half cooperators (plane blue) are framed by defectors (plain red). Moreover, both patches are surrounded by cooperators (light blue). The applied temptation to defect values are and . Importantly, invasions through the horizontal border are not permitted because we want to compare the independent evolution of both sub-systems. Since defectors are not present, cooperators have no natural predator. As a consequence, the whole system will evolve into a pure (light blue) phase. However, the really interesting aspect of this animation is how the mentioned sub-systems reach this state. In the top half, defectors are less aggressive, and therefore their invasions are less salient. This has two important consequences. In the first place, their payoffs are not that high for the other strategies to imitate them, and so the border is fluctuating rather strongly. Secondly, defectors do not form a homogeneous front along this border. The latter would be essential for a fast invasion of cooperators (light blue), who are their predators. In other words, the effective invasion of cooperators can only happen via the invasion of defectors. The latter conditions is completely fulfilled in the bottom half where defectors are more aggressive. Here defectors form not just a more compact invasion front, but they also form a thick, uniform stripe, which is an easy target for cooperators. Consequently, the more aggressive defectors will die out much faster than their less potent counterparts in the top half of the square lattice.

This process just described is actually very common when the value of the temptation to defect in one population is significantly larger than the corresponding values in other populations. Of course, the extinction of the most aggressive defector frequently involves also the extinction of its cooperator prey. Sometimes, however, if the system size is large enough, it may happen that the prey of the more aggressive defectors manages to separate itself in an isolated part of the lattice and hang on until his predators die out. Such a situation is illustrated in Fig. 7, where the white ellipses and circles mark plain blue cooperator spots who got rid of their natural predators (plain red). In the absence of the latter, the arguably weakest cooperators become the strongest, and they eventually rise to complete dominance by invading defectors from the other two populations who themselves continuously invade their cooperators. The whole evolutionary process can be seen in the animation in [51], where we have used prepared initial patches of the six competing strategies to make the spatiotemporal dynamics that leads to the described pattern formation better visible. Additionally, for a faster evolution, we have used a smaller linear system size. In effect, the plain blue cooperators use the defectors from the other two populations as a Trojan horse to invade the whole available space. And despite of starting as the weakest, they turn out to be the dominant due to the greediness of their direct predators.

## 4 Discussion

We have studied the spatiotemporal dynamics of cooperation in a system where several neutral populations are simultaneously present. The evolutionary prisoner’s dilemma game has been used as the backbone of our mathematical model, where we have assumed that strategies between the populations are payoff-neutral but competing freely with one another as determined by the interaction graph topology. Within a particular population the classical definition of the prisoner’s dilemma game between cooperators and defectors has been applied. We have observed fascinating spatiotemporal dynamics and pattern formation that is unattainable in a single population setup. From the spontaneous emergence of cyclic dominance to the survival of the weakest due to the greediness of the strongest, our results have revealed that the simultaneous presence of neutral populations significantly expands the complexity of evolutionary dynamics in structured populations. From the practical point of view, cooperation in the proposed setup is strongly promoted and remains viable even under extremely adverse conditions that could never be bridged by network reciprocity alone. The consideration of simultaneously present neutral populations thus allows us to understand the extreme persistence and stability of cooperation without invoking strategic complexity, and indeed in the simplest possible terms as far as population structure and overall complexity of the mathematical model is concerned.

The central observation behind the promotion of cooperation is that, if we put two payoff-neutral populations together, then only cooperators can benefit from it in the long run. While the advantage of mutual cooperation is readily recognizable already in a single population, and it is in fact the main driving force behind traditional network reciprocity, the extend of it remains limited because cooperators at the frontier with defectors always remain vulnerable to invasion. This danger is here elegantly avoided when a cooperative cluster meets with the defectors of the other population. In the latter case the positive consequence of network reciprocity is augmented and cooperators can easily invade the territory of the foreign defectors. Importantly, this evolutionary success of cooperators in one population works vice versa for cooperators in the other population(s) too. Due to this symmetry, it is easy to understand that, as we have shown, the larger the number of populations forming the system, the more effective the promotion of cooperation.

We have also shown that the already mentioned positive impact can be enhanced further if we allow different temptations to defect in different populations. Counterintuitively, in a system where the population specific temptation to defect values are diverse enough, defectors die out first whose temptation value is the largest. And this turns out to be detrimental for defectors in other populations too. An extreme aggressive invasion namely leads to the fast depletion of the prey – in this case the cooperators from the corresponding population – which in turn leads to the extinction of the predators. However, the reverse situation is not valid: if the most vulnerable cooperators somehow manage to survive, they eventually rise to complete dominance, using defectors from other populations as Trojan horses to invade cooperators from other populations. This gives rise to the dominance of the weakest due to the greediness of the strongest, and it also reminds us that dynamical processes in different populations should not be too diverse because this jeopardizes the stability of the whole system.

## References

### References

- Stanley H E 1971 Introduction to Phase Transitions and Critical Phenomena (Oxford: Clarendon Press)
- Marro J and Dickman R 1999 Nonequilibrium Phase Transitions in Lattice Models (Cambridge, U.K.: Cambridge University Press)
- Hinrichsen H 2000 Adv. Phys. 49 815–958
- Castellano C, Fortunato S and Loreto V 2009 Rev. Mod. Phys. 81 591–646
- Helbing D, Brockmann D, Chadefaux T, Donnay K, Blanke U, Woolley-Meza O, Moussaid M, Johansson A, Krause J and Perc M 2015 J. Stat. Phys. 158 735–781
- D’Orsogna M R and Perc M 2015 Phys. Life Rev. 12 1–21
- Pastor-Satorras R, Castellano C, Van Mieghem P and Vespignani A 2015 Rev. Mod. Phys. 87 925
- Szabó G and Fáth G 2007 Phys. Rep. 446 97–216
- Perc M and Szolnoki A 2010 BioSystems 99 109–125
- Perc M, Gómez-Gardeñes J, Szolnoki A and Floría and Y Moreno L M 2013 J. R. Soc. Interface 10 20120997
- Wang Z, Wang L, Szolnoki A and Perc M 2015 Eur. Phys. J. B 88 124
- Zimmermann M G and Eguíluz V M 2005 Phys. Rev. E 72 056118
- Santos F C and Pacheco J M 2005 Phys. Rev. Lett. 95 098104
- Santos F C, Pacheco J M and Lenaerts T 2006 Proc. Natl. Acad. Sci. USA 103 3490–3494
- Pacheco J M, Traulsen A and Nowak M A 2006 Phys. Rev. Lett. 97 258103
- Gómez-Gardeñes J, Campillo M, Floría L M and Moreno Y 2007 Phys. Rev. Lett. 98 108103
- Masuda N 2007 Proc. R. Soc. B 274 1815–1821
- Fu F, Wu T and Wang L 2009 Phys. Rev. E 79 036101
- Floría L M, Gracia-Lázaro C, Gómez-Gardeñes J and Moreno Y 2009 Phys. Rev. E 79 026106
- Lee S, Holme P and Wu Z X 2011 Phys. Rev. Lett. 106 028702
- Tanimoto J, Brede M and Yamauchi A 2012 Phys. Rev. E 85 032101
- Mobilia M 2012 Phys. Rev. E. 86 011134
- Szolnoki A and Perc M 2016 EPL 113 58004
- Fu F and Chen X 2017 New J. Phys. 19 071002
- Battiston F, Perc M and Latora V 2017 New J. Phys. 19 073017
- Szolnoki A and Perc M 2017 Phys. Rev. X 7 041027
- Hrdy S B 2011 Mothers and Others: The Evolutionary Origins of Mutual Understanding (Cambridge, MA: Harvard University Press)
- Nowak M A and Highfield R 2011 SuperCooperators: Altruism, Evolution, and Why We Need Each Other to Succeed (New York: Free Press)
- Axelrod R 1984 The Evolution of Cooperation (New York: Basic Books)
- Albert R and Barabási A L 2002 Rev. Mod. Phys. 74 47–97
- Boccaletti S, Latora V, Moreno Y, Chavez M and Hwang D 2006 Phys. Rep. 424 175–308
- Holme P and Saramäki J 2012 Phys. Rep. 519 97–125
- Kivelä M, Arenas A, Barthelemy M, Gleeson J P, Moreno Y and Porter M A 2014 J. Complex Netw. 2 203–271
- Boccaletti S, Bianconi G, Criado R, del Genio C, Gómez-Gardeñes J, Romance M, Sendiña-Nadal I, Wang Z and Zanin M 2014 Phys. Rep. 544 1–122
- Roca C P, Cuesta J A and Sánchez A 2009 Phys. Life Rev. 6 208–249
- Szolnoki A, Mobilia M, Jiang L L, Szczesny B, Rucklidge A M and Perc M 2014 J. R. Soc. Interface 11 20140735
- Perc M, Jordan J J, Rand D G, Wang Z, Boccaletti S and Szolnoki A 2017 Phys. Rep. 687 1–51
- Nowak M A and May R M 1992 Nature 359 826–829
- Nowak M A 2006 Science 314 1560–1563
- Rand D G, Nowak M A, Fowler J H and Christakis N A 2014 Proc. Natl. Acad. Sci. USA 111 17093–17098
- Nash J 1951 Ann. Math. 54 286–295
- figshare 2017 http://dx.doi.org/10.6084/m9.figshare.5281549
- Szabó G, Vukov J and Szolnoki A 2005 Phys. Rev. E 72 047107
- figshare 2017 http://dx.doi.org/10.6084/m9.figshare.5281570
- Dornic I, Chaté H, Chave J and Hinrichsen H 2001 Phys. Rev. Lett. 87 045701
- figshare 2017 http://dx.doi.org/10.6084/m9.figshare.5281591
- Frean M and Abraham E D 2001 Proc. R. Soc. B 268 1323–1327
- figshare 2017 http://dx.doi.org/10.6084/m9.figshare.5281606
- Szabó G and Szolnoki A 2008 Phys. Rev. E 77 011906
- figshare 2017 http://dx.doi.org/10.6084/m9.figshare.5281624
- figshare 2017 http://dx.doi.org/10.6084/m9.figshare.5281627