Imitating emotions instead of strategies in spatial games elevates social welfare
The success of imitation as an evolutionary driving force in spatial games has often been questioned, especially for social dilemmas such as the snowdrift game, where the most profitable may be the mixed phase sustaining both the cooperative as well as the defective strategy. Here we reexamine this assumption by investigating the evolution of cooperation in spatial social dilemma games, where instead of pure strategies players can adopt emotional profiles of their neighbors. For simplicity, the emotional profile of each player is determined by two pivotal factors only, namely how it behaves towards less and how towards more successful neighbors. We find that imitating emotions such as goodwill and envy instead of pure strategies from the more successful players reestablishes imitation as a tour de force for resolving social dilemmas on structured populations without any additional assumptions or strategic complexity.
Dynamics of social systems Dynamics of evolution Structures and organization in complex systems
Societies facing a social dilemma are at risk of failing to uphold wellbeing in their ranks because there exist strong incentives to put success of individuals above that of the society as a whole. It is therefore in the best, although not completely obvious, interest of all if social dilemmas are mitigated or, if at all possible, altogether avoided. Cooperative behavior  is something of a holly grail when it comes to resolving social dilemmas. To cooperate traditionally means to sacrifice some fraction of personal benefits for elevating social welfare. However, in the face of natural selection, favoring the fittest and the strongest amongst us, the concept quickly becomes misty and the outlook for cooperators to survive murky. Enter evolutionary games [2, 3, 4], which are frequently employed to help us reveal and understand the mechanisms and reasons why cooperation nevertheless prevails and is in fact much more common than one could assume. Examples of recent research works aimed towards this direction include [5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18].
One of the most rewarding observations in recent history related to the resolution of social dilemmas was that spatial reciprocity can maintain cooperative behavior without additional assumptions or mechanism weighing down on defectors . Other well known mechanisms promoting cooperation include kin selection , direct and indirect reciprocity , as well as group  and multilevel selection [23, 24]. These as well as related mechanism for the promotion of cooperation have been comprehensively reviewed in , and there are a number of recent reviews devoted to evolutionary games that capture succinctly recent advances made along this very vibrant avenue of research [26, 27, 28, 29]. Focusing on spatial reciprocity, however, one finds that certain social dilemmas are not susceptible to its workings, and that indeed well-mixed conditions may represent a more favorable environment. Hauert and Doebeli  reported that, especially for the snowdrift game, the promotion of cooperation by means of imitation on structured populations is problematic because the Nash equilibrium is a mixed phase of cooperators and defectors. Consequently, it is advantageous to imitate strategies that are opposite to neighboring strategies, which ultimately leads to a failure of utilizing advantages of spatial reciprocity. Moreover, while some experimental findings question the importance of imitation , others find that humans may imitate even in situations that may decrease their chance of further success , suggesting that such seemingly maladaptive behavior may be due to the inherent evolutionary usefulness of imitation in other situations.
Here we study the evolution of cooperation in spatial social dilemma games, but departing from the traditional assumption that strategies of players are the ones to potentially be imitated. Although it is certainly reasonable to assume that if one strategy is performing good imitating it is bound to yield positive results, we caution that this may not always be the case. Indeed, it is easy to come up with many such examples, the most obvious one being that imitating defection from a player that is surrounded by cooperators is a very bad idea if oneself is surrounded by defectors. Of course this scenario is more or less likely depending on the overlap between the neighborhoods of the two players, and may be more applicable to human societies than a grouping of simple microorganisms, yet it nevertheless is motivating enough for us to reconsider the concept of imitation. In particular, we refine it by allowing players not simply to imitate pure strategies, but rather to imitate emotional profiles of each other. In order to keep the model simple but still capturing the most relevant new features, we assign to every player two properties that define its emotional profile, namely the probability to cooperate with a more successful neighbor and the probability to cooperate with a less successful neighbor. With the first we determine envy or spite, while with the second property we determine goodwill or charity of each individual. In this way the strategy becomes link-specific rather than player-specific, as is the case in the traditional model. Obviously, other interpretations of the two probabilities are possible as well. Interestingly, we find that, without any additional assumptions, by imitating the more successful emotional profiles instead of simply the more successful strategies, the evolution of cooperation is significantly promoted and substantially higher social welfare is attainable, even in games where the most favorable is the mixed strategy phase. Thus, a simple fine tuning of the concept of imitation, or rather of what is possible to imitate, reestablishes imitation as an important and globally beneficial behavior in evolutionary processes.
The remainder of this letter is organized as follows. First, we describe the considered social dilemmas and the protocol for the imitation of emotional profiles. Next we present the results, whereas lastly we summarize and discuss their implications.
2 Social dilemmas and emotional profiles
Assuming that mutual cooperation yields the reward , mutual defection leads to punishment , and the mixed choice gives the cooperator the sucker’s payoff and the defector the temptation , we have the prisoner’s dilemma game if , the snowdrift game if , and the stag-hunt game if , thus covering all three major social dilemma types where players can choose between cooperation and defection. Following common practice, we set and , thus leaving the remaining two payoffs to occupy and , as depicted schematically in Fig. 1.
In the traditional model, irrespective of the governing social dilemma, each player occupies a node on the square lattice and is initially designated either as a cooperator or defector with equal probability, while evolution of the two strategies is performed in accordance with the Monte Carlo simulation procedure comprising the following elementary steps. First, a randomly selected player acquires its payoff by playing the game with all its four neighbors. Next, one randomly chosen neighbor of , denoted by , also acquires its payoff by playing the game with all its four neighbors. Traditionally player then imitates the strategy of player with the probability , where determines the level of uncertainty by strategy adoptions , which can be attributed to errors in judgment due to mistakes and external influences that affect the evaluation of the opponent. Without loss of generality we set , implying that better performing players are readily imitated, but it is not impossible to adopt the strategy of a player performing worse. This value of is representative for a wide range of finite selection intensities. The weak-selection limit [34, 35, 36] (), however, is not studied in the present work. For this traditional setup the stationary fraction of cooperators in the parameter plane is as depicted in Fig. 1. Well known results include the widespread dominance of defectors in the prisoner’s dilemma quadrant, as well as the possibility of cooperator dominance and coexistence with the defectors in the snowdrift and the stag-hung quadrant, yet only for sufficiently favorable combinations of and . These results will primarily be used for comparison purposes with the main findings that will be presented in the next section.
In order to depart from the traditional setup of spatial social dilemma games summarized above, we introduce an emotional profile to each player , which is determined by the parameter pair . Here is the probability that player will cooperate with player if , while is the probability that player will cooperate with player if . Essentially thus, the two parameters determine how a given player will behave when facing more or a less successful opponent. Initially, to enable the start of the evaluation process, each player is assigned a random ) pair and a payoff from the reachable interval. Subsequently, every payoff value is updated by considering the proper neighborhoods of a player and the actual emotional parameters. Importantly, after the accumulation of new payoffs, player does not imitate the strategy of player with the previously established adoption probability , but rather its emotional profile, i.e. the and/or value. Such a profile implicitly allows a player to behave differently (to cooperate and/or defect) towards different neighbors at the same time. Since the emotional profile consist of two parameters, however, the imitation is done separately for the two to avoid potential artificial propagation of freak (extremely successful) pairs. Naturally, the same probability is applied for both imitations. Finally, after each imitation the payoff of player is updated using its new emotional profile, whereby each full Monte Carlo step involves all players having a chance to adopt the emotional profile from one of their neighbors once on average. Prior to presenting the result of this model, it is important to note that there will always be a fixation of pairs, i.e. irrespective of and only a single pair will eventually spread across the whole population. Naturally, the fixation time depends on the system size as well as game parametrization, which we have taken properly into account by sufficiently long simulations times prior to recording the final and value. It is also worth pointing out that once the fixation occurs, the evolutionary process stops. The characteristic probability of encountering cooperative behavior on the spatial grid, which is equivalent to the stationary fraction of cooperators in the traditional version of the game, can thus be determined by means of averaging over the final states that emerge from different initial conditions.
We start by presenting the color map encoding the final values of on the parameter plane in Fig. 2. Since is the probability that players will cooperate with their less successful neighbors, i.e. despite the fact that their payoff is lower, this can be interpreted either as goodwill or charity. From the presented results it follows that for the snowdrift quadrant this behavior is practically completely dominant, irrespective of the details of game parametrization. Thus, if the governing social dilemma is of the snowdrift type, then players will always () cooperate with their neighbors provided their payoff is lower. For the stag-hunt game, on the other hand, the region of corresponds roughly to the region of cooperator dominance in the traditional model (compare with results presented in Fig. 1), although it extends somewhat further towards smaller and larger values. Results in the lower right quadrant, corresponding to the prisoner’s dilemma game, are equally positive, indicating that as long as is not too low, cooperation with less successful neighbors will be the dominant behavior. This holds virtually independent of , although surprisingly as increases the minimal still warranting decreases. It can thus be concluded that raising the temptation to defect may even facilitate charitable actions in that they are upheld even by lower values of .
Since the final values of reveal only half of the behavior on the spatial gird, it is next of interest to examine the color map encoding the final values of on the parameter plane. Results presented in Fig. 3 reveal at a glance that it is significantly more difficult to achieve cooperation with more successful neighbors than vice versa (compare with results presented in Fig. 2). While the results for the stag-hung game for are practically identical to those for , the situation is much different for the snowdrift and the prisoner’s dilemma game. In the snowdrift quadrant the total dominance of is replaced by near complete dominance of , indicating that players will not cooperate with their neighbors if the later are more successful. Envy thus appears to be an important agonist for the evolution of defection, rather than cooperation, in the snowdrift game. Only for values of slightly above , and irrespective of , will players choose to cooperate with their more successful neighbors, but otherwise not. For the prisoner’s dilemma game the results are equally negative, further restricting cooperation with more successful neighbors not only to small values of , but also only to moderately negative values of . As we will reveal below, however, unwillingness to cooperate with the more successful neighbors has negative consequences mainly for the evolution of cooperation in the prisoner’s dilemma game, while for the snowdrift game this fact actually favors the emergence of the globally optimal mixed phase warranting the highest level of social welfare.
By considering the results presented in Figs. 2 and 3 combined, we arrive at the probability to encounter cooperative behavior on the spatial grid, as depicted in Fig. 4. Here denotes the average level of cooperative behavior on the spatial grid after the evolution of emotional profiles has stopped, i.e. after the fixation of and . Since the regions of and in the stag-hunt quadrant overlap completely, it is natural that in this region also the probability to encounter cooperation will be equal to . Comparing this to the results presented in Fig. 1, it can be concluded that replacing the imitation of strategies with the imitation of emotional profiles in the stag-hunt game promotes cooperation by extending the region towards larger values of and smaller values of . For the snowdrift and the prisoner’s dilemma game full dominance of cooperative behavior can be observed where and regions overlap, while if and the probability to encounter cooperative behavior equals . Naturally, where both and are equal to zero also . Altogether, by comparing results presented in Figs. 1 and 4, it can be concluded that imitating emotional profiles, and thus having the liberty to behave differently towards different players, instead of adopting pure strategies, strongly promotes the evolution of cooperation in all three considered spatial social dilemma games. Particularly players engaging in the snowdrift game profit immensely from the new imitation procedure, which is surprising since especially for spatial games having a mixed Nash equilibrium imitation has acquired quite a negative reputation .
Since the success of imitation for spatial games where the Nash equilibrium is a mixed phase (coexistence of cooperators and defectors), as is the case for the snowdrift game, has often been questioned, it is thus of interest to examine results in this particular region of the parameter plane more precisely. Foremost, it should be emphasized that fine-tuning the imitation (what to imitate) procedure clearly restores the successfulness of imitation to arrive at a final state that is optimal for the society as a whole (see also results presented in Fig. 6 further below). The snapshot depicted in the left panel of Fig. 5 presents a typical final configuration of players, color-coded in such a way that if the player behaves cooperatively more (less) frequently than defectively toward its neighbors it is depicted green (red), while if the two actions are equally frequent it is depicted yellow. The presented snapshot reveals a characteristic checkerboard distribution of expected strategies, which is made even clearer by the enlargement of a typical region of the spatial grid depicted in the right panel of Fig. 5. Noteworthy, as a result of the evolutionary process, and despite of diverse strategies, players exhibit identical willingness to either cooperate or to defect, i.e. are characterized by the same emotional profile. This indicates that under the newly proposed imitation procedure players indeed share roles of cooperation and defection in order to arrive at the “socially optimal” configuration. Put differently, the spatial arrangement of players demonstrates that using the same attitude towards more or less successful players may result in a spatial mixture of cooperative and defective actions that warrants the highest mutual payoff.
Finally, as the last, and perhaps most persuading evidence for the successfulness of the newly proposed imitation procedure, it is thus instructive to examine the difference in payoffs between the traditional version of the games (results presented in Fig. 1) and the one introduced here adopting imitation of emotional profiles instead of strategies. Results presented in Fig. 6 reveal most clearly the extent of cooperation promotion in the stag-hunt quadrant (the black stripe in the lower left quadrant corresponds accurately to the enlarged area of cooperator dominance), as well as the transition towards the socially optimal mixed phase in large regions of the snowdrift (upper right) quadrant. The prisoner’s dilemma game, arguably constituting the most demanding conditions for the evolution of cooperation, also presents itself as very much susceptible to the positive impact of the new imitation procedure, if only the sucker’s payoff is not too negative. Note also that the difference in the harmony quadrant and partly also in the stag-hunt quadrant is zero because both models yield a full phase. With these final results, we conclude that imitating emotions such as goodwill and envy instead of unconditionally copying pure strategies from the more successful players reestablishes imitation as perfectly suitable for resolving social dilemmas on structured populations, even for games where the Nash equilibrium is a mixed phase.
Before concluding, we note that the dynamics of the model proposed in this letter is significantly different from the one emerging when the evolutionary process is governed by stochastic reactive strategies [37, 26]. In the latter case, the choice of action in a given round is only affected by the opponent’s behavior in the previous round, and consequently, a special form of reciprocity can emerge between neighbors because a cooperative act will likely trigger a similar reaction (to cooperate) from the targeted player. As we have argued, this is not necessarily true when emotions are subject to imitation. The role-separating mixed phase in the snowdrift quadrant has already been observed in spatial games, but it needed a significantly different – the so-called myopic strategy update – where a player can change the strategy independently from its neighborhood [38, 39]. Results presented here reveal that such a state can evolve also by means of imitation. To highlight the robustness of our findings, we have also applied the so-called death-birth updating , but found very similar results. Furthermore, the application of weak mutation, allowing the emergence of independent pairs, does not interfere with the evolution towards unique emotion profiles, as we have described above.
In sum, we have proposed and studied an alternative form of imitation, focusing specifically on its impact on the evolution of cooperation in the three most frequently considered spatial social dilemma games, namely the spatial snowdrift, stag-hunt and the prisoner’s dilemma game. By replacing the imitation of strategies by the imitation of emotional profiles of players, as defined by the probability to cooperate with the more and less successful neighbors, we have found that players are much more likely to cooperate with less successful neighbors than they do with those who are more successful. Thus, while goodwill and charity appear to be important agonists facilitating the evolution of cooperation, envy and spite act detrimental, favoring the evolution of defection instead. Importantly, this duality in the evolution of the two emotional traits of players actually leads to rather unexpected benefits in the snowdrift game, where the Nash equilibrium is a mixed phase. Although imitation was previously thought to be unsuitable for achieving the socially optimal state in this type of spatial games, our results indicate that the limitations lie not in the act of imitation itself, but rather in what is available for imitation. By replacing the strategy with a slightly more elaborate concept of an emotional profile, we have found that imitation is fully capable of guiding the population towards the globally optimal state warranting the highest level of social welfare. The stag-hunt as well as the prisoner’s dilemma game are also susceptible to the promotion of cooperation by means of the newly proposed imitation procedure. But while in the stag-hunt game benefits from both the cooperation with less as well as with the more successful neighbors are attainable, in the prisoner’s dilemma game the positive impact on the evolution of cooperation is (almost) entirely due to players being willing to cooperate with their less successful neighbors. Envy, being prohibitive to act cooperatively with more successful neighbors, thus appears to be a major inhibitor of higher levels of cooperative behavior in the prisoner’s dilemma game. Altogether, we find that more elaborate forms of imitation may reveal new mechanisms of promoting the evolution of cooperation in ways that appear to be more closely associated with complex societies, where the strategies alone may carry insufficient information to fully exploit the benefits of imitation.
Acknowledgements.Authors acknowledge support from the Hungarian National Research Fund (grant K-73449), the Bolyai Research Scholarship, the Natural Science Foundation of the Anhui Province of China (grant 11040606M119), and the Slovenian Research Agency (grants Z1-2032 and J1-4055).
- \NameAxelrod R. \BookThe Evolution of Cooperation (Basic Books, New York) 1984.
- \NameHofbauer J. Sigmund K. \BookEvolutionary Games and Population Dynamics (Cambridge University Press, Cambridge, UK) 1998.
- \NameNowak M. A. \BookEvolutionary Dynamics (Harvard University Press, Cambridge, MA) 2006.
- \NameSigmund K. \BookThe Calculus of Selfishness (Princeton University Press, Princeton, MA) 2010.
- \NameSantos F. C., Pacheco J. M. Lenaerts T. \REVIEWProc. Natl. Acad. Sci. USA10320063490.
- \NameHauert C. \REVIEWJ. Theor. Biol.2402006627.
- \NameFu F., Chen X., Liu L. Wang L. \REVIEWPhys. Lett. A371200758.
- \NameGómez-Gardeñes J., Campillo M., Moreno Y. Floría L. M. \REVIEWPhys. Rev. Lett.982007108103.
- \NameRong Z., Wu Z.-X. Wang W.-X. \REVIEWPhys. Rev. E822010026101.
- \NameTomassini M., Luthi L. Pestelacci E. \REVIEWInt. J. Mod. Phys. C1820071173.
- \NameSzolnoki A. Perc M. \REVIEWEPL86200930007.
- \NameCremer J., Reichenbach T. Frey E. \REVIEWNew J. Phys.112009093029.
- \NamePoncela J., Gómez-Gardeñes J., Floría L. M., Moreno Y. Sánchez A. \REVIEWEPL88200938003.
- \NameDai Q., Li H., Cheng H., Li Y. Yang J. \REVIEWNew J. Phys.122010113015.
- \NameHelbing D. Lozano S. \REVIEWPhys. Rev. E812010057102.
- \NameLiu R.-R., Jia C.-X. Wang B.-H. \REVIEWPhysica A38920105719.
- \NameLee S., Holme P. Wu Z.-X. \REVIEWPhys. Rev. Lett.1062011028702.
- \NameVan Segbroeck S., Santos F. C., Lenaerts T. Pacheco J. M. \REVIEWNew J. Phys.32011013007.
- \NameNowak M. A. May R. M. \REVIEWNature3591992826.
- \NameHamilton W. D. \REVIEWJ. Theor. Biol.719641.
- \NameAxelrod R. Hamilton W. D. \REVIEWScience21119811390.
- \NameWilson D. S. \REVIEWAm. Nat.1111977157.
- \NameTraulsen A. Nowak M. A. \REVIEWProc. Natl. Acad. Sci. USA103200610952.
- \NameSzolnoki A. Perc M. \REVIEWNew J. Phys.112009093033.
- \NameNowak M. A. \REVIEWScience31420061560.
- \NameSzabó G. Fáth G. \REVIEWPhys. Rep.446200797.
- \NameSchuster S., Kreft J.-U., Schroeter A. Pfeiffer T. \REVIEWJ. Biol. Phys.3420081.
- \NameRoca C. P., Cuesta J. A. Sánchez A. \REVIEWPhys. Life Rev.62009208.
- \NamePerc M. Szolnoki A. \REVIEWBioSystems992010109.
- \NameHauert C. Doebeli M. \REVIEWNature4282004643.
- \NameGrujić J., Fosco C., Araujo L., Cuesta J. A. Sánchez A. \REVIEWPLoS ONE52010e13749.
- \NameCook R., Bird G., Lünser G., Huck S. Heyes C. \REVIEWProc. R. Soc. B2011.
- \NameSzabó G. Tőke C. \REVIEWPhys. Rev. E58199869.
- \NameAltrock P. M. Traulsen A. \REVIEWNew J. Phys.112009013012.
- \NameOhtsuki H. \REVIEWJ. Theor. Biol.2642010136.
- \NameWu B., Altrock P. M., Wang L. Traulsen A. \REVIEWPhys. Rev. E822010046106.
- \NameNowak M. Sigmund K. \REVIEWActa Appl. Math.201990247.
- \NameSysi-Aho M., Saramäki J., Kertész J. Kaski K. \REVIEWEur. Phys. J. B442005129.
- \NameSzabó G., Szolnoki A., Varga M. Hanusovszky L. \REVIEWPhys. Rev. E802010026110.
- \NameOhtsuki H. Nowak M. A. \REVIEWJ. Theor. Biol.243200686.