Emergence of Cooperation in Nonscalefree Networks
Abstract
Evolutionary game theory is one of the key paradigms behind many scientific disciplines from science to engineering. Previous studies proposed a strategy updating mechanism, which successfully demonstrated that the scalefree network can provide a framework for the emergence of cooperation. Instead, individuals in random graphs and smallworld networks do not favor cooperation under this updating rule. However, a recent empirical result shows the heterogeneous networks do not promote cooperation when humans play a Prisoner’s Dilemma. In this paper, we propose a strategy updating rule with payoff memory. We observe that the random graphs and smallworld networks can provide even better frameworks for cooperation than the scalefree networks in this scenario. Our observations suggest that the degree heterogeneity may be neither a sufficient condition nor a necessary condition for the widespread cooperation in complex networks. Also, the topological structures are not sufficed to determine the level of cooperation in complex networks.
I Introduction
The longterm nature of competition leads to a confusing outcome that an agent’s payoff does not depend on a decision in one round but on a reasonable updating rule GTE (); ETG (); BAMS40479 (). Thus in evolutionary game theory, the updating rule has played a key role, for instance, the well known TitforTat TEC (), Pavlovian strategies TD2647 (), WinStayLoseShift Nature36456 (); TD35107 (), Stochastic reactive strategies GEB11364 (), among others. Borrowing the concept of complex networks RMP7447 (); AP511079 (), people realized that abstracting interpersonal relationships into social networks SIAMR45167 (); PR424175 () may be an effective way to understand games in real human society.
To understand the pattern of games on social networks, a variety of network models were then intensively investigated PR44697 (), such as smallworld networks PRL95098104 (); PRE72056128 () and scalefree networks PRL95098104 (); PRL98108103 (). In these networks, the group of opponents surrounding an individual is interpreted as its neighbors. The limited local interactions are the interactions restricted among the individual and their neighbors. As a successful attempt, the updating rule proposed by Santos and Pacheco PRL95098104 (); PNAS1033490 (); JEB19726 (); NATURE454213 () stood out. Simulation results PRL95098104 () show that the appearance of connected hubs in scalefree networks promotes the emergence of largescale cooperation. This conclusion is striking in that it helps researchers find an origin of cooperative behavior in social networks. Also, it indicates that the interconnected hubs are usually occupied by cooperators, when selection favors cooperation. In Santos and Pacheco’s updating rule, an individual randomly picks a neighbor as its reference. If the neighbor’s payoff is higher than the individual, he/she adopt the neighbor’s strategy with a certain probability. Given its feature of promoting cooperation in the networks with a powerlaw degree distribution PRL95098104 (); PNAS1033490 (); JEB19726 (), the rule is extensively employed in the following works PRE72056128 (); PRL98108103 (); PNAS1033490 (); JEB19726 (); NATURE454213 (); EPL7730004 (). Interestingly, their conclusion that degreeheterogeneity promotes the level of cooperation was then questioned theoretically in the public goods game NJP13123027 (). This difference indicates that the game model is a nontrivial factor as well. Indeed, the influence from updating rules or dynamics may be more influential PLR6208 (); BS10282 (); BS10385 (). Next, an empirical work reported by C. GraciaLázaro et al directly provides a realistic example PNAS10912922 ().
In this paper, we discuss an evolutionary game with a pure rational agentbased updating mechanism. In this game, we consider a local deterministic nature selection. An individual always adopts a local betterperforming strategy in the next round of game. Briefly, in a group including an individual and their neighbors, if the cooperative individuals can get a higher average payoff than the defective ones, the central individual will be a cooperator in the next round and vice versa. As in the wellmixed populations GTE (); EGPD (), for all the games investigated in this paper, defection dominates the population when the temptation to defect is powerful. However, for our updating rule, payoff memory PRE88032127 () can remarkably promote the levels of cooperation in the Watts and Strogatz’s smallworld network (WS) NATURE393440 () and random graphs, which clearly exceed that in the Barabási and Albert’s scalefree network (BA) SCI286509 (). Here, the payoff memory denotes the number of rounds during which the payoff of an individual is aggregated. Our observation indicates that degree heterogeneity may be neither a sufficient condition nor a necessary condition for the widespread cooperation in complex networks. In this respect, our conclusion is consistent with the recent empirical result PNAS10912922 ().
To stay consistent with the previous studies, we adopt the Prisoner’s Dilemma (PD) as the game model. As a heuristic framework, the Prisoner’s Dilemma describes a commonly identified paradigm in many realworld situations EC (); JCR (); GCEC (); NATURE398441 (). It has been widely studied as a standard model for the confrontation between cooperative and selfish behaviors. The selfish behavior here is manifested by a defective strategy, aspiring to obtain the greatest benefit from the interaction with others. This PD game model ETG (); GTE () considers two prisoners who are placed in separate cells. Each prisoner must decide to confess (defect) or keep silence (cooperate). A prisoner may receive one of the following four different payoffs depending on both its own strategy and the other prisoner’s strategy.

(temptation to defect) for defecting a cooperator.

(reward for mutual cooperation) for cooperating with a cooperator.

(punishment for mutual defection) for defecting a defector.

(sucker’s payoff) for cooperating with a defector.
Normally, the four payoff values are defined as: . At the next round, the prisoner will know the strategy of the other one in the previous round. It can then adjust its strategy according to the game updating rule. For the case that two individuals just play the prisoner’s dilemma for one round, there is a safe strategy, i.e., a prisoner always gets a payoff not less than his/her opponent if s/he defects. ’s strategy is denoted by . takes vectors and for the cooperative and defective strategy respectively. For convenience, and are denoted by and hereafter. In one round of game playing with the other prisoner , payoff can be rewritten as
(1) 
Ii Updating Rule
In our updating rule, the PD is repeated in the following way: in the first stage, a PD is played by every pair of individuals connected by a link in the network. For an individual in the round, their payoff reads as
(2) 
where is an entry of the adjacency matrix of networks, taking values (, , , ) whenever the individual and are connected and otherwise. Each individual then updates their accumulated payoff, which is the sum of payoffs they receive from the last rounds in memory. The sum of payoffs for all the cooperative neighbors (defectors) of in the round is denoted by () as shown in Fig. 1. We define as the span of the payoff memory. For , keeps aggregating for rounds. At the beginning of the th round, the whole payoff system is reset. The purpose of introducing the payoff memory into the model is to simulate a points system of accumulating the players’ payoffs. For example, a season in the English Premier League normally lasts rounds, in which denotes the memory span. In the second stage, each individual updates their strategy based on their payoff and all their neighbors’ payoffs. Consider a group of individuals including an individual and all ’s neighbors, if the cooperative individuals in this group have a higher average payoff than the defective ones, will be a cooperator in the next round, and vice versa. If the cooperative individuals’ average payoff equals the defective ones’ average payoff, will keep its strategy unchanged. For the cooperative neighbors at the round, the sum of their payoffs reads as:
(3) 
where and . The parameter . The function returns the largest integer lower than . For the defective neighbors, the sum of their payoffs reads as:
(4) 
Thus the average payoffs of the cooperative and defective neighbors are
(5) 
and
(6) 
respectively.
For , . For , we derive
(7) 
where
(8) 
The analytical process can be applied to other games by changing the payoff matrix . One can find out that the evolution process is deterministic after setting the initial roles.
Iii Gaming on Social Networks
We consider four classes of networks, the WS smallworld network NATURE393440 (), BA scalefree network SCI286509 (), regular and random graph. We generate ten WS networks, BA networks, regular graphs, and random graphs by random seeds. The regular graphs are formed by a number of individuals with an identical degree . The WS networks and random graphs are generated by randomly rewiring and of the links, respectively. Each network has finite individuals and links. The BA networks are grown by attaching new individuals to existing individuals.
For the PD, the payoff parameters are set as , , and , where represents the temptation to defect PRL95098104 (). The larger is, the more favorable defection becomes. We denote as the frequency of cooperators after reaching a network gaming equilibrium. We run time steps for each simulation, in which steps to guarantee that the system reaches a dynamical equilibrium. Next, we measure and average from to steps, in which the standard deviation of is less than . For a given network, we initially assign a fraction of individuals as cooperators at random, and the remaining individuals as defectors. The influences of the initial frequency of cooperators are shown in Fig. 2.
Fig. 2 shows that the initial frequencies of cooperators have a weak connection with , which is the average frequency of cooperators in gaming equilibrium. For different initial frequencies of cooperators, after a period of initial turbulence, the system can always reach a dynamical equilibrium, where the number of cooperators (or defectors) is stabilized at the particular value with minimum fluctuation. Thus we set the initial frequency of cooperators to 0.5 in all the following simulations. In Fig. 2(c)(d), one can observe the WS small world networks and random graphs are clearly proper platforms of cooperation.
Fig. 3 shows as a function of the payoff memory span, , for the four types of networks. The fraction of cooperators in a gaming equilibrium is determined by the network topology, game model, updating rule, payoff memory span and game parameters. The five factors govern the level of collaboration together. For , panel I(c) and II(c) show that is close to 0 in the BA scalefree networks and regular graphs. This observation indicates that collaboration is highly restrained in the system governed by our updating rule, even when the payoff memory is large. Instead, can keep a relatively high level in the WS networks (see panel III(c)) and random graphs (see panel IV(c)) when the payoff memory is large. For the Santos and Pacheco’s updating rule and the Nowak and May’s updating rule, just few cooperators can survive in the regular and random graphs with a large payoff memory. The observations indicate that our model is a counter example of the previous conclusion PRL98108103 () that the scalefree networks provide a uniform platform of cooperation. Again, the behaviors observed in the WS smallworld networks and random graphs indicate that even in a network without clear degree heterogeneity, cooperators still can survive in it.
Although we change both the updating rule and payoff memory to show the observations, these behaviors actually originate from the updating rule, since the payoff memory promotes cooperation in the social networks governed by the updating rules mentioned in this paper. For the Santos and Pacheco’s updating rule with the payoff memory, the BA scalefree network is still the best platform of cooperation compared with the other topologies mentioned in this paper. In the BA scalefree network governed by our updating rule, the attraction of two extreme states and is much stronger than that in the WS smallworld network and random graph. When the temptation to defect is powerful (), the basin of attraction of full cooperation is smaller than full defection. However, in the WS smallworld network and random graph, the attraction of full cooperation and defection does not exist. Thus the level of cooperation in the BA scalefree network is clearly lower than that in the WS smallworld network and random graph. When the temptation to defect is not so large, for example , the basin of attraction of full defection is not so large any more. Fig. 4 shows that the payoff memory dramatically promotes the level of cooperation in the BA networks for . In this case, the BA networks are still the proper platform of cooperation. For and , the basin of attraction of full defection is large enough to restrain the level of cooperation in the BA networks. Thus the valid range of for our observations in Fig. 3 is roughly .
By comparing the BA network with the uncorrelated network (configuration model RSA6161 ()), previous studies PRL95098104 () indicated that hubs are preferred by the cooperators. Interconnected hubs protect cooperation effectively PRL95098104 (). These two features whereas don’t exist in our system. Fig. 5 shows the distributions of cooperators and defectors for BA networks, WS smallworld networks, and random graphs, respectively. For these three classes of networks, one can observe that the distributions of cooperators are similar to the degree distributions of the networks. These observations indicate that cooperators do not preferentially occupy the hubs any more. They are evenly distributed.
Iv Discussion
As shown in Fig. 3 I(b), Santos and Pacheco’s updating rule PRL95098104 () dramatically promotes the level of cooperation in the BA scalefree networks, while it restrains cooperation in the other networks such as the WS smallworld networks and random graphs. Based on an intuitive local strategy optimization, we introduce a new strategy updating rule to check whether the network structures are sufficient to determine the level of cooperation. Through extensive numerical simulations, we observe the network structure is just one of the key factors. In a proper condition, the level of cooperation in the WS smallworld networks and random graphs can also exceed that in the BA scalefree networks.
To create such a proper condition, we consider the payoff memory in this paper. Actually, this parameter is not strange to most of us. In the previous game models, the payoff memory span used to be set to , i.e., an individual’s payoff in the current round will be cleared up after the strategy updating. In this condition, cooperation is highly restrained, even under our updating rule. With a small enhancement of the memory span, the evolutionary selection starts favoring cooperation. In term of this behavior, we presented a general explanation in our previous work PRE88032127 (). Admittedly, the influence of the payoff memory is limited. Nevertheless, it is sufficed to enable the random graphs and smallworld networks to be proper platforms for the emergence of cooperation. We believe this example may shed some lights on the origins of the widespread altruistic behaviors in the nonscalefree networks.
V Conclusion
In summary, we have discussed a question whether the degree heterogeneity promotes the level of cooperation for the Prisoner’s Dilemma in complex networks. Previous studies showed that the degree heterogeneity promotes the level of cooperation in the system governed by a particular strategy updating rule. A recent empirical study questions this conclusion in the case that humans play the Prisoner’s Dilemma. In this paper, we have proposed a strategy updating mechanism with payoff memory. Our simulation results show that the frequency of cooperators in the random graphs and WS smallworld networks exceeds that in the BA scalefree networks apparently, especially when the temptation to defect is powerful. Our observations indicate that the degree heterogeneity is neither a sufficient condition nor a necessary condition for the emergence of cooperation. In another word, the scalefree networks do not always promote cooperation in complex networks. In this respect, our results confirm the empirical result mentioned above. In a particular condition, cooperation can be widespread in any topological structure. Our observations may provide a better understanding of the widespread cooperation in the networks without degree heterogeneity.
We wish to thank Martin Parsley and Wenxu Wang for their help with the manuscript. Y. Z. and M. A. A.A. and C. B. are supported by the region Haute Normandie, France, and the ERDF RISC. S. Z. is supported by the UK Royal Academy of Engineering and the Engineering and Physical Sciences Research Council (EPSRC) under Grant No. 10216/70.
References
References
 (1) Gintis H 2000 Game theory evolving (princeton University, Princeton, NJ).
 (2) Maynard Smith J 1982 Evolution and the theory of games (Cambridge University Press, Cambridge, England).
 (3) Hofbauer J and Sigmund K 2003 Evolutionary game dynamics Bull. Amer. Math. Soc. (N.S.) 40 479519.
 (4) Axelrod R 1984 The evolution of cooperation (Basic Books, New York).
 (5) Kraines D and Kraines V 1989 Pavlov and the prisoner’s dilemma Theor. Decis. 26 4779.
 (6) Nowak M A and Sigmund K 1993 A strategy of winstay, loseshift that outperforms titfortat in the prisoner’s dilemma game Nature 364 5658.
 (7) Kraines D and Kraines V 1993 Learning to cooperate with pavlov an adaptive strategy for the iterated prisoner¡¯s dilemma with noise Theor. Decis. 35 107150.
 (8) Nowak M A and Sigmund K 1995 Invasion dynamics of the finitely repeated prisoner’s dilemma Games Econ. Behav. 11 364390.
 (9) Albert R and Barabási A L 2002 Statistical mechanics of complex networks Rev. Mod. Phys. 74 4797.
 (10) Dorogvtsev S N and Mendes J F F 2002 Evolution of networks Adv. Phys. 51 10791187.
 (11) Newman M E J 2003 The structure and function of complex networks SIAM Rev. 45 167256.
 (12) Boccaletti S, Latora V, Moreno Y, Chavez M and Hwanga D U 2006 Complex networks: structure and dynamics Phys. Rep. 424 175308.
 (13) Szabóa G and Fáth G 2007 Evolutionary games on graphs Physics Reports 446 97216.
 (14) Santos F C and Pacheco J M 2005 Scalefree networks provide a unifying framework for the emergence of cooperation Phys. Rev. Lett. 95 098104.
 (15) Santos F C, Rodrigues J F and Pacheco J M 2005 Epidemic spreading and cooperation dynamics on homogeneous smallworld networks Phys. Rev. E 72 056128.
 (16) GómezGardeñes J, Campillo M, Floría L M and Moreno T 2007 Dynamical organization of cooperation in complex topologies Phys. Rev. Lett. 98 108103.
 (17) Santos F C, Pacheco J M and Lenaerts T 2006 Evolutionary dynamics of social dilemmas in structured heterogeneous populations Proc. Natl. Acad. Sci. U.S.A. 103 34903494.
 (18) Santos F C and Pacheco J M 2006 A new route to the evolution of cooperation J. Evol. Biol. 19 726733.
 (19) Santos F C, Santos M D and Pacheco J M 2008 Social diversity promotes the emergence of cooperation in public goods games Nature 454 213216.
 (20) Szolnoki A and Szabó G 2007 Cooperation enhanced by inhomogeneous activity of teaching for evolutionary prisoner’s dilemma games Europhys. Lett. 77 30004.
 (21) Matjaz P 2011 Does strong heterogeneity promote cooperation by group interactions? New J. Phys. 13 123027.
 (22) Roca C P, Cuesta J A, and Sánchez A (2009) Evolutionary game theory: Temporal and spatial effects beyond replicator dynamics Physics of Life Reviews, 6 208249.
 (23) Yamauchi A, Tanimoto J, and Hagishima A (2010). What controls network reciprocity in the Prisoner’s Dilemma game? BioSystems, 102 8287.
 (24) Yamauchi A, Tanimoto J, and Hagishima A (2011). An analysis of network reciprocity in Prisoner’s Dilemma games using Full Factorial Designs of Experiment BioSystems, 103 8592.
 (25) GraciaLázaro C, Ferrer A, Ruiza G, Tarancón A, Cuesta J A, Sánchez A and Moreno Y 2012 Heterogeneous networks do not promote cooperation when humans play a Prisoner’s Dilemma Proc. Natl. Acad. Sci. USA 109 1284612847.
 (26) Hofbauer J and Sigmund K 1998 Evolutionary Games and Population Dynamics (Cambridge University Press, Cambridge, England).
 (27) Zhang Y, AzizAlaoui M A, Bertelle C, Zhou S and Wang W 2013 Fencesitters protect cooperation in complex networks Phys. Rev. E 88 032127.
 (28) Watts D J and Strogatz S H 1998 Collective dynamics of ¡®smallworld¡¯ networks Nature (London) 393 440442.
 (29) Barabási A L and Albert R 1999 Emergence of scaling in random networks Science 286 509512.
 (30) Axelrod R 1984 The Evolution of Cooperation (Basic Books, New York).
 (31) Axelrod R 1980 More Effective Choices in the Prisoner’s Dilemma J. Conflict Resolut. 24 379403.
 (32) Hammerstein P 2003 Genetic and Cultural Evolution of Cooperation (MIT, Cambridge, MA).
 (33) Turner P E and Chao L 1999 Prisoner’s dilemma in an RNA virus Nature (London) 398 441443.
 (34) Nowak M A and May R M 1992 Evolutionary game and spatial chaos Nature 359 826829.
 (35) Molloy M and Reed B 1995 A critical point for random graphs with a given degree sequence Random Struct. Algorithms 6 161180.BioSystems, 103(1), 8592.