A belief-based evolutionarily stable strategy
As an equilibrium refinement of the Nash equilibrium, evolutionarily stable strategy (ESS) is a key concept in evolutionary game theory and has attracted growing interest. An ESS can be either a pure strategy or a mixed strategy. Even though the randomness is allowed in mixed strategy, the selection probability of pure strategy in a mixed strategy may fluctuate due to the impact of many factors. The fluctuation can lead to more uncertainty. In this paper, such uncertainty involved in mixed strategy has been further taken into consideration: a belief strategy is proposed in terms of Dempster-Shafer evidence theory. Furthermore, based on the proposed belief strategy, a belief-based ESS has been developed. The belief strategy and belief-based ESS can reduce to the mixed strategy and mixed ESS, which provide more realistic and powerful tools to describe interactions among agents.
keywords:Evolutionarily stable strategy, Evolutionary game, Mixed strategy, Dempster-Shafer evidence theory, Belief function
definitionDefinition \newdefinitionexampleExample \newdefinitionremarkRemark \newdefinitiontheoremTheorem
Game theory von2007theory (); myerson2013game () provides an effective mathematical framework to explain and study the interactions among individuals. In many situations, the preferences, aims, and goals of participating individuals are potentially in conflict myerson1991game (). A canonical example is the prisoner’s dilemma game poundstone2011prisoner (), which exhibits an apparent social dilemma that human cooperation disappears when there exists a conflict between individual and collective rationality. Due to its significant advantages of depicting the essence underlying many phenomena in nature and society, game theory has been widely used in scientific disciplines from economics, psychology to biology, as well as operational research and political science.
Recently, with the ample introduction of temporal dynamics and spatial topology, traditional theory has been elevated to a new flat: evolutionary game theory lewontin1961evolution (); hamilton1967extraordinary (); smith1973the (); smith1982evolution (); hammerstein1994game (); dugatkin1998game (); masuda_pla03 (); huttegger2013methodology (); nowak2004evolutionary (), which provides a paradigmatic framework to study the evolution of cooperation within population dynamics hofbauer1998evolutionary (); santos2006evolutionary (); axelrod2006evolution (); szabo2007evolutionary (); nowak2006evolutionary (); santos2012dynamics (). Along this research line, the mechanisms of promoting the emergence of cooperative behaviors have been greatly proposed nowak2006five (); zimmermann2005cooperation (); santos2005scale (); gomez2007dynamical (); hauert_jtb06b (); wang2013insight (); ohtsuki2006simple (); QingJinSR20144095 (); sinatra2009ultimatum (); genki_sr (); wang2013impact (); szolnoki2012wisdom (); li2014comprehensive (); wang2014rewarding (). Typical examples include the the mobility of players vainstein_jtb07 (); meloni_pre09 (); xia12 (); jiang_ll_pre10 (); helbing_pnas09 (), heterogeneous activity szolnoki_epl07 (); zhen_plos10 (), spatial structured population roca_prl06 (); LazaroPNAS12 (); traulsen_pre04 (); cao_xb_pa10 (); wu_zx_pre06 (), and coevolutionary selection of dynamical rules jun_pre12 (); perc_bs10 (), to name but a few. In spite of plentiful achievements, a basic conception, evolutionarily stable strategy (ESS), which was first proposed by Smith and Price smith1973the () and further explained in smith1982evolution (); maynard1974theory (), always attracts the firm attention from theoretical and experimental viewpoints taylor1978evolutionary (); taylor2004evolutionary (); nowak2004emergence (); nowak2006evolutionary (); tarnita2009mutation (); shakarian2012review (). An ESS, an equilibrium refinement of the Nash equilibrium nash1950equilibrium (), can be regarded as a solution of one specific game, which is self enforcing and where no player can gain benefit by unilaterally deviating from it. At variance with Nash equilibrium, ESS is evolutionarily stable, and it can be either pure or mixed. Previous study smith1982evolution () demonstrated that a game with two pure strategies always has an ESS, despite it is either a pure ESS or a mixed ESS in a infinite population.
The mixed strategy usually reflects the randomness of strategies. For example, in a given game with two pure strategies, these two pure strategies can be represented by , . In this line, the mixed strategy can be expressed as , where determines the probability of strategy to be selected and takes a value within the interval . However, due to the impact of many factors, the selection probability of pure strategy keeps fluctuating in a range rather than being constant. While this disturbance can be caused by environmental noise, agent’s rationality degree and other factors, which gives rise to more uncertainty than that of mixed strategy.
Aiming to represent the uncertainty, one novel strategy, belief strategy, is proposed in this paper. Interestingly, the belief strategy is based upon the Dempster-Shafer evidence theory Dempster1967 (); Shafer1976 (), which is a tool of expressing uncertainty and exploring questions under the uncertain environment denoeux2004evclus (); denoeux2013maximum (); XinyangDAHP (); Kang2012 (); Deng2013Environment (); yang2013evidential (); DengTOPPER2013 (); yager2013decision (). Then, the proposed belief strategy is used to extend the ESS, namely, the belief strategy is a generalization of mixed strategy and the belief-based ESS is a generalization of mixed ESS. This setup provides more realistic and powerful frameworks to describe interactions among agents. The remainder of this paper is organized as follows. We will first describe the Hawk-Dove game, evolutionarily stable strategy, and Dempster-Shafer evidence theory; subsequently, we will present the proposed belief strategy and belief-based ESS; and finally we will summarize our conclusions.
2.1 Hawk-Dove game and evolutionarily stable strategy (ESS)
Hawk-Dove game smith1973the (); smith1982evolution (); szabo2007evolutionary () is a simple, paradigmatic model to simulate the competition between animals. Assume there is a population of animals, in which each individual aggressiveness is different during the interaction with others. Accordingly, their behaviors can be divided into two types: the aggressive type and the cooperative type. The aggressive type corresponds to strategy “Hawk” (H), the cooperative type is associated with strategy “Dove” (D). Within each interaction, two animals meet and compete for a resource (). When two Hawks meet, they will fight so that both of them have the opportunity to get , where is the cost of injury in the fight. When two Doves meet, they will share the resource, which means each individual obtains . If, however, a Hawk meets a Dove, the former will fight and the latter can only escape. As a result, the Hawk obtains the entire resource without any cost of injury, the Dove is left with nothing. In this sense, The payoff matrix of Hawk-Dove game is shown in Figure 1.
Evolutionarily stable strategy (ESS) is a key concept in evolutionary game theory. According to its definition smith1973the (), in a given environment an ESS is such a strategy that can not be invaded by any other alternative strategy which is initially rare. The condition required by an ESS can be formulated as smith1982evolution (); maynard1974theory ():
for all , where strategy is an ESS, is an alternative strategy, and is the payoff of strategy playing against strategy .
The conditions given in above equations (namely, Eqs.(1) and (2)) are on the basis of these assumptions including infinite population, asexual inheritance, complete mixing, pairwise and symmetric contests. If the evolutionarily stable strategy (ESS) is a pure strategy, is called a pure ESS. On the contrary, once is a mixed strategy, becomes the so-called mixed ESS. In smith1982evolution (), it has been proven that a game with two pure strategies always has an ESS (pure ESS or mixed ESS). Take the Hawk-Dove game as an example. In that game, pure strategy is an ESS if because . Conversely, if , the ESS of Hawk-Dove game is a mixed strategy.
While among the mixed ESS, the Bishop-Canning theorem bishop1978generalized () can provide great help. Herein, a statement given by Smith smith1982evolution () is directly adopted to display the Bishop-Canning theorem.
Bishop-Canning theorem: If is a mixed ESS with support , , , , then
where , , are said to be the “support” of if these pure strategies are played with non-zero probability in the mixed strategy.
Based on this theorem, the mixed ESS of Hawk-Dove game, denoted by , where is the probability choosing strategy , can be expressed as
Extending Eq.(4), we get
Hence, the mixed ESS is . It is easy to verify that the condition displayed in Eq.(2) has been meet in . The mixed strategy is stable against invasion.
2.2 Dempster-Shafer evidence theory
Dempster-Shafer evidence theory Dempster1967 (); Shafer1976 (), also called Dempster-Shafer theory or evidence theory, has been first proposed by Dempster Dempster1967 () and then developed by Shafer Shafer1976 (). This theory needs weaker conditions than the Bayesian theory of probability, so it is often regarded as an extension of the Bayesian theory. As a theory of reasoning under the uncertain environment, Dempster-Shafer theory has an advantage of directly expressing the “uncertainty” by assigning the probability to the subsets of the set composed of multiple objects, rather than to each of the individual objects. The probability assigned to each subset is limited by a lower bound and an upper bound, which respectively measure the total belief and the total plausibility for the objects in the subset. For the simplicity of explanation, a few basic concepts are introduced as follows.
Let be a set of mutually exclusive and collectively exhaustive events, indicated by
where the set is called a frame of discernment. The power set of is indicated by , namely
The elements of or subset of are called propositions. For a frame of discernment , a mass function is a mapping from to , formally defined by:
which satisfies the following condition:
where a mass function is also called a belief function or a basic probability assignment (BPA). The assigned basic probability number measures the belief being exactly assigned to proposition and represents how strongly the evidence supports .
Given a belief function , we can calculate the associated belief measure and plausibility measure, indicated by function and function, respectively. For a proposition , the belief function is defined as
The plausibility function is defined as
where . Obviously, if , and are the lower limit function and upper limit function of the probability to which proposition is supported, respectively. According to Shafer’s explanation Shafer1976 (), the difference between the belief and the plausibility of a proposition expresses the ignorance of the assessment for the proposition . The uncertainty expressed by belief and plausibility is shown in Figure 2.
3 Proposed belief-based evolutionarily stable strategy
In this section, firstly, a belief strategy is proposed based on Dempster-Shafer evidence theory, which extends the concept of mixed strategy. Secondly, in terms of the proposed belief strategy, a belief-based ESS is developed.
3.1 Belief strategy
As above mentioned, in game theory the strategies can be divided two types: (i) pure strategy, such as and in the Hawk-Dove game; (ii) mixed strategy, for instance . As for the mixed strategy, the parameter determines the selection probability of the given pure strategy. However, due to the impact of many factors, such as, environmental noise and individual rationality degree, the selection probability of pure strategy is not constant, but changes with a disturbance. Such disturbance induces the probability fluctuating in an interval . In order to express such uncertainty, in this paper a new strategy type, belief strategy, is proposed based on Dempster-Shafer evidence theory. The definition of belief strategy is given as below.
Definition of belief strategy: Let be the set of all pure strategies in a game, a belief strategy is a mapping from to , formally defined by
If , is called a support of belief strategy . Essentially, a belief strategy can be expressed by one belief function. Take the Hawk-Dove game as an example. Assume there is an individual who adopts strategy with a probability and adopts strategy with a probability , where and . The reminder is indistinguishable so that it is assigned to the mixture of and , namely . So the individual strategy is indicated by
According to the definitions of and functions, the following relation is satisfied,
Hence, in contrast with mixed strategy, the belief strategy shown in Eq.(16) can also represented as
Conceptually, the mixed strategy is a generalization of pure strategy, the belief strategy is a generalization of mixed strategy. If the set of supports of belief strategy only consists of single pure strategy, is reduced to a mixed strategy.
3.2 Belief-based ESS
Based on the above belief strategy, the belief-based ESS can been proposed. Similar to the conditions of classically pure ESS and mixed ESS, a belief strategy can become the belief-based ESS, which is stable against the invasion of alternative strategy , only if
for all , is the expected payoff of strategy playing against strategy . The above conditions are also just suitable for infinite population, pairwise and symmetric contests.
In order to find the belief-based ESS, it is necessary to calculate the lower limit and upper limit of selection probability of each pure strategy, whereat the Bishop-Canning theorem is also used. Take the Hawk-Dove game as an example with . The belief strategy is , . Herein we assume that is uniformly distributed in the interval , and its probability density function is displayed as follows,
Due to and are the supports of , we get,
According to the Bishop-Canning theorem, the following condition is used to find the belief-based ESS :
Formally, the belief-based ESS is shown as below.
where and . Also, the belief-based ESS can represented as the forms of belief function,
Figure 3 features the mixed ESS and belief-based ESS in the Hawk-Dove game. It is explicit that in a game with two pure strategies the mixed ESS is a point, while the belief-based ESS is a segment determined by parameter geometrically. When , the belief-based ESS is totally reduced to the mixed ESS. The parameter is a measure to reflect the uncertainty of belief strategy or belief-based ESS.
In terms of the Bishop-Canning theorem, . In order to further verify the stability of against invasion, the condition given in Eq.(23) is examined.
It can be found , and , when . Hence, these formulas prove that the belief-based ESS is stable against invasion.
In short, we have reviewed the concept of ESS and the uncertainty involved in the mixed strategy. This uncertainty mainly comes from the disturbance of the selection probability of pure strategies. In order to reflect such uncertainty, a belief strategy has been proposed based on Dempster-Shafer evidence theory. The proposed belief strategy is a generalization of mixed strategy. If the set of supports of a belief strategy only consists of single pure strategy, the belief strategy can reduce to a mixed strategy. What’s more, on the basis of the belief strategy, a belief-based ESS is proposed, which, to large extent, extends the mixed ESS. The proposed belief strategy and belief-based ESS can provide more powerful tools to describe complicated interaction among agents.
The work is partially supported by National Natural Science Foundation of China (Grant No. 61174022), Specialized Research Fund for the Doctoral Program of Higher Education (Grant No. 20131102130002), R&D Program of China (2012BAH07B01), National High Technology Research and Development Program of China (863 Program) (Grant No. 2013AA013801), the open funding project of State Key Laboratory of Virtual Reality Technology and Systems, Beihang University (Grant No.BUAA-VR-14KF-02), Fundamental Research Funds for the Central Universities (Grant No. XDJK2014D034).
- (1) J. von Neumann, O. Morgenstern, Theory of Games and Economic Behavior (60th Anniversary Commemorative Edition), Princeton University Press, 2007.
- (2) R. B. Myerson, Game Theory, Harvard University Press, 2013.
- (3) R. B. Myerson, Game Theory: Analysis of Conflict, Harvard University Press, 1991.
- (4) W. Poundstone, Prisoner’s Dilemma, Random House LLC, 2011.
- (5) R. C. Lewontin, Evolution and the theory of games, Journal of Theoretical Biology 1 (3) (1961) 382–403.
- (6) W. D. Hamilton, Extraordinary sex ratios, Science 156 (3774) (1967) 477–488.
- (7) J. M. Smith, G. Price, The logic of animal conflict, Nature 246 (1973) 15–18.
- (8) J. M. Smith, Evolution and the Theory of Games, Cambridge university press, 1982.
- (9) P. Hammerstein, R. Selten, Game theory and evolutionary biology, Handbook of game theory with economic applications 2 (1994) 929–993.
- (10) L. A. Dugatkin, H. K. Reeve, Game Theory and Animal Behavior, Oxford University Press, 1998.
- (11) N. Masuda, K. Aihara, Spatial prisoner’s dilemma optimally played in small-world networks, Physical Review A 313 (2003) 55–61.
- (12) S. M. Huttegger, K. J. Zollman, Methodology in biological game theory, The British Journal for the Philosophy of Science 64 (3) (2013) 637–658.
- (13) M. A. Nowak, K. Sigmund, Evolutionary dynamics of biological games, science 303 (5659) (2004) 793–799.
- (14) J. Hofbauer, K. Sigmund, Evolutionary Games and Population Dynamics, Cambridge University Press, 1998.
- (15) F. C. Santos, J. M. Pacheco, T. Lenaerts, Evolutionary dynamics of social dilemmas in structured heterogeneous populations, Proceedings of the National Academy of Sciences of the United States of America 103 (9) (2006) 3490–3494.
- (16) R. M. Axelrod, The Evolution of Cooperation, Basic books, 2006.
- (17) G. Szabó, G. Fath, Evolutionary games on graphs, Physics Reports 446 (4) (2007) 97–216.
- (18) M. A. Nowak, Evolutionary Dynamics: Exploring the Equations of Life, Harvard University Press, 2006.
- (19) M. D. Santos, F. L. Pinheiro, F. C. Santos, J. M. Pacheco, Dynamics of N-person snowdrift games in structured populations, Journal of Theoretical Biology 315 (2012) 81–86.
- (20) M. A. Nowak, Five rules for the evolution of cooperation, science 314 (5805) (2006) 1560–1563.
- (21) M. G. Zimmermann, V. M. Eguíluz, Cooperation, social networks, and the emergence of leadership in a prisoner’s dilemma with adaptive local interactions, Physical Review E 72 (5) (2005) 056118.
- (22) F. C. Santos, J. M. Pacheco, Scale-free networks provide a unifying framework for the emergence of cooperation, Physical Review Letters 95 (9) (2005) 098104.
- (23) J. Gómez-Gardeñes, M. Campillo, L. Floría, Y. Moreno, Dynamical organization of cooperation in complex topologies, Physical Review Letters 98 (10) (2007) 108103.
- (24) C. Hauert, Spatial effects in social dilemmas, Journal of Theoretical Biology 240 (2006) 627–636.
- (25) Z. Wang, S. Kokubo, J. Tanimoto, E. Fukuda, K. Shigaki, Insight into the so-called spatial reciprocity, Physical Review E 88 (4) (2013) 042145.
- (26) H. Ohtsuki, C. Hauert, E. Lieberman, M. A. Nowak, A simple rule for the evolution of cooperation on graphs and social networks, Nature 441 (7092) (2006) 502–505.
- (27) Q. Jin, L. Wang, C. Xia, Z. Wang, Spontaneous symmetry breaking in interdependent networked game, Scientific Reports 4 (2014) 4095.
- (28) R. Sinatra, J. Iranzo, J. Gomez-Gardenes, L. M. Floria, V. Latora, Y. Moreno, The ultimatum game in complex networks, Journal of Statistical Mechanics: Theory and Experiment 2009 (09) (2009) P09012.
- (29) G. Ichinose, M. Saito, H. Sayama, D. S. Wilson, Adaptive long-range migration promotes cooperation under tempting conditions, Scientific Reports 3 (2013) 2509.
- (30) Z. Wang, C.-Y. Xia, S. Meloni, C.-S. Zhou, Y. Moreno, Impact of social punishment on cooperative behavior in complex networks, Scientific Reports 3 (2013) 3055.
- (31) A. Szolnoki, Z. Wang, M. Perc, Wisdom of groups promotes cooperation in evolutionary social dilemmas, Scientific Reports 2 (2012) 0576.
- (32) Y. Li, X. Lan, X. Deng, R. Sadiq, Y. Deng, Comprehensive consideration of strategy updating promotes cooperation in the prisoner’s dilemma game, Physica A: Statistical Mechanics and its Applications 403 (2014) 284–292.
- (33) Z. Wang, A. Szolnoki, M. Perc, Rewarding evolutionary fitness with links between populations promotes cooperation, Journal of Theoretical Biology 349 (2014) 50–56.
- (34) M. H. Vainstein, A. T. C. Silva, J. J. Arenzon, Does mobility decrease cooperation?, Journal of Theoretical Biology 244 (2007) 722–728.
- (35) S. Meloni, A. Buscarino, L. Fortuna, M. Frasca, J. Gomez-Gardeñes, V. Latora, Y. Moreno, Effects of mobility in a population of prisoner’s dilemma players, Physical Review E 79 (2009) 067101.
- (36) C.-Y. Xia, S. Meloni, Y. Moreno, Effects of environmental knowledge on cooperation and agglomeration in spatial PGG games, Advances in Complex Systems 15 (2012) 1250056.
- (37) L.-L. Jiang, W.-X. Wang, Y.-C. Lai, B.-H. Wang, Role of adaptive migration in promoting cooperation in spatial games, Physical Review E 81 (2010) 036108.
- (38) D. Helbing, W. Yu, The outbreak of cooperation among success-driven individuals under noisy conditions, Proceedings of the National academy of Sciences of the United States of America 106 (2009) 3680–3685.
- (39) A. Szolnoki, G. Szabó, Cooperation enhanced by inhomogeneous activity of teaching for evolutionary prisoner’s dilemma games, EPL 77 (2007) 30004.
- (40) M. Perc, Z. Wang, Heterogeneous aspirations promote cooperation in the prisoner’s dilemma game, PLoS ONE 5 (2010) e15117.
- (41) C. P. Roca, J. A. Cuesta, A. Sánchez, Time scales in evolutionary dynamics, Physical Review Letters 97 (2006) 158701.
- (42) C. Gracia-Lázaro, A. Ferrer, G. Ruiz, A. Tarancón, J. A. Cuesta, A. Sánchez, Y. Moreno, Heterogeneous networks do not promote cooperation when humans play a prisoner’s dilemma, Proceedings of the National Academy of Sciences of the United States of America 109 (32) (2012) 12922–12926.
- (43) A. Traulsen, J. C. Claussen, Similarity based cooperation and spatial segregation, Physical Review E 70 (2004) 046128.
- (44) X.-B. Cao, W.-B. Du, Z.-H. Rong, Evolutionary public goods game on scale-free networks with heterogeneous investment, Physica A 389 (2010) 1273–1280.
- (45) Z.-X. Wu, X.-J. Xu, Z.-G. Huang, S.-J. Wang, Y.-H. Wang, Evolutionary prisoner’s dilemma game with dynamic preferential selection, Physical Review E 74 (2006) 021107.
- (46) J. Tanimoto, M. Brede, A. Yamauchi, Network reciprocity by coexisting learning and teaching strategies, Physical Review E 85 (2012) 032101.
- (47) M. Perc, A. Szolnoki, Coevolutionary games - a mini review, BioSystems 99 (2010) 109–125.
- (48) J. M. Smith, The theory of games and the evolution of animal conflicts, Journal of Theoretical Biology 47 (1) (1974) 209–221.
- (49) P. D. Taylor, L. B. Jonker, Evolutionary stable strategies and game dynamics, Mathematical Biosciences 40 (1) (1978) 145–156.
- (50) C. Taylor, D. Fudenberg, A. Sasaki, M. A. Nowak, Evolutionary game dynamics in finite populations, Bulletin of Mathematical Biology 66 (6) (2004) 1621–1644.
- (51) M. A. Nowak, A. Sasaki, C. Taylor, D. Fudenberg, Emergence of cooperation and evolutionary stability in finite populations, Nature 428 (6983) (2004) 646–650.
- (52) C. E. Tarnita, T. Antal, M. A. Nowak, Mutation-selection equilibrium in games with mixed strategies, Journal of theoretical biology 261 (1) (2009) 50–57.
- (53) P. Shakarian, P. Roos, A. Johnson, A review of evolutionary graph theory with applications to game theory, Biosystems 107 (2) (2012) 66–80.
- (54) J. F. Nash, et al., Equilibrium points in n-person games, Proceedings of The National Academy of Sciences 36 (1) (1950) 48–49.
- (55) A. P. Dempster, Upper and lower probabilities induced by a multivalued mapping, Annals of Mathematics and Statistics 38 (2) (1967) 325–339.
- (56) G. Shafer, A Mathematical Theory of Evidence, Princeton University Press, Princeton, 1976.
- (57) T. Denoeux, M.-H. Masson, EVCLUS: evidential clustering of proximity data, IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 34 (1) (2004) 95–109.
- (58) T. Denoeux, Maximum likelihood estimation from uncertain data in the belief function framework, IEEE Transactions on Knowledge and Data Engineering 25 (1) (2013) 119–130.
- (59) X. Deng, Y. Hu, Y. Deng, S. Mahadevan, Supplier selection using AHP methodology extended by D numbers, Expert Systems with Applications 41 (1) (2014) 156–167.
- (60) B. Kang, Y. Deng, R. Sadiq, S. Mahadevan, Evidential cognitive maps, Knowledge-Based Systems 35 (2012) 77–86.
- (61) X. Deng, Y. Hu, Y. Deng, S. Mahadevan, Environmental impact assessment based on D numbers, Expert Systems with Applications 41 (2) (2014) 635–643.
- (62) J.-B. Yang, D.-L. Xu, Evidential reasoning rule for evidence combination, Artificial Intelligence 205 (2013) 1–29.
- (63) X. Deng, Q. Liu, Y. Hu, Y. Deng, TOPPER: Topology prediction of transmembrane protein based on evidential reasoning, The Scientific World Journal 2013 (2013) Article ID 123731, 8 pages. doi:10.1155/2013/123731.
- (64) R. R. Yager, N. Alajlan, Decision making with ordinal payoffs under Dempster-Shafer type uncertainty, International Journal of Intelligent Systems 28 (11) (2013) 1039–1053.
- (65) D. Bishop, C. Cannings, A generalized war of attrition, Journal of Theoretical Biology 70 (1) (1978) 85–124.