Bio-inspired Evolutionary Game Dynamics on Complex Networks under Uncertain Cross-inhibitory Signals \thanksreffootnoteinfo
Given a large population of players, each player has three possible choices between option 1 or 2 or no option. The two options are equally favorable and the population has to reach consensus on one of the two options quickly and in a distributed way. The more popular an option is, the more likely it is to be chosen by uncommitted players. Uncommitted players can be attracted by those committed to any of the other two options through a cross-inhibitory signal. This model originates in the context of honeybees swarms, and we generalize it to duopolistic competition and opinion dynamics. The contributions of this work include (1) the formulation of an evolutionary game model to explain the behavioral traits of the honeybees, (2) the study of the individuals and collective behavior including equilibrium points and stability, (3) the extension of the results to the case of structured environment via complex network theory, (4) the analysis of the impact of the connectivity on consensus, and (5) the study of absolute stability for the collective system under time-varying and uncertain cross-inhibitory parameter.
First]Leonardo Stella Second]Dario Bauso
Department of Automatic Control and Systems Engineering, University of Sheffield, Mappin St. Sheffield, S1 3JD, UK.
Department of Automatic Control and Systems Engineering, University of Sheffield, Mappin St. Sheffield, S1 3JD, United Kingdom, and Dip. di Ing. Chimica, Gestionale, Informatica, Meccanica, Università di Palermo, 90128 Palermo, IT.
Key words: Game Theory; Consensus; Multi-Agent Systems.
We consider a large population of players who can choose option 1, option 2 or no option (uncommitted state). The two options are equally favorable and the population has to reach consensus on one of the two options quickly and in a distributed way. Players i) benefit from choosing the more popular option, ii) they can recruit uncommitted players, and iii) they can send cross-inhibitory signals to players committed to a different option.
Highlights of contributions. First, we provide an interpretation as game dynamics by modelling the evolution of the frequency of each strategy. We also introduce a new notion of game dynamics, which we call expected gain pairwise comparison, according to which the players change strategy with a probability that is proportional to the expected gain. We also extend the model to duopolistic competition and opinion dynamics. A second contribution is the analysis of stability of the individuals behaviors. Our analysis shows that, if the cross-inhibitory parameter exceeds a threshold, which we calculate explicitly, players reach consensus on one of the two options. Otherwise they distribute uniformly across the two options at the equilibrium. As third contribution, we reframe the above results in the case of structured environment. The structure of the environment is captured by a complex network, with given degree distribution. The nodes are the players and the degree of a node represents its connectivity. This allows us to study the role of heterogeneity. The following is a list of additional results with respect to the conference paper, see Stella and Bauso (2017). First, we provide a convergence analysis as a function of the connectivity. Then, we prove that higher connectivity increases the number of uncommitted players. Last, we prove absolute stability under time-varying uncertain cross-inhibitory parameter.
Related literature. The proposed model originates in the context of a swarm of honeybees, see Britton et al. (2002), and Pais et al. (2013). The analogy with duopolistic competition in marketing has been inspired by Bressan (2010), and the link to opinion dynamics in social networks by Hegselmann and Krause (2002). Although the role of cross-inhibitory parameter was studied in Pais et al. (2013), here we stress a different perspective based on the Lyapunov’s direct method for stability analysis and control design. Evolutionary dynamics in structured environment is discussed in Tan et al. (2014), Piccardi and Casagrandi (2008), Ranjbar-Sahraei et al. (2014). A game perspective in collective decision making is provided in Salhab et al. (2015). Consensus and games are studied in Yin et al. (2012).
This paper is organized as follows. In Section 2, we describe the game. In Section 3, we discuss applications. In Sections 4 and 5, we consider unstructured and structured environments, respectively. In Section 6, we study the asymmetric case. In Section 7, we study absolute stability under uncertain and time-varying cross-inhibitory signal. In Section 8, we provide numerical analysis. In Section 9, we provide conclusions and future works.
2 Game Dynamics
Given a large population of players, each player chooses within a set of three pure strategies. Let us denote the frequency of strategy , namely the portion of the population who has selected that strategy, by , , for . Let be the payoff matrix defined as follows:
The non-zero entries of matrix simulate a coordination game, whereby the row player benefits from matching the column player’s strategy. The row player earns and dollars for matching strategy or , otherwise he loses or if playing strategy or while the column player plays the other strategy. Uncommitted players do not gain nor loose anything in random-matching with opponents. The above matrix models a crowd-seeking scenario where the benefit of choosing a strategy between and depends on the frequency of that strategy. In addition, before choosing strategy or , players must be in an uncommitted state, namely in strategy . The evolution of the frequencies of each strategy is in accordance with the following game dynamics which links to the notion of innovative dynamics as in Hofbauer (2011). Let be the transition rate from to :
The following is the definition of expected gain comparison given for our game dynamics, which constitutes the first contribution of this paper.
(Expected gain comparison) Given a payoff matrix , by changing from strategy to the expected gain pairwise payoff comparison is defined as
where denotes the positive part of , and is an offset.
The above definition models the expected revenue obtained by considering the probability only of a payoff increase and ignoring payoff decreases in correspondence to a unilateral change of strategy.
For the payoff matrix in (2), we then have , , and , where the offset is , , or in each specific case. By substituting the previous equations in dynamics (2) and using the conservation of mass law, for which it holds , the formulation of the system can be reduced to a two-dimensional system as follows:
where we take , and (symmetric case). The above system is obtained in the case of unstructured environment, i.e., it does not consider any interaction topology. Such a system, in the asymmetric case, where the parameters are different for the two options, admits the Markov chain representation displayed in Fig. 1.
In the case of structured environment, let a complex network be given where is the probability distribution of the node degrees. Also let be the portion of the population with connections (class in short) using strategy , and let be the parameter capturing the connections of the players in the network. Furthermore, let be the mean value of , and let be the probability that a link randomly chosen will point to a player using strategy . The counterpart of system (2) for every class is:
We can view the above system as a microscopic model of the players in class parametrized by the macroscopic parameters and .
This section discusses three examples of applications of the game model in (2), namely honeybees swarm, duopolistic competition and opinion dynamics.
Swarm of honeybees. System (2) was first developed in the context of honeybees swarms, see Pais et al. (2013). The swarm has to choose between two nest-boxes. The two options have same value . Scout bees recruit uncommitted bees via a “waggle dance”. The parameters weight the strength of the cross-inhibitory signals. We can interpret as the portion of swarm selecting option , the portion of swarm selecting and the portion of swarm in the uncommitted state . Transitions from option to involve a amount of independent bees that choose spontaneously and a quantity of bees attracted by those who are already in . On the other hand, consider the case where bees move from strategy to : are those that spontaneously abandon their commitment to strategy and takes into account the cross-inhibitory signal sent from bees using option .
Duopolistic competition in marketing. System (2) provides an alternative model of duopolistic competition in marketing, see e.g. Example 9, p. 27 in Bressan (2010). The classical scenario captured by the well-known Lanchester model is as follows. Two manufacturers produce the same product in the same market. The variables represent the market share of the manufacturer at time . The cross-inhibitory signal and the “waggle dance” term describe different advertising efforts, which may enter the problem as parameters or controlled inputs in the analysis or design of the advertising campaign. Thus system (2), likewise the Lanchester model, describes the evolution of the market share. In the case of structured environment, system (2) captures the social influence of the advertisement campaigns of both manufacturers. A stronger cross-inhibitory signal can be used to model the capability of reaching out to a larger number of potential clients.
Opinion dynamics. Consider a population of individuals, each of which can prefer to vote left or right, see Hegselmann and Krause (2002). This is represented by the Markov chain depicted in Fig. 1 where nodes and represent the left and right. The distribution of individuals in each state is subject to transitions from one state to the other. Persuaders who campaign for the left can influence the transitions from the right to the uncommitted state in a similar way honeybees use cross-inhibitory signals. At the same time uncommitted individuals select left or right proportionally to the level of popularity of the two options. In the case of structured environment, system (2) captures the social influence of each individual. In other words, the cross-inhibitory signal is stronger for those individuals who have more connections.
4 Unstructured environment
In this section, we study stability under unstructured environment and symmetric cross-inhibitory parameters.
Given and an initial state , the equilibrium points of game dynamics (2) are:
Case 1. When ,
Case 2. When ,
Case 3. When and ,
Cases and refer to equilibrium points that are symmetric, i.e. we have the same number of individuals committed to option and .
Let , the equilibria converge to and in Case 2 and to in Case 3.
Note that the equilibrium points and correspond to consensus to option 1 and 2 respectively, while means that players are uniformly distributed between the two options. These results will be used in the following sections, when we will consider a time-varying cross-inhibitory signal , which is one of the novelties of this work. The next result establishes local asymptotic stability of the symmetric equilibrium described in Case 1.
Given and an initial state , the symmetric equilibrium point in Case 1 is locally asymptotically stable if and only if
In the special case where and our results are in accordance with the threshold value reported in equation (4) in Pais et al. (2013).
5 Structured environment
In this section we extend to the case of structured environment the results on equilibrium points and stability provided in the previous section. Let us consider game dynamics (2) and analyze the mean-field response obtained for a given class of players assuming that the distribution of the rest of the population is fixed. From , game dynamics (2) becomes
We can rewrite the above system in matrix form and, under the assumption that , we have
From Fig. 2, we can see that the connectivity shifts the eigenvalue further away from the origin (the ones for the first case are labelled above the -axis, while the ones for the second case are below). Thus, higher connectivity speeds up convergence.
Let and an initial state , for class , the equilibrium points are
Furthermore, at the equilibrium, the distribution of uncommitted players increases with connectivity .
The physical interpretation of the above result is that by increasing the connectivity of the network we bring more uncertainty into the collective decision making process. This reflects in an increase of the percentage of uncommitted players at steady-state.
Let us now develop a model combining a macroscopic and microscopic dynamics. By averaging on both sides of (8) using we have the following macroscopic model:
where and .
Given and an initial state , the symmetric equilibrium point in the case of structured environment is locally asymptotically stable if and only if
6 The asymmetric case
In the asymmetric case we consider only the cross-inhibitory signal sent from players in to players in and the spontaneous migration from to and with rate and respectively. The resulting model is
The above dynamics share striking similarities with the susceptible-infected-removed (SIR) model. Actually, , and can be viewed as the percentage of susceptible, infected and recovered agents, respectively. Parameter is the rate at which individuals decay into the recovered class and parameter is the rate at which the infection is spread among the population. The counterpart of (12) in the case of heterogeneous connectivity is
where the coefficients have the known meaning and is a function of time that captures the probability that any given link points to a player in . System (13) can be represented by the Markov chain in Fig. 3.
Furthermore we define function as:
and its first derivative as:
Consider the second derivative of :
The above second-order differential equation corresponds to the following bidimensional first-order system:
The above system shares similarities with a mass-spring-damper, where plays the role of viscous term, while the eigenvalues determine the amplitude of the oscillations. Implication of such a mechanical analogy will be highlighted and discussed further in the section on the numerical analysis.
7 Uncertain cross-inhibitory coefficient
In this section, we show that stability properties are not compromised even if the cross-inhibitory coefficient is uncertain and changes with time within a pre-specified interval. To do this, we first isolate the nonlinearity related to the cross-inhibitory signal in the feedback loop and prove absolute stability using the Kalman-Yakubovich-Popov lemma, see Chapter 10.1 in Khalil (2002). The feedback scheme used in this section is depicted in Fig. 4.
Now, the system described by the following set of equations is considered:
In the following assumption we introduce the sector nonlinerarities.
Let the cross-inhibitory coefficient be in .
We assume, for simplicity, that . Thus, we can write
The linearized version of (18) is
Building on the Kalman-Yakubovich-Popov lemma, absolute stability is linked to strictly positive realness of where and is the transfer function of system (20), yet to be calculated. Before addressing absolute stability, we first investigate conditions under which matrix is Hurwitz. To be Hurwitz, the trace of matrix must be negative, i.e. , and the determinant must be positive, i.e. . For the first condition, we can neglect the multiplier and have
where the equality holds from the condition which implies, in turn, that can be at most 0.5. In the case where is sufficiently small, and can be set sufficiently large to guarantee the condition . For the condition on the determinant, we have
which is satisfied, when , and is still true by choosing a proper in all the other cases. Now, we isolate the nonlinearities in , and we set , where denotes the identity matrix. Let us now obtain the transfer function associated with system (20):
where and . Then, for we obtain
where . We are ready to establish the following result.
We can extend our robustness analysis to the asymmetric system described by the following set of equations:
System (23) is the asymmetric version of (2), when and are negligible. This system admits two equilibrium points, i.e. . As in the previous sections, by applying the Lyapunov linearisation method we study the stability of these equilibrium points. For the equilibrium point and , the Jacobian matrices are given by
For the trace is and the determinant is , which means that is stable. Analogously, for , the trace is and the determinant is , which means that is a saddle. The corresponding bidimensional first-order system is
We denote the matrix of as matrix , the constant vector as , the vector as and the vector as . Here, we denote for the transpose of matrix . For a first approximation, we will not consider vector . The resulting calculations for the transfer function are:
Now, we check whether the transfer function is positive real, to ensure stability of system (23). To be positive real, the following conditions must hold true:
(1) is stable, i.e. no poles Re.
(2) Re, i.e. .
Condition (1) is trivially verified, since the real part of both poles and is equal to or less than zero. By inspection, condition (2) can be easily verified by plotting the imaginary part in the -axis and the real part in the -axis. This is depicted in Fig. 5, where it can be seen that, for a fixed , the condition translates into , which is always verified. Similarly, for a fixed , we have the symmetric case in which , which is always verified. Thus, is positive real and system (24) is absolutely stable.
8 Numerical Simulations
In this section we simulate the system in the case of structured environment, using the Barabási-Albert complex network. We assume that only a few nodes have high connectivity, whereas a large number of nodes have very low connectivity. We use a discretized version of the following power-law distribution, see Moreno et al. (2002):
In the rest of the section, we write to mean that players in class are connected to of the population. The sum of all players of all classes is in accordance with (26), i.e. , for all . The complex network is depicted in Fig. 6.
The asymmetric case. In the asymmetric case the system shares similarities with a mass-spring-damper model, as formulated in (17). We investigate the role of the cross-inhibitory signal parameter . The plot of the population distribution is displayed in Figs. 7-8 for and , respectively. The simulations involve only the population connected to only 5% of the whole population. Such population amounts to 30% of the total. As initial state, the population in state is equal to 10% of the total and in state is equal to the rest 90%. The plots show that a higher value of leads to a higher transient response of the third state component and to a faster response of the first two state components.
Mean-Field Response. We now simulate the mean-field response assuming a constant value , for two classes of players, namely those with connectivity and . As for the initial state, the population is split among the three states as: 60% in state , 20% in both states and . From the plots in Figs. 9-10, it is evident that the class with higher connectivity converges to an equilibrium point with higher values of (the uncommitted state). Theorem 4 justifies this behaviour, i.e. the role of parameter .
Micro-macro model. The last set of simulations involves the micro-macro model in (7) and (10). The classes are identical to the previous set, while as for the initial state, the population is split among the three states as: 70% in state and 30% in state . The plots in Fig. 11 show that at the equilibrium the value of increases with the connectivity, when is constant, namely we have more players in the uncommitted state. Again, this is in accordance to theorem 4.
For a collective decision making process originating in the context of honeybees swarms, we have provided an evolutionary game interpretation and we have studied stability in the case of structured and unstructured environment. Furthermore, we have investigated the role of the connectivity in terms of speed of convergence and characterisation of the equilibrium point. Finally, we have analysed the system in case of uncertain cross-inhibitory signal, which generalizes the constant coefficient used in the previous studies.
- Bressan (2010) Bressan, A. (2010). Noncooperative differential games at http://www.math.psu.edu/bressan/PSPDF/game-lnew.pdf.
- Britton et al. (2002) Britton, N. F., Franks, N. R., Pratt, S. C., Seeley, T. D. (2010). Deciding on a new home: how do honeybees agree? R. Soc. Lond. B Biol. Sci. 269, 1383–1388.
- Hegselmann and Krause (2002) Hegselmann, R. and Krause, U. (2002). Opinion dynamics and bounded confidence models, analysis, and simulations. J. Artificial Soc. Social Simul., 5(3), 1–33.
- Hofbauer (2011) Hofbauer, J. (2011). Deterministic evolutionary game dynamics. In K. Sigmund, editor, Evolutionary Game Dynamics, 61–79. American Math. Soc., RI.
- Khalil (2002) Khalil, H. K. (2002). Nonlinear systems. Prentice Hall, second edition, 2002.
- Moreno et al. (2002) Moreno, Y., Pastor-Satorras, R., and Vespignani, A. (2002). Epidemic outbreaks in complex heterogeneous networks. The European Physical J. B, 26, 521–529.
- Pais et al. (2013) Pais, D., Hogan, P. M., Schlegel, T., Franks, N. R., Leonard, N. E., Marshall, J. A. R. (2013). A Mechanism for Value-Sensitive Decision-Making. PLoS ONE, 8(9): e73216. doi:10.1371/journal.pone.0073216.
- Piccardi and Casagrandi (2008) Piccardi, C., Casagrandi, R. (2008). Inefficient epidemic spreading in scale-free networks. Phys. Rev. E 77, 026113, 2008.
- Ranjbar-Sahraei et al. (2014) Ranjbar-Sahraei, B., Bloembergen, D., Ammar, H. B., Tuyls, K., and Weiss, G. (2014). Effects of Evolution on the Emergence of Scale Free Networks. Proc. of the 14th International Conf. on the Synthesis and Simulation of Living Systems, ALIFE 14, 14(4), 36–50.
- Salhab et al. (2015) Salhab, R., Malhame, R.P., Le Ny, J. (2015). A dynamic game model of collective choice in multi-agent systems. 54th IEEE Conference on Decision and Control (CDC 2015), 4444–4449..
- Stella and Bauso (2017) Stella, L., Bauso, D. (2017). Evolutionary Game Dynamics for Collective Decision Making in Structured and Unstructured Environments. Proc. of the 20th IFAC 2017 World Congress, 9-14 July, Toulouse, France.
- Tan et al. (2014) Tan, S., Lü, J., Chen, G., Hill, D. J. (2014). When Structure Meets Function in Evolutionary Dynamics on Complex Networks. IEEE Circuits and Systems Magazine, 14(4): 36–50, 2014.
- Yin et al. (2012) Yin, H., Mehta, P. G., Meyn, S. P., Shanbhag U. V. (2012). Synchronization of Coupled Oscillators is a Game. IEEE Transactions on Automatic Control, 57(4). 920–935.
Proof of Theorem 1. To study equilibrium points, we first impose and obtain , which leads to two solutions: and , both studied in the first two cases, while the third case analyses the scenario in which both hold true.
[Case 1] When , the equilibrium point is the root of a second degree polynomial, . From and , we have the following equilibrium point, the roots of the above polynomial are given by:
[Case 2] When , setting and replacing in (2), we obtain , which in turn implies From the roots of the above polynomial, the two equilibrium points are:
[Case 3] A special case is when and . . Since , this leads to: which is equivalent to: From the previous equations, we obtain the following value for :
Thus we have the equilibrium point
To analyse the stability of system (29), we compute the Jacobian matrix around an equilibrium point, i.e. , as
for which we have a saddle point when the following condition for the determinant of the Jacobian holds: The latter is true when which in turn implies The latter yields . From considering for the equilibrium in Case 3, it follows
This concludes our proof.
Proof of Theorem 3.
The determinant of the above matrix is always positive. To see this, note that
Also, the trace of the above matrix is negative, i.e., and therefore the system is asymptotically stable. From we can conclude that the equilibrium point is an asymptotically stable node.
As for the speed of convergence, let us focus on the eigenvalues of the Jacobian. To this purpose, let us calculate the determinant which is given by
Then, Thus, the eigenvalues of the Jacobian matrix are In the two extreme case of no connectivity and full connectivity .
Proof of Theorem 4. We can compute the following
Again, when considering the above two cases we get
Therefore, we can also say that higher connectivity increases the number of players in the uncommitted state.
Proof of Theorem 5. To compute the equilibrium, let us set and obtain:
To inspect the existence of saddle points we need to study conditions under which the determinant of the Jacobian is less than 0. Then, we take and impose that the right-hand side is greater than the left-hand side
By taking the square root on both sides, since the left-hand side is strictly negative, we have , and after some basic algebra, we get (11).
Proof of theorem 6. Let us first prove that is strictly positive real. Thus, we study the properties of matrix , specifically the positive realness. To be strictly positive real, the following conditions must hold true:
is Hurwitz, i.e poles of all elements of have negative real parts;
First, we prove that is Hurwitz. Thus, all the poles must be negative, i.e. . which holds true, after considering the discussion on the trace of matrix as a direct consequence. Now, we check the second condition. It follows that
and . Thus, the second condition can be rewritten as