Imitation, internal absorption and the reversal of local drift in stochastic evolutionary games

Imitation, internal absorption and the reversal of local drift in stochastic evolutionary games

Tobias Galla tobias.galla@manchester.ac.uk Theoretical Physics, School of Physics and Astronomy, The University of Manchester, Manchester M13 9PL, United Kingdom
July 6, 2019
Abstract

Evolutionary game dynamics in finite populations is typically subject to noise, inducing effects which are not present in deterministic systems, including fixation and extinction. In the first part of this paper we investigate the phenomenon of drift reversal in finite populations, taking into account that drift is a local quantity in strategy space. Secondly, we study a simple imitation dynamics, and show that it can lead to fixation at internal mixed-strategy fixed points even in finite populations. Imitation in infinite populations is adequately described by conventional replicator dynamics, and these equations are known to have internal fixed points. Internal absorption in finite populations on the other hand is a novel dynamic phenomenon. Due to an outward drift in finite populations this type of dynamic arrest is not found in other commonly studied microscopic dynamics, not even in those with the same deterministic replicator limit as imitation.

evolving populations, game theory, imitation, fixation and extinction, drift reversal
preprint:

I Introduction

Much of the current interest in dynamic phenomena in the frequency-dependent selection in population focuses on the differences between stochastic descriptions of evolutionary processes and their deterministic counterparts. Traditionally deterministic formulations have been more common in the biological literature, presumably due to the fact that the theory of non-linear deterministic differential equations is relatively well developed. Aspects of stochastic dynamics that have been studied in recent years include (i) probabilities with which mutants invade existing populations nowakbook (), (ii) fixation probabilities and time-to-fixation in finite populations nowakbook (); altrockinger (), (iii) the differences between dynamical stationary states of deterministic and stochastic processes traulsen (); traulsenreview (); mobilia () and (iv) the dynamics of pattern formation induced by noise frey ().

In the present paper we will focus on two dynamic features of stochastic evolutionary dynamics, namely those of (a) drift reversal and (b) fixation to internal points in strategy space. Drift reversal here refers to a phenomenon reported in traulsen2 (); claussen2 (); claussen1 (), and characterises the observation that some stochastic evolutionary systems tend to approach the boundaries of the strategy space at small population sizes, whereas a drift in the other direction, towards the centre of strategy space, occurs at larger sizes. In particular expressions for a critical population size, at which this reversal of drift occurs, were derived in traulsen2 (); claussen2 (); claussen1 (). It is important to stress that the meaning of the word ‘drift’ is here different from its usual use for example in the context of ‘random drift’, ‘genetic drift’ or ‘neutral drift’ kimura (); kimura2 (); hubbell (). Instead it measures a tendency of the stochastic system to approach an internal fixed point, or to approach the boundaries of strategy space respectively, this will become clear below, when we give a more precise mathematical characterisation.

Secondly we analyse the dynamics of a strict proportional imitation process in finite populations, the details of this dynamics will be described below. Game dynamical processes commonly described in the literature are typically of the following type: in a first step two individuals are drawn at random from the population. Secondly, if the two individuals are of distinct types, one individual adopts the strategy of the other with a certain rate, typically dependent on the fitnesses of the two individuals, and in some cases also on the mean fitness of all individuals in the population. This adaptation step is equivalent to the death of one individual, and the birth of another. This type of dynamics leads to two distinct sources of stochasticity, one coming from the selection of the two individuals, and a second from the randomness of the birth and death processes that occur subsequently. In traulsendet () an interesting limiting case was introduced and studied. Both types of randomness were eliminated, leading to an entirely deterministic process even in finite populations. Here we take a different route, and make the birth-death component of the update deterministic in that only transition towards strictly more successful strategies are allowed. The randomness in the selection of individuals for potential update remains however, and consequently the overall dynamics is still stochastic, but as we will see novel phenomena can occur, in particular the possibility of an absorbing fixed point in the interior of the strategy space.

The paper is organised as follows: In Sec. II we will introduce the required definitions and the general notation. In Sec. III we then analyse the phenomenon of drift reversal in detail, and provide a refinement of the findings of traulsen2 (); claussen2 (); claussen1 (). These are derived for the one-dimensional case of symmetric games with two strategies, but can be generalised to higher-dimensional dynamics. In Sec. IV we then introduce the strict imitation dynamics, and show how it leads to internal absorption in the context of stochastic one-dimensional dynamics. Sections V and VI and then focus on similar dynamics for cyclic games with three strategies per player and for two-population games. In the final section we summarise our findings.

Ii General definitions

ii.1 Stochastic dynamics in finite populations

We will consider populations of agents, and restrict the analysis to processes conserving the size of the population. Each individual can be of one of types (the different species or pure strategies), so that the state of the system is characterised by the vector , where indicates the number of individuals of type . One always has . For one-population models we will label the payoff matrix elements of the underlying game as , where . The quantity thus denotes the payoff a player of type receives in interaction with a player of type . Frequency-dependent fitnesses are then given by

(1)

where we have excluded interaction of one individual with itself. To specify a population dynamical process it is then sufficient to define the ‘conversion’ rates , corresponding to events in which a player of type is replaced by one of type . We will limit the discussion to processes of the general form

(2)

where and . The model parameter here denotes the strength of selection and takes values . The are often referred to as reproductive fitnesses. In this paper we will mostly consider pairwise comparison processes, and the precise choice of will often be unimportant, amounting essentially to choosing a time scale. The form (2) is found by, at each time step, selecting two players from the population (with replacement), one for potential reproduction and one for potential removal. The player selected for potential removal is assumed to be of type , the one for potential reproduction of type . For a given pair of selected players reproduction and death actually only occur at a rate proportional to . The time evolution of the probability density of finding the system in state at time is then given by

(3)

where is a shift operator, acting on functions of by increasing by one: . is the inverse operator, reducing by one. It is here important to note that the system only has degrees of freedom, given that . For reasons of notational compactness the above master equation (3) has been formulated for the full system . This equation will be the starting point for our further investigations of stochastic game dynamics in finite populations.

ii.2 Deterministic replicator dynamics

A description of evolutionary processes for infinite populations, , is given by the following replicator dynamics

(4)

Here is the fraction of players of type in the limiting infinite population, and the average fitness in the population

(5)

It is however important to note that the above replicator equation provides an accurate description of the dynamics in the limit only for a suitable choice of the microscopic dynamics, in particular of the functions defined above. This will be explained in more detail below, see also traulsen2 (); bladon () for further discussion. Throughout this paper the superscript will indicate that we are referring to quantities defined for infinite populations. These quantities will typically be functions of , and can be obtained by a simple replacement of by , in the corresponding quantities for finite populations. For example we have .

Iii Reversal of local drift in symmetric games

iii.1 Definitions

We will first consider the example of a two-player symmetric game with two strategies, which we will label . In particular we will address games for which the deterministic replicator dynamics has a stable internal fixed point. Specifically we will choose the so-called Hawk-Dove game, also referred to as the game of chicken, defined by the payoff matrix nowakbook ()

(6)

with model parameters , to be specified below. Here and the state of the population is fully specified by the number of individuals carrying strategy . If the population consists of players of type , and players of type one then has

(7)

The number of individuals of type at which the fitnesses of the two species are equal, is given

(8)

if this expression is integer-valued (otherwise there is not such point). We will in the following specifically choose and , so that . This expression is integer-valued whenever the total number of individuals in the population, , is even, and we will restrict the discussion to such cases in the following.

In order to discuss the phenomenon of local drift reversal we will focus on what is referred to as the so-called ‘local’ update process in the literature traulsen2 (); claussen2 (); claussen1 (). This process is defined by the following transition rates

(9)
(10)

In this two-strategy case we write as a short-hand for and for the transition rate in the opposite direction. is an additional model parameter, introduced to ensure transition rates remain positive. Specifically this model parameter denotes the maximally possible difference in fitness between the fittest and the least fit species traulsen2 (). Given the choices we will here use .

The deterministic dynamics corresponding to the local update rule, valid in the limit of infinite populations, can be obtained from these definitions for example from the lowest order terms of an expansion of the master equation (3) in the square root of the inverse system size. We will not present the mathematical details here, as these have been discussed at length in the literature bladon (); traulsen2 (); kampen (). The lowest-order result is obtained as

(11)

where is the fraction of players of type in the limit of infinite populations. It is obvious that this equation admits an internal fixed point at in addition to the trivial fixed points and . Up to a re-scaling of time equation (11) is a specific example of a replicator equation (4), as some elementary algebra shows.

iii.2 Inversion of local drift

The notion of drift in traulsen2 (); claussen2 (); claussen1 () refers to the tendency of the stochastic dynamics to evolve towards an internal fixed point, or away from it. More formally, given a measure of distance of the one-dimensional Hawk-Dove system from the fixed point, , we will ask whether this distance is more likely to increase than to decrease if the system is started at a certain point in phase space, or whether there is a higher probability for a reduction in distance than there is for an increase of distance. may here for example be the Euclidean distance from the internal fixed point of the deterministic dynamics, but for purposes of general considerations we will leave unspecified until a later point. Recall that following our above remarks we will use the notation to refer to the case of infinite populations, and for finite.

Figure 1: (Color online) Local drift in the Hawk-Dove game. Lines are from theory ( from top to bottom at the maximum), symbols from simulations (averaged over independent runs). Inset is a magnification of the central peak, where an outward drift is observed.

The expected change of the distance of the system from the fixed point, conditioned on a starting point , is then given by

(12)

where and refer to the first and second derivatives with respect to of the function . Focus sing now on the Euclidean distance we then have

(13)

For the local process as specified above we find

(14)

This result is illustrated and compared with numerical simulations in Fig. 1. Simulations are of the microscopic process are here carried out using the popular Gillespie algorithm gillespie (). As seen in the figure the theoretical result (13) is confirmed by these computer experiments. Results show that for any fixed system size , a local outward drift, occurs for starting points, , close to the deterministic fixed point. If the starting point is sufficiently far away from the fixed point an inward drift occurs instead. As demonstrated in the inset of Fig. 1 the region in which an outward drift occurs shrinks consistently as the system size is increased. This is illustrated further in Fig. 2, where we show where in strategy space inward and outward drifts occur as a function of system size. In particular we notice that drift is a local quantity, at a given system size and at fixed model parameters the drift may be inward for one starting point in strategy space, but outward in other regions. The range of starting points at which an outward drift is observed is reduced as the system size is increased. In infinite populations the drift is inward throughout strategy space, with a globally stable attractor.

An analytical estimate for the boundary separating the regions with inward and outward drifts can be obtained setting in Eq. (14). Re-arranging and ignoring sub-leading terms in higher powers of one finds that a reversal of local drift occurs when

(15)

i.e. when , establishing a typical distance from the deterministic fixed point. Eq. (15) thus indicates the point in strategy space at which, for a given system size, the reversal of local drift occurs. Conversely for a given point in strategy space the reversal of local drift is found at a population size determined by .

These results can be connected to the well established system-size expansion approach to evolutionary processes in finite populations. We here discuss an approach based on van Kampen’s system size expansion, an alternative line of reasoning, using a Kramers-Moyal expansion is outlined in the Appendix. Following kampen (); bladon () the following stochastic differential equation for fluctuations about the deterministic fixed points can be derived

(16)

The variable is here defined as the re-scaled deviation

(17)

from the deterministic fixed point, the noise variable is Gaussian and uncorrelated in time, specifically one obtains

(18)

from the van Kampen expansion. We will not report the details of the derivation of Eqs. (16,17,18), these follow the steps of kampen (); bladon (). It is however worth noting that these equations apply only in the long-time limit, when the deterministic dynamics have reached a fixed point. The van Kampen expansion can also be executed for transients or deterministic systems with attractors other than fixed points boland (), but this is not relevant for the purposes of the present paper.

Eqs. (17,18) define a linear Langevin equation, and it is hence straightforward to compute the variance of fluctuations in the stationary state, see e.g. risken () for details. One finds that , and hence at long times . This defines a ‘typical’ asymptotic distance from the deterministic fixed point in perfect agreement with Eq. (15). We therefore conclude that the stochasticity of the dynamics induces a typical asymptotic distance from the deterministic fixed point, and that an inward drift is observed for starting points in strategy space that are further away from the fixed point than this typical distance. An outward drift is found for starting points closer to the fixed point than the ‘preferred’ asymptotic distance, underlining the importance to think of drift as a local quantity. In claussen1 (); claussen2 () similar calculations were performed (in the context of different games), but a flat average over all possible initial conditions was then computed, resulting in a global notion of an inward or outward drift, depending solely on the system size , but with no ‘local’ character in strategy space. While expressions for the local drift were of course contained in the calculations of claussen1 (); claussen2 () as intermediate steps, we believe that our results and analysis provide an interesting addition and further insight into the dynamics of drift reversal.

Figure 2: (Color online) Dependence of the direction of local drift in the Hawk-Dove game as a function of the starting point and of the system size of the finite population. An outward drift occurs in a region close to the deterministic fixed point (shaded area, results are obtained from Eq. (13)), and inward drift further away from the deterministic fixed point. The range of starting points at which an outward drift is observed is reduced as the system size is increased. In infinite populations the drift is inwards throughout strategy space, with a globally stable attractor. Black lines show the estimate of Eq. (15).

Iv Imitation dynamics and internal absorption in one-dimensional systems

We now turn to the second main topic of this paper, and consider a microscopic update process which admits absorption at an internal fixed point even for finite populations. This is not the case in the previous section, where we have shown that finite systems under the local update rule instead tend to a stationary state with fluctuations of a typical magnitude, or distance from the fixed point, even if the corresponding deterministic dynamics have a stable fixed point within the interior of strategy space.

Specifically we will now consider an imitation process. As before two players are drawn at random from the population at the beginning of each microscopic update step. If they are of identical types, nothing happens. If one player is of type and the other one of type , then the two players compare their fitnesses, and . If the two fitnesses are equal, , the player of type converts into a player of type with a rate proportional to , and similarly switches to with rate . The non-negative constant is a model parameter, and denotes the rate with which ‘neutral’ transitions occur. If however one player has a strictly higher fitness than the other, then the player with the higher fitness remains unchanged. The player with the lower fitness adopts the higher-performing strategy with a rate proportional to the fitness difference, for example if , then player will turn into a player of type with a rate proportional to . No transitions to strategies with strictly lower fitness are allowed. Specifically we will consider the process defined by the following transition rates:

(19)

is the step function, i.e. we have , .

Figure 3: (Color online) Internal absorption in the imitation dynamics of the Hawk-Dove game. Left: Trajectories for players, started from different initial conditions. The horizontal line shows the internal absorbing state, , as defined by Eq. (8). Right: Average time to fixation at the internal fixed point in a population of agents, started at point in strategy space (solid line are results from Eq. (23), interpolated to non-integer for optical convenience, symbols are from simulations, averaged over independent runs).

The deterministic limit of this dynamics is obtained as

(20)

where and . Inserting the expressions for and of the Hawk-Dove game () we have

(21)

We will here mostly focus on the case , which we will refer to this as strict imitation, or strict proportional imitation. In this case the choice of and is mostly irrelevant, and equivalent to setting the time scale. Unless specified otherwise we will mostly use in the following. The word ‘proportional’ indicates that adaptation towards better strategies occurs with a rate proportional to the improvement in fitness. As first shown in helbing () one recovers the standard replicator dynamics for , see also hofbauer (); gintis (). Indeed Eq. (21) reduces to Eq. (11) for up to an overall re-scaling of time by a constant factor .

Even though the local process discussed in the previous section and the strict imitation dynamics have identical deterministic limits the dynamical features of the respective stochastic processes in finite populations are quite different. This is illustrated in Fig. 3, where we show a set of trajectories obtained from the strict imitation dynamics () for the Hawk-Dove game (), started at different initial conditions. As seen in the figure all trajectories are eventually absorbed at the internal fixed point given by Eq. (8).

This behaviour can be understood from considerations of local drift taking into account that at we have for all , i.e only one of the two transition rates is ever non-zero for any given . If for example , then we have , and hence . Excluding the trivial case , which is an absorbing state itself, the resulting drift is then given by

(22)

with the last inequality holding as and for . A similar argument applies when . At the fixed point we have , and therefore , confirming that is indeed an absorbing state. One therefore concludes that fixation at the internal attractor will occur for any non-trivial starting point (i.e. ), and provided that is an even number (ensuring that is an integer).

To complete the picture it remains to compute the mean time to fixation, given a starting point . Assume to this end that . Then , and the system will move to state with the rate , reaching the state after an average waiting time of . If the system has not yet reached absorption. A second exponential clock will start ticking, and the system will hop to state after an average waiting period . This procedure then iterates until the absorbing state is reached, and we conclude that the mean time to fixation for trajectories starting at is given by

(23)

Similar results can be derived for starting points . These findings are in full consistency with the results of e.g. traulsendet () (see e.g. Eqs. (7) and (8) of this reference), taking into account for example that for all , and that absorption occurs at the internal fixed point with probability one. Comparison with simulations confirms the validity of our calculations, see the right panel of Fig. 3. As a technical detail we point out that the time scale used in the figure is obtained from that of Eq. (23) via the re-scaling . This is to ensure a well defined thermodynamic limit, , for example one may notice that the sum in (23) contains an number of terms, so that normalisation as just described is appropriate.

V Two-player symmetric games with cyclic interaction

v.1 Internal absorption

We will now turn to a different class of games, and consider symmetric two-players games with three pure strategies and cyclic interaction. Specifically we will investigate a generalisation of the well-known rock-paper-scissors (RPS) game, defined by the payoff matrix

(24)

where we will generally consider the case . Two players of identical species (strategies) will score zero payoff when they play each other. Strategy will ‘beat’ strategy two (receiving a payoff of unity), but lose against strategy three, and similar for all cyclic permutations. The loser’s payoff is . The central point of the -dimensional strategy simplex, at which all three species are equally populated ( for ) is known to be a stable fixed point of the corresponding replicator equation, provided gintis (); claussen1 (); mobilia (). At , corresponding to the standard RPS game, the centre is a neutrally stable fixed point, and the replicator dynamics admits an integral of motion, with periodic orbits about the centre fixed point. At finally the central fixed point is unstable, and deterministic trajectories approach the boundary of the simplex in a periodic manner. In absence of mutation, the case we will consider here, the three corner points of the simplex () are fixed points for any value of .

The strict imitation dynamics in finite population is then defined by the following transition rates

(25)

We here use and .

Figure 4: (Color online) Left: Trajectory obtained for the generalised RPS game under strict proportional imitation (, ). The the three curves show the concentrations of the three strategies as a function of time, convergence to the internal attractors is reached at about . Right: Circles indicate the probability to fixate at one of the corners of the strategy simplex (pure strategies) if started from a random initial condition (), squares show the probability of absorption at the central attractor (mixed strategy). Results are from simulations ( samples, run until fixation), and shown as a function of the model parameter .

As shown in the left panel of Fig. 4, this dynamics admits absorption at the internal fixed point. At the centre of the strategy simplex all three strategies are equally abundant, and by symmetry their fitnesses are equal. Hence all transition rates in Eq. (25) vanish. This is the case for any value of , however as seen in the right panel of Fig. 4 fixation at the centre occurs with non-zero probability only at . To understand this in more detail, it is helpful to keep in mind that the system can either fixate at the centre (corresponding to a mixed strategy profile) or at one of the corners of the strategy simplex (pure strategies). Due to stochastic fluctuations fixation at one of the four attractors will occur eventually in finite systems. This is confirmed in simulations, detailed analysis of the data in the right panel of Fig. 4 shows that all trajectories fixate eventually. Generally the stochastic dynamics will be governed by two contributions, first the flow of the deterministic limit about which the finite system can be thought to fluctuate, and secondly stochastic drift due to finite-size fluctuations. Stochasticity tends to drive the system to the boundaries of the simplex, and can hence cause fixation in one of the pure strategies (once the system hits one of the edges of the strategy simplex, fixation into one of the corners occurs). The results of Fig. 4 can then be understood as follows:

Figure 5: (Color online) Probability that fixation occurs at one of the corners of the strategy simplex in the generalise rock-paper-scissors game under strict proportional imitation dynamics (). Results are shown as a function of the size of the population. Symbols represent simulations, averaged over samples (depending on ), started from random initial conditions. The solid lines has a scaling of . Inset: Average time to fixation at the boundary (open symbols) and the centre (filled symbols) for

.

  1. At both the deterministic flow and the stochastic fluctuations drive the system towards the exterior of the simplex, and fixation occurs at one of the pure strategies.

  2. At the deterministic flow pulls the system towards the centre of the strategy simplex, where fixation occurs, if the centre is reached. Stochastic fluctuations however provide a tendency to drift outwards, and provided one of the edges of the simplex is reached fixation occurs at a pure strategy. The deterministic pull towards the centre will be strong for values of much smaller than one, and fixation at the centre is likely. For values of close to one, the deterministic fixed point becomes less and less stable, and fluctuations more influential. The system fixates at the borders of the simplex with increasing probability, as seen in Fig. 4 (right panel), notice in particular the non-zero probability of fixation at a pure strategy for values of just below one.

Measurements of the probability with which the system fixates at the boundary for fixed values of , and at varying system sizes confirm that external absorption (i.e. fixation at one of the pure strategies) is indeed a stochastic effect induced by fluctuations. Fig. 5 shows that the rate with which the system fixates at the edges of the simplex scales as at large values of .

This can be further understood from Fig. 6, where we show what the most likely point of fixation is for different starting points of the dynamics. If started in the central region of the strategy simplex (indicated by grey symbols) it is more likely that the system will fixate at the centre than at any of the three pure strategy absorbing states. For starting points closer to the edges of the simplex, fixation is likely to occur at one of the corners, as indicated by the coloured symbols. The left and right panels compare the basins of attraction of the internal absorbing state for different system sizes. As seen in the figure the domain from which the central attractor is reached grows as the system size is increased, and essentially spans the bulk of the two-dimensional strategy simplex for sufficiently large systems. Fixation at the edges only occurs for starting points at the periphery. This leads to the conclusion that the results of Fig. 5 may potentially be interpreted as an effect of bulk versus periphery. A starting point in the central region (which has a size ) favours fixation at the centre, a start in the peripheral region (which has a size ) predominantly leads fixation at one of the corners. At large and started from a random initial condition fixation occurs with a probability scaling with the relative areas of the peripheral and central basins of attraction, leading to a probability of external fixation scaling as . As a final remark we report that the average time to fixation at the interior fixed points (averaged over all initial conditions) is logarithmic in time (see inset of Fig. 5).

Figure 6: (Color online) Fixation in the generalised rock-paper-scissors game at under strict adaption dynamics. The figure shows results from simulations indicating where the system is most likely to fixate given a starting position. Red symbols for example indicate fixation in the upper corner of the simplex. If started from the grey area near the centre the system is found to most likely fixate at the internal fixed point . System sizes are (left) and (right). Simulations are run for in excess of independent samples per data point.

v.2 Drift reversal

We will now briefly turn to a discussion of local drift reversal in the generalised rock-papers-scissors game. As pointed out above such phenomena can only be expected when the considered dynamics has a stable attracting interior fixed point in the deterministic limit. Also we need to exclude internal absorption. We therefore focus on the above local process, and the generalised RPS game, Eq. (24), with .

Figure 7: (Color online) Drift reversal in generalised RPS: The figure shows time series from simulations of the local process at two different system sizes (). In both cases the dynamics were once started from a fixed initial condition close to the central deterministic fixed point (increasing curves), and once from a point closer to one of the edges of the simplex (decreasing curves). Trajectories which fixate during the course of the simulation up to are excluded from the statistics, simulation data is from in excess of non-fixated runs. As seen in the figure the initial drift is away from the fixed point in the former case, and towards the fixed point in the latter. The dashed lines show the asymptotic value of as predicted by Eq. (32).

The limiting replicator equations for the degrees of freedom and , denoting the concentrations of the three strategies, are then known to admit the Liapunov function gintis (); traulsen2 ()

(26)

Specifically one has under the replicator dynamics (), and the deterministic dynamics converges to the central fixed point , corresponding to . In claussen1 () the occurrence of drift reversal was reported for stochastic dynamics for this game in finite populations. Specifically, random initial conditions in the strategy simplex were considered, and it was shown that different microscopic processes tend to lead to an increase of in small populations (drift away from the central fixed point), and to decreasing values of in large populations (drift towards the fixed point). The ‘critical’ population size, at which the drift reverses sign, can be computed within an expansion in the inverse system size, see claussen1 (); claussen2 (). Crucially however, drift reversal was considered only as a global phenomenon, averaged over all possible initial conditions in strategy space. The main objective of this section is to refine this analysis, and to stress that drift and its reversal are again local phenomena.

This can be observed for example in Fig. 7, where we show that the initial drift is towards the fixed point (decreasing curves in the figure), when the dynamics is started sufficiently far away from the centre of the simplex, but that a drift away from the centre (increasing time series of ) occurs for starting points close to the center.

Figure 8: (Color online) Drift reversal in a generalised RPS population () with individuals, subject to the local update process. Yellow symbols near the centre indicate the region in strategy space at which the drift has an outward direction, darker symbols in the outer region indicate an inward drift. These were obtained from numerically evaluating the sign of the drift, using an analogue of Eq. (12). The solid black line indicates points with the value predicted by Eq. (32). Small deviations of the yellow region from the inside of the perimeter defined by are due to finite-size effects.

The simulation results in Fig. 7 suggest that for any given population size the system chooses an asymptotic value, , of , and that drift always occurs towards this preferred value of . Generally will depend on the system size. Thus if, at a given value of , the initial condition is such that the system is further away from the fixed point than indicated by , an inward drift occurs. Similarly, for starting points too close to the centre, an outward drift, together with an increase of is found, until is reached. One should here stress that states with are of course not absorbing, but that statements such as ‘the system will reach a value of ’, instead only hold statistically speaking, and when averaged over realisations of the stochastic dynamics.

Analytical progress can here be made within a system-size expansion, which allows one to describe the fluctuations about the deterministic fixed point. Again we do not discuss the detailed mathematics, as these are available in the literature, but report only the final result, consisting of the the following Langevin equations for fluctuations in the and components of the dynamics (fluctuations in the component then follow trivially due to overall conservation of particle number). One has

(27)

where are the fluctuations of the -component, more precisely one has for . The variables and denote zero-average Gaussian white noise with the following covariance properties

(28)
(29)

as obtained from the van Kampen expansion. The linearity of Eqs. (27) allows further progress and in particular the long-time limit of the following quantities can be computed

(30)

see risken () for further details. Denoting these asymptotic values by and respectively, one then has

(31)

where and are zero-average Gaussian random variables with statistics for , and where denotes an average over and . From this one finds

(32)

This estimate of the stationary value of , accurate for large, but finite , compares well with simulations, see Fig. 7, and confirms the picture discussed above. Depending on the system size the dynamics tends to a stationary value , see Eq. (32). Interpreting as a ‘distance’ from the central deterministic fixed point, the dynamics is thus driven towards a state of a preferred distance from the central point. If started from closer to the fixed point than this distance then the initial drift is outwards, if started from a point outside the asymptotic perimeter set by , then the drift is inwards. This is illustrated in Fig. 8.

As a final technical and more subtle point we would like to add that the corners of the strategy simplex still remain absorbing states, as we are working in the absence of mutation. One should hence expect that any trajectory of the system in finite populations will fixate on a pure strategy eventually. While this may at first seem to be in contradiction with the results just discussed, it is important to keep in mind that this type of fixation occurs on an exponentially slow time scale for , see e.g. mobilia (). This fixation regime is not captured by the van Kampen expansion. Instead the expansion addresses a regime in which a stationary Gaussian distribution is assumed, and where fixation at the edges of the simplex can be discarded. For the comparison of the theoretical predictions obtained from the van Kampen expansion against numerical data, we have therefore excluded trajectories which fixate in our simulations (see Fig. 7).

Vi Imitation dynamics in two-player games

In this section we will now consider a final example, and move on to a more complex asymmetric two-population game, Darwin’s so-called ‘Battle of the Sexes’, also known as the ‘Matching Pennies’ game. Here there are two separate populations, each of size , and labelled and for simplicity. Players in population (‘male’) only play the game with players in population (‘female’), and vice versa. The payoff of a pure strategy in the male population is hence determined by the composition of the female population and vice versa. Payoffs can then be encoded in bi-matrix form as follows:

(33)

The pure strategies in population are labelled and , the pure strategies in population are referred to as and . The first element in the brackets for each of the four cases is the payoff of the male player, the second the payoff of the female. In simple terms a male -player prefers to interact with a female -player rather than a female , and a male preferably interacts with a female (anti-coordination). Female players take the opposite view, a female receives a higher payoff when meeting a male as compared to meeting a male , and similarly a female prefers a male over a male (coordination). Of course the labelling ‘male’ and ‘female’ is not meant to have any direct real-world interpretation, the game would be identical with these labels interchanged. We denote the number of male players by and the number of female players by . The payoff structure is then described by fitness functions defined as follows

(34)

Strict imitation dynamics occurs within the two populations, for example two male players may be chosen for potential adaptation, compare their fitnesses and then one of them may change their strategy to adopt that of the other male. This leads to the following transition rates

(35)
Figure 9: (Color online) Battle of the Sexes: Panel a) shows an illustration of the deterministic flow, and the five fixed points. Panel b) shows the region near the central fixed point, arrows indicate the transitions that are possible under the strict imitation dynamics. The central fixed point can never be reached. The arrows into and out of the central square have been colored in pairs. The two arrows of any given color will give a net outflow taking into account that transition rates scale in proportion to ratios of the type for example.

The resulting flow in the deterministic replicator limit is shown in Fig. 9 (panel a), and has the well known cyclic motion with neutrally stable cycles about the central fixed point at . While this point is an absorbing state under the strict imitation dynamics in finite populations, Eqs. (35), simulations reveal that fixation never occurs at the centre, but that instead all trajectories are absorbed at one of the four corners of strategy space eventually. This is illustrated in Fig. 10, where we indicate the most likely point of absorption for all possible initial conditions of the imitation dynamics for . The absence of internal absorption in this game can be understood from a closer inspection of the imitation dynamics close to the central fixed point (see Fig. 9 panel b). In this figure we show the central fixed point (centre circle), and the surrounding states in finite populations (the population size is assumed to be even, so that the central fixed point is within the allowed strategy space). The arrows indicate transitions from one state to another that are possible under imitation, if no arrow is present the corresponding transition is not admissible. As seen in the figure the central fixed point cannot be reached from any of the neighbouring states, hence internal absorption can not occur. A closer analysis shows that there is indeed a net flux away from the region of the central fixed point, see the caption of Fig. 9.

Figure 10: (Color online) Fixation in the battle of the sexes under strict imitation dynamics. Colors indicate the absorbing state that is most likely to be reached if the dynamics is started from a given initial condition. Simulations are for , realisations. The central fixed point at is left blank.

Vii Conclusions

In summary we have studied a strict imitation dynamics in games played by finite populations of players. It is here important to stress that strict imitation is at best a stylised dynamical process, and in the light of recent experiments traulsenpnas () may well be ruled out for real-word adaptation and learning of players. Still this dynamics has interesting dynamical features, as underlined by the early studies of Helbing helbing (), linking imitation to replicator dynamics. We have here shown that proportional imitation can give rise to what we call internal absorption, a novel dynamical phenomenon in which dynamic arrest occurs at an internal point in strategy space, corresponding to the stable attractor of the deterministic replicator dynamics. This effect occurs in simple symmetric games, provided the payoff matrix is chosen such that the replicator equations have a stable internal fixed point. We have also verified the occurrence of internal absorption in generalise rock-paper-scissors games, again the stability of the internal fixed point under the replicator dynamics appears to be a necessary condition. In the Battle of the Sexes, where the replicator equations have neutrally stable cycles, no absorption at the internal fixed point is found.

The second main contribution of this paper is to analyse in detail the phenomenon of drift reversal, first reported in claussen1 (); claussen2 (). We have shown that the reversal of drift is in fact a local phenomenon, dependent on position in strategy space. Provided its deterministic limiting dynamics has a stable internal fixed point, a given microscopic update rule will select a ‘typical’ distance from this fixed point at which it operates asymptotically. This distance will depend on the population size, and will be smaller for large populations than for small ones. If started closer to the fixed point than the asymptotic distance, the system will have a tendency to move away from the fixed point (outward drift), if started further away from the fixed point than the preferred distance, an inward drift will be observed. The precise domains of inward and outward drift can be obtained analytically within a system-size expansion, valid for large, but finite populations. In this limit we have also provided an analytical estimate for the typical asymptotic distance from the deterministic fixed point.

Acknowledgements.
The author acknowledges funding by the Research Councils UK (RCUK reference EP/E500048/1), and would like to thank A. Traulsen and J. C. Claussen for useful discussions.

References

  • (Altrock & Traulsen 2009a) P. M. Altrock, A. Traulsen, Fixation times in evolutionary games under weak selection, New. J. Phys, 11 013012 (2009)
  • (Altrock & Traulsen 2009b) P. M. Altrock, A. Traulsen, Deterministic evolutionary games in finite populations, Phys. Rev. E 80 011909 (2009)
  • (Bladon et al 2010) A. J. Bladon, T. Galla, A. J. McKane, Evolutionary dynamics, intrinsic noise and cycles of co-operation, Phys. Rev. E, at press (2010), preprint arXiv:1006.0825v1 [q-bio.PE]
  • (Boland et al 2009) R. P. Boland, T. Galla, A. J. McKane, Limit cycles, complex Floquet multipliers, and intrinsic noise, Phys. Rev. E 79 051131 (2009)
  • (Claussen & Traulsen 2008) J. C. Claussen, A. Traulsen, Cyclic Dominance and Biodiversity in Well-Mixed Populations, Phys. Rev. Lett. 100 058104 (2008)
  • (Claussen 2007) J. C. Claussen, Drift reversal in asymmetric coevolutionary conflicts: influence of microscopic processes and population size, Eur. Phys. J. B 60 391 (2007)
  • (Cremer et al 2008) J. Cremer, T. Reichenbach, E. Frey, Anomalous finite-size effects in the Battle of the Sexes, Eur. Phys. J. B 63 373 (2008)
  • (Gillespie 1976) D. T. Gillespie, A General Method for Numerically Simulating the Stochastic Time Evolution of Coupled Chemical Reactions, J. Comput. Phys. 22, 403 (1976)
  • (Gintis 2000) H. Gintis, Game theory evolving, Princeton Univ. Press, Princeton NJ (2000)
  • (Helbing 1992) D. Helbing, A mathematical model for behavioural changes by pair interactions and its relation to game theory, in: G. Haag, U. Mueller and K. G. Troitzsch (Eds), Economic Evolution and Demographic Change. Formal Models in Social Sciences, Springer (Berlin) 1992
  • (Hofbauer & Sigmund 1998) J. Hofbauer, K. Sigmund, Evolutionary games and population dynamics, Cambridge University Press, Cambridge (UK) 1998
  • (Hubbell 2001) S .P. Hubbell, The United Neutral Theory of Biodiversity and Biogeography (Princeton University Press, Princeton, 2001)
  • (Kimura 1983) M. Kimura, The neutral theory of molecular evolution, Cambridge University Press, Cambridge, UK (1983)
  • (Kimura & Crow 1964) M. Kimura, J. F. Crow, The number of alleles that can be maintained in a finite population, Genetics 49 (1964) 725
  • (Mobilia 2010) M. Mobilia, Oscillatory dynamics in rock-paper-scissors games with mutations, J. Theor. Biol. 264 1 (2010)
  • (Nowak 2006) M. A. Nowak, Evolutionary dynamics, The Belknap Press of Harvard University Press, Cambridge MA (2006)
  • (Reichenbach et al 2007) T. Reichenbach, M. Mobilia, E. Frey, Mobility promotes and jeopardizes biodiversity in rock-paper-scissors games, Nature 448 1046 (2007)
  • (Risken 1996) H. Risken 1996, The Fokker-Planck equations, 2nd ed., Springer-Verlag Berlin Heidelberg (1996)
  • (Traulsen et al 2005) A. Traulsen, J. C. Claussen, C. Hauert, Coevolutionary Dynamics: From Finite to Infinite Populations, Phys. Rev. Lett. 95 238701 (2005)
  • (Traulsen et al 2006) A. Traulsen, J. C. Claussen, C. Hauert, Coevolutionary dynamics in large, but finite populations, Phys. Rev. E 74 011901 (2006)
  • (Traulsen & Hauert 2009) A. Traulsen, C. Hauert, Stochastic evolutionary game dynamics, in Reviews of Nonlinear Dynamics and Complexity Vol. II ed. H.-G. Schuster, Wiley-VCH: Weinheim (2009)
  • (Traulsen et al 2010) A. Traulsen, D. Semmann, R.D. Sommerfeld, H.-J. Krambeck, and M. Milinski, Human strategy updating in evolutionary games, Proc. Natl. Acad. Sci. USA 107 2962 (2010)
  • (van Kampen 1992) N. G. van Kampen, Stochastic processes in physics and chemistry, Elsevier, Amsterdam (1992).

Appendix

The point in strategy space at which drift reversal occurs for the one-dimensional Hawk-Dove example can also be obtained from a Kramers-Moyal expansion of the master equation (3). As shown in traulsen2 (); traulsen () this formalism results in a multiplicative stochastic process of the form

(36)

where denotes the increment of a standard Wiener process. One has and . From this one then finds (respecting the rules of Ito calculus)

(37)

for the time evolution of the distance from the interior fixed point. This then gives

(38)

i.e.

(39)

The notation here indicates the expected change in distance per unit time at time conditioned to trajectories starting at position at . The stationary point, is therefore given by the condition . For the Euclidean distance this simplifies to , and finally assuming that is of order , we self-consistently find

(40)

For the case of the Hawk-Dove game with local dynamics we find and so that we have

(41)

reproducing the result of Eq. (15).

Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
""
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
   
Add comment
Cancel
Loading ...
69799
This is a comment super asjknd jkasnjk adsnkj
Upvote
Downvote
""
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters
Submit
Cancel

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test
Test description