Generalized entropies and anomalous diffusion
Abstract.
Many physical, biological or social systems are governed by historydependent dynamics or are composed of strongly interacting units, showing an extreme diversity of microscopic behavior. Macroscopically, however, they can be efficiently modeled by generalizing concepts of the theory of Markovian, ergodic and weakly interacting stochastic processes. In this paper, we model stochastic processes by a family of generalized FokkerPlanck equations whose stationary solutions are equivalent to the maximum entropy distributions according to generalized entropies. We show that at asymptotically large times and volumes, the scaling exponent of the anomalous diffusion process described by the generalized FokkerPlanck equation and the phase space volume scaling exponent of the generalized entropy bijectively determine each other via a simple algebraic relation. This implies that these basic measures characterizing the transient and the stationary behavior of the processes provide the same information regarding the asymptotic regime, and consequently, the classification of the processes given by these two exponents coincide.
Real world processes are often characterized by the presence of a large number of interacting phenomena at multiple time or length scales [1], and thus, they are usually described by stochastic models that are strongly interacting or historydependent [2, 3, 4, 5]. A way to understand and classify these processes in terms of stationary and nonstationary probability densities is to generalize the concepts of statistical mechanics that already proved to be very powerful for describing weakly interacting, ergodic and Markovian systems [6, 7]. One such concept is entropy, which assigns a likelihood to macrostates, that is, to stationary distributions over microstates [8]. Maximizing this likelihood, possibly in the presence of external constraints, yields the most probable stationary distribution characterizing the system, called the Maximum Entropy (MaxEnt) distribution, which plays a key role in describing the stationary behavior of stochastic systems.
For example, the BoltzmannGibbs entropy form, , where runs over the microstates, follows from the assumption that the system realizations are independent and distinguishable: based on prior microstate probabilities , the likelihood that the fraction of N realizations is in microstate , is given by a multinomial distribution as
(1) 
In the expression above, the likelihood can be factorized to a independent multiplicity and a dependent factor corresponding to the prior distribution over the microstates. Taking the logarithm of both sides and rescaling by yields
(2) 
When considering the thermodynamic limit, becomes the relative entropy between the prior distribution and the realized histogram ; is corresponding to the BoltzmannGibbs entropy, and is the socalled crossentropy. Finding the most likely histogram, taking into account the possible nonuniform prior distribution , is in fact equivalent to the minimization of the relative entropy, not to the maximization of the entropy. Nevertheless, with uniform prior , where is corresponding to the phase space volume, the resulting distributions coincide.
In general, however, the realizations are not independent. Instead, their interaction can be macroscopically modeled by a corresponding entropy functional, which in principle can take infinitely many different forms. Similarly to the theory of renormalization group describing critical phenomena, an apprehensive characterization of these entropies can be made by observing what are the relevant and irrelevant parameters as we approach infinite system size [9]. Both axiomatic and combinatorial considerations suggest that the asymptotic scaling of the generalized entropy forms with provides a meaningful classification of the entropies.
This classification is based on the fundamental result by Hanel and Thurner[10] about the entropy functionals that can be written as a sum of a pointwise function over microstates, . As they showed, the first three ShannonKhinchin (SK1SK3) axioms [11, 12]

is continuous in

is maximal for the uniform distribution,

is invariant under adding a zeroprobability state to the system,
permit only the following asymptotic scaling relation for any entropic forms:
(3) 
or, equivalently, in terms of ,
(4) 
with and .
Hence, the scaling exponent can be used to parametrize the equivalence classes of the generalized entropy forms. For example, the BoltzmannGibbs entropy, where , is corresponding to , whereas the Tsallis entropy , with is corresponding to . Consequently, each such equivalence class can be represented by a Tsallis entropy.
Using combinatorial arguments, Hanel, Thurner and GellMann showed that generalizing the MaxEnt principle (2) to nonmultinomial processes yields SK1SK3 compatible entropies with asymptotic form 3, hence, such entropies can also be classified in the same manner by means of the exponent .
Note that the fourth ShannonKhinchin axiom is not considered in this analysis, therefore, the entropy of a joint distribution is not always decomposable to the entropy of the marginal and the entropy of the conditional distribution , averaged over .
In this paper we consider continuous entropy forms, and our main focus is on the relation between the MaxEnt distribution and the solution of the partial differential equation describing the macroscopic nonstationary dynamics. According to that, any entropy we discuss here, determined by a function , is assumed to be written as
(5) 
where is asymptotically characterized by (4) and is a timeindependent scalar function of the phase space coordinate. The partial differential equation describing the dynamics is a generalized FokkerPlanck equation, governing the timeevolution of a probability density , written in the form of
(6) 
where and are constants, describes some external potential, and is analogous to an effective density [13, 14, 15, 16]. Such nonstationary processes can be classified phenomenologically by the asymptotic scaling of the spread of over time, termed as anomalous diffusion [17, 18]:
(7) 
where the scaling factor of the space coordinate keeps the probability density invariant when the timescale is changed as . In general, (7) is satisfied only in the asymptotic limit, i.e., when . Nevertheless, this scaling relation, parametrized by , classifies the governing dynamics described by (6). For example, the diffusion equation falls into the equivalence class .
A key relation between the stationary behavior of the process, determined by the corresponding generalized entropy given in the form of (5), and the nonstationary behavior governed by a generalized FokkerPlanck equation is that the stationary solution of (6) agrees with the MaxEnt distribution [14, 19]. E.g., in case of the BoltzmannGibbs entropy, where , the MaxEnt distribution restricted by a constraint on the expected value of takes the form of . However, this is also equivalent to the stationary solution of the FokkerPlanck equation describing ordinary diffusion in the presence of some external potential ,
(8) 
which is a special case of 6 with . Setting zero net flux at the boundaries yields .
Based on the above, the BoltzmannGibbs entropy, belonging to the entropy class , is corresponding to the FokkerPlanck equation describing simple diffusion, which in turn is a member of the anomalous diffusion scaling class . A natural question arising based on this observation is the following: Does every entropy belonging to the universality class correspond to a generalized FokkerPlanck equation from the anomalous diffusion class ? And does every generalized FokkerPlanck equation belonging to the class correspond to an entropy belonging to the class? In other words, does and give rise to the same equivalence class, therefore, bijectively determine each other? In this paper we show that this is true not only for and , but for every and , where the exponents and are connected by a simple algebraic relation. This implies that the asymptotic scaling of generalized entropies with phase space volume and the asymptotic anomalous diffusion scaling of the corresponding generalized FokkerPlanck equation classify the processes in the same way, and consequently, they provide the same information about their asymptotic behavior. In Fig.1. we show a schematic illustration of the above concept.
In order to derive a relationship between the asymptotic exponents and , let us first consider the MaxEnt distribution corresponding to entropies given in the form of (5). By following a variational principle approach and taking into account the normalization and expected value constraints we can write
(9) 
where the constants are omitted for simplicity and the and Lagrange multipliers are introduced for fixing the zeroth and first moment, respectively. From (9) we obtain
(10) 
where is the inverse of the MaxEnt distribution corresponding to the entropy defined by . In case of the BoltzmannGibbs entropy the inverse of the MaxEnt distribution is given by the (appropriately shifted and rescaled) logarithm function, . Therefore, is usually referred to as the generalized logarithm for any entropy in general [6, 20].
Based on a given entropy and the corresponding generalized logarithm , our next step is to find the related generalized FokkerPlanck equation in the form of (6), where is chosen such that the stationary solution of the equation becomes equivalent to the MaxEnt distribution of the entropy. By replacing with in (6) according to (10) and using the chain rule we obtain that the stationarity condition is fulfilled if
(11)  
Thus, defining the generalized FokkerPlanck equation via (6) can be given as
(12) 
In particular, for the BoltzmannGibbs entropy and , yielding , which results in . Since is formulated based on in (12), we call the resulting equation
(13) 
the FokkerPlanck equation in order to distinguish it from the many other possible generalizations.
In the following, let us consider the FokkerPlanck equation with no external potential,
(14) 
In the asymptotic limit when the scaling rule (7) applies to . Thus, if we change to the rescaled variables and , the derivatives according to the new variables can be written as
(15) 
Using (7) and (15), the FokkerPlanck equation with no external potential given in (14) in the rescaled variables can be formulated as
(16) 
where we denoted by for simplicity. The term on the right hand side can be further transformed based on the scaling of given in (4), where by a change of variable we obtain (being valid for ). By substituting this into (12) we obtain
(17) 
According to that, the FokkerPlanck equation (16) in the rescaled variables yields
(18) 
This equation is satisfied if and only if
(19) 
resulting in the general relation
(20) 
between the exponent related to the anomalous diffusion, characterizing the scaling of in the limit and the exponent , describing the scaling of the generalized entropy with the phase space volume.
In order to demonstrate this general result, in Table 1. we list a few different generalized entropy forms from the literature together with the corresponding FokkerPlanck equations and the related and exponents. Although the actual algebraic form of the entropies along with their phase space volume scaling, their MaxEnt distributions and the corresponding generalized FokkerPlanck equations are different for any , their asymptotic anomalous diffusion scaling is completely determined by via (20). This exemplifies the fact that although the mapping between entropies and FokkerPlanck equations are defined at any (phase space volume or time) scale, any entropy, characterized by asymptotic exponent , can only be mapped to a FokkerPlanck equation describing anomalous diffusion with asymptotic exponent given by (20). In close connection to Table 1., Fig.2. shows the finite scale phase space volume scaling of some generalized entropies, illustrating the numerous possible ways of convergence to the asymptotic value .
In conclusion, the starting point of our study was that the classification of strongly interacting or nonMarkovian systems is highly nontrivial due to the extreme diversity of possible underlying microscopic rules. Macroscopically, however, they collapse to asymptotic equivalence classes according to their stationary and nonstationary behavior, quantified by the corresponding generalized entropy’s phase space volume scaling and the anomalous diffusion scaling, respectively. Here we have shown that these two classifications coincide, suggesting that either of these asymptotic exponents is indeed providing a useful characterization of the systems themselves, and not just describing their behavior in the stationary or nonstationary regime. Furthermore, the surprising versatility of the theoretical framework behind Tsallis statistics to model various aspects of strongly interacting systems might be explained by the fact that the family of Tsallis entropies, characterized by their deformation index , provides an algebraically simple representative of each such asymptotic equivalence class.
Acknowledgements.
The research was partially supported by the European Union through project REDAlert (grant no.: 740688REDAlertH2020SEC20162017/H2020SEC201620171) and by the Novo Nordisk Foundation (grant no.: NovoNordisk2017CY78167/45935)
References
 [1] Y. BarYam and M. Bialik. Beyond big data: Identifying important information for real world challenges. Cambridge, NECSI, 2013.
 [2] F. Shibata, Y. Takahashi, and N. Hashitsume. A generalized stochastic liouville equation. nonmarkovian versus memoryless master equations. J. Stat. Phys., 17(4):171–187, 1977.
 [3] T.D. Frank. A note on the markov property of stochastic processes described by nonlinear fokker–planck equations. Physica A, 320:204–210, 2003.
 [4] R. Hanel and S. Thurner. Generalized (c, d)entropy and aging random walks. Entropy, 15(12):5324–5337, 2013.
 [5] B. CorominasMurtra, R. Hanel, and S. Thurner. Understanding scaling through historydependent processes with collapsing sample space. Proc. Natl. Acad. Sci. U.S.A., 112(17):5348–5353, 2015.
 [6] C. Tsallis. Introduction to nonextensive statistical mechanics. Springer, 2009.
 [7] C. Beck. Generalised information and entropy measures in physics. Contemp. Phys., 50(4):495–510, 2009.
 [8] R. K. Pathria. Statistical mechanics. 1972.
 [9] H. E. Stanley. Scaling, universality, and renormalization: Three pillars of modern critical phenomena. Rev. Mod. Phys., 71(2):S358, 1999.
 [10] R. Hanel and S. Thurner. A comprehensive classification of complex statistical systems and an axiomatic derivation of their entropy and distribution functions. Europhys. Lett., 93(2):20006, 2011.
 [11] C. E. Shannon. A mathematical theory of communication. Bell Sys. Tech. J., 27(3):379, 1948.
 [12] A. I. Khinchin. Mathematical foundations of information theory. 1957.
 [13] R. Toral and P. Colet. Stochastic numerical methods: an introduction for students and scientists. Wiley, 2014.
 [14] A. R. Plastino and A. Plastino. Nonextensive statistical mechanics and generalized fokkerplanck equation. Physica A, 222(14):347–354, 1995.
 [15] L. Borland. Microscopic dynamics of the nonlinear fokkerplanck equation: A phenomenological model. Phys. Rev. E, 57(6):6634, 1998.
 [16] T. D. Frank. Nonlinear FokkerPlanck equations: fundamentals and applications. Springer Science & Business Media, 2005.
 [17] J.P. Bouchaud and A. Georges. Anomalous diffusion in disordered media: statistical mechanisms, models and physical applications. Phys. Rep., 195(45):127–293, 1990.
 [18] A. A. Dubkov, B. Spagnolo, and V. V. Uchaikin. Lévy flight superdiffusion: an introduction. Int J Bifurcat Chaos, 18(09):2649–2672, 2008.
 [19] T.D. Frank and A. Daffertshofer. Nonlinear fokker–planck equations whose stationary solutions make entropylike functionals stationary. Physica A, 272(3):497–508, 1999.
 [20] R. Hanel, S. Thurner, and M. GellMann. Generalized entropies and logarithms and their duality relations. Proc. Natl. Acad. Sci. U.S.A., 109(47):19151–19154, 2012.
 [21] G.A. Tsekouras and C. Tsallis. Generalized entropy arising from a distribution of q indices. Phys. Rev. E, 71(4):046144, 2005.
 [22] E. M. F. Curado and F. D. Nobre. On the stability of analytic entropic forms. Physica A, 335(1):94–106, 2004.
 [23] C. Tsallis. Possible generalization of boltzmanngibbs statistics. J. Stat. Phys., 52(1):479–487, 1988.
 [24] G. Kaniadakis. Statistical mechanics in the context of special relativity. Phys. Rev. E, 66(5):056125, 2002.
 [25] F. Shafee. Lambert function and a new nonextensive form of entropy. IMA J. Appl. Math., 72(6):785–800, 2007.