Up-Hill Diffusion Creating Density Gradient - What is the Proper Entropy?

Up-Hill Diffusion Creating Density Gradient - What is the Proper Entropy?

N. Sato and Z. Yoshida Graduate School of Frontier Sciences, The University of Tokyo, Kashiwa, Chiba 277-8561, Japan
July 20, 2019

It is always some constraint that yields any nontrivial structure from statistical averages. As epitomized by the Boltzmann distribution, the energy conservation is often the principal constraint acting on mechanical systems. Here, we investigate a different type: the topological constraint imposed on ‘space’. Such constraint emerges from the null space of the Poisson operator linking energy gradient to phase space velocity, and appears as an adiabatic invariant altering the preserved phase space volume at the core of statistical mechanics. The correct measure of entropy, built on the distorted invariant measure, behaves consistently with the second law of thermodynamics. The opposite behavior (decreasing entropy and negative entropy production) arises in arbitrary coordinates. An ensamble of rotating rigid bodies is worked out. The theory is then applied to up-hill diffusion in a magnetosphere.

There are plenty of examples that seemingly violate the principle of entropy maximization. So-called up-hill diffusion, creating density gradients, is often observed in multi-phase fluids and solids undergoing spinodal decomposition Hashimoto (); Cahn (), in metallic alloys Dar (), nanoporous materials Lau (), and magmas Les (). By separating the different components of the mixture, Helmholtz free energy achieves a local minimum, characterized by non-uniform concentrations, that is stable against fluctuations Cahn (). With a completely different mechanism, astronomical plasmas accumulate within the magnetic fields of stars and planets through the process of inward diffusion Schulz (); Boxer (); Yoshida2010 (); SatoStoc () and generate an heterogeneous density profile. That the driving force is not the energy constraint is made apparent by the experimental observation of non-neutral plasma particles climbing up the potential hill Yoshida2010 (), as well as by numerical calculations concerning their thermal equilibrium SatoTherm (). Here, the underlying principle is the self-organization of a quasi-stationary state, governed by long-range interactions, that feeds upon the topological constraints (typically in the form of adiabatic or Casimir invariants TC ()) affecting canonical phase space. As long as the invariants are preserved, the ordered architecture, arising from the integral manifolds foliating phase space, seems to be conflicting with the second law of thermodynamics. Once the invariants are broken, the quasi-stationary state is destroyed and the systems progressively approach thermal death. Accretion of galaxies under the action of gravitation Lynden-Bell (); Chavanis (), ferromagnetism mediated by the magnetic field Antoniazzi (); Pakter (), spontaneous creation of planetary magnetospheres through the electromagnetic interaction Schulz (); Boxer (); Yoshida2010 (), vortical structures in magnetofluids preserving helicities YosMah2002 (), living organisms harvesting ‘negentropy’ Schrodinger (), self-organization of data flows in information theory Ikegami () are some of the most paradigmatic examples of such ordered structures that grow on the topological invariants affecting the relevant ‘phase space’.

In the present paper, we study the non-equilibrium statistical mechanics of Hamiltonian systems subjected to the aforementioned topological constraints. In particular, we show that the entropy defined on the invariant measure (the preserved phase space volume) of the system behaves consistently with the second law of thermodynamics. Due to the non-covariant nature of differential entropy Jaynes (); Jaynes2 (), the time evolution of the uncertainty measured in arbitrary coordinates may ‘flip’, and appear as an entropy decrease in the Cartesian perspective. It is the Jacobian of the coordinate change that yields the ordered structure, while the probability distribution is flattened in the proper variables.

The theory, which finds its roots in the phenomenological observation that particle density in planetary magnetopsheres tends to be homogenized in the magnetic coordinates Bir (); Has (), shows that the proper phase space upon which statistical mechanics can be formulated differs from the a priori variables used to represent a general physical system. These findings may pave the way for a new and rigorous understanding of the statistical mechanics governing constrained systems.

We start with a short review of the Hamiltonian formalism. Hamiltonian mechanics is the result of interaction between matter (energy or Hamiltonian function ) and space (Poisson operator ) according to the equation:


where is the flow velocity in -dimensional phase space. (1) admits two typologies of constants of motion: those that can be ascribed to the specific form of the Hamiltonian function, i.e. to the properties of matter, and the so-called Casimir invariants that originate from the eigenvectors with -eigenvalue (the null space or kernel) of the Poisson operator, i.e. from the properties of space. This second kind of invariants, which limits the accessible regions of phase space as a result of the constraining environment, is at the core of the theory developed in the present work. Specifically, due to antisymmetry , whenever the operator has a kernel such that , the system is subjected to topological constraints:


(2) can be thought as the formal definition of topological constraint. We remark that the above result holds for any Hamiltonian, and even if does not satisfy the Jacobi identity (see f1 ()). However, thanks to Darboux’s theorem Morrison1998 (), the Jacobi identity ensures that the kernel is integrable, i.e. that a Casmir invariant exists:


where, for now, we assumed that the rank of is (see rank ()), and the two functions and are integration factor and Casimir invariant () respectively.

It is now useful to make some considerations on the non-covariant nature of differential entropy. Extension of Shannon’s discrete entropy to continuous probability distributions is a delicate process Jaynes (); Jaynes2 (). Indeed, the quantity:


is not, in general, the entropy of the continuous probability distribution on the volume element . The reason is that is not covariant, i.e. its value changes depending on the chosen coordinate system, and (4) tacitly assumes that is an invariant measure. Unfortunately, this is not always the case and (4) has to be amended with Jaynes’ functional:


where the Jacobian compensates the coordinate dependence of the logarithm. In the Hamiltonian picture, one can always find a time-independent function nullifying the Lie derivative of with respect to the dynamical flow , i.e. such that . The obtained with (5) will then give the desired covariant form of entropy. It is useful to recast (5) as below:


Here, is the probability distribution of and is the invariant measure of the system satisfying , with .

We are now ready to test the theory with a simple D example. In D, equation (1) can always be cast in the form , where is a properly chosen vector (see f2 ()). The Euler’s rotation equation for the motion of a rigid body with angular momentum and moments of inertia , , and can be obtained by setting and . The kernel associated to this operator, i.e. the topological constraint (2) affecting the phase space of a rigid body, is soon identified to be . Indeed, . At the same time, one can verify that the Jacobi identity (see f3 ()) is satisfied , making the system Hamiltonian. The Jacobi identity also guarantees that the kernel is integrable (remember (3)) to give the integration factor and the Casimir invariant , so that . Furthermore, the invariant measure turns out to be , as follows from . Since this is the original statistical measure, one can directly apply (4) to define the entropy of an ensemble of such rigid bodies. However, suppose that we consider a slightly more complicated rotation pattern, such as:


where, for example, . Since , high values of will be less probable and (7) may represent the anisotropic rotation of a rigid body that tends to spin around the axis with angular momenta . (7) still satisfies the Jacobi identity, and thus represents an Hamiltonian system with the same Casimir element . However, the invariant measure becomes:


where we introduced new coordinates , with . Separating the constant of motion , the new D canonical equations are:


One can verify that (9) is divergence free.

In order to study the statistical mechanics of the new system, we now consider an ensemble of objects obeying (9) and let them interact by adding to the Hamiltonian an interaction potential . Its ensemble average must go zero , since the total energy of the system has to be preserved. In addition, and this is the key point of the paper, there are grounds for the ergodic hypothesis in the novel coordinates (and not in the original variables ) because of the invariant measure (8). In other words, it is licit to exchange ensemble averages with time averages only on (8):


with and Gaussian white noises and where the bar indicates long-time averaging. In (10) first we substituted ensemble averages with time averages, and then represented the various components with random processes of zero time average. We remark that this would not have been possible in the original coordinates , as they are not measure preserving. Neglecting the constant , the equations accounting for the interaction become:


Note that, while the Hamiltonian is no more a constant, is still a Casimir invariant: the rigid bodies will explore the surface of phase space defined by .

The next step is to build the Fokker-Planck equation associated to (11). We refer the reader to Gardiner (); SatoStoc () for a detailed description of the procedure. The result is:



Here is the probability distribution on and , are the diffusion coefficients associated with the white noises. Finally, we seek for an explicit expression of the entropy production rate of the system. Define the Fokker-Planck velocity to be the vector field such that (Up-Hill Diffusion Creating Density Gradient - What is the Proper Entropy?) is written as . Then, recalling (6):


The first term represents the ensemble average of the Fokker-Planck velocity divergence, while the second factor can be cast as a surface integral representing entropy flow out . It is straightforward to deduce that:


Substituting the expression of in (13a), we obtain:


In figure 1 we report the results of the numerical simulation of (Up-Hill Diffusion Creating Density Gradient - What is the Proper Entropy?). The Entropy , defined on the invariant measure (8) of the system, behaves consistently with the second law of thermodynamics and the associated entropy production is positive. On the contrary, the wrong measure of entropy , defined by the distribution function on the original phase space , decreases. Furthermore, diffusion flattens the distribution and since preservation of particle number requires , creates an ordered structure by approaching .

Figure 1: (a): and as a function of time . (b): as a function of time . Arbitrary units are used. Initial condition is a flat distribution on .

Let us show how the theory can be applied to the study of a real self-organizing system: a magnetosphere. In astronomical plasmas, charged particles are trapped by planetary magnetospheres as they spiral around the magnetic field , where is the flux-function and the toroidal angle of a cylindrical coordinate system . This dynamics (cyclotron motion) is characterized by preservation of the magnetic moment , where is the particle mass, the particle velocity perpendicular to magnetic field lines, and . Because of the topological constraint , it turns out SatoStoc (); SatoHeat () that the invariant measure of magnetized particles is , where and are length and velocity along respectively. The electromagnetic interaction diffuse the constrained particles on the statistical measure and maximize the associated entropy . Due to the inhomogeneous Jacobian , the process will appear as creating density gradients and temperature anisotropy in the Cartesian perspective, while the entropy defined on is minimized. This scenario is exemplified in figures 2 and 3 obtained by simulation of the Fokker-Planck equation derived in SatoStoc (); SatoHeat ().

Figure 2: (a): and as a function of time . (b): as a function of time . Arbitrary units are used. Initial condition is a Maxwell-Boltzmann distribution.
Figure 3: Self-organized plasma after entropy maximization. (a): spatial profile of particle density (a.u.). (b): temperature anisotropy . (c): parallel temperature . (d): perpendicular temperature . White, green, and purple lines represent contours of , , and .

This research was supported by JSPS KAKENHI Grant Nos. 23224014 and 15K13532.


  • (1) T. Hashimoto, K. Matsuzaka, E. Moses, and A. Onuki, Phys. Rev. Lett. 74, 1 (1995).
  • (2) J. W. Cahn, Acta Met. 9, 795-801 (1961).
  • (3) L. S. Darken, Trans. AIME 180, 430-438 (1949).
  • (4) A. Lauerer, T. Binder, C. Chmelik, E. Miersemann, J. Haase, D. M. Ruthven, and J. Karger, Nat. Comm. 6, 7697 (2015).
  • (5) C. E. Lesher, J. Geophys. Res. 99, B5 pp. 9585-9604 (1994).
  • (6) M. Schulz and L. J. Lanzerotti, Particle Diffusion in the Radiation Belts (Springer, New York, 1974).
  • (7) A. C. Boxer, R. Bergmann, J. L. Ellsworth, D. T. Garnier, J. Kesner, M. E. Mauel, and P. Woskov, Nature Phys. 6, 207 (2010).
  • (8) Z. Yoshida, H. Saitoh, J. Morikawa, Y. Yano, S. Watanabe, Y. Ogawa, Phys. Rev. Lett 104, 235004 (2010).
  • (9) N. Sato and Z. Yoshida, J. Phys. A: Math. Theor. 48, 205501 (2015).
  • (10) N. Sato, N. Kasaoka, and Z. Yoshida, Phys. Plasmas 22, (4) 042508 (2015).
  • (11) A rigorous definition of topological constraint based on the degeneracy of the Poisson algebra will be given later on. When the constraint is integrable, it is called a Casimir invariant in the context of non-canonical Hamiltonian mechanics. The correspondence with adiabatic invariants of classical mechanics is discussed in Z. Yoshida and P. J. Morrison, in Nonlinear physical systems: spectral analysis, stability and bifurcation, edited by O. N. Kirillov and D. E. Pelinovsky (ISTE and John Wiley and Sons, 2014), Chap. 18, pp. 401-419 and Z. Yoshida and S. M. Mahajan, Prog. Theor. Exp. Phys. 2014 073J01 (2014).
  • (12) D. Lynden-Bell and R. Wood, Mon. Not. R. Astron. Soc. 138, 495 (1968).
  • (13) P. H. Chavanis, Dynamics and Thermodynamics of Systems with Long-Range Interactions: An Introduction (Springer, 2002), pp. 208-289.
  • (14) A. Antoniazzi, D. Fanelli, S. Ruffo, and Y. Y. Yamaguchi. Phys. Rev. Lett. 99, 040601 (2007).
  • (15) R. Pakter and Y. Levin, Phys. Rev. Lett. 106, 200603 (2011).
  • (16) Z. Yoshida and S. M. Mahajan, Phys. Rev. Lett. 88, 095001 (2002).
  • (17) E. Schrodinger, What is life- the physical aspect of the living cell, (Cambridge University Press, 1944).
  • (18) T. Ikegami and M. Oka, ICAART 2014, pp. 237-242 (2014).
  • (19) E. T. Jaynes, Probability theory the logic of science (Cambridge University Press, 2003), pp. 374-376.
  • (20) E. T. Jaynes, Phys. Rev. 106, 4 pp. 620-630 (1957).
  • (21) T. J. Birmingham, T. G. Northrop, and C. G. Falthammar, Phys. Fluids 10, 11 (1967).
  • (22) A. Hasegawa, Phys. Scr. T 116, 72-74 (2005).
  • (23) This identity is satisfied by any Poisson operator and is the essential feature of the algebraic structure of Hamiltonian systems. Defining the Poisson bracket of two functions and as , the identity for three functions , , and reads .
  • (24) P. J. Morrison, Rev. Mod. Phys. 70, 2 (1998).
  • (25) If the rank of is and the Jacobi identity is satisfied, there will be Casimir invariants. Again, this is a consequence of Darboux’s theorem Morrison1998 ().
  • (26) If with the magnetic field, and setting , with the electric potential, one obtains the equations of motion for a magnetized particle performing drift. In vacuum, for some pontetial . In this case, is a Casimir invariant.
  • (27) In D the Jacobi identity reads as . When satisfied, the constraint is integrable Szekeres (), and thus two scalars and can be found such that . Furthermore, defines an invariant measure since , with the result that, in , Jacobi identity, Casimir invariant, and invariant measure imply each other provided that .
  • (28) P. Szekeres, A Course in Modern Mathematical Physics, (Cambridge University Press, 2004), pp. 454-455.
  • (29) C. W. Gardiner, Handbook of Stochastic Methods for Physics, Chemistry and the Natural Sciences, 2nd ed., (Springer-Verlag, 1985).
  • (30) N. Sato, Z. Yoshida, and Y. Kawazura, Plasma Fus. Res. 11, 2401009 (2016).
Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
Add comment
Loading ...
This is a comment super asjknd jkasnjk adsnkj
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test description