Entropy production and the geometry of dissipative evolution equations
Purely dissipative evolution equations are often cast as gradient flow structures, , where the variable of interest evolves towards the maximum of a functional according to a metric defined by an operator . While the functional often follows immediately from physical considerations (e.g., the thermodynamic entropy), the operator and the associated geometry does not necessarily so (e.g., Wasserstein geometry for diffusion). In this paper, we present a variational statement in the sense of maximum entropy production that directly delivers a relationship between the operator and the constraints of the system. In particular, the Wasserstein metric naturally arises here from the conservation of mass or energy, and depends on the Onsager resistivity tensor, which, itself, may be understood as another metric, as in the Steepest Entropy Ascent formalism. This new variational principle is exemplified here for the simultaneous evolution of conserved and non-conserved quantities in open systems. It thus extends the classical Onsager flux-force relationships and the associated variational statement to variables that do not have a flux associated to them. We further show that the metric structure is intimately linked to the celebrated Freidlin-Wentzell theory of stochastically perturbed gradient flows, and that the proposed variational principle encloses an infinite-dimensional fluctuation-dissipation statement.
pacs:46.05.+b, 05.70.Ln, 05.40.-a
Dissipative evolution equations (e.g., heat conduction, mass diffusion, interface motion) often follow variational principles, such as Onsager’s least dissipation of energy Onsager (1931a, b) and extensions, in particular those based on maximum entropy production (MEPPs Martyushev and Seleznev (2006); Dewar et al. (2013)) or Steepest Entropy Ascent (SEA) Beretta (1987, 2014); Montefusco et al. (2014)). Mathematically, these equations are often of gradient flow type, that is, they can be described by the steepest ascent/descent of a functional, such as the entropy. Here descent has to be measured in a metric, which is neither provided by the aforementioned variational approaches, nor it is always intuitive (e.g., Wasserstein metric for diffusion processes). In this article, we establish a variational framework based on the ansatz of maximal entropy production which sheds light on the geometry of purely dissipative evolution equations. This new approach (1) delivers a construction of the gradient flow metric from conservation constraints in the variational formulation; (2) extends Onsager’s principle to simultaneously account for conserved and non-conserved quantities in open systems; and (3) encloses an infinite-dimensional fluctuation-dissipation statement, as shown from a large deviation argument for stochastically perturbed gradient flows. The diagram of Fig. 1 summarizes the connections established in this paper.
We sketch some of the most closely related variational principles and provide a short summary on gradient flows. The body of literature, both classic and recent, on these two topics is too large to be reviewed comprehensively here.
i.1 Entropy production
Onsager, in his celebrated papers Onsager (1931a, b) generalized the transport laws, such as those by Fourier, Ohm or Fick, to account for a possible coupling between different physical processes. He proposed a general linear kinematic constitutive relation between fluxes and forces , that is, . The conductivity matrix may depend on the state variables (temperature, pressure, chemical potential, etc.), but not on their gradient Gyarmati (1970), and is symmetric as a result of the time reversal of the underlying atomistic equations of motion, . These two properties of the constitutive relations — linearity and symmetry of the conductivity tensor — can be equivalently expressed by means of the principle of least dissipation of energy Onsager (1931a) (following Rayleigh’s nomenclature Rayleigh (1913)). Namely, let be the entropy production and denote a local dissipation potential, with the resistivity tensor being positive definite, then the variational principle reads
In Onsager’s words Onsager (1931a), ‘the rate of increase of the entropy plays the role of a potential’. Several generalizations of this extremum principle have since emerged in different fields encompassing climate Paltridge (1975), soft matter physics Doi (2011), plasticity Ziegler (1983), biology Dewar (2010) and quantum mechanics Beretta (1981) among others, and appear under the names of Maximum Entropy Production Principles(MEPPs) Martyushev and Seleznev (2006); Dewar et al. (2013) and Steepest Entropy Ascent (SEA) Beretta (1987, 2014); Montefusco et al. (2014). This latter framework provides a geometric interpretation of the resistivity tensor and generalizes to arbitrary (but a priori unknown) metric spaces. Another approach to nonequilibrium thermodynamics, which combines reversible and irreversible dynamics, is the General Equation for NonEquilibrium Reversible-Irreversible Coupling (GENERIC) Grmela and Öttinger (1997); Öttinger (2005). The structure of this formalism can be derived using contact forms in the setting of the Gibbs-Legendre manifold Grmela (2014, 2015); it can be cast variationally; and it allows for a systematic multiscale approach Grmela et al. (2015) as well as a treatment of fluctuations Grmela (2014, 2012).
i.2 Gradient flow structures
From a mathematical perspective, purely dissipative evolution equations can often be described as gradient flow structures Ambrosio et al. (2006). This means that the vectorial variable of interest (components are, for example, energy, density or interface position) evolves according to the steepest ascent of a functional (or descent for ) in a geometry given by a metric associated with a positive semi-definite operator ,
where if the inverse is defined, and is a force. Note that (2) is precisely the irreversible component of GENERIC. Then is a Lyapunov functional, , where denotes the dual parity between elements of the tangent and the cotangent space.
respectively, with and positive semi-definite. The latter equation is symbolically expressed as , with . Further details on the weak formulations of both flows and the norms involved are given in the Appendix.
It is noteworthy that the same equation can have different gradient flow representations. For example, the diffusion equation
can be interpreted both as flow (with mobility and Dirichlet integral ) and Wasserstein flow (with mobility and Boltzmann entropy ). The Wasserstein formulation is a natural choice since it involves the physical entropy. This flow and its associated metric will be automatically singled out by the variational principle proposed here, as we show next.
Ii Entropy production and deterministic evolution
In this section, we present a new variational principle for purely dissipative evolution equations based on the ansatz that systems evolve in the direction of maximum entropy production (see Eq. (9) below) 111The entropy production can be expressed as the entropy rate of the system minus the entropy increase induced by heat flux exchange with the ambient space Glansdorf and Prigogine (1971)., so as to reach the equilibrium configuration as fast as possible. The philosophy is therefore similar to SEA and MEPPs, yet different in its detailed formulation. In particular, the proposed principle will provide a direct relation between the operator and physical constraints in the system, thus shedding some light on the geometry of dissipative equations.
For simplicity, we first consider closed systems defined by a scalar variable and later generalize the obtained results to open systems and the vectorial setting. Illustrative examples are then chosen to demonstrate the applicability of the principle for both conserved and non-conserved fields, with explicit consideration of the boundary conditions. We note that non-conserved quantities do not have a flux associated to them, and therefore lie outside of the direct scope of Onsager’s principle (1).
For a closed system out of equilibrium characterized by a scalar state variable , the maximum entropy production ansatz is mathematically equivalent to the search of the velocity maximizing , where is the entropy density and the total entropy of the system. The maximization is pointwise in the tangent space for fixed , c.f. Fig. 2. However, this problem is not well-posed unless the length of the vector is prescribed, in which case the problem is reduced to the search of the optimal direction. This constraint is easily incorporated with a Lagrange multiplier, yielding a variational principle with Lagrangian
where the precise value of the length, which may depend on , has been obviated since it does not participate in variations for fixed . The evolution is then obtained by variations of (6) with respect to , giving , with , since entropy would decrease otherwise. This shows that the gradient flow (3) with functional naturally results from the maximum entropy production principle in the absence of any physical constraint.
However, the evolution of is often subjected to conservation constraints of the form
which naturally occurs when represents mass or energy. In this situation, the maximal dissipation occurs within the manifold of conserved ,
where on the boundary for a closed system. With an additional Lagrange multiplier , the variational principle at each point can then be written as
where the length constraint (measured with metric tensor ) has now been placed on the unknown variable . We note that constraining the length of as in (6) would leave partially undetermined, and so would be the constitutive relations, such as Fourier’s law for the case of heat conduction.
Variation with respect to in (7) delivers
which, after integration by parts, yields . Variations with respect to and give
Altogether, this leads to a Wasserstein gradient flow with functional and weight positive semi-definite, . The Wasserstein gradient flow (4) can be thus be understood as an gradient flow restricted to the manifold of conserved quantities.
In general, systems are characterized by a set of state variables , some of which are conserved, , (e.g., energy, concentration), and some of which are not, , (e.g., interface position), i.e., . In this case the variational principle can be written as
where now is a vectorial Lagrange multiplier, and and are second-order tensors. Similar derivations as above yield the evolution equations
which have an analogous structure to those previously obtained. However, for anisotropic materials, coupling between variables of different tensorial quantities is possible, and in this case, the Lagrangian shall be written as
Variations of this functional with respect to and give
with . Then, the evolution equations read
where the symbol indicates how the operator is applied to the vector . Further, , i.e.,
This simple viewpoint of dissipative evolution equations via constrained maximization will be exemplified below for the equation of heat transfer and interface motion in open system, as blueprint for the derivation of other equations in a similar manner.
Example: the heat equation and Fourier’s law.
We now show that Fourier’s law and the heat equation follow directly from the postulate of maximum entropy production. For an open system, the Lagrangian of the maximum entropy production principle is constructed by subtracting the entropy flow entering the boundary of the domain from the total entropy rate. Then the entropy increase considered exclusively originates from the internal production, in accordance with the second law of thermodynamics. Assuming that the system is completely characterized by the internal energy, and taking also the conservation of energy into account, the Lagrangian reads
where is the outer normal to the domain, is the heat flux, and and represent the entropy and energy per unit volume, respectively. From basic thermodynamic relations, assuming local thermodynamic equilibrium, . Therefore, variations with respect to , and , assuming boundary conditions in (boundary conditions in would imply on , and lead to the same evolution equation) yield
which combined give the equation of heat transfer, with ,
Example: Interface motion in an isotropic medium.
Next, we consider a two-phase system separated by an interface, which we characterize by an additional variable in the spirit of a phase field model (Provatas and Elder, 2011). Following a similar strategy as in the previous case, the evolution of the interface coupled to the heat equation can be obtained as the extremum of
Assuming the existence of a thermodynamic relation for the energy density of the form ,
where subscripts indicate the variables that are held fixed. Its Legendre transform with respect to the entropy density is the Helmholtz free energy ,
One then obtains
As in the previous example, we obtain the heat equation from variations of with respect to and , while variations with respect to yield the evolution of the interface,
Thus the interface is driven by the Massieu potential , whose relevance has been noted in SEA Beretta (2006, 2007, 2009), in the GENERIC setting Mielke (2011) as well as in large deviation theory Touchette (2009). We note that the derivation of this evolution in the Onsager formalism is nontrivial as does not have a flux and a corresponding thermodynamic force.
Iii Stochastic evolution and large deviations
In this section, we show that the proposed variational formulation for purely dissipative equations based on physical considerations is further supported by a large deviation principle (LDP) associated to stochastically perturbed gradient flows. The LDP provides the probability of a given evolution to occur, and therefore intrinsically contains a variational principle for the most likely path. Large deviation arguments have recently been used to connect particle models to gradient flows, for example in Adams et al. (2011, 2013); Mielke et al. (2014), and have also led to variational formulations of systems in GENERIC form Duong et al. (2013).
Specifically, let be a vector field that evolves in according to a stochastic gradient flow with small noise,
where is a vector of independent Brownian sheets, i.e., , with the Kronecker delta function and the Dirac delta function. Further, is an operator acting on , and is a small parameter controlling the strength of the noise. The stochastic calculus is to be understood in the Itô sense.
The probability distribution for satisfying (13) may be obtained from that of simpler processes using the theory of large deviations and the contraction principle Freidlin and Wentzell (1984). Indeed, by Schilder’s theorem, the probability distribution of the solutions to the vectorial ordinary differential equation , with a vector of time white noises, , follows
is called the rate functional. In words, the probability for undergoes an exponential decay with rate , and narrows as around the deterministic solution . Then, the probability distribution for satisfying can be obtained by expanding and with orthonormal basis functions for the domain Faris and Jona-Lasinio (1982); Freidlin (1988),
where are independent Brownian motions (direct computations show that ). The partial differential equation is then equivalent to the system of vectorial ordinary differential equations ; and the rate functional of the associated large deviation principles, for (see, e.g., Faris and Jona-Lasinio (1982); Freidlin (1988)), can be readily obtained from (14)
The solutions to (13) can be seen as , where is an operator. If is continuous (see Budhiraja et al. (2008) for measurable functions), then, by the contraction principle, follows a large deviation principle Sowers (1992) with functional , i.e.,
assuming defines a norm, with being the adjoint operator of . This result follows the spirit of Onsager and Machlup Onsager and Machlup (1953), for general gradient flow structures; however, the probability distribution obtained is not a function of the thermodynamic forces and fluxes as in the original formulation by Onsager, but of the variable and . This difference is analogous to that of (10) and (12).
Iv Maximum entropy production from large deviations
Equation (17) shows that the most likely path is the one that maximizes the exponent and thus minimizes . This minimum is attained by pointwise optimization (over for fixed at every instant of time), giving
Equation (18) represents a variational principle for the deterministic gradient flow, which, for , is shown below to be equivalent to Eq. (6) for gradient flows, to Eq. (7) for the Wasserstein evolution, and to Eqs. (8) and (9) for the combined vectorial case. Indeed, expanding the squares in Eq. (18) yields the variational problem
with and , where the latter does not affect the optimal evolution. One has in the presence of fluctuations, whereas for the optimal path holds.
with , and as defined in the Appendix. An equivalent result is obtained for the Wasserstein gradient flow (), noting that the last term of Eq. (7), with and , can be rewritten as
with . For the coupled case considered in Eq. (9), is a full matrix and its inverse reads
where the inverted divergence and inverted gradient are to be interpreted in appropriate spaces. The relations immediately follow from . Then, one similarly obtains that
The variational principles of Eqs. (6)–(9) can therefore be written as . We thus observe that the diagram of Fig. 1 commutes if , which represents a fluctuation-dissipation relation in infinite dimensions.
Square root of the Wasserstein operator.
We now discuss the expression encountered in the fluctuation-dissipation statement above for the Wasserstein operator. In general, for a given positive semi-definite self-adjoint there are several choices . However, only appears in the generator and thus the solutions to the corresponding Fokker-Planck equations for different roots are statistically equivalent Öttinger (1996). For Wasserstein gradient flows we only consider of divergence form, to have a conservative noise, i.e., , where for simplicity. Then, for the Wasserstein metric with mobility
We provide two independent derivations of a variational principle governing dissipative evolution equations of the form . The first is based on the maximization of the entropy production within the manifold of constraints, extending Onsager’s original approach, and provides insight into the geometry of the gradient flow structure (). In particular, the principle captures multiple metrics: one which is related to a thermodynamic length, and others that may result from the constraints in the system, such as conservation of mass or energy. The first metric is here taken as the metric and is in principle unknown (an extension to general metrics, as in SEA, is yet to be explored), whereas the second one is an outcome of the variational statement. By means of this procedure, the Wasserstein metric is here shown to be equivalent to the constrained metric associated to conserved fields. The second approach for obtaining the variational statement is based on the large deviation principle for the gradient flows augmented by a noise term , and is shown to be equivalent to the previously derived principle for . This represents a fluctuation-dissipation relation in infinite dimensions and endows the exponent of the large deviation principle with the usual interpretation of an entropy (dissipation) shortfall between a given path and the optimal one Varadhan (2010).
We write the weighted norm as , and denote and (note that for square integrable functions, is equivalent to the duality pairing). Then the weak formulation of the gradient flow for the diffusion equation is ( and in Eq. (3))
For the Wasserstein gradient flow, if with on , the Wasserstein norm is
The second expression is known as the seminorm with weight , . We write (see (Feng and Kurtz, 2006, Appendix D) for details)
With this notation, it is straightforward to calculate the weak formulation of the diffusion equation as a Wasserstein gradient flow (, and in Eq. (4)),
Acknowledgments. The authors thank M. von Renesse, P. Ayyaswamy, D. Kelly, R. Jack, V. Maroulas, M. Renger and E. Vanden-Eijnden for valuable comments. This work was partially supported by the UK’s Engineering and Physical Sciences Research Council Grant EP/K027743/1 (to JZ), the Leverhulme Trust (RPG-2013-261) and GW4 grants GW4-IF2-026 and GW4-AF-005. We appreciate helpful suggestions from the reviewers.
- Onsager (1931a) L. Onsager, Phys. Rev. 37, 405 (1931a).
- Onsager (1931b) L. Onsager, Phys. Rev. 38, 2265 (1931b).
- Martyushev and Seleznev (2006) L. Martyushev and V. Seleznev, Physics reports 426, 1 (2006).
- Dewar et al. (2013) R. C. Dewar, C. H. Lineweaver, R. K. Niven, and K. Regenauer-Lieb, Beyond the Second Law (Springer, 2013).
- Beretta (1987) G. P. Beretta, in The Physics of Phase Space Nonlinear Dynamics and Chaos Geometric Quantization, and Wigner Function (Springer, 1987) pp. 441–443.
- Beretta (2014) G. P. Beretta, Physical Review E 90, 042113 (2014).
- Montefusco et al. (2014) A. Montefusco, F. Consonni, and G. P. Beretta, arXiv preprint arXiv:1411.5378 (2014).
- Gyarmati (1970) I. Gyarmati, Non-equilibrium Thermodynamics. Field Theory and Variational Principles (Springer, 1970).
- Rayleigh (1913) L. Rayleigh, The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science 26, 776 (1913).
- Paltridge (1975) G. W. Paltridge, Quarterly Journal of the Royal Meteorological Society 101, 475 (1975).
- Doi (2011) M. Doi, Journal of Physics: Condensed Matter 23, 284118 (2011).
- Ziegler (1983) H. Ziegler, An introduction to thermomechanics (Elsevier, 1983).
- Dewar (2010) R. C. Dewar, Philosophical Transactions of the Royal Society B: Biological Sciences 365, 1429 (2010).
- Beretta (1981) G. Beretta, On the general equation of motion of quantum thermodynamics and the distinction between quantal and nonquantal uncertainties, Ph.D. thesis, Massachusetts Institute of Technology (1981).
- Grmela and Öttinger (1997) M. Grmela and H. C. Öttinger, Physical Review E 56, 6620 (1997).
- Öttinger (2005) H. C. Öttinger, Beyond equilibrium thermodynamics (John Wiley & Sons, 2005).
- Grmela (2014) M. Grmela, Entropy 16, 1652 (2014).
- Grmela (2015) M. Grmela, Entropy 17, 5938 (2015).
- Grmela et al. (2015) M. Grmela, V. Klika, and M. Pavelka, Physical Review E 92, 032111 (2015).
- Grmela (2012) M. Grmela, Physica D: Nonlinear Phenomena 241, 976 (2012).
- Ambrosio et al. (2006) L. Ambrosio, N. Gigli, and G. Savaré, Gradient flows: in metric spaces and in the space of probability measures (Springer, 2006).
- Jordan et al. (1998) R. Jordan, D. Kinderlehrer, and F. Otto, SIAM J. Math. Anal. 29, 1 (1998).
- (23) The entropy production can be expressed as the entropy rate of the system minus the entropy increase induced by heat flux exchange with the ambient space Glansdorf and Prigogine (1971).
- Provatas and Elder (2011) N. Provatas and K. Elder, Phase-field methods in materials science and engineering (John Wiley & Sons, 2011).
- Beretta (2006) G. P. Beretta, Physical Review E 73, 026113 (2006).
- Beretta (2007) G. P. Beretta, International Journal of Quantum Information 5, 249 (2007).
- Beretta (2009) G. P. Beretta, Reports on Mathematical Physics 64, 139 (2009).
- Mielke (2011) A. Mielke, Cont. Mech. Thermodyn. 23, 233 (2011).
- Touchette (2009) H. Touchette, Phys. Rep. 478, 1 (2009).
- Adams et al. (2011) S. Adams, N. Dirr, M. A. Peletier, and J. Zimmer, Comm. Math. Phys. 307, 791 (2011).
- Adams et al. (2013) S. Adams, N. Dirr, M. Peletier, and J. Zimmer, Philos. Trans. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci. 371, 20120341, 17 (2013).
- Mielke et al. (2014) A. Mielke, M. A. Peletier, and D. R. M. Renger, Potential Anal. 41, 1293 (2014).
- Duong et al. (2013) M. H. Duong, M. A. Peletier, and J. Zimmer, Nonlinearity 26, 2951 (2013).
- Freidlin and Wentzell (1984) M. I. Freidlin and A. D. Wentzell, Random perturbations of dynamical systems (Springer-Verlag, New York, 1984).
- Faris and Jona-Lasinio (1982) W. G. Faris and G. Jona-Lasinio, J. Phys. A 15, 3025 (1982).
- Freidlin (1988) M. I. Freidlin, Trans. Amer. Math. Soc. 305, 665 (1988).
- Budhiraja et al. (2008) A. Budhiraja, P. Dupuis, and V. Maroulas, Ann. Prob. , 1390 (2008).
- Sowers (1992) R. Sowers, Probab. Theory Related Fields 92, 393 (1992).
- Onsager and Machlup (1953) L. Onsager and S. Machlup, Physical Review 91, 1505 (1953).
- Öttinger (1996) H. C. Öttinger, Stochastic processes in polymeric fluids: tools and examples for developing simulation algorithms (Springer Berlin, 1996).
- Eyink (1990) G. L. Eyink, J. Stat. Phys. 61, 533 (1990).
- Dean (1996) D. Dean, J. Phys. A 29, L613 (1996).
- Kawasaki (1998) K. Kawasaki, J. Stat. Phys. 93, 527 (1998).
- Chavanis (2011) P.-H. Chavanis, Phys. A 390, 1546 (2011).
- Varadhan (2010) S. R. S. Varadhan, in Proceedings of the International Congress of Mathematicians. Volume I (Hindustan Book Agency, New Delhi, 2010) pp. 622–639.
- Feng and Kurtz (2006) J. Feng and T. G. Kurtz, Large deviations for stochastic processes, Mathematical Surveys and Monographs, Vol. 131 (AMS, Providence, RI, 2006).
- Glansdorf and Prigogine (1971) P. Glansdorf and I. Prigogine, Structure, stability and fluctuations (1971).