1 Introduction

# Exact equalities and thermodynamic relations for nonequilibrium steady states

## Abstract

We study thermodynamic operations which bring a nonequilibrium steady state (NESS) to another NESS in physical systems under nonequilibrium conditions. We model the system by a suitable Markov jump process, and treat thermodynamic operations as protocols according to which the external agent varies parameters of the Markov process. Then we prove, among other relations, a NESS version of the Jarzynski equality and the extended Clausius relation. The latter can be a starting point of thermodynamics for NESS. We also find that the corresponding nonequilibrium entropy has a microscopic representation in terms of symmetrized Shannon entropy in systems where the microscopic description of states involves “momenta”. All the results in the present paper are mathematically rigorous.

Exact equalities and thermodynamic relations

Teruhisa S. Komatsu1, Naoko Nakagawa2, Shin-ichi Sasa3, and Hal Tasaki4

## 1 Introduction

### 1.1 Background and motivation

General Properties of physical systems in thermal equilibrium are relatively well understood both from physical and mathematical points of view. Thermodynamics characterizes macroscopic properties of equilibrium states, and poses strong constraints on possible transitions between equilibrium sates, especially when an outside agent makes operation to the system. Statistical mechanics provides a probabilistic description of equilibrium states based on the microscopic mechanical description of the system.

To develop similar universal theories for systems out of equilibrium has been a major remaining challenge in theoretical physics. Among various classes of nonequilibrium states, nonequilibrium steady states (which we shall abbreviated as NESS throughout the paper), which have no macroscopic changes but have nonvanishing flows, may be a promising ground for developing such theories.

Looking back into the history of equilibrium physics, we find that the thermodynamic definition of entropy (and free energy) in terms of thermodynamic operation was a fundamental starting point. The notion and the properties of the entropy and the free energy were essential guides for the later development of equilibrium statistical mechanics. If we trust the analogy to the history, one possible route toward universal theories for NESS may be to start from a thermodynamics for NESS and pin down relevant thermodynamic functions. In [1], Oono and Paniconi made a proposal of an operational thermodynamics for NESS and coined the term “steady state thermodynamics” (SST).

We should point out however that the notion of entropy for NESS is much more subtle than one might imagine in the beginning. One reason is that the entropy in equilibrium physics plays several essentially different roles; it is a thermodynamic function whose derivatives correspond to physically observable quantities, it is a quantitative measure of which adiabatic process is possible and which is not, it is also a large deviation functional governing the fluctuation of physical quantities. There is a possibility that this “degeneracy” is an accident observed only in equilibrium states, and the degeneracy is immediately lifted when one goes out of equilibrium. If this is the case we shall encounter more than one “nonequilibrium entropies” each of which characterizing different physical aspect of a NESS. It will then be important to clarify which physics of NESS is represented by which extension of entropy.

In our own attempt to develop SST and to extend the notion of entropy to NESS [2, 3], we have concentrated on the aspect which gave birth to the concept of entropy, namely, the Clausius relation in thermodynamics. We started from a microscopic description of a heat conducting NESS, and showed that a very natural generalization of the Clausius relation, in which the heat is replaced with its “renormalized” counterpart, is valid when the “order of nonequilibrium” is sufficiently small. This was a realization of the early phenomenological discussions by Landauer [4] and by Oono and Paniconi [1], and an extension of the similar result by Ruelle [5] for models with Gaussian thermostat. We also found that, in systems where microscopic states have no time-reversal symmetry (i.e., the microscopic description of states involves “momenta”), the microscopic representation of the entropy differs from the traditional Gibbs-Shannon form, and requires further symmetrization with respect to the time-reversal transformation.

This approach to SST was later extended to quantum systems [6]. A geometric interpretation of the thermodynamic relations and corresponding exact relations were discussed in [7, 8] both for classical and quantum systems.

Other schemes for SST, which are distinct from those in [2, 3], have been proposed [9, 10, 11, 13, 14]. See the end of section 1.2 for details. See also [15] for an earlier attempt at approaching SST from a phenomenological point of view, and [16, 17, 18] for discussions about the “zeroth law” in SST. Closely related problem of heat capacity in NESS is discussed in [19, 20]. See [21] for a unified treatment of thermodynamics in NESS and adiabatic pumping in equilibrium.

Among other various promising attempt at discussing entropy (or a related quantity) in NESS, let us refer to the macroscopic fluctuation theory developed by Bertini, De Sole, Gabrielli, Jona-Lasinio, and Landim [22, 23, 24, 25], exact solution of the large deviation functional by Derrida, Lebowitz, and Speer [26, 27], the proposal of the additivity principle by Bodineau and Derrida [28], and the recent interesting proposal based on adiabatic accessibility by Lieb and Yngvason [29]. As for closely related approaches based on “fluctuation theorems” see, e.g., the recent review [30].

In the present paper, which is the first mathematical paper in our series of works on SST, we treat a general class of Markov jump processes that model nonequilibrium systems, and prove physically important relations including the NESS version of the Jarzynski equality and the extended Clausius relation. Although most of the results have been announced before, they were all derived heuristically. We here present mathematically rigorous results for the first time.

Let us describe the organization of the present paper5.

The following section 1.2 has been prepared for the readers who are not familiar with our approach (and other related approaches) to nonequilibrium thermodynamics. We shall motivate our study by briefly describing the standard Clausius relation in equilibrium thermodynamics in a simple setting, and explaining the difficulties one encounters when trying to extending it naively to NESS. We then describe our scheme of “renormalization” and introduce the extended Clausius relation for NESS. Finally we compare our approach with other proposals of thermodynamics for NESS.

In section 2, we shall give almost complete definitions necessary in the present paper. In sections 2.1 and 2.2, we define the Markov jump process and the corresponding description in terms of paths. These definitions are standard. Of particular importance are the notion of protocol and the expectation value defined in (2.12). In section 2.3, we define essential “thermodynamic” quantities, namely, the entropy production , its nonequilibrium part , and the work . We also discuss the experimental measurability of these quantities. We shall discuss some examples in section 2.4.

In section 3 we discuss the Jarzynski type equality (3.1) that holds for thermodynamic operations between two NESS. The equality is exact and rigorous, and will be the basis of our main result, namely the extended Clausius relation. Let us stress that our Jarzynski type equality (3.1) for NESS is distinct from existing exact equalities for general stochastic processes in that it contains only (almost) measurable thermodynamic type quantities. It is challenging to design experimental verification of the equality (3.1) with modern techniques in calorimetry.

In section 4, we shall be heuristic, and describe thermodynamic relations that can be derived from the rigorous equality (3.1). In section 4.1, we start from the most important extended Clausius relation, and also discuss a different representation of the relation in terms of the excess entropy production. We further discuss higher order “thermodynamic” relations in section 4.2, and present a heuristic estimate of error terms in section 4.3.

In section 5, which is a core of the present paper, we discuss rigorous versions of the thermodynamic relations without going into the proofs. After fixing the class of models for simplicity, we state Theorem 5.1 which allows us to identify (with a certain precision) our nonequilibrium entropy with the Shannon entropy. Then in Theorems 5.2 and 5.3, we state the extended Clausius relation and its higher order generalizations for the step protocol and the quasi-static protocol, respectively. The extended Clausius relation written in terms of the excess entropy production is stated in Theorem 5.4. Finally in Theorem 5.5, we state an inequality corresponding to the extended Clausius relation.

Sections 6, 7, 8, and 9 are devoted to the proofs of the theorems.

In section 6, we present arguments based on time-reversal symmetry to prove exact equalities discussed in section 3. The arguments are basically standard, but our Jarzynski type equality for NESS is proved by using a new statement which we call “splitting Lemma” (Lemma 6.1).

In section 7 we present totally different approach based on the method of modified rate matrix. This method is used in section 7.2 to prove the splitting lemma, and in section 7.4 to justify the heuristic estimates in section 4.3 of the error in the thermodynamic relations. This completes the proof of the extended Clausius relation and the related relations stated in Theorems 5.2, 5.3, and 5.4.

In section 8, we shall use the results from the previous sections to prove Theorem 8.1 about a useful and suggestive representation (first written down by two of us, T.S.K. and N.N.) of the probability distribution of NESS. Then this representation is used to prove Theorem 5.1 about the nonequilibrium entropy.

In section 9, we prove Theorem 5.5 about the extended Clausius inequality for NESS. The proof makes use of the standard argument based on the relative entropy and a rigorous version of the linear response formula stated as Lemma 9.1.

In the final section 10, we discuss a slightly different class of models which include “momenta”. We show that essentially all the results (except for the extended Clausius inequality) automatically extend to this situation if one replaces the Shannon entropy with a new quantity (10.7) called the symmetrized Shannon entropy. The symmetrized Shannon entropy has a very suggestive form and might be a key for further understanding of the essential properties of NESS in systems with momenta. In section 10.3, we discuss a simple toy model which illustrates the need of the symmetrized Shannon entropy.

### 1.2 SST in a typical example

Here we shall briefly discuss the essence of SST, i.e., the extended Clausius relation in the simplest example of heat conducting system. We also try to place our work in the broader context of thermodynamics and statistical mechanics by discussing relevant background.

#### Clausius relation for operation between equilibrium states:

To motivate our extended Clausius relation, we first review the standard Clausius relation, which is the starting point of equilibrium thermodynamics. Consider a physical system (which can be basically anything) attached to a single heat bath whose temperature can be controlled. See Fig. 1. When the inverse temperature of the bath is fixed at , the system settles to the equilibrium state corresponding to after a sufficiently long time. Recall that a physical system in equilibrium exhibits no macroscopic changes, and has no macroscopic flows (of, e.g., matter or energy).

We next consider a thermodynamic operation. We start from a situation where the inverse temperature of the bath is and the system is in the corresponding equilibrium. Then we change the inverse temperature of the bath according to a protocol fixed in advance, i.e., a smooth function of time where and . We shall assume that is large and varies slowly. Let be the heat flux (the energy that flows within a unit time) from the bath to the system. Then the well known Clausius relation is

 S(β′)−S(β)≃∫τ−τdtβ(t)J(t), (1.1)

where is the entropy of the system in the equilibrium state corresponding to . (More precisely, the entropy is a function of the equilibrium state.) The relation (1.1) becomes an exact equality in the limit where varies infinitesimally slowly. The equality can be proved mathematically in various setting.

We note that times the right-hand side of (1.1) is interpreted as the total entropy production in the heat bath. To see this, take a short time interval from to . The energy (heat) that flows into the bath during this interval is . Then the corresponding increase (or production) of entropy in the bath is given by the standard relation , which becomes the minus of (1.1) after integration over the whole process.

Although the entropy was introduced above as a purely thermodynamic (or macroscopic) quantity, there is a neat expression in terms of microscopic probability distribution. If one represents the equilibrium state in terms of the canonical distribution (see section 2.1 for the notation), the same entropy is written as

 S(β)=−∑xρβxlogρβx, (1.2)

where the right-hand side is nothing but the Shannon entropy of the probability distribution .

#### Extended Clausius relation for operation between NESS:

In our approach to thermodynamics for NESS, we wish to focus on possible extensions of the Clausius relation (1.1) and the expression (1.2) of the entropy. The hope is that proper extensions might be a starting point of a full-fledged thermodynamics.

To be specific, we focus on a NESS in a heat conducting system, which is a typical nonequilibrium setting. Consider a system which is attached to two large heat baths whose temperatures can be controlled. See Fig. 2.

Suppose first that the inverse temperatures of the baths are fixed at and , respectively. It is expected that, after a sufficiently long time, the system settles to a stationary state which has a steady temperature gradient and a constant heat current through it. Such a state, which has no macroscopically observable changes, but has a nonvanishing flow of energy, is a typical example of NESS.

As in the case of equilibrium, we consider a thermodynamic operation to NESS. We start from the situation where the two heat baths have fixed inverse temperatures and , and the system is in the corresponding NESS. Then we change the inverse temperatures of the baths according to a fixed protocol, i.e., functions and of time . We write , , , and .

We wish to ask whether a relation analogous to the Clausius relation (1.1) holds in this setting. Since there are two heat baths, we need to consider two heat currents separately. By , where , we denote the heat flux from the -th bath to the system at time . Then a naive analogue to (1.1) is

 S(β′1,β′2)−S(β1,β2)\lx@stackrel?≃∑k=1,2∫τ−τdtβk(t)Jk(t), (1.3)

where the minus of the right-hand side is the total entropy production in the two heat baths. Here is a certain function of two inverse temperatures , , which should be called the nonequilibrium entropy.

But it turns out that a relation like (1.3) can never be valid. This is most clearly seen by examining the right-hand side in the case where both and are independent of . Suppose that . In the NESS characterized by the two inverse temperatures and , there is a steady heat flux from the bath 1 to the bath 2 through the system. We thus have and for any . The right-hand side of (1.3) is thus equal to , which is negative and grows proportionally with . The left-hand side, on the other hand, is vanishing since . The relation (1.3) is clearly invalid.

More generally, fix the initial inverse temperature and the final inverse temperature (for ), and take reference functions of such that and . For a given , we choose our protocol as . Note that, when becomes large, the right-hand side of (1.3) diverges (roughly) proportionally to because there always is a heat current going through the system. On the other hand the left-hand side is independent of , because should be a function of the two inverse temperatures. We again conclude that the relation (1.3) cannot be valid.

#### Extended Clausius relation:

We need to find a way to “renormalize” the divergence in the right-hand side of (1.3) to get a finite quantity. One strategy is to introduce the reverse operation as follows. See Fig. 3. We start from the situation where the two heat baths have fixed inverse temperatures and , and the system is in the corresponding NESS. Then we change the inverse temperatures of the baths according to the reverse protocol defined by the functions for . Again we denote by , where , the heat flux from the -th bath to the system at time in this process. We expect when the operation is slow enough6. But we don’t have the exact equality in general since the currents at a given moment may depend on the history of the system. The subtle difference between and can be a key to understand the nature of NESS.

Since , the total entropy production in the baths
for the reverse protocol should diverge as in the same manner as that in the original protocol, i.e., the minus of the right-hand side of (1.3). This observation suggests that their difference may be finite in the limit , and may play a meaningful role. This is indeed the case, and we shall prove, for a class of models close to equilibrium, that the extended Clausius relation

 S(β′1,β′2)−S(β1,β2)≃12∑k=1,2{∫τ−τdtβk(t)Jk(t)−∫τ−τdtβ†k(t)J†k(t)}, (1.4)

holds in the limit . See Theorem 5.3 for the precise statement. Note that the unwanted divergence is properly “renormalized” by considering the difference between the total entropy productions (in the baths) in the original and the time-reversed operations. The same relation can be written by using the notion of excess entropy production as in [2, 3]. See Theorem 5.4.

The nonequilibrium entropy in (1.4) satisfies , where is the equilibrium entropy. Likewise the extended Clausius relation (1.4) reduces to the original Clausius relation (1.1) if the temperatures of the baths are always identical with each other, i.e., for any . We can say that our relation (1.4) is a natural extension of the original Clausius relation (1.1) to operations between NESS. We also stress that the right-hand side of (1.4) can be, in principle, measured experimentally; one needs to perform a pair of experiments for the original and the reverse protocols, and measure the heat currents from the two baths.

There are however (at least) two serious drawbacks in our theory. First the extended Clausius relation (1.4) is an approximate relation which is meaningful only when the system is close to the equilibrium. Our theory says nothing about systems which are very far from equilibrium. Secondly the extended Clausius relation (1.4) holds only for protocols where the parameters are varied very slowly. In the equilibrium thermodynamics, the Clausius inequality is known to hold for processes which are not necessarily slow. We can also prove an inequality corresponding to (1.4) (only for models without “momenta”), but it contains an error term which is not perfectly under control. See Theorem 5.5 and section 10.

#### Other schemes of “renormalization”:

Let us note that the above procedure is certainly not the unique way of “renormalizing” the divergent entropy production. For the moment, at least three other schemes of renormalization are known.

The scheme by Hatano and Sasa [9] developed for the overdamped Langevin system was the first realization of SST based on microscopic (or mesoscopic) dynamics. A very close, but slightly different, scheme based on macroscopic fluctuation theory was recently proposed by Bertini, Gabrielli, Jona-Lasinio, and Landim [10, 11]. The scheme due to Maes and Netocny [13] makes a full use of the large deviation analysis. See [13, 14] for discussions about the relations between different schemes.

Unlike our scheme, all these three schemes lead to extended Clausius relations (or analogous equivalent relations) which are exact for systems arbitrarily far away from equilibrium. Moreover, these equalities are accompanied by corresponding inequalities which are valid for general processes. These are clear advantages of the three schemes.

On the other hand, the renormalization in these three schemes requires subtraction of rather involved quantities which are not directly observable in experiments. In this sense our scheme, which uses only directly measurable quantities, has an advantage. We also note that our scheme applies to a larger class of models than the others. Although the Maes-Netocny scheme is based on microscopic (or mesoscopic) dynamics, it does not apply to models with inertia (momenta) as it is. As for the Hatano-Sasa scheme it has been pointed out [31] that a consistent thermodynamic interpretation is impossible once the momentum degrees of freedom is introduced. See also [14]. Among the four, ours seems to be the only scheme which provides a consistent thermodynamic relation in models including momenta (although we lack inequalities). See section 10.

For the moment we cannot say anything definite about which (or, even any) of the four schemes is most promising. We believe that further investigations from mathematical, theoretical, and experimental points of view are necessary.

## 2 Setup and definitions

Here we introduce general Markov jump processes that we study, and fix the notation. Quantities specific to our approach to nonequilibrium physics are introduced in section 2.3. We also describe typical examples in section 2.4.

### 2.1 Markov jump process

Let the state space be a finite set. The elements are states (in a suitable mesoscopic description) of the system. The probability distribution is denoted in vector notation as where is the probability to find the system in a state .

We assume that there is a set of parameters which characterizes the system. For concreteness we assume that takes its values in a compact subset of for some . Fix an arbitrary time scale . During the time interval , an external agent performs an operation to the system by controlling according to a protocol (i.e., a function of time ) (with ) which is fixed in advance. The function need not be continuous. We write the initial and the final values of the parameters as and , respectively. We also take which is much larger than , and consider the time evolution of the system in the longer time interval . We extend the protocol to the whole time interval by simply setting for and for . See Figure 4. We denote the whole protocol as . A special protocol in which takes a constant value throughout is denoted as .

We consider a Markov jump process characterized by a protocol .

For given parameters and such that , let be the transition rate from the state to . Physically speaking transitions in our system is caused by interactions between the system and heat baths attached to it. We assume that implies for any . We also assume that the whole state space is “connected” by nonvanishing . More precisely, for any with , one can take a sequence such that , , and for any . We also define the escape rate at by

 λαx:=∑y∈S(y≠x)Rαx→y. (2.1)

The Markov jump process corresponding to the protocol is defined by the master equation

 dpx(t)dt=−λα(t)xpx(t)+∑y∈S(y≠x)py(t)Rα(t)y→x, (2.2)

for any and , where is the probability to find the system in at time . The equation (2.2) is neatly rewritten in the vector notation as

 dp(t)dt=Rα(t)p(t), (2.3)

where is regarded as a column vector. The transition rate matrix7 is defined by specifying its entries as for and . The formal solution of (2.3) is written as

 p(t)=exp←[∫t−τdsRα(s)]pinit, (2.4)

where is the initial distribution given at . The time-ordered exponential is defined by

 exp←[∫t−τdsRα(s)] :=limN↑∞exp[(t+τ)Rα(sN−1)N]exp[(t+τ)Rα(sN−2)N]⋯exp[(t+τ)Rα(s0)N], (2.5)

with . When is time-independent, (2.5) coincides with the usual exponential .

It is a well known consequence of the Perron-Frobenius theorem that, for any parameter , one has

 lims↑∞exp[sRα]pinit=ρα, (2.6)

where is an arbitrary initial probability distribution. Here is the unique stationary probability distribution characterized by the condition . It is also known that for any . Physically speaking is the probability distribution for the nonequilibrium steady state (NESS) of the system with constant parameters .

### 2.2 Description in terms of paths

It is sometimes more convenient to describe the Markov jump process in terms of a path (or a history) of the state. A path is naturally identified with a piecewise constant function , but we shall often specify it in terms of the history of jumps as

 ^x=(n,(x0,x1,…,xn),(t1,t2,…,tn)), (2.7)

where is the total number of jumps, (such that for ) are the states that the system has visited, and is the time at which the jump took place. They are ordered as , and we often write and . We also write and as and , respectively. See Figure 5.

Then the weight (more precisely, the transition probability density) associated with a path is

 T^α[^x]:=n∏j=1Rα(tj)xj−1→xjn∏j=0exp[−∫tj+1tjdtλα(t)xj]. (2.8)

The weight is normalized so that

 ∫D^xδx(−τ),xT^α[^x]=1 (2.9)

for any initial state , where the “integral” over all the paths is defined by

 ∫D^x(⋯):=∞∑n=0 ∑x0,…,xn∈S(xj−1≠xj) ∫τ−τdt1∫τt1dt2∫τt2dt3⋯∫τtn−1dtn(⋯). (2.10)

In this language, the general solution (2.4) is written as

 px(t)=∫D^xpinitx(−τ)δx(t),xT^α[^x]. (2.11)

One way to see the equivalence of (2.4) and (2.11) is to write the matrix product explicitly (in terms of the sums over ) in (2.5).

Let be an arbitrary function of . We define the expectation value of by

 ⟨f⟩^αst→:=∫D^xf[^x]ρα(−τ)x(−τ)T^α[^x], (2.12)

where the subscript “” indicates that the system starts from the steady state for parameter , and nothing is specified for the final condition. We stress that this is a physically natural expectation, which can be realized experimentally.

### 2.3 Entropy production, work, and time-reversal

Let us further specify our problem, and also introduce some important quantities.

We assume that each state is associated with its energy . Here is a parameter (or a set of parameters) that characterizes the Hamiltonian , and is a component of . See (2.30) and (2.34) for examples. We assume that takes its value in a compact subset of for some .

#### Entropy production:

For any such that , we define the entropy production in the heat baths8 associated with the transition by

 θαx→y:=logRαx→yRαy→x, (2.13)

or, equivalently, by the “local detailed balance condition”

 Rαy→x=e−θαx→yRαx→y. (2.14)

Clearly one has . Mathematically speaking (2.13) is a mere definition. With this definition of , we can justifies the “detailed fluctuation theorem” (6.4), which will be a basis of the present work. To give the quantity a physical interpretation as entropy production, we need some preparations.

A class of processes called equilibrium (stochastic) dynamics describe a system attached to heat baths with a single temperature and free from any non-conservative forces (see footnote 2.4). Such a system approaches the corresponding equilibrium state after a sufficiently long time9. The transition rates (where we have written ) in an equilibrium dynamics satisfy the detailed balance condition

 e−βHνxR(β,ν)x→y=e−βHνyR(β,ν)y→x, (2.15)

for any . Here is the single inverse temperature of the heat baths. It is well known (and easy to prove) that the condition (2.15) ensures that the corresponding stationary distribution is the canonical distribution , where is the normalization constant.

Under the detailed balance condition (2.15), the entropy production (2.13) becomes

 θ(β,ν)x→y=β(Hνx−Hνy)=−βqx→y, (2.16)

where is the change in the energy of the system, which is equal to the heat transferred from the baths to the system. The final expression in (2.16) is nothing but the well-known formula for the change (or the production) of entropy in equilibrium thermodynamics.

The main subject of the present work is non-equilibrium stochastic dynamics, for which the detailed balance condition (2.15) can never be satisfied for any choice of and . We nevertheless assume here that the entropy production is written as

 θαx→y=−βBqx→y, (2.17)

where is the inverse temperature of the single heat bath10 that is relevant to the transition , and is the energy (heat) transferred from the bath to the system during the transition. The idea behind the identification (2.17) is that heat baths are always in equilibrium states so that we can use the relation from equilibrium thermodynamics for each transition, even when the system never settles to equilibrium11.

Throughout the present work we assume that the nonequilibrium system can be interpreted as a perturbation to an equilibrium system. As for the entropy production we write

 θαx→y=ψαx→y+β(Hνx−Hνy), (2.18)

where is a certain reference inverse temperature, which may not be unique. The quantity should be called the nonequilibrium part of entropy production. It also satisfies .

For a given path as in (2.7), we can define the total entropy production in as

 Θ^α[^x]=n∑j=1θα(tj)xj−1→xj, (2.19)

and its nonequilibrium part as

 Ψ^α[^x]=n∑j=1ψα(tj)xj−1→xj. (2.20)

For any subinterval , we define partial entropy productions by

 Θ[τ1,τ2],^α[^x]=n∑j=1χ[tj∈[τ1,τ2]]θα(tj)xj−1→xj, (2.21) Ψ[τ1,τ2],^α[^x]=n∑j=1χ[tj∈[τ1,τ2]]ψα(tj)xj−1→xj, (2.22)

where and .

#### Work:

Let us write the protocol for the parameter of the Hamiltonian as , which is a component of the full protocol . For a path , we define

 W^ν[^x]:=n∑j=0(Hν(tj+1)xj−Hν(tj)xj)=n∑j=0∫tj+1tjdtdν(t)dt[∂Hνx(t)∂ν]ν=ν(t), (2.23)

where the final expression is valid only when is differentiable. Note that is the change in the energy of the system during the interval , in which the state of the system is always . Since this change in the energy is caused solely by the change in , we can identify it with the work done by the external agent who operates on the system. Therefore (2.23) is the total work done by the external agent to the system in the path . Note that, since varies only for , the summand in (2.23) vanishes if .

It is obvious from (2.18) that , , and are related with each other. In fact by summing up (2.18) for all the transitions in , we see

 Θ^α[^x] =Ψ^α[^x]+βn∑j=1(Hν(tj)xj−1−Hν(tj)xj) Missing or unrecognized delimiter for \bigl =Ψ^α[^x]+βW^ν[^x]+β(Hν(−τ)x(−τ)−Hν(τ)x(τ)). (2.24)

#### Time-reversal:

For a path as in (2.7), we define its time-reversal as

 ^x†=(n,(xn,xn−1,…,x0),(−tn,−tn−1,…,−t2,−t1)). (2.25)

If we use the language of function and denote original path as , the time-reversed path is . Similarly for a protocol and its component , we define their time-reversal as and , respectively.

One easily finds that the total entropy production, its nonequilibrium part, and the work are antisymmetric with respect to the time-reversal, i.e.,

 Θ^α[^x]=−Θ^α†[^x†],Ψ^α[^x]=−Ψ^α†[^x†],W^ν[^x]=−W^ν†[^x†]. (2.26)

#### Measurability of the quantities:

In operational approaches to thermodynamics we believe it essential to distinguish between physical quantities which are experimentally measurable (at least in principle) and which are not. In what follows we assume that a path has been realized, and ask whether the quantities , , and are measurable. This corresponds to the measurability of these quantities in a single experiment.

As in most treatments of equilibrium thermodynamics, we assume that the total work is measurable. The work is a purely mechanical quantity, and the external agent can, in principle, always determine it by precisely measuring the (generalized) force and the displacement.

We next argue that the total entropy production (in the baths) is also measurable. Suppose that the system is in touch with heat baths, where the inverse temperature of the -th bath is . Let be the total amount of heat that flowed into the system from the -th bath during the experiment, i.e., the sum of for every transition (which involve the -th bath) in the path . We assume that the total heat can be measured for each . This may not be a trivial assumption, but in principle we can think of carefully designed heat baths where heat flux can be monitored accurately12. Since the relation (2.17) means that the total entropy production is written as , we conclude that is measurable.

The measurability of the nonequilibrium part of the total entropy production is more subtle. We argue that is measurable or semi-measurable depending on the model. (See the next section for details of the models.) In the models for heat conduction, where the system exchanges energy only with heat baths, we find from (2.33). This means that is determined from the measurable total heat . In the models of systems driven by an external non-conservative force, on the other hand, the system exchanges energy with the external field as well as the heat baths. It then turns out (see (2.36)) that is identical to times the total work done to the system by the external field. The work done by the external field may be measured in principle13, but the measurement seems to be extremely difficult in general. We thus regard as a semi-measurable quantity in this case.

### 2.4 Examples

Although our theory applies to a large variety of physical models, it might be useful to have some concrete examples in mind. Here we define a standard class of equilibrium dynamics, and then describe two typical problems of nonequilibrium physics.

#### Equilibrium dynamics:

Before discussing nonequilibrium problems, let us discuss equilibrium dynamics, which will be the starting point.

To define transition rates, it is convenient to first specify the Hamiltonian and the connectivity function such that for any with . We assume that the state space is connected via nonvanishing , or more precisely, for any one can take a sequence such that , , and for any . We make no assumptions on the Hamiltonian except that it is real.

Then the transition rates for the equilibrium dynamics at the inverse temperature may be defined, for example, as

 R(β,ν)x→y=c(x,y)eβHνx, (2.27)

or

 R(β,ν)x→y=c(x,y)e(β/2)(Hνx−Hνy). (2.28)

It is clear that both the definitions satisfy the necessary conditions for transition rates including the detailed balance condition (2.15).

Suppose that one has and for some . Then the rate (2.28) becomes , and is independent of the inverse temperature .

The abstract scheme discussed above applies to various concrete physical settings. Let us describe a system of particles on a lattice. Let the lattice be a finite set whose elements are denoted as . We denote by the set of bonds on . More precisely the element of is a pair with some such that . We assume that is connected via the bonds in . The simplest example is the one-dimensional periodic lattice with , where we identify with .

We assume that there are particles on the lattice, and let denote a configuration of the particles on . More precisely, we set , where is the position of the -th particle (). One may or may not impose the hard-core condition, i.e., whenever . See Figure 6.

For any two configurations and , we set if for some and for any such that , and otherwise. In other words, if and only if one can modify the configuration into by moving one particle along a bond in .

As for the Hamiltonian, the standard choice is

 Hx:=N∑j=1V1(xj)+N∑j,k=1(j>k)V2(xj,xk), (2.29)

where the single particle potential and the two-particle interaction potential are arbitrary real valued functions on and , respectively.

#### Heat conduction:

Let us discuss an idealized model of heat conduction. We assume that the system interacts with heat baths with different temperatures. We label the baths by the index , and denote by the inverse temperature of the -th bath. The set of parameters that characterizes the model is

 α=(β1,…,βn,ν). (2.30)

With any such that we associate a unique index , which indicates that the -th bath is relevant for the transition between and . Then we define the transition rate as

 Rαx→y=R(βj(x,y),ν)x→y, (2.31)

for any such that , where the right-hand side is defined by (2.27) or (2.28).

From the definition (2.13), one finds

 θαx→y=βj(x,y)(Hνx−Hνy)=Δβj(x,y)(Hνx−Hνy)+β(Hνx−Hνy), (2.32)

where we have chosen the reference inverse temperature (somewhat arbitrarily), and wrote . Comparing with (2.18), one finds

 ψαx→y=Δβj(x,y)(Hνx−Hνy), (2.33)

which is indeed small when all the inverse temperatures are close to each other, and is chosen properly14.

#### Driven system:

We shall illustrate a system which is in contact with a single heat bath with the inverse temperature , but is driven by a non-conservative external force.

For each pair such that , we define a quantity which satisfies the antisymmetry . Physically, is interpreted as the displacement (in the direction of the non-conservative external force) of the particle associated with the transition . In the simplest example of particles on the one-dimensional periodic lattice, we set if a particle jumps to the right in the transition , and if a particle jumps to the left.

We assume that the non-conservative15 external force is applied to the system. The model is parameterized by

 α=(β,ν,f). (2.34)

We then define the transition rate by

 Rαx→y=eβfdx→y/2R(β,ν)x→y, (2.35)

for any such that , where the right-hand side is defined by (2.27) or (2.28).

From the definitions (2.13) and (2.18), one readily finds

 ψαx→y=βfdx→y. (2.36)

This means that the nonequilibrium part of the total entropy production (see (2.20)) can be interpreted as the total work done by the non-conservative external force to the system (multiplied by ).

## 3 Jarzynski-type equalities for NESS

We start by presenting some exact equalities which are valid for general operations (i.e., protocols) to NESS. They are reminiscent of the Jarzynski equality (3.2), which holds for operations to equilibrium states [32, 30].

We note, however, that the derivation of these equalities for NESS is not as straightforward as that of the original Jarzynski equality [32]. One of the main difficulties is that we do not know the explicit form of the probability distribution of NESS while the corresponding stationary distribution in the equilibrium case is the canonical distribution.

Our main equality is the following. We here consider a general protocol introduced in the beginning of section 2.1. See, in particular, Figure 4. As we have discussed at the end of section 2.3, we regard that the work is measurable, and the nonequilibrium part of the entropy production is measurable or semi-measurable depending on the model.

###### Theorem 3.1

There exists a function (that we call the free energy) of the parameters which coincides with the equilibrium free energy for an equilibrium system with , and we have for any protocol that

 Missing or unrecognized delimiter for \bigr (3.1)

This theorem will be proved in section 6.2, using the results from section 7.

We recall that is a component of (as in (2.30) and (2.34)), and likewise the protocol is a component of the full protocol . Note that we fix the operation time scale when taking the limit . It means that we are treating an arbitrary operation, including very “wild” ones.

The equality (3.1) expresses the difference of the (nonequilibrium) free energy in terms of the expectation values defined for nonequilibrium processes. In this sense it may be regarded as a nonequilibrium version of the Jarzynski equality

 F(β,ν′)−F(β,ν)=−1βlog⟨e−βW^ν⟩(β,^ν)eq→ (3.2)

for equilibrium processes. A fundamental difference of our equality from the original equality is that we must consider the expectation values for both the original protocol and its time-reversal