# Stochastic Thermodynamics with Odd Controlling Parameters

###### Abstract

Stochastic thermodynamics extends the notions and relations of classical thermodynamics to small systems that experience strong fluctuations. The definition of work and heat and the microscopically reversible condition are two key concepts in the current framework of stochastic thermodynamics. Herein, we apply stochastic thermodynamics to small systems with odd controlling parameters and find that the definition of heat and the microscopically reversible condition are incompatible. Such a contradiction also leads to a revision to the fluctuation theorems and nonequilibrium work relations. By introducing adjoint dynamics, we find that the total entropy production can be separated into three parts, with two of them satisfying the integral fluctuation theorem. Revising the definition of heat and the microscopically reversible condition allows us to derive two sets of modified nonequilibrium work relations, including the Jarzynski equality, the detailed Crooks work relation, and the integral Crooks work relation. We consider the strategy of shortcuts to isothermality as an example and give a more sophisticated explanation for the Jarzynski-like equality derived from shortcuts to isothermality.

## I Introduction

In the past several decades, growing interest in small systems has been boosted by the tremendous progress in nanotechnology Perkin1994 (); Liphardt2002 (); Collin2005 (); Ciliberto2017 () and biomolecular machines Balzani2000 (); Howard2001 (); Yildiz2004 (); Kay2007 (); Millic2014 (). Because small systems are susceptible to thermal fluctuations, the mean values of thermodynamic quantities such as work, heat, and entropy are not sufficient to predict the behavior of small systems. Fluctuations and probability distributions of thermodynamic quantities also play a vital role. The framework of stochastic thermodynamics has been developed to study fluctuating behaviors of thermodynamic quantities on individual trajectories Sekimoto2010 (); Jarzynski2011 (); Seifert2012 (); Klages2013 (); Qian2014 (). By applying the first law of thermodynamics to fluctuating trajectories, Sekimoto Sekimoto1997 (); Sekimoto1998 () first defined work and heat on individual trajectories. A steady-state thermodynamic framework was put forward later by Oono and Paniconi Oono1998 () and further refined by Hatano and Sasa Hatano2001 (). However, the generality of the current framework of stochastic thermodynamics remains a fertile topic to be applied to more complex conditions Jarzynski2017PRX (); Mandal2017 ().

The probability distributions of these thermodynamic quantities obey various exact fluctuation relations. These include the Jarzynski equality Jarzynski1997 (); Jarzynski1997PRE (); Hummer2001 (), which connects free energy differences between two equilibrium states with an exponential average over nonequilibrium work along fluctuating trajectories; the Crooks relation Crooks1999PRE (); Crooks2000PRE (), which relates the probability distribution of nonequilibrium work in the forward driving processes to the probability distribution of nonequilibrium work in the time-reversed driving processes by means of a detailed and an integral equality; and a series of fluctuation theorems Evans1993 (); GallavottiPRL1995 (); Evans2002 (); Seifert2005 (); Liu2009 (); Gong2015 (), which use the integral and detailed equalities, respectively, to describe the exponential average and the probability distributions of entropy production along fluctuating trajectories. Most of these fluctuation relations can be derived from a fundamental relation: the microscopically reversible condition connecting the probability functionals of the forward and the time-reversed trajectories with the stochastic heat along the forward trajectory Crooks1998 (); Crooks1999 (); Jarzynski2000J ().

The fluctuation relations listed above mainly focused on the probability distributions of entire thermodynamic quantities. According to the framework of steady-state thermodynamics Oono1998 (); Hatano2001 (), the total heat of a stochastic trajectory can be separated into a housekeeping and an excess part. The former is necessary to maintain the system in the nonequilibrium steady state, while the latter is associated with transitions between nonequilibrium steady states. Hatano and Sasa Hatano2001 () found that the part of the total entropy production related to excess heat satisfies the integral fluctuation theorem. Speck and Seifert Speck2005 () then demonstrated that the remaining part of the entropy production related to housekeeping heat also satisfies the integral fluctuation theorem. By introducing an adjoint dynamics Maes1999 (); Chernyak2006 (), Esposito and Van den Broeck Esposito2010PRL (); Esposito2010PRE (); Broeck2010PRE () generalized the Hatano-Sasa Hatano2001 () and the Speck-Seifert fluctuation theorem Speck2005 () into discrete stochastic systems and gave a detailed version of them. Spinney and Ford Spinney2012PRL (); Spinney2012PRE (); Ford2012PRE () recently investigated systems with odd dynamical variables (such as momentum that changes its sign under time-reversal operation) and found that the total entropy production can be separated into three parts, with two of them satisfying the integral fluctuation theorem. Lee et al. Lee2013 (); Yeo2016 () modified the separation rule for the total entropy production put forward in Spinney2012PRL (); Spinney2012PRE (); Ford2012PRE () and endowed each part of the total entropy production with clear physical origins.

The controlling parameters, which control small systems during stochastic processes, are usually assumed to be even variables. However, in recent studies we found that controlling parameters in small systems can also be odd variables. One example is from controlling theories in small systems. Shortcut to isothermality provides a unified controlling strategy for conducting a finite-rate isothermal transition between equilibrium states with the same temperature Geng2017 (). An auxiliary potential is introduced in this strategy to escort the evolution of small systems. It has been demonstrated that the auxiliary potential in shortcuts to isothermality always contains the time derivative of an even controlling parameter. As with the velocity variable, which is the time derivative of the position variable, the time derivative of the even controlling parameter should also change its sign under time-reversal operation, thus failing to guarantee the strategy of shortcuts to isothermality in the time-reversed driving process. In other controlling theories Adib2005 (); Vaikuntanathan2008 (); Ballard2009 (), the time derivative of the even controlling parameter is also contained in the external field, which results in the failure of the controlling theory in the time-reversed driving process. Active biological systems can be considered as another example. Mandal et al. Mandal2017 () studied entropy production and fluctuation theorems for active matter. The effective field in their formalism also contains the time derivative of an even controlling parameter. Moreover, the applied magnetic field in charged Brownian particle systems also change its sign under time-reversal operation. To distinguish between different types of controlling parameters, we call the parameters that retain their sign under time-reversal operation as even controlling parameters and those that change their sign under time-reversal operation as odd controlling parameters.

In this paper, we focus on small systems with both odd and even controlling parameters. However, to highlight the unique properties of odd controlling parameters, we still set the topic as stochastic thermodynamics with odd controlling parameters. As shown in figure 1, the research motivation of this paper is to determine whether the current framework of stochastic thermodynamics can be extended to systems with odd controlling parameters. If not, can we construct a matched framework of stochastic thermodynamics for systems with odd controlling parameters? In Section II, we apply the current framework of stochastic thermodynamics to systems with odd controlling parameters and show that there is a contradiction between the definition of heat and the microscopically reversible condition. Such a contradiction also leads to a revision to the fluctuation theorems and the nonequilibrium work relations. In Section III, we discuss the entropy production and fluctuation theorems for systems with odd controlling parameters. By introducing an adjoint dynamics, we can separate the total entropy production into three parts. The total entropy production and two of the three parts satisfy the integral fluctuation theorem. In Section IV, we discuss nonequilibrium work relations for systems with odd controlling parameters. We find that two sets of nonequilibrium work relations can be obtained by revising the definition of work and heat and the microscopically reversible condition, respectively. In Section V, we consider the strategy of shortcuts to isothermality as an example. Starting from two different choices of the definition of work, we give different explanations of the Jarzynski-like equality derived from shortcuts to isothermality. We conclude with a discussion in Section VI.

## Ii Contradiction Between The Definition of Heat and Microscopic Reversibility

The general setting considered in this paper involves a Brownian particle coupled to a thermal reservoir with a constant temperature . External driving is applied to the system by introducing a time-dependent potential within the time interval , with and representing even and odd controlling parameters, respectively. The motion of the stochastic system is governed by the Langevin equation

(1) |

where represents the coefficient of friction and denotes the standard Gaussian white noise satisfying and . The ensemble behavior of the stochastic system is described by the Fokker-Planck equation

(2) |

with representing the distribution function of the stochastic system. Throughout this paper, we set the mass of the system and the Boltzmann factor to be unit for simplicity.

Sekimoto proposed to endow the Langevin dynamics with a thermodynamic interpretation Sekimoto2010 (). The difference between odd and even controlling parameters was not considered in his original formalism. Stochastic work and heat are respectively defined along an individual stochastic trajectory as

(3) |

and

(4) |

We adopt the Stratonovich convention in this paper.

The condition of microscopic reversibility

(5) |

provides another way to define heat Crooks1998 (); Crooks1999 (); Jarzynski2000J (). Here, denotes the probability of the trajectory , given that the system started at and is driven by the protocol . The probability of the time-reversed trajectory with the time-reversed dynamics is expressed as , given that the system started at and is driven by the time-reversed protocol . Here represents the time-reversal operation. denotes the heat absorbed from the thermal reservoir by the system along the forward trajectory.

The mapping relation between the time-reversed trajectory and the forward trajectory reads

(6) |

The time-reversed driving process follows the same dynamical equation as the forward driving process, namely

(7) |

while the dynamical variables and the parameters are replaced by the time-reversed ones.

The probability of observing a trajectory of the system for given initial state can be obtained by using the path-integral methods Onsager1953 (); Wiegel1986 (); Klages2013 (). For the forward driving process, the conditional probability of observing increments and in a time interval can be written as

(8) |

The detailed derivations of this short-time conditional probability are provided in Appendix A. Similarly, the short-time conditional probability of the time-reversed driving process can be expressed as

(9) |

From microscopic reversibility (5), we can then give a definition of the heat along a stochastic trajectory as

(10) | |||||

where we have defined . Applying the first law of thermodynamics to a stochastic trajectory, we can then define the stochastic work as

(11) | |||||

with representing the energy of the system. To distinguish it from the previous definition, we use and to denote the stochastic heat and work derived from microscopic reversibility (5), respectively. In the particular situation , we can easily verify that the definitions of work (11) and heat (10) reduce to Eqs. (3) and (4), respectively.

Unlike systems containing only even controlling parameters, in which the two definitions of work and heat given above should be consistent Sekimoto2010 (); Crooks1999 (); Crooks1998 (), we find that with the inclusion of odd controlling parameters, the definition given by microscopic reversibility diverges substantially from the one given by the current framework of stochastic thermodynamics. This indicates that either the definition of work and heat or microscopic reversibility in the current framework of stochastic thermodynamics needs to be modified. Because of the fundamental status in stochastic thermodynamics of the work and heat definitions and of microscopic reversibility, such a contradiction may also lead to a revision to the fluctuation theorems and the nonequilibrium work relations.

## Iii Entropy Production and Fluctuation Theorems

Having introduced the contradiction in the definitions of work and heat, we next investigate the form of the stochastic entropy production and fluctuation theorems for systems with odd controlling parameters.

### iii.1 Total Entropy Production and Fluctuation Theorems

For a stochastic system evolving in the time interval , the probability ratio of the forward trajectory and time-reversed trajectory defines the total entropy production

(12) |

where and represent the initial distributions of the forward and the time-reversed driving processes, respectively.

It has already been proved that a random variable such as obeys an integral fluctuation theorem

(13) |

if this random variable can be expressed as the ratio of the forward trajectory probability functional and another transformed trajectory probability functional, such as . The transformed trajectory and the forward trajectory must have the Jacobian of unity, such as and Seifert2005 (); Esposito2010PRL (); Esposito2010PRE (); Broeck2010PRE (); Yang2019 ().

The initial distribution of the time-reversed driving process is chosen as the final distribution of the forward driving process, i.e.,

(14) |

Substituting Eqs. (8), (9), and (14) into Eq. (12), we can express the total entropy production as

(15) | |||||

Such a form of the total entropy production is much more complicated than the case containing only even controlling parameters Seifert2005 (); Seifert2012 (); Esposito2010PRL (); Esposito2010PRE (); Broeck2010PRE (). Therefore the inclusion of odd controlling parameters also changes the form of the total entropy production, but the fluctuation theorem for the total entropy production still holds.

### iii.2 Three Parts of the Total Entropy Production

To further exploit nonequilibrium properties of the entropy, we introduce an adjoint dynamics

(16) |

Note that such a counterintuitive dynamics is mathematically constructed just to separate the total entropy production into meaningful parts Esposito2010PRL (); Crooks2000PRE (); Chernyak2006 (). The short-time conditional probability of this adjoint dynamics satisfies the relation

(17) |

where represents the stationary distribution of the forward dynamics Maes1999 (). We can verify that the adjoint dynamics gives the same stationary distribution and the opposite stationary flux as the forward dynamics.

Based on the adjoint dynamics, we construct two adjoint driving processes and . The mapping relation between the adjoint driving process and the forward driving process reads

(18) |

while the mapping relation between the adjoint driving process and the forward driving process reads

(19) |

Herein, variables with hat and tilde represent the variables in the adjoint driving process and in the adjoint driving process, respectively. The initial distributions of the adjoint driving process and the adjoint driving process are respectively chosen as

(20) |

and

(21) |

By utilizing these two adjoint driving processes, we can further separate the total entropy production into three parts

(22) |

with the trajectory entropy related to the adjoint driving process

(23) |

the trajectory entropy related to the adjoint driving process

(24) |

and the remaining part

(25) |

According to the structure of the trajectory entropy (23) and (24), we can prove that and satisfy the integral fluctuation theorems

(26) |

The third trajectory entropy does not satisfy the fluctuation theorem since it can not be written in the form of , , and .

The concrete expressions of , , and can be further derived respectively as

(27) |

(28) | |||||

and

(29) |

The detailed derivations of the above expressions are shown in Appendix B. Note that if the potential satisfies , we can easily verify that and the total entropy production reduces to the case for systems containing only even controlling parameters Seifert2005 (); Seifert2012 (). The inclusion of odd controlling parameters leads to two extra entropy contributions and and fluctuation theorems (26).

## Iv Nonequilibrium Work Relations

In section II, we put forward the contradiction between the definition of heat and microscopic reversibility for stochastic systems with odd controlling parameters. In this section, we revise the definition of work and heat and microscopic reversibility in the current framework of stochastic thermodynamics and check the form of the nonequilibrium work relations, including the Jarzynski equality and the Crooks work relations.

### iv.1 Revising the Definitions of Work and Heat

In this subsection, we choose to retain the form of microscopic reversibility (5) and revise the definitions of work and heat in the current framework of stochastic thermodynamics to be Eqs. (11) and (10), respectively. In this situation, microscopic reversibility should be reformulated as

(30) |

We keep the form of microscopic reversibility (30) for the following reasons. First, starting from microscopic reversibility and choosing proper initial and final distributions, we can derive most of the fluctuation theorems and nonequilibrium work relations, such as the Jarzynski equality Crooks1998 (), the Crooks work relations Crooks1999PRE (); Crooks2000PRE (), and the total entropy production fluctuation theorems Seifert2005 (). In view of the vital bond role, many researchers believe that microscopic reversibility is fundamental in the framework of stochastic thermodynamics. Second, combining microscopic reversibility (30) with Eq. (12), we can naturally follow the statement of Seifert Seifert2005 () and separate the total entropy production into two contributions: the entropy variation of the stochastic system itself

(31) |

and the entropy variation of the thermal reservoir (or the medium)

(32) |

with .

Herein, we set the initial and the final distributions to be stationary distributions. The expression of the stationary distribution can be easily derived from the Fokker-Planck equation (2) as

(33) |

where

(34) |

represents the normalization factor. The stationary distribution (33) possesses the form of canonical distribution. However, combining the stationary distribution (33) with the short time conditional probability (8) and (9), we can verify that the principle of detailed balance Gardiner1985 () is broken

(35) |

Consequently the concept of equilibrium cannot be applied to systems with odd controlling parameters if the potential satisfies . Such a “nonequilibrium” behavior is caused by a potential with the broken symmetry of time inversion, rather than by usual nonequilibrium constraints such as non-conservative force. Without causing ambiguity, we refer to the traditional nonequilibrium thermodynamics and still call the stationary distribution (33) the steady-state distribution.

By combining microscopic reversibility (30) with the form of the steady-state distribution (33), we can obtain the relation

(36) |

which leads to three nonequilibrium work relations:

(37) |

(38) |

and

(39) |

Here represents the “steady-state” free energy difference. is a functional of the forward trajectory, while is the corresponding functional of the time-reversed dynamical trajectory. We have assumed that the functional satisfies . These three nonequilibrium work relations (37), (38) and (39) maintain the respective forms of the Jarzynski equality Jarzynski1997 (), the detailed Crooks work relation Crooks1999PRE (), and the integral Crooks work relation Crooks2000PRE (), while the stochastic work expression in these relations is replaced by the revised one (11).

Referring to the steady-state thermodynamics Oono1998 (); Hatano2001 (), we can associate part of the total stochastic heat with the trajectory entropy

(40) | |||||

Note that the expression of partial heat (40) is same as the definition of heat in the current framework of stochastic thermodynamics (4).

Considering transitions between steady states and substituting Eq. (40) into relation (23), we can obtain the relation

(41) |

where we have defined partial work as

(42) | |||||

According to relation (41), we can also derive three nonequilibrium work relations:

(43) |

(44) |

and

(45) |

Comparing with the previous three nonequilibrium work relations (37), (38), and (39), we can find that the above three nonequilibrium work relations (43), (44), and (45) also retain the respective forms of the Jarzynski equality Jarzynski1997 (), the detailed Crooks work relation Crooks1999PRE (), and the integral Crooks work relation Crooks2000PRE (). However, it is worth noting that the partial work we consider here is just part of the total work . In addition, the time-reversed driving process is replaced by the adjoint driving process.

### iv.2 Revising Microscopic Reversibility

In this subsection, we instead keep the form of the definitions of work (3) and heat (4) in the current framework of stochastic thermodynamics and instead revise microscopic reversibility. We retain the form of the definitions of work and heat for the following reasons. First, the form of the definitions of work (3) and heat (4) seem a natural means of extension from systems containing one kind of controlling parameter to systems containing two types of controlling parameters. Second, the form of the definitions of work (3) and heat (4) coincide with the thermodynamic interpretation of the Langevin dynamics given by Sekimoto Sekimoto2010 (). This makes the definitions of work (3) and heat (4) physically easier to understand than the alternative definitions of work (11) and heat (10).

According to the definition of heat (4), the microscopically reversible condition should be revised to be

(46) |

with

(47) |

representing the correction term caused by the inclusion of odd controlling parameters. In the particular situation , this correction term vanishes.

Considering transitions between stationary states and taking revised microscopic reversibility (46) into account, we can obtain the relation

(48) |

which leads to three nonequilibrium work relations:

(49) |

(50) |

and

(51) |

This indicates that for systems with odd controlling parameters, if we keep the form of the definitions of work (3) and heat (4) in the current framework of stochastic thermodynamics, a correction term needs to be added to revised versions of microscopic reversibility Crooks1998 (), the Jarzynski equality Jarzynski1997 (), and the Crooks work relations Crooks1999PRE (); Crooks2000PRE ().

On the other hand, if we start from Eq. (23) and consider transitions between stationary states, we can derive the relation

(52) |

Comparing with Eq. (48), we can find that the time-reversed driving process is replaced by the adjoint driving process in Eq. (52), but the additional quantity is successfully eliminated. Three different nonequilibrium work relations can also be derived from Eq. (52):

(53) |

(54) |

and

(55) |

These three nonequilibrium work relations (53), (54), and (55) mathematically recover the Jarzynski equality Jarzynski1997 (), the detailed Crooks work relation Crooks1999PRE (), and the integral Crooks work relation Crooks2000PRE (), respectively. However it is worth noting that the time-reversed driving process in the traditional detailed and integral Crooks work relations Crooks1999PRE (); Crooks2000PRE () is replaced by the adjoint driving process in Eqs. (54) and (55).

## V Example: Shortcuts to Isothermality

To explain our results explicitly, we consider the strategy of shortcuts to isothermality Geng2017 () as an example. In conventional thermodynamics, it is widely believed that the realization of an isothermal process needs to quasi-statically drive controlling parameters. The strategy of shortcuts to isothermality is designed to realize a finite-rate isothermal transition between two equilibrium states with the same temperature Geng2017 (). Within the above framework of stochastic thermodynamics for systems with odd controlling parameters, we can give a deeper understanding of shortcuts to isothermality as well as the Jarzynski-like equality derived from it.

We first briefly introduce the strategy of shortcuts to isothermality; please refer to Geng2017 () for further details. Within the framework of shortcuts to isothermality, an auxiliary potential is introduced to the system of interest with the Hamiltonian . Herein, is the controlling parameter. This auxiliary potential is required to escort the evolution of the system so that the system distribution is always in the instantaneous equilibrium distribution of the original Hamiltonian :

(56) |

with

(57) |

representing the free energy of the original system in equilibrium. To this end, the auxiliary potential is demonstrated to have the structure

(58) |

where function can be determined according to the method in Geng2017 () and . By imposing boundary conditions

(59) |

we can make the auxiliary potential vanish at the beginning and end of the driving process, which indicates that the distribution functions at the two endpoints of the driving process become equilibrium distributions. This is the main idea of shortcuts to isothermality.

As the time derivative of the controlling parameter changes its sign during time-reversal operation, i.e.,

(60) |

we can treat it as the odd controlling parameter and map shortcuts to isothermality into the stochastic thermodynamic framework for systems with odd controlling parameters. The mapping relations read

(61) |

and

(62) | |||||

Therefore, during the driving process of shortcuts to isothermality, the system cannot reach the equilibrium state because of the existence of the odd controlling parameter . Only at the two endpoints of the driving process, the odd controlling parameter vanishes, allowing the system to evolve to equilibrium state.

Three nonequilibrium work relations were derived under the framework of shortcuts to isothermality in Geng2017 (). Among the three nonequilibrium work relations, the third relation, a Jarzynski-like equality, is closely related to the choice of the definition of work. In the following, we discuss the relationship between this Jarzynski-like equality and the definition of work.

In Geng2017 (), the definition of work maintains the form (3). In this situation, the integral Crooks work relation is revised to be Eqs. (51) or (55).

Starting from Eq. (55) and assuming that , we can obtain the relation

(63) | |||||

Herein, represents the ensemble average over all trajectories starting from a fixed state in the adjoint driving process. and respectively represent the final state in the forward driving process and the corresponding steady state when the controlling parameter is fixed. represents the free energy difference of the system with the auxiliary potential (58). Applying the strategy of shortcuts to isothermality, we can evolve the system from an equilibrium state to another one at the same temperature, i.e., in Eq. (63). Therefore, we can derive the equality

(64) |

In the forward driving process, the system Hamiltonian is

(65) |

Considering the mapping relation between the adjoint driving process and the forward driving process (18), we can derive the system Hamiltonian in the adjoint driving process as

(66) |

Note that the system Hamiltonian in the adjoint driving process (66) has the same structure as the one in the forward driving process (65). This means that in the adjoint driving process, the strategy of shortcuts to isothermality can also drive the system from an equilibrium state to another one at the same temperature. According to this property of shortcuts to isothermality, we can omit the subscript ‘’ in Eq. (64) and simplify it to be

(67) |

This equation is a Jarzynski-like equality and implies that one can estimate by taking the exponential average of the work over trajectories that start from an arbitrary fixed state and then evolve under the strategy of shortcuts to isothermality.

Equation (67) is derived from the revised integral Crooks work relation (55). If we instead start from Eq. (51), we can also derive a Jarzynski-like equality

(68) |

In the time-reversed driving process, the system Hamiltonian is expressed as

(69) | |||||

Compared with the system Hamiltonian in the forward driving process (65), we find that in the time-reversed driving process, the strategy of shortcuts to isothermality cannot drive the system from an equilibrium state to another one at the same temperature. Therefore, the subscript ‘’ in Eq. (68) cannot be omitted directly.

The Jarzynski-like equalities (67) and (68) are all derived under the choice of keeping the form of the definition of stochastic work (3) unchanged. If we instead choose to retain the form of the microscopically reversible condition (30) and revise the definition of work to be (11), the integral Crooks work relation is revised to be Eqs. (39) or (45). Starting from these two relations, we can similarly derive two Jarzynski-like equalities

(70) |

and

(71) |

As the system Hamiltonian in the adjoint driving process (66) has the same structure as the one in the forward driving process (65), the subscript ‘’ in Eq. (71) can be omitted, i.e.,