Second law, entropy production, and reversibility in thermodynamics of information

# Second law, entropy production, and reversibility in thermodynamics of information

Takahiro Sagawa 111Department of Applied Physics, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8656, Japan
###### Abstract

We present a pedagogical review of the fundamental concepts in thermodynamics of information, by focusing on the second law of thermodynamics and the entropy production. Especially, we discuss the relationship among thermodynamic reversibility, logical reversibility, and heat emission in the context of the Landauer principle and clarify that these three concepts are fundamentally distinct to each other. We also discuss thermodynamics of measurement and feedback control by Maxwell’s demon. We clarify that the demon and the second law are indeed consistent in the measurement and the feedback processes individually, by including the mutual information to the entropy production.

## 1 Introduction

Thermodynamics is intrinsically related to information, as the entropy represents the lack of our knowledge. The first clue on the information-thermodynamics link was provided by Maxwell, who considered a thought experiment of “Maxwell’s demon” [1]. Later, Szilard suggested a quantitative connection between work extraction and information gain [2], even several decades before the establishment of information theory by Shannon [3]. The role of information in thermodynamics was investigated and controversies were raised throughout the twentieth century [4, 5, 6, 7, 8, 9, 10].

In this decade, thermodynamics of information has attracted renewed attention because of the development of the modern theory of nonequilibrium thermodynamics [11, 12, 17, 14, 13, 15, 16, 18, 19, 20, 21, 22, 23], which is often referred to as stochastic thermodynamics. Especially, a fundamental thermodynamic relation called the fluctuation theorem was discovered in 1990s [11, 12, 14, 15], which has opened up a new avenue of research. We note, however, that only a few seminal works have been done already in 1970s and 80s [24, 25, 26, 27, 28].

Thermodynamics of information can now be formulated based on stochastic thermodynamics [29], by incorporating information concepts such as Shannon entropy and mutual information. Specifically, the Landauer principle for information erasure [30, 31, 32, 33, 34, 35, 36, 37] and feedback control by Maxwell’s demon [38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51] have been investigated from the modern point of view.

Furthermore, the relationship between thermodynamics and information has become significant from the experimental point of view [52]. The first quantitative demonstration of the work extraction by Maxwell’s demon was performed in Ref. [53], and the Landauer bound for information erasure was demonstrated in Ref. [54]. Several fundamental experiments have been further performed both in the classical and quantum regimes [55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69].

In this article, we review the theoretical foundation of thermodynamics of information. Especially, we aim at clarifying the concept of reversibilities and the consistency between Maxwell’s and the second law, which we hope would unravel subtle conceptual problems.

We will put special emphasis on the following two observations. First, reversibility has several different aspects. In particular, thermodynamic reversibility and logical reversibility are fundamentally distinct concepts, which are associated with different kinds of the degrees of freedom of a thermodynamic system. Second, mutual information is a crucial concept to understand the consistency between the demon and the second law. We will see that the demon is consistent with the second law for the measurement and the feedback processes individually.

We here make some side remarks. First, in this article, we only consider classical systems in the presence of an infinitely large heat bath, though essentially the same argument applies to quantum systems. Second, this article is completely newly written, but is closely related to a paper [37] by the author, where the present article is intended to be more pedagogical and comprehensive. Finally, for simplicity of notation, we set the Boltzmann constant to unity (i.e., ) throughout the article.

This article is organized as follows. In the rest of this section, we briefly summarize the above-mentioned two observations. In Sec. 2, we review the second law and reversibility in conventional thermodynamics as a preliminary. In Sec. 3, we discuss the framework of stochastic thermodynamics and clarify what reversibility means there. In Sec. 4, we review reversibility in computation, which is referred to as logical reversibility. In Sec. 5, we discuss thermodynamics of information in a simple setup, and state the Landauer principle. The relationship between thermodynamic reversibility and logical reversibility is clarified in this simple setup. In Sec. 6, we generally formulate thermodynamics of computation. In Sec. 7, we slightly change the topic and discuss work extraction by Maxwell’s demon. In particular, we consider the upper bound of extractable work and formulate thermodynamic reversibility with feedback. In Sec. 8, we generally discuss the entropy balance during the measurement and the feedback processes of the demon and clarify how the demon is consistent with the second law in these processes. In Sec. 9, we make concluding remarks, where we briefly summarize some topics that are not mentioned in the preceding sections.

At this stage, let us summarize some key observations, which will be detailed in the following sections. We consider a binary memory that stores one bit of information (“” and “”), and suppose that the memory is in contact with a single heat bath (see Fig. 1). We focus on the relationship among thermodynamic reversibility, logical reversibility, and heat emission from the memory. We first note that there are three kinds of the degrees of freedom in this setup:

(i)

The computational states of the memory (i.e., “” and “” for the binary case). Such computational states should be robust against thermal fluctuations, in order to store information stably.

(ii)

Internal physical states of the memory, which represent the physical degrees of freedom associated with a single computational state.

(iii)

The degrees of freedom of the heat bath, which is assumed to be in thermal equilibrium.

Then, we have the following observations:

• Thermodynamic reversibility refers to the reversibility of the total system including the heat bath and thus is connected to the entropy change in (i)+(ii)+(iii).

• Logical reversibility refers to the reversibility of the computational states only and thus is connected to the entropy change in (i).

• Heat transfer to the bath is bounded by the entropy change of all the degrees of freedom of the memory, i.e., (i)+(ii).

Therefore, the above three concepts should be distinguished fundamentally, while some of them can become equivalent in some specific setups.

Entropy production characterizes thermodynamic reversibility, and thus is related to the degrees of freedom (i)+(ii)+(iii). A general version of the second law of thermodynamics states that entropy production is always nonnegative for any transition from a nonequilibrium state to another nonequilibrium state (see Sec. 3 for details). In particular, the entropy production of the total system, including an engine and the memory of Maxwell’s demon, is nonnegative for individual processes of measurement and feedback control. A crucial point here is that the mutual information between the engine and the demon should be counted as a part of the entropy production. By doing so, the demon is always consistent with the second law of thermodynamics, and we do not need to consider the information-erasure process to understand the consistency.

## 2 Reversibility in conventional thermodynamics

As a preliminary, we briefly review conventional thermodynamics, which has been established in the nineteenth century as a phenomenological theory for macroscopic systems. A remarkable feature of conventional thermodynamics lies in the fact that it can be formulated in a self-contained manner, without referring to underlying microscopic dynamics such as Newtonian mechanics and quantum mechanics. In fact, an equilibrium state is characterized by only a few macroscopic quantities such as the energy and the temperature. We can also define the thermodynamic entropy in a purely phenomenological manner by using, for example, the Clausius formula [70].

We note that conventional thermodynamics can be formulated as a mathematically rigorous axiomatic theory [71]. While in this article we do not formalize our argument in a rigorous manner, the following argument of this section can be made rigorous in line with the theory of Lieb and Yngvason [71].

We focus on the situation that a thermodynamic system is in contact with a single heat bath at temperature . Let be the inverse temperature. We consider a transition from an equilibrium state to another equilibrium state. During the transition, the system absorbs the heat from the bath, and changes its thermodynamic entropy by . We note that in conventional thermodynamics, the thermodynamic entropy is defined only for equilibrium states, and the second law only concerns a transition from an equilibrium state to another equilibrium state, though intermediate states can be out of equilibrium.

Then, the second law is stated as follows.

Second law of conventional thermodynamics. An equilibrium state can be converted into another equilibrium state with heat absorption Q, if and only if

 ΔST−βQ≥0. (1)

We note that the “only if” part (i.e., any possible state conversion satisfies inequality (1)) is the usual second law, while “if” part (i.e., a state conversion is possible if inequality (1) is satisfied) is also true under reasonable axioms [71].

The left-hand side of the second law (1) is referred to as the entropy production, which we denote by

 Σ:=ΔST−βQ. (2)

This terminology, the entropy production, dates back to Prigogine, who associated with the entropy change of the bath [72]. In this spirit, is regarded as the entropy change of the entire “universe” that consists of the system and the heat bath.

We can also rewrite the second law (1) in terms of the work and the free energy. Let be the work performed on the system, and be the change in the average internal energy. The first law of thermodynamics is given by

 W+Q=ΔE. (3)

By substituting the first law into inequality (1), we obtain

 W≥ΔFeq, (4)

where

 Feq:=E−TST (5)

is the equilibrium free energy. If the process is cyclic, inequality (4) reduces to , which is the Kelvin’s principle, stating that perpetual motion of the second kind is impossible (i.e., a positive amount of work cannot be extracted from an isothermal cycle).

We next formulate the concept of reversibility in conventional thermodynamics. Based on the standard textbook argument [70], we adopt the following definition:

Definition (Reversibility in conventional thermodynamics). A state transition from one to another equilibrium state is thermodynamically reversible, if and only if the final state can be restored to the initial state, without remaining any effect on the outside world.

We note that “effect” above is regarded as a “macroscopic effect” in conventional thermodynamics because microscopic changes (i.e., the subleading terms in the thermodynamic limit) are usually neglected.

A crucial feature of this definition is that thermodynamic reversibility is completely characterized by the entropy production, as represented by the following theorem.

Theorem. Thermodynamic reversibility is achieved if and only if the entropy production is zero, i.e., .

Proof. While one can find a proof of the above theorem in standard textbooks of thermodynamics (at least implicitly), we reproduce it here for the sake of self-containedness.

i) Suppose that a transition is thermodynamically reversible. Then there exists a reverse transition that satisfies the requirements in the definition of reversibility. From the requirement that there is no remaining effect in the outside world, should hold, because otherwise the energy of the heat bath is changed after the reverse transition. Combining this with , we have . On the other hand, both of and hold from of the second law. Therefore, .

ii) Suppose that is zero. Then is also zero. Therefore, the reverse transition is possible with , because of the “if” part of the second law.

We consider the concept of quasi-static process, by which we define that the system remains in equilibrium during the entire process. Thermodynamic reversibility is achieved if a process is quasi-static, because the quasi-static condition guarantees that . The quasi-static limit is achieved by an infinitely slow process in many situations in which

 ΔST=βQ+O(τ−1) (6)

holds, where is the time interval of the entire process. In the infinitely-slow limit , the equality in (1) is achieved. More precisely, means , where is the relaxation time of the system.

We note, however, that there is a subtle difference between quasi-static and infinitely slow processes in general. For example, suppose that there is a box separated by a wall at the center, and gas is only in the left side of the wall. No matter how slow the removal of the wall is, the free expansion of the gas to the right side is not quasi-static and thermodynamically irreversible. Such a situation has been experimentally demonstrated in Ref. [73] in the context of stochastic thermodynamics.

The foregoing argument can be straightforwardly generalized to situations with multiple heat baths at different temperatures. In particular, the zero entropy production of a heat engine with two baths implies the maximum efficiency of the heat-to-work conversion. For example, the Carnot cycle attains the maximum efficiency, where the entropy production is zero and the cycle is thermodynamically reversible. We note that it has been rigorously proved that any thermodynamically reversible process is infinitely slow, based on some reasonable assumptions (including that fluctuations of the system do not diverge) [74]. Since an infinitely slow process gives the zero power (i.e., the work per unit time is zero), thermodynamically reversible engines might be practically useless. For that reason, the efficiency at the maximum power has been intensively studied [75, 76].

The concept of reversibility in conventional thermodynamics is generalized to stochastic thermodynamics, as discussed in the next section.

## 3 Reversibility in stochastic thermodynamics

Stochastic thermodynamics is an extension of thermodynamics to situations where a system is not necessarily macroscopic, and the initial and final states are not necessarily in thermal equilibrium [21, 22, 23]. When a large heat bath is attached to a small system, thermal fluctuations affect the system, and its dynamics become stochastic. Correspondingly, thermodynamic quantities, such as the heat, the work, and the entropy, become stochastic. Biochemical molecular motors and colloidal particles are typical examples of stochastic thermodynamic systems, with which numerous experiments have been performed [52].

Because of thermal fluctuations, the second law of thermodynamics can be violated with a small probability in small systems. At the level of the ensemble average, however, the second law is still valid in an extended form. Furthermore, a universal relation called the fluctuation theorem has been established by taking into account the role of thermal fluctuations of the entropy production, from which the second law of thermodynamics can be reproduced. This is the reason why thermodynamics is still relevant to small systems.

To formulate the second law of stochastic thermodynamics, we need the concept of information entropy, in particular Shannon entropy. Let be a probability variable, which takes a particular value with probability . The Shannon entropy of is then defined as [77]

 S(X):=−∑xP(x)lnP(x)≥0. (7)

If the probability variable is continuous, we replace the summation above by an integral over . We note that if , is regarded as zero.

In contrast to the thermodynamic entropy that can be defined only for thermal equilibrium, the Shannon entropy can be defined for an arbitrary probability distribution. However, these entropies coincide in the canonical distribution , where is the equilibrium free energy and is the Hamiltonian (i.e., the internal energy of state ). In this case, the Shannon entropy is given by the difference between the average energy and the free energy:

 S(X):=−∑xPcan(x)lnPcan(x)=β(∑xPcan(x)E(x)−Feq), (8)

which is a statistical-mechanical expression of the thermodynamic entropy.

We suppose that a small system is described by a Markov jump process or overdamped Langevin dynamics [78, 79]. We also suppose that the system is driven by external parameters (e.g., the center and the frequency of optical tweezers), which are represented by time-dependent parameters in a master equation or a Langevin equation. The following argument is independent of the details of dynamics, and therefore we do not explicitly write down stochastic equations.

On the other hand, we assume that variables that break the time-reversal symmetry (i.e., that have the odd parity for the time-reversal transformation) are absent or negligible. In particular, the momentum term is negligible in dynamics of the system (in particular, the Langevin equation is overdamped), and the magnetic field is absent in the external potential. While some of the following arguments are straightforwardly generalized to systems with the momentum and the magnetic field, there are some subtle problems with odd-parity variables.

Let be the state of the system, which has a certain probability distribution. Correspondingly, we define the Shannon entropy of the system, written as . We consider a transition from the initial distribution to the final distribution of the system, where both the distributions are arbitrary and can be out of equilibrium. Let be the change in the Shannon entropy of the system, and be the ensemble average of the heat absorption. We then have the following version of the second law:

Second law of stochastic thermodynamics. A distribution can be converted into another distribution with the heat absorption Q, if and only if

 ΔS−βQ≥0. (9)

As in conventional thermodynamics, the left-hand side of inequality (9) is referred to as the (ensemble averaged) entropy production:

 Σ:=ΔS−βQ. (10)

This is regarded as the total entropy increase in the “whole universe” that consists of the system and the heat bath.

As discussed before, if the distribution is canonical, the Shannon entropy reduces to the thermodynamic entropy. Therefore, inequality (9) is a reasonable generalization of the conventional second law (1) to situations that the initial and final distributions are not necessarily canonical.

We can rewrite inequality (9) in terms of the work and the free energy. Let be the work performed on the system. We denote the average energy of the system by . The first law of thermodynamics is again given by

 W+Q=ΔE, (11)

where is the change in the average energy. By substituting (11) into inequality (9), we obtain

 W≥ΔF, (12)

where

 F:=E−TS (13)

is called the nonequilibrium free energy [13, 20]. We note that if the probability distribution is canonical, we have from Eq. (8) that . In such a case, inequality (12) reduces to , which is nothing but the second law of equilibrium thermodynamics.

We now consider the precise meaning of “if and only if” in the second law above. The “only if” part (i.e., any possible transition satisfies inequality (9)) has been proved for Markovian stochastic dynamics, including Markov jump processes and Langevin dynamics [13, 15, 18, 20]. Furthermore, inequality (9) has been proved for a setup in which the total system, including the heat bath, obeys Hamiltonian dynamics [16].

Before discussing the “if” part, we show that there exists a protocol that achieves the equality in (9), for given initial and final distributions and potentials [29]. Such a protocol is schematically shown in Fig. 2, which consists of sudden and infinitely slow changes of the external potential. While the initial distribution can be out of equilibrium, the potential is instantaneously adjusted to the distribution to make it equilibrium. Then, the potential is changed infinitely slowly, which brings the distribution to the final one. After that, the potential is again changed suddenly, and the final distribution can again be out of equilibrium.

It is easy to check that the entropy production is zero, , in this protocol. In fact, the heat absorption from (b) to (c) satisfies , while in the potential switching processes the heat absorption is zero and the Shannon-entropy change is also zero. Therefore, the entropy production of the entire process is zero. The essence of this protocol is that the distribution is always canonical, except for the very moments of the initial and final times. In this sense, this protocol is regarded as quasi-static. In other words, the probability distribution never spontaneously evolves during the entire process, which prohibits the entropy production from becoming positive.

Based on the above protocol, the “if” part of the second law (i.e., a transition is possible if inequality (9) is satisfied) can be shown, by explicitly constructing a protocol with . For example, we can add an auxiliary cyclic process to an intermediate step of the infinitely slow protocol ((b)-(c) in Fig. 2). If such a cyclic process is not slow, it simply “stirs” the system, and the system emits a positive amount of heat (i.e., ). By adding this heat emission, we have a positive amount of entropy production in the entire process.

We next discuss the concept of reversibility in stochastic thermodynamics. While the fundamental idea is the same as in conventional thermodynamics, we need to care about probability distributions in stochastic thermodynamics. We thus adopt the following definition:

Definition (Reversibility in stochastic thermodynamics). A stochastic process is thermodynamically reversible, if and only if the final probability distribution can be restored to the initial one, without remaining any effect on the outside world.

As in conventional thermodynamics, reversibility defined above is completely characterized by the entropy production.

Theorem. Reversibility in stochastic thermodynamics is achieved if and only if the entropy production is zero, i.e., .

The proof of this theorem is completely parallel to the case of conventional thermodynamics, just by replacing thermodynamic entropy by Shannon entropy.

From the above theorem, the protocol described in Fig. 2, satisfying , is thermodynamically reversible. We can also directly see this, because the final distribution is restored to the initial distribution, just by reversing the entire protocol step by step. In this reversed protocol, the heat absorption satisfies , and therefore no effect remains in the outside world.

## 4 Reversibility in computation

We next discuss the concept of reversibility in computation, which we will show is fundamentally distinct from thermodynamic reversibility.

Let be the set of the input states of computation. For example, if any input consists of binary bits, then . We can also consider being the set of the output states of computation, which can be different from in general. Any computation process is a map from to .

We see three simple examples of such computation.

NOT

The NOT gate simply flips the input bit: , and

 ^C(0)=1, ^C(1)=0. (14)
ERASE

The information erasure maps any input to a single “standard state.” If the input is one bit and the standard state is “”, then , and

 ^C(0)=0, ^C(1)=0. (15)
AND

For the AND gate, the input is two bits and the output is one bit: , , and

 ^C(00)=0, ^C(01)=0, ^C(10)=0, ^C(11)=1. (16)

Rigorously speaking, a computable map from to is defined as a partial recursive function, or equivalently, a partial function that can be implemented by a Turing machine [80]. However, this precise characterization of computability is not necessary for the following argument. We also note that we only consider deterministic computation in this article.

We now define logical reversibility of computation [6, 7, 9]. For a logically reversible computation, one can recover the original input from only the output, which is formalized as follows.

Definition (Logical reversibility). A deterministic computational process is logically reversible, if and only if it is an injection. In other words, is logically reversible if and only if, for any output, there is a unique input.

In the case of the aforementioned three examples, NOT is logically reversible, while ERASE and AND are logically irreversible. Figure 3 schematically illustrates these examples, where it is visually obvious that only NOT is injection and thus logically reversible.

We next show that logical reversibility can be characterized by the Shannon entropy of the computational states. For that purpose, we consider a probability distribution over inputs. Let be the probability of input . The probability distribution over the outputs is then given by

 p(m′)=∑m∈^C−1(m′)p(m), (17)

where means . Correspondingly, we define the Shannon entropies of the input and the output by

 S(M):=−∑m∈Mp(m)lnp(m), S(M′):=−∑m′∈M′p(m′)lnp(m′). (18)

Then, as a general property of the Shannon entropy [77], we have

 ΔS(M):=S(M′)−S(M)≤0. (19)

In fact,

 S(M)−S(M′)=−∑m′∈M′∑m∈^C−1(m′)p(m)lnp(m)+∑m′∈M′∑m∈^C−1(m′)p(m)lnp(m′)=∑m′∈M′∑m∈^C−1(m′)p(m)lnp(m′)p(m)≥0, (20)

where we used Eq. (17) to obtain the second term on the right-hand side of the first line, and used with to obtain the last inequality. Therefore, the Shannon entropy does not increase by any deterministic computation.

We show the entropy changes in the aforementioned three examples. We assume that the probability distribution of the input is uniform.

NOT

, and thus .

ERASE

, , and thus .

AND

, , and thus .

The equality in the last line of (20) is achieved, if and only if holds for any satisfying and . This is equivalent to the following: For any with , there exists a unique with . This means that is injection, when the domain of is restricted to the set of with . Therefore, we obtain the following theorem (in a slightly rough expression):

Theorem. A deterministic computational process is logically reversible, if and only if the Shannon entropy of the computational process does not change.

We are now in a position to discuss why logical and thermodynamic reversibilities are fundamentally distinct. In fact, logical reversibility is the reversibility of only computational states (i.e., the degrees of freedom (i) in Sec. 1), and thus characterized by the Shannon-entropy change of computational states, . On the other hand, thermodynamic reversibility is the reversibility of the entire system including the heat bath (i.e., the degrees of freedom (i)+(ii)+(iii) in Sec. 1), and thus characterized by the total entropy production . This observation is summarized in Table 1. We will further develop this observation in the subsequent sections, especially in the context of the Landauer principle.

We note that any logically irreversible process can be embedded in another logically reversible process by extending the space of computational states [7]. For example, if we prepare as an extended set of computational states, we can construct an extended map by

 ^C′ : (m,0)∈M×M′ ↦ (m,^C(m))∈M×M′, (21)

where is the standard state of . Strictly speaking, should be interpreted as a map from to . This extended map reproduces the original map , if we only look at of the input and of the output. A crucial feature of is that the input is kept in of the output of . Therefore, the extended map is logically reversible, even when the original map is logically irreversible. Such a construction of a logically reversible extension of a logically irreversible map has experimentally be demonstrated in Ref. [64] in the context of thermodynamics of computation.

## 5 Landauer principle

We now discuss thermodynamics of computation. Before a general argument, in this section we focus on a paradigmatic model: the conventional setup of the Landauer principle for information erasure [6, 30, 31].

The information erasure is nothing but the ERASE gate discussed in Sec. 4: The initial information of “” or “” is erased, so that the final computational state is always in “” that is called the standard state. This is a logically irreversible process as discussed before.

If the initial distribution of the input is uniform (i.e., ), the Shannon entropy of the input is , while that of the output is . The change in the computational entropy is given by , as already shown in Sec. 4. This is the erasure of one bit of information.

To physically implement information erasure, we consider a physical device that stores one bit of information, which is called a memory. Suppose that the memory is in contact with a single heat bath at temperature (). In the conventional setup, the memory is modeled by a particle in a symmetric double-well potential (see Fig. 4 (a)), which has already been discussed in Sec. 1 (Fig. 1). The memory stores “” (“”), if the particle is in the left (right) well. The particle moves stochastically under the effect of a heat bath and can be described by, for example, an overdamped Langevin equation. Let be the position of the particle, which is the physical degrees of freedom of this memory. We assume that the barrier between the wells is sufficiently high compared with the thermal energy that thermal tunneling between the wells is negligible.

As a simpler model of the memory, the symmetric double-well potential can be replaced by two boxes with an equal volume (Fig. 4 (b)), where the barrier of the double-well potential corresponds to the wall separating the boxes. In this two-box model, the memory stores “” (“”), if the particle is in the left (right) box.

In any setup (either the double-well model or the two-box model), the entire phase space, which we denote as , represents the position of the particle. is divided into two regions that represent computational states “” and “”.

The information-erasure process with the two-box model is represented in Fig. 4 (c). We suppose that the memory is in local equilibrium in the individual boxes in the initial and final distributions. Since the two boxes have the same volume, the change in the Shannon entropy of the entire phase space by the information erasure is the same as that of the computational states:

 ΔS(X)=ΔS(M)=−ln2. (22)

From the second law (9) with , we have

 −Q≥Tln2, (23)

which implies that the heat emission from the memory is bounded from below by . This bound on heat emission is referred to as the Landauer bound, and inequality (23) is called the Landauer principle. Experimental verifications of the Landauer principle with a symmetric memory have been performed in, for example, Refs. [54, 59, 60, 63, 62].

Let be the work performed on the memory during the erasure. Since the internal energy does not change during the erasure, holds from the first law of thermodynamics. Therefore, the Landauer principle (23) can be rewritten as

 W≥Tln2, (24)

which gives the fundamental lower bound of the work required for the information erasure in a symmetric memory.

The equality in (23) is achieved in the quasi-static limit, where the compression process in Fig. 4 (c) is infinitely slow. In fact, such a quasi-static protocol is a special case of the reversible protocol in Fig. 2, and therefore the equality in (23) is achieved from the general argument in Sec. 3. In this case, the entropy production defined in (10) is zero: . We note that we can also directly compute that in the infinitely-slow limit, by using the equation of states of the single-particle gas, which will be discussed in Sec. 6 for more general situations (the present situation is there).

Therefore, information erasure is thermodynamically reversible in the quasi-static limit. We note that the probability distribution is unchanged by the removal of the wall, which guarantees that the process is quasi-static. On the other hand, if information erasure is not quasi-static, the entropy production is positive: . In this case, information erasure is thermodynamically irreversible.

To be more explicit, we show in Fig. 4 (d) the time-reversal of the quasi-static information-erasure protocol with the two-box model, which indeed restores the probability distribution to the initial one, leading to the Shannon-entropy change . In this time reversal process, the heat of is absorbed from the heat bath during the expansion process, which has the inverse sign of the erasure process. We thus confirm that in the time-reversal.

In short, logically irreversible information erasure can be performed in a thermodynamically reversible manner. Of course, this is totally consistent, given the different definitions of the two reversibilities. In fact, as also discussed in Sec. 1, logical reversibility cares only about the reversibility of the computational states, while thermodynamic reversibility is characterized by reversibility in the entire universe that consists of the memory and the heat bath.

From the entropic point of view, logical reversibility implies , while thermodynamic reversibility implies . These are definitely different, even when as in the present case.

In Table 2, we summarize the relationship between thermodynamic and logical reversibilities in the standard setup of information erasure.

## 6 Thermodynamics of computation

In this section, we discuss a general framework of stochastic thermodynamics of computation. First, we remark that a physical state and a computational state are distinct concepts. In the standard setup of the Landauer principle in Sec. 5, the physical state is the position of the particle, and thus is a continuous variable, while the computational state is “left” or “right” of the double well (representing “” and “”), and thus is a binary varibable. In realistic situations of computation, a single computational state contains a huge number of microscopic physical states. This can be regarded as a coarse-graining of the physical phase space.

In general, we divide the physical phase space (i.e., the set of physical states) into several non-overlapping regions, where each region represents a computational state. Let be the set of physical states, and be the set of computational states, as in the previous sections. We consider a subset of , written as with index , which is the set of physical states that represent a computational state . is then divided as , where for all with the empty set .

We consider a probability distribution on the physical phase space. Let be the probability of physical state , and be that of computational state . Since all of represent a single computational state , we have

 p(m)=∑x∈XmP(x). (25)

We then define the conditional probability of under the condition that the computational state is (i.e., ):

 P(x|m)={P(x)/p(m)  (if  x∈Xm),0   (otherwise). (26)

We next consider the Shannon entropy associated with this probability distribution. The Shannon entropy of the physical states is given by

 S(X):=−∑xP(x)lnP(x), (27)

and the Shannon entropy of the computational states is given by

 S(M):=−∑mp(m)lnp(m). (28)

We also consider the conditional entropy of under the condition that the computational state is :

 S(X|m):=−∑x∈XmP(x|m)lnP(x|m), (29)

which represents fluctuations of physical states inside a single computational state.

A crucial property of the non-overlapping devision of the phase space is the corresponding decomposition of total (physical) entropy, which is represented as

 S(X)=S(M)+S(X|M), (30)

where

 S(X|M):=∑mp(m)S(X|m)=−∑m∈M∑x∈Xmp(m)P(x|m)lnP(x|m). (31)

This decomposition is a general property of probability theory [77], and its proof is given by

 S(X)=−∑x∈XP(x)lnP(x)=−∑m∈M∑x∈Xmp(m)P(x|m)ln[p(m)P(x|m)]=−∑m∈Mp(m)lnp(m)−∑m∈M∑x∈Xmp(m)P(x|m)lnP(x|m)=S(M)+S(X|M), (32)

where we used to obtain the third line. We note that does not diverge for all and .

Here, represents the entire fluctuation in the physical phase space, which is related to the heat through the second law (9). On the other hand, is the entropy of computational states, which is related to logical reversibility as discussed in Sec. 4. represents the average of fluctuations inside the individual computational states. We refer to as the physical entropy, as the computational entropy, and as the internal entropy.

We next consider dynamics on that realizes a computation . The dynamics can be stochastic on the entire phase space but should be deterministic on the computational space in order to realize a deterministic computation. Such a situation is realistic in practical computations, because physical states thermally fluctuate inside individual computational states, even when the output of computation is deterministic.

We consider the change in the entropy during computation. Let and be the initial and final computational states that are related deterministically as , and and be the initial and final physical states that are related stochastically. We use notations and ( and ) to refer to the probability variables of the initial and final physical (computational) states, respectively. The change in the Shannon entropies are then denoted as , , and .

The second law (9) is represented by the total entropy as

 ΔS(X)≥βQ, (33)

which is equivalent to, via decomposition (30),

 ΔS(M)+ΔS(X|M)≥βQ. (34)

Correspondingly, the entropy production (10) is decomposed as

 Σ=ΔS(M)+ΔS(X|M)−βQ. (35)

In the present setup, the nonequilibrium free energy is the same as Eq. (13):

 F(X):=E(X)−TS(X)=E(X)−TS(M)−TS(X|M), (36)

where is the average energy of the memory. Then, the fundamental lower bound of the work required for the computation is given by

 W≥ΔF(X). (37)

We consider the local canonical distribution inside a computational state , which is given by

 Pcan(x|m):={eβ(Feq(m)−E(x|m))  (if  x∈Xm),0   (otherwise), (38)

where is the Hamiltonian for a given , and

 Feq(m):=−Tln∑x∈Xme−βE(x|m) (39)

is the local equilibrium free energy under the condition of . If the memory is in local equilibrium inside individual computational states, the nonequilibrium free energy (36) reduces to [35]

 F(X)=Feq−TS(M), (40)

where . If the initial and final distributions are local canonical, inequality (37) reduces to

 W≥ΔFeq−TΔS(M). (41)

In the rest of this section, we assume that the initial and final distributions are local canonical. In fact, this is a reasonable assumption, given that the time scale of global thermalization is much longer than that of local thermalization, because of the potential wall between the computational states.

We now consider the role of the symmetry of the memory. We first consider the case that the memory is symmetric as in Sec. 5. In such a case, the local free energies of the two computational states are the same: , and therefore for any computation. Therefore, inequality (41) reduces to

 W≥−TΔS(M), (42)

which is a general expression of the Landauer principle. In fact, the original Landauer principle (24) is a special case of inequality (42) with . Inequality (42) has been directly verified in a recent experiment [62].

In terms of the internal entropy, the symmetry implies in local equilibrium, and therefore . Then, inequality (34) reduces to

 ΔS(M)≥βQ, (43)

or equivalently,

 −Q≥−TΔS(M). (44)

We next consider the case that the memory is asymmetric, where and in general. If , the entropy change in the computational states is not directly related to the heat (i.e., inequality (43) does not necessarily hold) [32, 34, 35]. In Fig. 5 (a), we show a simple example of a memory with an asymmetric double-well potential, where the left (right) well represents computational state “” (“”).

As is the case for the symmetric memory, we can replace the double-well potential by two boxes (Fig. 5 (b)). If the double-well potential is asymmetric, the volumes of the two boxes are not the same. Let () be the ratio of the volume of the left box. If the memory is symmetric, . For , the local free energies satisfy , and the internal entropies satisfy in local equilibrium. We emphasize that the initial probability distribution of and is arbitrary (i.e., not necessarily ), because the memory can store any information.

We consider information erasure with the asymmetric memory. For simplicity, we assume that the initial distribution is . The Shannon-entropy change in the computational states by the information erasure is then given by .

Figure 5 (c) shows the optimal information-erasure protocol, which achieves the equality of the second law (34[35]. A crucial point of this protocol is that the wall is first moved to the center infinitely slowly. Thanks to this process, the probability distribution of the particle (i.e., for both left and right) does not change by the removal of the wall. (If we removed the wall without moving it to the center, the probability distribution would spontaneously relax towards the uniform distribution over the box, which makes the process thermodynamically irreversible and the entropy production positive.) This is in the same spirit as the protocol in Fig. 2. Then, the box is compressed from to the left infinitely slowly, and the final position of the wall returns to the initial one. The total entropy production is zero in this process, and thus it is thermodynamically reversible. To see the thermodynamic reversibility more explicitly, we illustrate the time-reversal of the above protocol in Fig. 5 (d).

We can also directly compute the heat emission for the protocol in Fig. 5 (c). We assume the equation of states of the single-particle gas, i.e., , where is the pressure and is the volume of the box (and remind that ). The heat emission is then given by . We note that the work is not needed for the removal of the wall. Then, the heat emission during the entire process is given by

 −Q=Tln2+T2ln1−tt. (45)

On the other hand, we have and

 S(X′|m′=0)−S(X|m=1)=lnt1−t, (46)

and therefore,

 ΔS(X|M):=S(X′|m′=0)−12(S(X|m=0)+S(X|m=1))=12lnt1−t. (47)

Combining this with , we obtain

 ΔS(M)+ΔS(X|M)−βQ=−ln2+12lnt1−t+(ln2+12ln1−tt)=0, (48)

which achieves the equality in (34).

If , Eq. (45) reproduces that . On the other hand, if , we have . In particular, if , we have , which is below the Landauer bound (44). Of course, this does not contradict the second law. In such a case, the decrease in the computational entropy is compensated for by the increase in the internal entropy . Information erasure with such an asymmetric memory has experimentally been demonstrated in Ref. [61].

In summary, heat emission is connected to the change in the total physical entropy of the memory (i.e., (i)+(ii) in Sec. 1), which is decomposed into the computational and internal entropies as in Eq. (30). If the change in the internal entropy is not zero, the computational entropy is not directly related to heat emission. This is the reason why the information erasure below the Landauer bound (44) is possible with an asymmetric memory, while the general bound (34) is always true.

## 7 Work extraction and reversibility with feedback control

We next consider work extraction from heat engines through feedback control by Maxwell’s demon. As we will discuss below, the mutual information is the source of work extraction by the demon, and therefore we refer to such a heat engine as an information engine.

In this section, we do not explicitly formulate the memory of the demon itself. Instead, we only regard the demon as an external agent that affects the probability distribution of the engine through the measurement. The full analysis of the total system of engine and demon is postponed to the next section.

We first consider the Szilard engine, which is a simple model of an information engine (see Fig. 6). The Szilard engine consists of a Brownian particle (or a molecule) in a box that is attached to a single heat bath at temperature . During the process depicted in Fig. 6, the demon obtains one bit () of information corresponding to left or right, and performs feedback control. Then, the work of is extracted from the engine, and the same amount of heat is absorbed from the bath. The amount of the work is calculated by the same manner as in Sec. 6. Since the dynamics of the engine is cyclic, this positive work extraction apparently violates the second law. However, if we take into account the memory of the demon, then the total system is not cyclic, and therefore this is not a violation of the second law. As will be discussed in Sec. 8, we can understand more quantitatively the consistency between the demon and the second law by taking into account the mutual information between the engine and the demon.

We now consider a general upper bound of the extractable work with feedback control. We assume that the engine is in contact with a single heat bath at temperature . Let be the probability distribution of the engine immediately before the measurement by the demon. We note that we use notation (and ) to describe the engine, instead of the memory of the demon; we use notation for the measurement outcome obtained by the demon.

We suppose that the error of the measurement is described by the conditional probability , which is the probability of obtaining outcome under the condition that the true state of the engine is . If the measurement is error-free, we have with being the Kronecker’s delta (or the delta function if the state variable is continuous). The joint probability of and is given by , and the unconditional probability of is given by . From the Bayes rule, the conditional probability of under the condition of outcome is given by

 P(x|y)=P(x,y)P(y). (49)

Correspondingly, the conditional entropy of under the condition of a particular is given by

 S(X|y):=−∑xP(x|y)lnP(x|y). (50)

Its ensemble average over all is

 S(X|Y):=∑yP(y)S(X|y)=−∑xyP(x,y)lnP(x|y)=S(XY)−S(Y), (51)

where is the Shannon information of the joint distribution.

After the measurement, the protocol to control the engine depends on , which is the characteristic of feedback control. By noting that the initial distribution of the engine is given by under the condition of outcome , the second law of stochastic thermodynamics (9) can apply to the conditional distribution:

 S(X′|y)−S(X|y)≥βQy, (52)

where is the heat absorption with , and is the conditional entropy in the final distribution of the engine. By taking the ensemble average over all , we have

 S(X′|Y)−S(X|Y)≥βQ, (53)

where .

Before proceeding further, we here discuss mutual information, which quantifies a correlation between two probability variables. The mutual information between and is defined as

 I(X:Y):=S(X)+S(Y)−S(XY)=∑x,yP(x,y)lnP(x,y)P(x)P(y). (54)

It immediately follows that

 I(X:Y)=S(X)−S(X|Y)=S(Y)−S(Y|X). (55)

The mutual information satisfies the following inequalities:

 0≤I(X:Y)≤min{S(X),S(Y)}, (56)

where holds if and only if the two systems are not correlated (i.e., ).

Going back to the second law (53), it is rewritten as, by using Eq. (55),

 ΔS(X)−βQ≥−ΔI, (57)

where and . We note that if the feedback control works, should hold (and thus ). In fact, in the case of the Szilard engine, and hold, because there is no remaining correlation after the entire process. In general, since the correlation is also decreased by dissipation to the environment, gives an upper bound of the information that is utilized by feedback control. By noting that is nonnegative in the absence of feedback control, inequality (57) implies that we can reduce the entropy of the system by using feedback control, where the mutual information is the resource of the entropy reduction.

We consider the nonequilibrium free energy of , defined in the same manner as Eq. (13):

 F(X):=E(X)−TS(X), (58)

where is the average energy of the engine. We then rewrite inequality (57) as

 W≥ΔF(X)+TΔI. (59)

By defining the extracted work , we have

 Wext≤−ΔF(X)−TΔI. (60)

The right-hand side above can be further bounded as

 Wext≤−ΔF(X)+TI(X:Y), (61)

where we used . Inequality (61) implies that additional work can be extracted up to the mutual information obtained by the measurement.

We consider a special case that the initial distribution of the engine is canonical and the final distribution is also canonical under the condition of . More precisely, the final Hamiltonian can depend on , which we denote by , and the final distribution is given by , where is the final equilibrium free energy with . Let be the initial equilibrium free energy as usual . We then have

 S(X)=β(E(X)−Feq(X)), S(X′|y)=β(E(X′|y)−Feq(X′|y)), (62)

where that gives . We define the change in the equilibrium free energy by

 ΔFeq(X):=∑yP(y)Feq(X′|y)−Feq(X). (63)

By substituting Eq. (62) into inequality (59), we have

 W≥ΔFeq(X)−TI(X:Y), (64)

or equivalently,

 Wext≤−ΔFeq(X)+TI(X:Y). (65)

We emphasize that inequality (64) or (65) is exactly equivalent to (59) under the assumption that the initial and final distributions are (conditional) canonical, where we did not drop . In fact, to obtain inequality (64) or (65) from (59), is just absorbed into the definition of in Eq. (63). On the other hand, we dropped to obtain (61) from (59).

In the case of the Szilard engine, we have , , and . Therefore, the equality in (65) is achieved in the Szilard engine.

We note that inequality (65) has been derived in Refs. [41, 43]. The role of mutual information in thermodynamics has been experimentally demonstrated in, for example, Refs. [56, 67].

We consider thermodynamic reversibility with feedback control [81, 82]. We remember that the second law with feedback control is given by inequality (53), which is the ensemble average of inequality (52). Here, inequality (52) is equivalent to the second law (9) under the condition of . Therefore, it is reasonable to adopt the following definition [82]:

Definition (Thermodynamic reversibility with feedback). In the presence of feedback control, thermodynamic reversibility of the engine is achieved if and only if the equality in (53) is achieved, or equivalently, the equality in (52) is achieved for all .

In the rest of this section, we work on this definition of thermodynamic reversibility. We note, however, that this definition does not concern the reversibility of the memory of the demon during the measurement process. In fact, in this section we have just formulated the measurement process as the modification of the probability distribution from to , without explicitly considering dynamics of the memory. The full treatment of the memory will be discussed in Sec. 8 in detail.

By remembering the argument in Sec. 3, thermodynamic reversibility is achieved by the protocol in Fig. 2, where we now replace the distribution by the conditional one , and also the potential by the -dependent one . In other words, thermodynamic reversibility with feedback control is achieved if we adjust the potential such that the conditional distribution becomes always the canonical distribution of . In particular, we need to switch the potential immediately after the measurement, because the distribution is suddenly changed from to by the measurement. We again remark that this consideration neglects reversibility of the measurement process itself.

We revisit the Szilard engine as a special example. Since the Szilard engine achieves the equality in (65) as mentioned above, the Szilard engine is thermodynamically reversible. We can directly confirm that the Szilard engine is always in the canonical distribution under a particular measurement outcome.

To see this point clearer, let us consider a simple analogue of the Szilard engine, illustrated in Fig. 7. In this model, the particle is in one of the two sites with the same energy, which is in contact with a single heat bath at temperature (). The information gain and the work extraction in this model are the same as those in the Szilard engine, implying the thermodynamic reversibility of this model. It is obvious that this model is always in the conditional canonical distribution during the entire process.

We can generalize this model by incorporating a measurement error [82], which is illustrated in Fig. 8. We suppose that the error rate of the measurement is given by (); the conditional probabilities are given by () and () with being “right” or “left”. In this case, the mutual information obtained by this measurement is

 I(X:Y)=ln2+εlnε+(1−ε)ln(1−ε). (66)

Immediately after the measurement, we have () and (). To achieve thermodynamic reversibility, we need to make the canonical distribution for all . Consider the case that “left” as illustrated in Fig. 8. (The same argument applies to the case that “right”.) The demon switches the energy level of the right site to make the energy difference so that becomes canonical (Fig. 8 (c)):

 e−βδE1+e−βδE=ε, 11+e−βδE=1−ε. (67)

The work extraction by this switching is given by on average, because the particle is pushed up if it is in the right site.

The demon next lowers the energy level of the right site infinitely slowly, and the final distribution is the same as the initial one (Fig. 8 (d)). The extracted work during this process is given by , because the extracted work equals the minus of the equilibrium free-energy change in this situation (i.e., the free energy after the sudden switching is and that in the final distribution is ).

The total work extracted from the entire process is then given by

 Wext=−εδE+Tln11+e−βδE=T(ln2+εlnε+(1−ε)ln(1−ε)). (68)

We note that in the entire process. Therefore, the equality in (65) is achieved, i.e., , and thus we confirm that this protocol achieves the thermodynamic reversibility.

This type of the Szilard engine with measurement error has been proposed in Ref. [82], and experimentally demonstrated in Ref. [56] by using a single electron box. Other models that achieve the thermodynamic reversibility with feedback have been discussed in Refs. [83, 84, 85, 86, 46].

## 8 Entropy balance in Maxwell’s demon

In the previous section, we did not explicitly consider the measurement as a physical process, but as the modification of the probability distribution from to . In particular, we did not consider the entropy production in the memory of the demon itself.

In this section, we explicitly consider stochastic thermodynamics of the entire system of the engine and the memory of the demon  [50, 51]. Specifically, we focus on the entropy balance during the measurement and the feedback processes by explicitly considering the memory as a physical system. In this respect, we will reproduce the second law (53) with feedback control from a slightly different viewpoint from Sec. 7. We also discuss the fundamental energy cost required for the measurement process.

As a preliminary, we consider general dynamics of the bipartite system and in the presence of a heat bath at temperature . The entropy production in the total system is given by

 Σ(XY):=ΔS(XY)−βQXY, (69)

where is the change in the joint Shannon entropy, and is the heat absorbed by the total system. We can also define the entropy production in the subsystem by

 Σ(X):=ΔS(X)−βQX, (70)

where is the change in the Shannon entropy of , and is the heat absorbed by from the heat bath. In the same manner, we define

 Σ(Y):=ΔS(Y)−βQY. (71)

In many physical situations (e.g., a bipartite Markov jump process and a Langevin system with two variables driven by independent noise), we can suppose that the heat is additive:

 QXY=QX+QY. (72)

On the other hand, the Shannon entropy is generally not additive, and the mutual information appears:

 ΔS(XY)=ΔS(X)+ΔS(Y)−ΔI(X:Y). (73)

By using Eqs. (72) and (73), the total entropy production is decomposed as

 Σ(XY)=Σ(X)+Σ(Y)−ΔI(X:Y), (74)

where the total entropy production is not additive too, because of the mutual information term. This observation is crucial to understand the consistency between Maxwell’s demon and the second law, as discussed below. We emphasize that the second law of thermodynamics always applies to the total entropy production:

 Σ(XY)≥0. (75)

Correspondingly, a process is thermodynamically reversible if and only if .

We note that the terminology of the entropy “production” for the subsystems (i.e., and ) is a little bit of an abuse. More precisely, is the sum of the entropy increase in and that in the bath associated with dynamics of . Strictly speaking, the terminology of “production” should be reserved for the entropy increase of the total system, not for that of a subsystem. In the following, however, for the sake of simplicity, we refer to and just as the entropy production of the subsystems.

We now consider stochastic thermodynamics of the measurement and feedback processes (see Fig. 9 for a schematic). We suppose that subsystem is an engine measured and controlled by the demon, and subsystem plays the role of the memory of the demon. The Szilard engine discussed above is a special case of this setup; Fig. 10 shows the dynamics of the Szilard engine along with the memory of the demon. To avoid too much complication, we do not explicitly formulate the computational states of the demon in the following, while it is straightforward to consider them [37].

We first consider the measurement process. Before the measurement, the system and the demon are not correlated, and the mutual information is zero. Let be the initial state of the engine and the initial state of the demon. During the measurement, the dynamics of the demon depends on the initial state of the system. For simplicity, we assume that the measurement reads out the instantaneous value of , and the system does not evolve during the measurement. After the measurement, the state of the demon, denoted as , is correlated with . Here, is supposed to be equivalent to the measurement outcome in Sec. 7.

Let be the mutual information between and . The mutual-information change during the measurement is given by

 ΔImeas=I, (76)

which is positive if the demon gains information. From Eq. (74), the entropy production in the total system during the measurement is given by

 Σ(XY)meas=Σ(X)meas+Σ(Y)meas−I. (77)

From the assumption that the system does not evolve during the measurement, . Therefore, we obtain

 Σ(XY)meas=Σ(Y)meas−I. (78)

Since the second law applies to the total entropy production, , we obtain

 Σ(Y)meas≥I. (79)

This implies that the entropy production of the memory during the measurement is bounded from below by the mutual information.

In terms of the nonequilibrium free energy (13), we rewrite inequality (79) as

 Wmeas≥ΔF(Y)meas+TI, (80)

where . Inequality (80) reveals the fundamental lower bound of the energy cost for the measurement. Here, on the right-hand side comes from the right-hand side of Eq. (79), and represents the additional energy cost to obtain the mutual information . This inequality has been derived in Refs. [35, 50].

We next consider the feedback process, where the dynamics of the engine depends on the measurement outcome . For simplicity, we assume that the memory does not evolve during the measurement (i.e., remains unchanged). After the feedback, the final state of the system is , and the remaining correlation between and is denoted as . The mutual-information change during feedback is then given by

 ΔIfb=Irem−I. (81)

This is negative if the obtained information is used during the feedback by the demon as discussed in Sec. 7. We note that , , and respectively equal , , and in the notations of Sec. 7.

From Eq. (74), the entropy production in the total system during the feedback is given by

 Σ(XY)fb=Σ(X)fb+Σ(Y)fb+I−Irem. (82)

From the assumption that the memory does not evolve during the feedback,