Swiveled Rényi entropies

Swiveled Rényi entropies

Abstract

This paper introduces swiveled Rényi entropies as an alternative to the Rényi entropic quantities put forward in [Berta et al., Phys. Rev. A 91, 022333 (2015)]. What distinguishes the swiveled Rényi entropies from the prior proposal of Berta et al. is that there is an extra degree of freedom: an optimization over unitary rotations with respect to particular fixed bases (swivels). A consequence of this extra degree of freedom is that the swiveled Rényi entropies are ordered, which is an important property of the Rényi family of entropies. The swiveled Rényi entropies are however generally discontinuous at and do not converge to the von Neumann entropy-based measures in the limit as , instead bounding them from above and below. Particular variants reduce to known Rényi entropies, such as the Rényi relative entropy or the sandwiched Rényi relative entropy, but also lead to ordered Rényi conditional mutual informations and ordered Rényi generalizations of a relative entropy difference. Refinements of entropy inequalities such as monotonicity of quantum relative entropy and strong subadditivity follow as a consequence of the aforementioned properties of the swiveled Rényi entropies. Due to the lack of convergence at , it is unclear whether the swiveled Rényi entropies would be useful in one-shot information theory, so that the present contribution represents partial progress toward this goal.

1Introduction

In 1961, Alfred Rényi defined a parametrized family of entropies now bearing his name, by relaxing one of the axioms that singles out the Shannon entropy [30]. This led to both the -Rényi entropy and the -Rényi divergence, defined respectively for a parameter and probability distributions and as

where denotes the natural logarithm here and throughout the paper. The Shannon entropy and relative entropy are recovered in the limit as :

What began largely as a theoretical exploration ended up having many practical ramifications, especially in the contexts of information theory and statistics. For example, it is now well known that the Rényi entropies play a fundamental role in obtaining a sharpened understanding of the trade-off between communication rate, error probability, and number of resources in communication protocols, such as data compression and channel coding [9]. Smoothing the Rényi entropies [33] has also led to the development of one-shot information theory [31], with applications to cryptography.

Part of what makes the Rényi entropies so useful in applications is their properties: convergence to the Shannon and relative entropies in the limit as , monotonicity in the parameter, and additivity, in addition to others. The convergence to the Shannon and relative entropies ensures that, by taking this limit, one recovers asymptotic information-theoretic statements, such as the data compression theorem or the channel capacity theorem, from the more fine-grained statements. Monotonicity in the parameter ensures that gives more weight to low surprisal events for and vice versa for , helping to characterize the aforementioned trade-off in information-theoretic settings. The additivity property implies that the Rényi entropies can simplify immensely when evaluated for memoryless stochastic processes.

In light of the progress that the Rényi paradigm has brought to information theory, one is left to wonder if this could happen in more exotic settings, such as quantum information theory and/or for multipartite settings (here by multipartite, we mean three or more parties). This line of thought has led to the development of several non-commutative generalizations of the Rényi relative entropy in ( ?), which has in turn led to a sharpened understanding of several quantum information-theoretic tasks (see [8] and references therein) and refinements of the uncertainty principle [7]. As far as we are aware, the development of the multipartite generalization of the Rényi entropy in (Equation 1) is less explored, with the exception of a recent proposal [6] for a multipartite quantum generalization.

With the intent of developing either a multipartite classical or quantum generalization of (Equation 1), one might suggest after a moment’s thought to replace a quantity which features a linear combination of entropies by one with the same linear combination of Rényi entropies. However, this approach is objectively unsatisfactory in at least two regards: properties of the original information measure are not preserved by doing so and one is not guaranteed to have the powerful monotonicity in property mentioned above. For example, take the case of the conditional mutual information of a tripartite density operator defined as

where is the quantum entropy of a density operator on system . One of the most important properties of this quantity is that it is non-negative (known as strong subadditivity of quantum entropy [22]), and as a consequence, it is monotone non-increasing with respect to any quantum channel applied to the system [10] (by symmetry, the same is true for one applied to ). However, if we define a Rényi generalization of as , where , then explicit counterexamples reveal that this Rényi generalization can be negative, monotonicity with respect to quantum channels need not hold, and neither does monotonicity in [21].

To remedy these deficiencies, the authors of [6] put forward a general prescription for producing a Rényi generalization of a quantum information measure, with the aim of having the properties of the original measure retained while also satisfying the monotonicity in property. The work in [6] was only partially successful in this regard. Continuing with our example of conditional mutual information, consider the following Rényi generalization [5]:

For , the quantity is non-negative, monotone non-increasing with respect to quantum channels acting on the system, converges to in the limit as , and is conjectured to obey the monotonicity in property (with some numerical and analytical evidence in favor established) [5]. However, hitherto a proof of the monotonicity in property for remains lacking. It is also an open question to determine whether is monotone non-increasing with respect to quantum channels acting on the system—this partially has to do with the fact that is not symmetric with respect to exchange of the and systems, unlike the conditional mutual information in .

2Summary of results

In this paper, we modify the recently proposed Rényi generalizations of quantum information measures from [6] by placing swivels in a given chain of operators.1 As an example of the idea, consider that we can rewrite the quantity in (Equation 3) in terms of the Schatten 2-norm as follows:

The new idea is to modify this quantity to include swivels as follows:

where is the compact set of all unitaries commuting with the Hermitian operator . Thus, the fixed eigenbases of and act as swivels connecting adjacent operators in the operator chain above, such that the unitary rotations and about these swivels are allowed. Of course, such swivels make no difference when the density operator and its marginals commute with each other (the classical case), or when the system is trivial, in which case the above quantity reduces to a Rényi mutual information

We mention that we were led to the definition in (Equation 4) as a consequence of the developments in [45], in which similar swivels appeared in refinements of entropy inequalities such as monotonicity of quantum relative entropy and strong subadditivity.

The quantity in (Equation 4) satisfies some of the properties already established for in [5], which include non-negativity for and monotonicity with respect to quantum channels acting on the system. However, the extra degree of freedom in (Equation 4) allows us to prove that this swiveled Rényi conditional mutual information is monotone non-decreasing in for .

The swiveled Rényi entropies are in general discontinuous at and do not converge to the von Neumann entropy-based measures in the limit as . Thus, the present paper represents a work in progress toward the general goal of find Rényi generalizations of quantum information measures that satisfy all of the desired properties that one would like to have. It thus remains an open question to find Rényi quantities that meet all desiderata.

The rest of the paper proceeds by developing this idea in detail. We review some background material in Section 3, which includes various quantum Rényi entropies and the Hadamard three-line theorem, the latter being the essential tool for establishing monotonicity in for the swiveled Rényi entropies. We then focus in Section 4 on developing swiveled Rényi generalizations of the quantum relative entropy difference in (Equation 6), given that many different information measures can be written in terms of this relative entropy difference, including conditional mutual information (see, e.g., the discussions in [34]). Our main contributions are Theorems ? and ?, which state that these quantities are monotone non-decreasing in for particular values. We then briefly discuss how refinements of entropy inequalities follow as a consequence of the properties of the swiveled Rényi entropies. Section 5 discusses swiveled Rényi conditional mutual informations and justifies that they possess the properties stated above. We extend the idea in Section 6 to establish swiveled Rényi generalizations of an arbitrary linear combination of von Neumann entropies with coefficients chosen from the set . We finally show how our methods can be used to address an open question posed in [47]. Section 8 concludes with a summary and some open directions.

3Preliminaries

3.1Quantum states and channels

A quantum state is described mathematically by a density operator, which is a positive semi-definite operator with trace equal to one. A quantum channel is a linear, trace-preserving, completely positive map. For more background on quantum information theory, we refer to [26]. Our results apply to finite-dimensional Hilbert spaces. For most developments, we take , , and to be as given in the following definition:

Sometimes we need more restrictions, in which case we take , , and as follows:

We employ the common convention that functions of Hermitian operators are evaluated on their support. In more detail, the support of a Hermitian operator , written as , is defined as the vector space spanned by its eigenvectors whose corresponding eigenvalues are non-zero. Let an eigendecomposition of be given as for eigenvectors . Then . Let denote the projection onto the support of . A function of an operator is then defined as .

3.2Entropies and norms

Let , , and be as given in Definition ?. The quantum relative entropy [41] is defined as

whenever , and otherwise, it is defined to be equal to . The quantum relative entropy is monotone non-increasing with respect to quantum channels [20], in the sense that

Another relevant information measure is the quantum relative entropy difference, defined as

We can use the Schatten norms in order to establish Rényi generalizations of von Neumann entropies, which are more refined information measures for quantum states and channels that reduce to the von Neumann quantities in a limit. The Schatten -norm of an operator is defined as

where and (note that we sometimes use the notation even for values when the quantity on the right-hand side of (Equation 7) is not a norm). From the above definition, we can see that the following equalities hold for any operators and :

The quantum Rényi entropy of a state is defined for as

and reduces to the von Neumann entropy in the limit as :

There are at least two ways to generalize the quantum relative entropy, which we refer to as the Rényi relative entropy [27] and the sandwiched Rényi relative entropy [25]. They are defined respectively as follows:

if or if and . If and , then they are defined to be equal to . The rewritings in (Equation 9) and ( ?) are helpful for our developments in this paper and follow from (Equation 8)–( ?) and the following:

Both Rényi generalizations reduce to the quantum relative entropy in the limit as [27]:

The Rényi relative entropy is monotone non-increasing with respect to quantum channels when [27]:

and the sandwiched Rényi relative entropy possesses a similar monotonicity property when [16]:

By picking particular values of the Rényi parameter , the quantities above take on special forms and have meaning in operational contexts, being known as the zero-relative entropy [11], the collision relative entropy [12], the min-relative entropy [13], and the max-relative entropy [11], respectively:

where is the quantum fidelity [39].

3.3Hadamard three-line theorem

One of the most important technical tools for proving our main result is the operator version of the Hadamard three-line theorem given in [1], in particular, the very slight modification stated in [14]. We note that the theorem below is a variant of the Riesz-Thorin operator interpolation theorem (see, e.g., [3]).

3.4Rényi generalizations of the quantum relative entropy difference

Let , , and be as given in Definition ?. In [34], two Rényi generalizations of the relative entropy difference in (Equation 6) were defined as follows:

where . Let be an isometric extension of , so that

We can write the adjoint in terms of this isometric extension as follows:

This then allows us to write the definitions above in a simpler form:

It is known that the following limits hold for , , and taken as in Definition ? [34]:

The fact that these limits hold for , , and taken as in Definition ? and subject to follows from [45] and the development in Appendix A. [15] proved that for ,

and for :

when , , and are taken as in Definition ?. The latter inequality was refined recently in [45] for and for , , and taken as in Definition ? and subject to . It remains an open question to determine whether these quantities are non-decreasing in for any non-trivial range of (note that [34] argued that they are non-decreasing in in a neighborhood of ).

4Swiveled Rényi generalizations of the quantum relative entropy difference

In the spirit of the discussion in Section 2, we consider different definitions of and in order to allow for unitary rotations about swivels, i.e., an optimization over unitaries of the form and :

This slight extra degree of freedom allows us to establish that and are monotone non-decreasing in for particular values (see Theorems ? and ?).

4.1Reduction to Rényi relative entropy

Observe that by choosing , we find that reduces to the Rényi relative entropy whenever :

and to the sandwiched Rényi relative entropy whenever :

just as

4.2Behavior around

Here we discuss the behavior of and around , with the result being that these quantities are generally discontinuous at :

Proof. Let , which we will choose shortly. Define the function as

whenever , and as in ( ?). One can check that

for example by performing Taylor expansions to calculate the limit (see Appendix C for details of this calculation). The function is then continuous in , , and . Furthermore, it fulfills the conditions of Lemma ? in Appendix B if we choose for any and . Hence, we get that

is continuous on and thus

Repeating the same argument with yields that

is continuous on and thus

Given that , we can conclude the following inequality:

The arguments for the quantity are similar, so we just sketch them briefly. Define the function

for and set . One can then compute (again via Taylor expansions, e.g.) that

The rest of the argument proceeds as above, which leads to the other equalities in ( ?)-( ?).


4.3Monotonicity in the Rényi parameter

This section contains our main result, that both and are monotone non-decreasing with respect to for particular values.

Proof. The main tool for our proof is Theorem ?. We break the proof of inequality in ( ?) into several cases. We first consider . For some and , pick

which fixes . Then

We then apply Theorem ? to find that the following inequality holds for all and :

As a consequence, we can take the maximum over all and and apply the definition in ( ?) to establish that

We finally apply a logarithm to arrive at the conclusion that ( ?) holds for all .

To get the monotonicity for the range , we exchange and in (Equation 12)-( ?) and apply the same reasoning as in (Equation 13)-(Equation 14) to arrive at the following inequality:

Taking a negative logarithm and noting that then gives ( ?) for this range.

We are now left with proving the case and the dual parameter of , such that . Notice that . Let . We pick

so that . Consider that , so that

We then find that

Consider that

Thus, similarly, we have

Applying Theorem ? gives

which implies ( ?) for and . Putting the three cases together along with Proposition ? gives the inequality in ( ?) for , , and .


Proof. We handle the inequality in ( ?) in a similar way as in the previous proof. First, suppose that . Let and , and note that for the choices given. For some and , pick

which fixes . Then we find the following expression for

and the following ones for and :

Applying Theorem ?, we find that the following inequality holds for all and :

We can then take a maximum over all and and apply the definition in ( ?) to establish that

The inequality in ( ?) then follows for after taking a logarithm.

To get the monotonicity for the range , we exchange and in (Equation 15)-( ?) and apply the same reasoning as in (Equation 16)-(Equation 17) to arrive at the following inequality:

Taking a negative logarithm and noting that , so that