Secure Lossy Transmission of Vector Gaussian SourcesThis work was supported by NSF Grants CCF 07-29127, CNS 09-64632, CCF 09-64645 and CCF 10-18185.

Secure Lossy Transmission of Vector Gaussian SourcesThis work was supported by NSF Grants CCF 07-29127, CNS 09-64632, CCF 09-64645 and CCF 10-18185.

Abstract

We study the secure lossy transmission of a vector Gaussian source to a legitimate user in the presence of an eavesdropper, where both the legitimate user and the eavesdropper have vector Gaussian side information. The aim of the transmitter is to describe the source to the legitimate user in a way that the legitimate user can reconstruct the source within a certain distortion level while the eavesdropper is kept ignorant of the source as much as possible as measured by the equivocation. We obtain an outer bound for the rate, equivocation and distortion region of this secure lossy transmission problem. This outer bound is tight when the transmission rate constraint is removed. In other words, we obtain the maximum equivocation at the eavesdropper when the legitimate user needs to reconstruct the source within a fixed distortion level while there is no constraint on the transmission rate. This characterization of the maximum equivocation involves two auxiliary random variables. We show that a non-trivial selection for both random variables may be necessary in general. The necessity of two auxiliary random variables also implies that, in general, Wyner-Ziv coding is suboptimal in the presence of an eavesdropper. In addition, we show that, even when there is no rate constraint on the legitimate link, uncoded transmission (deterministic or stochastic) is suboptimal; the presence of an eavesdropper necessitates the use of a coded scheme to attain the maximum equivocation.

1Introduction

Information theoretic secrecy was initiated by Wyner in [1], where he studied the secure lossless transmission of a source over a degraded wiretap channel, and obtained the necessary and sufficient conditions. Later, his result was generalized to arbitrary, i.e., not necessarily degraded, wiretap channels in [2]. In recent years, information theoretic secrecy has gathered a renewed interest, where mostly channel coding aspects of secure transmission is considered, in other words, secure transmission of uniformly distributed messages is studied.

Secure source coding problem has been studied for both lossless and lossy reconstruction cases in [3]. Secure lossless source coding problem is studied in [3]. The common theme of these works is that the legitimate receiver wants to reconstruct the source in a lossless fashion by using the information it gets from the transmitter in conjunction with its side information, while the eavesdropper is being kept ignorant of the source as much as possible. Secure lossy source coding problem is studied in [10]. In these works, unlike the ones focusing on secure lossless source coding, the legitimate receiver does not want to reconstruct the source in a lossless fashion, but within a distortion level.

The most relevant works to our work here are [15]. In [15], the author considers the secure lossy transmission of a source over a degraded wiretap channel while both the legitimate receiver and the eavesdropper have side information about the source. In [15], in addition to the degradedness that the wiretap channel exhibits, the source and side information also have a degradedness structure such that given the legitimate user’s side information, the source and the eavesdropper’s side information are independent. For this setting, in [15], a single-letter characterization of the distortion and equivocation region is provided. In particular, the optimality of a separation-based approach, i.e., the optimality of a code that concatenates a rate-distortion code and a wiretap channel code, is shown. In [16], the setting of [15] is partially generalized such that in [16], the source and side information do not have any degradedness structure. On the other hand, as opposed to the noisy wiretap channel of [15], in [16], the channel between the transmitter and receivers is assumed to be noiseless. For this setting, in [16], a single-letter characterization of the rate, equivocation, and distortion region is provided.

Here, we consider the setting of [16] for jointly Gaussian source and side information. In particular, we consider the model where the transmitter has a vector Gaussian source which is jointly Gaussian with the vector Gaussian side information of both the legitimate receiver and the eavesdropper. In this model, the transmitter wants to convey information to the legitimate user in a way that the legitimate user can reconstruct the source within a distortion level while the eavesdropper is being kept ignorant of the source as much as possible as measured by the equivocation. A single-letter characterization of the rate, equivocation, and distortion region for this setting exists due to [16]. Although we are unable to evaluate this single-letter characterization for the vector Gaussian source and side information case to obtain the corresponding rate, equivocation, distortion region explicitly, we obtain an outer bound for this region. We obtain this outer bound by optimizing the rate and equivocation constraints separately. We note that a joint optimization of the rate and equivocation constraints for a fixed distortion level would yield the exact achievable rate and equivocation region for this fixed distortion level. Thus, optimizing the rate and equivocation constraints separately yields a larger region, i.e., an outer bound. We show that this outer bound is tight when we remove the rate constraint at the transmitter. In other words, we obtain the maximum achievable equivocation at the eavesdropper when the legitimate user needs to reconstruct the vector Gaussian source within a fixed distortion while there is no constraint on the transmission rate.

We note some implications of this result. First, we note that since there is no rate constraint on the transmitter, it can use an uncoded scheme to describe the source to the legitimate user, and, indeed, it can use any instantaneous (deterministic or stochastic) encoding scheme for this purpose. However, we show through an example that even when there is no rate constraint on the transmitter, to attain the maximum equivocation at the eavesdropper, in general, the transmitter needs to use a coded scheme. Hence, the presence of an eavesdropper necessitates the use of a coded scheme even in the absence of a rate constraint on the transmitter. Second, we note that the maximum equivocation expression has two different covariance matrices originating from the presence of two auxiliary random variables in the single-letter expression. We show through another example that both of these covariance matrices, in other words, both of these two auxiliary random variables, are needed in general to attain the maximum equivocation at the eavesdropper. The necessity of two covariance matrices, and hence two auxiliary random variables, implies that, in general, Wyner-Ziv coding scheme [17] is not sufficient to attain the maximum equivocation at the eavesdropper.

2Secure Lossy Source Coding

Here, we describe the secure lossy source coding problem and state the existing results. Let denote i.i.d. tuples drawn from a distribution . The transmitter, the legitimate user and the eavesdropper observe and , respectively. The transmitter wants to convey information to the legitimate user in a way that the legitimate user can reconstruct the source within a certain distortion, and meanwhile the eavesdropper is kept ignorant of the source as much as possible as measured by the equivocation. We note that if there was no eavesdropper, this setting would reduce to the Wyner-Ziv problem [17], for which a single-letter characterization for the minimum transmission rate of the transmitter for each distortion level exists.

The distortion of the reconstructed sequence at the legitimate user is measured by the function where denotes the legitimate user’s reconstruction of the source . We consider the function that has the following form

where is a non-negative finite-valued function. The confusion of the eavesdropper is measured by the following equivocation term

where , which is a function of the source , denotes the signal sent by the transmitter.

An code for secure lossy source coding consists of an encoding function at the transmitter and a decoding function at the legitimate user . A rate, equivocation and distortion tuple is achievable if there exists an code satisfying

The set of all achievable tuples is denoted by which is given by the following theorem.

The achievable scheme that attains the region has the same spirit as the Wyner-Ziv scheme [17] in the sense that both achievable schemes use binning to exploit the side information at the legitimate user, and consequently, to reduce the rate requirement. The difference of the achievable scheme that attains comes from the additional binning necessitated by the presence of an eavesdropper. In particular, the transmitter generates sequences and bins both sequences. The transmitter sends these two bin indices. Using these bin indices, the legitimate user identifies the right sequences, and reconstructs within the required distortion. On the other hand, using the bin indices of , the eavesdropper identifies only the right sequence, and consequently, does not contribute to the equivocation, see ( ?)1. Indeed, this achievable scheme can be viewed as if it is using a rate-splitting technique to send the message , since has two coordinates, one for the bin index of , and one for the bin index of . This perspective reveals the similarity of the achievable scheme that attains and the one that attains the capacity-equivocation region of the wiretap channel [2] where also rate-splitting is used. In particular, in the latter case, the message is divided into two parts such that is sent by the sequence and is sent by the sequence . The eavesdropper decodes whereas the other message contributes to the secrecy.

We note that Theorem ? holds for continuous by replacing the discrete entropy term with the differential entropy term . To avoid the negative equivocation that might arise because of the use of differential entropy, we replace equivocation with the mutual information leakage to the eavesdropper defined by

Once we are interested in the mutual information leakage to the eavesdropper, a rate, mutual information leakage, and distortion tuple is said to be achievable if there exists an code such that

The set of all achievable tuples is denoted by . Using Theorem ?, the region can be stated as follows.

3Vector Gaussian Sources

Now we study the secure lossy source coding problem for jointly Gaussian where the tuples are independent across time, i.e., across the index , and each tuple is drawn from the same jointly Gaussian distribution . In other words, we consider the case where is a zero-mean Gaussian random vector with covariance matrix , and the side information at the legitimate user and the eavesdropper are jointly Gaussian with the source . In particular, we assume that have the following form

where and are independent zero-mean Gaussian random vectors with covariance matrices and , respectively, and and are independent. We note that the side information given by (Equation 1)-( ?) are not in the most general form. In the most general case, we have

for some matrices. However, until Section Section 5, we consider the form of side information given by (Equation 1)-( ?), and obtain our results for this model. In Section 5, we generalize our results to the most general case given by (Equation 2)-( ?). We note that since the rate, information leakage and distortion region is invariant with respect to the correlation between and , the correlation between and is immaterial.

The distortion of the reconstructed sequence is measured by the mean square error matrix:

Hence, the distortion constraint is represented by a positive semi-definite matrix , which is achievable if there is an code such that

Throughout the paper, we assume that . Since the mean square error is minimized by the minimum mean square error (MMSE) estimator which is given by the conditional mean, we assume that the legitimate user applies this optimal estimator, i.e., the legitimate user selects its reconstruction function as

Once the estimator of the legitimate user is set as (Equation 3), using Theorem ?, a single-letter description of the region for a vector Gaussian source can be given as follows.

We also define the region as the union of the pairs that are achievable when the distortion constraint matrix is set to . Our main result is an outer bound for the region , hence for the region .

We will prove Theorem ? in Section 4. In the remainder of this section, we provide interpretations and discuss some implications of Theorem ?.

The outer bound in Theorem ? is obtained by minimizing the constraints on and individually, i.e., the rate lower bound in ( ?) is obtained by minimizing the rate constraint in ( ?) and the mutual information leakage lower bound in ( ?) is obtained by minimizing the mutual information leakage constraint in ( ?) separately. However, to characterize the rate and mutual information leakage region , one needs to minimize the rate constraint in ( ?) and the mutual leakage information constraint in ( ?) jointly, not separately. In particular, since the region is convex in the pairs as per a time-sharing argument, joint optimization of the rate constraint in ( ?) and the mutual information leakage constraint in ( ?) can be carried out by considering the tangent lines to the region , i.e., by solving the following optimization problem

for all values of , where . As of now, we have been unable to solve the optimization problem for all values of . However, as stated in Theorem ?, we solve the optimization problems and by showing the optimality of jointly Gaussian to evaluate the corresponding cost functions. In other words, our outer bound in Theorem ? can be written as follows.

We note that the constraint in ( ?), and hence , gives us the Wyner-Ziv rate distortion function [17] for the vector Gaussian sources. Moreover, we note that gives us the minimum mutual information leakage to the eavesdropper when the legitimate user wants to reconstruct the source within a fixed distortion constraint while there is no concern on the transmission rate . Denoting the minimum mutual information leakage to the eavesdropper when the legitimate user needs to reconstruct the source within a fixed distortion constraint by , the corresponding result can be stated as follows.

Theorem ? implies that if the transmitter’s aim is to minimize the mutual information leakage to the eavesdropper without concerning itself with the rate it costs as long as the legitimate receiver is able to reconstruct the source within a distortion constraint , the use of jointly Gaussian is optimal. Since in Theorem ?, there is no rate constraint, one natural question to ask is whether can be achieved by an uncoded transmission scheme. Now, we address this question in a broader context by letting the encoder use any instantaneous encoding function in the form of where can be a deterministic or a stochastic mapping. When is chosen to be stochastic, we assume it to be independent across time. We note that the uncoded transmission can be obtained from instantaneous encoding by selecting to be a linear function. Similarly, uncoded transmission with artificial noise can be obtained from instantaneous encoding by selecting , where denotes the noise. Hence, if the encoder uses an instantaneous encoding scheme, the transmitted signal is given by . Let be the minimum information leakage to the eavesdropper when the legitimate user is able to reconstruct the source with a distortion constraint while the encoder uses an instantaneous encoding. The following example demonstrates that, in general, cannot be achieved by instantaneous encoding.

This example shows that an uncoded transmission is not optimal even when there is no rate constraint. This is due to the presence of an eavesdropper; the presence of an eavesdropper necessitates the use of a coded scheme.

Another question that Theorem ? brings about is whether the minimum in ( ?) is achieved by a non-trivial . By a trivial selection for we mean either or . The former corresponds to the selection and the latter corresponds to the selection . We note that although ( ?) is monotonically decreasing in in the positive semi-definite sense, ( ?) is neither monotonically increasing nor monotonically decreasing in in the positive semi-definite sense. Hence, due to this lack of monotonicity of in , in general, we expect that both and may be necessary to attain the minimum in ( ?). The following example demonstrates that in general and may be necessary.

Example ? shows that, in general, we might need two covariance matrices, and hence two different auxiliary random variables, to attain the minimum information leakage. Indeed, if we have either or , the corresponding achievable scheme is identical to the Wyner-Ziv scheme [17]. Hence, the necessity of two different auxiliary random variables implies that, in general, Wyner-Ziv scheme [17] is suboptimal.

4Proof of Theorem

We now provide the proof of Theorem ?. As mentioned in the previous section, this outer bound is obtained by minimizing the rate constraint in ( ?) and the mutual information leakage constraint in ( ?) separately. We first consider the rate constraint in ( ?) as follows

where ( ?) comes from the fact that is maximized by jointly Gaussian , and ( ?) comes from the monotonicity of in positive semi-definite matrices. Now we introduce the following lemma.

The proof of Lemma ? is given in Appendix Section 11. Lemma ? and ( ?) imply ( ?).

Next, we consider the mutual information leakage constraint in ( ?) as follows

We note that the cost function of can be rewritten as follows

where (Equation 7) comes from the Markov chain and ( ?) comes from the Markov chain . We note that the first term in ( ?) is minimized by a jointly Gaussian as we already showed in obtaining the lower bound for the rate given by ( ?) above in (Equation 5)-( ?). On the other hand, the remaining term of ( ?) in the bracket is maximized by a jointly Gaussian as shown in [18]. Thus, a tension between these two terms arises if is selected to be jointly Gaussian. In spite of this tension, we will still show that a jointly Gaussian is the minimizer of . Instead of directly showing this, we first characterize the minimum mutual information leakage when is restricted to be jointly Gaussian, and show that this cannot be attained by any other distribution for . We note that any jointly Gaussian can be written as

where are zero-mean Gaussian random vectors with covariance matrices , respectively. Moreover, are independent of but can be dependent on each other. Before characterizing the minimum mutual information leakage when is restricted to be jointly Gaussian, we introduce the following lemma.

The proof of Lemma ? is given in Appendix Section 12. Using Lemma ?, the minimum mutual information leakage to the eavesdropper when is restricted to be jointly Gaussian can be written as follows:

We note that the minimization in (Equation 8) can be written as a minimization of the cost function in (Equation 8) over all possible matrices by expressing and in terms of . Instead of considering this tedious optimization problem, we consider the following one:

We note that due to the Markov chain , we always have . A proof of this fact is given in Appendix Section 13. Besides this inequality, and might have further interdependencies which are not considered in the optimization problem in (Equation 9). Since neglecting these further interdependencies among and enlarges the feasible set of the optimization problem in (Equation 8), we have, in general,

On the other hand, it can be shown that the value of can be obtained by some jointly Gaussian satisfying the Markov chain , as stated in the following lemma.

The proof of Lemma ? is given in Appendix Section 14.

Now we study the optimization problem in (Equation 9) in more detail. Let and be the minimizers for the optimization problem . They need to satisfy the following KKT conditions.

The proof of Lemma ? is given in Appendix Section 15.

Next, we use channel enhancement [19]. In particular, we enhance the legitimate user’s side information as follows.

This new covariance matrix has some useful properties which are listed in the following lemma.

The proof of Lemma ? is given in Appendix Section 16. Using this new covariance , we define the enhanced side information at the legitimate user as follows

where is a zero-mean Gaussian random vector with covariance matrix . Since we have and as stated in the second statement of Lemma ?, without loss of generality, we can assume that the following Markov chain exists.

Assuming that the Markov chain in (Equation 12) exists does not incur any loss of generality because the rate, mutual information leakage and distortion region depends only on the conditional marginal distributions but not on the conditional joint distribution . Now, we define the following optimization problem:

We note that we have due to the Markov chain in (Equation 12), which leads to the following fact:

Moreover, unlike the original optimization problem in (Equation 6), we can find the minimizer of the new optimization problem explicitly, as stated in the following lemma.

We note that Lemma ? implies that and a Gaussian leading to is the minimizer of the optimization problem . The proof of Lemma ? is given in Appendix Section 17.

Next, we show that indeed which, in view of (Equation 14), will imply . To this end, using Lemma ?, we have

where (Equation 15) comes from the last statement of Lemma ?, ( ?) follows from the fifth statement of Lemma ?, and ( ?) comes from the fourth statement of Lemma ?. In view of (Equation 14), ( ?) implies that ; completing the proof of Theorem ? as well as the proof of Theorem ? due to the fact that .

5General Case

We now consider the general case where the side information are given by

where without loss of generality, we can assume that the covariance matrices of Gaussian vectors and are given by identity matrices. We denote the singular value decomposition of and by and , respectively. Since any invertible transformation applied to the side information does not change the rate, information leakage, and distortion region, the side information given by (Equation 16)-( ?) and the side information obtained by multiplying (Equation 16)-( ?) by , respectively, yield the same rate, information leakage and distortion region. In other words, the side information given by (Equation 16)-( ?) and the side information given by

yield the same rate, information leakage and distortion region, where the covariance matrices of are given by identity matrices. Next, we claim that there is no loss of generality to assume that the side information and have the same length as the source . To this end, assume that the length of is smaller than the length of . In this case, simply, we can concatenate with some zero vector to ensure that both and have the same length. Next, assume that the length of is larger than the length of . In this case, will definitely have at least diagonal elements which are zero, and hence the corresponding entries in will come from only the noise. Since noise components are independent, dropping these elements of does not change the rate, information leakage and distortion region. Thus, without loss of generality, we can assume that , and hence without loss of generality, we can assume that is a square matrix. The same argument applies to the eavesdropper’s side information, and hence, without loss of generality, we can also assume that is a square matrix. Next, we define the following side information

where . We note that and are invertible matrices. Since multiplying the side information in (Equation 18)-( ?) by some invertible matrices does not change the rate, information leakage and distortion region, the side information in (Equation 18)-( ?) and the following side information

have the same rate, information leakage and distortion region, where the covariance matrices of and are given by

respectively. For a given distortion constraint , we denote the rate and information leakage region for the side information model given in (Equation 16)-( ?) by , where the subscript stands for the “original system”, and for the side information model given in (Equation 19)-( ?) by . We have the following relationship between and .

The proof of Lemma ? is given in Appendix Section 18. Next, using Theorem ?, we obtain an outer bound for the region , where this outer bound also serves as an outer bound for the region due to Lemma ?. The corresponding result is stated in the following theorem.

The proof of Theorem ? is given in Appendix Section 19. We prove Theorem ? in two steps. In the first step, by using Theorem ?, we obtain an outer bound for the region , and in the second step, we obtain the limit of this outer bound as . As the outer bound in Theorem ? basically comes from the outer bound in Theorem ?, all our previous comments and remarks about Theorem ? are also valid for the outer bound in Theorem ?. Similar to Theorem ?, Theorem ? also provides the minimum information leakage to the eavesdropper when the rate constraint on the transmitter is removed. Denoting the corresponding minimum information leakage by , we have the following theorem.

As Theorem ? basically comes from Theorem ?, all our previous comments and remarks about Theorem ? are also valid for Theorem ?.

6Conclusions

In this paper, we study secure lossy source coding for vector Gaussian sources, where the transmitter sends information about the source in a way that the legitimate user can reconstruct the source within a distortion level by using its side information. Meanwhile, the transmitter wants to keep the mutual information leakage to the eavesdropper to a minimum, where the eavesdropper also has a side information about the source. We obtain an outer bound for the achievable rate, mutual information leakage, and distortion region. Moreover, we obtain the minimum mutual information leakage to the eavesdropper when the legitimate user needs to reconstruct the source within a certain distortion while there is no constraint on the transmission rate.

7Proof of ()

We first define the following function

which is monotonically decreasing, continuous and convex in . Next, we note that when an instantaneous encoding scheme is used, the minimum-mean-square-error estimator is given by

where (Equation 21) comes from the independence of across time. Consequently, when an instantaneous encoding scheme is used, the minimum-mean-square-error is given by

Assume that there exists an instantaneous encoding scheme that achieves the distortion level :

We now obtain a lower bound for the minimum information leakage for this instantaneous encoding scheme as follows

where (Equation 23) comes from the independence of across time, ( ?) follows by setting , ( ?) comes from the definition of , ( ?) is due to the convexity of in , ( ?) follows from the fact that is continuous in , and ( ?) comes from (Equation 22) and the fact that is monotonically decreasing in .

8Proof of Lemma

We first introduce two lemmas that will be used in the proof of Lemma ?. Throughout this appendix, we use notation to denote “ and are independent” to shorten the presentation.

 Since a set of random variables is independent iff their joint characteristic function is the product of their individual characteristic functions, to prove Lemma ?, it is sufficient to show the following.

We can show this as follows

where (Equation 24) comes from the Markov chain and ( ?) follows from the fact that . Equation ( ?) implies the independence between and ; completing the proof of Lemma ?.

 Similar to the proof of Lemma ?, here also we use the fact that a set of random variables is independent iff their joint characteristic function is the product of their individual characteristic functions. To this end, since , we have

If we set in (Equation 25), we get

On the other hand, since , we have

where (Equation 27) comes from the fact that . In view of (Equation 26) and (Equation 27), we have

which implies that ; completing the proof of Lemma ?.

We now prove Lemma ?. We note that we have iff the Markov chain holds. We prove by contradiction that when , the Markov chain is not possible. To this end, we note that the side information at the eavesdropper can be written as

or in other words, we have where is a Gaussian random variable independent of with variance . Next, we note that the Markov chain implies in view of Lemma ?. Since are jointly Gaussian, can be written as

where , and as a consequence of this choice, we have . Hence, if we have the Markov chain

then, Lemma ? implies that , where is

Since , we have , and also due to the assumption that the Markov chain holds. Hence, in view of Lemma ?, we have . Moreover, since we have the Markov chain , implies that . Hence, if , we have . However, if , is not feasible, and this implies that the Markov chain is not possible; completing the proof of Lemma ?.

9Proof of ()

Here, we provide the proof of ( ?). To this end, we consider a slightly more general case where the joint distribution of the source and side information is given by

and the distortion constraint is imposed with a diagonal matrix whose diagonal entries are denoted by . From Theorem ?, the minimum information leakage is given by

We first introduce the following auxiliary random variables

which satisfy the Markov chain

which follows from (Equation 28) and the Markov chain .

Next, we introduce the following two lemmas.

Using Lemma ?, the following lemma can be proved.

Now, we proceed with (Equation 29) as follows