Communication with Disturbance Constraints

# Communication with Disturbance Constraints

Bernd Bandemer and Abbas El Gamal
Information Systems Laboratory, Stanford University,
350 Serra Mall, Stanford, CA 94305, USA
Email: bandemer@stanford.edu, abbas@ee.stanford.edu
This work is partially supported by DARPA ITMANET. Bernd Bandemer is supported by an Eric and Illeana Benhamou Stanford Graduate Fellowship.
###### Abstract

Motivated by the broadcast view of the interference channel, the new problem of communication with disturbance constraints is formulated. The rate–disturbance region is established for the single constraint case and the optimal encoding scheme turns out to be the same as the Han–Kobayashi scheme for the two user-pair interference channel. This result is extended to the Gaussian vector (MIMO) case. For the case of communication with two disturbance constraints, inner and outer bounds on the rate–disturbance region for a deterministic model are established. The inner bound is achieved by an encoding scheme that involves rate splitting, Marton coding, and superposition coding, and is shown to be optimal in several nontrivial cases. This encoding scheme can be readily applied to discrete memoryless interference channels and motivates a natural extension of the Han–Kobayashi scheme to more than two user pairs.

Communication with

Disturbance Constraints

Bernd Bandemer and Abbas El Gamal

Information Systems Laboratory, Stanford University,

350 Serra Mall, Stanford, CA 94305, USA

Email: bandemer@stanford.edu, abbas@ee.stanford.edu

00footnotetext:   This work is partially supported by DARPA ITMANET. Bernd Bandemer is supported by an Eric and Illeana Benhamou Stanford Graduate Fellowship.

## I Introduction

Alice wishes to communicate a message to Bob while causing the least disturbance to nearby Dick, Diane, and Diego, who are not interested in the communication from Alice. Assume a discrete memoryless broadcast channel between Alice , Bob , and their preoccupied friends as depicted in Figure 1. We measure the disturbance at side receiver by the amount of undesired information rate originating from the sender , and require this rate not to exceed in the limit. The problem is to determine the optimal trade-off between the message communication rate and the disturbance rates .

This communication with disturbance constraints problem is motivated by the broadcast side of the interference channel in which each sender wishes to communicate a message only to one of the receivers while causing the least disturbance to the other receivers. However, in this paper, which is an extended version of [1], we focus on studying the problem of communication with disturbance constraints itself. The application of the coding scheme developed in this paper to deterministic interference channels with more than two user pairs is discussed in [2].

For a single disturbance constraint, we show that the optimal encoding scheme is rate splitting and superposition coding, which is the same as the Han–Kobayashi scheme for the two user-pair interference channel [3, 4]. This motivates us to study communication with more than one disturbance constraint with the hope of finding good coding schemes for interference channels with more than two user pairs. To this end, we establish inner and outer bounds on the rate–disturbance region for the deterministic channel model with two disturbance constraints that are tight in some nontrivial special cases. In the following section we provide needed definitions and present an extended summary of our results. The proofs are presented in subsequent sections, with some parts deferred to the Appendix.

## Ii Definitions and main results

Consider the discrete memoryless communication system with disturbance constraints (henceforth referred to as DMC--DC) depicted in Figure 1. The channel consists of finite alphabets , , , , and a collection of conditional pmfs . A code for the DMC--DC consists of the message set , an encoding function , and a decoding function . We assume that the message is uniformly distributed over . A rate–disturbance tuple is achievable for the DMC--DC if there exists a sequence of codes such that

 limn→∞P(^M≠M) =0, limsupn→∞ (1/n)I(Xn;Znj) ≤Rd,j,j∈[1:K].

The rate–disturbance region of the DMC--DC is the closure of the set of all achievable tuples .

###### Remark 1.

Like the message rate , the disturbance rates , for , are measured in units of bits per channel use. (We use logarithms of base throughout.)

###### Remark 2.

The measure of disturbance can be expanded as . The first term is the entropy rate of the received signal and is caused by both the transmission itself and by noise inherent to the channel. Subtracting the second term separates out the noise part. (For channels with additive white noise, e.g., the Gaussian case, the second term is exactly the differential entropy of each noise sample.)

###### Remark 3.

Our results remain essentially true if disturbance is measured by instead. If the channel is deterministic, the two measures coincide.

###### Remark 4.

The disturbance constraint is reminiscent of the information leakage rate constraint for the wiretap channel [5, 6], that is, . Replacing with , however, dramatically changes the problem and the optimal coding scheme. In the wiretap channel, the key component of the optimal encoding scheme is randomized encoding, which helps control the leakage rate . Such randomization reduces the achievable transmission rate for a given disturbance constraint, hence is not desirable in our setting.

The rate–disturbance region is not known in general. In this paper we establish the following results.

### Ii-a Rate–disturbance region for a single disturbance constraint

Consider the case with a single disturbance constraint, i.e., , and relabel as and as . We fully characterize the rate–disturbance region for this case.

###### Theorem 1.

The rate–disturbance region of the DMC--DC is the set of rate pairs such that

 R ≤I(X;Y), Rd ≥I(X;Z|U), R−Rd ≤I(X;Y|U)−I(X;Z|U),

for some pmf with .

Let be the rate region defined by the rate constraints in the theorem for a fixed joint pmf . This rate region is illustrated in Figure 2. The rate–disturbance region is simply the union of these regions over all and is convex without the need for a time-sharing random variable.

The proof of Theorem 1 is given in Subsections III-A and III-B. Achievability is established using rate splitting and superposition coding. Receiver decodes the satellite codeword while receiver distinguishes only the cloud center. Note that this encoding scheme is identical to the Han–Kobayashi scheme for the two user-pair interference channel [3, 4].

We now consider three interesting special cases.

#### Ii-A1 Deterministic channel

Assume that and are deterministic functions of . We show that the rate–disturbance region in Theorem 1 reduces to the following.

###### Corollary 1.

The rate–disturbance region for the deterministic channel with one disturbance constraint is the set of rate pairs such that

 R ≤H(Y), R−Rd ≤H(Y|Z),

for some pmf .

Clearly, this region is convex. Alternatively, the region can be written as the set of rate pairs such that

 R ≤H(Y|Q), Rd ≥I(Y;Z|Q),

for some joint pmf with . Corollary 1 and the alternative description of the region are established by substituting in the region of Theorem 1 and simplifying the resulting region as detailed in Subsection III-C.

###### Remark 5.

Consider the injective deterministic interference channel with two user pairs depicted in Figure 3. Here, is a function that models the link from transmitter to receiver , for . The combining functions are assumed to be injective in each argument. This setting is a special case of the channel investigated in [7]. This can be seen by merging and of Figure 3 into a function that maps to . Likewise, define the function as the merger of and . The modified combining functions and are injective in and , respectively, and therefore satisfy the assumptions in [7]. It follows that the Han–Kobayashi scheme where the transmitters use superposition codebooks generated according to and achieves the capacity region of the channel in Figure 3.

On the other hand, Corollary 1 shows that the same encoding scheme achieves the disturbance-constrained capacity for the channels and , shown as dashed boxes in Figure 3. Here, and are the desired receivers, and and are the side receivers associated with disturbance constraints. Note that decodability of the desired messages at receivers and in the interference channel certainly implies decodability at and in the channels with disturbance constraint, respectively.

###### Example 1.

Consider the deterministic channel depicted in Figure 4(a) and its rate–disturbance region in Figure 4(b). Note that rates can be achieved with zero disturbance rate by restricting the transmission to input symbols (or ), which map to different symbols at , but are indistinguishable at . On the other hand, for sufficiently large , the disturbance constraint becomes inactive and is bounded only by the unconstrained capacity . In addition to the optimal region achieved by superposition coding, the figure also shows the strictly suboptimal region achieved by simple non-layered random codes.

#### Ii-A2 Gaussian channel

Consider the problem of communication with one disturbance constraint for the Gaussian channel

 Y =X+W1, Z =X+W2,

where the noise is and . Assume an average power constraint on the transmitted signal .

The case is not interesting, since then is a degraded version of and the disturbance rate is simply given by the data rate . If , is a degraded version of , and the rate–disturbance region reduces to the following.

###### Corollary 2.

The rate–disturbance region of the Gaussian channel with parameters and is the set of rate pairs such that

 R ≤C(αP), Rd ≥C(αP/N),

for some , where for .

Achievability is proved using Gaussian codes with power . The converse follows by defining such that and applying the vector entropy power inequality to , where is the excess noise. The details are given in Subsection III-D. Note that this is a degenerate form of the Han–Kobayashi scheme because the constraint from the multiple access side of the interference channel is not taken into consideration.

#### Ii-A3 Vector Gaussian channel

Now consider the vector Gaussian channel with one disturbance constraint

 Y =X+W1, Z =X+W2,

where and the noise and for some positive semidefinite covariance matrices . Assume an average transmit power constraint , where is the covariance matrix of . This case is not degraded in general.

###### Theorem 2.

The rate–disturbance region of the Gaussian vector channel with parameters , , and is the convex hull of the set of pairs such that

 R ≤12log|Ku+Kv+K1||K1|, R−Rd ≤12log|Kv+K1||Kv+K2||K2||K1|, Rd ≥12log|Kv+K2||K2|.

for some positive semidefinite matrices with .

Achievability of this rate–disturbance region is shown by applying Theorem 1. Using the discretization procedure in [8], it can be shown that the theorem continues to hold with the power constraint additionally applied to the set of permissible input distributions. The claimed region then follows by considering the special case where the input distribution is jointly Gaussian. To prove the converse, we use an extremal inequality in [9] to show that Gaussian input distributions are sufficient. The details of the proof are given in Subsection III-E.

### Ii-B Inner and outer bounds for the deterministic channel with two disturbance constraints

The correspondence between optimal encoding for the channel with one disturbance constraint and the Han–Kobayashi scheme for the interference channel suggests that the optimal coding scheme for disturbance constraints may provide an efficient (if not optimal) scheme for the interference channel with more than two user pairs. This is particularly the case for extensions of the two user-pair injective deterministic interference channel for which Han–Kobayashi is optimal [7] (see Remark 5). As such, we restrict our attention to the deterministic version of the DMC--DC.

First, we establish the following inner bound on the rate–disturbance region.

###### Theorem 3 (Inner bound).

The rate–disturbance region of the deterministic channel with two disturbance constraints is inner-bounded by the set of rate triples such that

 R ≤H(Y), (1) Rd,1+Rd,2 ≥I(Z1;Z2|U), (2) R−Rd,1 ≤H(Y|Z1,U), (3) R−Rd,2 ≤H(Y|Z2,U), (4) R−Rd,1−Rd,2 ≤H(Y|Z1,Z2,U)−I(Z1;Z2|U), (5) 2R−Rd,1−Rd,2 ≤H(Y|Z1,Z2,U)+H(Y|U) −I(Z1;Z2|U), (6)

for some pmf .

The inner bound is convex. The expression appears in three of the inequalities. As in Marton coding for the 2-receiver broadcast channel with a common message, it is the penalty incurred in encoding independent messages via correlated sequences. The region defined by the inequalities in the theorm for a fixed is illustrated in Figure 5.

###### Remark 6.

The right-hand side of condition (6) can be equivalently expressed as

 H(Y|Z1,Z2,U)+H(Y|U)−I(Z1;Z2|U) =H(Y|Z1,U)+H(Y|Z2,U)−I(Z1;Z2|U,Y),

This shows that the condition is stricter than the sum of conditions (3) and (4).

The encoding scheme for Theorem 3 involves rate splitting, Marton coding, and superposition coding. The analysis of the probability of error, however, is complicated by the fact that receiver wishes to decode all parts of the message as detailed in Subsection IV-A. Receivers and each observe a satellite codeword from a superposition codebook.

###### Remark 7.

The encoding scheme underlying the inner bound of Theorem 3 can be readily extended to the general (non-deterministic) DMC--DC.

To complement the inner bound, we establish the following outer bound on the rate–disturbance region of the deterministic channel with two disturbance constraints.

###### Theorem 4 (Outer bound).

If a rate triple is achievable for the deterministic channel with two disturbance constraints, then it must satisfy the conditions

 R ≤H(Y|Q), Rd,1 ≥I(Y;Z1|Q), Rd,2 ≥I(Y;Z2|Q),

for some pmf with .

The proof of this outer bound is given in Subsection IV-B. Note that this outer bound is very similar in form to the alternative description of Corollary 1 for the single-constraint deterministic case.

The inner bound in Theorem 3 and the outer bound in Theorem 4 coincide in some special cases. To discuss these, we introduce the following notation. Since all channel outputs are functions of , they can be equivalently thought of as set partitions of the input alphabet . Set partitions form a partially ordered set (poset) under the refinement relation. Since this poset is a complete lattice [10], the following concepts are well-defined. For two set partitions (functions) and , let denote that is a refinement of (equivalently, is degraded with respect to ), let be the intersection of the two set partitions (the function that returns both and ), and let denote the finest set partition of which both and are refinements (the Gács–Körner–Witsenhausen common part of and , cf. [11, 12]).

The inner bound of Theorem 3 coincides with the outer bound of Theorem 4 if or is a degraded version of , i.e., if the output together with the common part of and determine or completely.

###### Theorem 5.

The rate–disturbance region of the deterministic channel with two disturbance constraints is given by the outer bound of Theorem 4 if

 Y∧(Z1∨Z2) ≼Z1,or Y∧(Z1∨Z2) ≼Z2.

The theorem is proved by specializing Theorem 3 as detailed in Subsection IV-C. In the case where or is a degraded version of alone, achievability follows by setting in Theorem 3. Otherwise, we let . This is intuitive, since corresponds to the common-message step in the Marton encoding scheme.

###### Example 2.

Consider the deterministic channel depicted in Figure 6. The desired receiver output is a refinement of both side receiver outputs and , and hence, Theorem 5 applies. Figure 7(a) depicts the rate–disturbance region, numerically approximated by evaluating each grid point in a regular grid over the distributions and subsequently taking the convex hull. Figure 7(b) contrasts the single-constraint case (where is set to infinity, and thus inactive) with the case where both side receivers are under the same disturbance rate constraint (). As expected, imposing an additional disturbance constraint can significantly reduce the achievable message rate. Finally, Figure 7(c) illustrates the trade-off between the disturbance rates and at the two side receivers, for a fixed data rate .

We conclude this section by considering another case in which we can fully characterize the rate–disturbance region of the deterministic channel with two disturbance constraints. If is a degraded version of (or vice versa), the region of Theorem 3 is optimal and simplifies to the following.

###### Corollary 3.

The rate–disturbance region of the deterministic channel with two disturbance constraints with or is the set of rate triples such that

 R ≤H(Y), R−Rd,1 ≤H(Y|Z1), R−Rd,2 ≤H(Y|Z2).

for some pmf .

Achievability follows as a special case of Theorem 3. The encoding scheme underlying the theorem carefully avoids introducing an ordering between the side receiver signals and , but such ordering is naturally given by the channel here. Consequently, the corollary follows by setting the auxiliary equal to the output at the degraded side receiver. This turns the encoding scheme into superposition coding with three layers. The details are given in Subsection IV-D.

Note that the region of Corollary 3 is akin to the deterministic case with one disturbance constraint in Corollary 1. In both cases, the side receiver signals need not be degraded with respect to .

## Iii Proofs for a single disturbance constraint

### Iii-a Achievability proof of Theorem 1

Achievability is proved as follows.

Codebook generation. Fix a pmf .

1. Split the message into two independent messages and with rates and , respectively. Hence .

2. For each , independently generate a sequence according to .

3. For each , independently generate a sequence according to .

Encoding. To send message , transmit .

Decoding. Upon receiving , find the unique such that .

Analysis of the probability of error. We are using a superposition code over the channel from to . Using the law of large numbers and the packing lemma in [8], it can be shown that the probability of error tends to zero as if

 R1

Analysis of disturbance rate. We analyze the disturbance rate averaged over codebooks .

 I(Xn;Zn|C) ≤H(Zn,M0|C)−H(Zn|Xn,C) =H(M0)+H(Zn|M0,C)−H(Zn|Xn) (a)≤nR0+H(Zn|Un)−nH(Z|X) ≤nR0+nH(Z|U)−nH(Z|X,U) =nR0+nI(X;Z|U) ≤nRd, (9)

where (a) follows since is a function of the codebook and . Substituting and using Fourier–Motzkin elimination on inequalities (7), (8), and (9) completes the proof of achievability.

### Iii-B Converse of Theorem 1

Consider a sequence of codes with as and the joint pmf that it induces on assuming . Define the time-sharing random variable , independent of everything else. We use the identification , and let , , and . Note that is consistent with the channel. Then

 R ≤I(X;Y)+εn,

as in the converse proof for point-to-point channel capacity, which uses the same identifications of random variables. On the other hand,

 nRd ≥I(Xn;Zn) =H(Zn)−H(Zn|Xn) =n∑i=1(H(Zi|Zi−1)−H(Zi|Xi)) ≥n∑i=1H(Zi|Zi−1,Yni+1)−nH(Z|X) =nH(Z|U)−nH(Z|X,U) =nI(X;Z|U).

Finally,

 n(Rd−R) ≥I(Xn;Zn)−nR (a)≥H(Zn)−H(Zn|Xn)−I(M;Yn)−nεn (b)=n∑i=1(H(Zi|Zi−1)−I(M;Yi|Yni+1))−nH(Z|X)−nεn =n∑i=1(H(Zi|Zi−1,Yni+1)+I(Yni+1;Zi|Zi−1) −H(Yi|Yni+1)+H(Yi|M,Yni+1))−nH(Z|X)−nεn (c)=n∑i=1(H(Zi|Zi−1,Yni+1)+I(Yi;Zi−1|Yni+1) −H(Yi|Yni+1)+H(Yi|Xi))−nH(Z|X)−nεn =n∑i=1(H(Zi|Zi−1,Yni+1)−H(Yi|Zi−1,Yni+1) +H(Yi|Xi,Zi−1,Yni+1))−nH(Z|X)−nεn =n∑i=1(H(Zi|Zi−1,Yni+1)−I(Xi;Yi|Zi−1,Yni+1)) −nH(Z|X)−nεn (d)=nH(Z|U)−nI(X;Y|U)−nH(Z|X,U)−nεn =nI(X;Z|U)−I(X;Y|U)−nεn,

where (a) uses Fano’s inequality, (b) single-letterizes the noise term with equality due to memorylessness of the channel, (c) applies Csiszár’s sum identity to the second term and channel memorylessness to the fourth term, and (d) uses the previous definitions of auxiliary random variables. Finally, the cardinality bound on is established using the convex cover method in [8].

### Iii-C Proof of Corollary 1

Using the deterministic nature of the channel, the region in Theorem 1 reduces to the set of rate pairs such that

 R ≤H(Y), (10) Rd ≥H(Z|U), (11) Rd ≥R+H(Z|U)−H(Y|U), (12)

for some pmf . Now fixing a rate and a pmf and varying to minimize , the right hand sides of (11) and (12) are lower bounded by

 H(Z|U)≥0,

and

 R+H(Z|U)−H(Y|U) =R+H(Z|U)−H(Y,Z|U)+H(Z|Y,U) =R−H(Y|Z,U)+H(Z|Y,U) ≥R−H(Y|Z).

Note that the particular choice simultaneously achieves both lower bounds with equality and is therefore sufficient. The rate–disturbance region thus reduces to Corollary 1.

For a fixed pmf , this region has exactly two corner points: and . As we vary , there is one corner point that dominates all other points. The pmf for this dominant can be constructed by maximizing as follows. For each , define to be the set of symbols that are compatible with . Let be a symbol that maximizes . For each element of , pick exactly one that is compatible with it and . Finally, place equal probability mass on each of these values, and zero mass on all others. This pmf on yields the dominant corner point , namely . Moreover, for this distribution, coincides with . Therefore, the net contribution (modulo convexification) of each pmf to the rate–disturbance region amounts to its corner point . This implies the alternative description of the region. Lastly, the cardinality bound on in the alternative description is follows from the convex cover method in [8].

### Iii-D Proof of Corollary 2

Achievability is straightforward using a random Gaussian codebook with power control, and upper-bounding the disturbance rate at receiver by white Gaussian noise. The converse can be seen as follows. Clearly, . Let be such that . Then

 nC(α⋆P)=nR ≤I(Xn;Yn)+nεn =h(Yn)−h(Yn|Xn)+nεn,

and therefore,

 h(Yn) ≥n2log(2πe)+nC(α⋆P)−nεn =n2log(2πe(1+α⋆P))−nεn

Since , we can write the physically degraded form of the channel as , , where is the excess noise that receiver experiences in addition to receiver . Applying the vector entropy power inequality to , we conclude

 1nh(Zn) ≥12log(22nh(Yn)+22nh(~Wn2)) ≥12log(2−2εn⋅2πe(1+α⋆P)+2πe(N−1)) ≥12log(2πe(N+α⋆P))−εn,

and finally,

 Rd ≥1nI(Xn;Zn) =1nh(Zn)−12log(2πeN) ≥C(α⋆P/N)−εn.

### Iii-E Proof of Theorem 2

Recall the shape of depicted in Figure 2. The coordinates of the corner points and are given by

 A(U,X):R =h(X+W1)−h(W1), (13) Rd =h(X+W2|U)+h(X+W1)−h(X+W1|U)−h(W2), (14) B(U,X):R =h(X+W1|U)−h(W1), (15) Rd =h(X+W2|U)−h(W2). (16)
###### Proof of achievability.

We specialize Theorem 1. Consider the specific constructed as follows. For given positive semidefinite matrices with , let

 U ∼N(0,Ku), V ∼N(0,Kv), X =U+V,

where and are independent. Then, the terms in Theorem 1 evaluate to

 I(X;Y) =h(Y)−h(W1)=12log|Ku+Kv+K1||K1|, I(X;Y|U) =h(Y|U)−h(W1)=12log|Kv+K1||K1|, I(X;Z|U) =h(Z|U)−h(W2)=12log|Kv+K2||K2|.

Simplifying the right hand sides and introducing time-sharing leads to the desired result.

For completeness, the coordinates of and for given matrices , are

 A(Ku,Kv):R =12log|Ku+Kv+K1||K1|, (17) Rd =12log|Kv+K2||K2||K