Gaussian Channel with Noisy Feedback and Peak Energy Constraint

# Gaussian Channel with Noisy Feedback and Peak Energy Constraint

Yu Xiang and Young-Han Kim The authors are with the Department of Electrical and Computer Engineering, University of California, San Diego, La Jolla, CA 92093 USA e-mail: (yxiang@ucsd.edu; yhk@ucsd.edu).
###### Abstract

Optimal coding over the additive white Gaussian noise channel under the peak energy constraint is studied when there is noisy feedback over an orthogonal additive white Gaussian noise channel. As shown by Pinsker, under the peak energy constraint, the best error exponent for communicating an -ary message, , with noise-free feedback is strictly larger than the one without feedback. This paper extends Pinsker’s result and shows that if the noise power in the feedback link is sufficiently small, the best error exponent for conmmunicating an -ary message can be strictly larger than the one without feedback. The proof involves two feedback coding schemes. One is motivated by a two-stage noisy feedback coding scheme of Burnashev and Yamamoto for binary symmetric channels, while the other is a linear noisy feedback coding scheme that extends Pinsker’s noise-free feedback coding scheme. When the feedback noise power is sufficiently small, the linear coding scheme outperforms the two-stage (nonlinear) coding scheme, and is asymptotically optimal as tends to zero. By contrast, when is relatively larger, the two-stage coding scheme performs better.

Noisy feedback, error exponent, peak energy constraint, Gaussian channel.

## I Introduction and Main Results

We consider a communication problem for an additive white Gaussian noise (AWGN) forward channel with feedback over an orthogonal additive white Gaussian noise backward channel as depicted in Fig. 1.

Suppose that the sender wishes to communicate a message over the (forward) additive white Gaussian noise channel

 Yi=Xi+Zi,

where , , and respectively denote the channel input, channel output, and additive Gaussian noise. The sender has a causal access to a noisy version of over the feedback (backward) additive white Gaussian noise channel

 ~Yi=Yi+~Zi,

where is the Gaussian noise in the backward link. We assume that the forward noise process and the backward noise process are independent of each other, and respectively white Gaussian and .

We define an code with the encoding functions , , and the decoding function . We assume a peak energy constraint

 P{n∑i=1x2i(w,~Yi−1)≤nP}=1for all w. (1)

The probability of error of the code is defined as

 P(n)e =P{W≠^W(Yn)} =1MM∑w=1P{W≠^W(Yn)|W=w},

where is distributed uniformly over and is independent of .

As is well known, the capacity of the channel (the supremum of such that there exists a sequence of codes with ) stays the same with or without feedback. Hence, our main focus is the reliability of communication, which is captured by the error exponent

 limn→∞−1nlnP(n)e

of the given code. The error exponent is sensitive to the presence of noise in the feedback link. Schalkwijk and Kailath showed in their celebrated work  that noise-free feedback can improve the error exponent dramatically under the expected energy constraint

 n∑i=1E[x2i(w,~Yi−1)]≤nPfor all% w, (2)

(in fact, decays much faster than exponentially in ). Kim, Lapidoth, and Weissman  studied the optimal error exponent under the expected energy constraint and noisy feedback, and showed that the error exponent is inversely proportional to for small .

Another important factor that affects the error exponent is the energy constraint on the channel inputs—the peak energy constraint in (1) vs. the expected energy constraint in (2). Wyner  showed that the error probability of the Schalkwijk–Kailath coding scheme  degrades to an exponential form under the peak energy constraint. In fact, Shepp, Wolf, Wyner, and Ziv  showed that for the binary-message case (), the best error exponent under the peak energy constraint is achieved by simple nonfeedback antipodal signaling, regardless of the presence of feedback. This negative result might lead to an impression that under the peak energy constraint, even noise-free feedback does not improve the reliability of communication. Pinsker  proved the contrary by showing that the best error exponent for sending an -ary message does not depend on and, hence can be strictly larger than the best error exponent without feedback for .

In this paper, we show that noisy feedback can improve the reliability of communication under the peak energy constraint, provided that the feedback noise power is sufficiently small. Let

 EM(α):=limsupn→∞−1nlnP∗e(M,n),

where denotes the best error probability over all codes for the AWGN channel with the noisy feedback. Thus, denotes the best error exponent for communicating an -ary message over the AWGN channel without feedback. Shannon  showed that

 EM(∞)=M4(M−1)P. (3)

This follows by first upper bounding the error exponent with the sphere packing bound and then achieving this upper bound by using a regular simplex code on the sphere of radius , that is, each codeword satisfies and is at the same Euclidean distance from every other codeword. In particular, for ,

 xn(1) =√nP⋅(0,1,0,…,0), xn(2) =√nP⋅(−1/2,√3/2,0,…,0), xn(3) =√nP⋅(−1/2,−√3/2,0,…,0),

and

 E3(∞)=38P.

At the other extreme, denotes the best error exponent for communicating an -ary message over the AWGN channel with noise-free feedback. Pinsker  showed that

 EM(0)≡P2

for all . In particular,

 E3(0)=P2.

Clearly, is decreasing in and

 EM(∞)≤EM(α)≤EM(0)

for every and .

Is strictly larger than (i.e., is noisy feedback better than no feedback)? Does tend to as (i.e., does the performance degrade gracefully with small noise in the feedback link)? What is the optimal feedback coding scheme that achieves ? To answer these questions, we establish the following results.

###### Theorem 1

For ,

 EM(α∗(s))≥P2(1−3(M−2)M(s2−2s+4)+3(M−2)),

where

 α∗(s)=3s24(s2−2s+4).

By comparing the lower bound with (3) and identifying the critical point , we obtain the following.

###### Corollary 1
 EM(α)>EM(∞)for α<14.

Thus, if the noise power in the feedback link is sufficiently small, then the noisy feedback improves the reliability of communication even under the peak energy constraint. The proof of Theorem  is motivated by recent results of Burnashev and Yamamoto in a series of papers , , where they considered a communication model with a forward BSC() and a backward BSC(), and showed that when is sufficiently small, the best error exponent is strictly larger than the one without feedback.

The lower bound in Theroem  shows that , which is strictly less than . To obtain a better asymptotic behavior for , we establish the following.

###### Theorem 2
 EM(α) ≥P211+α+4(⌊M/2⌋)2α+4(⌊M/2⌋)√α(1+α) ≥P21(√αM+√1+α)2.

This theorem leads to the following.

###### Corollary 2
 limα→0EM(α)=EM(0).

Thus, the lower bound in Theorem  is tight for . The proof of Theorem  extends Pinsker’s linear noise-free feedback coding scheme  to the noisy case.

Fig. 2 compares the two bounds for the case. The linear noisy feedback coding scheme performs better when is sufficiently small, while the two-stage noisy feedback coding scheme performs better when is relatively larger.

The rest of the paper is organized as follows. In Section II, we study a two-stage noisy feedback coding scheme motivated by recent results of Burnashev and Yamamoto and establish Thereom . In Section III, we extends Pinsker’s noise-free linear feedback coding scheme to the noisy feedback case and establish Theorem . Section IV concludes the paper.

## Ii Two-stage Noisy Feedback Coding Scheme

### Ii-a Background

It is instructive to first consider a two-stage noise-free feedback coding scheme for . This two-stage scheme has been studied by Schalkwijk and Barron  and Yamamoto and Itoh  for a general .

Encoding. Fix some . For simplicity of notation, assume throughout that is an integer. To send message , during the transmission time interval (namely, stage ), the encoder uses the simplex signaling:

 xλn(w)=⎧⎪⎨⎪⎩√λnP⋅(0,1,0,…,0) % for w=1,√λnP⋅(−1/2,√3/2,0,…,0) for w=2,√λnP⋅(−1/2,−√3/2,0,…,0) for w=3. (4)

Based on the feedback , the encoder then chooses the two most probable message estimates and , where

 p(^w1|yλn)≥p(^w2|yλn)≥p(^w3|yλn) (5)

and in case of a tie the one with the smaller index is chosen. Since the channel is Gaussian and is uniform, can be written as

 ||xλn(^w1)−yλn||≥||xλn(^w2)−yλn||≥||xλn(^w3)−yλn||,

where denotes the Euclidean distance. During the transmission time interval (stage ), the encoder uses antipodal signaling for if and transmits all-zero sequence otherwise:

 xnλn+1(w)=⎧⎪⎨⎪⎩√(1−λ)nP⋅(1,0,0,…,0) if w=min{^w1,^w2},√(1−λ)nP⋅(−1,0,0,…,0) if w=max{^w1,^w2},(0,0,…,0) otherwise.

Decoding. At the end of stage , the decoder chooses the two most probable message estimates and based on as the encoder does. At the end of stage , the decoder declares that is sent if

 ^w =argminw∈{^w1,^w2}||xn(w)−yn|| =argminw∈{^w1,^w2}(||xλn(w)−yλn||2+||xnλn+1(w)−ynλn+1||2)1/2.

Analysis of the probability of error. Let and denote the two most probable message estimates at the end of stage . The decoder makes an error if and only if one of the following events occurs:

 E1 ={W≠^W1 and W≠^W2}, E2 ={W∈{^W1,^W2} and ^W≠W}.

Thus, the probability of error is

 P(n)e=P(E1)+P(E2).

By symmetry, we assume without loss of generality that is sent. For brevity, we do not explicitly condition on the event in probability expressions in the following, whenever it is clear from the context. Refering to Fig. 3, let

 A23={yλn:||xλn(1)−yλn||≥||xλn(2)−yλn|| and ||xλn(1)−yλn||≥||xλn(3)−yλn||},

we have

 P(E1) =P{Yλn∈A23} ≤Q(d1) (a)≤12exp(−λnP2),

where follows since (see [11, Problem ]). Fig. 3: The error event E1 when W=1. Here d1=√λnP and 1, 2, and 3 denote xλn(1), xλn(2), and xλn(3), respectively.

On the other hand, is determined by the distance between the simplex signaling in stage and the distance between the antipodal signaling in stage (see Fig. 4). In particular,

 ||Xn(^W1)−Xn(^W2)||=√d22+d23=√(4−λ)nP.

Thus,

 P(E2) =Q(||Xn(^W1)−Xn(^W2)||2) =Q(√(1−λ4)nP) ≤12exp(−12(1−λ4)nP).

Therefore, the error exponent of the two-stage feedback coding scheme is lower bounded as

 E′3(0) =limsupn→∞−1nlnP(n)e =limsupn→∞−1nmax{lnP(E1),lnP(E2)} ≥min{λP2,P2(1−λ4)}.

Now let . Then it can be readily verified that both terms in the minimum are the same and we have

 E3(0)≥E′3(0)≥2P5.
###### Remark 1

Since , this two-stage noise-free feedback coding scheme is strictly suboptimal.

###### Remark 2

We need only three transmissions: two for stage and one for stage . Thus actually divides only the total energy , not the block length .

### Ii-B Two-stage Noisy Feedback Coding Scheme

Based on the two-stage noise-free feedback coding scheme in the previous subsection and a new idea of signal protection introduced by Burnashev and Yamamoto , , we present a two-stage noisy feedback coding scheme for . The coding scheme for an arbitrary is given in Appendix A.

In the two-stage noise-free feedback coding scheme, the encoder and decoder agree on the same set of message estimates and at the end of stage . When there is noise in the feedback link, however, this coordination is not always possible. To solve this problem, we assign a signal protection region , , to each signal as depicted in Fig. 5. Let and denote the transmitted and received signals, respectively, and denote the feedback sequence at the encoder. Let and the signal protection region for , , is defined as

 Bw={yλn: ||xλn(w)−yλn||≤||xλn(w′)−yλn|| for w′≠w and ∣∣||xλn(w′)−yλn||−||xλn(w′′)−yλn||∣∣≤td′ for w′,w′′≠w} (6)

which means that message is the most probable and the other messages and are of approximately equal posterior probabilities. Here is a fixed parameter which will be optimized later in the analysis.

Encoding. In stage , the encoder uses the same simplex signaling as in the noise-free feedback case (see (4)). Then based on the noisy feedback , the encoder chooses and such that

 ||xλn(^w1)−yλn||≥||xλn(^w2)−yλn||≥||xλn(^w3)−yλn||,

In stage , the encoder uses antipodal signaling for if and transmits all-zero sequence otherwise.

Decoding. The decoder makes a decision immediately at the end of stage if the received signal lies in one of the signal protection regions, i.e., for . Otherwise, it chooses the two most probable message estimates and and wait for the transmission in stage . At the end of stage , the decoder declares that is sent if

 ^w =argminw∈{^w1,^w2}||xn(w)−yn|| =argminw∈{^w1,^w2}(||xλn(w)−yλn||2+||xnλn+1(w)−ynλn+1||2)1/2. Fig. 5: Signal protection regions. The shaded areas Bw for w=1,2,3 are the signal protection regions for xλn(1), xλn(2), and xλn(3), respectively. Here d4=sd1/2=(s/2)√λnP for some parameter s=s(t)∈[0,1] to be optimized later.
###### Remark 3

The signal protection region corresponds to the case in which the two least probable messages are of approximately equal posterior probabilities, i.e., .

Analysis of the probability of error. Let (, ) and (, ) denote the pairs of the two most probable message estimates at the encoder and the decoder, respectively. As before, we assume that is sent. Refering to Fig. 5, let

 A′ww′=Aww′∖(∪w′′Bw′′),w,w′∈[1:3]

where .

The decoder makes an error only if one or more of the following events occur:

• decoding error at the end of stage

 E1={Yλn∈B2∪B3∪A′23},
• miscoordination due to the feedback noise

 ~E12 ={Yλn∈A′12,~Yλn∈A13∪A23}, ~E13 ={Yλn∈A′13,~Yλn∈A12∪A23},
• decoding error at the end of stage

 E2={W∈{^W1,^W2}={~W1,~W2} and ^W≠W}.

Thus, the probability of error is upper bounded as

 P(n)e ≤P(E1)+P(~E12)+P(~E13)+P(E2) =P(E1)+2P(~E12)+P(E2).

To simplify the analysis, we introduce a new parameter such that . It can be easily checked that corresponds to and that this constraint guarantees that (see Fig. 6(a)). Hence, for the first term

 P(E1) =P{Yλn∈A′23∪B2∪B3} ≤2Q(d5) (7) ≤exp(−λnP8(s2−2s+4)).

The second term can be upper bounded (see Fig. 6(b)) as

 P(~E12) =P{Yλn∈A′12,~Yλn∈A13∪A23} ≤P{~Yλn∈A13∪A23|Yλn∈A′12} ≤2Q(d6√α) (8) ≤exp(−3s2λnP32α). Fig. 6: (a) The error event E1 when W=1. Since 0≤s≤1, we have d5=√d21+d24−d1d4=√(λnP/4)(s2−2s+4). (b) The error event E2 when W=1 and {^W1,^W2}={1,3}. Here d6=(√3/2)d4=s√(3λnP/16).

Finally, the third term can be upper bounded in the exactly same manner as in the noise-free feedback case:

 P(E3)≤12exp(−12(1−λ4)nP).

Therefore, the error exponent of the two-stage noisy feedback coding scheme is lower bounded as

 E′3(α) =limsupn→∞−1nlnP(n)e ≥limsupn→∞−1nmax{lnP(E1),lnP(~E12),lnP(E2)}

Now let

 α=α∗(s)=3s24(s2−2s+4)

and

 λ=λ∗(s)=4s2−2s+5.

Then it can be readily verified that all the three terms in the minimum are the same and we have

 E′3(α∗(s))≥P2s2−2s+4s2−2s+5=:ϕ(s). (9)

Note that if ,

 ϕ(s)>38P=E3(∞),

and is monotonically increasing over . Thus

 E3(α)>E3(∞)for α<α∗(1)=14.

This completes the proof of Theorem for the case.

###### Remark 4

It can be easily checked that the lower bound in is tight and characterizes the exact error exponent of the two-stage noisy feedback coding scheme.

## Iii Linear Noisy Feedback Coding Scheme

### Iii-a Background

It is instructive to revisit (a slightly simplified version of) the linear noise-free feedback coding scheme by Pinsker , which shows that for all . This lower bound is tight since   and is nonincreasing in .

Encoding. To send message , the encoder transmits

 X1(w)=⎧⎨⎩L+1−wL√P if M=2L+1,L+1/2−wL√P if M=2L. (10)

Because of the feedback , the encoder can learn the noise . Subsequently it transmits

 Xi=(1+δ)Zi−1,i∈[2:η],

and afterwards, where will be optimized later and the random time is the largest such that

 k∑i=1X2i≤nP.

Decoding. Upon receiving , the decoder estimates by

 ^X1=¯n∑i=1(−1)i−1Yi(1+δ)i−1

and declares that is sent if

 ^w=argminw∈[1:M]|X1(w)−^X1|.
###### Remark 5

It can be easily checked that each time , the encoder transmits the error

 i−1∑j=1(−1)j−1Yj(1+δ)j−1−X1=(−1)i−2Zi−1(1+δ)i−2

in the decoder’s current estimate of the initial transmission (up to scaling). Thus, Pinsker’s coding scheme is another instance of iterative refinement used in the Schalkwijk-Kailth coding scheme  for the Gaussian channel and the Horstein coding scheme  for the binary symmetric channel.

Analysis of the probability of error. For simplicity of notation, assume throughout that is an integer. We use to denote a generic sequence of nonnegative numbers that tends to zero as . When there are multiple such functions , we denote them all by with the understanding that . It is easy to see that decoding error occurs only if . The probability of error is thus upper bounded as

 P(n)e=P{W≠^W}≤P{|X1−^X1|>√P2L}.

The key idea in the analysis is to introduce a “virtual” transmission

 X′i=⎧⎨⎩X1if i=1,(1+δ)Zi−1if i∈[2:¯n],0otherwise. (11)

Let

 Y′i=X′i+Zi (12)

and define the estimate of as

 ^X′1=¯n∑i=1(−1)i−1Y′i(1+δ)i−1. (13)

Then, it can be easily shown that

 ^X′1=X1+(−1)¯n−1Z¯n(1+δ)¯n−1.

Thus we have

 P{|X1−^X1|>√P2L} ≤P{|X1−^X′1|+|^X′1−^X1|>√P2L} ≤P{|X1−^X′1|>√P2L}+P{|^X′1−^X1|>0} =:P1+P2.

Now we upper bound the two terms. For the first term, we have

 P1 =P{∣∣∣Z¯n(1+δ)¯n−1∣∣∣>√P2L} =2Q(√P(1+δ)¯n−12L) ≤exp(−P(1+δ)2(¯n−1)8L2).

For the second term, note that for all if and only if , and thus that only if . Therefore,

 P2 ≤P{¯n∑i=1X2i>nP} (a)≤P{¯n∑i=2(1+δ)2Z2i−1>(n−1)P} =P{χ2¯n−1>(n−1)P(1+δ)2},

where follows since (recall ) and denotes a chi-square random variable with degrees of freedom. By upper bounding the tail probability of the chi-square random variable  as

 P{χ2k>x}≤exp(−x2+k2logexk) for any k≥1 and x≥k, (14)

we have

 P2 ≤P{χ2¯n−1>(n−1)P(1+δ)2} ≤exp(−12(n−1)P(1+δ)2+¯n−12loge(n−1)P(¯n−1)(1+δ)2) ≤exp(−12(n−1)P(1+δ)2+¯n−12loge(n−1)P(¯n−1)) ≤exp(−12nP(1+δ)2+nϵn),

where tends to zero as . Therefore, the error exponent of the linear feedback coding scheme is lower bounded as

 E′′M(0) ≥limsupn→∞−1nlnP(n)e =limsupn→∞−1nmax{lnP1,lnP2} ≥limsupn→∞min{P(1+δ)2(¯n−1)8nL2,P2(1+δ)2}.

for any . Now let

 δ=δ(n)=ln(4nL2)2¯n,

which tends to zero as . Then the limits of both terms in the minimum are the same. Therefore,

 E′′M(0)≥limsupn→∞P2(1+δ(n))2=P2,

which completes the proof of achievability.

### Iii-B Linear Noisy Feedback Coding Scheme

Now we formally describe and analyze a linear noisy feedback coding scheme based on Pinsker’s noise-free feedback coding scheme.

Encoding. Fix some . To send message , the encoder transmits

 X1(w)=⎧⎨⎩L+1−wL√λnP if M=2L+1,L+1/2−wL√λnP if M=2L. (15)

Because of the noisy feedback , the encoder can learn . Subsequently it transmits

 Xi=(1+δ)(Zi−1+~Zi−1),i∈[2:η],

where will be optimized later and the random time