Wyner-Ziv Coding over Broadcast Channels: Digital Schemes
This paper addresses lossy transmission of a common source over a broadcast channel when there is correlated side information at the receivers, with emphasis on the quadratic Gaussian and binary Hamming cases. A digital scheme that combines ideas from the lossless version of the problem, i.e., Slepian-Wolf coding over broadcast channels, and dirty paper coding, is presented and analyzed. This scheme uses layered coding where the common layer information is intended for both receivers and the refinement information is destined only for one receiver. For the quadratic Gaussian case, a quantity characterizing the overall quality of each receiver is identified in terms of channel and side information parameters. It is shown that it is more advantageous to send the refinement information to the receiver with “better” overall quality. In the case where all receivers have the same overall quality, the presented scheme becomes optimal. Unlike its lossless counterpart, however, the problem eludes a complete characterization.
Consider a sensor network of nodes taking periodic measurements of a common phenomenon. We study the communication scenario in which one of the sensors is required to transmit its measurements to the other nodes over a broadcast channel. The receiver nodes are themselves equipped with side information unavailable to the sender, e.g., measurements correlated with the sender’s data. This scenario, which is depicted in Figure 1, can be of interest either by itself or as part of a larger scheme where all nodes are required to broadcast their measurements to all the other nodes. Finding the capacity of a broadcast channel is a longstanding open problem, and thus, limitations of using separate source and channel codes in this scenario may never be fully understood. In contrast, a very simple joint source-channel coding strategy is optimal for the special case of lossless coding . More specifically, it was shown in  that in Slepian-Wolf coding over broadcast channels (SWBC), as the lossless case was referred to, for a given source , side information , and a broadcast channel , lossless transmission (in the Shannon sense) is possible with channel uses per source symbol if and only if there exists a channel input distribution such that
for . In the optimal coding strategy, every typical source word is randomly mapped to a channel codeword , where and are so that . If (1) is satisfied, there exists a channel codebook such that with high probability, there is a unique index for which is jointly typical with the side information and is jointly typical with the channel output simultaneously, at any receiver . This result exhibits some striking features which are worth repeating here.
The optimal coding scheme is not separable in the classical sense, but consists of separate components that perform source and channel coding in a broader sense. This results in the separation of source and channel variables as in (1).
If the broadcast channel is such that the same input distribution achieves capacity for all individual channels, then (1) implies that one can utilize all channels at full capacity. Binary symmetric channels and Gaussian channels are the widely known examples of this phenomenon.
The optimal coding scheme does not explicitly involve binning, which is commonly used in network information theory. Instead, with the simple coding strategy of , each channel can be thought of as performing its own binning. More specifically, the channel output at each receiver can be viewed as corresponding to a virtual bin
1containing all source words that map to channel codewords jointly typical with . In general, the virtual bins can overlap and correct decoding is guaranteed by the size of the bins, which is about .
In this paper, we consider the general lossy coding problem in which the reconstruction of the source at the receivers need not be perfect. We shall refer to this problem setup as Wyner-Ziv coding over broadcast channels (WZBC). We present a coding scheme for this scenario and analyze its performance in the quadratic Gaussian and binary Hamming cases. This scheme uses ideas from SWBC  and dirty paper coding (DPC) [3, 6] as a starting point. The SWBC scheme is modified a) to allow quantization of the source, and b) to handle channel state information (CSI) at the encoder by using DPC. The modification with DPC is then employed in a layered transmission scheme with receivers, where there is common layer (CL) information destined for both receivers and refinement layer (RL) information meant for only one of the receivers. The channel codewords corresponding to the two layers are superposed and the resultant interference is mitigated using DPC. We shall briefly discuss other possible layered schemes obtained by varying the encoding and the decoding orders of the two layers and using successive coding or DPC to counteract the interference, although for the bandwidth matched Gaussian and binary Hamming cases, we observe that these variants perform worse.
DPC is used in this work in a manner quite different from the way it was used in , which concentrated on sending private information to each receiver in a broadcast channel setting, where the information that forms the CSI and the information that is dirty paper coded are meant for different receivers. Therefore, although the DPC auxiliary codewords are decoded at one of the receivers, unlike in our scheme, this is of no use to that receiver. For our problem, this difference leads to an additional interplay in the choice of channel random variables. The DPC techniques in this work are most similar to those in [16, 20], where, as in our scheme, the CSI carries information about the source and hence decoding the DPC auxiliary codeword helps improve the performance. However, our results indicate a unique feature of DPC in the framework of WZBC. In particular, in our layered scheme, the optimal Costa parameter for the quadratic Gaussian problem turns out to be either 0 or 1. When it is 0, there is effectively no DPC, and when it is 1, the auxiliary codeword is identical to the channel input corrupted by the CSI. To the best of our knowledge, although the latter choice is optimal for binary symmetric channels, it has never been shown to be optimal for a Gaussian channel in a scenario considered before.
When an appropriately defined “combined” channel and side information quality is constant at each receiver, the new scheme is shown to be optimal in the quadratic Gaussian case. We also derive conditions for the same phenomenon to occur in the binary Hamming case, although the expressions are not as elegant as in the quadratic Gaussian problem. Unlike in , however, the scheme that we derive is not always optimal. A simple alternative approach is to separate the source and channel coding. Both Gaussian and binary symmetric broadcast channels are degraded. Hence their capacity regions are known  and further, there is no loss of optimality in confining ourselves to two layer source coding schemes. The corresponding source and side information pairs are also degraded. Although a full characterization of the rate-distortion performance is available for the quadratic Gaussian case , only a partial characterization is available for the binary Hamming problem [15, 17]. In any case, we obtain an achievable distortion tradeoff of separate source and channel coding by combining the known rate-distortion results with the capacity results. For the quadratic Gaussian problem, we show that our scheme always performs at least as well as separate coding. The same phenomenon is numerically observed for the binary Hamming case.
For the two examples we consider, a second alternative is uncoded transmission if there is no bandwidth expansion or compression. This scheme is optimal in the absence of side information at the receivers in both the quadratic Gaussian and binary Hamming cases. However, in the presence of side information, the optimality may break down. We show that, depending on the quality of the side information, our scheme can indeed outperform uncoded transmission as well. In particular, if the combined quality criterion chooses the worse channel as the refinement receiver (because it has much better side information), then our layered scheme outperforms uncoded transmission for the quadratic Gaussian problem.
The paper is organized as follows. In Section 2, we formally define the problem and present relevant past work. Our main results are presented in Section 3 and Section 4, namely the extensions of the scheme in  that we develop for the lossy scenario. We then analyze a layered scheme in particular for the quadratic Gaussian and binary Hamming cases in Sections 5 and 6, respectively. For these cases, we compare the derived schemes with separate source and channel coding, and with uncoded transmission. Section 7 concludes the paper by summarizing the results and pointing to future work.
2 Background and Notation
Let be random variables denoting a source with independent and identically distributed (i.i.d.) realizations. Source is to be transmitted over a memoryless broadcast channel defined by . Decoder has access to side information in addition to the channel output . Let single-letter distortion measures be defined at each receiver, i.e.,
An code consists of an encoder
and decoders at each receiver
The rate of the code is channel uses per source symbol.
A distortion tuple is said to be achievable at a rational rate if for every , there exists such that for all integers with , there exists an code satisfying
where and denotes the channel output corresponding to .
In this paper, we present some general WZBC techniques and derive the corresponding achievable distortion regions. We study the performance of these techniques for the following cases.
Quadratic Gaussian: All source and channel variables are real-valued, and we use the notation to denote the variance of any Gaussian random variable . The source and side information are jointly Gaussian and the channels are additive white Gaussian, i.e., where is Gaussian and is independent of . There is an input power constraint on the channel:
where . Without loss of generality, we assume that and with and . Thus, , denotes the mean squared-error in estimating from , or equivalently, from since . Reconstruction quality is measured by squared-error distance: .
Binary Hamming: All source and channel alphabets are binary. The source is , where denotes the Bernoulli distribution with . The channels are binary symmetric with transition probabilities , i.e., where and and are independent with denoting modulo 2 addition (or the XOR operation). The side information sequences at the receivers are also noisy versions of the source corrupted by passage through virtual binary symmetric channels; that is, with and and are independent. Reconstruction quality is measured by Hamming distance: .
The problems considered in [9, 13, 19] can all be seen as special cases of the WZBC problem. However, the quadratic Gaussian and the binary Hamming cases with non-trivial side information have never, to our knowledge, been analyzed before. Nevertheless, separate source and channel coding and uncoded transmission are obvious strategies. We shall evaluate the performance of these alternative strategies and present numerical comparisons with our proposed scheme.
2.1 Wyner-Ziv Coding over Point-to-Point Channels
Before analyzing the WZBC problem in depth, we shall briefly discuss known results for Wyner-Ziv coding over a point-to-point channel, i.e., the case . Since , we shall drop the subscripts that relate to the receiver. The Wyner-Ziv rate-distortion performance is characterized in  as
where is an auxiliary random variable, and the capacity of the channel is well-known (cf. ) to be
It is then straightforward to conclude that combining separate source and channel codes yields the distortion
We are further interested in the evaluation of , as well as in the test channels achieving it, for the quadratic Gaussian and binary Hamming cases. We will use similar test channels in our WZBC schemes.
It was shown in  that the optimal backward test channel is given by
where and are independent Gaussians.
For the rate we have
The optimal reconstruction is a linear estimate , which yields the distortion
It was implicitly shown in  that the optimal auxiliary random variable is given by
where are all independent, and are Ber() and Ber() with and , respectively, and is an erasure operator, i.e.,
This choice results in
with denoting the binary convolution, i.e., , and denoting the binary entropy function, i.e.,
It is easy to show that when , is increasing in and decreasing in .
Since and Ber(), the corresponding optimal reconstruction function boils down to a maximum likelihood estimator given by
The resultant distortion is given by
implying together with (7) that
where the extra constraint is imposed because is a provably suboptimal choice. It also follows from the discussion in  that there exists a critical rate above which the optimal test channel assumes and , and below which it assumes and . The reason why we discussed other values of above is because we will use the test channel in its most general form in all WZBC schemes.
2.2 A Trivial Converse for the WZBC Problem
At each terminal, no WZBC scheme can achieve a distortion less than the minimum distortion achievable by ignoring the other terminals. Thus,
For the binary Hamming case, using (9) and , the converse becomes
2.3 Separate Source and Channel Coding
For a general source and channel pair, the source and channel coding problems are extremely challenging. The set of all achievable rate triples (common and two private rates) for general broadcast channels are not known. The corresponding source coding problem has not been explicitly considered in previous work either. But there is considerable simplification in the quadratic Gaussian and binary Hamming cases since the channel and the side information are degraded in both cases: we can assume that one of the two Markov chains, or , holds (for arbitrary channel input ) for the channel, and similarly either or holds for the source. The capacity region for degraded broadcast channels is fully known. In fact, since any information sent to the weaker channel can be decoded by the stronger channel, we can assume that no private information is sent to the weaker channel. As a result, two layer source coding, which has been considered in [15, 17, 18], is sufficiently general.
To be able to analyze and simultaneously, we denote the random variables, rates, and distortion levels associated with the ood channel by the subscript and those associated with the ad one by , i.e., the channel variables always satisfy where is either 1 or 2 and takes the other value. Let denote the capacity region for channel uses, i.e., the region of all pairs of total rates that can be simultaneously decoded by each receiver. As shown in [1, 5], is the convex closure of all such that there exist a channel input and an auxiliary random variable satisfying , the power constraint (if any) , and
Let be the set of total rates that must be sent to each source decoder to enable the receivers to reconstruct the source within the respective distortions and . A distortion pair is achievable by separate source and channel coding with channel uses per source symbol if and only if
Note that we use cumulative rates at the good receiver.
Despite the simplification brought by degraded side information, there is no known complete single-letter characterization of for all sources and distortion measures when . Let be defined as the convex closure of all such that there exist source auxiliary random variables with either or , and reconstruction functions satisfying
for , and
It was shown in  that when . On the other hand,  showed that even when , for the quadratic Gaussian problem. For all other sources and distortion measures, we only know in general when . We shall present explicit expressions for the complete tradeoff in the quadratic Gaussian case in Section 5 and an achievable tradeoff for the binary Hamming case in Section 6.
2.4 Uncoded Transmission
In the bandwidth-matched case, i.e., when , if the source and channel alphabets are compatible, uncoded transmission is a possible strategy. For the quadratic Gaussian case, the distortion achieved by uncoded transmission is given by
for . This, in turn, is also because the channel is the same as the test channel up to a scaling factor. More specifically, when is transmitted and corrupted by noise , one can write with , where is an appropriately scaled version of the received signal and
For the binary Hamming case, this strategy achieves the distortion pair
for . That is because the channel is the same as the test channel that achieves with . The distortion expression in (18) then follows using (8). One can also show that (18) coincides with only when or . Once again, these respectively correspond to trivial and zero .
3 Basic WZBC Schemes
In this section, we present the basic coding schemes that we shall then develop into the schemes that form the main contributions of this paper. In what follows, we only present code constructions for discrete sources and channels. The constructions can be extended to the continuous case in the usual manner. Our coding arguments rely heavily on the notion of typicality. Given a random variable defined over a discrete alphabet the typical set at block length is defined as 
where denotes the number of times appears in .
The first scheme, termed Common Description Scheme (CDS), is a basic extension of the scheme in  where the source is first quantized before transmission over the channel. Even though our layered schemes are constructed for the case of receivers, CDS can be utilized for any . Unlike in , where typical source words are placed in one-to-one correspondence with a channel codebook, the source words are first mapped to quantized versions and it is these quantized versions that are mapped to the channel codebook. Like , there is no explicit binning, but the channel performs virtual binning. Before discussing the performance of the CDS, we shall present an extension of the CDS for a more general coding problem.
Suppose that there is CSI available solely at the encoder, i.e., the broadcast channel is defined by the transition probability and the CSI with some , where is some fixed distribution defined on the CSI alphabet , is available non-causally at the encoder. Given a source and side information at the decoders , codes and achievability of distortion pairs is defined as in the WZBC scenario except that the encoder now takes the form . The following theorem characterizes the performance of an extension of the CDS, which we term CDS with DPC.
A distortion pair is achievable at rate if there exist random variables , and functions with and such that
The code construction is as follows. For fixed , a source codebook is chosen from . A set of bins , where each is chosen randomly at uniform from , is also constructed. Given a source word and CSI , the encoder tries to find a pair such that and . If it is unsuccessful, it declares an error. If it is successful, the channel input is drawn from the distribution . At terminal , the decoder goes through all pairs until it finds the first pair satisfying and simultaneously. If there is no such pair, the decoder sets . Once is decided, coordinate-wise reconstruction is performed using with and .
We define the error events as
Using standard typicality arguments, it can be shown that for fixed , if
then , and that and for any and large enough . Similarly, it follows that if
This probability also vanishes if thanks to (19). This completes the proof.
Note that, if is a trivial random variable, independent of the channel, the scenario becomes the original WZBC setup and CDS with DPC becomes CDS. By equating and , we obtain the following corollary that characterizes the performance of the CDS.
A distortion tuple is achievable at rate for the WZBC problem if there exist random variables , and functions with such that
The coding scheme in the proof of Theorem 1 can also decode successfully.
It then suffices to show that for large enough . Indeed, since ,
The assumption is not restrictive at all, because otherwise no information can be delivered to terminal to begin with.
The significance of Corollary 2 is that decoding provides information about the CSI . This information, in turn, will be very useful in our layered WZBC schemes where the CSI is self-imposed and related to the source itself.
Examining the proof of Theorem 1, we notice an apparent separation between source and channel coding in that the source and channel codebooks are independently chosen. Furthermore, successful transmission is possible as long as the source coding rate for each terminal is less than the corresponding channel coding rate for a common channel input. However, the decoding must be jointly performed and neither scheme can be split into separate stand-alone source and channel codes. Nevertheless, due to the quasi-independence of the source and channel codebooks we shall refer to source codes and channel codes separately when we discuss layered WZBC schemes. This quasi-separation was shown to be optimal for the SWBC problem and was termed operational separation in .
4 A Layered WZBC Scheme
In this section, we focus on the case of receivers. In CDS, the same information is conveyed to both receivers. However, since the side information and channel characteristics at the two receiving terminals can be very different, we might be able to improve the performance by layered coding, i.e., by not only transmitting a common layer (CL) to both receivers but also additionally transmitting a refinement layer (RL) to one of the two receivers. The resultant interference between the CL and RL can then be mitigated by successive decoding or by dirty paper encoding. Since there are two receivers, we are focusing on coding with only two layers because intuitively, more layers targeted for the same receiver can only degrade the performance.
Unless the better channel also has access to better side information, it is not straightforward to decide which receiver should receive only the CL and which should additionally receive the RL. We shall therefore refer to the decoders as the CL decoder and the RL decoder (which necessarily also decodes the CL) instead of using the subscripts and . For the quadratic Gaussian problem, we will later develop an analytical decision tool. For all other sources and channels, one can combine the distortion regions resulting from the two choices, namely, CL decoder and RL decoder and vice versa. For ease of exposition, for a given choice of CL and RL decoders, we also rename the source and channel random variables by replacing the subscripts 1 and 2 by (for random variables corresponding to the CL information or to the CL decoder) and (for random variables corresponding to the RL information or to the receiver that decodes both CL and RL).
As mentioned earlier, the inclusion of an RL codeword changes the effective channel observed while decoding the CL. It is on this modified channel that we send the CL using CDS or CDS with DPC, and the respective channel rate expressions in (21) and (19) must be modified in a manner that we describe in the following subsections where we also present the capacity of the effective channel for transmitting the RL. Each possible order of channel encoding and decoding (at the RL decoder) leads to a different scheme. We shall concentrate on the scheme that has the best performance among the four in the Gaussian and binary Hamming cases, deferring a discussion of the other three to Appendix .1. In this scheme, illustrated in Figure 2, the CL is coded using CDS with DPC with the RL codeword acting as CSI. We shall refer to this scheme as the Layered Description Scheme (LDS). We characterize the source and channel coding rates for LDS in the following. We will only sketch the proofs of the theorems, as they rely only on CDS with DPC, and other standard tools.
4.1 Source Coding Rates for LDS
The RL is transmitted by separate source and channel coding. In coding the source, we restrict our attention to systems where the communicated information satisfies where corresponds to the CL and is the RL. The source coding rate for the RL is therefore (cf. ). This has to be less than the RL capacity. Due to the separability of the source and channel variables in the required inequalities we can say that a distortion pair is achievable if
Here, is the “capacity” region achieved by either LDS or any of its variations discussed in Appendix .1, and is the set of all triplets so that there exist and reconstruction functions and satisfying and
The subscripts and are used to emphasize transmission of the CL to receivers and , respectively. Similarly, the subscript refers to transmission of RL to receiver .
4.2 Channel Coding Rates for LDS
The next theorem provides the effective channel rate region for LDS.
Let be the union of all for which there exist , , and with and such that
The various random variables that appear in Theorem 2 have the following interpretation: and are the channel outputs when the input is . and correspond to the partial channel codewords that are superposed to form the channel input. Finally is the auxiliary random variable used in DPC with forming the CSI.
In LDS, a trivial together with reduces to CDS.
We construct an RL codebook with elements from . We then use the CDS with DPC construction with the chosen RL codeword acting as CSI. It follows from Theorem 1 that the CL information can be successfully decoded (together with the auxiliary codeword ) at both receivers if (28) and (29) are satisfied. This way, the effective communication system for transmission of RL becomes a channel with as input and the pair and as output. For reliable transmission, (30) is then sufficient.
5 Performance Analysis for the Quadratic Gaussian Problem
In this section, we analyze the distortion tradeoff of the LDS for the quadratic Gaussian case. While CDS with DPC is developed only as a tool to be used in layered WZBC codes, CDS itself is a legitimate WZBC strategy. We thus analyze its performance in some detail first before proceeding with LDS. It turns out, somewhat surprisingly, that CDS may in fact be the optimal strategy for an infinite family of source and channel parameters. Understanding the performance of CDS also gives insight into which receiver should be chosen as receiver , and which one as receiver . We remind the reader that the variance of a Gaussian random variable will be denoted by .
5.1 CDS for the Quadratic Gaussian Problem
for . In other words,
For all that achieve the minimum in (31), we have
Thus, as seen from (11), . This, in particular, means that if
is a constant, CDS achieves the trivial converse and there is no need for a layered WZBC scheme. Specialization of (31) to the case is also of interest:
In particular, all maximizing achieve . Thus, the trivial converse is achieved if is a constant.
5.2 LDS for the Quadratic Gaussian Problem
For LDS, we begin by analyzing the channel coding performance and then the source coding performance in terms of achievable channel rates. Then closely examining the channel rate regions, we determine whether , or is more advantageous given , , , , , and . The resultant expression when exhibits an interesting phenomenon which we will make use of in deriving closed form expressions for the tradeoff in LDS.
Channel Coding Performance
For LDS, we choose channel variables and as independent zero-mean Gaussians with variances and , respectively, with , and use the superposition rule