Slepian-Wolf Coding Over Cooperative Relay Networks

Slepian-Wolf Coding Over Cooperative Relay Networks

Abstract

This paper deals with the problem of multicasting a set of discrete memoryless correlated sources (DMCS) over a cooperative relay network. Necessary conditions with cut-set interpretation are presented. A Joint source-Wyner-Ziv encoding/sliding window decoding scheme is proposed, in which decoding at each receiver is done with respect to an ordered partition of other nodes. For each ordered partition a set of feasibility constraints is derived. Then, utilizing the sub-modular property of the entropy function and a novel geometrical approach, the results of different ordered partitions are consolidated, which lead to sufficient conditions for our problem. The proposed scheme achieves operational separation between source coding and channel coding. It is shown that sufficient conditions are indeed necessary conditions in two special cooperative networks, namely, Aref network and finite-field deterministic network. Also, in Gaussian cooperative networks, it is shown that reliable transmission of all DMCS whose Slepian-Wolf region intersects the cut-set bound region within a constant number of bits, is feasible. In particular, all results of the paper are specialized to obtain an achievable rate region for cooperative relay networks which includes relay networks and two-way relay networks.

1Introduction

Consider a group of sensors measuring a common phenomenon, like weather. In this paper, we investigate a communication scenario in which some sensors desire to obtain measurements of the other nodes with the help of some existing relay nodes in the network. In the language of information theory, we can consider measurements of sensors as outputs of discrete memoryless correlated sources and model the communication network as a cooperative relay network in which each node can simultaneously be a transmitter, a relay and a receiver. So the problem can be defined as below:

Given a set of sources observed at nodes respectively ( is the set of nodes in the network) and a set of receivers at nodes which is not necessarily disjoint from , what conditions must be satisfied to enable us to reliably multicast to all the nodes in over the cooperative relay network?

The problem of Slepian-Wolf (SW) coding over multi-user channels has been considered for some special networks. First in [1], Tuncel investigated the problem of multicasting a source over a broadcast channel with side information at the receivers. He proposed a joint source-channel coding scheme which achieves operational separation between source coding and channel coding in the sense that the source and channel variables are separated. He also proved the optimality of his scheme. In a recent work [2], this problem was generalized to the problem of lossy multicasting of a source over a broadcast channel with side information. In [3], a necessary and sufficient condition for multicasting a set of correlated sources over acyclic Aref networks [4] was derived. The problem of multicasting correlated sources over networks was also studied in the network coding literature [5].

Cooperative relay network has been widely studied in terms of achievable rate region for relay networks [7], multiple access relay channels [9] and multi-source, multi-relay and multi-destination networks [10]. In all the mentioned works, two main strategies of Cover and El Gamal for relay channels [11], namely, decode and forward (DF) and compress and forward (CF) were generalized for the cooperative relay networks. In a more general setting [12], Gündüz, et.al., consider a compound multiple access channel with a relay in which three transmitters where, one of them acts as a relay for the others, want to multicast their messages to the two receivers. Several Inner bounds to the capacity region of this network were derived using DF, CF and also structured lattice codes. Although finding the capacity of the simple relay channel is a longstanding open problem, an approximation for the Gaussian relay network with multicast demands has been recently found in [13]. In these works, the authors propose a scheme that uses the Wyner-Ziv coding at the relays and a distinguishability argument at the receivers.

In this paper, we first study the problem of multi-layer Slepian-Wolf coding of multi-component correlated sources, in which each source should encode its components according to a given hierarchy. Using the sub-modularity of the entropy function and a covering lemma, we prove an identity which states that for any points of SW-region with respect to joint encoding/decoding of the components, there exists a multi-layer SW-coding which achieves it. To the best of our knowledge, this identity is new and we call it the SW-identity. Then, we propose a joint Source-Wyner-Ziv encoding/sliding window decoding scheme for Slepian-Wolf coding over cooperative networks. In this scheme, each node compresses its channel observation using Wyner-Ziv coding and then jointly maps its source observation and compressed channel observation to a channel codeword. For decoding, each receiver uses sliding window decoding with respect to an ordered partition of other nodes. For each ordered partition, we obtain a set of DMCS which can reliably be multicast over the cooperative relay network. By utilizing the SW-identity, we obtain the union of the sets of all feasible DMCS with respect to all ordered partitions. Our scheme results in operational separation between the source and channel coding. In addition, this scheme does not depend on the graph of the network, so the result can easily be applied to any arbitrary network. We show that the sufficient conditions for our scheme, are indeed necessary conditions for the Slepian-Wolf coding over arbitrary Aref networks and linear finite-field cooperative relay networks. Moreover, we prove the feasibility of multicasting of all DMCS whose Slepian-Wolf region overlap the cut-set bound within a constant number of bits over a Gaussian cooperative relay network. This establishes a large set of DMCS that belongs to the set of DMCS which can reliably be multicast in the operational separation sense. Note that the model considered in this paper, encompasses the model of multiple access channel with correlated sources. So the set of feasible DMCS in the operational separation sense is a subset of all feasible DMCS. We extract an achievable rate region for cooperative relay networks by reducing sufficient conditions for reliable multicasting. We show that this achievable rate region subsumes some recent achievable rates based on the CF strategy [8]. In addition, we estimate the capacity region of Gaussian cooperative relay networks within a constant number of bits from the cut-set bound. Our result improves capacity approximation of Gaussian relay networks given in [15].

The rest of the paper is organized as follows. In Section 2, we introduce notations and definitions used in this paper. Section 3 derives necessary conditions for reliable multicasting of DMCS over cooperative networks. Section 4 studies the multi-layer Slepian-Wolf coding, in particular, a novel identity related to the entropy function is derived. In Section 5, we obtain feasibility constraints which are the main results of the paper. In sections Section 6 and Section 7, we derive necessary and sufficient conditions for multicasting of DMCS over some classes of semi-deterministic networks and Gaussian cooperative relay networks, respectively. Section 8 employs results of the previous sections to derive an inner bound and an outer bound for the capacity region of a cooperative relay networks. Section 9 concludes the paper.

2Preliminaries and Definitions

2.1Notation

We denote discrete random variables with capital letters, e.g., , , and their realizations with lower case letters , . A random variable takes values in a set . We use to denote the cardinality of a finite discrete set , and to denote the probability mass function (p.m.f.) of on , for brevity we may omit the subscript when it is obvious from the context. We denote vectors with boldface letters, e.g. , . The superscript identifies the number of samples to be included in a given vector, e.g., . We use to denote the set of -strongly typical sequences of length , with respect to p.m.f. on . Further, we use to denote the set of all -sequences such that are jointly typical, w.r.t. . We denote the vectors in the th block by a subscript . For a given set , we use the shortcuts and . We use to denote the set theoretic difference of and . We say that , if for each and sufficiently large , the relation holds.

2.2Sub-modular Function

Let be a finite set and be a power set of it, i.e., the collection of all subsets of . A function is called sub-modular, if for each ,

Function is called super-modular, if is sub-modular. Given two sets and a sub-modular function , we define .
Let be DMCS with distribution . For each , we define the entropy function as where denotes the entropy of random variable . It is well-known that the entropy function is a sub-modular function over the set [17]. The sub-modularity property of the entropy function plays an essential role in the remainder of the paper, (in contrast to the non-decreasing property of the entropy, i.e, ).

2.3Some Geometry

A polytope is a generalization of polygon to a higher dimension. Point, segment and polygon are polytopes of dimension , and , respectively. A polytope of dimension can be considered as a space bounded by a set of polytopes of dimension . The boundary polytope of dimension is called facet. For a given polytope , a collection of polytopes is called a closed covering of , if .

The proof is provided in the Appendix Section 10.

Lemma ? provides a powerful tool for dealing with the regions which are described with a set of inequalities.

It is easy to show that majorization has the following simple property:

The essential polytope of the sub-modular function over the set is a polytope of dimension , which has facets, each corresponding to intersection of hyperplane with for each non-empty subset . By , we denote the facet corresponding to the subset . Since is a super-modular function, one can easily show that is a non-empty polytope of dimension (see for example, [18]) .

The proof is provided in Appendix Section 11.

2.4System Model

A cooperative relay network is a discrete memoryless network with nodes , and a channel of the form

At each time , every node sends an input , and receives an output , which are related via .

According to Definition ?, the joint probability distribution of the random variables factors as,

3Cut-set type necessary conditions for reliable multicasting

In this section, we prove necessary conditions for reliable multicasting of correlated sources over cooperative network.

Using Fano’s inequality, imposing the condition as , it follows that:

with as . We also have . For each such that and , we have:

where follows from the fact that is a function of and the fact that , follows since conditioning reduces entropy, follows because form a Markov chain, is obtained by introducing a time-sharing random variable which is uniformly distributed over the set and is independent of everything else, follows by allowing with and defining and .

4Multi-Layer Slepian-Wolf Coding

Before describing our scheme and the related results, in this section, we deal with the problem of multi-layer Slepian-Wolf coding (ML-SW). Study of the ML-SW enables us to find a new tool to analyze the main problem. In the previous works (for example [19], [20]), ML-SW is used to describe a source with some small components (for example, by a binary representation of it) and then successively encoding these components with SW-coding instead of encoding the whole source at once. For example, if we describe an i.i.d. source by , i.e., , instead of encoding by bits/symbol, we can first describe by bits/symbol and then apply SW-coding to describe by bits/symbol, assuming that the receiver knows from decoding the previous layer information as a side information. Since the total bits required to describe in two layers is , it follows that there is no loss in the two-layer SW-coding compared with the jointly encoding of the source components. A natural question is: How can this result be generalized to a more general setting of multi-terminal SW-coding?

Figure 1: Two-Layer Slepian-Wolf coding for a pair of two-component correlated sources. This coding is suboptimal in the sense that it does not achieve the entire of Slepian-Wolf coding.
Figure 1: Two-Layer Slepian-Wolf coding for a pair of two-component correlated sources. This coding is suboptimal in the sense that it does not achieve the entire of Slepian-Wolf coding.

At first, let us look at the two-terminal SW-coding. Suppose two sources and are given. Joint SW-coding yields that lossless description of with rates is feasible, provided that . Now suppose the following simple ML-SW. Assume in the first layer, and are encoded by SW-coding with rates and in the next layer and are encoded by SW-coding with rates assuming that the receiver knows from decoding of the previous layer information (See Figure 1). The lossless description of in this manner is possible, if:

Figure 2: Slepian-Wolf rate region vs rate regions with two and three layers Slepian-Wolf coding. Segments AC correspond to the three-layer SW-coding, in which in the first layer, X_2 is encoded, then in the second layer (Y_2,X_1) is encoded assuming that X_2 is already decoded at the receiver and in the third layer Y_1 is encoded assuming that (X_2,Y_2,X_1) is already available at the receiver. Segment CD corresponds to two-layer SW-coding of the Fig. . Segment DB is obtained from a similar three layer SW-coding to that of segment AC. Notice that each corner point of any multi-layer SW-coding that lies inside the SW-region is coincident to a corner point of another multi-layer SW-coding.
Figure 2: Slepian-Wolf rate region vs rate regions with two and three layers Slepian-Wolf coding. Segments correspond to the three-layer SW-coding, in which in the first layer, is encoded, then in the second layer is encoded assuming that is already decoded at the receiver and in the third layer is encoded assuming that is already available at the receiver. Segment corresponds to two-layer SW-coding of the Fig. . Segment is obtained from a similar three layer SW-coding to that of segment . Notice that each corner point of any multi-layer SW-coding that lies inside the SW-region is coincident to a corner point of another multi-layer SW-coding.

This shows that this simple layering can not achieve all the points in the SW-region, in particular the corner points and can not be achieved by this scheme(See Figure 2). But the point can be achieved by successive SW-coding of , , and regarding that the previous sources are available at the receiver. This method suggests that instead of dividing the SW-coding in two layers, SW-coding can be performed in three layers: in the first layer is described for the receiver with rate , in the second layer are encoded by SW-coding in the presence of at the receiver, and finally in the last layer is described using SW-coding assuming are available to the receiver. Analyzing this strategy, yields that are achievable if,

From this strategy, the corner point is achieved, but the corner point is not achieved. In addition, as it can be seen in Figure 2, the other corner point of this scheme () is coincident with one of the corner points of the two-layer scheme. By symmetry, the corner point is achieved by a three-layer scheme in which , and are encoded in the first, second and third layer respectively. In addition, as it can be seen in Figure 2, the union of the regions of the three different layering schemes is a closed covering of the SW-region. Note that in all the three schemes, there is a hierarchy in the sense that the first component of each source (i.e., ) is encoded prior to the second component of it (i.e., ). The result of the two-terminal SW-coding suggests that to obtain the entire SW-region of multi-components DMCS, it suffices to consider all possible layering schemes such that a given hierarchy on each source is satisfied.

Consider a DMCS with two component sources, i.e., . Now we describe ML-SW with respect to a given ordered partition . In addition, we assume that the decoder has access to side information which is correlated with according to an arbitrary distribution .

  1. In the first layer, using SW-coding, is encoded with rates in which for , we set . The receiver can reliably decode provided that

    Define the function as

    Now using the sub-modularity of the entropy function, we have

    Hence is sub-modular. In addition, we have: . Note that , thus is equivalent to

    Now it follows from Definition ? that is contained in the SW-region of the first layer, iff it majorizes the essential polytope of , i.e., .

  2. In the layer , assuming that has been decoded at the receiver from the previous layers (where ), using SW-coding is encoded with rates in which for , we set . The receiver can reliably decode provided that,

    Define the function as follows:

    Now in similar manner to , it can be shown that is sub-modular. Following similar steps described in the previous stage, we conclude that is contained in the SW-region of the layer , iff it majorizes the essential polytope of , i.e., .

Define (which is the overall rate vector) and . We showed that . On the other side, suppose that the point majorizes , so there is a point such that . Applying Lemma ? to , we have . Hence there are points such that . Let where . Now we have and for all , . Thus, each rate vector satisfying can be achieved using ML-SW coding with respect to . Therefore the set of all achievable rates with respect to is given by:

The next theorem, is the main result of this section.

Define the function with . is a sub-modular function with the essential polytope . By definition, a point belongs to SW-region iff it majorizes . To prove the theorem, we must show that

Applying Equation to the RHS of yields,

Thus, to prove the theorem, we only need to show that is a closed covering of . We prove this by strong induction on . For as base of induction, it is clear (The case was proved separately in the beginning of the section). For assume that the theorem holds for any with size . We show that and satisfy the conditions of Lemma ?, thus is a closed covering of .

Proof of Claim ?

. First note that, (See equation )

where follows from the fact that with equality holds if and conditioning does not reduce the entropy, and follows by the chain rule, since is a partition of . Now we can conclude the claim from .

Proof of Claim ?

.By Lemma ?, is given by:

In which and are the associated essential polytopes of sub-modular functions and with domains and , respectively. More precisely, and are given by:

Now, since the size of and are smaller than , by applying the induction assumption to essential polytopes and (with side information at the decoder), we obtain:

where , and the functions and whose domain are and , are defined by:

Using and , we obtain and as:

Let be the concatenation of and . We assert that

By Lemma ?, belongs to , iff

To evaluate , consider

where we have used the fact that . Now, we compute the RHS of :