Capacity Results for Multicasting Nested Message Sets over Combination Networks

Capacity Results for Multicasting Nested Message Sets over Combination Networks

Shirin Saeedi Bidokhti, Vinod M. Prabhakaran, Suhas N. Diggavi S. Saeedi Bidokhti was with the School of Computer and Communication Sciences, Ecole Polytechnique Fédéral de Lausanne. She is now with the Department of Electrical Engineering department for Communications Engineering, Technische Universität München (e-mail: shirin.saeedi@tum.de).V. Prabhakaran is with the School of Technology and Computer Science, Tata Institute of Fundamental Research (e-mail: vinodmp@tifr.res.in).S. Diggavi is with the Department of Electrical Engineering, UCLA, USA e-mail: (suhas@ee.ucla.edu). The work of S. Saeedi Bidokhti was partially supported by the SNSF fellowship no.146617. The work of Vinod M. Prabhakaran was partially supported by a Ramanujan Fellowship from the Department of Science and Technology, Government of India. The work of S. Diggavi was supported in part by NSF awards 1314937 and 1321120. The material in this paper was presented in parts at the 2009 IEEE Information Theory Workshop, Taormina, Italy, 2012 IEEE Information Theory Workshop, Lausanne, Switzerland, and the 2013 IEEE International Symposium on Information Theory, Istanbul, Turkey.
Abstract

The problem of multicasting two nested messages is studied over a class of networks known as combination networks. A source multicasts two messages, a common and a private message, to several receivers. A subset of the receivers (called the public receivers) only demand the common message and the rest of the receivers (called the private receivers) demand both the common and the private message. Three encoding schemes are discussed that employ linear superposition coding and their optimality is proved in special cases. The standard linear superposition scheme is shown to be optimal for networks with two public receivers and any number of private receivers. When the number of public receivers increases, this scheme stops being optimal. Two improvements are discussed: one using pre-encoding at the source, and one using a block Markov encoding scheme. The rate-regions that are achieved by the two schemes are characterized in terms of feasibility problems. Both inner-bounds are shown to be the capacity region for networks with three (or fewer) public and any number of private receivers. Although the inner bounds are not comparable in general, it is shown through an example that the region achieved by the block Markov encoding scheme may strictly include the region achieved by the pre-encoding/linear superposition scheme. Optimality results are founded on the general framework of Balister and Bollobás (2012) for sub-modularity of the entropy function. An equivalent graphical representation is introduced and a lemma is proved that might be of independent interest.

Motivated by the connections between combination networks and broadcast channels, a new block Markov encoding scheme is proposed for broadcast channels with two nested messages. The rate-region that is obtained includes the previously known rate-regions. It remains open whether this inclusion is strict.

Network Coding, Combination Networks, Broadcast Channels, Superposition Coding, Linear Coding, Block Markov Encoding

I Introduction

The problem of communicating common and individual messages over general networks has been unresolved over the past several decades, even in the two extreme cases of single-hop broadcast channels and multi-hop wireline networks. Special cases have been studied where the capacity region is fully characterized (see [1] and the references therein, and [2, 3, 4, 5]). Inner and outer bounds on the capacity region are derived in [6, 7, 8, 9, 10, 11, 12]. Moreover, new encoding and decoding techniques are developed such as superposition coding [13, 14], Marton’s coding [15, 16, 17], network coding [2, 18, 19], and joint unique and non-unique decoding [20, 21, 22, 23].

Surprisingly, the problem of broadcasting nested (degraded) message sets has been completely resolved for two users. The problem was introduced and investigated for two-user broadcast channels in [17] and it was shown that superposition coding was optimal. The problem has also been investigated for wired networks with two users [24, 25, 26] where a scheme consisting of routing and random network coding turns out to be rate-optimal. This might suggest that the nested structure of the messages makes the problem easier in nature. Unfortunately, however, the problem has remained open for networks with more than two receivers and only some special cases are understood [22, 27, 28, 29, 30]. The state of the art is not favourable for wired networks either. Although extensions of the joint routing and network coding scheme in [24] are optimal for special classes of three-receiver networks (e.g., in [31]), they are suboptimal in general depending on the structure of the network [32, Chapter 5], [33]. It was recently shown in [34] that the problem of multicasting two nested message sets over general wired networks is as hard as the general network coding problem.

In this paper, we study nested message set broadcasting over a class of wired networks known as combination networks [35]. These networks also form a resource-based model for broadcast channels and are a special class of linear deterministic broadcast channels that were studied in [29, 30]. Lying at the intersection of multi-hop wired networks and single-hop broadcast channels, combination networks are an interesting class of networks to build up intuition and understanding, and develop new encoding schemes applicable to both sets of problems.

We study the problem of multicasting two nested message sets (a common message and a private message) to multiple users. A subset of the users (public receivers) only demand the common message and the rest of the users (private receivers) demand both the common message and the private message. The term private does not imply any security criteria in this paper.

Combination networks turn out to be an interesting class of networks that allow us to discuss new encoding schemes and obtain new capacity results. We discuss three encoding schemes and prove their optimality in several cases (depending on the number of public receivers, irrespective of the number of private receivers). In particular, we propose a block Markov encoding scheme that outperforms schemes based on rate splitting and linear superposition coding. Our inner bounds are expressed in terms of feasibility problems and are easy to calculate. To illustrate the implications of our approach over broadcast channels, we generlize our results and propose a block Markov encoding scheme for broadcasting two nested message sets over general broadcast channels (with multiple public and private receivers). The rate-region that is obtained includes previously known rate-regions.

I-a Communication Setup

A combination network is a three-layer directed network with one source and multiple destinations. It consists of a source node in the first layer, intermediate nodes in the second layer and destination nodes in the third layer. The source is connected to all intermediate nodes, and each intermediate node is connected to a subset of the destinations. We refer to the outgoing edges of the source as the resources of the combination network, see Fig. 1. We assume that each edge in this network carries one symbol from a finite field . We express all rates in symbols per channel use ( bits per channel use) and thus all edges are of unit capacity.

The communication setup is shown in Fig. 1. A source multicasts a common message of bits and a private message of bits. and are independent. The common message is to be reliably decoded at all destinations, and the private message is to be reliably decoded at a subset of the destinations. We refer to those destinations who demand both messages as private receivers and to those who demand only the common message as public receivers. We denote the number of public receivers by and assume, without loss of generality, that they are indexed . The set of all public receivers is denoted by and the set of all private receivers is denoted by .

Encoder

Decoder

Decoder

Decoder

Fig. 1: A combination network with two public receivers indexed by and one private receiver indexed by . All edges are of unit capacity.

Encoding is done over blocks of length . The encoder encodes and into sequences , , that are sent over the resources of the combination network (over uses of the network). Based on the structure of the network, each user receives a vector of sequences, , that is a certain collection of sequences that were sent by the source. Given , public receiver , , decodes and given , private receiver , , decodes . A rate pair is said to be achievable if there is an encoding/decoding scheme that allows the error probability

approach zero (as ). We call the closure of all achievable rate-pairs the capacity region. Although we allow error probability in the communication scheme (for example in proving converse theorems), the achievable schemes that we propose are zero-error schemes. Therefore, in all cases where we characterize the capacity region, the error capacity region and the zero-error capacity region coincide. This is not surprising considering the deterministic nature of our channels/networks.

I-B Organization of the Paper

The paper is organized in eight sections. In Section II, we give an overview of the underlying challenges and our main ideas through several examples. Our notation is introduced in Section III. We study linear encoding schemes that are based on rate splitting, linear superposition coding, and pre-encoding in Section IV. Section V proposes a block Markov encoding scheme for multicasting nested message sets, Section VI discusses optimality results, and Section VII generalizes the block Markov encoding scheme of Section V to general broadcast channels. We conclude in Section VIII.

Ii Main Ideas at a Glance

The problem of multicasting messages to receivers which have (two) different demands over a shared medium (such as the combination network) is, in a sense, finding an optimal solution to a trade-off. On the one hand, public receivers (which presumably have access to fewer resources) need information about the common message only so that each can decode the common message. On the other hand, private receivers require complete information of both messages. It is, therefore, desirable from private receivers’ point of view to have these messages jointly encoded (especially when there are multiple private receivers). This may be in contrast with public receivers’ decodability requirement. This tension is best seen through an example. Example 1 below shows that an optimal encoding scheme should allow joint encoding of the common and private messages but in a restricted and controlled manner, so that only partial information about the private message is “revealed” to the public receivers and decodability of the common message is ensured.

Example 1.

Fig. 2: The source multicasts and (of rates and ) in channel use.

Consider the combination network shown in Fig. 2 in channel use. The source communicates a common message and a private message to four receivers. Receivers and are public receivers and receivers and are private receivers. Since Receivers and each have min-cuts less than , they are not able to decode both the common and private messages. For this reason, random linear network coding does not ensure decodability of at the public receivers. We note that to communicate , it is necessary that some partial information about the private message is revealed to public receiver .

Split the private message into of rate and of rate . is the part of that is not revealed to the public receivers and is the part that is revealed to Receiver (but not Receiver ). By splitting into independent pieces, we make sure that only a part of , in this case , is revealed to Receiver ; In other words, only is encoded into sequences ’s that are received by Receiver . A linear scheme based on this idea is illustrated in Fig. 2 and could be represented as follows:

(1)

Note that by this construction we have ensured that the private receivers get a full rank transformation of all information symbols, and the public receivers get a full rank transformation of a subset of the information symbols (including the information symbol of ). ∎

Our first coding scheme (Proposition 1 in Section IV-C) builds on Example 1 by splitting the private message into independent pieces (of rates , , to be optimized) and using linear superposition coding. The rate split parameters should be designed such that they satisfy several rank constraints (for different decoders to decode their messages of interest). We characterize the achievable rate-region by a linear program with no objective function (a feasibility problem). The solution of this linear program gives the optimal choice of for the scheme. We show that our scheme is optimal for combination networks with two public and any number of private receivers.

For networks with three or more public and any number of private receivers, the above scheme may perform sub-optimally. It turns out that one may, depending on the structure of resources, gain by introducing dependency among the partial (private) information that is revealed to different subsets of public receivers.

Example 2.

Fig. 3: Multicasting of rate and of rate . A multicast code such as , , ensures achievability of .

Consider the combination network of Fig. 3 where destinations , , are public receivers and destinations , , are private receivers. The source wants to communicate a common message of rate (i.e., ) and a private message of rate . Clearly this rate pair is achievable using the multicast code shown in Fig. 3 (or simply through a random linear network code). However, the scheme we outlined before is incapable of supporting this rate-pair. This may be seen by looking at the linear program characterization of the scheme: is achievable by our first encoding scheme if there is a solution to the following feasibility problem (see Section IV for details of derivation).

(2)
(3)
for all that is a permutation of : (4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)

By testing the feasibility of the above linear program, it can be seen that the above problem is infeasible (by Fourier-Motzkin elimination or using MATLAB). On a higher level, although the optimal scheme in Fig. 3 reveals partial (private) information to the different subsets of the public receivers, this is not done by splitting the private message into independent pieces and there is a certain dependency structure between the symbols that are revealed to receivers , , and . This is why our first linear superposition scheme does not support the rate-pair .

We use this observation to modify the encoding scheme and achieve the rate pair . First, pre-encode message , through a pre-encoding matrix , into a pseudo private message of larger “rate”:

(14)

Then, encode using rate splitting and linear superposition coding:

(15)

Suppose is a MDS (maximum distance separable) code. Each private receiver is able to decode two symbols out of the three symbols of and can thus decode . It turns out that the achievable rate-region of this scheme is a relaxation of (2)-(13) when may be negative. ∎

Our second approach (Theorem 1 in Section IV-D) builds on Example 2 and the underlying idea is to allow dependency among the pieces of information that are revealed to different sets of public receivers by an appropriate pre-encoder that encodes the private message into a pseudo private message of a larger rate, followed by a linear superposition encoding scheme. We prove that the rate-region achieved by our second scheme is tight for (or fewer) public receivers and any number of private receivers. To prove the converse, we first write an outer-bound on the rate-region which looks similar to the inner-bound feasibility problem and is in terms of some entropy functions. Next, we use sub-modularity of entropy to write a converse for every inequality of the inner bound. In this process, we develop a visual tool in the framework of [36] to deal with the sub-modularity of entropy and prove a lemma that allows us show the tightness of the inner bound without explicitly solving its corresponding feasibility problem.

Generalizing the pre/encoding scheme to networks with more than three public receivers is difficult because of the more involved dependency structure that might be needed, in a good code, among the partial (private) information pieces that are to be revealed to the subsets of public receivers. Therefore, we propose an alternative encoding scheme that captures these dependencies over sequential blocks, rather than the structure of the one-time (one-block) code. This is done by devising a simple block Markov encoding scheme. Below, we illustrate the main idea of the block Markov scheme by revisiting the combination network of Fig. 3.

Example 3.

Consider the combination network in Fig. 3 over which we want to achieve the rate pair . Our first code design using rate splitting and linear superposition coding (with no pre-encoding) was not capable of achieving this rate pair. Let us add one resource to this combination network and connect it to all the private receivers. This gives an extended combination network, as shown in Fig. 4, that differs from the original network only in one edge. This “virtual” resource is shown in Fig. 4 by a bold edge. One can verify that our basic linear superposition scheme achieves over this extended network by writing the corresponding linear program and finding a solution:

(16)
(17)

Let the message be a pseudo private message of larger rate () that is communicated over the extended combination network (in one channel use), and let , , , be the symbols that are sent over the extended combination network. One code design is given below. We will use this code to achieve rate pair over the original network.

(18)

Since the resource edge that carries is a virtual resource, we aim to emulate it through a block Markov encoding scheme. Using the code design of (18), all information symbols (, , ) are decodable at all private receivers. One way to emulate the bold virtual resource is to send its information (the symbol carried over it) in the next time slot using one of the information symbols , , that are to be communicated in the next time slot.

More precisely, consider communication over transmission blocks, and let be the message pair that is encoded in block . In the block, encoding is done as suggested by the code in (18). Nevertheless, to provide private receivers with the information of (as promised by the virtual resource), we use in the next block to convey . Since this symbol is ensured to be decoded at the private receivers, it indeed emulates the virtual resource. In the block, we simply encode and directly communicate it with the private receivers. Upon receiving all the blocks at the receivers, we perform backward decoding [37]. So in transmissions, we send symbols of and new symbols of over the original combination network; i.e., for , we achieve the rate-pair .

Fig. 4: The extended combination network of Example 3. A block Markov encoding scheme achieves over the original combination network. At time , information symbol contains the information of symbol .

Out third coding scheme (Theorem 2 in Section V) builds on Example 3. When there are or more public receivers, our block Markov scheme is more powerful that the first two schemes and we are not aware of any example where this scheme is sub-optimal. In Section V, we describe our block Markov encoding scheme and characterize the rate region it achieves. We show, for three (or fewer) public and any number of private receivers, that this rate-region is equal to the capacity region and, therefore, coincides with the rate-region of Theorem 1. Furthermore, we show through an example that the block Markov encoding scheme could outperform the previously discussed linear encoding schemes when there are or more public receivers.

In Section VII, we further adapt this scheme to general broadcast channels with two nested message sets and obtain a rate region that includes previously known rate-regions. We do not know if this inclusion is strict.

Iii notation

We denote the set of outgoing edges from the source by with cardiality , and we refer to those edges as the resources of the combination network. The resources are labeled according to the public receivers they are connected to; i.e., we denote the set of all resources that are connected to every public receiver in , , and not connected to any other public receiver by . Note that the edges in may or may not be connected to the private receivers. We identify the subset of edges in that are also connected to a private receiver by . Fig. 1 shows this notation over a combination network with four receivers. In this example, , , , , , , , , , , , .

Fig. 5: A combination network with two public receivers indexed by and two private receivers indexed by .

Throughout this paper, we denote random variables by capital letters (e.g., , ), the sets and by and , respectively, and subsets of by script capital letters (e.g., and ). We denote the set of all subsets of by and in addition denote its subsets by Greek capital letters (e.g., and ).

All rates are expressed in in this work. The symbol carried over a resource of the combination network, , is denoted by which is a scalar from . Similarly, its corresponding random variable is denoted by . We denote by , , the vector of symbols carried over resource edges in , and by , , , the vector of symbols carried over resource edges in . To simplify notation, we sometimes abbreviate the union sets , and , by , and , respectively, where is a subset of . The vector of all received symbols at receiver is denoted by , . When communication takes place over blocks of length , all vectors above take the superscript ; e.g., , , , , .

We define superset saturated subsets of as follows. A subset is superset saturated if inclusion of any set in implies the inclusion of all its supersets; e.g., over subsets of , is superset saturated, but not .

Definition 1 (Superset saturated subsets).

The subset is superset saturated if and only if it has the following property.

  • is an element of only if every , , is an element of .

For notational matters, we sometimes abbreviate a superset saturated subset by the few sets that are not implied by the other sets in . For example, is denoted by , and similarly is denoted by .

Iv Rate Splitting and Linear Encoding Schemes

Throughout this section, we confine ourselves to linear encoding at the source. For simplicity, we describe our encoding schemes for block length , and highlight cases where we need to code over longer blocks.

We assume rates and to be non-negative integer values111There is no loss of generality in this assumption. One can deal with rational values of and by coding over blocks of large enough length and working with integer rates and . Also, one can attain real valued rates through sequences of rational numbers that approach them.. Let and be variables in for messages and , respectively. We call them the information symbols of the common and the private message. Also, let vector be defined as the vector with coordinates in the standard basis . The symbol carried by each resource is a linear combination of the information symbols. After properly rearranging all vectors , , we have

where is the encoding matrix. At each public receiver , , the received signal is given by , where is a submatrix of corresponding to , . Similarly, the received signal at each private receiver , , is given by , where is the submatrix of the rows of corresponding to , .

We design to allow the public receivers decode and the private receivers decode . We then characterize the rate pairs achievable by our code design. The challenge in the optimal code design stems from the fact that destinations receive different subsets of the symbols that are sent by the encoder and they have two different decodability requirements. On the one hand, private receivers require their received signal to bring information about all information symbols of the common and the private message. On the other hand, public receivers might not be able to decode the common message if their received symbols depend on “too many” private message variables. We make this statement precise in Lemma 1. In the following, we find conditions for decodability of the messages.

Iv-a Decodability Lemmas

Lemma 1.

Let vector be given by (19), below, where , , , and .

(19)

Message is recoverable from if and only if and the column space of is disjoint from that of .

Corollary 1.

Message is recoverable from in equation (19) only if

We defer the proof of Lemma 1 to Appendix A and instead discuss the high-level implication of the result. Let where . Matrix can be written as , where is a full-rank matrix of dimension and is a full-rank matrix of dimension . is essentially just a set of linearly independent columns of spanning its column space. In other words, we can write

(20)
(21)

Since , are decodable if is full-rank.

Defining as a new of dimension and defining a null matrix in Lemma 1, we reach at the trivial result of the following corollary.

Corollary 2.

Messages are recoverable from in equation (19) if and only if

Since every receiver sees a different subset of the sent symbols, it becomes clear from Corollary 1 and 2 that an admissible linear code needs to satisfy many rank constraints on its different submatrices. In this section, our primary approach to the design of such codes is through zero-structured matrices, discussed next.

Iv-B Zero-structured matrices

Definition 2.

A zero-structured matrix is an matrix with entries either zero or indeterminate222Although zero-structured matrices are defined in Definition 2 with zero or indeterminate variables, we also refer to the assignments of such matrices as zero-structured matrices. (from a finite field ) in a specific structure, as follows. This matrix consists of blocks, where each block is indexed on rows and columns by the subsets of . Block , , is an matrix. Matrix is structured so that all entries in block are set to zero if , and remain indeterminate otherwise. Note that and .

Equation (22), below, demonstrates this definition for .

(22)

The idea behind using zero-structred encoding matrices is the following: the zeros are inserted in the encoding matrix such that the linear combinations that are formed for the public receivers do not depend on “too many” private information symbols (see Corollary 1).

In the rest of this subsection, we find conditions on zero-structured matrices so that they can be made full column rank.

Lemma 2.

There exists an assignment of the indeterminates in the zero-structured matrix (as specified in Definition 2) that makes it full column rank, provided that

(23)

For , (23) is given by

(24)
(25)
(26)
(27)
(28)
(29)

We briefly outline the proof of Lemma 2, because this line of argument is used later in Section IV-D. For simplicity of notation and clarity of the proof, we give details of the proof for . The same proof technique proves the general case.

Let be a zero-structured matrix given by equation (22). First, we reduce the problem of matrix being full column rank to an information flow problem over the equivalent unicast network of Fig. 6. Then we find conditions for feasibility of the equivalent unicast problem. The former is stated in Lemma 3 and the latter is formulated in Lemma 4, both to follow.

source

sink

Fig. 6: The source communicates a message of rate to the sink over a unicast network. The capacity of each edge is marked beside it. The bold edges are of infinite capacity. The network is tailored so that the mixing of the information which happens at the third layer mimics the same mixing of the information that is present in the rows of matrix in (22).

The equivalent unicast network of Fig. 6 is formed as follows. The network is a four-layer directed network with a source node in the first layer, four (in general ) nodes , , in the second layer, another four (in general ) nodes , , in the third layer, and finally a sink node in the fourth layer. The source wants to communicate a message of rate to the sink. We have (unit capacity) edges from the source to each node . Also, we have (unit capacity) edges from each node to the sink . The edges from the second layer to the third layer are of infinite capacity and they connect each node to all nodes where . The equivalent unicast network is tailored so that the mixing of the information which happens at each node (at the third layer) mimics the same mixing of the information that is present in the rows of matrix . This equivalence is discussed formally in Lemma 3 and its proof is deferred to Appendix B.

Lemma 3.

Given the zero-structured matrix in (22), the following two statements are equivalent.

  1. There exists an assignment of variables in that makes it full column rank.

  2. A message of rate could be sent over its equivalent unicast network in Fig. 6.

Using Lemma 3, could be made full-rank if is less than or equal to the min-cut between nodes and over the equivalent unicast network. The min-cut between and is given by Lemma 4 and the proof is delegated to Appendix C.

Lemma 4.

The min-cut separating nodes and over the network of Fig. 6 is given by

(30)

Lemma 3 and Lemma 4 give conditions for matrix to become full rank. The proof for is along the same lines.

Iv-C Zero-structured linear codes: an achievable rate-region

In our initial approach, we design the encoding matrix to be zero-structured. The idea is to have the resource symbols that are available to the public receivers not depend on “too many” private message variables. Equation (31) below shows such an encoding matrix for two public and any number of private receivers.

(31)

Here, the non-zero entries are all indeterminate and to be designed appropriately. Also, parameters , , and are non-negative structural parameters, and they satisfy

(32)

In effect, matrix splits message into four independent messages, , , , , of rates , , , , respectively. The zero structure of ensures that only messages , and are involved in the linear combinations that are received at public receiver and that only messages , and are involved in the linear combinations that are received at public receiver . In general, matrix splits message into independent messages of rates , , such that

(33)

Note that the zero-structure allows messages to be involved (only) in the linear combinations that are sent over resources in where . When referring to a zero-structured encoding matrix , we also specify the rate-split parameters , .

Remark 1.

As defined above, parameters , , are assumed to be integer-valued. Nonetheless, one can let these parameters be real and approximately attain them by encoding over blocks of large enough length.

Conditions under which all receivers can decode their messages of interest are as follow:

  • Public receiver : is the vector of all the symbols that are available to receiver . Using the zero-structure of in (31), we have

    (34)
    (35)

    Generally, is given by

    (36)

    where is a submatrix of corresponding to , . It has at most non-zero columns. We relate decodability of message at receiver to a particular submatrix of being full column rank. Let

    (37)

    Choose to be a largest submatrix of the columns of that could be made full column rank (over all possible assignments). Define

    (38)

    We have the following two lemmas.

    Lemma 5.

    Each public receiver can decode from (35) if , as defined above, is full column rank.

    Proof:

    Conditions for decodability of are given in Lemma 1 (and its following argument). By the manner and are defined, whenever is full column rank, not only is full column rank, but also columns of span all the column space of (for all possible assignments– otherwise a larger would have been chosen) and the span of the column space of is thus disjoint from that of . ∎

    Lemma 6.

    For each public receiver , there exists an assignment of (specific to ) such that is full column rank, provided that

    (39)
    Proof:

    The proof is deferred to Appendix D. ∎

  • Private receiver : is the vector of all the symbols that are carried by the resources in , . We have

    (40)