Generalized Sphere Packing Bound

Generalized Sphere Packing Bound

Arman Fazeli,1 Alexander Vardy,1 and Eitan Yaakobi2 {afazelic,avardy}@ucsd.edu,  yaakobi@caltech.edu 1University of California San Diego, La Jolla, CA 92093, USA
2California Institute of Technology, Pasadena, CA 91125, USA
Abstract

Kulkarni and Kiyavash recently introduced a new method to establish upper bounds on the size of deletion-correcting codes. This method is based upon tools from hypergraph theory. The deletion channel is represented by a hypergraph whoseedges are the deletion balls (or spheres), so that a deletion-correcting code becomes a matching in this hypergraph. Consequently, a bound on the size of such a code can be obtained from bounds on the matching number of a hypergraph. Classical results in hypergraph theory are then invoked to compute an upper bound on the matching number as a solution to a linear-programming problem: the problem of finding fractional transversals.

The method by Kulkarni and Kiyavash can be applied not only for the deletion channel but also for other error channels. This paper studies this method in its most general setup. First, it is shown that if the error channel is regular and symmetric then the upper bound by this method coincides with the well-known sphere packing bound and thus is called here the generalized sphere packing bound. Even though this bound is explicitly given by a linear programming problem, finding its exact value may still be a challenging task. The art of finding the exact upper bound (or slightly weaker ones) is the assignment of weights to the hypergraph’s vertices in a way that they satisfy the constraints in the linear programming problem. In order to simplify the complexity of the linear programming, we present a technique based upon graph automorphisms that in many cases significantly reduces the number of variables and constraints in the problem. We then apply this method on specific examples of error channels. We start with the channel and show how to exactly find the generalized sphere packing bound for this setup. Next studied is the non-binary limited magnitude channel both for symmetric and asymmetric errors, where we focus on the single-error case. We follow up on the deletion channel, which was the original motivation of the work by Kulkarni and Kiyavash, and show how to improve upon their upper bounds for single-deletion-correcting codes. Since the deletion and grain-error channels resemble a very similar structure for a single error, we also improve upon the existing upper bounds on single-grain error-correcting codes. Finally, we apply this method for projective spaces and find its generalized sphere packing bound for the single-error case.

I Introduction

One of the basic and fundamental results in coding theory asserts that an upper bound on a length- binary code with minimum Hamming distance is

where . This is known as the classical sphere packing bound. This bound can be applied for other cases as well. Let be a finite set with some distance function . Assume that the volume of every ball is the same, that is, if then for all , for some fixed value . Then, the resulting sphere packing bound on a code with minimum distance becomes . However, what happens if the size of all balls is not the same? Clearly, a naive solution is to use as the minimum size of all balls and then to apply the same bound, but this approach can give a very weak upper bound. The goal of this paper is to study a generalization of the sphere packing bound for setups where the size of all balls is not necessarily the same.

The lower counter bound for the sphere packing one is the well-known Gilbert-Varshamov bound [10, 22]. This bound states that if the size of all balls of radius is the same, , then a lower bound on a code with minimum distance becomes . In [21], a similar study was carried for the Gilbert-Varshamov bound in case that the size of all balls is not necessarily the same. Using Turán’s theorem, it was shown that the same derivation on a lower bound of a code still holds, with the modification of using the average size of the balls. That is, if , then a generalized Gilbert-Varshamov bound asserts that there exists a code with minimum distance and of size at least . Thus, an immediate question to ask is whether the same analogy holds for the sphere packing bound: Is an upper bound on a code with minimum distance ? Even though in most of the cases we study in this work this derivation does hold, the answer in general to this question is negative. However, it is interesting to find some conditions under which this bound will always be satisfied.

The deletion channel [18] is one of the examples where the balls can have different sizes. Recently, in [15], Kulkarni and Kiyavash showed a technique, based upon tools from hypergraph theory [2], in order to derive explicit non-asymptotic upper bounds on the cardinalities of deletion-correcting codes. These upper bounds were given both for binary and non-binary codes as well as for deletion-correcting codes for constrained sources. Since the method in [15] can be applied for other similar setups, more results were presented shortly after for different channel models. Upper bounds on the cardinalities of grain-error-correcting codes were given in [8] and [12] and similar bounds for multipermutations codes with the Kendall’s distance were derived in [4].

This paper has two main goals. First, we extend the method studied for the deletion channel by Kulkarni and Kiyavash [15] and analyze it in its most general setting. We assume that the error channel is characterized by a directed graph, which depicts for a given transmitted word, its set of possible received words. Then, an upper bound will be given on codes which can correct errors, for some fixed . This bound is established by the solution of a linear programming given from a hypergraph that is derived from the error channel graph. In particular, it is shown that the sphere packing bound is a special case of this bound. We also study properties of this bound and show a scheme, based upon graph automorphisms, that in many cases can significantly reduce the complexity of the linear programming problem. In the second part of this work, we provide specific examples on the application of this method to setups where the balls have different sizes. These examples include the channel, non-binary channels with limited magnitude errors (symmetric and asymmetric), deletion channel, grain-error channel, and finally, projective spaces. In some of these examples we improve upon the existing results which use this method to calculate the upper bound on the code cardinalities. When possible in these examples, we compare the bounds we receive with the state-of-the-art ones.

In order to describe our results, we need to introduce some notation. Let be a hypergraph, where is its vertices set and is its hyperedges set. Let be the incidence matrix of , so if . A transversal in is a subset that intersects every hyperedge in . The transversal number of , denoted by , is the size of the smallest transversal. Every transversal can be represented by a binary vector which needs to satisfy . However, if the vector can have values over and still satisfies the last inequality, then it is called a fractional transversal. Under this setup, it is known that , where is the linear programming relaxation of , defined as

(1)

Let be a directed graph which describes an error channel. The vertices set is the set of all possible transmitted words, and the edges set consists of all pairs of vertices of distance one. The distance between , is the path metric in and is denoted by . Note that since the graph is directed, it is possible to have . For every , its radius- ball is the set which was defined above and its degree is . The largest cardinality of a length- code in with minimum distance is denoted by . Given some positive integer , the graph is associated with a hypergraph where and . Observing that every code of minimum distance is a matching in (which is a collection of pairwise disjoint edges), the following upper bound on was verified in [15],

(2)

One of the first properties we present asserts that if the graph is regular such that for all , and the distance function is symmetric, then the bound coincides with the sphere packing bound, that is, . Therefore, in this work the bound is called the generalized sphere packing bound.

The expression provides an explicit upper bound on . However, it may still be a hard problem to calculate this value since it requires the solution of a linear programming problem that can have an exponential number of variables and constraints. Clearly, one would inspire to find this exact value, but if this is not possible to accomplish, it is still valuable to give an upper bound on , which, in essence, is an upper bound on as well. Such an upper bound will be given by finding any fractional transversal and the goal will be to find one with small weight. In fact, all the upper bound results presented in [4, 8, 12, 15] follow this approach and an upper bound on the value in each case is given.

The rest of the paper is organized as follows. Section II establishes the rest of the definitions and tools required in this paper and demonstrates them on the channel. This channel will be used throughout the paper as a running example and a case study we rigorously investigate. In Section III, we start with basic properties on the generalized sphere packing bound. In particular, we show upper and lower bounds on its value and prove that if the graph is regular and symmetric then the sphere packing bound coincides with the generalized sphere packing bound. We also show several examples which establish a dissenting answer to the question brought earlier about the upper bound validity of an average sphere packing value. We then proceed to define a special monotonicity property on the graph which states that a graph is monotone if for all and two vertices and , if then . This property is useful in order to give a general formula for a fractional transversal and a corresponding upper bound. In fact, this property and fractional transversal were used in the previous works [8, 12, 15]. Lastly in this section, we use tools from automorphisms on graphs in order to simplify the complexity of the linear programming problem in (1). Noticing that in many channels there are groups of vertices with similar behavior motivates us to treat them as the same vertex and thus significantly reduce the number of variables and constraints in the linear programming (1). In Section IV, we study the channel. Our main contribution here is finding a method to calculate the generalized sphere packing bound for all radii. In Section V we carry a similar task for the limited-magnitude channel with symmetric and asymmetric errors. We focus only the single error case of radius one in both cases and find fractional transversals and corresponding upper bounds. Section VI follows upon the original work of [15], improving the bounds derived therein for the deletion channel (for the case of a single deletion). Since the structure of the deletion and grain-error channel is very similar, especially for a single error, we continue with the same approach to improve upon the existing upper bounds from [8, 12] on the cardinalities of single-grain error-correcting codes. Section VII studies bounds on projective spaces and in particular we give an optimal solution for the radius-one case under this channel. Finally, Section VIII concludes the paper and proposes some problems which remained open.

Ii Definitions and Preliminaries

In this section we formally define the tools and definitions used throughout the paper. We mainly follow the same definitions and properties from [15].

Let be a hypergraph where , and its incidence matrix. A matching in is a collection of pairwise disjoint hyperedges and the matching number of , denoted by , is the size of the largest matching. The matching number of , , is the solution of the integer linear programming problem

Note that the transversal number , defined in the previous section, is the solution of the integer linear programming problem

These two problems satisfy weak duality and thus . Furthermore, they can be slightly modified such that the vectors in the minimization and maximization problems can have values in , and still they give the values of and , that is,

The relaxation of these integer linear programmings allows the variables and to take values in , which are not necessarily integers. The value of this linear programming relaxation for the matching number is denoted by

and the corresponding one for the transversal number is the value , stated in (1). Note that the real solutions can be significantly different than the integer solutions and since and satisfy strong duality, the following property holds [15]

and in particular, for any fractional transversal ,

Lastly, we mention here that we will usually denote the fractional transversal by , such that corresponds to the value that is assigned to the vertex . However, when it will be clear from the context, the notation will be used to refer to the value of , where .

Every error channel studied in this work will be depicted by some directed graph , where the set defines the set of all pairs of vertices of distance one from each other. The distance between every two vertices , denoted by , is the length of the shortest path from to in the graph , and if such a path does not exist. Note that this definition of distance is not necessarily symmetric and thus it may happen that . However if for all , , then we say that is symmetric, and otherwise it is not symmetric. For any , we let be the sets and . The out-degree of is and the in-degree is . The definition of and coincide with the ones in the Introduction for and , respectively. To ease the notation in the paper we will follow the ones from the Introduction for the “out” case and use the ones defined above for the “in” case.

If a word is transmitted and at most errors occurred then any word in can be received. A code in this graph is said to have minimum distance if for all , . We let be the largest cardinality of a code in of length and minimum distance . If for every , there exists some fixed such that for every , , then we say that the graph is regular and otherwise it is called non-regular.

For any positive integer , is a hypergraph associated with such that and . As was stated in (2), the value is an upper bound on and is called in this work the generalized sphere packing bound.

The average size of a ball of radius in is defined to be

In [21], using Turán’s theorem a generalized Gilbert-Varshamov bound was shown to hold also for the cases where the size of all balls is not the same. This bound asserts that a lower bound on is given by

Let us remind the question we brought in the Introduction about the analogy of the last bound to the sphere packing bound. Namely, does the following inequality hold

We call the value the average sphere packing value and denote it by . We do not call this value a bound since, as we shall see later, it is not necessarily a valid upper bound.

The following example demonstrates the definitions and concepts introduced in this section for the channel.

Example 1

. The channel is a channel with binary inputs and outputs where the errors are asymmetric. Here, we assume that errors can only change a 1 to 0 with some probability , but not vice versa; see Fig 1.

Fig. 1: The -channel.

The corresponding graph is , where and

and denotes the Hamming weight of . Let be some fixed positive integer. For every ,

and .

The corresponding hypergraph is , such that and . The generalized sphere packing bound becomes

(3)

The average size of a ball with radius is

For , and thus we get

Therefore, the average sphere packing value in this case becomes

In particular, for we get

In the sequel it will be verified that the average sphere packing value for is a valid upper bound for the channel.

Even though the generalized sphere packing bound gives an explicit upper bound on the cardinality of error-correcting codes, it is not necessarily immediate to calculate it. To accomplish this task, one needs to solve a linear programming which, in general, does not necessarily have an efficient solution. Furthermore, note that in many of the communication channels the number of variables and constraints can be very large and in particular exponential with the length of the words. Our main discussion in this paper will be dedicated towards approaches for deriving the value for different graphs . However, in cases where it will not be possible to derive this explicit value, we note that every fractional transversal provides a valid upper bound and thus we inspire to give the best fractional transversal we can find.

Iii General Results and Observations

In this section we start by proving basic properties on the value of the generalized sphere packing bound as specified in (1). We then show some approaches for finding fractional transversals. Finally, we present a scheme, based upon automorphisms on graphs, that in many cases can significantly reduce the complexity of the linear programming problem for calculating the value . As specified in Section II, we assume throughout this section that the error channel is depicted by some directed graph and for a fixed integer , is its associated hypergraph.

Iii-a Basic Properties of the Generalized Sphere Packing Bound

We start here by proving some basic properties and giving insights on the value of . The next lemma proves a lower bound on the generalized sphere packing bound in case that its in-degree is upper bounded.

Lemma 1

. If for all , , then

Proof:

Since , for all , the weight of every column of the incidence matrix of is at most , that is, for all . Let be a fractional transversal in . Then, for every , , and thus

However, note that

and therefore

Hence, we conclude that . ∎

Next, we show an upper bound on the generalized sphere packing bound in case that its out-degree is lower bounded.

Lemma 2

. If for all , , then

Proof:

If for all then the vector is a fractional transversal and thus . ∎

According to the last two lemmas we can show that if the graph is regular and symmetric then the generalized sphere packing bound coincides with the sphere packing bound.

Corollary 3

. If the graph is symmetric and regular then the generalized sphere packing bound and the sphere packing bound coincide. Furthermore, , where for all , .

Proof:

Since is regular then for all , and according to Lemma 2, we have . Since is also symmetric we have that for all , and according to Lemma 1, we get . Therefore, . ∎

The next example proves that the requirement on the graph to be symmetric is necessary in order to have equality between the sphere packing and the generalized sphere packing bound.

Example 2

. In this example the graph has six vertices, so . For , there is an edge from to and finally there is an edge from to ; see Fig. 2.

Fig. 2: The graph .

Therefore, for all , so the graph is regular and the sphere packing bound becomes . However, the vector is a fractional transversal, which is optimal, and thus the generalized sphere packing bound of equals 1.

In the next example, we show a graph that does not obey to the average sphere packing value. This provides a negative answer to the earlier question we asked in the Introduction regarding the validity of the average sphere packing value as a valid bound.

Example 3

. The graph in this example has five vertices, so . There is an edge from the first vertex to all other four vertices; see Fig. 3.

Fig. 3: The graph .

The average size of a ball is and thus the average sphere packing value becomes . However, the minimum distance of the code in is , and in particular, it can be a code with minimum distance , which contradicts the average sphere packing value.

Example 3 depicts a directed, i.e. not symmetric, graph where the average sphere packing value does not hold. Next we show an example of a symmetric graph that does not satisfy the average sphere packing value either.

Example 4

. Assume there are vertices partitioned into two groups: the first one consists of vertices and the other group of the remaining vertices. Every vertex from the first group is connected (symmetrically) to a set of exactly vertices from the second group such that there is no overlap between these sets. The vertices in the second group are all connected to each other. Thus, the average radius-one ball size is

Therefore, the average sphere packing value is less than 2. However, it is possible to construct a single-error correcting code with the vertices of the first group.

Examples 3 and 4 prove that the average sphere packing value does not hold in all cases. In fact, from Example 4, we do not only conclude that it does not hold in general, but also that the ratio between this value and a size of a code can be arbitrarily small. However, it is still very interesting to find some minimal conditions such that this bound holds.

Iii-B Monotonicity and Fractional Transversals

Remember that a vector is a fractional transversal if and for ,

A first example for choosing a fractional transversal is stated in the next lemma.

Lemma 4

. The vector given by

for , is a fractional transversal.

Proof:

It is easy to verify that . For every , if , then and thus

Therefore, we get

A graph is said to satisfy the monotonicity property, or is monotone, if for every , and ,

In this case, the fractional transversal from Lemma 4 can be stated more explicitly.

Lemma 5

. If is monotone then the vector given by

for , is a fractional transversal.

Proof:

If is monotone then for every , . Therefore, the fractional transversal from Lemma 4 simply becomes

As a result of Lemma 5, if is monotone, then the following expression is an upper bound on ,

(4)

We call this bound the monotonicity upper bound, which holds in case that is monotone, and denote it by . We will build upon Example 1 to exemplify the monotonicity upper bound for the channel.

Example 5

. It is straightforward to verify that the graph from Example 1 satisfies the monotonicity property since for every , if then . Thus, according to Lemma 5, the vector given by

is a fractional transversal. Therefore, the monotonicity upper bound derived in (4) is calculated to be

For example, for , we get

Note that the average sphere packing value, calculated in Example 1, for is , is stronger than the monotonicity upper bound. In fact, this hints that in some cases, which will be studied in the sequel, it is possible to improve upon the monotonicity upper bound. Indeed, it is possible to verify that in this case the fractional transversal according to Lemma 5 is not optimal by showing that the vector , where

for and , is a fractional transversal. The corresponding bound for this fractional transversal becomes

which verifies the validity of the average sphere packing value. However, this choice of fractional transversal is still suboptimal and hence we seek to find a further improvement. Finding the exact value will be the topic and problem we solve in Section IV.

The deletion channel which was studied in [15], the overlapping grain-error model studied in [8] and the non-overlapping grain error-error model for studied in [12] all satisfy the monotonicity property. Indeed, all these works applied the monotonicity upper bound in order to derive upper bounds on the cardinalities of error-correcting codes in every channel. However, as will be shown in this work, the choice of the fractional transversal according to Lemma 5 is not necessarily optimal. This will be verified by providing different fractional transversals which yield stronger upper bounds than the ones achieved by the monotonicity upper bound.

Iii-C Automorphisms on Graphs

One of the main obstacles in calculating the value of is the large number of variables and constraints in the linear programming in (1). However, most of the graphs studied in this work contain symmetries between their vertices. For example, the linear programming in Example 1 for the channel has variables and constraints in order to find the value of , but it is not hard to notice that vectors of the same weight have identical behavior, and thus, one would expect to assign the same weight to these vertices. This will reduce the number of variables and constraints from to , which significantly simplifies the linear programming problem in (3). This subsection presents a scheme, based upon graph automorphisms, that in many cases can be used in order to significantly reduce the number of variables and constraints to calculate the bound . We will show the general scheme along with a demonstration how it is applied on our continued example of the channel.

Let us first remind some tools derived from properties on automorphisms of graphs. Let be a directed graph with vertices. An automorphism of is a permutation of its vertices that preserves adjacency. That is, an automorphism of is a permutation such that for all , if and only if . Assume , we let be the set of all permutations of elements. The set of all automorphisms of is

It is known that is a subgroup of the symmetric group under the operation of functions composition.

The group induces a relation on such that if and only if there exists where . It is possible to verify that is an equivalence order and hence is partitioned into equivalence classes, denoted by . Furthermore, we denote .

For any , let us define the set

Given a partition of , we say that a fractional transversal is -regular if for all and every , .

Given a fractional transversal and an automorphism , the vector is defined by . The next lemma proves that the vector is a fractional transversal as well.

Lemma 6

. Let be a fractional transversal and an automorphism. Then, the vector is a fractional transversal as well.

Proof:

It is clear to verify that . We need to show that for all the following inequality holds

Since is an automorphism, if and only if and therefore

where the last inequality holds since is a fractional transversal. ∎

Our main result in this part is stated in the next theorem and corollary.

Theorem 7

. For every , if then contains an -regular fractional transversal.

Proof:

Let be a fractional transversal. If is -regular then the property holds. Otherwise, let and as defined above. Note that

and together with Lemma 6 we get that . Similarly, we can show that . Let be some order of the automorphisms in . We can similarly derive that the vector

belongs to as well.

We finally show that is -regular. For all and

Now, let be such that and note that

Thus, we get

Lastly, we note that Theorem 7 holds not only for the automorphism group but also for every subgroup of . Given a subgroup of , assume it partitions the vertices set into equivalence classes . Let be an adjacency matrix corresponding to the subgroup , such that for ,

(5)

The next Corollary summarizes this discussion.

Corollary 8

. Let be a subgroup of and is its partition of into equivalence classes. Then, the generalized sphere packing bound from (1) becomes

(6)
Proof:

According to Theorem 7, it is enough to consider only fractional transversals which are -regular. Such a fractional transversal can be represented by a vector such that for , is the weight given to all the vectors in the set .

The condition from (1) can be stated as for all , . However, for all the number of vertices which belong to some set is fixed and is given by the value . Therefore, for every , this condition can be written as . Finally, since there are vectors which are assigned with weight we get that the weight of this -regular fractional transversal is and thus the corollary holds. ∎

The next example shows how to apply the automorphisms scheme presented in this subsection for the channel.

Example 6

. In Example 1, we saw that in order to find the value according to (3), it is required to solve a linear programming with variables and constraints. Let us demonstrate how the automorphism scheme studied in this subsection can reduce both the number of variables and constraints to be .

First, we define the following set of automorphisms on . For every , a permutation is defined such that for all , . It is possible to verify that the set is a subgroup of . Furthermore, the set is partitioned under into equivalence classes , where , for . Therefore, according to equation (6) in Corollary 8, it is enough to limit our search and find only fractional transversals which are -regular. Hence, the problem in (3) is simplified to be

(7)

In the next section we will continue Example 3 and show exactly how to solve the problem in (7).

Iv The Channel

The channel was already discussed before in Examples 1, 5, and 6. We derived the linear programming problem to find the value in (3) and calculated its average sphere packing value. Then, we saw that is monotone and thus we calculated its monotonicity upper bound. Finally, we showed how to use the graph automorphism approach in order to derive a more compact linear programming problem to calculate