LP decoding of expander codes: a simpler proof

LP decoding of expander codes: a simpler proof

Michael Viderman
Computer Science Department
Technion — Israel Institute of Technology
Haifa 32000, Israel
viderman@cs.technion.ac.il
Research supported by an ERC-2007-StG grant number 202405.
Abstract

A code is a -expander code if it has a Tanner graph, where every variable node has degree , and every subset of variable nodes such that has at least neighbors.

Feldman et al. (IEEE IT, 2007) proved that LP decoding corrects errors of -expander code, where .

In this paper, we provide a simpler proof of their result and show that this result holds for every expansion parameter .

1 Introduction

The central algorithmic problem in coding theory is the explicit construction of error-correcting codes with best possible parameters together with fast encoding and decoding algorithms. Recently, this area has benefited enormously from insights and viewpoints originating in complexity theory, and numerous interconnections between codes and complexity theory have been discovered, which are surveyed for example in [9, 12]. The former survey [9] focuses on a notion called list-decoding and the second survey [12] focuses mainly on sub-linear algorithms for local testing and local decoding. Basically, there are two different kinds of noise models: adversarial and probabilistic. In this paper we consider an adversarial noise model where we only assume a bound on the number of errors and not how they are distributed. We refer a reader to the seminal paper of Richardson and Urbanke [7] for details concerning probabilistic noise model.

Surprisingly, all known constructions of asymptotically good error-correcting codes which can be decoded in linear time when a constant fraction of symbols is adversarially corrupted are based on expander codes. In particular, when the parity check graph of the low density parity check (LDPC) code has good expansion properties the associated code is called an expander code (see Definition 2). Expander codes and their decoding algorithms are often (implicitly) involved as basic building blocks in the constructions of asymptotically good codes which are linear-time decodable.

More formally, a linear code is a -expander code if it has a Tanner graph [11] , where every variable node has degree and every subset of variable nodes such that has at least neighbors.

The celebrated result of Sipser and Spielman [8] showed that regular expander codes with expansion parameter are decodable in linear time from errors. They achieved this by defining an extremely simple decoding algorithm, called Flip Algorithm. Feldman et al. [3] showed that the linear programming (LP) decoding (suggested in [4]) corrects errors in polynomial time if the underlying code is a -expander with expansion parameter . It is worth to notice that the running time of the LP decoder was strongly improved later (see e.g. [2, 10, 14]), although linear running time was not achieved. Then it was shown that similar result can be obtained using a different decoding algorithm [13], which in particular runs in linear time. Moreover, [13, Proposition D.1] points out that some regular expander codes with expansion parameter have minimum Hamming distance 2 and thus cannot correct even a single error. This leaves open the intriguing question: whether all regular expander codes with expansion parameter more than are polynomial time decodable from the constant fraction of errors.

While the decoding algorithm provided in [13] reaches better (proven) result than LP decoding in [3] with regards to regular expander codes, the LP decoder still looks much more powerful. Hence we might hope that the future of studying of LP decoding capabilities (see [1, 15, 16, 6, 5]) will resolve the above question.

The proof of the main result of Feldman et al. [3] was indirect and took the LP dual witness twice by using the min-cut max-flow theorem. We show a straightforward proof for the result of [3] which in particular holds for every expansion parameter . We think that our simplified proof sheds more light on the intuition lying behind the LP decoding of expander codes.

2 Preliminaries

Let be the binary field and be the set . In this work, we consider only linear codes. Let be a linear code over a field . For , let and . We define the distance between two words to be . The minimum distance of a code is defined by . The dual code is defined by . For we say that if for all we have .

Now we define expanders and expander codes. We start from the definition of expanders and then proceed to the definition of expander codes.

{definition}

[Expander Codes] Let be a linear code and let be its Tanner graph [11], where represents the variable nodes and represents the parity check nodes. Note that for every we have if and only if . For and it holds that if and only if . For and , let

  • be the set of neighbors of ,

  • be the set of neighbors of the node .

Let be constants. Then, is called a -expander if every vertex has degree and for all subsets such that we have .

We say that a code is a -expander code if it has a parity check graph that is a -expander.

3 Main Result

Feldman et al. [3] proved the following theorem.

{theorem}

[[3]] If is a -expander code, where and is an integer then is decodable from at most errors (in polynomial time) by LP decoding.

Their proof took the LP dual witness twice and in particular, they applied min-cut max-flow theorem to find such a witness. Feldman et al. [3] mentioned that it would be interesting to see a more direct proof of this result.

We reprove their result in the more straightforward manner and obtain a small improvement to the required expansion parameter, i.e., we assume only that .

{theorem}

[Main Theorem] If is a -expander code, where and is an integer then is decodable from at most errors (in polynomial time) by LP decoding.

In Section 4 we recall an implicit result (Theorem 4) that was shown in [3]. We shall use Theorem 4 in the proof of Theorem 3, which appears in Section 5.

4 Implicit result of Feldman et al. [3]

Let be a code and be its parity check graph. Assume that is an input word (a corrupted codeword) and is a set of error coordinates, i.e., if all bits in were flipped then would be the codeword of . We say that if and if .

{definition}

[Feasible weights] Let be the parity check graph of the code . A setting of edge weights is called feasible with respect to if

  1. For all and distinct we have .

  2. For all we have .

The following theorem was implicit in [3] and was argued in [3, Proposition 2] and by the discussions in [3, Sections 2 and 3].

{theorem}

[Implicit in [3]] If for all , there exists a feasible settings of weights then LP decoding for the code corrects errors. Moreover, if is an LDPC code then LP decoding runs in polynomial time.

The nice point here is that one can use Theorem 4 without any background on the LP decoding technique. We also note that this Theorem is related to the LDPC codes in general, not necessarily to the expander codes.

5 Proof of Theorem 3

We start the proof by defining the concept of matching (slightly different and simpler than in [3]).

{definition}

[-Matching] Let be a subset and let be an integer. With some abuse of notations we say that is a matching if for every we have at most one such that . Given a subset we say that is a -matching with respect to if

  1. is a matching.

  2. For every we have at least nodes such that .

For the rest of the proof, let be the set of error locations in the input word, and

We let . The following proposition is similar to the corresponding proposition in [3], where an appropriate modification was made to fit our definition of matching (Definition 5).

{proposition}

Let and assume . If there exists an -matching with respect to then there is a feasible edge weight assignment (as in Definition 4).

Proof.

We assign weights as follows.

  • For such that there is an and , set , and for all other set

  • For all other , for all set .

Clearly, this weighting satisfies the first condition of a feasible weight assignment. For condition 2 we distinguish three cases. We recall that is an integer.

  1. For we have . Also, at least edges that are incident to are in (and each has weight ). All other incident edges have weight or . So, the total weight of the incident edges to is at most . This is less than as long as .

  2. If then . At least of ’s incident edges are in , but (trivially) not incident to . These edges have weight 0. The other incident edges have weight either 0 or . So, the total weight is at most which is smaller than as long as .

  3. When then . We know that has less than neighbors in , i.e., at most neighbors in . All other edges have weight 0. I.e., the total weight is at most which is smaller than as long as .

Summarizing our conditions on , we have . There is such an if , i.e., . ∎

It is not hard to see that a sufficiently strong upper bound on implies an upper bound on .

{proposition}

Assume that . Then .

Proof.

Recall that .

We prove that . Assume the contrary and let such that (note that ).

We argue that which implies that with contradiction. Due to the fact that is a -expander code we have

Then we have and . Contradiction. ∎

We are ready to prove Theorem 3.

Proof of Theorem 3.

Let be a set of error locations. By assumption, and so, by Proposition 5 we have . By Corollary 5.1 (stated below) there exists an -matching with respect to . By Proposition 5 there exists a feasible edge weight assignment. Theorem 4 implies that the LP decoder succeeds and runs in polynomial time. ∎

5.1 Hall’s Theorem

Let us recall that is the bipartite graph with vertex sets and , and the edge set . We also recall Definition 5. We let to be a subset of .

{theorem}

[Hall’s Marriage Theorem] Assume that for every we have . Then there exists a -matching with respect to .

{corollary}

[Polygamous form of Hall’s Theorem] Let be an integer. Assume that for all we have . Then there exists -matching with respect to .

Proof.

Replace each with nodes connected to all nodes in . The corollary follows from the Hall’s theorem (Theorem 5.1). ∎

Acknowledgements

We thank the anonymous referees for valuable comments on an earlier version of this paper.

References

  • [1] Sanjeev Arora, Constantinos Daskalakis, and David Steurer. Message passing algorithms and improved LP decoding. In Michael Mitzenmacher, editor, Proceedings of the 41st Annual ACM Symposium on Theory of Computing, STOC 2009, Bethesda, MD, USA, May 31 - June 2, 2009, pages 3–12. ACM, 2009.
  • [2] David Burshtein. Iterative approximate linear programming decoding of LDPC codes with linear complexity. IEEE Transactions on Information Theory, 55(11):4835–4859, 2009.
  • [3] Jon Feldman, Tal Malkin, Rocco A. Servedio, Clifford Stein, and Martin J. Wainwright. LP decoding corrects a constant fraction of errors. IEEE Transactions on Information Theory, 53(1):82–89, 2007.
  • [4] Jon Feldman, Martin J. Wainwright, and David R. Karger. Using linear programming to decode binary linear codes. IEEE Transactions on Information Theory, 51(3):954–972, 2005.
  • [5] Nissim Halabi and Guy Even. LP decoding of regular LDPC codes in memoryless channels. IEEE Transactions on Information Theory, 57(2):887–897, 2011.
  • [6] Ralf Koetter and Pascal O. Vontobel. On the block error probability of LP decoding of LDPC codes, February 25 2006.
  • [7] Thomas J. Richardson and Rüdiger L. Urbanke. The capacity of low-density parity-check codes under message-passing decoding. IEEE Transactions on Information Theory, 47(2):599–618, 2001.
  • [8] Michael Sipser and Daniel A. Spielman. Expander codes. IEEE Transactions on Information Theory, 42(6):1710–1722, 1996. Preliminary version appeared in FOCS 1994.
  • [9] Madhu Sudan. List decoding: algorithms and applications. SIGACT News, 31(1):16–27, 2000.
  • [10] Mohammad H. Taghavi, Amin Shokrollahi, and Paul H. Siegel. Efficient implementation of linear programming decoding. IEEE Transactions on Information Theory, 57(9):5960–5982, 2011.
  • [11] Robert Michael Tanner. A recursive approach to low complexity codes. IEEE Transactions on Information Theory, 27(5):533–547, 1981.
  • [12] Luca Trevisan. Some applications of coding theory in computational complexity. In ECCC: Electronic Colloquium on Computational Complexity, technical reports, 2004.
  • [13] Michael Viderman. Linear time decoding of regular expander codes. In Innovations in Theoretical Computer Science 2012 (ITCS 2012), Cambridge, MA, USA, January 8-10, 2012, pages 168–182. ACM, 2012.
  • [14] Pascal O. Vontobel. Interior-point algorithms for linear-programming decoding. In Proceedings of the Information Theory and its Applications Workshop, UC San Diego, La Jolla, CA, USA, 2008.
  • [15] Pascal O. Vontobel and Ralf Koetter. Bounds on the threshold of linear programming decoding, February 25 2006.
  • [16] Pascal O. Vontobel and Ralf Koetter. Towards low-complexity linear-programming decoding, February 25 2006.
Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
""
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
   
Add comment
Cancel
Loading ...
276632
This is a comment super asjknd jkasnjk adsnkj
Upvote
Downvote
""
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters
Submit
Cancel

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test
Test description