Two-Bit Bit Flipping Decoding of LDPC Codes
In this paper, we propose a new class of bit flipping algorithms for low-density parity-check (LDPC) codes over the binary symmetric channel (BSC). Compared to the regular (parallel or serial) bit flipping algorithms, the proposed algorithms employ one additional bit at a variable node to represent its “strength.” The introduction of this additional bit increases the guaranteed error correction capability by a factor of at least 2. An additional bit can also be employed at a check node to capture information which is beneficial to decoding. A framework for failure analysis of the proposed algorithms is described. These algorithms outperform the Gallager A/B algorithm and the min-sum algorithm at much lower complexity. Concatenation of two-bit bit flipping algorithms show a potential to approach the performance of belief propagation (BP) decoding in the error floor region, also at lower complexity.
High speed communication systems such as flash memory, optical communication and free space optics require extremely fast and low complexity error correcting schemes. Among existing decoding algorithms for LDPC codes  on the BSC, the bit flipping (serial or parallel) algorithms are least complex yet possess desirable error correcting abilities. First described by Gallager , the parallel bit flipping algorithm was shown by Zyablov and Pinsker  to be capable of asymptotically correcting a linear number of errors (in the code length) for almost all codes in the regular ensemble with left-degree . Later, Sipser and Spielman  used expander graph arguments to show that this algorithm and the serial bit flipping algorithm can correct a linear number of errors if the underlying Tanner graph is a good expander. Note that their arguments also apply for regular codes with left-degree . It was then recently shown by Burshtein  that regular codes with left-degree are also capable of correcting a linear number of errors under the parallel bit flipping algorithm.
Despite being theoretically valuable, the above-mentioned capability to correct a linear number of errors is not practically attractive. This is mainly because the fraction of correctable errors is extremely small and hence the code length must be large. Besides, the above-mentioned results do not apply for column-weight-three codes, which allow very low decoding complexity. Also, compared to hard decoding message passing algorithms such as the Gallager A/B algorithm, the error performance of the bit flipping algorithms on finite length codes is usually inferior. This drawback is especially visible for column-weight-three codes for which the guaranteed error correction capability is upper-bounded by (to be discussed later), where is the girth of a code. The fact that a code with or can not correct certain error patterns of weight two indeed makes the algorithm impractical regardless of its low complexity.
In recent years, numerous bit-flipping-oriented decoding algorithms have been proposed (see  for a list of references). However, almost all of these algorithms require some soft information from a channel with capacity larger than that of the BSC. A few exceptions include the probabilistic bit flipping algorithm (PBFA) proposed by Miladinovic and Fossorier . In that algorithm, whenever the number of unsatisfied check nodes suggests that a variable (bit) node should be flipped, it is flipped with some probability rather than being flipped automatically. This random nature of the algorithm slows down the decoding, which was demonstrated to be helpful in practical codes whose Tanner graphs contain cycles. The idea of slowing down the decoding can also be found in a bit flipping algorithm proposed by Chan and Kschischang . This algorithm, which is used on the additive white Gaussian noise channel (AWGNC), requires a certain number of decoding iterations between two possible flips of a variable node.
In this paper, we propose a new class of bit flipping algorithms for LDPC codes on the BSC. These algorithms are designed in the same spirit as the class of finite alphabet iterative message passing algorithms . In the proposed algorithms, an additional bit is introduced to represent the strength of a variable node. Given a combination of satisfied and unsatisfied check nodes, the algorithm may reduce the strength of a variable node before flipping it. An additional bit can also be introduced at a check node to indicate its reliability. The novelty of these algorithms is three-fold. First, similar to the above-mentioned PBFA, our class of algorithms also slows down the decoding. However they only do so when necessary and in a deterministic manner. Second, their deterministic nature and simplicity allow simple and thorough analysis. All subgraphs up to a certain size on which an algorithm fails to converge can be found by a recursive algorithm. Consequently, the guaranteed error correction capability of a code with such algorithms can be derived. Third, the failure analysis of an algorithm gives rise to better algorithms. More importantly, it leads to decoders which use a concatenation of two-bit bit flipping algorithms. These decoders show excellent trade offs between complexity and performance.
Let denote an () LDPC code over the binary field GF(2). is defined by the null space of , an parity check matrix. is the bi-adjacency matrix of , a Tanner graph representation of . is a bipartite graph with two sets of nodes: variable nodes and check nodes. In a -left-regular code, all variable nodes have degree . Each check node imposes a constraint on the neighboring variable nodes. A check node is said to be satisfied by a setting of variable nodes if the modulo-two sum of its neighbors is zero, otherwise it is unsatisfied. A vector is a codeword if and only if all check nodes are satisfied. The length of the shortest cycle in the Tanner graph is called the girth of .
In this paper, we consider 3-left-regular LDPC codes with girth , although the class of two-bit bit flipping algorithms can be generalized to decode any LDPC code. We assume transmission over the BSC. A variable node is said to be corrupt if it is different from its original sent value, otherwise it is correct. Throughout the paper, we also assume without loss of generality that the all-zero codeword is transmitted. Let denote the input to an iterative decoder. With the all-zero codeword assumption, the support of , denoted as is simply the set of variable nodes initially corrupt. In our case, a variable node is corrupt if it is 1 and is correct if it is 0.
A simple hard decision decoding algorithm for LDPC codes on the BSC, known as the parallel bit flipping algorithm [2, 3] is defined as follows. For any variable node in a Tanner graph , let and denote the number of satisfied check nodes and unsatisfied check nodes that are connected to , respectively.
Iii The class of two-bit bit flipping algorithms
The class of two-bit bit flipping algorithms is described in this section. We start with two motivating examples. The first one illustrates the advantage of an additional bit at a variable node while the second illustrates the advantage at a check node.
Iii-a First Motivating Example: Two-bit Variable Nodes
In this subsection, symbols and denote a correct and a corrupt variable node while and denote a satisfied and an unsatisfied check node. Let be a 3-left-regular LDPC code with girth and assume that the variable nodes and form an eight cycle as shown in Fig. 1. Also assume that only and are initially in error and that the parallel bit flipping algorithm is employed. In the first iteration illustrated in Fig. 1\subreffp13, and are unsatisfied while and are satisfied. Since and , and are flipped and become correct. However, and are also flipped and become incorrect since and . In the second iteration (Fig. 1\subreffp24), the algorithm again flips and . It can be seen that the set of corrupt variable nodes alternates between and , and thus the algorithm does not converge.
The parallel bit flipping algorithm fails in the above situation because it uses the same treatment for variable nodes with and . The algorithm is too “aggressive” when flipping a variable node with . Let us consider a modified algorithm which only flips a variable node with . This modified algorithm will converge in the above situation. However, if only and are initially in error (Fig. 1\subreffp12) then the modified algorithm does not converge because it does not flip any variable node. The modified algorithm is now too “cautious” to flip a variable node with .
Both decisions (to flip and not to flip) a variable node with can lead to decoding failure. However, we must pick one or the other due the assumption that a variable node takes its value from the set . Relaxing this assumption is therefore required for a better bit flipping algorithm.
Let us now assume that a variable node can take four values instead of two. Specifically, a variable node takes its value from the set , where () stands for “strong zero” (“strong one”) and () stands for “weak zero” (“weak one”). Assume for now that a check node only sees a variable node either as 0 if the variable node is or , or as 1 if the variable node is or . Recall that is the number of unsatisfied check nodes that are connected to the variable node . Let be the function defined in Table I.
Consider the following bit flipping algorithm.
Compared to the parallel bit flipping algorithm and its modified version discussed above, the TBFA1 possesses a gentler treatment for a variable node with . It tries to reduce the “strength” of before flipping it. One may realize at this point that it is rather imprecise to say that the TBFA1 flips a variable node from to or vice versa, since a check node still sees as 0. However, as the values of can be represented by two bits, i.e., can be mapped onto the alphabet , the flipping of should be understood as either the flipping of one bit or the flipping of both bits.
It is easy to verify that the TBFA1 is capable of correcting the error configurations shown in Fig. 1. Moreover, the guaranteed correction capability of this algorithm is given in the following proposition.
The TBFA1 is capable of correcting any error pattern with up to errors in a left-regular column-weight-three code with Tanner graph which has girth and which does not contain any codeword of weight .
The proof is omitted due to page limits. \qed
It can be shown that the guaranteed error correction capability of a 3-left-regular code with the parallel bit flipping algorithm is strictly less than . Thus, the TBFA1 increases the guaranteed error correction capability by a factor of at least 2.
In , we have shown that the Gallager A/B algorithm is capable of correcting any error pattern with up to errors in a 3-left-regular code with girth . For codes with girth and minimum distance , the Gallager A/B algorithm can only correct up to two errors. This means that the guaranteed error correction capability of the TBFA1 is at least as good as that of the Gallager A/B algorithm (and better for codes with ). It is also not difficult to see that the complexity of the TBFA1 is much lower than that of the Gallager A/B algorithm.
Now that the advantage of having more than one bit to represent the values of a variable node is clear, let us explore the possibility of using more than one bit to represent the values of a check node in the next subsection.
Iii-B Second Motivating Example: Two-bit Check Nodes
In this subsection, we use the symbols and to denote a variable node and a variable node, respectively. The symbols used to denote a variable node and a variable node are shown in Fig. 2\subreffvp2 where is a variable node and is a variable node . The symbols and still represent a satisfied and an unsatisfied check node.
Assume a decoder that uses the TBFA1 algorithm. Fig. 2\subreffvp1, \subreffvp2 and \subreffvp3 illustrates the first, second and third decoding iteration of the TBFA1 on an error configuration with four variable nodes and that are initially in error. We assume that all variable nodes which are not in this subgraph remain correct during decoding and will not be referred to. In the first iteration, variable nodes and are strong and connected to two unsatisfied check nodes. Consequently, the TBFA1 reduces their strength. Since variable nodes and are strong and only connected to one unsatisfied check node, their values are not changed. In the second iteration, all check nodes retain their values (satisfied or unsatisfied) from the first iteration. The TBFA1 hence flips and from to and flips and from to . At the beginning of the third iteration, the value of any variable node is either or . Every variable node is connected to two satisfied check nodes and one unsatisfied check node. Since no variable node can change its value, the algorithm fails to converge.
The failure of the TBFA1 to correct this error configuration can be attributed to the fact that check node is connected to two initially erroneous variable nodes and , consequently preventing them from changing their values. Let us slightly divert from our discussion and revisit the PBFA proposed by Miladinovic and Fossorier . The authors observed that variable node estimates corresponding to a number close to unsatisfied check nodes are unreliable due to multiple errors, cycles in the code graph and equally likely a priori hard decisions. Based on this observation, the PBFA only flips a variable node with some probability . In the above error configuration, a combination of two unsatisfied and one satisfied check nodes would be considered unreliable. Therefore, the PBFA would flip the corrupt variable nodes and as well as the correct variable node and with the same probability . However, one can see that a combination of one unsatisfied and two satisfied check nodes would also be unreliable because such combination prevents the corrupt variable nodes and from being corrected. Unfortunately, the PBFA can not flip variable nodes with less than unsatisfied check nodes since many other correct variable nodes in the Tanner graph would also be flipped. In other words, the PBFA can not evaluate the reliability of estimates corresponding to a number close to unsatisfied check nodes. We demonstrate that such reliability can be evaluated with a new concept introduced below.
Revisit the decoding of the TBFA1 on the error configuration illustrated in Fig. 2. Notice that in the third iteration, except check node , all check nodes that are unsatisfied in the second iteration become satisfied while all check nodes that are satisfied in the second iteration become unsatisfied. We will provide this information to the variable nodes.
A satisfied (unsatisfied) check node is called previously satisfied (previously unsatisfied) if it was satisfied (unsatisfied) in the previous decoding iteration, otherwise it is called newly satisfied (newly unsatisfied).
The possible transitions of a check node are illustrated in Fig. 3. Let , , and be the number of previously satisfied check nodes, previously unsatisfied check nodes, newly satisfied check nodes and newly unsatisfied check nodes that are connected to a variable node , respectively. Let be a function defined as follows:
Consider the following bit flipping algorithm:
The TBFA2 considers a combination of one newly unsatisfied, one newly satisfied and one previously satisfied check node to be less reliable than a combination of one previously unsatisfied and two previously satisfied check nodes. Therefore, it will reduce the strength of and at the end of the third iteration. Consequently, the error configuration shown in Fig. 2 can now be corrected after 9 iterations. Proposition 1 also holds for the TBFA2.
Remarks: Let be the set of all functions from . A natural question to ask is whether can be replaced with some such that the TBFA1 algorithm can correct the error configuration shown in Fig. 2. Brute force search reveals many of such functions. Unfortunately, none of those functions allow the algorithm to retain its guaranteed error correction capability stated in Proposition 1.
We recap this section by giving the formal definition of the class of two-bit bit flipping algorithms.
For the class of two-bit bit flipping algorithms, a variable node takes its value from the set . A check node sees a and a variable node as 0 and sees a and a variable node as 1. According to Definition 1, a check node can be previously satisfied, previously unsatisfied, newly satisfied or newly unsatisfied. An algorithm is defined by a mapping , where is the column-weight of a code.
Different algorithms in this class are specified by different functions . In order to evaluate the performance of an algorithm, it is necessary to analyze its failures. To that task we shall now proceed.
Iv A framework for failure analysis
In this section, we describe a framework for the analysis of two-bit bit flipping algorithms (the details will be provided in the journal version of this paper). Consider the decoding of a two-bit bit flipping algorithm on a Tanner graph . Assume a maximum number of iterations and assume that the channel makes errors. Let denote the subgraph induced by the variable nodes that are initially in error. Let be the set of all Tanner graphs that contain . Let be the subset of with the following property: if then there exists an induced subgraph of such that (i) is isomorphic to and (ii) the two-bit bit flipping algorithm fails to decode on after iterations if the initially corrupt variable nodes are variable nodes in . Let be a subset of such that any graph contains a graph and no graph in contains another graph in . With the above formulation, we give the following proposition.
Algorithm will converge on after decoding iterations if the induced subgraph is not contained in any induced subgraph of that is isomorphic to a graph in .
If fails to converge on after iterations then , hence must be contained in an induced subgraph of that are isomorphic to a graph in . \qed
We remark that Proposition 2 only gives a sufficient condition. This is because might be contained in an induced subgraph of that is not isomorphic to any graph in . Nevertheless, can still be used as a benchmark to evaluate the algorithm . A better algorithm should allow the above sufficient condition to be met with higher probability. For a more precise statement, we give the following.
The probability that a Tanner graph is contained in a Tanner graph with variable nodes is less than the probability that is contained in a Tanner graph with variable nodes if
Let be a Tanner graph with variable nodes such that contains . Since and both have variable nodes, the probability that is contained in equals the probability that is contained in . On the other hand, since contains , the probability that is contained in is less than the probability that is contained in by conditional probability. \qed
Proposition 3 suggests that a two-bit bit flipping algorithm should be chosen to maximize the size (in terms of number of variable nodes) of the smallest Tanner graph in . Given an algorithm , one can find all graphs in up to a certain number of variable nodes by a recursive algorithm. Let denote the set of corrupt variable nodes at the beginning of the -th iteration. The algorithm starts with the subgraph , which is induced by the variable nodes in . Let be the set of check nodes that are connected to at least one variable node in . In the first iteration, only the check nodes in can be unsatisfied. Therefore, if a correct variable node becomes corrupt at the end of the first iteration then it must connect to at least one check node in . In all possible ways, the algorithm then expands recursively by adjoining new variable nodes such that these variable nodes become corrupt at the end of the first iteration. The recursive introduction of new variable nodes halts if a graph in is found. Let be the set of graphs obtained by expanding . Each graph in is then again expanded by adjoining new variable nodes that become corrupt at the end of the second iteration. This process is repeated times where is the maximum number of iterations.
V Numerical results and Discussion
We demonstrate the performance of two-bit bit flipping algorithms on a regular column-weight-three quasi-cyclic LDPC code of length . The code has rate and minimum distance . Two different decoders are considered. The first decoder, denoted as BFD1, employs a single two-bit bit flipping algorithm. The BFD1 may perform iterative decoding for a maximum number of 30 iterations. The second decoder, denoted as BFD2, is a concatenation of 55 algorithms, namely . Associated with algorithm is a maximum number of iterations . The BFD2 operates by performing decoding using algorithm on an input vector for or until a codeword is found. The maximum possible number of decoding iterations performed by the BFD2 is . Details on the algorithms as well as the parity check matrix of the quasi-cyclic LDPC code can be found in .
Simulations for frame error rate (FER) are shown in Fig. 4. Both decoders outperform decoders which use the Gallager A/B algorithm or the min-sum algorithm. In particular, the FER performance of the BFD2 is significantly better. More importantly, the slope of the FER curve of the BFD2 is larger than that of the BP decoder. This shows the potential of two-bit bit flipping decoders with comparable or even better error floor performance than that of the BP decoder. It is also important to remark that although the BFD2 uses 55 different decoding algorithms, at cross over probability , more than 99.99% of codewords are decoded by the first algorithm. Consequently, the average number of iterations per output word of the BFD2 is not much higher than that of the BFD1, as illustrated in Fig. 5. This means that similar to the BFD1, the BFD2 has an extremely high speed.
This work was funded by NSF under the grants CCF-0963726 and CCF-0830245.
-  R. G. Gallager, Low Density Parity Check Codes. Cambridge, MA: M.I.T. Press, 1963.
-  V. Zyablov and M. Pinsker, “Estimation of the error-correction complexity for Gallager low-density codes,” Probl. Inf. Transm., vol. 11, no. 6, pp. 18–26, 1976.
-  M. Sipser and D. Spielman, “Expander codes,” IEEE Trans. Inf. Theory, vol. 42, no. 6, pp. 1710–1722, Nov. 1996.
-  D. Burshtein, “On the error correction of regular LDPC codes using the flipping algorithm,” IEEE Trans. Inf. Theory, vol. 54, no. 2, pp. 517–530, Feb. 2008.
-  X. Wu, C. Ling, M. Jiang, E. Xu, C. Zhao, and X. You, “New insights into weighted bit-flipping decoding,” IEEE Trans. Commun., vol. 57, no. 8, pp. 2177–2180, Aug. 2009.
-  N. Miladinovic and M. Fossorier, “Improved bit-flipping decoding of low-density parity-check codes,” IEEE Trans. Inf. Theory, vol. 51, no. 4, pp. 1594–1606, Apr. 2005.
-  A. Chan and F. Kschischang, “A simple taboo-based soft-decision decoding algorithm for expander codes,” IEEE Commun. Letters, vol. 2, no. 7, pp. 183–185, Jul. 1998.
-  S. Planjery, D. Declercq, S. Chilappagari, and B. Vasic and, “Multilevel decoders surpassing belief propagation on the binary symmetric channel,” in IEEE Int. Symp. Inf. Theory, Jun. 2010, pp. 769–773.
-  S. K. Chilappagari, D. V. Nguyen, B. V. Vasic, and M. W. Marcellin, “Error correction capability of column-weight-three LDPC codes under the Gallager A algorithm - Part II,” IEEE Trans. Inf. Theory, vol. 56, no. 6, pp. 2626–2639, Jun. 2010.
-  “Error floors of LDPC codes - Multi-bit bit flipping algorithm.” [Online]. Available: http://www2.engr.arizona.edu/~vasiclab/Projects/CodingTheory/ErrorFloor%Home.html