Soft decision decoding of Reed-Muller codes: recursive lists
Recursive list decoding is considered for Reed-Muller (RM) codes. The algorithm repeatedly relegates itself to the shorter RM codes by recalculating the posterior probabilities of their symbols. Intermediate decodings are only performed when these recalculations reach the trivial RM codes. In turn, the updated lists of most plausible codewords are used in subsequent decodings. The algorithm is further improved by using permutation techniques on code positions and by eliminating the most error-prone information bits. Simulation results show that for all RM codes of length 256 and many subcodes of length 512, these algorithms approach maximum-likelihood (ML) performance within a margin of 0.1 dB. As a result, we present tight experimental bounds on ML performance for these codes.
Index terms – Maximum-likelihood performance, Plotkin construction, posterior probabilities, recursive lists, Reed-Muller codes.
The main goal of this paper is to design feasible error-correcting algorithms that approach ML decoding on the moderate lengths ranging from 100 to 1000 bits. The problem is practically important due to the void left on these lengths by the best algorithms known to date. In particular, exact ML decoding has huge decoding complexity even on the blocks of 100 bits. On the other hand, currently known iterative (message-passing) algorithms have been efficient only on the blocks of thousands of bits.
To achieve near-ML performance with moderate complexity, we wish to use recursive techniques that repeatedly split an original code into the shorter ones. For this reason, we consider Reed-Muller (RM) codes, which represent the most notable example of recursive constructions known to date. These codes - denoted below - have length and Hamming distance They also admit a simple recursive structure based on the Plotkin construction which splits the original RM code into the two shorter codes of length . This structure was efficiently used in recursive decoding algorithms of -, which derive the corrupted symbols of the shorter codes and from the received symbols. These recalculations are then repeated until the process reaches the repetition codes or full spaces, whereupon new information symbols can be retrieved by any powerful algorithm - say, ML decoding. As a result, recursive algorithms achieve bounded distance decoding with a low complexity order of , which improves upon the complexity of majority decoding .
We also mention two list decoding algorithms of  and , which substantially reduce the error rates at the expense of a higher complexity. In both algorithms, RM codes are represented as the generalized concatenated codes, which are repeatedly decomposed into the shorter blocks similarly to the Plotkin construction. In all intermediate steps, the algorithm of  tries to estimate the Euclidean distance to the received vector and then retrieves the codewords with the smallest estimates. To do so, the algorithm chooses some number of codewords from both constituent codes and Then the product list is constructed for the original code. These lists are recursively re-evaluated and updated in multiple runs. The second technique  is based on a novel sequential scheme that uses both the main stack and the complementary one. The idea here is to lower-bound the minimum distance between the received vector and the best code candidates that will be obtained in the future steps. This “look-ahead” approach gives low error rates and reduces the decoding complexity of .
Recently, new recursive algorithms were considered in  and . In particular, for long RM codes of fixed code rate recursive decoding of  corrects most error patterns of weight ( instead of the former threshold of This is done without any increase in decoding complexity. However, the new decoding threshold is still inferior to that of a much more powerful ML decoding.
In the sequel, we advance the algorithm of , also applying list decoding techniques. This approach mostly follows  and differs from the prior results in a few important aspects. First, we use exact posterior probabilities in our recursive recalculations instead of the distance approximations employed before. This allows us to design a tree-like recursive algorithm that can better sort out all plausible candidates in intermediate steps and avoid multiple decoding runs. Second, we shall see that the output error rate significantly varies for the different information symbols derived in the recursive process. Therefore, we also consider subcodes of RM codes obtained by removing the least protected information bits. Finally, decoding will be improved by applying a few permutations on code positions. As a result, we closely approach the performance of ML decoding on the lengths 256 and 512, which was beyond the reach of the former techniques.
The material is organized as follows. In Section 2, we briefly summarize some recursive properties of RM codes and their decoding procedures. In Section 3, we describe our list decoding algorithm . Finally, in Section 4 we discuss the improvements obtained by eliminating the least protected information bits and using permutation techniques.
2 Recursive encoding and decoding for RM codes
The following description is detailed in . Let any codeword of RM code be represented in the form where and . We say that is split onto two “paths” and . By splitting both paths, we obtain four paths that lead to RM codes of length and so on. In each step of our splitting, we assign the path value to a new -component and to a new -component. All paths end at the repetition codes or full spaces where
Thus, we can consider a specific binary path
that leads from the origin to some left-end code . For any right-end node the same process gives a subpath of length
A similar decomposition can be performed on the block of information bits that encode the original vector In this way, any left-end path gives only one information bit associated with its end node Any right-end path gives information bits associated with the end code We can also add an arbitrary binary suffix of length to the right-end paths, and obtain a one-to-one mapping between the extended paths and information bits
2.2 Basic decoding with posterior probabilities
Let any binary symbol be mapped onto . Then any codeword of RM code belongs to and has the form This codeword is transmitted over a memoryless channel The received block consists of the two halves and , which are the corrupted images of vectors and . The decoder first takes the symbols and for any position and finds the posterior probabilities of transmitted symbols and
To simplify our notation, below we use the associated quantities
Note that is the difference between the two posterior probabilities and of and in position of the left half. Similarly, is obtained on the right half. The following basic recursive algorithm is described in  and Section IV of  in more detail.
Step 1. Let be the posterior probability of any symbol of the codeword We find the corresponding quantity which is (see formula (18) in )
Symbols form the vector of length Then we use some soft-decision decoder that gives a vector and its information block
Step 2. Now we assume that Step 1 gives correct vector Let be the posterior probability of a symbol Then the corresponding quantity is (see formula (19) in )
where The symbols form the vector of length We use some (soft decision) decoding algorithm to obtain a vector and its information block
In a more general scheme , vectors and are not decoded but used as our new inputs These inputs are recalculated multiple times according to (2) and (3). Finally, we reach the end nodes and . Here we perform maximum-likelihood (ML) decoding as follows.
At any node , our input is a newly recalculated vector of length with the given differences between posterior probabilities of two symbols . Rewriting definition (1), we assign the posterior probability
to a symbol . In this way, we can find the posterior probability
of any codeword , and choose the most probable codeword , where
Secondly, we use ML decoding instead of the minimum distance decoding that chooses with the maximum inner product:
Thirdly, we employ a different rule and stop at the repetition codes instead of the biorthogonal codes used in . This last change will make it easier to use the list decoding described in the following section.
Similarly to , this recursion gives decoding complexity
Thus, complexity has maximum order of which is twice the complexity of the algorithm of .
3 List decoding
To enhance algorithm , we shall use some lists of or fewer codewords obtained on any path . This algorithm – called – increases the number of operations at most times and has the overall complexity order of Given any integer parameter , we say that the list have size if decoding outputs either all available records or records, whichever is less This algorithm performs as follows.
At any step of the algorithm , our input consists of records
Each record is formed by some information block its cost function and the corresponding input which is updated in the decoding process These three entries are defined below.
Decoding starts at the root node Here we set and take one record
where is the input vector. Decoding takes the first path (denoted to the leftmost code and recalculates vector similarly to the algorithm However, now we take both values of the first information symbol and consider both codewords and of length in the repetition code . The posterior probabilities (4) of these two vectors will also define the cost function of the new information block
In our list decoding, we represent the two outcomes as the initial edges mapped with their cost functions Then we proceed to the next code which corresponds to the subsequent path denoted Given two different decoding results our recursion (2), (3) gives two different vectors arriving at this node. Therefore, decoding is performed two times and gives the full tree of depth 2. More generally, at any step , decoding is executed as follows.
Suppose that the first paths are already processed. This gives information blocks
of length and the corresponding records Each vector is then recalculated on the new path using formulas (2) and (3), in the same way it was done in Let this path end on some left-end code . Decoding of the new information symbol yields extended blocks
of depth marked by their cost functions
Step is completed by choosing blocks with the highest cost functions
The decoding on the right-end nodes is similar. The only difference is that the full spaces include codewords defined by information blocks of length In this case, we can choose the two most probable vectors (in essence, making bit-by-bit decisions) and set in our cost calculations (8). Another - more refined version of the algorithm - chooses four different vectors of the code whenever The best record is chosen at the last node More generally, the algorithm is executed as follows.
Discussion. Consider the above algorithm on the AWGN channel Using the results of , it can be proven that on this channel, the -component is decoded on the channel with the new noise power
The first approximation is tight for very small (though the channel is no longer Gaussian), while the second one performs well on the “bad” channels with Thus, the noise power always increases in the -direction; the more so the worse the original channel is. By contrast, the -channel can be approximated by the smaller power . These observations also show that the first information symbol - which is obtained on the binary path - is protected the least, and then the decoding gradually improves on the subsequent paths.
Now we see that the algorithm with the list of size delays our decision on any information symbol by steps, making this decision better protected. In the particular case of a bad channel, it can be verified that the first symbol is now decoded when the noise power is reduced times. For this reason, this list decoding substantially reduces the output word error rates (WER) even for small size
For the algorithm processes all the codewords of the first biorthogonal code and is similar to the algorithm of . On the other hand, algorithm updates all cost functions, while chooses one codeword on each end node. Therefore can be considered as a generalization of that continuously updates decoding lists in all intermediate steps. The result is a more powerful decoding that comes along with a higher complexity.
Simulation results. Below we present our simulation results for the AWGN channels. Here we also counted all the instances, when for a given output the decoded vector was more probable than the transmitted one. Obviously, these specific events also represent the errors of ML decoding. Thus, the fraction of these events gives a lower bound on the ML decoding error probability. This lower bound is also depicted in the subsequent figures for all the codes tested.
Our simulation results show that for all RM codes of lengths 128 and 256, the algorithm rapidly approaches ML performance as the list size grows. For RM codes of length 128 and distance , we summarize these results in Fig. 1. For each RM code, we present tight lower bounds for the error probability of ML decoding. To measure the efficiency of the algorithm we also exhibit the actual list size at which approaches the optimal ML decoding within a small margin of
This performance loss is measured at the output word error rate (WER) however, we found little to no difference for all other WER tested in our simulation. In Table 1, we complement these lists sizes with the two other relevant parameters:
the signal-to-noise ratios (SNR per information bit) at which algorithm gives the WER
the complexity estimates counted as the number of floating point operations.
Table 1. RM codes of length 128: the list size decoding complexity, and the corresponding SNR at which algorithm performs within dB from ML decoding at WER
For RM codes of length 256, we skip most decoding results as these will be improved in the next section by the permutation techniques. In our single example in Fig. 2, we present the results for the code This code gives the lowest rate of convergence to the ML decoding among all RM codes of length In other words, all other codes require the smaller lists to achieve the same performance loss This example and other simulation results show that the algorithm performs within 0.5 dB from ML decoding on the lengths 128 and 256 using lists of small or moderate size.
4 Further improvements
4.1 Subcodes of RM codes
More detailed results also show that many codes of length require lists of large size to approach ML decoding within the small margin of 0.25 dB. Therefore for , we also employ a different approach. Namely, the decoding performance can be improved by eliminating those paths, where recursive decoding fails more often. Here we use the results of , which show that the leftmost paths are the least protected.
Recall that each left-end path corresponds to one information symbol. Therefore, decoding on these paths can be eliminated by setting the corresponding information bits as zeros. In this way, we employ the subcodes of the original code . Note that our decoding algorithm runs virtually unchanged on subcodes. Indeed, the single difference arises when some information block takes only one value 0 on the corresponding left node (or less than values on the right node). Therefore, on each step we can proceed as before, by taking only the actual blocks left at this node after expurgation.
In the algorithm , this expurgation starts with the least protected information path that ends at the node It can be shown that for long RM codes of fixed order eliminating even the single weakest path increases the admissible noise power times Thus, the lowest orders yield the biggest gain (10log dB, which equals 0.75 dB and 0.375 dB, respectively.
To proceed further, we eliminate the next weakest path However, the theoretical analysis becomes more complicated on the subsequent bits and it is unclear which bits should be eliminated first. For this reason, we optimized this pruning process in our simulation by making a few ad hoc trials and eliminating subsequent bits in different order.
The corresponding simulation results are presented in Figure 3 for the -code and its -subcode. We see that pruning substantially improves code performance. It is also interesting to compare Figures 2 and 3. We see that the subcode approaches the optimal ML performance much faster than the original code does. In particular, the same margin of dB can be reached with only codewords instead of codewords needed on the code. In all other examples, the subcodes also demonstrated a much faster convergence, which leads to a lesser complexity.
In Fig. 4, we present similar results for the -subcode of the -code Here in Table 2, we also give a few list sizes the corresponding SNRs needed to reach the output WER and the complexity estimates counted by the number of floating point operations. Similar results were also obtained for the subcodes of other RM codes of length 512.
Table 2. (, )-subcode of the (, ) RM code List sizes the corresponding SNRs and complexity estimates needed at WER
These simulation results show that combining both techniques - eliminating the least protected bits and using small lists of codewords - gives a gain of 3 to 4 dB on the lengths over the original non-list decoding algorithm For subcodes, we also approach ML decoding with the lists reduced 4 to 8 times relative to the original RM codes.
4.2 New permutation techniques
The second improvement to the algorithm utilizes the rich symmetry group of RM codes  that includes permutations of positions To employ fewer permutations, we first permute the indices of all positions Thus, we first take a permutation
of indices and consider the corresponding permutations of positions
Remark. Note that the indices represent the different axes in Thus, any permutation of indices is the permutation of axes of For example, the permutation of indices leaves unchanged the first and the fourth quarters of all positions but changes the order of the second and the third quarters.
Given a permutation consider the subset of original indices (axes) that were transformed into the first axes by the permutation . We say that two permutations and are equivalent if these images form the identical (unordered) subsets
Now consider any subset of permutations (9) that includes exactly one permutation from each equivalent class. Thus, includes permutations, each of which specifies a subset of the first indices. Recall that these indices correspond to the axes that are processed first on the subpath (for example, we can start with the axis instead of in which case we first fold the adjacent quarters instead of the halves of the original block). Thus, this subset specifies all possible ways of choosing unordered axes that will be processed first by the algorithm
Given some positive integer (which is smaller than the former parameter we then incorporate these permutations into the list decoding Namely, we form all permutations of the received vector and apply algorithm to each vector . However, at each step of the algorithm, we also combine different lists and leave only best candidates in the combined list, each counted once.
Note that this technique makes only marginal changes to our conventional list decoding Indeed, the single vector in our original setting (7) is replaced by permutations Thus, we use parameter in our initial setting but keep parameter for all decoding steps. If then the number of records drops to almost immediately, after the first decoding is performed on the path
Also, information bits are now decoded in different orders depending on a specific permutation Note that we may (and often do) get the same entries repeated many times. Therefore, in steps 2 and 3 we must eliminate identical entries. This is done in all steps by applying inverse permutations and comparing the corresponding blocks . This permutation-based algorithm is called below and has complexity similar to for all the codes tested.
The motivation for this algorithm is as follows. The specific order of our axes also defines the order in which the decoding algorithm folds the original block into the subblocks of lengths then and so on. Now note that this folding procedure will likely accumulate the errors whenever erroneous positions substantially disagree on the two halves (correspondingly, quarters, and so on). This can also happen if the errors are unevenly spread over the two halves of the original block. By using many permutations, we make it more likely that the error positions are spread more evenly even if they get accumulated in the original setting or any other specific setting. In this way, permutation techniques serve the same functions as interleaving does on the bursty channels.
Simulation results for the moderate lengths 256 and 512 show that the algorithm approaches the optimal ML performance even when the combined list of most probable candidates is reduced two to eight times relative to the original algorithm . For RM codes of length 256, we summarize these results in Fig. 5. For each RM code, we first present the lower bounds for the ML decoding error probability. Similarly to Fig. 1, we then find the minimum size that makes the algorithm perform only within dB away from ML decoding These sizes and complexity estimates are also given in Table 3 Note that both algorithms give smaller lists once this performance loss is slightly increased. In particular, the results in Table 4 show that the lists are reduced two times for dB.
In summary, the permutation algorithm performs within 0.5 dB from ML decoding on the length 256, by processing vectors for all RM codes. To date, both techniques - permutation decoding of complete RM codes and list decoding of their subcodes - yield the best trade-offs between near-ML performance and its complexity known on the lengths
Table 3. RM codes of length 256: the list sizes, complexities, and the corresponding SNRs, at which the permutation algorithm performs within dB from ML decoding at WER
Table 4. RM codes of length 256: the list sizes, complexities, and the corresponding SNRs, at which the permutation algorithm performs within dB from ML decoding at WER
Note, however, that the algorithm gives almost no advantage for the subcodes considered in the previous subsection. Indeed, these subcodes are obtained by eliminating the leftmost (least protected) information bits. However, any new permutation assigns the new information bits to these leftmost nodes. Thus, the new bits also become the least protected. Another unsatisfactory observation is that increasing the size of the permutation set - say, to include all permutations of all indices - helps little in improving decoding performance. More generally, there are a number of important open problems related to these permutation techniques. We name a few:
find the best permutation set for the algorithm ;
analyze the algorithm analytically;
modify the algorithm for subcodes.
5 Concluding remarks
In this paper, we considered recursive decoding algorithms for RM codes that can provide near-maximum likelihood decoding with feasible complexity for RM codes or their subcodes on the moderate lengths .
Our study yet leaves many open problems. Firstly, we need to tightly estimate the error probabilities on the different paths To optimize our pruning procedures for specific subcodes, it is important to find the order in which information bits should be removed from the original RM code. Finally, it is yet an open problem to analytically estimate the performance of the algorithms and
- I.S. Reed, “A class of multiple error correcting codes and the decoding scheme,” IEEE Trans. Info. Theory, vol. IT-4, pp. 38-49, 1954.
- S.N. Litsyn, “On decoding complexity of low-rate Reed-Muller codes,” Proc. 9 All-Union Conf. on Coding Theory and Info. Transmission, Part 1, Odessa, USSR, pp. 202-204, 1988 (in Russian).
- F. Hemmati, “Closest coset decoding of codes,” IEEE Selected Areas Commun., vol. 7, pp. 982-988, 1989.
- G.A. Kabatyanskii, “On decoding of Reed-Muller codes in semicontinuous channels,” Proc. 2 Int. Workshop “Algebr. and Comb. Coding Theory”, Leningrad, USSR, 1990, pp. 87-91.
- R. Lucas, M. Bossert, and A. Dammann, “Improved soft-decision decoding of Reed-Muller codes as generalized multiple concatenated codes,” Proc. ITG Conf. on Source and Channel Coding, Aahen, Germany, 1998, pp. 137-141.
- N. Stolte and U. Sorger, “Soft-decision stack decoding of binary Reed-Muller codes with “Look-Ahead” technique,” Proc. 7 Int. Workshop “Algebr. and Comb. Coding Theory”, Bansko, Bulgaria, June 18-24, 2000, pp. 293-298.
- F.J. MacWilliams, N.J.A. Sloane, The Theory of Error-Correcting Codes, North-Holland, Amsterdam, 1981.
- I. Dumer, “Recursive decoding of Reed-Muller codes,” Proc. 37 Allerton Conf. on Commun., Contr., and Computing, Monticello, IL, Sept. 22-24, 1999, pp. 61-69.
- I. Dumer and K. Shabunov, “Recursive constructions and their maximum likelihood decoding,” Proc. 38 Allerton Conf. on Commun., Contr., and Computing, Monticello, IL, USA, 2000, pp. 71-80.
- I. Dumer, “Soft decision decoding of Reed-Muller codes: a simplified algorithm,” IEEE Trans. Info. Theory, this issue.