Optimal Thresholds for GMD Decoding with –extended Bounded Distance Decoders
We investigate threshold–based multi–trial decoding of concatenated codes with an inner Maximum–Likelihood decoder and an outer error/erasure –extended Bounded Distance decoder, i.e. a decoder which corrects errors and erasures if , where is the minimum distance of the outer code and . This is a generalization of Forney’s GMD decoding, which was considered only for , i.e. outer Bounded Minimum Distance decoding. One important example for –extended Bounded Distance decoders is decoding of –Interleaved Reed–Solomon codes. Our main contribution is a threshold location formula, which allows to optimally erase unreliable inner decoding results, for a given number of decoding trials and parameter . Thereby, the term optimal means that the residual codeword error probability of the concatenated code is minimized. We give an estimation of this probability for any number of decoding trials.
One of Forney’s seminal contributions to algebraic coding was the invention of Generalized Minimum Distance (GMD) decoding [1, 2]. It provides a means to exploit soft information from the channel using a hard–decision algebraic decoder by multi–trial error/erasure decoding with a varying number of erased unreliable input symbols. Most intriguing about the GMD scheme is that it performs as good as Maximum Likelihood (ML) decoding if the channel is good. This gives rise to the frequent application of GMD decoding for concatenated codes. There, the inner code is responsible for correcting a considerable amount of transmission channel errors. Thus, the input symbols for the outer decoder can be viewed as being transmitted over a super channel, which is composed of the transmission channel and the inner decoder. This super channel is always good if the parameters of the inner code are chosen appropriately.
Any decoder’s performance can be measured by its guaranteed decoding radius and its residual codeword error probability, the latter one being a function of the channel. The fundamental research problem of threshold–based multi–trial decoding is to find for the set of thresholds , , which optimizes the respective performance measure. Generally, the output of multi–trial decoding is a result list. In this paper, we denote the case that the transmitted codeword is among the elements of the result list as a decoding success. Both the decoding radius and the residual codeword error probability are to be understood in this context.
Maximization of the guaranteed decoding radius of GMD decoding for concatenated codes was considered by Blokh and Zyablov . Using Linear Programming, they obtained optimal threshold sets when both inner and outer code are BMD–decoded. In previous work [4, 5], we generalized their result to the case where the outer code is –extended Bounded Distance (BD)–decoded for the full (real) range .
Blokh and Zyablov also considered the probably more practical performance measure, i.e. minimization of the residual codeword error rate. For concatenated codes with inner ML and outer BMD decoding they derived optimal threshold sets using results on the Binary Symmetric Channel (BSC) error exponent from Gallager [6, 7] and Forney . In this paper, we tackle the generalization to the case of an outer –extended Bounded Distance (BD) decoder, building up on our previous results . Thereby, .
The paper is organized as follows. In Section II, we describe the structure and threshold–based multi–trial decoding of concatenated codes, in Section III, we derive necessary and sufficient conditions for an optimal threshold set. We do this on a high level, using the error- and erasure probabilities for each threshold pair , , as parameters. In Section IV, we recall Forney’s generalization of Gallager’s error exponent of the BSC channel in the error/erasure decoding case. Simple approximations of the error- and erasure probabilities are derived in Section V. This allows to analytically calculate the set of optimal thresholds in Section VI together with results on the residual codeword error probability. In Section VII, we wrap up the paper and draw conclusions for further research.
Ii GMD Decoding of Concatenated Codes
A concatenated code consists of two constituent codes and . We denote as the inner code, as the outer code and as the concatenated code. Since is binary, such is . W.l.o.g. we restrict ourselves to this most practical case.
An information vector is first encoded into a codeword . The –ary symbols , , of are then converted into binary vectors and encoded into . The binary matrix consisting of the is then transmitted over a BSC channel with crossover probability .
At the receiver, erroneous vectors are received. They are fed into an ML decoder for .The resulting codeword estimates are mapped to their information vectors and converted into symbols , where is the erasure symbol. The vector and the number of thresholds are the input for the GMD decoder of .
Inside the GMD decoder, is processed in the following way. First, for every symbol , the reliability value ,
is calculated. Then, the threshold set , , is applied as
resulting in an input list , where . Thus, a decoding result of the inner decoder is discarded in decoding trial , , if its reliability value falls below the threshold . Finally, an error/erasure BD decoder (in our case ) is applied to every element of resulting in a result list . Whenever we have a decoding success.
Iii Necessary and Sufficient Conditions
This section generalizes our result from  which was obtained for the simple case where the inner code is BPSK modulation and the outer BD decoder has parameter , i.e. it is a BMD decoder.
Considering (2), it is clear that two cases can occur while advancing from threshold to . First, it is possible that a correct symbol from the super channel is erased for while it was not for . Second, it is possible that a wrong symbol is erased for while it was not for . The probabilities for these two cases are defined by
. Additionally, we define
for the three border cases. We shall find useful approximations for these probabilities in Section V. Note, that .
With each of the probabilities we associate the number of symbols from falling into the case, i.e. . Obviously, .
Let and be the numbers of erroneous and erased symbols, respectively, in . An BD decoder for succeeds in decoding as long as
This inequality can be expressed by as
since it follows from (2) and the orderliness of the threshold set that a symbol is erased by if it is erased by . Unequality (4) is then obtained by simply counting all symbols which are errors and erasures, respectively, for decoding trial and replacing and in (3). Let
we obtain an exact formula for the residual codeword error probability of the GMD decoder with . We can replace the condition by
to obtain a good approximation of for . Condition can be compressed to
if we consider . The latter set of equalities can be seen by subtracting two subsequent equations of from each other. Since the super channel can be assumed to be good, we can further approximate by
The previous observations allow to prove the following theorem.
The following conditions are necessary and sufficient for an optimal threshold set , which minimizes the residual codeword error rate .
Equivalent to the maximization in (5), we can also express the approximation for in logarithmic form, i.e.
Then, the maximization term is a linear function of the . Thus, its maximum is attained at some boundary point fulfilling condition , i.e.
In non–logarithmic form:
Let fulfill the statement of the theorem and let be a set of thresholds where at least one threshold is different than in . Assume that is optimal. The only possible way for to achieve a smaller would be to decrease all terms in (9) simultaneously. This is impossible, decreasing any of the probabilities would increase at least one of the others. This proves that is both unique and optimal. ∎
Iv Forney’s Generalization of Gallager’s Error Exponent
Let us for a while consider one specific decoding trial , and the corresponding threshold . Reliability values are calculated according to (1) and thresholds are applied as in (2). Hence, by erasing, the inner ML decoder becomes a decoder with erasing option. Its decoding criterion is defined by
It was shown by Forney that this criterion is optimal in a sense that no other criterion can decrease both the error and error–or–erasure probability . In the same publication, it was shown that both probabilities can be expressed in terms of Gallager’s error exponent for the BSC [6, 7]. They are given by
where is Gallager’s exponent and the corresponding optimization parameter, .
V Approximated Probabilities
In this section, we shall find simple approximations for the probabilities , and , which were defined in Section III. The approximations are required to obtain an analytic threshold location formula fulfilling the necessary and sufficient conditions of Theorem 1. Let us start with the following observation. With Gallager’s exponent, the error probability of an ML decoder is . For such decoding, there exists a Hamming distance radius such that decoding of with almost always succeeds and decoding of with almost always yields an erroneous result. This radius can be thought of as an approximation of the borders of the Voronoi cells of . For decoding with erasure option as in (10) and the error exponents from (11) and (12) this gives the following approximations of and . Recall, that is the crossover probability of the BSC.
. Since , we also have . Of course, the probabilities and radii vary for different thresholds. Hence, we append the threshold index as a parameter, i.e. we denote the probabilities and radii for by , and , respectively. Note, that from follows and .
Let us consider the probability for a symbol, which is erased for each threshold from . Its Hamming distance to the transmitted codeword must be at least and at most . Otherwise, there would be a threshold for which the symbol would not be erased. Consequently, we have
Now, let us approximate the probability for a wrong symbol which is never erased. Its Hamming distance to the transmitted codeword must be at least . We obtain
The probabilities , , can be approximated as follows. Symbols counting towards must lie between and as can be seen in Figure 1. For we observe that the symbols must lie between and . We obtain
With equations (13), (14), (15), and (16) we expressed the probabilities , and in terms of the probabilities and , for which we can use the error- and erasure exponents from (11) and (12), respectively. After some rather technical simplifications, this yields the following simple expressions.
The probabilities and can be approximated by
Vi Optimal Thresholds and Residual Codeword Error Probability
With the results from Sections III and V we can now derive an analytic formula for the optimal thresholds , . Consider Theorem 1. It basically states a system of equations, the optimal threshold set with elements being its solution. Let us express the equations using the approximated probabilities from Lemma 1. For (6), this gives
The optimal threshold set for GMD decoding of a concatenated code, with inner ML and outer BD decoding, , is given by
where is Gallager’s error exponent for the BSC and is the corresponding optimization parameter, .
For outer BMD decoding, i.e. , the optimal threshold set is given by
The corollary coincides with the results of Blokh and Zyablov . Thus, we obtain their result as a special case of our main result, i.e. Theorem 2. Note, that both and – and thereby also – are functions of the BSC’s crossover probability . Also note, that in both cases is non–decreasing in .
We shall now state the residual codeword error probability, which can be achieved using an optimal set of thresholds for . To do this, we return to Theorem 1, more precisely to (9) in its proof. We saw that all terms in the maximization must be equal. Hence, we have the expression
Using Lemma 1 gives
i.e. is defined by the largest threshold within .
If we are restricted to one single threshold, becomes
The opposite extremal case is an unlimited number of thresholds. To calculate , we require the largest possible threshold, i.e. . L’Hospital’s rule for (20) yields
In the same manner as before, we obtain
For BSC crossover probability , the probabilities in (24) are almost equal. Moreover, approaches the ML error probability as goes to zero. By (24), this happens even faster for and . Morever, our experiments show that the ML error probability is already achieved for moderate channel conditions, especially if .
Figure 2 shows exemplary residual error probability curves for a concatenated code with inner code , outer code and outer BD–decoding with a varying number of optimal thresholds. We observe that BD decoding always beats BMD decoding. This could have been expected, since it can be shown that
We investigated threshold–based multi–trial decoding of concatenated codes with inner ML and outer BD decoding. For any integer number of decoding trials, i.e. thresholds, we gave an analytic formula for the optimal locations of the thresholds in a sense that the residual codeword error probability is minimized. We showed that for an arbitrary number of thresholds, outer BD decoding outperforms outer BMD decoding and gave a range of achievable error probabilities. Within this range, the system designer can select to meet given performance and complexity constraints.
Our results can be applied to standardized concatenated coding schemes, e.g. for the CCSDS Telemetry Channel . It utilizes a set of outer Reed–Solomon (RS) codes and an inner convolutional code. For a small modification of the standard , the RS odes can be decoded collaboratively, i.e. they can be considered as an –IRS code. For such codes, an efficient error/erasure BD decoding algorithm has been proposed in . Its complexity is the same as for decoding the RS codes separately. Hence, the complexity of multi–trial decoding with the outer BD decoder grows only linearly in .
-  G. D. Forney, “Generalized Minimum Distance decoding,” IEEE Trans. Inform. Theory, vol. IT-12, pp. 125–131, April 1966.
-  G. D. Forney, Concatenated Codes. Cambridge, MA, USA: M.I.T. Press, 1966.
-  E. L. Blokh and V. V. Zyablov, Linear Concatenated Codes. Nauka, 1982. In Russian.
-  C. Senger, V. R. Sidorenko, M. Bossert, and V. V. Zyablov, “Decoding generalized concatenated codes using interleaved Reed–Solomon codes,” in Proc. IEEE Int. Symposium on Inform. Theory, (Toronto, ON, Canada), July 2008.
-  C. Senger, V. R. Sidorenko, M. Bossert, and V. V. Zyablov, “Multi-trial decoding of concatenated codes using fixed thresholds.” Preprint, 2008.
-  R. G. Gallager, “A simple derivation of the coding theorem and some applications,” IEEE Trans. Inform. Theory, vol. IT-11, pp. 3–18, Jan 1965.
-  R. G. Gallager, Information Theory and Reliable Communication. New York: John Wiley & Sons, 1968. ISBN 0-471-29048-3.
-  G. D. Forney, “Exponential error bounds for erasure, list, and decision feedback schemes,” IEEE Trans. Inform. Theory, vol. IT-14, pp. 206–220, March 1968.
-  C. Senger, V. R. Sidorenko, and V. V. Zyablov, “On Generalized Minimum Distance decoding thresholds for the AWGN channel,” in Proc. XII Symposium Problems of Redundancy in Information and Control Systems, (St. Petersburg, Russia), May 2009.
-  Consultative Committee for Space Data Systems, Telemetry Channel Coding, October 2002. Recommendation for Space Data System Standards, CCSDS 101.0-B-6, Blue Book, Issue 6.
-  G. Schmidt, C. Senger, and M. Bossert, “Concatenated code designs with outer interleaved Reed–Solomon codes and inner tailbiting convolutional codes,” in Proc. International ITG Conference on Source and Channel Coding, (Ulm, Germany), January 2008.
-  G. Schmidt, V. R. Sidorenko, and M. Bossert, “Collaborative decoding of interleaved Reed–Solomon codes and concatenated code designs,” IEEE Trans. Inform. Theory, vol. IT-55, pp. 2991–3012, July 2009.
-  I. I. Dumer, “Concatenated codes and their multilevel generalizations,” in Handbook of Coding Theory, vol. II, ch. 23, Amsterdam: North-Holland, 1998. ISBN 0-444-50087-1.