Proving Threshold Saturation for Nonbinary SC-LDPC Codes on the Binary Erasure Channel

# Proving Threshold Saturation for Nonbinary SC-LDPC Codes on the Binary Erasure Channel

Alexandre Graell i Amat, Iryna Andriyanova, Amina Piemontese
Department of Signals and Systems, Chalmers University of Technology, Gothenburg, Sweden
ETIS Laboratory, ENSEA/University of Cergy-Pontoise/CNRS, Cergy-Pontoise, France
Università di Parma, Dipartimento di Ingegneria dell’Informazione, Viale G. P. Usberti 181/A, Parma, Italy
Research supported by the Swedish Research Council under grant #2011-5961, the Swedish Foundation for Strategic Research (SSF) under the Gustaf Dalén project IMF11-0077, and the Adlerbert Research Foundation.
###### Abstract

We analyze nonbinary spatially-coupled low-density parity-check (SC-LDPC) codes built on the general linear group for transmission over the binary erasure channel. We prove threshold saturation of the belief propagation decoding to the potential threshold, by generalizing the proof technique based on potential functions recently introduced by Yedla et al.. The existence of the potential function is also discussed for a vector sparse system in the general case, and some existence conditions are developed. We finally give density evolution and simulation results for several nonbinary SC-LDPC code ensembles.

## I Introduction

Spatial coupling of low-density parity-check (LDPC) codes has revealed as a powerful technique to construct codes that universally achieve capacity for many channels under belief propagation (BP) decoding. The main principle behind this outstanding behavior is the convergence of the BP threshold to the maximum a posteriori (MAP) threshold of the underlying block code ensemble, a phenomenon known as threshold saturation [1]. The concept of spatial coupling is not exclusive of LDPC codes, and also applies to other scenarios, such as relaying, compressed sensing, and statistical physics. In the realm of coding, spatial coupling has also been recently applied to turbo codes [2].

Nonbinary LDPC codes defined over GF have received an increasing interest in the recent years, since for short-to-moderate block lengths they have been shown to outperform their binary counterparts. Nonbinary spatially-coupled LDPC (SC-LDPC) codes have been considered more recently in, e.g., [3, 4]. In [3] a method to compute an upper bound on the MAP threshold for nonbinary LDPC codes on the binary erasure channel (BEC) was proposed, and it was shown that the MAP threshold of regular ensembles improves with and approaches the Shannon limit. It was also empirically shown in [3] that threshold saturation occurs for nonbinary SC-LDPC codes.

In this paper, we prove111The proofs of the main results in this paper can be found in the extended version of the paper [5]. threshold saturation of the BP threshold of nonbinary SC-LDPC codes on the BEC to the so called potential threshold, which is conjectured to coincide with the MAP threshold. Our proof is based on the proof technique proposed in [6, 7] to prove threshold saturation for (binary) SC-LDPC codes. This technique is based on the observation that the density evolution (DE) equations of LDPC codes form an admissible system for which it is possible to properly define a potential function, and that a fixed point of the DE corresponds to a stationary point of the corresponding potential function. Our proof is a non-trivial generalization of the proof in [7] to accomodate nonbinary SC-LDPC codes. In particular, we discuss the necessary conditions for the existence of the potential function for a vector sparse system in the general case, and show that the potential function in the form of [7] does not exist for nonbinary codes. We also give DE results and simulation results for several nonbinary SC-LDPC code ensembles.

### I-a Notation and Some Definitions

We use upper case letters to denote scalar functions, bold lowercase letters to denote vectors, and bold uppercase letters for matrices. We assume all vectors to be row vectors, and we denote by the row vector obtained by transposing the vector of stacked columns of matrix .

Let be a non-negative vector of length . The Jacobian of a scalar function is defined as . Also, we define the Jacobian of a vector function as , where and , and the Hessian of a vector function as .

## Ii Density Evolution for (dv,dc,m) and (dv,dc,m,L,w) LDPC Code Ensembles over GF(2m)

We consider transmission over a BEC with erasure probability , denoted by BEC(), using nonbinary LDPC codes defined over the general linear group. The code symbols are elements of the binary vector space GF, of dimension , and we transmit on the BEC the -tuples representing their binary image. We denote a regular nonbinary LDPC code ensemble over GF as , where and denote the variable node degree and the check node degree, respectively. In this paper, we will also consider the regular SC-LDPC code ensembles described in [1], where denotes the spatial dimension, and is the smoothing parameter. This ensemble is obtained by placing sets of variable nodes of degree at positions . A variable node at position has connections to check nodes at positions from the range . For each connection, the position of the check node is uniformly and independently chosen from that range. A (terminated) SC-LDPC code ensemble is defined by the parity-check matrix

 H=⎡⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢ ⎢⎣H0(1)⋮⋱Hw−1(1)H0(L)⋱⋮Hw−1(L)⎤⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥⎦.

Each submatrix is a sparse nonbinary matrix, where is the number of variable nodes in each position and the number of check nodes in each position. It is important to note that the check node degrees corresponding to the first and last couple of positions is lower than , i.e., the graph shows some irregularities. These irregularities lead to a locally better decoding (at the expense of a rate loss, which vanishes with ) and are the responsible for the outstanding performance of SC-LDPC codes.

In general, the messages exchanged in the BP decoding of nonbinary LDPC codes are real vectors of length , the th element of which representing the a posteriori probability that the symbol is . For the DE on the BEC, however, it is sufficient to keep track of the dimension of the messages exchanged. Therefore, the DE simplifies to the exchange of messages of length , where the th entry of the message is the probability that the message has dimension .

### Ii-a (dv,dc,m) Regular LDPC Code Ensemble over GF(2m)

Consider a ensemble over GF, used for transmission over the BEC(). Let and be probability vectors of length , where (resp. ) is the probability that a message from (resp. to) variable nodes at iteration has dimension , . The DE updates for the variable nodes and the check nodes at iteration are described by

 xℓ∘=f∘(yℓ∘;ε),    yℓ∘=g∘(xℓ−1∘).

where and are functions from to , defined as

 f∘(y∘;ε) =p∘(ε)⊡(⊡dv−1y∘), (1) g∘(x∘) =⊠dc−1x∘. (2)

For two probability vectors and of length , the operations and in (1)-(2) are defined as

 [a∘⊡b∘]k =m∑i=kk+m−1∑j=kVmi,j,ka∘ib∘j,k=0,…,m, (3) k =k∑i=0k∑j=k−iCmi,j,ka∘ib∘j,k=0,…,m, (4)

where is the probability of choosing a subspace of dimension whose intersection with a subspace of dimension has dimension , and is the probability of choosing a subspace of dimension whose sum with a subspace of dimension has dimension (see [5] for details). Moreover, we define with terms (i.e., ), and with terms (i.e., ).

In (1) is a row vector of length , the th element of which being the probability that the channel message has dimension ,

 p∘i(ε)=(mi)εi(1−ε)m−i,i=0,⋯,m. (5)

Also, .

The fixed-point DE equation for is

 x∘=f∘(g∘(x∘);ε). (6)

Note that decoding is successful when the DE equation converges to .

The proof technique in [7] requires monotone vector functions for the variable node and the check node updates. It can be shown that and are not monotone, hence cannot be used directly. In the following, we rewrite the DE equation in (6) in a more suitable form to prove threshold saturation.

###### Definition 1

Given a probability vector , define the CCDF vector , where . We also define . Then, it follows that . Note also that . For simplicity of further notation, let denote a right shift of with a prepended 1.

Considering the CCDF vectors , and , we can define new vector functions and , with

 fi =m∑k=if∘k(y∘;ε)=m∑k=i[(p−1−p)⊡(⊡dv−1(y−1−y))]k gi =m∑k=ig∘k(x∘)=m∑k=i[⊠dc−1(x−1−x)]k.

Then, the DE equation (6) can be written in an equivalent form as

 x=f(g(x);ε). (7)
###### Theorem 1

The functions and are increasing in .

###### Corollary 1

The density evolution for regular nonbinary LDPC codes given by (7) converges to a fixed point.

Successful decoding corresponds to convergence of the DE equation (7) to the fixed point .

For later use, we denote by the set of all possible values of . Likewise, we denote by / the set of all possible values of /. For nonbinary codes and for some ,

 E : 0≤ε≤1, X : 0≤xi≤pε,i,   Y: 0≤yi≤1,   1≤i≤m.

Vector functions and have several properties which will be useful for the proof of threshold saturation in Section III.

###### Lemma 1

Consider and defined above. For and ,

1. and are nonnegative vectors;

2. is differentiable in and is twice differentiable in ;

3. ;

4. , and it is invertible for ;

5. is strictly increasing with .

### Ii-B (dv,dc,m,L,w) SC-LDPC Code Ensemble over GF(2m)

Assume a ensemble over GF and transmission over the BEC(). In the form of (7), the fixed-point DE equations for the ensemble can be written as

 xi=1ww−1∑k=0f(yi−k;εi−k),    yi=1ww−1∑j=0g(xi+j−k),

where , and

 εi={ε,1≤i≤L0,1≤i−L

Collect all CCDF vectors and into the matrices and , respectively. Also, let be the matrix

The fixed-point DE equation for the ensemble can be written in matrix form, similarly as in [7],

 X=ATF(AG(X);ε),

where is an matrix, , is an matrix, and .

## Iii Potential Function and a Proof of Threshold Saturation

The DE equation (7) for the regular ensemble describes a vector admissible system for which we can properly define a potential function, similarly to [7].

###### Definition 2

The potential function of the system defined by functions and above, is given by

 U(x;ε)=g(x)DxT−G(x)−F(g(x);ε), (8)

where and are scalar functions that satisfy , , , and , for a symmetric matrix with positive elements and a non-zero determinant.

The definition of above is slightly more general than the one in [7], since is assumed to be symmetric with non-zero determinant, instead of being diagonal as in [7]. The properties of , and the calculation of and are addressed in Section IV.

###### Definition 3

For and , is a fixed point of the DE if ; is a stationary point of the potential function if .

Let the fixed point set be defined as

 F={(x;ε)|x=f(g(x);ε)}.
###### Lemma 2

For the vector system defined by and , the following assertions hold.

1. is a fixed point if and only if it is a stationary point of the potential ;

2. is strictly decreasing in , for and ;

3. is strictly decreasing in .

4. For some and such that , if and , then .

Thanks to the decreasing property of , we can now define the BP and the potential thresholds, denoted respectively by and .

###### Definition 4

The BP threshold is

 εBP=supε(ε∈E|U′(x;ε)>0, ∀x∈X).
###### Definition 5

The potential threshold is

 ε∗=supε(ε∈(εBP,1] | ΔE(ε)≥0),

where , and

In other words, is the lowest value of for which does not have a critical point, whereas is the lowest value of for which for all such that .

It has been shown for several systems [6], that the MAP threshold and the potential threshold are identical. We conjecture that the potential threshold of nonbinary LDPC codes is also identical to the MAP threshold.

###### Definition 6

The potential function for the spatially-coupled case is defined similarly as in [7]

 U(X;ε)=Tr(G(X)DXT)−G(X)−F(AG(X);ε), (9)

where , and .

The main result of our paper is stated below. It proves successful decoding for , i.e., the BP decoder saturates to the potential threshold for large enough values of .

###### Theorem 2

Consider the spatially-coupled LDPC code ensemble, and let be the upper bound on the norm for its corresponding potential function . Then, for and , the only fixed point of the system is .

## Iv Properties of D, and Calculation of F(y;ε) and G(x)

The existence of and is not guaranteed by the definition of . Here, we derive a condition on the existence of and and investigate how it depends on the form of the matrix . Without loss of generality, we consider the case of the ensemble as an example of a coupled vector system.

###### Theorem 3

Consider the ensemble and let be given by Definition 2. Then, and exist (hence exists) if there exist sets of values , and that satisfy

 ⎧⎨⎩isφ(i1,…,is,…,im)=∑mj=1djsϕ(j)(i1,…is−1,…,im)(ε)ktμ(k1,…,kt,…,km)=∑mj=1djtγ(j)(k1,…,kt−1,…km), (10)

for all possible -uples and and all and varying from to , where

 ϕ(j)(i1,…,im)(ε) =coeff(fj(x;ε),xi11⋯ximm), (11) γ(j)(k1,,…km) =coeff(gj(x),xi11⋯ximm). (12)

Theorem 3 can be extended to the coupled ensemble in a straightforward manner.

We now give a necessary condition on the existence of with diagonal . We use the following definition.

###### Definition 7

For a vector function , define the coefficient sets as

 Sfj ={(i1,…,im):coeff(fj(x),xi11⋯ximm)≠0}, (13)

for all .

###### Theorem 4

Assume a diagonal matrix . Then, the system of equations (10) exists if, for all from to

 (i1,…,im)∈Sfi⇔(i1,…,ii+1,…,ij−1,…,im)∈Sfj (i1,…,im)∈Sgi⇔(i1,…,ii+1,…,ij−1,…,im)∈Sgj

for some values of .

###### Proposition 1

For the nonbinary LDPC code ensemble, if is a diagonal matrix, the solution of (10) does not exist.

The consequence of Proposition 1 is that the potential function as defined in [7] does not exist for nonbinary LDPC codes. However, we can prove the following proposition.

###### Proposition 2

A positive symmetric matrix is sufficient for the existence of a solution of (10).

Thus, for nonbinary LDPC codes we can define the potential functions as in Definition 2 and 6, which are then used to prove threshold saturation.

## V Numerical Results

For the numerical results, we consider the coupled ensemble defined in [1], properly extended to the nonbinary case. In Table I we give the BP thresholds for several ensembles and and , denoted by , , , and , respectively, for . It is observed that the threshold improves with . In particular, a significant improvement is observed from (binary) to . It is interesting to note that approaches the Shannon limit as increases (the last column of the table gives the gap to the Shannon limit for the coupled ensembles with , ). We also observed for all values of that the BP threshold tends to the MAP threshold with increasing values of , suggesting threshold saturation to the MAP threshold. As an example, we report in the table the MAP threshold for .

In Fig. 1 we give bit error rate (BER) results for several nonbinary SC-LDPC codes with , and codeword length K bits. The code rate is , where is the rate loss due to finite . As a comparison, we also plot the performance for (binary code). In agreement with the DE results, the nonbinary SC-LDPC codes outperform their binary counterparts.

## Vi Conclusions

We proved threshold saturation for nonbinary SC-LDPC codes, when transmission takes place over the BEC, extending the proof in [7] to accommodate nonbinary SC-LDPC codes. We showed that nonbinary SC-LDPC codes achieve better BP threshold than their binary counterparts. Interestingly, the BP threshold approaches the Shannon limit with increasing values of , suggesting that capacity can be achieved with nonbinary SC-LDPC codes. Finite length performance results confirm that nonbinary SC-LDPC codes may perform better than binary codes for a given (binary) block length.

## References

• [1] S. Kudekar, T. J. Richardson, and R. L. Urbanke, “Threshold saturation via spatial coupling: Why convolutional LDPC ensembles perform so well over the BEC,” IEEE Trans. Inform. Theory, vol. 57, no. 2, pp. 803–834, Feb. 2011.
• [2] S. Moloudi, M. Lentmaier, and A. Graell i Amat, “Spatially coupled turbo codes,” 2014. [Online]. Available: http://arxiv.org/abs/1404.2393
• [3] A. Piemontese, A. Graell i Amat, and G. Colavolpe, “Nonbinary spatially-coupled LDPC codes on the binary erasure channel,” in Proc. IEEE Int. Conf. Commun., Jun. 2013.
• [4] L. Wei, K.-A. Toshiaki, D. G. M. Mitchell, T. E. Fuja, and D. J. Costello Jr., “Threshold analysis of non-binary spatially-coupled LDPC codes with windowed decoding,” in Proc. IEEE Int. Symp. Information Theory, 2014.
• [5] I. Andriyanova and A. Graell i Amat, “Threshold saturation for nonbinary SC-LDPC codes on the binary erasure channel,” 2013. [Online]. Available: http://arxiv.org/abs/1311.2003
• [6] A. Yedla, Y.-Y. Jian, P. S. Nguyen, and H. D. Pfister, “A simple proof of Maxwell saturation for coupled scalar recursions,” 2013, IEEE Trans. Inf. Theory, submitted for publication. [Online]. Available: http://arxiv.org/abs/1309.7910/
• [7] A. Yedla, Y.-Y. Jian, P. Nguyen, and H. Pfister, “A simple proof of threshold saturation for coupled vector recursions,” in Proc. Information Theory Workshop, Sep. 2012.
You are adding the first comment!
How to quickly get a good reply:
• Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
• Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
• Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
The feedback must be of minimum 40 characters and the title a minimum of 5 characters