# Adaptive Switched Lattice Reduction-Aided Linear Detection Techniques for MIMO Systems

## Abstract

Lattice reduction (LR) aided multiple-input-multiple-out (MIMO) linear detection can achieve the maximum receive diversity of the maximum likelihood detection (MLD). By emloying the most commonly used Lenstra, Lenstra, and L. Lov´asz (LLL) algorithm, an equivalent channel matrix which is shorter and nearly orthogonal is obtained. And thus the noise enhancement is greatly reduced by employing the LR-aided detection. One problem is that the LLL algorithm can not guarantee to find the optimal basis. The optimal lattice basis can be found by the Korkin and Zolotarev (KZ) reduction. However, the KZ reduction is infeasible in practice due to its high complexity. In this paper, a simple algorithm is proposed based on the complex LLL (CLLL) algorithm to approach the optimal performance while maintaining a reasonable complexity.

## 1 Introduction

MIMO systems are fundamental for the next generation wireless networks for their great potential in improving the system capacity and performance. Two key factors that are responsible for the advantages of MIMO system are how to design the optimized signal transmission form at the transmit side and the appropriate signal detection at the receive side. The nonlinear maximum likelihood detection (MLD) can guarantee the best bit error rate (BER) performance. However, the MLD is usually impractical due to its complexity that grows exponentially with the number of constellation points and the number of transmitted streams. The linear detectors such as zero-forcing (ZF) and minimum mean square error (MMSE) receivers have the lower complexity while the simplification is the cost of sacrificing BER performance [2].

Recently, the lattice reduction (LR) aided detection attracted significant research efforts for it can achieve near-optimal performance with very low complexity. The LR algorithm in conjunction with MIMO detection techniques was first considered by Yao and Wornell [5]. From the simulations of [5], the symbol error rate (SER) curves can parallel those of the MLD algorithm by using LR-aided detection schemes. It was proved in [3], [4] that the LR-aided MIMO linear receivers can achieve the maximum diversity as the MLD. Hence, a great deal of interest has been devoted to exploring the application of LR in MIMO systems. The LR-aided detection schemes with respect to the MMSE criterion have been extended by Wuebben et al [6]. In [7], not only the LR-aided SU-MIMO detection but also the LR-aided SU-MIMO precoding has been investigated. LR-aided MIMO precoding for decentralized receivers was discussed in [8]. The quantitative error-rate analysis of LR aided detection was given in [12].

The optimal lattice basis can be found by Korkin and Zolotarev (KZ) reduction [13]. KZ reduciton is an exponential-time lattice reduction algorithm, hence, it is infeasible in practice due to its high complexity [11], [14]. The most commonly used LR reduction is the polynomial-time LLL algorithm which was first proposed by Lenstra, Lenstra, and L. Lov´asz in [9]. The LLL algorithm guarantees to find a lattice basis within a factor to the optimal one in polynomial time [4], [11]. The essence of the LLL algorithm is try to orthogonalize the columns of the channel matrix and reduce its size as well. The Gram-Schmidt orthogonalization (GSO) procedure and size reduction are the two core components of the LLL algorithm. Through the LLL algorithm, only a real value-based matrix can be processed which may lead to extra unnecessary complexity when the channel has large dimensions. In order to reduce the computational complexity further, the complex LLL (CLLL) algorithm was proposed in [10]. The overall complexity of CLLL algorithm is nearly half of the LLL algorithm without sacrificing any performance.

In this paper, we will employ the CLLL algorithm to implement the LR-aided detection techniques for the MIMO system. As mentioned earlier, the CLLL algorithm can not guarantee to find the optimal lattice basis, hence, there will be a room left for us to improve the performance of CLLL algorithm further and maintain a low complexity at the same time. This is the motivation of this paper, and then the switched scheme is developed and proposed to improve the performance of LR-aided detection techniques.

This paper is organized as follows. In Section II the system model and brief review of the algorithms are given. In Section III the proposed switched LR-aided MIMO detection algorithms are described in detail. Simulation results and conclusions are presented in Section IV and Section V.

Notation: Matrices and vectors are denoted by upper and lowercase boldface letters, and the transpose, inverse, pseudoinverse of a matrix by , , , respectively. The and prefixes denote the real and imaginary parts. rounds to a closest integer, while to the closet integer smaller than or equal to .

## 2 Preliminaries

### 2.1 System Model

In this paper, we assume an uncoded spatial multiplexing (SM) MIMO broadcast channel, where transmit antennas are employed at the base station (BS) and receive antennas are equipped at the user terminal (UT). Actually, MIMO techniques can be combined with orthogonal frequency division multiplexing (OFDM) technique. By decomposing the channel into multiple orthogonal sub-channels, the OFDM techniques can successfully transform frequency selective fading channels into flat fading channels. Hence, we assume a flat fading MIMO channel in this paper. The system model is illustrated in Fig.1, and the received signal is given by:

(1) |

where is the complex Gaussian channel matrix with zero mean and unit variance. We assume is a full-rank MIMO channel, i.e., consists of ( ) linearly independent row vectors. is the transmitted data vector, and is the Gaussian noise with i.i.d. entries of zero mean and variance .

### 2.2 Complex Lattice Reduction Algorithm

A complex lattice is a set of points , where contains the bases of the lattice . Actually, any matrix can generate the same lattice as if and only if , where is a unimodular matrix () and all elements of are Gaussian integers, i.e. .

The aim of the LR algorithm is to find a new basis which is shorter and nearly orthogonal compared with the original matrix . Given the QR decomposition of , , where is an orthogonal matrix with unit length () and the upper-triangular matrix is a rotated and reflected representation of . Thus, each column vector of is given by [6]

(2) |

where is the columns of . If are close to zero, we can say that is almost orthogonal to the space spanned by . Similarly the QR decomposition of is . Then, the basis for is CLLL-reduced if both of the following conditions are satisfied:

(3) |

(4) |

where influences the quality of the reduced basis and the computational complexity. We usually choose to achieve a trade-off between good performance and complexity [9]. The main steps of the CLLL reduction algorithm are described clearly in [10].

Obviously, is not the unique basis for the lattice . The lattice can have infinitely many different bases other than . For any unimodular matrix which satisfy and , there will be a corresponding basis . Among all the bases, the optimal one can be found by KZ reduction. From [11], an arbitrary matrix is KZ-reduced if and only if its upper-triangular representation is KZ-reduced. For convenient, we study the upper-triangular matrix of ,

(5) |

the KZ reduction is recursively defined if each of the following three conditions holds:

(6) |

(7) |

(8) |

Any KZ-reduced matrix is clearly also CLLL-reduced. The optimal basis can be obtained by using KZ reduction, however, finding a KZ reduced basis would be too time-consuming and infeasible in practice. Considering the computational complexity, it is reasonable to employ the low complexity CLLL reduction algorithm to implement the lattice reduction. Consequently, there will be a performance gap between the CLLL and KZ algorithms. Regarding the performance gap, strategies are developed in section III to approach the optimal performance and offer a trade-off between performance and complexity.

### 2.3 CLR Aided MIMO Detection

The LR-reduced channel matrix has better channel quality compared to the original channel matrix . Therefore, if the MIMO receivers were designed based on , a better detector performance can be achieved due to less noise enhancement increased by . The LR aided MIMO detection structure is illustrated in Fig.2,

In Fig.2, is the transmit M-QAM signal. The set of M-QAM constellation is given by with representing the modulation index. The parameter is used for normalizing the power of the transmit signals to 1. From Fig.2, the received signal is,

(9) |

where is the CLLL-reduced channel matrix and is the equivalent transmit signal. Similarly to the conventional MIMO detector, there are several CLR-aided MIMO detection techniques:

1) CLR-aided ZF Detector

ZF detection strategy is the simplest one, obviously, if we want to cancel out the impact of the fading channel at the receiver side, we can set the receiver filter as the pseudoinverse of [2],

(10) |

And then, the received signal is .

Since is a unimodular matix, the statistical properties of are identical to those of . Therefore, the estimation error covariance matrix of ZF detection is,

(11) |

From , it is clear that increased the noise interference on the signal, and thus the BER performance of the ZF detector will be degraded by . Since is based on the CLR-reduced channel , it is more likely to be well conditioned compared to which is based on , hence the effect of noise enhancement will be moderated by .

2) CLR-Aided MMSE Detector

As an improvement to reduce the effects of noise amplification caused by the ZF filter, the MMSE filter taking the noise term into account:

(12) |

As shown in [15], the MMSE detection is equal to ZF with respect to an extended system model. The extended channel matrix and the extended received signal is given by,

(13) |

From (14), it is the condition of that determines the noise amplification not the condition of in the CLR-aided MMSE detection case. We can compute according to (15), however, the BER performance will be discounted for this mismatching. A better performance can be obtained by performing the CLR for the extended channel matrix , i.e. , and by computing

(14) |

then, the received signal is .

3) CLR-Aided SIC Detector

The filers employed in Successive Interference Cancellation (SIC) detection are mainly based on the linear filters which were discussed above. The key idea of SIC detection is layer peeling, that is, the first symbol is decoded first, and then cancelling the decoded symbol in the next layer peeling, repeat this manipulation layer by layer until all the symbols are decoded from the received signal. By layer peeling, the interference caused by the already detected symbols is canceled. The SIC detection can be equivalently implemented by a QR decomposition, , and then computes . The detection steps is summarized bellowed,

(15) | |||

(16) |

where . And finally, .

After the receive filter, the estimated transmit signal can be obtained by . However, the elements of are far from integers and serious performance loss will be caused if was quantized directly. In order to avoid the quantization error caused by rounding as much as possible, proper shifting and scaling work should be done before multiplying the received signal by . The estimation of is,

(17) |

where . Finally the estimated transmit signal can be easily obtained by .

## 3 Proposed Switched LR-Aided Detection

The proposed algorithm is motivated by the fact that the CLLL-reduced channel matrix is neither the optimal nor the unique basis in the lattice space. Therefore, we can build more LR-reduced candidate matrices and choose the best one during each detection stage. Observing the formulation , it is not difficult to find that there are two ways if we want to get more LR-reduced candidates.

The first way is achieved by constructing different satisfied matrix. As long as always meet two requirements, (1) , (2) all elements of are Gaussian integers. The physical meaning of the first requirement is that we cannot change the transmit power through the LR transformation. Under the first limitation, we need to construct the different matrices by the four units 1, -1, j and -j.

The second way to get more candidates can be achieved by changing , that is, we can randomly swap columns of channel matrix at the receiver side before the LR transformation. Actually, interchanging the columns of is equivalent to performing a linear transformation of .

Comparing these two ways, it is not easy to construct different satisfied matrices by the four units, while, the second way is more convenient to implement. Hence, we focus on the second scheme to generate more candidates of . The art of switched techniques are also discussed in [19].

### 3.1 The System Structure of the Proposed Algorithm

The LR-reduced candidate channel matrices are roughly orthogonal, consequently, we need to develop a metric to measure the orthogonality and choose the best one among them. The system model of the proposed randomly switched CLR-aided detection algorithm is illustrated in Fig.3.

From Fig.3 denotes the randomly permuting matrix which exclude the original order, the corresponding LR-reduced channel matrix is given by,

(18) |

where is the candidate, and is the total number of candidates. Considering the complexity of the algorithm, the number of candidate matrices is limited to no more than 10 when the channel matrix has high dimensions, that is, .

### 3.2 The Selection Criterion

Obviously, designing powerful selection criterion is crucial to the proposed algorithm. From matrix theory [23], the condition number (CN) can be used as the indication of the orthogonality of the reduced matrix. The CN is defined as,

(19) |

where and is the largest and smallest singular value of . For the orthogonal channel matrix the value of CN is 1, there will be no noise amplification effect during the detection. The matrix with low CN is well-conditioned, while the matrix with a high CN is ill-conditioned. In [16] the CN is used to study the impact of channel matrix on the different detection techniques. It is showed by the simulation results that all the detection schemes can achieve very good performance for . However, CN mainly reflects the worst situation of the channel matrix and thus it is only a rough measurement. Therefore, CN is not a reliable selection criterion due to the fact that CN cannot fully measure the channel performance.

Luckily, we found a stable and reliable metric to serve as the selection criterion called orthogonality defect factor (ODF) which is defined in [3] as,

(20) |

where s are the columns of the basis . Clearly, with equality of an orthogonal basis. Therefore, for candidates and , we can say is better reduced than if . By using the ODF selection metric, we can choose the best candidate from , and then the received signal can be rewritten as,

(21) |

consequently, the receive filter is calculated based on as well. Since has better channel quality, the BER performance will be improved by the proposed algorithm.

The complexity of the proposed algorithm is determined by the total candidate number . The different trade-offs between performance and complexity can be achieved by altering depending on specific situation. For example, we can set , and compare the orthogonality between the candidate and the original channel matrix. Then, choose the better one to compute the detection filter. Hence, the performance will be always better or equal to the conventional CLR-aided detection.

A graceful trade-off between BER performance and computational complexity offered by the proposed algorithm which is summarized in table I. The simulation results are given in the next section.

## 4 Simulation Results

In this section, the linear MIMO detection techniques are given and compared with the proposed algorithms. A system with transmit antennas and receive antennas is considered. We assume an uncorrelated block fading channel, that is, the channel is static during each transmit packet and there is no correlation among the antennas. The number of trials used to average the curves is 1000 and the packet length is 100 symbols for the simulations. The SNR is defined as , and the is defined as with is the number of information bits transmitted per channel symbol .

Fig.4 showed the BER performance of the proposed randomly switched CLR-aided schemes with ZF detection. From Fig.4, we can find that the ZF detection has the worst BER performance compared to the other techniques since the high noise amplification without LR. The BER performance of CLR-aided ZF detection is better than the conventional ZF detection while worse than the proposed algorithm. In addition, the performance of the proposed scheme gradually improved as the candidates number increases. It is clear that the proposed scheme have better performance compared to ZF detection and LR-aided ZF detection. At the BER of , the proposed scheme with has more than 4.5dB gains compared to the CLR-aided ZF detection. It is worth noting that even with , the proposed scheme has almost 2dB gains over the CLR-aided ZF linear detection.

Fig.5 illustrated the MMSE,MMSE-SIC and their corresponding proposed algorithms respectively. The MMSE based detection algorithms have better BER performance compared to their existing ZF counterparts in Fig.4. At 25dB, the BER of conventional MMSE and CLR-aided extended MMSE detection are all higher than . While, the BER of the proposed scheme achieved very good performance at 25dB, they are all below and even approached with . Since there is a interference cancellation during each SIC detection layer, the improved performance of the proposed randomly switched CLR-aided MMSE-SIC is quite limited compared with the randomly switched technique with linear detection. However, the proposed algorithm still offer almost 2dB gain with at compared to CLR-aided MMSE-SIC.

Fig.6 illustrated the BER performance with 16-QAM modulation. Obviously, the BER performance of all corresponding algorithms are becoming worse compared to the QPSK modulation scenario due to the smaller signal point distance. Similarly, the proposed schemes offered better performance in the 16-QAM modulation.

These simulation results support the analysis that the LR algorithms based on LLL can not guarantee an optimal basis, however, we can approach the optimal performance by using the proposed scheme. Actually, if the equivalent channel is strictly orthogonal between each other, the linear detection is identical to MLD [6].

## 5 conclusion

In this paper, the randomly switched technique was proposed to improve the performance of LR-aided MIMO detection, which gracefully offered a tradeoff between the complexity and performance. Simulation results evidence that our proposed algorithms have substantial performance gains compared to the existing MIMO linear and LR-aided linear detection. As for nonlinear SIC detection, the gain improved by the proposed algorithm is limited. How to reduce the gap between the proposed algorithm based on SIC detection and the ML or SD detection need further study.

### References

- A. Paulraj, R. Nabar, and D. Gore, Introduction to Space-Time Wireless Communications. Cambridge University Press, 2003.
- M. Taherzadeh, A. Mobasher and A. K. Khandani, “LLL Reduction Achieves the Receiver Diversity in MIMO Decoding,” IEEE Trans. Inf. Theory, vol. 53, no. 12, pp. 4801–4805, Dec. 2007.
- X. Ma and W. Zhang, “Performance Analysis for MIMO Systems with Lattice-Reduction Aided Linear Equalization,” IEEE Trans. Commun.,vol. 56, no. 2, pp. 309–318, Feb. 2008.
- H. Yao and G. Wornell, ”Lattice-Reduction-Aided Detectors for MIMO Communication Systems,” in IEEE Proc. Globecom, Taipei, Taiwan, November 2002.
- D. Wübben, R. Böhnke, V. Kühn, and K.-D. Kammeyer, ”Near-maximum-likelihood detection of MIMO systems using MMSE-based lattice-reduction,” in Proc. 2004 Int. Conf. Commun. (ICC’04), June 2004, pp. 798-802.
- C. Windpassinger and R. Fischer, ”Low-complexity near-maximum likelihood detection and precoding for MIMO systems using lattice reduction,” in Proc. IEEE Inf. Theory Workshop, Paris, France, March 2003, pp. 345-348.
- R. F.H. Fischer, C. A. Windpassinger, ”Improved MIMO precoding for decentralized receivers resembling concepts from lattice reduction,” in Proc. of IEEE Global Telecommunications Conf., San Francisco, CA, USA, December 2003, pp. 1852 - 1856.
- A. K. Lenstra, H. W. Lenstra, and L. Lov´asz, ”Factoring Polynomials with Rational Coefficients,” Math. Ann, vol. 261, pp. 515-534, December 1982.
- Y. H. Gan, C. Ling, and W. H. Mow, ”Complex lattice reduction algorithm for low-complexity full-diversity MIMO detection,” IEEE Transactions on Signal Processing, vol. 57, no. 7, pp. 2701 - 2710, July 2009.
- E. Agrell, T. Eriksson, A. Vardy, and K. Zeger, ”Closest point search in lattices,” IEEE Trans. Inf. Theory, vol. 48, no. 8, pp. 2201–2214, Aug. 2002.
- C. Ling, ”On the Proximity Factors of Lattice Reducition-Aided Decoding,” IEEE Transactions on Signal Processing, vol. PP, no. 99, pp. 1–14, March 2011.
- A. Korkine and G. Zolotareff, “Sur les formes quadratiques,” Math. Annalen, vol. 6, pp. 366–389, 1873.
- C. P. Schnorr and M. Euchner, “Lattice basis reduction: improved practical algorithms and solving subset sum problems,” Math Programming, vol. 66, pp. 181–191, 1994.
- D. Wübben, R. Böhnke, V. Kühn, and K.-D. Kammeyer, ”MMSE Extension of V-BLAST bascd on Sorted QR Decomposition,” in IEEE Proc. Vehicular Technology Conference, Orlando, Florida, USA, Octorber 2003.
- D. Wübben, R. Böhnke, V. Kühn, and K.-D. Kammeyer, ”MMSE-Based Lattice-Reduction for Near-ML Detection of MIMO Systems,” in 2004 ITG Workshop on Smart Antennas.
- R. C. de Lamare, R. Sampaio-Neto, ”Adaptive MBER decision feedback multiuser receivers in frequency selective fading channels”, IEEE Commun. Lett., vol. 7, no. 2, pp. 73-75, 2003.
- L. Song, R. C. de Lamare, A. G. Burr, “Successive interference cancellation schemes for time-reversal space-time block codes”, IEEE Transactions on Vehicular Technology, vol. 57, vol. 1, pp. 642-648, 2008.
- R. C. de Lamare and R. Sampaio-Neto, ”Minimum Mean Squared Error Iterative Successive Parallel Arbitrated Decision Feedback Detectors for DS-CDMA Systems,” IEEE Transactions on Communications, May, 2008.
- R.C. de Lamare and R. Sampaio-Neto, “Adaptive Reduced-Rank Equalization Algorithms Based on Alternating Optimization Design Techniques for MIMO Systems,” IEEE Trans. Vehicular Technology, vol. 60, no. 6, pp.2482-2494, July 2011.
- P. Li, R. C. de Lamare and R. Fa, “Multiple Feedback Successive Interference Cancellation Detection for Multiuser MIMO Systems,” IEEE Transactions on Wireless Communications, vol. 10, no. 8, pp. 2434-2439, August 2011.
- P. Li and R. C. de Lamare, “Adaptive Decision Feedback Detection with Constellation Constraints for MIMO Systems”, IEEE Transactions on Vehicular Technology, 2012.
- G. Golub, C. Van Loan, Matrix Computaitons. The Johns Hopkins University Press, 1996.