# On Max-SINR Receiver for Hexagonal Multicarrier Transmission Over Doubly Dispersive Channel

###### Abstract

In this paper, a novel receiver for Hexagonal Multicarrier Transmission (HMT) system based on the maximizing Signal-to-Interference-plus-Noise Ratio (Max-SINR) criterion is proposed. Theoretical analysis shows that the prototype pulse of the proposed Max-SINR receiver should adapt to the root mean square (RMS) delay spread of the doubly dispersive (DD) channel with exponential power delay profile and U-shape Doppler spectrum. Simulation results show that the proposed Max-SINR receiver outperforms traditional projection scheme and obtains an approximation to the theoretical upper bound SINR performance within the full range of channel spread factor. Meanwhile, the SINR performance of the proposed prototype pulse is robust to the estimation error between the estimated value and the real value of time delay spread.

## I Introduction

Orthogonal frequency division multiplexing (OFDM) systems with guard-time interval or cyclic prefix can prevent inter-symbol interference (ISI). OFDM has overlapping spectra and rectangular impulse responses. Consequently, each OFDM sub-channel exhibits a sinc-shape frequency response. Therefore, the time variations of the channel during one OFDM symbol duration destroys the orthogonality of different subcarriers, and results in power leakage among subcarriers, known as inter-carrier interference (ICI), which causes degradation in system performance. In order to overcome the above drawbacks of OFDM system, several pulse-shaping OFDM systems were proposed [1, 2, 3, 4].

It is shown that signal transmission through a rectangular lattice is suboptimal for doubly dispersive (DD) channel [5, 6, 7]. By using results from sphere covering theory, the authors have demonstrated that lattice OFDM (LOFDM) system, which is OFDM system based on hexagonal-type lattice, providing better performance against ISI/ICI [5]. However, LOFDM confines the transmission pulses to an orthogonal set. As pointed out in [2], these orthogonalized pulses destroy the time-frequency (T-F) concentration of the initial pulses, hence lower the robustness to the time and frequency dispersion caused by the DD propagation channel.

In [6, 7, 8], the authors abandoned the orthogonality condition of the modulated pulses and proposed a multicarrier transmission scheme on hexagonal T-F lattice named as hexagonal multicarrier transmission (HMT). To optimally combat the impact of the DD propagation channels, the lattice parameters and the pulse shape of modulation waveform are jointly optimized to adapt to the channel scattering function from a minimum energy perturbation point of view. It is shown that the hexagonal multicarrier transmission systems obtain lower energy perturbation by incorporation the best T-F localized Gaussian pulses as the elementary modulation waveform, hence outperform OFDM and LOFDM systems from the robustness against channel dispersion point of view [9, 10, 11].

In HMT system, there is no cyclic prefix and the data symbols of HMT signal are transmitted at the hexagonal lattice point of T-F plane and subcarriers are interleaved in the T-F space, as is illustrated in Fig. 1. The basic mathematical operation on the received signal performed by the demodulator is a projection onto an identically structured function set generated by the prototype pulse function [10, 11, 12], i.e. an optimal match filter. It is shown in [15, 13, 14] that the optimum sampling time of wireless communication systems over DD channel depends on the power distribution of the channel profiles, and that zero timing offset does not always yield the best system performance. Max-SINR ISI/ICI-Shaping receiver for multicarrier modulation system is discussed in [2], the Max-SINR prototype pulse can be obtained by maximizing the generalized Rayleigh quotient.

We will present in this paper that receivers proposed in [6, 7, 8] are suboptimal approaches in the view of Signal-to-Interference-plus-Noise Ratio (SINR) performance. A novel receiver based on maximizing SINR (Max-SINR) criterion for HMT system is proposed in this paper. Theoretical analyses and simulation results show that the proposed optimal Max-SINR receiver outperforms the traditional projection receiver in [6, 7, 8] and obtains an approximation to the theoretical upper bound SINR performance. Meanwhile, the proposed scheme is robust to the estimation error between the estimated value and the real value of root mean square (RMS).

## Ii Hexagonal Multicarrier Transmission System

In the view of signal transmission on lattice in the T-F plane, the system performance is mainly determined by two factors: a) energy concentration of the elementary modulation pulse, a better T-F concentrated pulse would lead to more robustness against the energy leakage and b) distance between the transmitted symbols in the T-F plane, it is obvious that the larger the distance, the less the perturbation among the transmitted symbols. As pointed out in [6, 7, 8], for a given signaling efficiency, the information-bearing pulses arranged on a hexagonal T-F lattice, as is illustrated in Fig. 1, can be separated as sufficiently as possible in the T-F plane.

In HMT systems, the transmitted baseband signal can be expressed as [6]

(1) |

where and are the lattice parameters, which can be viewed as the symbol period and the subcarrier separation, respectively; is the user data, which is assumed to be taken from a specific signal constellation and independent and identically distributed (i.i.d.) with zero mean and average power ; and are the position index in the T-F plane; and denote the sets from which can be taken, with cardinalities M and N, respectively. The prototype pulse is the Gaussian window

(2) |

with being a parameter controlling the energy distribution in the time and frequency directions. The ambiguity function of Gaussian pulse is defined by

(3) |

where denotes the complex conjugate.

It is shown in Fig. 1 that the original hexagonal lattice can be expressed as the disjoint union of a rectangular sublattice and its coset . The transmitted baseband signal in (1) can be rewritten as

(4) |

where =, and represent the symbols coming from two sets and , respectively. is the transmitted pulse on -th symbol and -th subcarrier.

The baseband DD channel can be modeled as a random linear operator H [16]

(5) |

where and are the maximum multipath delay spread and the maximum Doppler frequency, respectively[17]. is called the delay-Doppler spread function, which is the Fourier transform of the time-varying channel impulse response with respect to . The product is referred to as the channel spread factor (CSF). If , the channel is said to be underspread; otherwise, it is overspread. Practical wireless channels usually satisfy the assumption of wide-sense stationary uncorrelated scattering (WSSUS), and [16].

In the WSSUS assumption the channel is characterized by the second order statistics

(6) |

where denotes the expectation and is called the scattering function, which characterizes the statistics of the WSSUS channel. Without loss of generality, we use .

As shown in (5), the propagation channel introduces energy
perturbation among the transmitted symbols. It is shown in
[5, 6, 7] that the symbol energy perturbation
function is dependent on the channel scattering function and the
pulse shape. The properly designed HMT system with Gaussian
prototype pulse achieves minimum symbol energy perturbation over DD
channel. The optimality of such system in combating the ISI/ICI
caused by the DD channel is guaranteed by the Heisenberg uncertainty
principle and the sphere-packing theory. The choice of the system
parameters to minimize such undesired energy
perturbation is extensively dealt with in [5, 6, 7].
In general, the pulse shape of the Gaussian window and the
transmission pattern parameters and should be matched to the
channel scattering function. The optimal system parameter for DD
channels with exponential-U scattering function can be chosen as
[6]^{1}^{1}1In the case of no time and
frequency dispersion,
, the matching
criteriaes (7) and (8) are also reduced to
and ,
respectively. and are the centralized temporal and
spectral second-order moments, respectively [17].

(7) |

and

(8) |

where is the RMS delay of the DD channel and the coefficient for various signaling efficiency is listed in Table I.

0.5 | 1.0 | 2.0 | 4.0 | |

2.25 | 2.00 | 1.90 | 1.85 |

The received signal can be expressed as

(9) |

where is the AWGN with variance .

## Iii Max-SINR Receiver for HMT System

The basic mathematical operation of the received signal performed by the demodulator is a projection onto an identically structured function set generated by the prototype pulse function, i.e. an optimal match filter[7, 11]. To obtain the data symbol , the match filter receiver projects the received signal on the prototype pulse function set =, i.e.,

(10) |

where =, and is the prototype pulse at the receiver. The energy of the received symbol, after projection on the prototype pulse set can be expressed as

(11) |

Under the assumptions of WSSUS channel and source symbols are statistically independent, (11) can be rewritten as

(12) |

The SINR of received signal can be expressed as

(13) |

where the interference-plus-noise energy

(14) |

Clearly, the interference-plus-noise energy function is dependent on the channel scattering function and the pulse shape (through its ambiguity function). According to the form of channel scattering functions, the Max-SINR receiver can be discussed in two cases [6]: Case A: DD channel with exponential power delay profile and U-shape Doppler spectrum. Case B: DD channel with uniform power delay profile and uniform Doppler spectrum. Due to the space limitations, this paper only to discuss the Case A, that is the Max-SINR receiver for HMT system over DD channel with exponential power delay profile and U-shape Doppler spectrum.

For the DD channel with exponential power delay profile and U-shape Doppler spectrum, the scattering function can be expressed as [18]

(15) |

with . We assume that ^{2}^{2}2It is shown in [15, 13, 14] that the optimum
sampling time of wireless communication systems over DD channel
depends on the power distribution of the channel profiles, and that
zero timing offset does not always yield the best system
performance. In other words, there is a timing offset between the
prototype pulses at the transmitter and the receiver. In
[2], the Max-SINR prototype pulse g of
multicarrier transmission system with rectangular T-F lattice over
DD channel is obtained by maximizing the generalized Rayleigh
quotient
.
The solution is the generalized eigenvector of the matrix pair
corresponding to the
largest generalized eigenvalue. It is shown that there is a delay
between the transmitted Gaussian prototype pulse and the received
Gaussian prototype pulse. In this paper, the close form time offset
expressions between the transmitted and received prototype pulse is
derived. -, the theoretical SINR of the
received signal over the DD channel with exponential power delay
profile and U-shape Doppler spectrum can be expressed as

(16) |

Substituting (15) into (14), the interference-plus-noise energy function in (16) can be expressed as

(17) |

The theoretical SINR upper bound of the received signal can be expressed as

(18) |

Plugging (16) and (17) in (18), the Max-SINR prototype pulse can be expressed as and (see Appendix)

(19) |

We can see from equation (19) that the prototype pulse of the proposed Max-SINR receiver is a function of RMS. In HMT system, the prototype pulse shape of the Gaussian window and the transmission pattern parameters and should be matched to the channel scattering function including RMS. In other word, the RMS delay spread and the maximum Doppler frequency are known at the HMT transceiver. Hence, it is reasonable to assume that the RMS delay spread priori known in HMT system.

## Iv Simulation and Discussion

In this section, we test the proposed Max-SINR receiver via computer simulations based on the discrete signal model. In the following simulations, the number of subcarriers for HMT system is chosen to =, and the length of prototype pulse =. The center carrier frequency is =GHz and the sampling interval is set to =s. The system parameters of HMT system are =kHz, = and is set to =. Traditional projection receiver proposed in [6, 7, 8] is named as Traditional Projection Receiver (TPR) in the following simulation results. WSSUS channel is choosen as DD channel with exponential power delay profile and U-shape Doppler spectrum.

The prototype pulses of Max-SINR receiver at = and = are given in Fig. 2. The prototype pulse of TPR scheme is also depicted for comparison. We can see from Fig. 2 that there is a delay between the TPR prototype pulse and the Max-SINR prototype pulse. The delay is obtained according to equation (19), and increases with the increasing of CSF .

The SINR performance of different receivers with the variety of for HMT system over DD channel is depicted in Fig. 3. The CSFs are set to =0.07 and =0.2, respectively. We can see from Fig. 3 that the SINR performance of the proposed Max-SINR receiver outperforms TPR scheme about 1 at =0.07 and 1.5 at =, respectively. The SINR gap between the proposed Max-SINR receiver and the theoretical SINR upper bound is smaller than 0.5dB and 0.1dB at =, respectively.

The SINR performance of the proposed Max-SINR receiver with the variety of is given in Fig. 4. We assume that there is an estimation error between the estimated value of RMS delay spread and the real value . We can conclude from Fig. 4 that the proposed Max-SINR receiver is robust to the estimation error of RMS delay spread.

The SINR performance with the variety of channel spread factors at =20dB is depicted in Fig. 5. The SINR performance of the TPR scheme is depicted for comparison. It can be seen that there is a degradation of SINR with the increasing of channel spread factor. The proposed Max-SINR receiver obtains an approximation to the theoretical upper bound SINR performance within the full range of . There is an about 2.5dB maximum SINR gap between the Max-SINR receiver and the TPR scheme at =0.35, and the SINR gap decreases as the CSF decreases.

## V Conclusion

The Max-SINR receiver is proposed for HMT system over DD channel in this paper. After a detailed analysis, we present that the prototype pulse of the Max-SINR receiver should adapt to the RMS delay spread. Theoretical analysis shows that the proposed Max-SINR receiver outperforms the TPR scheme in SINR and obtains an approximation to the theoretical upper bound SINR performance within the full range of CSF. Simulation results show that the proposed scheme is robust to the estimation error of RMS and is suitable for the actual HMT system.

## Appendix A Proof of Equation(19)

Under the assumption that the prototype pulse at the receiver is , the SINR of the received signal can be expressed as

(20) |

where in (20) can be rewritten as

(21) |

where is the complementary error function. If , we may obtain an approximate solution of the complementary error function by [19]

(22) |

The Max-SINR receiver can be obtained by solving the gradient of with respect to

(23) |

where and can be expressed as

(24) |

hence, equation (23) can be rewritten as

(25) |

Equation (25) can be simplified to a quadratic equation. Under the constraint of , the solution of the quadratic equation can be expressed as

(26) |

## Acknowledgment

This work was supported by the National Natural Science Foundation of China (No. 60972050) and Major Special Project of China under Grant (2010ZX03003-003-01) and the Jiangsu Province National Science Foundation under Grant (BK2011002) and the Young Scientists Pre-research Fund of PLAUST under Grant (No. KYTYZLXY1211).

## References

- [1] Kumbasar V., Kucur O. ICI reduction in OFDM systems by using improved sinc power pulse. Digital Signal Processing, vol.17, no.6, pp.997-1006, Nov. 2007.
- [2] Das S., Schniter P. Max-SINR ISI/ICI-Shaping multicarrier communication over the doubly dispersive channel. IEEE Transactions on Signal Processing, vol.55, no.12, pp.5782-5795, Dec. 2007.
- [3] Abbas H.K., Waleed A. M., Nihad S., The performance of multiwavelets based OFDM system under different channel conditions. Digital Signal Processing, vol.20, no.2, pp.472-482, Mar. 2010.
- [4] Gao X. Q., Wang W., Xia X. G., et al. Cyclic prefixed OQAM-OFDM and its application to single-carrier FDMA. IEEE Transactions on Communications, vol.59, no.5, pp.1467-1480, May 2011.
- [5] Strohmer T, Beaver S. Optimal OFDM design for time-frequency dispersive channels. IEEE Trans. Commun. vol.51, no.7, pp.1111-1122, Jul. 2003.
- [6] Han F. M., Zhang X. D. Hexagonal multicarrier modulation: A robust transmission scheme for time-frequency dispersive channels. IEEE Transactions on Signal Processing, vol.55, no.5, pp.1955-1961, May 2007.
- [7] Han F. M., Zhang X. D. , MLSD for hexagonal multicarrier transmission with time-frequency localized pulses, IEEE Transactions on Vehicular Technology, vol.58, no.3, pp.1598-1604, Mar. 2009.
- [8] Han F. M., Zhang X.D. Asymptotic Equivalence of Two Multicarrier Transmission Schemes in Terms of Robustness Against Time¨CFrequency Dispersive Channels. IEEE Transactions on Vehicular Technology,vol.59, no.2, pp.1598-1604, Feb. 2010.
- [9] Xu K., Shen Y. H. Effects of carrier frequency offset, timing offset, and channel spread factor on the performance of hexagonal multicarrier modulation systems. EURASIP Journal on Wireless Communications and Networking, vol.2009, pp.1-8, Jan. 2009.
- [10] Xu K., Xu Y., Zhang D. SINR analysis of hexagonal multicarrier transmission systems in the presence of insufficient synchronization for doubly dispersive channel. Frequenz, vol.65, no.5, pp.149-157, Aug. 2011.
- [11] Xu K., Lv Z., Xu Y., Zhang D. Max-SINR Based Timing Synchronization Scheme in Hexagonal Multicarrier Transmission. Wireless Personal Communications, Jan. 2012, DOI: 10.1007/s11277-012-0550-5.
- [12] Jung P., Wunder G. The WSSUS pulse design problem in multicarrier transmission. IEEE Transactions on Communications, vol.55, no.10, pp.1918-1928, Oct. 2007.
- [13] Jingxian Wu, Chengshan Xiao, Performance Analysis of Wireless Systems With Doubly Selective Rayleigh Fading, IEEE Transactions on Vehicular Technology, vol.56, no.2, pp.721-730, Mar. 2007.
- [14] Jingxian Wu, Yahong Rosa Zheng, Khaled Ben Letaief, Chengshan Xiao, On the Error Performance of Wireless Systems with Frequency Selective Fading and Receiver Timing Phase Offset, IEEE Transactions on Wireless Communications, vol.6, no.2, pp.720-729, Feb. 2007.
- [15] Jingxian Wu, Yahong Rosa Zheng, Khaled Ben Letaief, Chengshan Xiao, Matched Filter Bound of Wireless Systems over Frequency Selective Channels with Receiver Timing Phase Offset. in Proc. IEEE Global Telecommun. Conf. GLOBECOM¡¯05, pp.3758-3762, Dec. 2005.
- [16] Bello P. A.,Characterization of randomly time-variant linear channels, IEEE Transactions on Communication System, vol.11, no.4, pp.360-393, Dec. 1963.
- [17] L. Cohen, Time-frequency analysis. Englewood Cliffs, NJ: Prentice-Hall, 1995.
- [18] P. Matthias, Mobile fading channels. West Sussex, England: John Wiley & Sons, Ltd, 2002.
- [19] Kingsbury, N. Approximation Formulae for the Gaussian Error Integral, Q(x), Connexions Web site. http://cnx.org/content/m11067/2.4/, Jun 7, 2005.