An Augmented Nonlinear Complex LMS for Digital Self-Interference Cancellation in Full-Duplex Direct-Conversion Transceivers

An Augmented Nonlinear Complex LMS for Digital Self-Interference Cancellation in Full-Duplex Direct-Conversion Transceivers


In future full-duplex communications, the cancellation of self-interference (SI) arising from hardware nonidealities, will play an important role in the design of mobile-scale devices. To this end, we introduce an optimal digital SI cancellation solution for shared-antenna-based direct-conversion transceivers (DCTs). To establish that the underlying widely linear signal model is not adequate for strong transmit signals, the impacts of various circuit imperfections, including power amplifier (PA) distortion, frequency-dependent I/Q imbalances, quantization noise and thermal noise, on the performance of the conventional augmented complex least mean square (ACLMS) based SI canceller, are analyzed. In order to achieve a sufficient signal-to-noise-plus-interference ratio (SNIR) when the nonlinear SI components are not negligible, we propose an augmented nonlinear CLMS (ANCLMS) based SI canceller for a joint cancellation of both the linear and nonlinear SI components by virtue of a widely-nonlinear model fit. A rigorous mean and mean square performance evaluation is conducted to justify the performance advantages of the proposed scheme over the conventional ACLMS solution. Simulations on orthogonal frequency division multiplexing (OFDM)-based wireless local area network (WLAN) standard compliant waveforms support the analysis.


Full-duplex, I/Q imbalance, self-interference, augmented complex LMS (ACLMS), augmented nonlinear complex LMS (ANCLMS), mean and mean square analysis


1 Introduction


The full-duplex (FD) technology aims at doubling the radio link data rate through simultaneous and bidirectional communication at the same center frequency, and is widely considered as a driving-force for more spectrally efficient wireless networks and a potential candidate to fulfill 5G’s ambition of a 1000-fold gain in capacity [1, 2]. One of the major challenges in FD communications is the so-called self-interference (SI) problem, a strong transmit signal coupled into the receiver (Rx) path. Since the transmitter (Tx) and Rx chains are closely linked together in each transceiver node of FD communication systems, the SI power leaked into and reflected from the Tx chain could be even 50 dB to 110 dB higher than the Rx sensitivity level in either wireless local area network (WLAN) or cellular scenarios [3, 4, 5]. The design of FD transceivers has long been considered impossible for practical realizations and implementations, and it is only recently that its feasibility was experimentally demonstrated using the wireless open-access research platform (WARP) with WiFi waveforms [6, 7, 8, 9, 10, 11]. Based on this feasibility result, it was recently suggested that a preferable FD network should consist of backhaul nodes operating in the FD mode and access nodes remaining in the legacy half-duplex (HD) mode [12, 13, 14]. However, recent researcher has showed that operating access nodes in FD mode significantly leverages the gain in degrees of freedom (DoF) in either ergodic or fast-fading channel [15], and an imperative is to design a hardware structure suitable for mass-production. Owing to the small-size, low-cost and low-energy-consumption, direct-conversion transceivers (DCTs) are widely applied in HD wireless systems, and are also suitable for far-end device implementation in the context of FD communication systems.

In order to provide SI cancellation, there exist numerous types of hardware solutions. According to the antenna placement strategies, these can be classified into separate-antennas-based and shared-antenna-based schemes. When each transceiver node is equipped with more than two separate antennas, SI attenuation can be achieved by improving the electromagnetic insulation between the antennas. Owing to the inherent closed-loop of FD systems, the knowledge of the SI channel matrix can be obtained by involving either placing extra transmit antennas or allocating specific spatial resources [16, 17]. On the other hand, the shared-antenna-based design aims to separate the transmit and receive signals by sharing a common antenna [9, 10], the key component of which is a three port routing device, known as a circulator, used to isolate the incoming and outgoing signals. Requiring only off-the-shelf RF components, the shared-antenna structure stands out as a cost-effective and energy-saving choice for the design of mobile-scale FD transceivers, and demonstrations on the WARP have shown that even 110 dB and 103 dB SI cancellation can be respectively achieved in SISO [9] and MIMO systems[10].

In a shared-antenna structure, there is still need for further non-trivial analog and digital SI cancellation, due to the leakage of the circulator, single-path reflection from the antenna, and multi-path interference from the surrounding environment. The purpose of analog SI cancellation is to prevent the saturation of the SI power level of the Rx low-noise amplifier (LNA), and of the same time, to ensure that the difference between the power of residual SI and the received signal of interest does not exceed the dynamic range of an analog-to-digital converter (ADC) [18]. Subsequently, a further digital baseband cancellation is performed to deal with the residual SI components, as well as with RF circuit non-idealities, mainly including nonlinear distortion, I/Q imbalance and phase noise. The nonlinearity is largely caused by the power amplifier (PA), while IQ imbalance and phase noise are mainly induced by the imperfect local oscillator (LO). The impact of PA nonlinear distortion on FD DCTs has been reported and investigated in [19, 20], while the effect of phase noise was analyzed in [21, 22]. Since the I/Q imbalance is essentially reflected in the mismatch between in-phase and quadrature components of the complex-valued I/Q signal, it is reflected in an image interference associated with the original signal [23, 24, 25].

The impact of the image interference caused by Tx I/Q imbalance on the SI cancellation has been studied in [26], indicating that it heavily limits the receiver path signal-to-noise-plus-interference ratio (SNIR). However, due to size constraints of FD DCTs, the Rx and Tx may share a common imperfect LO, therefore, a more accurate analysis of the impact of image interference on SI cancellation should be performed by a joint consideration of both Tx and Rx I/Q imbalances. Motivated by this finding, a widely linear framework was then developed, whereby not only the original transmit signal, but also its complex conjugate, i.e., the image interference, are jointly processed to form an estimate of the SI signal, followed by block-based parameter estimation methods to estimate the cancellation parameters through widely linear least-squares model fitting [27]. By exploiting the advantages of widely linear adaptive estimation algorithms as compared with block-based ones, such as their lower computational complexity and faster adaptation for potential time-varying channels, the augmented (widely linear) complex least mean square (ACLMS) adaptive filtering algorithm [28, 29, 30] has been employed in a DSP-assisted analog SI cancellation process, and its theoretical performance in the presence of Tx and Rx IQ imbalances has been evaluated [31]. However, for simplicity, in [31], the I/Q imbalances within transmitters and receivers were considered to be frequency-independent, which is not the case in wideband scenarios, where their frequency selectivity has been extensively reported and justified in [24]. Furthermore, although it has been illustrated by simulations that due to the undermodeling problem, ACLMS yields suboptimal SI cancellation results in the presence of PA nonlinearity, a theoretical understanding of this suboptimality and ways of its mitigation are still lacking.

Therefore, in this paper, we first provide a comprehensive mean and mean square performance analysis of the ACLMS based SI canceller, in both transient and steady-state stages, to theoretically establish its suboptimality in wideband FD DCTs. For rigor, both the PA nonlinear distortion and frequency-dependent image interferences are also considered. Next, in order to achieve a sufficient amount of SNIR when the nonlinear SI components are not negligible, an augmented nonlinear CLMS (ANCLMS) is proposed for a joint cancellation of both the image and the nonlinear SI components by virtue of a widely-nonlinear model fitting, and a theoretical performance evaluation is conducted to demonstrate its performance advantages over ACLMS. Moreover, to facilitate its use in practical applications, a data pre-whitening scheme is employed to speed up its convergence. Simulations on the proposed canceller with orthogonal frequency division multiplexing (OFDM)-based WLAN standard compliant waveforms applied support the analysis.

Notations: Lowercase letters are used to denote scalars, a, boldface letters for column vectors, a and boldface uppercase letters for matrices, A. The superscripts , , and denote respectively the complex conjugation, transpose, Hermitian transpose and matrix inversion operation. The operator represents the trace of a matrix, while operators , , respectively denote the convolution, Kronecker product and Frobenius norm. The statistical expectation operator is denoted by , matrix determinant by , while operators and extract respectively the real and imaginary parts of a complex variable and . The operator stands for a column vector composed by the diagonal elements of the matrix A. Matrix vectorization is designated by , which returns a column vector transformed by stacking the successive columns of matrix, and its inverse operation, i.e., restoring the matrix from the its vectorized form, is denoted by . The extraction of matrix diagonal element into a vector is denoted by .

Figure 1: The architecture of a shared-antenna FD DCT.

2 Full-Duplex Transceiver and Its Widely-Linear Baseband Equivalent Model

2.1 Fd Dct

The structure of a typical shared-antenna FD DCT is given in Fig. 1, and due to its simplicity this structure is widely adopted in modern wireless transceivers [32]. We here briefly discuss the physics for I/Q imaging and PA distortion, the two vital impairments within a FD DCT, and we refer to [20] for a complete characterization of the effective SI waveform in different stages of the transceiver.

The I/Q imbalance impairment, characterized by an amplitude difference between the I/Q oscillators, and/or a phase shift from the nominal , is a consequence of the imperfections of I/Q mixers at both the transmitter and receiver chains. In wideband systems, the mismatch between the low-pass filters of the in-phase and quadrature branches also contributes to the frequency selective nature of I/Q imbalance [33, 24]. The Tx frequency-dependent I/Q imbalance process occurs in the red shaded region in Fig. 1, for which input-output relation in discrete-time baseband can be expressed as


where is the original SI waveform before digital-to analog-conversion, perfectly known by the receiver, and is the I/Q imbalanced output of the Tx I/Q mixer. The channel impulse responses (IRs) for the direct signal and its image component , that is, and , can be further described as [24]


The physical meaning of , , and are provided in Table 1. The type of transformations in (1) and (2) , where both the direct and complex-conjugated signals are filtered and finally summed together, are called widely linear [34, 35]. The quality of the I/Q mixer can be quantified by the image rejection ratio (IRR), defined by1 . Note that the I/Q imbalance may also exist in the Rx mixer, and can be characterized in a similar way.

Symbol Denotation
Tx frequency-dependent IR of in-phase branch
Tx frequency-dependent IR of quadrature branch
Tx frequency-independent gain imbalance
Tx frequency-independent phase imbalance
I/Q imbalance IR of Tx direct component
I/Q imbalance IR of Tx image component
I/Q imbalance IR of Rx direct component
I/Q imbalance IR of Rx image component
Image rejection ratio
PA gain
Gain of nonlinear PA
VGA Gain
LNA Gain
Tx IQ mixer gain
Rx IQ mixer gain
LNA thermal noise
ADC quantization noise
Memory polynomials of PA
Estimation error of analog cancellation
Receiver sensitivity
SNR requirement
Dynamic range of ADC
Table 1: Notations and symbols used.

Another considerable impairment within a FD DCT is the PA nonlinearity, which may occur in the green shaded region in Fig. 1. The low-cost PA, which is used in the DCT to enhance the power of a signal before transmission, inevitably causes nonlinear distortion, and the common approach to describe its wideband behavior is via the Wiener-Hammerstein model, given by [36]


The physical meanings for , and in (3) are provided in Table. 1. Note that the gain from Tx variable gain amplifier (VGA) is omitted, since it is linear and can be absorbed into the SI waveform . The nonlinear term on the right hand side (RHS) of (3) is the third-order intermodulation term, which is much stronger compared with other terms beyond third-order. For simplicity, we shall now omit the descriptions of other modules within the FD DCT in Fig. 1, as well as other less significant impairments, such as phase noise and timing error, to yield a simplified widely linear relation from the SI waveform in the transmitter to the digitalized signal in the receiver, given by [20]


where and are the end-to-end channel IRs for the direct component and its image counterpart , given by


The physical meanings of , , and are listed in Table 1. Suppose that Tx and Rx mixers have the same level of IRR at dB, then, from (5) and (6), we observe that the second term on the RHS of (5) is negligible, since it is dB weaker than the first one, and the power of is considered to be dB lower than that of . The third term on the RHS of (4), that is, , is the received signal of interest, whose power before the digital SI cancellation can be calculated as


where is the Rx mixer gain, defined as . The explanations for the other parameters can be also found in Table. 1. The composite noise term, that is, in (4), represents the sum of interference components, including PA nonlinearity, thermal noise and quantization noise from an ADC, given by [20]




in which, represents the third-order intermodulated (IMD) component in (3), and is the Tx mixer gain. The approximation in (9) is made by considering a realistic situation for typical FD I/Q imbalance parameters, whereby the energy of the signal output from the Tx mixer is mainly concentrated in the light-of-sight path [33]. The corresponding channel IRs for and its image counterpart are denoted respectively by and . The interference, resulted from thermal noise during the digital SI cancellation process, is represented by in (12), where denotes the thermal noise induced by a LNA, and in (13) is an image counterpart of . The quantization noise is denoted by , whose power is subject to the number of ADC bits and the peak-to-average-power-ratio (PAPR) of .

2.2 Widely-linear Modelling in a Vectorized Form and Signal Assumptions

For mathematical simplicity, we consider a vectorized form to represent the widely linear relation between the observed signal and its corresponding SI waveform in (4), given by


where and are the optimal end-to-end channel IRs, determined by transmit and receive frequency-dependent I/Q imbalances, PA memory and residual of analog cancellation, as given in (5) and (6). is of length , whereby the SI waveform is a zero-mean proper white Gaussian random variable with variance . The Gaussianity and properness assumptions on are valid for wideband OFDM waveforms. Indeed, the work in [37] verified that a bandlimited uncoded OFDM symbol converges to a proper Gaussian random process as the number of subcarriers increases, and its whiteness is guaranteed due to the fact that is independent temporally. The analysis in [27] illustrates that under practical conditions, the power of the image thermal noise signal is much lower than those of the other components. Therefore, based on (8), the expression for the overall noise signal can be simplified as


in which and with respectively represent the end-to-end channel IRs of the IMD SI components, that is, , and its complex conjugate . Then, from (9), we know that is a zero-mean complex-valued random variable, whose variance is given by


By assuming that is a zero-mean complex-valued additive white Gaussian noise (AWGN), its variance can be determined as


where is the sensitivity level of the receiver and is the SNR requirement, and hence, from (12), the AWGN nature of can be also guaranteed, and its variance can be obtained as


The quantization noise is assumed to be another zero-mean AWGN process with variance , and is independent of . In this way, the VGA gain , used to ensure the received signal fit within the voltage range of the ADC, can be calculated as

where is the dynamic range of the ADC.

2.3 Component Analysis

From (4), observe that in a possible FD DCT, the baseband signal before digital SI cancellation is composed of various interference components, including SI , IMD SI , thermal noise and their image counterparts, as well as quantization noise . In order to visualize which terms are counted as primary interferences, simulations were carried out to illustrate their relative powers within FD DCTs. Typical system parameters, as suggested by [27], were chosen, and they are listed in Table 2, which shows that two types of FD DCTs with practical levels of analog SI cancellation are considered. The Type 2 DCT exhibits an inferior analog SI cancellation capability than its Type 1 counterpart, resulting in a weaker received signal of interest and thermal noise. The power of the thermal noise was chosen to be 15 dB below the receiver sensitivity, while that of the received signal of interest at the input of the receiver chain was exactly at the sensitivity level, as defined in (7). The impact of the analog cancellation on the DCT is indicated by the power of the analog cancellation error vector , which is incorporated into the channel IRs , , and . The coefficients of the filters are given to ensure the mixer gain or is 6 dB and IRR level is 25 dB, a specific requirement of 3GPP LTE [38]. The PA input-referred third-order intercept point (IIP3), represented by , is a fictitious point used to determine the power level of the third-order nonlinear distortion in the Tx output.

Parameter Value Notation
Receiver Sensitivity -89 dBm
SNR requirement 15 dB
Thermal noise floor -104 dBm
RF separation
40 dB (Type 1)
30 dB (Type 2)
RF attenuation
30 dB (Type 1)
20 dB (Type 2)
IRR 25 dB
Tx mixer gain 6 dB
Rx mixer gain 6 dB
PA gain 27 dB
PA IIP3 20 dBm
LNA gain 25 dB
Transmit power dB
ADC dynamic range 7 dB
Quantization noise power -60 dBm
Table 2: System Parameters of Typical FD DCTs
(a) Type 1
(b) Type 2
Figure 2: The power comparison among different signal components in representative FD DCTs, before digital SI cancellation and against different levels of transmit powers. (a) Type 1. (b) Type 2.

As shown in Fig. a, in a Type 1 FD DCT, the SI component and its image counterpart were both the dominant interferences to the signal of interest in the entire transmit power range, and when the transmit power becomes higher, the IMD SI component linearly increases to be another major interference. On the other hand, as indicated by Fig. b, in a Type 2 FD DCT, when the transmit power goes above 20 dBm, the thermal noise would become weaker than the quantization noise and the image IMD SI . This is because either a stronger nonlinear SI or a less efficient analog cancellation results in a lower VGA gain, which in turn yields a weaker amplification of signal of interest and thermal noise , as indicated by (7), (17) and (LABEL:kBB).

3 Conventional ACLMS based SI Canceller and Its Performance Analysis

As discussed in Section 2, the effect of of Tx and Rx IQ imbalances is to make both the SI component and its image counterpart the dominant interferences in a FD DCT. To mitigate this issue, a widely linear model based augmented complex least mean square (ACLMS) adaptive filtering algorithm was employed in [31] as a further DSP-assisted canceller after the analog SI cancellation procedure, and a preliminary SI cancellation performance analysis of ACLMS was conducted. For simplicity, the I/Q imbalances within transmitter and receiver were considered to be frequency-independent in [31], which, however, is not the case in wideband scenarios, where frequency selectivity has been rigourously justified [24]. Furthermore, it has been illustrated by simulations that ACLMS yields suboptimal SI cancellation results in the presence of PA nonlinearity, since it arbitrarily excludes the nonlinear SI components from its underlying observation model, which in fact can be outstanding among all the imperfections, as shown in Section 2.3. Meanwhile, a theoretical quantification of this suboptimality is still missing. Therefore, in this section, we provide a comprehensive mean and mean square convergence analysis of the conventional ACLMS based SI canceller in the presence of both frequency-dependent Tx and Rx I/Q imbalances and PA nonlinear distortion. A unified statistical framework to quantify the SI cancellation performance of ACLMS in both transient and steady-state stages is investigated. For rigor, the proposed analysis covers both the cases of low and high transmit powers. For the compactness of analysis, we first represent the widely linear model in (14) in an augmented form, given by


where is the augmented SI vector, and is the augmented optimal end-to-end system IRs, which model the transmit and receive frequency-dependent I/Q imbalances, PA distortion and the residual of analog SI cancellation.

The ACLMS estimates the set of system parameters by minimising the MSE cost function , defined as [28, 29, 30]


where is the instantaneous output error, given by


in which the augmented weight vector of ACLMS, that is, , is updated as


where is the step size [39, 28].

3.1 Mean Convergence Analysis

Upon introducing the weight error vector


the filter output in (22) becomes


From (23), the recursion for the update of the weight error vector can be derived as


where is an identity matrix.

The mean behavior of can now be determined by applying the statistical expectation operator to both sides of (26) and upon employing the standard independence assumptions [40, 41, 42, 43, 44, 45], that is, the composite noise is statistically independent of any other signal in ACLMS, and is statistically independent of the augmented SI input vector , to yield


where is the covariance matrix of the augmented SI vector , defined as


Therefore, the convergence of ACLMS in the mean is guaranteed if the step-size satisfies [44]


From Fig. a and Fig. b in Section 2.3, we observe that the relative power relationships among different SI components vary as the transmit power changes. Therefore, in order to accurately describe the statistical mean behaviour of the ACLMS based SI canceller, we next consider two case studies as

Low transmit power

When the transmit power is low, both the nonlinear distortion component and the quantization noise at the receiver are negligible, since their powers are much weaker than that of the thermal noise in (15). Therefore, from (20), we have and hence, at the steady-state, i.e., , from (27)


High transmit power

When the transmit power is high, the nonlinear PA distortion component becomes the third strongest interference among all the imperfections considered. Upon using the independence assumptions and (15), the second term on the RHS of (27) can be derived as


where represents an zero vector. The last step is performed by using the Gaussian fourth order moment factorizing theorem and since is proper, we have [40, 41, 42, 43]. From (27), the steady-state value of the weight error vector, that is, , can be evaluated as


Remark 1: The upper bound on the step size for the mean convergence of ACLMS for a low transmit power FD DCT is identical to that for a high transmit power one. At the steady state, when the transmit power is low, ACLMS converges in the mean to the optimal weight coefficients associated with , that is, in (20), in an unbiased manner. However, as indicated by (32), when the transmit power is high, this yields a bias in the estimation of out of entries of the weight error vector , quantified by . The level of this bias depends upon the level of undermodeling, that is, the transmitter mixer gain , the transmit SI signal power , and the channel IRs associated with the IMD SI components, that is, and , whose values are determined by receiver frequency-dependent I/Q imbalances, PA memory and residual of analog SI cancellation, as shown in (10) and (11).

3.2 Mean-Square Convergence Analysis

From (21) and (25), and again by employing the standard independence assumptions stated in Section 3.1, the MSE of ACLMS based SI canceller, that is, , can be further evaluated as


where is the covariance matrix of the augmented weight error vector . It can be observed from (33) that the mean square convergence analysis of ACLMS now rests upon both the first and second order statistical properties of . To this end, we first apply the Hermitian operator to both sides of (26), to yield


Upon multiplying (34) to both sides of (26) with (34) and taking the statistical expectation , the evolution of the weight error covariance matrix now becomes




It can be observed that is independent from the IMD component , and hence [41, 42, 43]


where is the pseudocovariance matrix of the augmented SI vector , given by

in which is an identity matrix. The term on the RHS of (37) can be decomposed as , where is a vector, whose entries are the diagonal elements of , defined as


and denotes a vector in which all the entries are unities. Then, based on (35), the evolution of becomes


The convergence of the recursion for the vector in (39) is subject to two conditions: 1) the terms , , and are bounded, which is guaranteed if is bounded; 2) all the eigenvalues of the transition matrix are less than unity [46]. From (29), Condition 1) holds when . Furthermore, the eigenvalues of , denoted by , where , can be obtained by solving , and from (39), it is easy to find that is Toeplitz, for which the diagonal elements are , and off-diagonal ones are . Hence, after a few manipulations, we have


Note that since , we have , and hence, Condition 2) is satisfied if , to yield


Remark 2: The upper bound in (42) is tighter than that in Condition 1), and therefore, the mean square convergence of ACLMS based SI canceller in the presence of frequency-dependent IQ imbalances and PA distortion is guaranteed if the step-size satisfies (42).

3.3 Steady State Analysis

Suppose that step-size is chosen such that the mean square stability of ACLMS is guaranteed [47]. Consider and based on (33) and (38), its steady-state MSE can be expressed as