Comments on “Design of fractional-order variants of complex LMS and NLMS algorithms for adaptive channel equalization”
The purpose of this note is to highlight some critical flaws in recently proposed fractional-order variants of complex least mean square (CLMS) and normalized least mean square (NLMS) algorithms in “Design of Fractional-order Variants of Complex LMS and Normalized LMS Algorithms for Adaptive Channel Equalization” [Non-linear Dyn. 88(2), 839-858 (2017)]. It is substantiated that these algorithms do not always converge whereas they have no advantage over the CLMS and NLMS algorithms whenever they converge. Our claims are based on analytical reasoning supported by simulations.
keywords:Least mean squares algorithm, Fractional-order variant of LMS, Complex LMS, Normalized LMS.
The least mean square (LMS) is a widely used algorithm in adaptive signal processing LMSBook1 (). It has many variants to deal with assorted signals and environmental conditions CLMS (); LMS11 (); QKLMS (); NLMS (); qLMS (). Recently, some fractional-order variants of the CLMS and the NLMS (henceforth acronymed as the FCLMS and the FNLMS resp.) are proposed in CFLMS () pretending improved steady-state and convergence performances in an adaptive filtering framework. A system input vector is passed through an tap channel with finite impulse response. A vector is formed using the noisy channel output and subsequently used as an input to tap filter with sought weight vector so that the error during the data transmission, , is minimum. Accordingly, the mean squared error (MSE) based objective function
is considered and solved using the FCLMS and the FNLMS. In above and hereinafter, the superposed , , and indicate complex conjugate, transpose and Hermitian transpose, respectively.
In this note, we argue that the proposed variants, the FCLMS and the FNLMS, have no improvement over the CLMS and the NLMS, and there are serious flaws in the design and simulation setup of these algorithms CFLMS (). We present our main remarks in Section 2 and provide some supporting simulation results in Section 3.
2 Main Remarks
In order to facilitate the ensuing discussion, we use same notations and equation numbers as in CFLMS (). The corrected equations will be marked by superposed asterisk and new equation numbers will be in roman.
2.1 Inappropriate objective function for the FCLMS
where is the cross-correlation between the input and the output, is the auto-correlation matrix of the output , and is the average power of the input . In CFLMS (), is defined by (without modulus) which in fact corresponds to the real systems only. The correct form of the objective function for the CLMS is well-known (see, e.g., (LMSBook1, , Eq. 2.99)) and is given by
Herein, and denote the real and the imaginary parts, respectively. In fact, an inappropriate use of the relation led to the simplified expression (5). However, this is not possible for a complex system unless for all , i.e., the cross-correlation between and , is strictly real, which is very strong and unrealistic assumption for complex system identification, if not impossible. Regardless of the fact, the objective function in Eq. (5) appears to be a complex-valued function. It is simply due to the presence of (instead of as in (5*)) which is complex. We emphasize that the MSE is supposed to be a real-valued quadratic function. Therefore, any variant of the CLMS based on the objective function (5) is highly felicitous to diverge or even fail. The proposed FCLMS can be expected to work only in the real cases. However, in that situation, it will be simply a fractional-order variant of the LMS introduced in FLMS () but not the CLMS. The performance of similar fractional-order variants has already been debated in bershad2017comments (), where it is established that they have no advantage over the conventional LMS. As will be discussed later on (see Section 2.3), there is a discrepancy between the pseudo-code implementation in (CFLMS, , Table 1) and the suggested theoretical derivation of the FCLMS. Consequently, this flaw is only relevant to the theoretical presentation of the algorithm which does not correspond to the implementation.
2.2 Inappropriate use of fractional calculus
The update rules for the FCLMS and FNLMS are, respectively, defined by
Herein, and are the controlling parameters for the integral and fractional updates, is the fractional-order, and is the fractional gradient with respect to defined in terms of the left Riemann-Liouville fractional derivative as in (CFLMS, , Eq. (14)). Using the formula (see, e.g.,Kilbas ())
the fractional gradient term for the FCLMS is presented in CFLMS () as
where represents the Gamma function and the exponent on is component-wise. For the FNLMS, the same gradient term is used without complex conjugate on .
The expression (36) is not justified in CFLMS (). We argue that it is incorrect. Indeed, if we consider the simple case of the real fractional gradient for the FNLMS and rigorously use fractional calculus for the Riemann-Liouville derivatives, it turns out that
where and are the components of the cross-correlation vector and the autocorrelation matrix given by
For the FCLMS, the weight is complex. Therefore, the fractional gradient of the real-valued function with respect to a complex vector should be calculated in the sense of Wirtinger calculus (see, for instance, Delgado ()). Therefore, the expression (36) for the fractional gradient of the non-holomorphic function with respect to complex vector is unwarranted.
2.3 Inappropriate design
Let us ignore the mathematical jargon discussed in Section 2.2 for an instance. Precisely, assume that the update equations,
for the FCLMS and the FNLMS, respectively, are constituted by intuition. Then, their design is similar to the fractional LMS algorithm in FLMS (). Accordingly, they inherit the problems of the fractional LMS already discussed in bershad2017comments (). More specifically, we have the following remarks.
Eq. (37) also substantiates that for negative values of the fractional term will be complex. However, the FCLMS may converge thanks to the integral part of the update equation (37) that corresponds to the CLMS algorithm. Nevertheless, it will converge to a high steady-state residual error generated by the fractional term (see, e.g., Fig. 3).
Interestingly, on contrary to all the theoretical discussion and equations, the pseudo-code implementation in (CFLMS, , Table 1) is done by just augmenting the corresponding update equation of the CLMS by the right hand side of Eq. (36) and the rest of the code remains the same. More specifically, there is a discrepancy between the theoretical analysis and the numerical implementation. However, this appears to be a blessing in disguise in some situations where the convergence of the CLMS algorithm is not stymied by the fractional part of the update equation for the FCLMS.
2.4 Simulation bias
In adaptive signal processing, performance comparison between algorithms can be made on the basis of different criteria. Three important measures of performance are: i) convergence rate, ii) steady-state error, and iii) computational complexity. From equation (37) and (45), it can be seen that the fractional-order variants are computationally very expensive, we therefore focus only on convergence and steady-state measures for our experiments.
For a fair evaluation, the conventional algorithms and their proposed counterparts must be setup at either an equal convergence (for the steady-state performance comparison) or an equal steady-state (for the convergence performance). Also, if one algorithm can perform better than the other in both aspects then higher convergence rate at the cost of low steady-state error must be shown.
We argue that the simulation parameters used in (CFLMS, , Sect. 4.1) are biased. Specifically, with the learning rate values adopted in (CFLMS, , Sect. 4.1), the LMS and the NLMS algorithms converge slowly (see (CFLMS, , Fig. 3-5)). Other issues are listed below.
The results in CFLMS () are reported for one simulation run only and no Monte Carlo simulation information is given, which makes reproducibility of the similar results very difficult if not impossible.
In (CFLMS, , Fig. 3-5), the linear scale is used for axis which makes it difficult to compare the steady-state error.
The performance shown in (CFLMS, , Sect. 4.1) for FNLMS does not delineate the actual convergence trend. With the given simulation parameters the FNLMS shows divergence (see, Section 3.1).
In a random desired weight vector scenario ((CFLMS, , Sect. 4.1)), the information of the random distribution is not provided. If the weights are obtained from a Gaussian distribution that may result in a random negative value then the given results are not possible/reproducible, in view of the remarks in Section 2.3.
To evaluate the performance of the FNLMS and the FCLMS algorithms, we considered the problem of system identification. The FNLMS algorithm is evaluated for two evaluation protocols: i) the system with negative desired weights under noisy environment with signal-to-noise ratio (SNR) of dB; ii) the system with all positive weights without any noise. The FCLMS algorithm is evaluated for negative weights only under noisy environment with SNR of dB.
The NLMS, the CLMS, and their fractional order variants are configured to equal performance at . The performance of the FNLMS and the FCLMS is observed for , , , , , and .
For real inputs, we considered a random signal of length obtained from a zero-mean Gaussian distribution with variance . For the complex signal, a similar configuration is used however, the signal is obtained from a circular complex Gaussian distribution instead of a real random source.
The experiments are repeated for independent rounds and mean results are reported. For each independent round, the weights were initialized with zeros. The performance of all the algorithms is evaluted on mean deviation (MD) which is the norm of the difference between the sought and the obtained weights, i.e.,
where and are the sought and approximated weight vectors at th iteration, respectively. Here, is the norm and is the length of the filter vector.
3.1 Performance evaluation of the FNLMS
3.1.1 Evaluation protocol 1
We consider a system with impluse response values
The step-size for the NLMS is set to , whereas those for the FNLMS are set to be (see, (CFLMS, , Sect. 4)). Figure 1 shows the learning curves for the NLMS and the FNLMS. We setup both algorithms on equal convergence rate, and compared the steady-state performance of both algorithms. It can be observed that, the FNLMS algorithm failed to identify the system with negative weights for all the listed values of .
3.1.2 Evaluation protocol 2
In evaluation protocol 2, we choose the desired weight
as it is given in (CFLMS, , Sect. 4.1). For the NLMS the step-size is set to , whereas for the FNLMS the step-sizes are set to be . Figure 2 shows the learning curves for the NLMS and the FNLMS. We setup both algorithms at an equal convergence rate and compared the steady-state performance. From Fig. 1, it can be seen that the FNLMS algorithm is diverging for all the listed values of and the results obtained herein do not agree with those reported in (CFLMS, , Sect. 4.1).
3.2 Performance evaluation of the FCLMS
For the evaluation of the FCLMS, we consider a system with impulse response values of
The step-size of CLMS is set to , whereas the step-size for the FCLMS and are set to be , and , respectively. Figure 3, shows the learning curves for the CLMS and the FCLMS algorithms. We setup both algorithms at an equal convergence performance and compared the steady-state error. From Fig. 3, it is evident that the CLMS algorithm performs better than the FCLMS algorithm under all conditions. The final MD values for the FCLMS and the CLMS are reported in Table 1. Note that the fractional-term in the FCLMS has no benefit at all. On contrary, it is stymieing the steady-state performance of the integral part corresponding to the CLMS without even improving the convergence rate.
|FCLMS: steady-state error (dB) at different fractional powers|
In this comment, we have analyzed the fractional-order variants of the complex least mean square (CLMS) and the normalized least mean square (NLMS) proposed in CFLMS (). We have highlighted serious flaws in the theoretical derivation, design, and simulation setup in CFLMS (). We conclude that the proposed algorithms either diverge or do not show any improvement in the performance in terms of convergence and steady-state error over the conventional algorithms.
Appendix A Derivation of Riemann-Liouville fractional gradient for real case
Recall that, for any ,
where we have used the fact that . Notice, that the first term is constant with respect to . Therefore, by the definition of the Riemann-Liouville derivative and invoking the rule (32), one arrives at
- (1) P. S. R. Diniz, Adaptive Filtering: Algorithms and Practical Implementation, Springer, New York, 2013.
- (2) B. Widrow, J. McCool, and M. Ball, The complex LMS algorithm, Proceedings of the IEEE, 63: (1975), 719-720.
- (3) A. Khalili, A. Rastegarnia, and S. Sanei, Quantized augmented complex least-mean square algorithm: Derivation and performance analysis, Signal Processing 121:(2016), 54–59.
- (4) B. Chen, S. Zhao, P. Zhu, and J. C. Príncipe, Quantized kernel least mean square algorithm, IEEE Transactions on Neural Networks and Learning Systems 23 (1): (2012), 22–32.
- (5) J. I. Nagumo and A. Noda, A learning method for system identification, IEEE Transactions on Automatic Control, 12 (3): (1967), 282–287.
- (6) U. M. Al-Saggaf, M. Moinuddin, M. Arif, and A. Zerguine, The q-least mean squares algorithm, Signal Processing 111: (2015), 50–60.
- (7) S. M. Shah, R. Samar, N. M. Khan, and M. A. Z. Raja, Design of fractional-order variants of complex LMS and NLMS algorithms for adaptive channel equalization, Nonlinear Dynamics 88(2):(2017), 839–858.
- (8) M. A. Z. Raja, I. M. Qureshi, A modified least mean square algorithm using fractional derivative and its application to system identification, Eur. J. Sci. Res. 35(1): (2009), 14–21.
- (9) N. J. Bershad, F. Wen, and H. C. So, Comments on “Fractional LMS algorithm”, Signal Processing 133:(2017), 219–226.
- (10) A. A. Kilbas, H. M. Srivastava, and J. J. Trujillo, Theory and Applications of Fractional Differential Equations, Elsevier, 2016.
- (11) K. Kreutz-Delgado, The complex gradient operator and the CR-calculus, arXiv preprint (arXiv:0906.4835), 2009 [Retrieved on 15 Dec. 2017].