Minimum Error Entropy Kalman Filter

Minimum Error Entropy Kalman Filter

[    [    [    [    [
Abstract

To date most linear and nonlinear Kalman filters (KFs) have been developed under the Gaussian assumption and the well-known minimum mean square error (MMSE) criterion. In order to improve the robustness with respect to impulsive (or heavy-tailed) non-Gaussian noises, the maximum correntropy criterion (MCC) has recently been used to replace the MMSE criterion in developing several robust Kalman-type filters. To deal with more complicated non-Gaussian noises such as noises from multimodal distributions, in the present paper we develop a new Kalman-type filter, called minimum error entropy Kalman filter (MEE-KF), by using the minimum error entropy (MEE) criterion instead of the MMSE or MCC. Similar to the MCC based KFs, the proposed filter is also an online algorithm with recursive process, in which the propagation equations are used to give prior estimates of the state and covariance matrix, and a fixed-point algorithm is used to update the posterior estimates. In addition, the minimum error entropy extended Kalman filter (MEE-EKF) is also developed for performance improvement in the nonlinear situations. The high accuracy and strong robustness of MEE-KF and MEE-EKF are confirmed by experimental results.

Paestum]Badong Chen, Paestum]Lujuan Dang, Rome]Yuantao Gu, Paestum]Nanning Zheng, Paestum,Baiae]José C. Príncipe

Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Xi’an, 710049, China 

Beijing National Research Center for Information Science and Technology (BNRist) and Department of Electronic Engineering, Tsinghua University, Beijing, 100084, China 

Department of Electrical and Computer Engineering, University of Florida, Gainesville, FL, 32611, USA 


Key words:  Kalman filtering, Minimum Error Entropy (MEE), robust estimation, non-Gaussian noises.

 

11footnotetext: This paper was not presented at any IFAC meeting. Corresponding author Badong Chen. Tel. 86-29-82668802 ext.8009. Fax. 86-29-82668672.

1 Introduction

Kalman filtering is a powerful technology for estimating the states of a dynamic system, which finds applications in many areas including navigation, guidance, data integration, pattern recognition, tracking and control systems [1, 2, 3, 4, 5]. The original Kalman filter (KF) was derived for a linear state space model with Gaussian assumption [6, 2]. To cope with nonlinear estimation problems, a variety of nonlinear extensions of the original Kalman filter have been proposed in the literature, including extended Kalman filter (EKF) [7, 8], unscented Kalman filter (UKF) [9], cubature Kalman filter (CKF) [10] and many others. However, most of these Kalman filters are developed based on the popular minimum mean square error (MMSE) criterion and will face performance degradation in case of complicated noises, since in general MMSE is not a good choice for the estimation in non-Gaussian noises.

In recent years, to solve the performance degradation problem in heavy-tailed (or impulsive) non-Gaussian noises, some robust Kalman filters have been developed by using certain non-MMSE criterion as the optimality criterion [11, 12]. Particularly, the maximum correntropy criterion (MCC) [13, 14] in information theoretic learning (ITL) [11, 12] has been successfully applied in Kalman filtering to improve the robustness against impulsive noises. Typical examples include the maximum correntropy based Kalman filters [15, 16, 17, 18, 19, 20, 21, 22, 23, 24], maximum correntropy based extended Kalman filters [25, 26, 27], maximum correntropy based unscented Kalman filters [28, 29, 30], maximum correntropy based square-root cubature Kalman filters [31, 32] and so on. Since correntropy is a local similarity measure and insensitive to large errors, these MCC based filters are little influenced by large outliers [13, 33].

The MCC is a nice choice for dealing with heavy-tailed non-Gaussian noises, but its performance may not be good when facing more complicated non-Gaussian noises, such as noises from multimodal distributions. The minimum error entropy (MEE) criterion [34, 35] is another important learning criterion in ITL, which has been successfully applied in robust regression, classification, system identification and adaptive filtering [34, 35, 36, 37, 38]. Numerous experimental results show that MEE can outperform MCC in many situations although its computational complexity is a little higher [39, 40]. In addition, the superior performance and robustness of MEE have been proved in [41]. The goal of this work is to develop a new Kalman-type filter, called minimum error entropy Kalman filter (MEE-KF), by using the MEE as the optimality criterion. The proposed filter uses the propagation equations to obtain the prior estimates of the state and covariance matrix, and a fixed-point algorithm to update the posterior estimates and covariance matrix, recursively and online. To further improve the performance in the nonlinear situations, the MEE criterion is also incorporated into EKF, resulting in minimum error entropy extended Kalman filter (MEE-EKF).

The rest of the paper is organized as follows. In section II, we briefly review the KF algorithm and MEE criterion. In section III, we develop the MEE-KF algorithm. Sections IV and V provide the computational complexity and convergence analysis, respectively. In section VI, the MEE-EKF is developed. The experimental results are presented in section VII and finally, the conclusion is given in section VIII.

2 Background

2.1 Kalman Filter

Consider a linear dynamic system with unknown state vector and available measurement vector . To estimate the state , Kalman filter (KF) assumes a state space model described by the following state and measurement equations:

(1)
(2)

where and are the state-transition matrix and measurement matrix, respectively. Here, the process noise and measurement noise are mutually independent, and satisfy

(3)
(4)
(5)

where and are the covariance matrices of and , respectively. In general, the KF includes two steps:

(1) Predict: The a-priori estimate and the corresponding error covariance matrix are calculated by

(6)
(7)

(2) Update: The a-posteriori estimate and the corresponding error covariance matrix are obtained by

(8)
(9)
(10)

where is the Kalman filter gain.

2.2 Minimum Error Entropy Criterion

Different from the MMSE [34] and MCC [35], the MEE aims to minimize the information contained in the error. In MEE, the error information can be measured by the Renyi’s entropy:

(11)

where is the order of Renyi’s entropy, and denotes the information potential defined by

(12)

where is the probability density function (PDF) of error and denotes the expectation operator. In practical applications, the PDF can be estimated by Parzen’s window approach [11]:

(13)

where denotes the Gaussian kernel with kernel size ; are error samples. Combining (12) and (13), one can obtain an estimate of the second order () information potential :

(14)

Since the negative logarithmic function is monotonically decreasing, minimizing the error entropy means maximizing the information potential .

3 Minimum Error Entropy Kalman Filter

3.1 Augmented Model

First, we denote the state prediction error as

(15)

Combining the above state prediction error with the measurement equation (2.1), one can obtain an augmented model

(16)

where denotes a identity matrix, and

(17)

is the augmented noise vector comprising of the state and measurement errors. Assuming that the covariance matrix of the augmented noise is positive definite, we have

(18)

where , and are obtained by the Cholesky decomposition of , and , respectively. Multiplying both sides of (16) by gives

(19)

where

(20)
(21)
(22)

with , , and .

3.2 Derivation of MEE-KF

Based on (14), the cost function of MEE-KF is given by

(23)

Then, the optimal solution to is achieved by maximizing the cost function (23), that is

(24)

Setting the gradient of the cost function regarding to zero, we have

(25)

where

(26)
(27)
(28)
(29)
(30)
(31)

From (3.2), can be solved by a fixed-point iteration:

(32)

where

(33)

where . The explicit expressions of , , and are

(34)
(35)
(36)
(37)

According to Eqs. (20), (21) and (33), we arrive at

(38)

By (38), the Eq. (3.2) can be rewritten as

(39)

where

By using the matrix inversion lemma

(40)

with the identifications

(41)

one can reformulate (3.2) as

(42)

where

(43)

Then, the posterior covariance matrix can be updated by

(44)

With the above derivations, the proposed MEE-KF algorithm can be summarized as Algorithm 1.

Step 1: Initialize the state priori estimate and state prediction error covariance matrix ; set a proper kernel size and a small positive number .
Step 2: Use Eqs. (2.1) and (2.1) to obtain and , respectively; use the Cholesky decomposition of to obtain and ; use Eqs. (20) and (21) to obtain and , respectively.
Step 3: Let and , where denotes the estimated state at the fixed-point iteration .
Step 4: Use available measurements to update:

(45)

with

(46)
(47)
(48)
(49)
(50)
(51)
(52)

Step 5: Compare and

(53)

If the above condition holds, set and continue to Step 6. Otherwise, , and return to Step 4.
Step 6: Update and the posterior error covariance matrix by

(54)

and return to Step 2.

Algorithm 1 Minimum Error Entropy Kalman Filter (MEE-KF)

4 Computational Complexity

This section provides the comparison of the computational complexities of KF, maximum correntropy Kalman filter (MCKF) [15] and MEE-KF in terms of the floating point operations.

The KF updates with Eqs. (2.1)-(10), and the corresponding floating point operations are given in Table 1. From Table 1, we can conclude that the computational complexity of KF is

(55)

According to [15], the computational complexity of MCKF is

(56)

where denotes the fixed-point iteration number, which is relatively small in general as shown in simulations in Section VII.

The updates of MEE-KF involve Eqs. (2.1), (2.1), (45)-(52) and (54), and the corresponding floating point operations are shown in Table 1. According to Table 1, the computational complexity of MEE-KF is

(57)

The MEE-KF has an additional computational burden induced by the error entropy functions in comparison to KF, and has a slightly higher computational complexity than MCKF. In the sense of order of magnitude, the computational complexities of the MEE-KF, MCKF and KF have no significant difference.

Equation Addition/subtraction and multiplication Division, matrix inversion, Cholesky decomposition and exponentiation
(2.1)
(2.1)
(2.1)
(2.1)
(10)
(45)
(46)
(47)
(48)
(49)
(50)
(51)
(52)
(54)
Table 1: Computational Complexities of Some Equations

5 Convergence Issue

This section provides a sufficient condition to ensure the convergence of the fixed point iterations in MEE-KF, where the proof is similar to [35] and thus will not be provided here.

First, from Eq. (3.2), we can rewrite

(58)

with and .

Thus, a Jacobian matrix of with respect to gives

(59)

with . Define as an -norm () of a vector or an induced norm of a matrix as