Distributed Dimensionality Reduction Fusion Estimation with Communication Delays in Cyber-Physical Systems
This paper studies the distributed dimensionality reduction fusion estimation problem with communication delays for a class of cyber-physical systems (CPSs). The raw measurements are preprocessed in each sink node to obtain the local optimal estimate (LOE) of a CPS, and the compressed LOE under dimensionality reduction encounters with communication delays during the transmission. Under this case, a mathematical model with compensation strategy is proposed to characterize the dimensionality reduction and communication delays. This model also has the property to reduce the information loss caused by the dimensionality reduction and delays. Based on this model, a recursive distributed Kalman fusion estimator (DKFE) is derived by optimal weighted fusion criterion in the linear minimum variance sense. A stability condition for the DKFE, which can be easily verified by the exiting software, is derived. In addition, this condition can guarantee that estimation error covariance matrix of the DKFE converges to the unique steady-state matrix for any initial values, and thus the steady-state DKFE (SDKFE) is given. Notice that the computational complexity of the SDKFE is much lower than that of the DKFE. Moreover, a probability selection criterion for determining the dimensionality reduction strategy is also presented to guarantee the stability of the DKFE. Two illustrative examples are given to show the advantage and effectiveness of the proposed methods.
istributed Fusion Estimation, Kalman Filtering, Bandwidth Constraints, Communication Delays, Stability Analysis, Cyber-Physical Systems.
Information fusion has attracted considerable research interest during the past decades, and has found applications in a variety of areas, including internet of things  and cyber-physical systems (CPSs) . Particularly, multi-sensor fusion estimation utilizes useful information contained in multiple sets of data for the purpose of estimating a quantity or parameter in a process . It is widely used in practical applications because it can potentially improve estimation accuracy and enhance reliability and robustness against faults [3, 4, 5]. Many fusion estimation approaches have been presented in the literature (see [6, 7, 8, 9, 10, 11, 12], and the references therein). At the same time, advances in embedded computing, communication, and related hardware technologies have recently brought the paradigm of CPSs to a new research frontier . Moreover, CPSs have found applications in a broad range of areas such as intelligent transportation systems , multi-robot systems , and smart grid systems . As one of important issues in CPSs, real-time state estimation based on sensor measurements has recently attracted considerable research interests because state estimate can provide a CPS with the real-time monitoring and control capability [17, 18]. For example, estimating the real voltage from sensor information must be completed before taking certain actions to regulate the voltage into some desired range in a power grid . It is noted that the accuracy of state estimation has an important impact on computing control commands for safe and efficient operation of a CPS [17, 18, 19]. Therefore, it is of theoretical significance and practical relevance to investigate the problem of information fusion estimation for the CPSs [20, 21].
There mainly exist two kinds of fusion architectures: centralized fusion structure and distributed fusion structure. However, the distributed fusion structure is generally more robust and fault-tolerant as compared with the centralized fusion structure [4, 5, 6, 7, 8]. This motivates us to consider the distributed fusion estimation problem in this paper for a class of CPS architecture (see Fig.1), where system state is spatially distributed in the physical space. When the local estimates are transmitted to the fusion center (FC) via communication channels, bandwidth constrains and communication delays are unavoidable in communication networks . Moreover, the above two factors can degrade the fusion estimation performance because of the information loss caused by bandwidth and delay constrains [23, 24, 25]. Thus, how to design distributed fusion methods in the presence of bandwidth and delay constraints is essential for real-time state estimate of CPSs.
1.1 Related Work
Regarding the problem of bandwidth constraints in multi-sensor systems, as pointed out in , there are mainly two approaches to reduce the communication traffic: the quantization method (see [27, 28, 29], and references therein) and the dimensionality reduction method (see [30, 31, 32], and the references therein). Particularly, the dimensionality reduction method to the original multi-sensor observations was designed in  based on the principal components analysis, while the dimensionality reduction strategy with the quantization error was developed in  to deal with stable multi-sensor fusion systems. Under the distributed fusion structure, when the physical state x(t) as shown in Fig.1 is multi-dimensional (or even high-dimensional) in a CPS, it is unrealistic to completely send the local estimate of the state x(t) to the FC via a bandwidth-constrained communication channel. In this sense, bandwidth constraint in the CPSs is the primary consideration when designing a distributed state fusion estimator. Notice that, to reduce the communication traffic, the idea of the dimensionality reduction method is that a multi-dimensional signal is directly converted into a low-dimensional signal, while the idea of the quantization method is that the number of coding bits for each component of a multidimensional signal is reduced before being transmitted. Meanwhile, the quantization usually results in nonlinear dynamics, and it is difficult to find a data compression operator analytically, particularly, for the multidimensional signals. Therefore, the dimensionality reduction method can provide an attractive alternative to solve the distributed fusion estimation problem with bandwidth constraints in the CPSs.
Though the dimensionality reduction fusion estimation algorithms have been proposed in [30, 31, 32, 33, 34] to reduce the communication traffic, the communication delays, which occurs during the transmission, were not taken into account. With the communication delays, the dimensionality reduction fusion estimation must solve two challenging issues: one is how to compensate the information loss caused by the communication delays and bandwidth constraints under a unified mathematical model; The other one is how to fuse the asynchronous local compressed estimates because of communication delays. Notice that the centralized and distributed fusion estimation algorithms have been proposed in [23, 35, 36, 37, 38, 39, 40] based on different communication delay models, however, the main results in [23, 35, 36, 37, 38, 39, 40] cannot be extended to the case of the dimensionality reduction estimation with communication delays. The reason is that the data compression and information compensation in dimensionality reduction may change the property of the original measurements (e.g., the statistical correlation in [30, 32] has been changed under the Kalman fusion structure). Under this case, we have studied the information fusion estimation problem in [20, 24] for the CPSs with bandwidth constraints and communication delays. It should be pointed out that the steady-state fusion estimator with simple calculation cannot be obtained based on the proposed communication model in , while the covariance intersection (CI) fusion strategy in  was suboptimal because fusion estimator was determined by minimizing an upper bound of estimation error covariance.
Motivated by the aforementioned analysis, we study the distributed stochastic dimensionality reduction fusion estimation problem with communication delays for the CPSs. Notice that the information loss is inevitable because of the dimensionality reduction and communication delays, and such a fusion estimation with incomplete information will degrade the estimation performance. Since the delays are caused by communication channels, the key issue is how to design an efficient dimensionality reduction strategy to guarantee the stability of the distributed fusion estimator. Although our previous works in [20, 24, 30] have studied the related stochastic dimensionality fusion estimation problems, there are still fundamental problems that cannot be solved up to now. In detail,
When only considering stochastic dimensionality reduction strategy, the stable probability selection criterion in  was derived from the inequality relaxation of the matrix trace. However, the inequality relaxation will lead to certain conservatism, thus how to find a new derivation idea to reduce the conservatism is very important for the application of the proposed dimensionality reduction strategy. Notice that the stability conditions in  were directly derived from the similar derivation in , and thus the corresponding conservatism cannot also be avoided in .
When considering stochastic dimensionality reduction strategy under communication delays, the distributed CI fusion estimator in  was suboptimal because the corresponding optimization objective was an upper bound of the estimation error covariance matrix. Particularly, the CI fusion results in  needed to solve non-convex nonlinear optimization problems online at each time, which may lead to a large number of calculation. Though the distributed fusion estimator in  was optimal based on the optimal weighted fusion criterion, the model of communication delays cannot be applicable to the case of time-varying delays. More importantly, the computational complexity of the fusion estimator in  was also high. Obviously, the common disadvantage of the results in  and  is the high computation cost, and the optimal weighed fusion criterion can provide the optimal and analytic solutions. Therefore, based on the optimal weighted fusion criterion, how to design steady-state dimensionality reduction fusion estimators with simple calculation is of great significance in the presence of communication delays.
We shall solve the above two problems, and the main contributions of this paper can be summarized as follows:
An optimal distributed Kalman fusion estimator (DKFE) is derived in the linear minimum variance sense when there are bandwidth and communication delay constraints in CPSs, and each weighting fusion matrix is calculated by the analytic form.
A delay-dependent and probability-dependent stability condition is derived such that the fusion estimation error covariance matrix of the DKFE converges to a unique steady-state matrix for any initial values. Under this condition, the steady-state DKFE, which has much lower computational complexity as compared with the DKFE, is given. Moreover, when each communication delay is known, the probability selection criterion for determining dimensionality reduction strategy is presented to guarantee the stability of the DKFE.
Compared with the fusion estimation method in , the model of communication delays in this paper does not require that each sink node knows the communication delay in advance, and the steady-state DKFE with simple calculation is derived (see Remark 1). Since the covariance intersection fusion criterion in  is suboptimal, the estimation performance of the designed DKFE must be better than that of the fusion estimator in  when each communication delay is constant. Moreover, the computation cost of the steady-state DKFE must be lower than that of the CI fusion estimator in  (see Remark 2).
When there is no communication delay for the scenario described in Fig.1, it is shown that the stability condition in this paper has less conservatism than the result in . This is because a new derivation idea without any inequality relaxation is proposed to design the stochastic dimensionality reduction strategy. Moreover, when considering communication delays, the corresponding stability analysis is also based on this new derivation idea. Notice that it is difficult to obtain the stability condition by using the derivation idea in  when the communication delay is modeled in this paper (see Remarks 7-8).
The rest of this paper is organized as follows. Section II presents the problem formulation. The finite-horizon DKFE is designed in Section III. In Section IV, the stability condition and the steady-state DKFE are derived, and the probability selection criteria are given to determine satisfactory compression operators. Two illustrative examples are presented in Section V to show the advantage and effectiveness of the proposed approaches, and then the conclusions are drawn in Section VI.
Notations: The notations used throughout the paper are fairly standard. The superscript represents the transpose, and is the mathematical expectation. represents the identity matrix of size , while stands for a block diagonal matrix. means the occurrence probability of the event , while denotes the trace of the matrix . represent the 2-norm of the matrix . denotes that and are orthogonal vectors, and represents the column vector that is composed of the elements . The symbol is the least common multiple of and , while denotes the rank of the matrix . The function is defined by , and denotes a positive-definite (negative-definite) matrix.
2 Problem Formulation
2.1 Dimensionality Reduction and Communication Delays
Consider the physical process in Fig.1 described by the following discrete state-space model:
where is the state of the process, is the system noise, and is a constant matrix with appropriate dimension. As pointed out in , the model (1) is widely adopted for describing state dynamics of CPSs including power systems, smart grid infrastructures, and building automation systems, etc. When the measurements from each sensor are sent to sink nodes, the sink node’s measurement is modeled by:
where is the measurement matrix with appropriate dimension, and is the measurement noise. Moreover, and are uncorrelated zero-mean Gaussian white noises satisfying
where is defined by:
Then, based on the measurements , the local optimal estimate (LOE) is given by the Kalman filter:
Define . Then, the optimal gain matrix and the local estimation error covariance matrix are calculated by
Under the distributed fusion structure, each LOE must be sent to the FC to design an optimal fusion estimator. However, it is unrealistic to send the complete information included in to the FC over communication networks because almost all communication network can only carry a finite amount of information per unit time. This problem is especially prominent in the fusion estimation for the large-scale CPSs integrated by wireless sensor networks. To reduce communication traffic, only components of the LOE are allowed to be transmitted to the FC at each time, and other components are discarded. Compared with the original LOE , the dimension of the transmitted signal is reduced. In this sense, the above method can be viewed as one of the dimensionality reduction strategies. According to this dimensionality reduction strategy, the allowed sending components (ASC) of has possible cases, where . Then, at a particular time, only one vector signal, which is taken from one group of the above cases, is selected and transmitted to the FC, and this selected signal is denoted by . When is sent to the FC by the sink node, the FC will receive the data packet containing at time because of communication delay. Let denote the local estimation information received by the FC at time . Then, in the FC is given by:
Up to now, the problem of dimensionality reduction and communication delays has been presented, and the process diagram is shown in Fig.2.
It is noted that the signal only takes one element from the following finite set:
where represents one group of ASCs. To characterize the determining process of , we introduce the following indicator functions:
where are required to satisfy
such that only takes one ASC from the set (10) at time , i.e.,
At time , if the fusion estimate of is directly designed based on , the fusion estimation performance must be poor because of the communication delays and the un-transmitted component of . In this case, the compensating state estimate (CSE) of , denoted by , can be modeled as follows:
where is determined by
where means that the component of is selected and sent to the FC, while means that the component of is discarded. Particularly, at time , the compensation strategy in the CSE model (15) is reflected by the following aspects:
The un-transmitted components of are compensated by the one-step prediction based on .
The delayed information is compensated by the -step prediction based on , where .
Remark 1. In , at the sink node, the -step prediction based on the local estimate was given by . Due to the bandwidth constraints, only components of were allowed to be sent. Then, the CSE of , denoted as , was given by (i.e., the model (18) in ):
where the definition of is the same as that of , and denotes the fusion estimate designed by . For the CSE model (18), the -step prediction must be completed at the sink node, which implies that each sink node must know the communication delay from the sink node to the FC in advance. Under this case, when the communication delay is unknown for the sink node or time-varying, the model (18) will be invalid. Different from the modeling method in , the CSE model (15) does not require that each sink node knows the communication delay in advance, and thus the model (15) can be more easily implemented in a practical system. Particularly, when considering the time-varying communication delay , the local estimation information received by the FC, denoted as , is given by:
where denotes the selected ASC at the sink node. Meanwhile, it is reasonable to consider that the time-varying delay is bounded in practical applications, and satisfies . Then, by resorting to the buffers at the FC, each time-varying delay can be prolonged to its upper bound at each time, i.e., the model (19) is reduced to:
Since the structure of (20) is the same as that of (9), the case of time-varying delays can still be modeled by (15). Notice that the CSE model (18) in  will not be applicable to this case, because the time-varying communication delays are only known to the FC, and each sink node impossibly know the time-varying delays a priori. On the other hand, the stability condition in  could only guarantee the MSE of the fusion estimator converged to a steady-state value. It should be pointed out that the computational complexity of the fusion estimator in  is a slightly high, yet the corresponding steady-state fusion estimator cannot be derived from the stability condition in . In contrast, the steady-state DKFE with simple calculation can be designed based on the stability condition in Theorem 3.
Remark 2. For the case of time-varying delays, the estimation error cross-covariance matrices cannot be obtained under the dimensionality reduction strategy in this paper. Fortunately, the covariance intersection (CI) fusion criterion does not need the cross-covariance matrices. Therefore, the distributed CI fusion estimation algorithm was developed in  to deal with the time-varying delays. Notice that the CI fusion criterion is not optimal because the optimization objective is an upper bound of estimation error covariance matrix, and each weighting matrix is obtained by solving non-convex nonlinear optimization problems at each time. Different from the fusion criterion in , the optimal weighted fusion criterion with analytic solutions is used to design the DKFE in this paper. Thus, when considering the constant communication delays, the estimation performance of the DKFE is better than that of the fusion estimator in . On the other hand, as pointed out in Remark 1, the designed fusion estimation algorithms in this paper can be also applicable to the case of time-varying communication delays. However, it is difficult to show whose estimation performance is optimal between the DKFE in this paper and the fusion estimator in  when dealing with time-varying delays. This is because the conservatism in this paper is introduced from the delay model (i.e., prolonging the time-varying delay to its upper bound at each time), while the conservatism in  is introduced from the CI fusion criterion (i.e., minimizing an upper bound of the fusion estimation error covariance). However, from the perspective of computational complexity, the steady-state DKFE in this paper is better than the CI fusion estimator in  whenever considering the constant delays or time-varying delays.
2.2 Problem of Interest
It is concluded from (13) that the selected ASC at the sink node is determined by the binary variables . On the other hand, it is known from (15) that the design of optimal must be completed at the FC, because the communication delay (from the sink node to the FC) and each CSE are only obtained by the FC, but these information are unknown to each sink node. Therefore, an optimal may be difficult to be designed at the sink node. Based on the above consideration, let each binary variable be generated in a random way at the sink node, and let random variables obey the categorical distribution satisfying
Under this case, a group of ASC in the set (10) is randomly selected as the at time . Moreover, the occurrence probabilities of the cases and are given by and , where the selection probability satisfies:
where . This means that when each selection probability is given by (21–22), in (23) will be determined by (24). Notice that the selection probabilities are to be designed in this paper for guaranteeing the stability of the DKFE.
where is determined by the following function:
According to the CSEs in the FC, the DKFE for the addressed CPSs is given by:
where , and combining (27) yields that the DKFE is unbiased if .
Consequently, the problems to be solved in this paper are described as follows:
1) When the selection probabilities satisfying (22) are given in advance, the aim is to design optimal weighting matrices such that the MSE of the DKFE is minimal at each time step, i.e.,
2) Find stability conditions, which are dependent on the communication delay in (9) and the selection probability in (22), such that the estimation error covariance matrix of the DKFE converges to a unique positive matrix, i.e.,
and is independent of the initial values.
Remark 3. When the sink node knows the selection probability in advance, the binary variables obeying the categorical distribution will be randomly generated at each time step, and then the selected ASC can be determined by (13) at the sink node. Under this case, one of the important issues in this paper is how to design the satisfactory probability selection criteria, which will be solved in Section IV. On the other hand, when the result (30) holds, the limit of each weighting matrix must exist, and will be independent of the initial values. This is because the estimation error covariance matrix of the DKFE is dependent on each time-varying matrix . In such a case, the steady-state DKFE with simple calculation will be given in this paper.
3 Finite-Horizon DKFE for the CPSs
In this section, the recursive DKFE will be derived by using the optimal fusion criterion weighted by matrices in the linear minimum variance sense. Define and . Then, from the results in [7, 8], the optimal weighting matrices in (29) and the corresponding fusion estimation error covariance matrix can be calculated by:
where the weighting matrices determined by (31) satisfy the constraint , and
In what follows, six lemmas will be given before deriving the recursive form of . For notational convenience, the following indicator function is introduced:
Meanwhile, if , it will be specified that and , where and represent different matrix functions with respect to the variable .
Lemma 1  For stochastic matrices , , , where
If each random variable in is independent of any random variables of and then
where “” is defined as , and the product “” for the matrices and is defined by
Lemma 2 Define
See A.1 in Appendix.
Lemma 3 Define
Then, , ,