Distributed Cooperative Online Estimation With Random Observation Matrices, Communication Graphs and Time-Delays

# Distributed Cooperative Online Estimation With Random Observation Matrices, Communication Graphs and Time-Delays

## Abstract

We analyze convergence of distributed cooperative online estimation algorithms by a network of multiple nodes via information exchanging in an uncertain environment. Each node has a linear observation of an unknown parameter with randomly time-varying observation matrices. The underlying communication network is modeled by a sequence of random digraphs and is subjected to nonuniform random time-varying delays in channels. Each node runs an online estimation algorithm consisting of a consensus term taking a weighted sum of its own estimate and neighbours’ delayed estimates, and an innovation term processing its own new measurement at each time step. By stochastic time-varying system, martingale convergence theories and the binomial expansion of random matrix products, we transform the convergence analysis of the algorithm into that of the mathematical expectation of random matrix products. Firstly, for the delay-free case, we show that the algorithm gains can be designed properly such that all nodes’ estimates converge to the real parameter in mean square and almost surely if the observation matrices and communication graphs satisfy the stochastic spatial-temporal persistence of excitation condition. Especially, this condition holds for Markovian switching communication graphs and observation matrices, if the stationary graph is balanced with a spanning tree and the measurement model is spatially-temporally jointly observable. Secondly, for the case with time-delays, we introduce delay matrices to model the random time-varying communication delays between nodes, and propose a mean square convergence condition, which quantitatively shows the intensity of spatial-temporal persistence of excitation to overcome time-delays.

Distributed online estimation, cooperative estimation, random graph, random time-delay, mean square convergence, almost sure convergence.

## I introduction

Estimation algorithms have important applications in many fields, e.g. navigation systems, space exploration, machine learning and power systems ([1]-[4]), etc. In a power system, measurement devices such as remote terminal units and phasor measurement units, send the measured active and reactive power flows, bus injection powers and voltage amplitudes to the Supervisory Control and Data Acquisition (SCDA) system, then the voltage amplitudes and phase angles at all buses are estimated for secure and stable operation of the system ([5]-[6]). Generally speaking, there are mainly two categories of estimation algorithms in term of information structure, i.e. centralized and distributed algorithms. In a centralized algorithm, a fusion center is used to collect all nodes’s measurements and gives the global estimate. The centralized information structure heavily relies on the fusion center and lacks robustness and security. In a distributed algorithm, a network of multiple nodes is employed to cooperatively estimate the unknown parameter via information exchanging, where each node is an entity with integrated capacity of sensing, computing and communication, and occasional node/link failures may not destroy the entire estimation task. Hence, distributed cooperative estimation algorithms are more robust than centralized ones ([7]-[8]).

There exist various kinds of uncertainties in real networks. For example, sensors are usually powered by chemical or solar cells, and the unpredictability of cell power leads to random node/link failures, which can be modeled by a sequence of random communication graphs. Besides, node sensing failures or measurement losses ([9]) can be modeled by a sequence of random observation matrices (regression matrices). There are lots of literature on distributed online estimation problems with random graphs. Ugrinovskii [10] studied distributed estimation with Markovian switching graphs. Kar & Moura [11] and Sahu [12] considered distributed estimation with i.i.d. graph sequences, where Kar & Moura [11] showed that the algorithm achieves weak consensus under a weak distributed detectability condition and Sahu [12] proved that the algorithm converges almost surely if the mean graph is balanced and strongly connected. Simões & Xavier [13] proposed a distributed estimation algorithm with i.i.d. undirected graphs and proved that the convergence rate of mean square estimation error is asymptotically equal to that of the centralized algorithm. Distributed cooperative online estimation based on diffusion strategies was addressed in [14]-[18] with spatially-temporally independent observation matrices, i.e. the sequence of observation matrices of each node is an independent random process and those of different nodes are mutually independent. Piggott & Solo [19]-[20] studied distributed estimation with temporally correlated observation matrices and a fixed communication graph. Ishihara & Alghunaim [21] studied distributed estimation with spatially independent observation matrices. Kar  [22] and Kar & Moura [23] proposed consensus+innovation distributed estimation algorithms with random graphs and observation matrices, where the sequences of communication graphs and observation matrices are both i.i.d. and the mathematical expectation of observation matrices needs to be known. They proved that the algorithm converges almost surely if the mean graph is balanced and strongly connected. Zhang & Zhang [24] considered distributed estimation with finite Markovian switching graphs and i.i.d. observation matrices, and proved that the algorithm converges in mean square and almost surely if all graphs are balanced and jointly contain a spanning tree. Zhang [25] proposed a robust distributed estimation algorithm with the communication graphs and observation matrices being mutually independent with each other and both uncorrelated sequences. In summary, most existing literature on distributed cooperative estimation algorithms required balanced mean graphs and special statistical properties of communication graphs and observation matrices, such as i.i.d. or Markovian switching graph sequences, spatially or temporally independent observation matrices with the fixed mathematical expectation, which are also independent of communication graphs.

Besides random communication graphs and observation matrices, random communication delays are also common in real systems ([26]-[28]). Due to congestions of communication links and external interferences, time-delays are usually random and time-varying, whose probability distribution can be approximately estimated by statistical methods. However, to our best knowledge, there has been no literature on distributed online estimation with general random time-varying communication delays. Zhang  [29] and Millán  [30] considered distributed estimation with uniform deterministic time-invariant and time-varying communication delays, respectively, where Millán  [30] established a LMI type convergence condition by the Lyapunov-Krasovskii functional method.

In this paper, we analyze convergence of distributed cooperative online parameter estimation algorithms with random observation matrices, communication graphs and time-delays. Each node’s algorithm consists of a consensus term taking a weighted sum of its own estimate and delayed estimates of its neighbouring nodes, and an innovation term processing its own new measurement at each time step. The sequences of observation matrices, communication graphs and time-delays are not required to satisfy special statistical properties, such as mutual independence and spatial-temporal independence. Furthermore, neither the sample paths of the random graphs nor the mean graphs are necessarily balanced. These relaxations together with the existence of random time-varying time-delays bring essential difficulties to the convergence analysis, and most existing methods are not applicable. For examples, the frequency domain approach ([29],[31]) is only suitable for deterministic uniform time-invariant time-delays, and the Lyapunov-Krasovskii functional method leads to a non-explicit LMI type convergence condition ([30]). Liu [32] and Liu [33] addressed distributed consensus with deterministic time-varying communication delays and i.i.d. communication graphs, whose analysis method relying on the condition of balanced mean graphs, is not applicable to unbalanced mean graphs.

We introduce delay matrices to model the random time-varying communication delays between each pair of nodes. By stochastic time-varying system, martingale convergence theories and the binomial expansion of random matrix products, we transform the convergence analysis of the algorithm into that of the mathematical expectation of random matrix products. Firstly, for the delay-free case, we show that the algorithm gains can be designed properly such that all nodes’ estimates converge to the real parameter in mean square and almost surely if the observation matrices and communication graphs satisfy the stochastic spatial-temporal persistence of excitation condition. Especially, we show that for Markovian switching communication graphs and observation matrices, this condition holds if the stationary graph is balanced with a spanning tree and the measurement model is spatially-temporally jointly observable. Secondly, for the case with time-delays, we propose a mean square convergence condition, which explicitly relies on the conditional expectations of delay matrices, observation matrices and weighted adjacency matrices of communication graphs over a sequence of fixed-length time intervals. This condition quantitatively shows the intensity of spatial-temporal persistence of excitation to overcome additional effects of time-delays. Compared with the existing literature, our contributions are summarized as below.

• The delay-free case

• We show that it is not necessary that the sequences of observation matrices and communication graphs are mutually independent or spatially-temporally independent. Also, the mean graphs are not necessarily time-variant and balanced. We establish the stochastic spatial-temporal persistence of excitation condition under which the distributed cooperative online estimation algorithm with random graphs and observation matrices converges in mean square and almost surely. For a network consisting of completely isolated nodes, the stochastic spatial-temporal persistence of excitation condition degenerates to several independent stochastic persistence of excitation conditions for centralized algorithms.

• Especially, for the case with Markovian switching communication graphs and observation matrices, we prove that the stochastic spatial-temporal persistence of excitation condition holds if the stationary graph is balanced with a spanning tree and the measurement model is spatially-temporally jointly observable, implying that neither local observability of each node nor instantaneous global observability of the entire measurement model is necessary.

• The case with time-delays

• We introduce delay matrices to model the random time-varying time-delays between each pair of nodes. By the method of binomial expansion of random matrix products, we obtain a mean square convergence condition, which explicitly relies on the conditional expectations of the delay matrices, observation matrices and weighted adjacency matrices of communication graphs over a sequence of fixed-length time intervals, and shows that the communication graphs and observation matrices need to be persistently excited with enough intensity to attenuate the random time-delays.

• The nonuniform random time-varying communication delays considered in this paper are more general, and we allow correlated communication delays, graphs and observation matrices.

The rest of the paper is arranged as follows. In Section II, we formulate the problem. In Section III, we describe the distributed cooperative online parameter estimation algorithm with random observation matrices, communication graphs and time-delays. The convergence analysis for the delay-free case and the case with random time-varying time-delays are given in Sections IV and V, respectively. Finally, we conclude the paper and give some future topics in Section VI.

Notation and symbols:
: Hadamard product;
: Kronecker product;
: trace of matrix ;
: 2-norm of matrix ;
: transpose of matrix ;
: probability of event ;
: dimensional identity matrix;
: spectral radius of matrix ;
absolute value of real number ;
: dimensional real vector space;
: is positive semidefinite;
: the largest integer less than or equal to ;
: the smallest integer greater than or equal to ;
: mathematical expectation of random variable ;
: minimum eigenvalue of real symmetric matrix ;
: dimensional column vector with all entries being one;
:  dimensional matrix with all entries being zero;
: , where  is a sequence of real numbers, is a sequence of real positive numbers;  : ;
For a sequence of  dimensional matrices  and a sequence of scalars  , denote

 ΦZ(j,i)={Z(j)⋯Z(i), j≥iIn, j

For any arbitrary nonnegative integers and , denote the Kronecker function by

 Ii,j={1,i=j0,i≠j.

## Ii problem formulation

### Ii-a Measurement model

Consider a network of nodes. Each node is an estimator with integrated capacity of sensing, computing, storage and communication. The estimators/nodes cooperatively estimate an unknown parameter vector via information exchanging. The relation between the measurement vector of estimator and the unknown parameter is represented by

 zi(k)=Hi(k)x0+vi(k), i=1,⋯,N, k≥0. (1)

Here, is the random observation (regression) matrix at time instant with , and is the additive measurement noise. Denote , and . Rewrite (1) by the compact form

 z(k)=H(k)x0+v(k), k≥0. (2)
###### Remark 1.

In many real applicaitons, the relations between the unknown parameter and the measurements can be represented by (1). For examples, in the distributed multi-area state estimation in power systems, the grid is partitioned into multiple geographically non-overlapping areas, and each area is regarded as a node. The grid state to be estimated consists of voltage amplitudes and phase angles at all buses. The measurement of each area/node consists of the active and reactive power flow, bus injection powers and voltage amplitude information measured by remote terminal units and phasor measurement units in the -th area. By the DC power flow approximation ([34]), the grid state degenerates to the voltage phase angles at all buses and the relation between the measurement of each area and the grid state can be represented by (1). In distributed parameter identification, each node’s measurement equation is given by

 zi(k)=n∑j=1cjzi(k−j)+vi(k)=[zi(k−1),⋯,zi(k−n)][c1,⋯,cn]T+vi(k).

For this case, the unknown parameter and the observation matrix (generally called regressor) is an dimensional row vector. In addition, sensing failures in real networks can be modeled by a Markov chain or an i.i.d. sequence of Bernoulli variables . Then , where is the sequence of observation matrices without sensing failures.

### Ii-B Communication models

Assume that there exist nonuniform random time-varying communication delays for the communication links between each pair of nodes. We use a sequence of random variables , to represent the time-delays associated with the link from node to node , where the positive integer  represents the maximum time-delay. This sequence is subjected to the discrete probability distribution

 P{λji(k) =q}=pji,q(k) with d∑q=0pji,q(k)=1. (3)

We stipulate that , , . Denote the dimensional matrices , , called delay matrices. By the definition of Kronecker function, we know that for each , is a sequence of random matrices and its sample paths are sequences of  matrices. By (3), we know that  and

 d∑q=0I(k,q)=1N1TN a.s. (4)

We use a sequence of random communication graphs , , to describe the possible link failures among nodes, where is the node set and is the weighted adjacency matrix of the communication graph in which  a.s. for all and and if and only if the link from node to node exists at time instant for all . The neighborhood of node is . The degree matrix of the graph is  and the Laplacian matrix of the graph is  ([35]). Let

 ¯¯¯¯A(k,q)=(AG(k)∘I(k,q))⊗In. (5)

Then, by (4) and the above, we have

 d∑q=0¯¯¯¯A(k,q)=AG(k)⊗In. (6)

## Iii distributed cooperative online estimation algorithm

Let  be the estimate by node for the unknown parameter at time instant . Starting at the initial estimate , at any time instant , node  takes a weighted sum of its own estimate and delayed estimates received from its neighbours, and then adds a correction term based on the local measurement information (innovation) to update the estimate . Specifically, the distributed cooperative online parameter estimation algorithm with random observation matrices, communication graphs and time-delays is given by

 xi(k+1) = xi(k)+a(k)HTi(k)(zi(k)−Hi(k)xi(k)) (8) +b(k)∑j∈Ni(k)aij(k)(xj(k−λji(k))−xi(k)), i∈V, k≥0,

where and are called the innovation gain and the consensus gain, respectively.

Denote the fileds , , with . For the algorithm (8), we have the following assumptions.

A1.a The sequence is independent of , , , and , , .

A1.b The sequence  is a martingale difference sequence and there exists a constant  such that

A2.a

A2.b There exist positive constants  and such that  and

A3.a and are positive real sequences monotonically decreasing to zero, satisfying , for any given positive integer , , and

A3.b .

A3.c

 supk≥0b(k)

where the constant satifies , .

###### Remark 2.

Note that, in Assumption A1.a, neither mutual independence nor spatial-temporal independence is assumed on the observation matrices, communication graphs and time-delays.

###### Remark 3.

It is easy to find , and , satisfying Assumptions A3.a and A3.b. If , , , , then both Assumptions A3.a and A3.b hold.

By the definition of , we know that . Then by (8), we have

 xi(k+1) = xi(k)+a(k)HTi(k)[zi(k)−Hi(k)xi(k)] (10) +b(k)∑j∈Ni(k)aij(k)[d∑q=0xj(k−q)Iλji(k),q−xi(k)], i∈V.

Denote and . By (5), rewrite (10) as

 x(k+1) = [INn−b(k)DG(k)⊗In−a(k)HT(k)H(k)]x(k) (12) +b(k)d∑q=0¯¯¯¯A(k,q)x(k−q)+a(k)HT(k)z(k).

Denote the overall estimation error vector . Note that . By (2) and (6), subtracting on both sides of (12) leads to

 e(k+1) =[INn−b(k)DG(k)⊗In−a(k)HT(k)H(k)]x(k)+b(k)d∑q=0¯¯¯¯A(k,q)x(k−q) +a(k)HT(k)z(k)−1N⊗x0 =[INn−b(k)DG(k)⊗In−a(k)HT(k)H(k)](x(k)−1N⊗x0+% 1N⊗x0) +b(k)d∑q=0¯¯¯¯A(k,q)(x(k−q)−1N⊗x0+1N⊗x0)+a(k)HT(k)z(k)−1N⊗x0 =[INn−b(k)DG(k)⊗In−a(k)HT(k)H(k)](e(k)+1N⊗x0) +b(k)d∑q=0¯¯¯¯A(k,q)(e(k−q)+1N⊗x0)+a(k)HT(k)z(k)−1N⊗x0 =[INn−b(k)DG(k)⊗In−a(k)HT(k)H(k)]e(k) −(b(k)DG(k)⊗In+a(k)HT(k)H(k))(1N⊗x0) +b(k)d∑q=0¯¯¯¯A(k,q)e(k−q)+b(k)d∑q=0¯¯¯¯A(k,q)(1N⊗x0)+a(k)HT(k)z(k) =[INn−b(k)DG(k)⊗In−a(k)HT(k)H(k)]e(k)−a(k)HT(k)H(k)(1N⊗x0)

Noting that , by the above, we obtain the overall estimation error equation

 e(k+1) = [INn−b(k)DG(k)⊗In−a(k)HT(k)H(k)]e(k) (14) +b(k)d∑q=0¯¯¯¯A(k,q)e(k−q)+a(k)HT(k)v(k), k≥0.

## Iv the delay-free case

In this section, we give the convergence conditions of the algorithm (8) for the delay-free case, i.e., , a.s. , . All proofs of this section are put in Appendix B.

Denote . Specifically, if  is balanced, then is the Laplacian matrix of the symmetrized graph of , ([36]). For any given positive integers  and , denote

 λhm=λmin[(m+1)h−1∑k=mh(b(k)a(k)E[ˆLG(k)|F(mh−1)]⊗In+E[HT(k)H(k)|F(mh−1)])].
###### Theorem IV.1.

If Assumptions A1.a, A1.b and A3.a hold, and there exist a positive integer  and positive constants and  such that

 (b.1)  infm≥0λhm≥θ>0 a.s.; (b.2)  supk≥0[E[(∥LG(k)∥+∥HT(k)H(k)∥)2max{h,2}|F(k−1)]]12max{h,2}≤ρ0 a.s.,

then the algorithm (8) converges in mean square, i.e.,

###### Theorem IV.2.

If the conditions in Theorem IV.1 hold and Assumptions A2.a and A3.b hold, then the algorithm (8) converges almost surely, i.e.,

###### Remark 4.

The condition (b.1) in Theorems IV.1 and IV.2 is the key convergence condition. We call it the stochastic spatial-temporal persistence of excitation condition, where “spatial-temporal” represents the reliance of the condition on all nodes’ observation matrices and communication graphs (spatial dimension) over a sequence of fixed-length time intervals (temporal dimension) and “persistence of excitation” represents that the minimum eigenvalues of matrices consisting of spatial-temporal observation matrices and Laplacian matrices are uniformly bounded away from zero. Guo [37] considered centralized estimation algorithms with random observation matrices and proposed the “stochastic persistence of excitation” condition to ensure convergence. The condition (b.1) can be regarded as the generalization of “stochastic persistence of excitation” condition in [37] to that for distributed algorithms. For a network with isolated nodes, a.s., and the condition (b.1) degenerates to independent “stochastic persistence of excitation” conditions.

###### Remark 5.

Most existing literature on distributed estimation required balanced mean graphs ([22],[24]). Here, the condition (b.1) may still holds even if the mean graphs are unbalanced. For example, consider a simple fixed graph with and let and . Obviously, is unbalanced. By some direct calculations, we have , which implies the condition (b.1).

In the most existing literature, it was also required that the sequence of observation matrices is i.i.d. and independent of the sequence of communication graphs, neither of which is necessary in Theorems IV.1 and IV.2. Subsequently, we further give more intuitive convergence conditions for Markovian switching communication graphs and observation matrices, as stated in the following assumption.

A4 is a homogeneous and uniform ergodic Markov chain with a unique stationary distribution .
Here, with , where is the state space of observation matrices of node and being the state space of the weighted adjacency matrices, , , , and with representing .

###### Corollary IV.1.

If Assumptions A1.a, A1.b, A3.a, A3.b and A4 hold, , , and

(c.1) the stationary weighted adjacency matrix is nonnegative and its associated graph is balanced with a spanning tree;

(c.2) the measurement model (1) is spatially-temporally jointly observable, i.e.,

 λmin(N∑i=1(∞∑l=1πlHTi,lHi,l))>0, (15)

then the algorithm (8) converges in mean square and almost surely, i.e., , and

###### Remark 6.

Most of the existing distributed estimation algorithms used the mathematical expectation of observation matrices which is restricted to be time-invariant and difficult to be obtained ([22],[24]). They required instantaneous global observability in the statistical sense for the measurement model, i.e., is positive definite, where is a fixed matrix with , for all , . Differently, we only use the sample paths of observation matrices in the algorithm (8). The mathematical expectations of observation matrices are allowed to be time-varying. We prove that for homogeneous and uniform ergodic Markovian switching observation matrices and communication graphs, the stochastic spatial-temporal persistence of excitation condition holds if the stationary graph is balanced with a spanning tree and the measurement model is spatially-temporally jointly observable, that is, (15) holds, implying that neither local observability of each node, i.e. , , nor instantaneous global observability of the entire measurement model, i.e. , , is needed.

## V the case with random Time-varying communication delays

In this section, we further analyze the convergence of the algorithm (8) with random observation matrices, communication graphs and time-delays simultaneously. All proofs of this section are put in Appendix C.

The random time-varying communication delays bring about that the mean square convergence analysis of the algorithm becomes very difficult. To this end, we transform (14) into the following equivalent system ([32]-[33]).

 r(k+1) = F(k)r(k)+g(k), (16) g(k) = d∑q=1Cq(k)g(k−q)+a(k)HT(k)v(k), (17)

where , , , satisfy

 F(k)+C1(k)=INn−b(k)DG(k)⊗In−a(k)HT(k)H(k)+b(k)¯¯¯¯A(k,0), (18) C1(k)F(k−1)−C2(k)=−b(k)¯¯¯¯A(k,1), (19) C2(k)F(k−2)−C3(k)=−b(k)¯¯¯¯A(k,2), (20) ⋮ (21) Cd−1(k)F(k−d+1)−Cd(k)=−b(k)¯¯¯¯A(k,d−1), (22) Cd(k)F(k−d)=−b(k)¯¯¯¯A(k,d). (23)

Let , and , . It can be verified that if , , then , , i.e. the system (14) and the system (16)-(18) are equivalent.

We first establish a lemma as the basis of convergence analysis.

###### Lemma V.1.

If Assumptions A2.b and A3.c hold, then there exists a constant  such that is invertible a.s. and  a.s., .

If Assumptions A2.b and A3.c hold, then is invertible a.s. by Lemma V.1. Then by (18), we have

 F(k) = INn−b(k)DG(k)⊗In−a(k)HT(k)H(k)+b(k)¯¯¯¯A(k,0)−C1(k) (24) = INn−b(k)DG(k)⊗In−a(k)HT(k)H(k)+b(k)¯¯¯¯A(k,0) (27) −(C2(k)−b(k)¯¯¯¯A(k,1))F−1(k−1) ⋮ = INn−[b(k)DG(k)⊗In+a(k)HT(k)H(k) (29) −b(k)d∑q=0¯¯¯¯A(k,q)[ΦF(k−1,k−q)]−1]=INn−G(k), k≥0,

where

 (30)

For any given positive integers  and , denote

 λhm′=λmin[(m+1)h−1∑k=mh(2b(k)a(k)E[ˆLG(k)|F(mh−1)]⊗In+2E[HT(k)H(k)|F(mh−1)] (31) −b(k)a(k)d∑q=0E[¯¯¯¯A(k,q)[[ΦF(k−1,k−q)]−1−INn]|F(mh−1)] (32) −b(k)a(k)d∑q=0E[[[ΦF(k−1,k−q)]−1−INn]T¯¯¯¯AT(k,q)|F(mh−1)])]. (33)
###### Theorem V.1.

If Assumptions A1.a, A1.b, A2.b, A3.a and A3.c hold, and there exist a positive integer  and a constant  such that , then the algorithm (8) converges in mean square, i.e. .

For any given positive integers  and , denote

 Δhm = (m+1)h−1∑k=mhb(k)a(k)[d∑q=0∥E[¯¯¯¯A(k,q)([ΦF(k−1,k−q)]−1−INn)|F(mh−1)]∥].

Subsequently, we present a corollary which reflects the impact of communication delays more intuitively.

###### Corollary V.1.

If Assumptions A1.a, A1.b, A2.b, A3.a and A3.c hold and there exist a positive integer  and a constant such that , then the algorithm (8) converges in mean square, i.e. .

###### Remark 7.

Theorem V.1 gives an explicit convergence condition under which all nodes’ estimates converge to the real parameter in mean square. Existing literature used the Lyapunov-Krasovskii functional method to deal with time-delays and obtained the non-explicit LMI type convergence condition ([30]). In this section, we transform the system with random time-varying communication delays into an equivalent delay-free system by introducing an auxiliary system and then adopt the method of binomial expansion of random matrix products to transform the mean square convergence analysis of the delay-free system into that of the mathematical expectation of random matrix products, and obtain the key convergence condition which explicitly relies on the conditional expectations of delay matrices, observation matrices and weighted adjacency matrices of communication graphs over a sequence of fixed-length time intervals. In Corollary V.1, we further obtain the more intuitive convergence condition which shows that the communication graphs and observation matrices need to be persistently excited with enough intensity to attenuate additional effects of time-delays. When time-delays don’t exist, these conditions both degenerate to the stochastic spatial-temporal persistence of excitation condition in Theorem IV.1.

## Vi conclusion

In this paper, we analyzed the convergence of the distributed cooperative online parameter estimation algorithm in an uncertain environment. Each node has a partial linear observation of the unknown parameter with random time-varying observation matrices. The underlying communication network is modeled by a sequence of random digraphs and is subjected to nonuniform random time-varying delays in channels. For the delay-free case, we proved that if the observation matrices and the graph sequence satisfy the stochastic spatial-temporal persistence of excitation condition, then the algorithm gains can be designed properly such that all nodes’ estimates converge to the real parameter in mean square and almost surely. Specially, for Markovian switching communication graphs and observation matrices, this condition holds if the stationary graph is balanced with a spanning tree and the measurement model is spatially-temporally jointly observable. For the case with communication delays, we introduced delay matrices to model the random time-varying communication delays, adopted the method of binomial expansion of random matrix products to transform the mean square convergence analysis of the algorithm into that of the mathematical expectation of random matrix products, and obtained mean square convergence conditions explicitly relying on the conditional expectations of delay matrices, observation matrices and weighted adjacency matrices of communication graphs over a sequence of fixed-length intervals and showing that the communication graphs and observation matrices need to be persistently excited with enough intensity to attenuate additional effects of time-delays. Furthermore, when time-delays don’t exist, these conditions degenerate to the stochastic spatial-temporal persistence of excitation condition obtained for the delay-free case.

Future topics may include generalizing this work to case with asynchronous measurements and communication, the case with input delays and communication noises. Meanwhile, the convergence rate analysis is also an interesting topic for future investigation.

## Appendix A several useful lemmas

###### Definition A.1.

([38]) A Markov chain on a countable state space with a stationary distribution , and transition function is called uniform ergodic, if there exist positive constants and such that for all ,

 ∥Pn(x,⋅)−π∥≤Rr−n.

Here, .

###### Lemma A.1.

([39]) For any given matrix , denote . If there exists a constant  such that , then  is invertible and

###### Lemma A.2.

([40]) Assume that  and are real sequences satisfying , and exists. Then

 limk→∞k∑i=1s1(i)k∏l=i+1(1−s2(l))=limk→∞s1(k)s2(k).
###### Lemma A.3.

([41]) Assume that are all nonnegative adaptive sequences, satisfying

 E[x(k+1)|F<