Distributed Estimation of Dynamic Parameters : Regret Analysis
Abstract
This paper addresses the estimation of a timevarying parameter in a network. A group of agents sequentially receive noisy signals about the parameter (or moving target), which does not follow any particular dynamics. The parameter is not observable to an individual agent, but it is globally identifiable for the whole network. Viewing the problem with an online optimization lens, we aim to provide the finitetime or nonasymptotic analysis of the problem. To this end, we use a notion of dynamic regret which suits the online, nonstationary nature of the problem. In our setting, dynamic regret can be recognized as a finitetime counterpart of stability in the meansquare sense. We develop a distributed, online algorithm for tracking the moving target. Defining the pathlength as the consecutive differences between target locations, we express an upper bound on regret in terms of the pathlength of the target and network errors. We further show the consistency of the result with static setting and noiseless observations.
I Introduction
Distributed detection, learning and estimation has been a main topic of interest in the past three decades [1, 2, 3, 4]. With widespread applications in sensor, robotic, economic and social networks, distributed algorithms have received a considerable attention in science and engineering [5, 6]. In these scenarios, a group of agents aim to learn or estimate the value of a parameter. Each individual agent receives partially informative data about the parameter; however, the global spread of information in the network allows agents to accomplish the task collaboratively. Many of these information aggregation methods use consensus protocols as a crucial component [7, 8]. Distributed algorithms are popular for their ability to handle large data sets, low computation burden on agents and robustness to node failures.
On the other hand, online learning and optimization has been intensively studied in the literature of machine learning [9, 10], proving to be a powerful tool to model sequential decision problems. The problem can be modeled as a game between a learner and an adversary. The learner sequentially selects actions, and the adversary reveals the corresponding losses to the learner. The term online refers to the fact that the learner receives data in a sequential fashion. The popular performance metric for online algorithms is called regret. Regret measures the performance of algorithm versus a predefined benchmark. For instance, the benchmark could be the best fixed action had the learner known all the losses in advance. In a broad sense, when the benchmark is a fixed sequence, the regret is called static, whereas a timevarying benchmark sequence brings forward the notion of dynamic regret [10, 11, 12].
The goal of this paper is to develop a distributed, online algorithm for tracking timevarying parameters. To this end, a unifying observation is to view the distributed estimation problem as an online optimization. A network of agents aim to track a moving target which is only partially observable to each agent. The parameter can represent a sequence of highdimensional data where agents only receive a lowdimensional version of it. Therefore, they must communicate with each other to track the target. Each time agents form their estimates of the parameter, the whole network incurs a loss (network loss). We formalize the problem as an instance of distributed, online optimization whose dynamic regret can be characterized in the general form of
\displaystyle\textbf{Regret}=\text{Decentralized Loss  Centralized Loss}. 
In other words, the regret compares the performance of the decentralized algorithm versus its centralized counterpart.
Our main contribution is to provide the finitetime or nonasymptotic analysis of the problem using the notion of regret. In this context, regret can also be interpreted as a finitetime analog of stability in meansquare sense. In contrast to existing approaches, we do not restrict the dynamics of the parameter, i.e., the target is allowed to move along an arbitrary trajectory. The freedom given to the dynamics broadens the application of the problem. For instance, the parameter can represent the price of a product which does not have a particular pattern, or the location of a vehicle that moves along an unpredictable trajectory. In turn, our estimation update is akin to consensus+innovation updates in the literature with an important distinction that it does not assume any dynamics on the parameter.
We then characterize the behavior of regret in finitetime regime. To this end, we quantify the pathlength in terms of the consecutive differences between target locations, and prove that the regret can be bounded in terms of the pathlength of the target. Interestingly, our regret bound interpolates smoothly from static to dynamic setting. That is, when the parameter is static, we can recover the corresponding results with static regret (with appropriate tuning of problem parameters). We further simplify the bound in two cases: static target and noiseless observation.
Related Work
There exists a large body of literature on estimation of dynamic parameters. These works mostly assume that the parameter follows a known dynamics, say, the model can be represented as an LTI system. Perhaps the most classical example is the celebrated Kalman filtering [13]. The elegance of Kalman filter has motivated a lot of researchers to investigate the problem in the context of networks. We cannot hope to do the justice to the extensive literature on distributed Kalman filtering, and refer the avid reader to a series of works on this topic[14, 15, 16, 17]. The works of [18, 19, 20, 21] are also in the same spirit in which the parameter follows linear dynamics. Acemoglu et al. [18] present a ruleofthumb learning rule, and provide the asymptotic behavior of their update. In [19], Khan et al. investigate the tradeoff between stability of the linear dynamics and connectivity of the network. They show that their update can potentially track unstable linear models driven by noise. Similarly, an algorithm consisting of pseudoinnovations is developed in [20]. Moreover, the authors of [21] consider a scalar linear model, and characterize an explicit expression for the meansquare deviation. Restricting their attention to noiseless case, De et al. pose an inverse problem in [22]. In their model, the parameter can be observed whereas the dynamics is unknown and must be estimated. On the other hand, Atanasov et al. [23] propose a distributed, linear estimator in wireless sensor networks which encompasses an auxiliary localization procedure. The authors prove the meansquare consistency of the joint localization and target estimation algorithm. Moreover, Sayin et al. [24] present a distributed, online algorithm for state estimation applicable to Big Data. The authors of [25] develop an optimizationbased, predictioncorrection method for tracking the trajectory of a moving target. Finally, our work is also related to [26], where the problem is considered in the context of social networks.
Organization
The rest of this paper is organized as follows. In the next section we introduce the notation, and formalize the estimation problem using the notion of regret. In Section III, we present our technical results and their consequences. Notably, we demonstrate a pathlength bound on regret. Section IV consists of concluding remarks, and in Section V we provide the proof of our technical results.
II Problem Formulation
IIA Nomenclature
We adhere to the following notation in the exposition of our results:
[n]  The set \{1,2,...,n\} for any integer n 

x^{\top}  Transpose of the vector x 
I_{d}  Identity matrix of size d\times d 
\\cdot\  The spectral norm operator 
\mathbbm{1}_{d}  Vector of all ones with dimension d 
\mathbf{0}_{d}  Vector of all zeros with dimension d 
\lambda_{i}(P)  The ith largest eigenvalue of matrix P 
\rho(P)  The spectral radius of matrix P 
Throughout, the variables in bold are random, and all the vectors are in column form. When referring to vector of all ones or zeros, sometimes the dimension is omitted and can be inferred from the context.
IIB Observation Model and Regret Definition
Our objective is to track a timevarying parameter that is not confined to a certain dynamics. The parameter could represent the location of a moving target following an arbitrary trajectory. We denote the parameter by \theta_{t}\in\mathbbm{R}^{d} at each time t\in[T]. A network of n agents collaborate with each other to track the parameter which is only partially observable to each agent. More formally, the observation model for agent i\in[n] can be expressed in the following form
\displaystyle\mathbf{y}_{i,t}=H_{i}\theta_{t}+\mathbf{w}_{i,t},  (1) 
where H_{i}\in\mathbbm{R}^{m_{i}\times d} and \mathbf{w}_{i,t}\in\mathbbm{R}^{m_{i}} denote the observation matrix and noise, respectively. For instance, \theta_{t} can be a highdimensional data where agents only observe a lowdimensional version of that, i.e. m_{i} might be much smaller than d. We respect the standard assumption of zeromean and finitecovariance for the observation noise. That is, we assume that \mathbbm{E}\left[\mathbf{w}_{i,t}\right]=\mathbf{0} and \mathbbm{E}\left[\mathbf{w}_{i,t}^{\top}\mathbf{w}_{i,t}\right]=W_{i}, respectively. We further assume the independence over time and space; that is, \mathbbm{E}\left[\mathbf{w}_{i,t}^{\top}\mathbf{w}_{j,s}\right]=0 for i\neq j or s\neq t.
Note that when the parameter is static, i.e. when \theta_{t}=\theta, we recover the classical distributed estimation problem (see e.g. [2]). Though not locally observable to each agent, we assume that the parameter is globally identifiable from the standpoint of all agents in the network.
Assumption 1
The sequence \{\theta_{t}\}_{t=1}^{T} is globally identifiable, i.e., it holds that the matrix
\displaystyle H=\frac{1}{n}\sum_{i=1}^{n}H_{i}^{\top}H_{i}, 
is invertible.
The estimation procedure follows an online protocol. At time t, each agent i forms an estimate \hat{\boldsymbol{\theta}}_{i,t} of \theta_{t} based on observations \{\mathbf{y}_{i,\tau}\}_{\tau=1}^{t1}. After that, the new signal \mathbf{y}_{i,t} becomes available to the agent. The online nature of our estimation problem allows us to pose it as an instance of online optimization. Therefore, before deriving the explicit update for \hat{\boldsymbol{\theta}}_{i,t}, we need to introduce a few notions based on the literature of online optimization. Let us define a local square loss
\displaystyle\ell_{i,t}(\theta):=\mathbbm{E}\left[\left\\mathbf{y}_{i,t}H_{i% }\theta\right\^{2}\right],  (2) 
for each agent i\in[n], followed by the network loss as
\displaystyle\ell_{t}(\theta):=\sum_{i=1}^{n}\mathbbm{E}\left[\left\\mathbf{y% }_{i,t}H_{i}\theta\right\^{2}\right].  (3) 
Note that Assumption 1 guarantees that \ell_{t}(\theta) has a unique minimizer at each time t. Agents aim to minimize the accumulated network loss over time. Equivalently, the goal of the network is to minimize the regret defined as
\displaystyle\mathbf{Reg}_{T}:=\frac{1}{T}\sum_{t=1}^{T}\bigg{(}\frac{1}{n}% \sum_{j=1}^{n}\ell_{t}(\hat{\boldsymbol{\theta}}_{j,t})\ell_{t}(\theta_{t})% \bigg{)}.  (4) 
The regret quantifies the difference between the realized and ideal network loss: the estimation \hat{\boldsymbol{\theta}}_{j,t} of each agent j is evaluated at the function \ell_{t}(\cdot), and the average (among agents) is subtracted by the loss of exact, ideal estimate \theta_{t}. Evidently, the ideal estimate (solution to the network loss (3)) is not available to an individual agent; hence, one can also interpret the regret as the difference between an algorithm that receives local loss (decentralized) versus another algorithm that accesses the network loss (centralized).
IIC Network Model and Estimation Update
The interactions of agents, which in turn defines the network, is captured with the matrix P. Formally, we denote by [P]_{ij}, the ijth entry of the matrix P. When [P]_{ij}>0, agent i communicates with agent j. We assume that P is symmetric, stochastic with positive diagonal elements. The assumption simply guarantees the information flow in the network. Alternatively, from technical point of view, we respect the following hypothesis.
Assumption 2
We assume the Markov chain P is irreducible and aperiodic, and P^{\top}=P.
The assumption implies that the Markov chain P has a unique stationary distribution i.e., \mathbbm{1}^{\top}P=\mathbbm{1}^{\top} is the unique (unnormalized) left eigenvector corresponding to \lambda_{1}(P)=1. It also entails that \lambda_{1}(P) is unique, and the other eigenvalues of P are less than unit in magnitude [27].
To follow the trajectory of the parameter, we propose an update rule which can be cast as a distributed variant of Online Gradient Descent [10] with noisy feedback. It takes the form
\displaystyle\hat{\boldsymbol{\theta}}_{i,t}=\sum_{j=1}^{n}[P]_{ij}\hat{% \boldsymbol{\theta}}_{j,t1}+\alpha H_{i}^{\top}\left(\mathbf{y}_{i,t1}H_{i}% \hat{\boldsymbol{\theta}}_{i,t1}\right),  (5) 
where \alpha\in\mathbbm{R} is the step size. The update is akin to consensus+innovation updates in the literature (see e.g. [19, 28]) with a distinction that it does not assume any dynamics on the parameter \theta_{t}. The consensus part forces agents to keep their estimates close to each other, and the innovation part takes into account the new observation. We can now shed light on our previous interpretation of regret: had the agents known the network loss \ell_{t}(\theta), they could have formed estimates based on Online Gradient Descent (which is a centralized algorithm). Since agents only know the local loss (2), regret (4) is comparing the distributed algorithm versus its centralized counterpart.
We further define the pathlength of the moving target as
\displaystyle C_{T}:=\sum_{t=1}^{T}\\theta_{t}\theta_{t1}\^{2},  (6) 
which indicates how much the parameter drifts over time. In particular, when bounding the regret (4), the pathlength C_{T} becomes useful. As we shall explore in the next section, there are connections between meansquare stability and regret; hence, we close this section by defining the meansquare stability.
Definition 1
We say the process is stable in the meansquare sense, whenever
\displaystyle\frac{1}{n}\sum_{i=1}^{n}\mathbbm{E}\left[\big{\}\hat{% \boldsymbol{\theta}}_{i,t}\theta_{t}\big{\}^{2}\right]\longrightarrow\sigma, 
where \sigma\geq 0 is a finite constant.
III Main Results
In this section, we provide our technical results and their consequences. More specifically, our objective is to find an anytime bound on the regret involving the pathlength of target. We further interpret the bound in several circumstances.
IIIA Error Process and Stability
Stability (in mean or meansquare sense) is closely attached to the behavior of estimation error. Let us define the local error process as follows,
\displaystyle\mathbf{e}_{i,t}:=\hat{\boldsymbol{\theta}}_{i,t}\theta_{t}.  (7) 
Stacking the local errors in a vector, we represent the global error by
\displaystyle\mathbf{e}_{t}:=[\mathbf{e}_{1,t},\ldots,\mathbf{e}_{n,t}]^{\top}.  (8) 
In the following lemma, we show that the error process can be represented as an LTI system.
Lemma 1
The error process (8) can be characterized via an LTI system as follows
\displaystyle\mathbf{e}_{t}=Q\mathbf{e}_{t1}+\mathbf{u}_{t}, 
where
\displaystyle Q:=P\otimes I_{d}\alpha~{}\textit{{diag}}\left[H_{1}^{\top}H_{1% },\ldots,H_{n}^{\top}H_{n}\right], 
and
\displaystyle\mathbf{u}_{t}:=\mathbbm{1}_{n}\otimes(\theta_{t1}\theta_{t})+% \alpha~{}\left[\begin{array}[]{ccc}H_{1}^{\top}\mathbf{w}_{1,t1}\\ \vdots\\ H_{n}^{\top}\mathbf{w}_{n,t1}\end{array}\right]. 
We observe from Lemma 1 that to ensure mean stability (to have \mathbbm{E}[\mathbf{e}_{t}]\rightarrow\mathbf{0} as t\rightarrow\infty), the conditions
\displaystyle\rho(Q)<1\ \ \ \ \ \text{and}\ \ \ \ \ \ \mathbbm{E}[\theta_{t}% \theta_{t1}]\rightarrow\mathbf{0}, 
need to be satisfied. To push the spectral radius of Q inside the unit circle, we can simply tune the step size \alpha. We quantify this condition in Section IIIC, and show the role of global identifiability (Assumption 1) and connectivity of network (Assumption 2) in existence of such \alpha.
However, the second condition entirely depends on the problem environment. For instance, the condition is evidently satisfied in the static case (\theta_{t}=\theta). It also holds when \theta_{t} is generated randomly based on a stationary distribution, i.e. whenever \mathbbm{E}[\theta_{t}]=\theta. Regardless of stability, we can always characterize the regret behavior in terms of pathlength of the target.
IIIB A PathLength Bound for Regret
In this section, we analyze the regret (4), which is a measure of finitetime performance. The following lemma exhibits the connection of meansquare stability (Definition 1) and regret (4). In particular, the lemma is used to bound the regret in terms of pathlength of the target (6).
Lemma 2
The regret defined in (4) can be bounded as,
\displaystyle\mathbf{Reg}_{T}\leq\frac{1}{nT}\sum_{t=1}^{T}\sum_{i=1}^{n}\left% \H_{i}\right\^{2}\mathbbm{E}\left[\left\\mathbf{e}_{t}\right\^{2}\right]. 
Lemma 2 proves that stability implies boundedness of asymptotic regret since
\displaystyle\mathbbm{E}\left[\left\\mathbf{e}_{t}\right\^{2}\right]=\sum_{i% =1}^{n}\mathbbm{E}\left[\big{\}\hat{\boldsymbol{\theta}}_{i,t}\theta_{t}\big% {\}^{2}\right], 
and once the limit exists, the Cesàro mean preserves it. However, we are interested in nonasymptotic analysis of the problem. We present our finitetime statement in the subsequent theorem.
Theorem 1
Let the sequence \{\theta_{t}\}_{t=1}^{T} be globally identifiable (Assumption 1), and the Markov chain P be irreducible and aperiodic (Assumption 2). If each agent i\in[n] generates the estimate sequence \{\hat{\boldsymbol{\theta}}_{i,t}\}_{t=1}^{T} according to the update rule (5), the regret satisfies
\displaystyle\mathbf{Reg}_{T}  \displaystyle\leq\frac{1}{n}\sum_{i=1}^{n}\left\H_{i}\right\^{2}\frac{\alpha% ^{2}\sum_{i=1}^{n}\H_{i}\^{2}W_{i}}{1\Q\}  
\displaystyle~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}+\frac{1}{T% }\sum_{i=1}^{n}\left\H_{i}\right\^{2}\frac{C_{T}}{(1\Q\)^{2}}. 
Theorem 1 provides a pathlength bound for regret. The underlying intuition behind the term C_{T} is as follows: as agents decide on the next value based on previous observations (online prediction), they are always one step behind in estimation. Even when the observations are noiseless (W_{i}=0 for all i\in[n]), the second term still remains, which is an artifact of agent i using \mathbf{y}_{i,t1} to predict \theta_{t}.
We remark that pathlength regret bounds were previously studied in the context of online optimization for centralized frameworks (see e.g. [10, 11, 12]). Here, we specialized to quadratic losses, and proved a pathlength bound in distributed setting. Therefore, our bound involves an additional network penalty comparing to the centralized framework.
IIIC Tuning the Step Size
We now discuss several aspects of the regret bound derived in Theorem 1. To this end, we need to clarify the dependence of \Q\ on step size \alpha to analyze the bound. Recall the closedform of Q in Lemma 1 as well as stochasticity of P which implies P\mathbbm{1}_{n}=\mathbbm{1}_{n}; pre/post multiply Q by a vector of all ones with dimension nd (\mathbbm{1}=\mathbbm{1}_{n}\otimes\mathbbm{1}_{d}) to get
\displaystyle\mathbbm{1}^{\top}Q\mathbbm{1}  \displaystyle=\mathbbm{1}^{\top}\left(P\otimes I_{d}\alpha~{}\textit{{diag}}% \left[H_{1}^{\top}H_{1},\ldots,H_{n}^{\top}H_{n}\right]\right)\mathbbm{1}  
\displaystyle=\mathbbm{1}^{\top}\left(\mathbbm{1}\alpha~{}\textit{{diag}}% \left[H_{1}^{\top}H_{1},\ldots,H_{n}^{\top}H_{n}\right]\mathbbm{1}\right)  
\displaystyle=\mathbbm{1}^{\top}\mathbbm{1}\alpha~{}\mathbbm{1}^{\top}\textit% {{diag}}\left[H_{1}^{\top}H_{1},\ldots,H_{n}^{\top}H_{n}\right]\mathbbm{1}  
\displaystyle=\mathbbm{1}^{\top}\mathbbm{1}n\alpha~{}\mathbbm{1}_{d}^{\top}H% \mathbbm{1}_{d}, 
as H=\frac{1}{n}\sum_{i=1}^{n}H_{i}^{\top}H_{i}. Therefore, since \mathbbm{1}^{\top}\mathbbm{1}=nd, we obtain
\displaystyle\frac{\mathbbm{1}^{\top}Q\mathbbm{1}}{\mathbbm{1}^{\top}\mathbbm{% 1}}  \displaystyle=1n\alpha~{}\frac{\mathbbm{1}_{d}^{\top}H\mathbbm{1}_{d}}{% \mathbbm{1}^{\top}\mathbbm{1}}=1\alpha\frac{\mathbbm{1}_{d}^{\top}H\mathbbm{1% }_{d}}{\mathbbm{1}_{d}^{\top}\mathbbm{1}_{d}}. 
In view of Assumption 2, \lambda_{1}(P)=1 is unique, and no direction other than \mathbbm{1}_{n} can recover the trivial eigenvalue. Hence, depending on the null space of \textit{{diag}}\left[H_{1}^{\top}H_{1},\ldots,H_{n}^{\top}H_{n}\right], \alpha can be set small enough such that the following upper bound holds
\displaystyle\lambda_{1}(Q)\leq 1\alpha\lambda_{n}(H).  (9) 
By the same token, Weyl’s eigenvalue inequality implies a lower bound the smallest eigenvalue of Q as
\displaystyle\lambda_{n}(Q)\geq\lambda_{n}(P)\alpha\lambda_{1}(\textit{{diag}% }\left[H_{1}^{\top}H_{1},\ldots,H_{n}^{\top}H_{n}\right]).  (10) 
Once again choosing small enough \alpha guarantees that the RHS of (9) is larger than the absolute value of the RHS of (10). Let us distinguish such regime as \alpha\leq\alpha_{\max}. Then, symmetry of Q warrants that
\displaystyle\Q\\leq 1\alpha\lambda_{n}(H),\ \ \ \ \forall\alpha\leq\alpha_% {\max}. 
Therefore, disregarding the constants (dependence on \{H_{i}\}_{i=1}^{n} and H) in Theorem 1, we can simplify the regret bound using above as follows
\displaystyle\mathbf{Reg}_{T}  \displaystyle\leq\mathcal{O}\left(\alpha\sum_{i=1}^{n}W_{i}+\frac{C_{T}}{T% \alpha^{2}}\right). 
The following comments are now in order:

If the target is fixed, i.e. \theta_{t}=\theta for every t\in[T], we have C_{T}=0, and the second term in the bound vanishes. Then, we can set \alpha=\min\{\frac{1}{T},\alpha_{\max}\} to maintain the \mathcal{O}(\frac{1}{T}) rate. This choice of step size and the corresponding result is consistent with the results in static setting.

When the observations are noiseless (W_{i}=0) the first term becomes zero, and \alpha=\min\{1,\alpha_{\max}\} recovers the \mathcal{O}(\frac{C_{T}}{T}) rate. In this regime, a sublinear pathlength C_{T} always guarantees a zero asymptotic regret.

In the general case tuning \alpha=\min\{C_{T}^{1/3}T^{1/3},\alpha_{\max}\} yields a regret of \mathcal{O}\left(C_{T}^{1/3}T^{1/3}\right) which holds for any T.
IV Concluding Remarks
In this paper, we considered the distributed estimation problem in dynamic environments. A network of agents partially observe the parameter of interest which is timevarying with no particular dynamics. However, the parameter is globally observable from the standpoint of all agents together. We pose the problem as an instance of distributed, online optimization, where agents suffer some loss after each round of estimation. In turn, they aim to minimize the network loss defined as the sum of individual losses. We formulated the problem using the notion of dynamic regret from online optimization literature. Our main contribution was to provide the nonasymptotic analysis of the dynamic regret. We first showed that the regret can be related to the finitetime counterpart of stability. Next, we defined the pathlength as the consecutive differences between target locations, and derived an upper bound on regret in terms of the pathlength. We finally demonstrated that our bound is consistent with special cases of static setting and noiseless observations.
There are several directions in which our work can be improved. Our setting does not include any prior assumption on the dynamics of the parameter. While the generality of this setup is appealing, it is also interesting to investigate the problem for particular dynamics. More specifically, once agents know the dynamics of the parameter (even in the nonlinear case) they can incorporate it into the algorithm, and obtain more accurate estimates. Furthermore, the choice of step size in the general case requires a prior knowledge of the pathlength. An alternative approach is to have an online feedback of the trajectory as the algorithm moves forward. Therefore, an adaptive, online algorithm for tracking dynamic parameters is still an open problem.
V Appendix : Proofs
Proof of Lemma 1. In view of equations (1) and (5), we write
\displaystyle\hat{\boldsymbol{\theta}}_{i,t}\theta_{t}  \displaystyle=\sum_{j=1}^{n}[P]_{ij}\hat{\boldsymbol{\theta}}_{j,t1}\theta_{t}  
\displaystyle+\alpha H_{i}^{\top}\left(\mathbf{y}_{i,t1}H_{i}\hat{% \boldsymbol{\theta}}_{i,t1}\right)  
\displaystyle=\sum_{j=1}^{n}[P]_{ij}(\hat{\boldsymbol{\theta}}_{j,t1}\theta_% {t1})+\theta_{t1}\theta_{t}  
\displaystyle+\alpha H_{i}^{\top}\left(H_{i}\theta_{t1}+\mathbf{w}_{i,t1}H_% {i}\hat{\boldsymbol{\theta}}_{i,t1}\right),  (11) 
since \sum_{j=1}^{n}[P]_{ij}=1. Rewriting above based on the definition of local error process (7), we obtain
\displaystyle\mathbf{e}_{i,t}  \displaystyle=\sum_{j=1}^{n}[P]_{ij}\mathbf{e}_{j,t1}\alpha H_{i}^{\top}H_{i% }\mathbf{e}_{i,t1}  
\displaystyle+\theta_{t1}\theta_{t}+\alpha~{}H_{i}^{\top}\mathbf{w}_{i,t1}. 
Representing above in the matrix form using the Kronecker product completes the proof.\hfill\blacksquare
Proof of Lemma 2. Note equations (1) and (3) to observe
\displaystyle\ell_{t}(\theta_{t})=\sum_{i=1}^{n}\mathbbm{E}\left[\\mathbf{w}_% {i,t}\^{2}\right]=\sum_{i=1}^{n}W_{i},  (12) 
and
\displaystyle\ell_{t}(\hat{\boldsymbol{\theta}}_{j,t})  \displaystyle=\sum_{i=1}^{n}\mathbbm{E}\left[\left\\mathbf{y}_{i,t}H_{i}\hat% {\boldsymbol{\theta}}_{j,t}\right\^{2}\right]  
\displaystyle=\sum_{i=1}^{n}\mathbbm{E}\left[\left\H_{i}\theta_{t}+\mathbf{w}% _{i,t}H_{i}\hat{\boldsymbol{\theta}}_{j,t}\right\^{2}\right]  
\displaystyle=\sum_{i=1}^{n}\mathbbm{E}\left[\left\H_{i}\mathbf{e}_{j,t}% \mathbf{w}_{i,t}\right\^{2}\right], 
where we used (7) in the last step. Given that noise realizations are independent over time, we can simplify above
\displaystyle\ell_{t}(\hat{\boldsymbol{\theta}}_{j,t})  \displaystyle=\sum_{i=1}^{n}\mathbbm{E}\left[\left\H_{i}\mathbf{e}_{j,t}% \mathbf{w}_{i,t}\right\^{2}\right]  
\displaystyle=\sum_{i=1}^{n}\left(\mathbbm{E}\left[\left\H_{i}\mathbf{e}_{j,t% }\right\^{2}\right]+W_{i}\right).  (13) 
Therefore, combining (12) and (13), we get
\displaystyle\frac{1}{n}\sum_{j=1}^{n}\ell_{t}(\hat{\boldsymbol{\theta}}_{j,t}% )\ell_{t}(\theta_{t})  \displaystyle=\frac{1}{n}\sum_{j=1}^{n}\sum_{i=1}^{n}\mathbbm{E}\left[\left\H% _{i}\mathbf{e}_{j,t}\right\^{2}\right]  
\displaystyle\leq\frac{1}{n}\sum_{j=1}^{n}\sum_{i=1}^{n}\left\H_{i}\right\^{% 2}\mathbbm{E}\left[\left\\mathbf{e}_{j,t}\right\^{2}\right]  
\displaystyle=\Big{(}\frac{1}{n}\sum_{i=1}^{n}\left\H_{i}\right\^{2}\Big{)}% \mathbbm{E}\left[\left\\mathbf{e}_{t}\right\^{2}\right], 
where we used (8) in the last line. Summing over t\in[T] completes the proof.\hfill\blacksquare
Proof of Theorem 1. In view of Lemma 1, we have
\displaystyle\mathbbm{E}\left[\\mathbf{e}_{t}\^{2}\right]  \displaystyle\leq\Q\^{2}\mathbbm{E}\left[\\mathbf{e}_{t1}\^{2}\right]  
\displaystyle+\alpha^{2}\sum_{i=1}^{n}\H_{i}\^{2}W_{i}+n\\theta_{t1}% \theta_{t}\^{2}  
\displaystyle+2[\mathbbm{1}_{n}\otimes(\theta_{t1}\theta_{t})]^{\top}Q% \mathbbm{E}[\mathbf{e}_{t1}].  (14) 
since \mathbbm{E}[\mathbf{w}_{i,t1}]=\mathbf{0}. Using the simple fact that for any \beta>0
\displaystyle 2ab\leq\frac{1}{\beta}a^{2}+\beta b^{2}, 
based on AMGM inequality, we apply the CauchySchwarz inequality to bound
\displaystyle 2[\mathbbm{1}\otimes(\theta_{t1}\theta_{t})]^{\top}Q\mathbbm{E% }[\mathbf{e}_{t1}]\leq  
\displaystyle~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}\frac{1}{\beta}\% \mathbbm{1}_{n}\otimes(\theta_{t1}\theta_{t})\^{2}+\beta\Q\^{2}E[\% \mathbf{e}_{t1}\]^{2}, 
for any \beta>0, and simplify (14) to
\displaystyle\mathbbm{E}\left[\\mathbf{e}_{t}\^{2}\right]  \displaystyle\leq(1+\beta)\Q\^{2}\mathbbm{E}\left[\\mathbf{e}_{t1}\^{2}\right]  
\displaystyle+\alpha^{2}\sum_{i=1}^{n}\H_{i}\^{2}W_{i}+\frac{\beta+1}{\beta}% n\\theta_{t1}\theta_{t}\^{2}, 
which entails
\displaystyle\Big{(}1(1+\beta)\Q\^{2}\Big{)}\mathbbm{E}\left[\\mathbf{e}_{% t}\^{2}\right]  
\displaystyle~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}\leq(1+\beta)\Q\^{2}\Big{(}% \mathbbm{E}\left[\\mathbf{e}_{t1}\^{2}\right]\mathbbm{E}\left[\\mathbf{e}% _{t}\^{2}\right]\Big{)}  
\displaystyle~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}+\alpha^{2}\sum_{i=1}^{n}\H_{i}% \^{2}W_{i}+\frac{\beta+1}{\beta}n\\theta_{t1}\theta_{t}\^{2}.  (15) 
Using the convention \mathbbm{E}\left[\\mathbf{e}_{0}\^{2}\right]=0, the following sum telescopes
\displaystyle\sum_{t=1}^{T}\mathbbm{E}\left[\\mathbf{e}_{t1}\^{2}\right]% \mathbbm{E}\left[\\mathbf{e}_{t}\^{2}\right]\leq\mathbbm{E}\left[\\mathbf{e% }_{0}\^{2}\right]=0, 
and summing (15) over t\in[T], we derive
\displaystyle\sum_{t=1}^{T}\mathbbm{E}\left[\\mathbf{e}_{t}\^{2}\right]  \displaystyle\leq\frac{\alpha^{2}T\sum_{i=1}^{n}\H_{i}\^{2}W_{i}}{1(1+\beta% )\Q\^{2}}  
\displaystyle~{}~{}~{}~{}~{}~{}~{}~{}~{}+\frac{\beta+1}{\beta}\frac{n\sum_{t=1% }^{T}\\theta_{t1}\theta_{t}\^{2}}{1(1+\beta)\Q\^{2}}. 
Recall that 0<\beta<\Q\^{2}1 is a free parameter in the analysis. Letting \beta=\Q\^{1}1, and recalling Definition 6, we calculate the bound above to get
\displaystyle\sum_{t=1}^{T}\mathbbm{E}\left[\\mathbf{e}_{t}\^{2}\right]  \displaystyle\leq\frac{\alpha^{2}T\sum_{i=1}^{n}\H_{i}\^{2}W_{i}}{1\Q\}+% \frac{nC_{T}}{(1\Q\)^{2}}. 
Appealing to Lemma 2 completes the proof.\hfill\blacksquare
References
 [1] J. N. Tsitsiklis, “Decentralized detection by a large number of sensors,” Mathematics of Control, Signals and Systems, vol. 1, no. 2, pp. 167–182, 1988.
 [2] S. S. Stanković, M. S. Stanković, and D. M. Stipanović, “Decentralized parameter estimation by consensus based stochastic approximation,” IEEE Transactions on Automatic Control, vol. 56, no. 3, pp. 531–543, 2011.
 [3] K. Drakopoulos, A. Ozdaglar, and J. N. Tsitsiklis, “On learning with finite memory,” IEEE Transactions on Information Theory, vol. 59, no. 10, pp. 6859–6872, 2013.
 [4] S. Shahrampour, A. Rakhlin, and A. Jadbabaie, “Distributed detection : Finitetime analysis and impact of network topology,” IEEE Transactions on Automatic Control, vol. 61, 2016.
 [5] F. Bullo, J. Cortés, and S. Martinez, Distributed Control of Robotic Networks: A Mathematical Approach to Motion Coordination Algorithms: A Mathematical Approach to Motion Coordination Algorithms. Princeton University Press, 2009.
 [6] D. Acemoglu, M. A. Dahleh, I. Lobel, and A. Ozdaglar, “Bayesian learning in social networks,” The Review of Economic Studies, vol. 78, no. 4, pp. 1201–1236, 2011.
 [7] A. Jadbabaie, J. Lin, and A. S. Morse, “Coordination of groups of mobile autonomous agents using nearest neighbor rules,” IEEE Transactions on Automatic Control, vol. 48, no. 6, pp. 988–1001, 2003.
 [8] R. OlfatiSaber and R. M. Murray, “Consensus problems in networks of agents with switching topology and timedelays,” IEEE Transactions on Automatic Control, vol. 49, no. 9, pp. 1520–1533, 2004.
 [9] N. CesaBianchi and G. Lugosi, Prediction, learning, and games. Cambridge University Press, 2006.
 [10] M. Zinkevich, “Online convex programming and generalized infinitesimal gradient ascent,” in International Conference on Machine Learning, 2003.
 [11] E. Hall and R. Willett, “Online convex optimization in dynamic environments,” IEEE Journal of Selected Topics in Signal Processing, vol. 9, no. 4, pp. 647–662, June 2015.
 [12] A. Jadbabaie, A. Rakhlin, S. Shahrampour, and K. Sridharan, “Online optimization: Competing with dynamic comparators,” in Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics, 2015, pp. 398–406.
 [13] R. E. Kalman, “A new approach to linear filtering and prediction problems,” Journal of Fluids Engineering, vol. 82, no. 1, pp. 35–45, 1960.
 [14] R. OlfatiSaber, “Distributed kalman filter with embedded consensus filters,” in IEEE Conference on Decision and Control (CDC), European Control Conference (ECC), 2005, pp. 8179–8184.
 [15] ——, “Distributed kalman filtering for sensor networks,” in IEEE Conference on Decision and Control (CDC), 2007, pp. 5492–5498.
 [16] R. Carli, A. Chiuso, L. Schenato, and S. Zampieri, “Distributed kalman filtering based on consensus strategies,” IEEE Journal on Selected Areas in Communications, vol. 26, no. 4, pp. 622–633, 2008.
 [17] R. OlfatiSaber, “Kalmanconsensus filter: Optimality, stability, and performance,” in IEEE Conference on Decision and Control (CDC) held jointly with Chinese Control Conference (CCC), 2009, pp. 7036–7042.
 [18] D. Acemoglu, A. Nedić, and A. Ozdaglar, “Convergence of ruleofthumb learning rules in social networks,” in IEEE Conference on Decision and Control (CDC), 2008, pp. 1714–1720.
 [19] U. Khan, S. Kar, A. Jadbabaie, J. M. Moura et al., “On connectivity, observability, and stability in distributed estimation,” in IEEE Conference on Decision and Control (CDC), 2010, pp. 6639–6644.
 [20] S. Das and J. M. Moura, “Distributed state estimation in multiagent networks,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2013, pp. 4246–4250.
 [21] S. Shahrampour, S. Rakhlin, and A. Jadbabaie, “Online learning of dynamic parameters in social networks,” in Advances in Neural Information Processing Systems, 2013.
 [22] A. De, S. Bhattacharya, P. Bhattacharya, N. Ganguly, and S. Chakrabarti, “Learning a linear influence model from transient opinion dynamics,” in Proceedings of the 23rd ACM International Conference on Information and Knowledge Management. ACM, 2014, pp. 401–410.
 [23] N. Atanasov, R. Tron, V. M. Preciado, and G. J. Pappas, “Joint estimation and localization in sensor networks,” in IEEE Conference on Decision and Control (CDC), 2014, pp. 6875–6882.
 [24] M. O. Sayin, N. D. Vanli, I. Delibalta, and S. S. Kozat, “Optimal and efficient distributed online learning for big data,” in IEEE International Congress on Big Data (BigData Congress), 2015, pp. 126–133.
 [25] A. Simonetto, A. Mokhtari, A. Koppel, G. Leus, and A. Ribeiro, “A decentralized predictioncorrection method for networked timevarying convex optimization,” in Proceedings of the 6th IEEE International Workshop on Computational Advances in MultiSensor Adaptive Processing,(Cancun, Mexico), 2015.
 [26] R. van Oosten, “Learning from neighbors in a changing world,” 2016.
 [27] J. S. Rosenthal, “Convergence rates for markov chains,” SIAM Review, vol. 37, no. 3, pp. 387–405, 1995.
 [28] S. Kar, J. M. Moura, and K. Ramanan, “Distributed parameter estimation in sensor networks: Nonlinear observation models and imperfect communication,” IEEE Transactions on Information Theory, vol. 58, no. 6, pp. 3575–3605, 2012.