Distributed Compressed Estimation for Wireless Sensor Networks Based on Compressive Sensing
This letter proposes a novel distributed compressed estimation scheme for sparse signals and systems based on compressive sensing techniques. The proposed scheme consists of compression and decompression modules inspired by compressive sensing to perform distributed compressed estimation. A design procedure is also presented and an algorithm is developed to optimize measurement matrices, which can further improve the performance of the proposed distributed compressed estimation scheme. Simulations for a wireless sensor network illustrate the advantages of the proposed scheme and algorithm in terms of convergence rate and mean square error performance.
signal processing algorithms are of great importance for statistical inference in wireless networks and applications such as wireless sensor networks (WSNs) . Distributed processing techniques deal with the extraction of information from data collected at nodes that are distributed over a geographic area . In this context, for each node a set of neighbor nodes collect and process their local information, and transmit their estimates to a specific node. Then, each specific node combines the collected information together with its local estimate to generate improved estimates.
In many scenarios, the unknown parameter vector to be estimated can be sparse and contain only a few nonzero coefficients. Many algorithms have been developed in the literature for sparse signal estimation . However, these techniques are designed to take into account the full dimension of the observed data, which increases the computational cost, slows down the convergence rate and degrades mean square error (MSE) performance.
Compressive sensing (CS)  has recently received considerable attention and been successfully applied to diverse fields, e.g., image processing , wireless communications  and MIMO radar . The theory of CS states that an –sparse signal of length can be recovered exactly with high probability from measurements. Mathematically, the vector with dimension that carries sufficient information about () can be obtained via a linear model 
where is the measurement matrix.
The application of CS to WSNs has been recently investigated in , . A compressive wireless sensing scheme was developed in  to save energy and bandwidth, where CS is only employed in the transmit layer. In , a greedy algorithm called precognition matching pursuit was developed for CS and used at sensors and the fusion center to achieve fast reconstruction. However, the sensors are assumed to capture the target signal perfectly with only measurement noise. The work of  introduced a theory for distributed CS based on jointly sparse signal recovery. However, in  CS techniques are only applied to the transmit layer, whereas distributed CS in the estimation layer has not been widely investigated. A sparse model that allows the use of CS for the online recovery of large data sets in WSNs was proposed in , but it assumes that the sensor measurements could be gathered directly, without an estimation procedure. In summary, prior work has focused on signal reconstruction algorithms in a distributed manner but has not considered both compressed transmit strategies and estimation techniques.
In this work, we focus on the design of an approach that exploits lower dimensions, reduces the required bandwidth, and improves the convergence rate and the MSE performance. Inspired by CS, we introduce a scheme that incorporates compression and decompression modules into the distributed estimation procedure. In the compression module, we compress the unknown parameter into a lower dimension. As a result, the estimation procedure is performed in a compressed dimension. After the estimation procedure is completed, the decompression module recovers the compressed estimator into its original dimension using an orthogonal matching pursuit (OMP) algorithm . We also present a design procedure and develop an algorithm to optimize the measurement matrices, which can further improve the performance of the proposed scheme. Specifically, we derive an adaptive stochastic gradient recursion to update the measurement matrix. Simulation results illustrate the performance of the proposed scheme and algorithm against existing techniques.
This paper is organized as follows. Section II describes the system model. In Section III, the proposed distributed compressed estimation scheme is introduced. The proposed measurement matrix optimization is illustrated in Section IV. Simulation results are provided in Section V. Finally, we conclude the paper in Section VI.
Notation: We use boldface uppercase letters to denote matrices and boldface lowercase letters to denote vectors. We use to denote the inverse operator, for conjugate transposition and for complex conjugate.
2System Model and Problem Statement
A wireless sensor network (WSN) with N nodes, which have limited processing capabilities, is considered with a partially connected topology. A diffusion protocol is employed although other strategies, such as incremental  and consensus  could also be used. A partially connected network means that nodes can exchange information only with their neighbors as determined by the connectivity topology. In contrast, a fully connected network means that, data broadcast by a node can be captured by all other nodes in the network . At every time instant , the sensor at each node takes a scalar measurement according to
where is the input signal vector with zero mean and variance , is the noise at each node with zero mean and variance . From (Equation 1), we can see that the measurements for all nodes are related to an unknown parameter vector with size that should be estimated by the network. We assume that is a sparse vector with non-zero coefficients. The aim of such a network is to compute an estimate of in a distributed fashion, which minimizes the cost function
where denotes expectation. Distributed estimation of is appealing because it provides robustness against noisy measurements and improved performance as reported in . To solve this problem, a cost-effective technique is the adapt–then–combine (ATC) diffusion strategy 
where indicates the set of neighbors for node , is the local estimator of node , denotes the cardinality of and is the combination coefficient, which is calculated with respect to the Metropolis rule
and should satisfy
Existing distributed sparsity-aware estimation strategies, e.g., , are designed using the full dimension signal space, which reduces the convergence rate and degrades the MSE performance. In order to improve performance, reduce the required bandwidth and optimize the distributed processing, we incorporate at each node of the WSN the proposed distributed compressed estimation scheme based on CS techniques, together with a measurement matrix optimization algorithm.
3Proposed Distributed Compressed Estimation Scheme
In this section, we detail the proposed distributed compressed estimation (DCE) scheme based on CS. The proposed scheme, depicted in Fig. ?, employs compression and decompression modules inspired by CS techniques to perform distributed compressed estimation. In the proposed scheme, at each node, the sensor first observes the vector , then with the help of the measurement matrix obtains the compressed version , and performs the estimation of in the compressed domain. In other words, the proposed scheme estimates the vector instead of the vector , where and the –dimensional quantities are designated with an overbar. At each node, a decompression module employs a measurement matrix and a reconstruction algorithm to compute an estimate of . One advantage for the DCE scheme is that fewer parameters need to be transmitted between neighbour nodes.
We start the description of the proposed DCE scheme with the scalar measurement given by
where and is the input signal vector. This operation is depicted in Fig. ? as the compression module.
Fig. ? illustrates the proposed DCE scheme. The scheme can be divided into three steps:
In the adaptation step, at each time instant =1,2, . . . , I, each node =1,2, …, N, generates a local compressed estimator through
where and .
Given the network topology structure, only the local compressed estimator will be transmitted between node and all its neighbor nodes. The measurement matrix will be kept locally.
At each time instant =1,2, . . . , I, the combination step starts after the information exchange is finished. Each node will combine the local compressed estimators from its neighbor nodes and itself through
to compute the updated compressed estimator .
After the final iteration , each node will employ the OMP reconstruction strategy to generate the decompressed estimator . Other reconstruction algorithms can also be used. The decompression module described in Fig. ? illustrates the details. In summary, during the DCE procedure, only the local compressed estimator will be transmitted over the network resulting in a reduction of the number of parameters to be transmitted from to . The proposed DCE scheme is given in Table ?.
The computational complexity of the proposed DCE scheme is , where is the number of nodes in the WSN and is the number of time instants. The distributed NLMS algorithm has a complexity , while the complexity of the sparse diffusion NLMS algorithm  is . For the distributed compressive sensing algorithm of , the computational complexity is . In the proposed DCE scheme, only the local compressed estimator with parameters will be transmitted through the network, which means the transmission requirement is greatly reduced as compared with the standard schemes that transmit with parameters.
|For each time instant =1,2, . . . , I-1|
|For each node =1,2, …, N|
|and is the random measurement matrix|
|For each node =1,2, …, N|
|After the final iteration|
|For each node =1,2, …, N|
|where is the final decompressed estimator.|
4Measurement Matrix Optimization
To further improve the performance of the proposed DCE scheme, an optimization algorithm for the design of the measurement matrix , which is now time–variant, is developed here. Unlike prior work , this optimization is distributed and adaptive. Let us consider the cost function
where . To minimize the cost function, we need to compute the gradient of with respect to and equate it to a null vector, i.e., . As a result, only the first three terms in (Equation 3) need to be considered. Taking the first three terms of (Equation 3) we arrive at
Because the random variable is statistically independent from the other parameters and has zero mean, (Equation 4) can be further simplified as
Then, we have
where , and . Equating (Equation 5) to a null vector, we obtain
The expression in (Equation 6) cannot be solved in closed–form because is an unknown parameter. As a result, we employ the previous estimate to replace . However, and depend on each other, thus, it is necessary to iterate (Equation 6) with an initial guess to obtain a solution. In particular, we replace the expected values with instantaneous values. Starting from (Equation 5), we use instantaneous estimates to compute
According to the method of steepest descent , the updated parameters of the measurement matrix at time are computed by using the simple recursive relation
where is the step size and is the unknown parameter vector that must be estimated by the network. Then, the parameter vector is used to reconstruct the estimate of as follows
where the operator denotes the OMP reconstruction algorithm. Note that other reconstruction algorithms could also be employed. Replacing by , we arrive at the expression for updating the measurement matrix described by
The computational complexity of the proposed scheme with measurement matrix optimization is .
We assess the proposed DCE scheme and the measurement matrix optimization algorithm in a WSN application, where a partially connected network with = 20 nodes is considered. We compare the proposed DCE scheme with uncompressed schemes, including the distributed NLMS (dNLMS) algorithm (normalized version of ), sparse diffusion NLMS algorithm , sparsity-promoting adaptive algorithm , and the distributed compressive sensing algorithm , in terms of MSE performance. Note that other metrics such as mean-square deviation (MSD) could be used but result in the same performance hierarchy between the analyzed algorithms.
The input signal is generated as and , where is a correlation coefficient and is a white noise process with variance , to ensure the variance of is . The compressed input signal is obtained by . The measurement matrix is an i.i.d. Gaussian random matrix that is kept constant. The noise samples are modeled as complex Gaussian noise with variance . The unknown parameter vector has sparsity , where =50, =10 and =3. The step size for the distributed NLMS, distributed compressive sensing, sparse diffusion LMS and the proposed DCE algorithms is 0.45. The parameter that controls the shrinkage in  is set to 0.001. For , the number of hyperslabs equals 55 and the width of the hyperslabs is 0.01.
Fig. ? illustrates the comparison between the DCE scheme with other existing algorithms, without the measurement matrix optimization. It is clear that, when compared with the existing algorithms, the DCE scheme has a significantly faster convergence rate and a better MSE performance. These advantages consist in two features: the compressed dimension brought by the proposed scheme and CS being implemented in the estimation layer. As a result, the number of parameters for transmission in the network is significantly reduced.
In the second scenario, we employ the measurement matrix optimization algorithm to in the DCE scheme. The parameter for the measurement matrix optimization algorithm is set to 0.08 and all other parameters remain the same as in the previous scenario. In Fig. ?, we observe that with the help of the measurement matrix optimization algorithm, DCE can achieve a faster convergence when compared with DCE without the measurement matrix optimization.
In the third scenario, we compare the DCE scheme with the distributed NLMS algorithm with different levels of resolution in bits per coefficient, reduced dimension and sparsity level . The x-axis stands for the reduced dimension and their corresponding sparsity level can be found in Fig. ?. In Fig. ?, it is clear that with the increase of the sparsity level the MSE performance degrades. In addition, the MSE performance will increase when the transmission has more bits per coefficient. For the DCE scheme, the total number of bits required for transmission is times the number of bits per coefficient, whereas for the distributed NLMS algorithm it is times the number of bits per coefficient. A certain level of redundancy is required between the sparsity level and the reduced dimension due to the error introduced by the estimation procedure.
We have proposed a novel DCE scheme and algorithms for sparse signals and systems based on CS techniques and a measurement matrix optimization. In the DCE scheme, the estimation procedure is performed in a compressed dimension. The results for a WSN application show that the DCE scheme outperforms existing strategies in terms of convergence rate, reduced bandwidth and MSE performance.
- “Diffusion least-mean squares over adaptive networks: Formulation and performance analysis,”
C. G. Lopes and A. H. Sayed, IEEE Transactions on Signal Processing, vol. 56, no. 7, pp. 3122–3136, July 2008.
- “Distributed conjugate gradient strategies for distributed estimation over sensor networks,”
S. Xu and R. C. de Lamare, in Proc. Sensor Signal Processing for Defence 2012, London, UK, 2012.
- “Adaptive link selection strategies for distributed estimation in diffusion wireless networks,”
S. Xu, R. C. de Lamare, and H. V. Poor, in Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing, Vancouver, Canada 2013.
- “Diffusion lms strategies for distributed estimation,”
F. S. Cattivelli and A. H. Sayed, IEEE Transactions on Signal Processing, vol. 58, pp. 1035–1048, March 2010.
- “Sparse LMS for system identification,”
Y. Chen, Y. Gu, and A.O. Hero, in Proc. IEEE ICASSP, pp. 3125–3128, Taipei, Taiwan, May 2009.
- “Distributed spectrum estimation for small cell networks based on sparse diffusion adaptation,”
P. Di Lorenzo, S. Barbarossa, and A.H. Sayed, IEEE Signal Processing Letters, vol. 20, no. 12, pp. 1261–1265, Dec 2013.
- “Online adaptive estimation of sparse signals: Where rls meets the –norm,”
D. Angelosante, J.A Bazerque, and G.B. Giannakis, IEEE Transactions on Signal Processing, vol. 58, no. 7, pp. 3436–3447, July 2010.
- “A sparsity promoting adaptive algorithm for distributed learning,”
S. Chouvardas, K. Slavakis, Y. Kopsinis, and S. Theodoridis, IEEE Transactions on Signal Processing, vol. 60, no. 10, pp. 5412–5425, Oct 2012.
- “Distributed sparse linear regression,”
G. Mateos, J. A. Bazerque, and G. B. Giannakis, IEEE Transactions on Signal Processing, vol. 58, no. 10, pp. 5262–5276, Oct 2010.
- “Sparse distributed learning based on diffusion adaptation,”
P. Di Lorenzo and A. H. Sayed, IEEE Transactions on Signal Processing, vol. 61, no. 6, pp. 1419–1433, March 2013.
- “Distributed sparse recursive least-squares over networks,”
Z. Liu, Y. Liu, and C. Li, IEEE Transactions on Signal Processing, vol. 62, no. 6, pp. 1386–1395, March 2014.
- “Compressive diffusion strategies over distributed networks for reduced communication load,”
M. O. Sayin and S. S. Kozat, IEEE Transactions on Signal Processing, vol. 62, no. 20, pp. 5308–5323, Oct 2014.
- “Adaptive reduced-rank mmse filtering with interpolated fir filters and adaptive interpolators,”
R. C. de Lamare and R. Sampaio-Neto, IEEE Signal Processing Letters, vol. 12, no. 3, pp. 177–180, March 2005.
- “Reduced-rank adaptive filtering based on joint iterative optimization of adaptive filters,”
R. C. de Lamare and R. Sampaio-Neto, IEEE Signal Processing Letters, vol. 14, no. 12, pp. 980–983, December 2007.
- “Adaptive reduced-rank processing based on joint and iterative interpolation, decimation and filtering,”
R. C. de Lamare and R. Sampaio-Neto, IEEE Transactions on Signal Processing, vol. 57, no. 7, pp. 2503–2514, July 2009.
- “Reduced-rank stap schemes for airborne radar based on switched joint interpolation, decimation and filtering algorithm,”
R. Fa, R. C. de Lamare, and L. Wang, IEEE Transactions on Signal Processing, vol. 58, no. 8, pp. 4182–4194, August 2010.
- “Set-membership adaptive algorithms based on time-varying error bounds for CDMA interference suppression,”
R. C. de Lamare and P. S. R. Diniz, IEEE Transactions on Vehicular Technology, vol. 58, no. 2, pp. 644–654, February 2009.
- “Blind adaptive constrained constant-modulus reduced-rank interference suppression algorithms based on interpolation and switched decimation,”
R. C. de Lamare, R. Sampaio-Neto, and M. Haardt, IEEE Transactions on Signal Processing, vol. 59, no. 2, pp. 681–695, February 2011.
- “Compressed sensing,”
D. L. Donoho, IEEE Transactions on Information Theory, vol. 52, no. 4, pp. 1289–1306, April 2006.
- “An introduction to compressive sampling,”
E.J. Candes and M.B. Wakin, IEEE Signal. Proc. Mag., vol. 25, no. 2, pp. 21–30, March 2008.
- “Imaging via compressive sampling,”
J. Romberg, IEEE Signal Processing Magazine, vol. 25, no. 2, pp. 14–20, March 2008.
- “Compressive wireless sensing,”
W. Bajwa, J. Haupt, A. Sayeed, and R. Nowak, in Proc. IEEE Inform. Process. Sens. Netw., Nashville, TN, April 2006.
- “MIMO radar using compressive sampling,”
Y. Yao, A. P. Petropulu, and H. V. Poor, IEEE Journal on Selected Topics in Signal Processing, vol. 4, no. 1, pp. 146–163, Feb 2010.
- “Distributed compressive sensing reconstruction via common support discovery,”
C. Wei, M. Rodrigues, and I. J. Wassell, in IEEE International Conference on Communications, Kyoto, Japan, June 2011.
- “Distributed compressive sensing,”
D. Baron, M. F. Duarte, M. B. Wakin, S. Sarvotham, and R. G. Baraniuk, in ECE Department Tech. Report TREE-0612, Rice University, Houston, TX, November 2006.
- “Sensing, compression, and recovery for WSNs: Sparse signal modeling and monitoring framework,”
G. Quer, R. Masiero, G. Pillonetto, M. Rossi, and M. Zorzi, IEEE Transactions on Wireless Communications, vol. 11, no. 10, pp. 3447–3461, October 2012.
- “Signal recovery from random measurements via orthogonal matching pursuit,”
J. A. Tropp and A. C. Gilbert, IEEE Transactions on Information Theory, vol. 53, no. 12, pp. 4655–4666, December 2007.
- “Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition,”
Y. C. Pati, R. Rezaiifar, and P. S. Krishnaprasad, in Proc. 27th Annu. Asilomar Conf. Signals, Systems,and Computers, pp. 40–44 vol.1, Paciﬁc Grove, CA, Nov, 1993.
- “Adaptive greedy approximations,”
G. Davis, S. Mallat, and M. Avellaneda, Constructive Approximation, vol. 13, no. 1, pp. 57–98, 1997.
- “Incremental adaptive strategies over distributed networks,”
C. G. Lopes and A. H. Sayed, IEEE Transactions on Signal Processing, vol. 48, no. 8, pp. 223–229, Aug 2007.
- “Fully distributed state estimation for wide-area monitoring systems,”
L. Xie, D.-H. Choi, S. Kar, and H. V. Poor, IEEE Transactions on Smart Grid, vol. 3, no. 3, pp. 1154–1169, September 2012.
- “Distributed adaptive node–specific signal estimation in fully connected sensor networks–part II: Simultaneous and asynchronous node updating,”
A. Bertrand and M. Moonen, IEEE Transactions on Signal Processing, vol. 58, no. 10, pp. 5292–5306, 2010.
- “Measurement matrix design for compressive sensing–based MIMO radar,”
Y. Yao, A. P. Petropulu, and H. V. Poor, IEEE Transactions on Signal Processing, vol. 59, no. 11, pp. 5338–5352, November 2011.
- Adaptive Filter Theory,
S. Haykin, Prentice Hall, Upper Saddle River, NJ, USA, third edition, 1996.