Simultaneous Block-Sparse Signal Recovery Using Pattern-Coupled Sparse Bayesian Learning
In this paper, we consider the block-sparse signals recovery problem in the context of multiple measurement vectors (MMV) with common row sparsity patterns. We develop a new method for recovery of common row sparsity MMV signals, where a pattern-coupled hierarchical Gaussian prior model is introduced to characterize both the block-sparsity of the coefficients and the statistical dependency between neighboring coefficients of the common row sparsity MMV signals. Unlike many other methods, the proposed method is able to automatically capture the block sparse structure of the unknown signal. Our method is developed using an expectation-maximization (EM) framework. Simulation results show that our proposed method offers competitive performance in recovering block-sparse common row sparsity pattern MMV signals.
Simultaneous Block-Sparse Signal Recovery Using Pattern-Coupled Sparse Bayesian Learning
|Hang Xiao, Zhengli Xing , Linxiao Yang, Jun Fang, and Yanlun Wu|
|National Key Laboratory of Science and Technology on Communications,|
|University of Electronic Science and Technology of China, Chengdu 611731, China,|
|China Academy of Engineering Physics, Mianyang 621900, China.|
Index Terms— Compressed sensing, block-sparse signal, multiple Measurement Vectors (MMV), sparse Bayesian Learning.
Compressing sensing is a new paradigm for data acquisition and reconstruction through exploiting the inherent sparsity signals of interest. In practice, sparse signals usually have additional structure that can be exploited to enhance the recovery performance. For example, the atomic decomposition of multi-band signals or audio signals usually results in block-sparse structure, in which the non-zeros coefficients occurs in cluster. A number of algorithms, e.g. block-OMP, mixed norm-minimization were proposed to recover the block-sparse signals. These methods address only the SMV recovery problem. However, in real world the assumption of signals share the same sparsity pattern hold valid. We usually obtain multiple observations, where the recovery performance can be enhanced by exploiting the joint estimation. There are a great deal of signals in the real-world share the unchanged sparsity pattern over time, such as communication signals are assigned to a specific bands of frequency spectrums, thus, these communication signals often sharing the same sparsity pattern in frequency domain. It has been shown that compared to SMV case, the successful recovery rate can be greatly improved using multiple measurement vectors. Wipf and Rao first introduced SBL to sparse signal recovery for SMV model, and later extended it to the MMV model, deriving the MSBL algorithm. However, MSBL algorithm does not take the block-sparse properties of MMV signals into consideration. Zhang and Rao develop the BSBL algorithm to solve the temporally correlated block-sparse MMV signals. Nevertheless, BSBL algorithm requires block-partition known a priori.
In this paper, we extend our former work to the MMV scenario. To exploit the statistical dependencies, we propose pattern-coupled hierarchical Gaussian prior model which characterizes both the block sparseness of the coefficient and the statistical dependency between neighboring coefficients. A key assumption in the considered MMV model is that the support ( i.e. the indexes of nonzero entries) of each column in MMV signals are identical, and the block-structure of each signal is entirely unkown. In our hierarchical Bayesian model, the prior for each coefficient not only involves its own hyperparameter, but also the hyperparameters of its immediate neighboring coefficients. An expectation-maximization (EM) algorithm is developed to learn and the hyperparameters and to estimate the block-sparse MMV signals. Simulation results are provided to illustrate the effectiveness of the proposed algorithm.
2 Problem Formulation
We consider the problem of simultaneously recovering a set of block-sparse signals from an basic underdetermined system
where , and denote the th measurements, the sensing matrix and the noise, respectively. The signal has a block-sparse structure and all share the same support. We note that the block partition of is unknown. We aim to recover by exploiting their block-sparsity and the property that sharing the same row sparsity pattern.
It is easy to see that the model (1) can be rewritten in matrix form, given by
where , , and is unknown noise matrix. We assume that the elements of are i.i.d white noise following Gaussian distribution with zero mean and variance. Then the distribution of conditional on is given as
To simultaneously capture the property of column-wise block-sparsity and the common row sparsity pattern of , we assign a Gaussian prior on each row of . Specifically, we impose a Gaussian prior distribution on the th row of , i.e. , with zero mean and covariance matrix, i.e.,
where is a positive matrix characterizing the dependency of the elements of . We note that all the rows of share the same which has been shown that such a prior is able to promote the low-rankness of , i.e., automatically capture the correlation among the rows of . In (4), are positive scalars controlling the sparsity of the rows of . We assume and for the end rows and , is a parameter indicating the relevance between the coefficient and its neighboring coefficients. It has been shown that by coupling the neighbor elements using , such a prior has potential to encourage a block-sparse solution. Then the joint distribution of given and can be written as
where to be a diagonal matrix with its th diagonal element equals to .
3 Proposed Bayseian Inference Algorithm
In this section, we proceed to develop a sparse Bayesian learning method for block-sparse MMV signal recovery. Based on the above hierarchical model, the posterior distribution of can be computed as
The log-posterior of can be written as
where and denote the vectorization of and , respectively, and denotes the operation of Kronecker product. Then we arrive at that the posterior distribution of follows the Gaussian distribution with mean and covariance matrix given as
Given a set of estimated hyperparameters , , and the observed , the maximum a posterior (MAP) estimate of is the mean of its posterior distribution, i.e.
Our problem therefore reduces to estimate the value of the hyperparameters , , and . A strategy to maximize the likelihood function of these hyperparameters is to exploit the expectation-maximization (EM) formulation, in which we first introduce a hidden variable and then iteratively maximize a lower bound of the likelihood function (this lower bound is also referred to as the Q-function). Briefly speaking, the algorithm alternates between an E-step and a M-step. In the E-step, we compute a new Q-function by taking expectation of the log joint distribution of data and hidden variable with respect to the posterior of hidden variable which computed using current estimation of the hyperparameters. In the M-step, we update the hyperparameters by maximizing the Q-function with respect to them.
We define the hyperparameters and recognize as the hidden variable. Then Q-function can be expressed as
and, consequently, can be updated by maximizing -function, i.e.,
where denotes the operator that taking expectation using the distribution , and is given as
where we define , with , and , in which , with denotes the the row of . Similarly, we set . Then the optimal solution should satisfy
Since all the hyper parameters are non-negative, we have
Hence the term on the left-hand side of (27) is lower and upper bounded respectively by
combining above equations (27)-(31), we arrive at
Due to the high computational complexity of calculating an accuracy solution of (16), we employ an sub-optimal solution of it, i.e., just set to its upper bound, which arrives at
Although We employ a sub-optimal solution (23) to update the hyperparameter in M-step, numerical results show that the sub-optimal update rule is quite effective. This is because the sub-optimal solution (23) provide a reasonable estimate of the optimal solution.
We then consider solving the problem (12). Similarly, we set the first derivative of the Q-function with respect to to zero, i.e.,
and arrive at that the optimal solution is given by
To estimate , the Q-function can be simplified to
By computing the derivative of (26) and setting it to zero, we arrive at
Some of the expectations and moments used during the update are summarized as
where is a vector with its th element equals to the th of ’s and is a matrix with its th entry equals to the th of .
For clarity, we summarize our algorithm as follows.
4 Simulation Results
In our simulations, we study how the proposed algorithm benefit from multiple measurement vectors for the block-sparse common row sparsity pattern signals recovery problem. Suppose each -dimensional sparse vector contains nonzero coefficients which are partitioned into blocks with random sizes and random location. The over-complete dictionary are randomly generated with each entry independently drawn from a normal distribution.
We examine the recovery performance of our proposed algorithm, also referred as the MMV pattern-coupled sparse Bayesian learning algorithm (MPCSBL), under different choice of . As indicated earlier in our paper, is a parameter quantifying the dependencies among the neighboring coefficients. Fig.1 depicts the success rates vs. the ratio for different choices of in noiseless case, where we set , , . Results are averaged over 1000 independent runs, with the measurement matrix and the sparse signal randomly generated for each run. The performance of conventional sparse Bayesian learning method ( donated as ”MSBL”) is also included for our comparision. When , our proposed algorithm achieves a significant performance improvement as compared with MSBL through exploiting the underlying block-sparse structure, even without knowing the exacting locations and the sizes of the non-zero blocks. We also observe that our proposed algorithm is not very sensitive to the choice of as long as . The success rates of proposed method and MSBL as a function of the sparsity level are plotted in Fig.2, where , and . We see that our proposed algorithm present the better performance than the MSBL method. In noisy case, we setting , , and , Fig.3 shows that the normalized mean square errors (NMSE) vs. SNR with nonzero rows for different choices of . We also see that our proposed method achieves the better estimation accuracy than existing methods.
We proposed a new Bayesian method for recovery block-sparse MMV signals with common row sparsity pattern. A pattern-coupled hierarchical Gaussian prior model was introduced to characterize both the sparseness of the coefficients and the statistical dependency between neighboring coefficients of the signal. Through exploiting the underlying block-structure, our method outperforms other existing methods in block-sparse MMV signals recovery with common row sparsity pattern. Numerical results show that the proposed method presents superior performance in recovery block-sparse MMV signals with common row sparsity pattern.
-  M. Mishali and Y. C. Eldar, “Blind multi-band signal reconstruction compressed sending for analog signals,” IEEE Trans. Signal Processing, vol. 57, no. 3, pp. 993–1009, 2009.
-  Rémi Gribonval and Emmanuel Bacry, “Harmonic decomposition of audio signals with matching pursuit,” IEEE Trans. Signal Processing, vol. 51, no. 1, pp. 101–111, Jan. 2003.
-  Yonina C. Eldar, Patrick Kuppinger, and Helmut Bölcskei, “Block-sparse signals: uncertainty relations and efficient recovery,” IEEE Trans. Information Theory, vol. 58, no. 6, pp. 3042–3054, June 2010.
-  Y. C. Eldar and M. Mishali, “Robust recovery of signals from a strctured union of subspaces,” IEEE Trans. Inform. Theory, vol. 55, no. 11, pp. 5302–5316, 2009.
-  Yonina C. Eldar and Moshe Mishali, “Robust recovery of signals from a structured union of subspaces,” IEEE Trans. Information Theory, vol. 55, no. 11, pp. 5302–5316, Nov. 2009.
-  Y. C. Eldar and H. Rauhut, “Average case analysis of multichannel sparse recovery using convex relaxation,” IEEE Trans. Inform. Theory, vol. 56, no. 1, pp. 505–519, 2010.
-  D. P. Wipf and B. D. Rao, “An empirical bayesian stragety for solving the simultaneous sparse approximation problem,” IEEE Trans. Signal Processing, vol. 55, no. 7, pp. 3704–3716, 2010.
-  Zhilin Zhang and Bhaskar D. Rao, “Sparse signal recovery with temporally correlated source vectors using sparse Bayesian learning,” IEEE Journal of Selected Topics in Signal Processing, vol. 5, no. 5, pp. 912–926, Sept. 2011.
-  Jun Fang, Yanning Shen, Hongbin Li, and Pu Wang, “Pattern-coupled sparse Bayesian learning for recovery of block-sparse signals,” IEEE Transactions on Signal Processing, vol. 63, no. 2, pp. 360–372, 2015.
-  D. Malioutov, M. Cetin, , and A. S. Willsky, “A sparse signal reconstructional perspective for source localization with sensor arrays,” IEEE Trans. Signal Processing, vol. 53, no. 8, pp. 3010–3022, 2005.
-  Bo Xin, Yizhou Wang, Wen Gao, and David Wipf, “Exploring algorithmic limits of matrix rank minimization under affine constraints,” IEEE Transactions on Signal Processing, vol. 64, no. 19, pp. 4960–4974, 2016.