Hyperspectral Image Restoration via Multi-mode and Double-weighted Tensor Nuclear Norm Minimization

Hyperspectral image (HSI) has been widely used in many fields [4, 1] due to its wealthy spatial and spectral information of a real scene. However, the observed HSIs are usually corrupted by different noises, e.g., Gaussian noise, impulse noise, deadlines, stripes and their mixtures. Therefore, HSI restoration, as a preprocessing step to remove mixed noise for various subsequent applications, is a valuable and active research topic.

HSIs can be treated as 3-order tensors. Its low-rankness is a critical property for HSI restoration tasks. Due to the nonunique definitions of the tensor rank, different tensor decompositions and their corresponding tensor ranks are proposed, such as the Tucker decomposition [18, 12], PARAFAC decomposition [14, 6], and tensor singular value decomposition (t-SVD) [15, 24, 9], to exploit the low-rankness of HSIs.

Among them, the tensor tubal rank induced by t-SVD can characterize the low-rank structure of a tensor very well [23]. Its convex relaxation is the tensor nuclear norm (TNN) [10]. TNN is effective to keep the intrinsic structure of tensors; t-SVD can be calculated easily in the Fourier domain and TNN minimization problem can be efficiently solved by convex optimization algorithms. Hence, TNN has attracted extensive attention for HSI restoration problems in recent years [24, 20, 21]. However, during the definition of TNN, there are three kinds of prior knowledge that are underutilized for further exploiting the low-rankness in HSIs. Firstly, in the Fourier transform domain of HSIs, the low-frequency slices carry the profile information of HSIs, while the high-frequency slices mainly carry the detail and noise information of HSIs. Secondly, in each frequency slices, bigger singular values mainly contain information on clean data and smaller singular values mainly contain information on noise. Thirdly, low-rankness not only exists in the spectral dimension but also lies in the spatial dimensions [18]. The classical TNN only takes the Fourier transform to connect the spatial dimensions with the spectral dimension and lacks flexibility for handling different correlations along with different modes of HSIs [24].

In this letter, to take full advantage of the above prior knowledge and improve the capability and flexibility of TNN, we propose a multi-mode and double-weighted TNN (MDWTNN) for HSI restoration tasks. The merits of our model are four-fold. First, according to information types in different frequency slices in the Fourier transform domain, we adaptively assign bigger weights to slices that mainly contain noise information and smaller weights to slices that mainly contain profile information, which can depress noise more and simultaneously preserve the profile information of clean HSIs better. Second, in each frequency slice, we use the partial sum of singular values (PSSV) to only shrink small singular values, which can better protect the clean data information contained in big singular values. Third, we apply the double-weighted TNN in all modes of HSIs, which can achieve a more flexible and accurate characterization of HSI low-rankness. Finally, we develop an alternating direction method of multiplier (ADMM) based algorithm to efficiently solve the proposed model, and obtain the best restoration performance both on the synthetic and real HSI dataset in comparison to all competing HSI restoration methods.

MDWTNN in (9) takes full advantage of physical meanings in frequency components, singular values, and modes of HSIs, which can provide a better approximation to the tensor rank. Then we use MDWTNN to replace the regularization term Rank in (2) and propose the HSI restoration model as follows:

Introducing auxiliary variables, model (10) is equivalent to

By augmented Lagrangian multiplier method, the Lagrangian function of model (11) can be written as

where $\Lambda$ and ${\Gamma }_p$ are the Lagrangian multipliers; $\beta$ and ${\mu }_p$ are the Lagrange penalty parameters. Its minimization problem can be efficiently solved in the framework of ADMM [2]. At the $(n+1)$-th iteration, each variable in the Lagrangian function can be updated by solving its corresponding subproblem respectively when other variables are fixed at the $n$-th iteration.

For $Z_p$, $p=1,2,3$, their corresponding subproblems can be written as

The closed-form solution of (12) obtained from theorem 1 are as follows:

For $S$, its corresponding subproblem can be written as

It can be solved by the soft-thresholding operator [13] as:

For $N$, its corresponding subproblem can be written as

It has the closed-form solution as follows :

For multipliers ${\Gamma }_p$ and $\Lambda$, they can be updated as follows:

The proposed algorithm for our HSI restoration model is summarized in Algorithm 1.

To verify the effectiveness of our MDWTNN based HSI restoration model, various experiments are performed on a set of challenging simulated and real HSI datasets. From the Washington DC Mall dataset¹, we choose a sub-blocks with a size of $256\times 256\times 191$ as a simulation dataset. From the Indian Pines dataset², we choose a sub-blocks with a size of $145\times 145\times 220$ as a real dataset. For comparison, four state-of-the-art HSI denoising methodes are employed as the benchmark in the experiments, i.e., BM4D [16], LRMR [22], LRTDTV [18] and 3DTNN [24]. Since the BM4D method is only suitable to remove Gaussian noise, we implement it on HSIs which are preprocessed by the RPCA restoration method [15].

In the simulation experiments, the hybrid of white Gaussian and impulse noises with 5 different intensity levels are added to the simulation dataset band by band. Let $G$ and $P$ denote the variance of Gaussian white noise and percentage of impulse noise, respectively. In noise case 1-3, the same intensity noise is added to all the bands. In noise case 1, $G$=0.1 and $P$=0.2; In noise case 2, $G$=0.2 and $P$=0.2; In noise case 3, $G$=0.1 and $P$=0.4; In noise case 4 and 5, the noise intensities are different for different bands. In noise case 4, $G$ is randomly selected from 0.1 to 0.2 and $P$=0.2; In noise case 5, $G$=0.1 and $P$ is randomly selected from 0.2 to 0.4.

For quantitatively evaluating the restoration results of all the test methods, the CPU times and the means of PSNR [7], SSIM [19] and SAM [11] in each band, i.e., MPSNR, MSSIM and MSAM, are listed in Table I. Also, the PSNR and SSIM of each band in all restoration results are presented in Fig. 1. It is clear that our proposed model enjoys a superior performance over the other popular approaches. Although the CPU times of our model are not the shortest, one can update $Z_p$ by (13) in parallel to further shorten the CPU times of our model. For visual evaluation in Fig. 2, we show the 80-th band in all restoration results of the simulation dataset under noise case 5. In Fig. 3, we list the 150-th in all restoration results of the real dataset. It can be seen that the image restored by our model maintains the best structure and texture information.

In this letter, we propose a multi-mode and double-weighted TNN for HSI restoration tasks. The proposed TNN can efficiently characterize the physical meanings of the frequency components, singular values, and orientations ignored by the standard TNN. And the weight parameters also can be obtained adaptively. They powerfully improve capability and flexibility for describing low-rankness in HSIs. The experiments conducted with both simulate and real HSI datasets show that our MDWTNN based HSI restoration model is a competitive method to remove the hybrid noise. Besides, our proposed MDWTNN regularization term can also be applied to other low-rankness based tasks, i.e., hyperspectral imagery classification, tensor completion, MRI reconstruction.