Leveraging Deep Stein’s Unbiased Risk Estimator for Unsupervised X-ray Denoising

Leveraging Deep Stein’s Unbiased Risk Estimator for Unsupervised X-ray Denoising

Fahad Shamshad, Muhammad Awais, Muhammad Asim,
Zain ul Aabidin Lodhi, Muhammad Umair, Ali Ahmed
Department of Electrical Engineering, Information Technology University, Lahore, Pakistan
Department of Civil Engineering, University of Toronto, Canada

Among the plethora of techniques devised to curb the prevalence of noise in medical images, deep learning based approaches have shown the most promise. However, one critical limitation of these deep learning based denoisers is the requirement of high-quality noiseless ground truth images that are difficult to obtain in many medical imaging applications such as X-rays. To circumvent this issue, we leverage recently proposed approach of (soltanayev2018training, ) that incorporates Stein’s Unbiased Risk Estimator (SURE) to train a deep convolutional neural network without requiring denoised ground truth X-ray data. Our experimental results demonstrate the effectiveness of SURE based approach for denoising X-ray images.


Leveraging Deep Stein’s Unbiased Risk Estimator for Unsupervised X-ray Denoising

  Fahad Shamshad, Muhammad Awais, Muhammad Asim, Zain ul Aabidin Lodhi, Muhammad Umair, Ali Ahmed Department of Electrical Engineering, Information Technology University, Lahore, Pakistan Department of Civil Engineering, University of Toronto, Canada


noticebox[b]Machine Learning for Health (ML4H) Workshop at NeurIPS 2018.\end@float

1 Introduction

X-ray images provide crucial support for diagnosis and decision making in many diverse clinical applications. However, X-ray images may be corrupted by statistical noise, thus seriously deteriorating the quality and raising the difficulty of diagnosis (thanhreview, ). Therefore, X-ray denoising is an essential pre-processing step for improving the quality of raw X-ray images and their relevant clinical information content.

Deep learning with massive amounts of training data has revolutionized many image processing and computer vision tasks including image denoising (lecun2015deep, ). Deep learning based denoisers have been recently shown to produce state of the art results (zhang2017beyond, ), and have been extensively investigated for denoising X-ray images for enhanced diagnosis reliability (gondara2016medical, ; chen2017low, ). These deep learning based denoisers are usually trained by minimizing mean squared error (MSE). This requires access to abundant high quality and clean ground truth X-ray images that are hard to acquire.

In this work, we leverage recently proposed approach of (soltanayev2018training, ) to train a deep convolutional neural network for denoising, using only noisy X-ray data. Denoising approach of (soltanayev2018training, ) is based on the classical idea of Stein’s Unbiased Risk Estimator (SURE) (stein1981estimation, ). SURE gives an unbiased estimate of MSE, however, it does not require ground truth data for tuning parameters of denoising algorithm thus circumventing the main hurdle for deep learning based denoisers that require clean ground truth for training.

Figure 1: We incorporate SURE loss (3) to train a convolutional neural network for X-ray image denoising. SURE loss allows us to train our network using only corrupted images without requiring ground truth clean images.

2 Methodology

We consider recovering true X-ray image from its noisy measurements of the form


where is noise corrupted image, denotes independent and identically distributed Gaussian noise i.e. where is identity matrix and is standard deviation that is assumed to be known. We are interested in a weakly differentiable function parametrized by that maps noisy X-ray images to clean ones . We model by a convolutional neural network (CNN) where are weights of this network. CNN based denoising methods are typically trained by taking a representative set of clean ground truth images along with corresponding set of noise corrupted observations . The network then learns the mapping from noisy observations to clean images by minimizing a supervised loss function; typically mean squared error (MSE). MSE minimizes the error between true images and the network output as follows:


Note the dependence of MSE on ground truth clean images . Instead of minimizing MSE, we employ SURE loss that optimizes neural network parameters by minimizing


where denotes divergence and is defined as


The first term in (3), minimizes the error between observations and corresponding estimates at network output . The second term penalizes neural network based denoiser for varying as its input image is changed. Calculating divergence of the denoiser is a central challenge for SURE based estimators. We estimate divergence via fast Monte Carlo approximation, see (ramani2008monte, ) for details. In short, instead of utilizing a supervised loss of MSE in (2), we optimize network weights in an unsupervised manner using (3), that does not require ground truth; see Figure 1. We leverage the auto-differentiation function of Tensorflow (abadi2016tensorflow, ) to calculate the gradient of the SURE base loss function, that is hard to compute otherwise.

3 Experiments

To evaluate the proposed denoising approach, we use Indiana University’s Chest X-ray database (IndianaXrays, ). The database consists of chest X-ray images of varying sizes, out of which we select images for training due to the scarcity of computational resources. Training images are re-scaled, cropped, and flipped, to form a set of overlapping patches each of size . We use an end-to-end trainable denoising convolutional neural network (DnCNN) (zhang2017beyond, ) that have recently shown promising denoising results. DnCNN consists of 16 sequential convolutional layers with residual connections. Training was conducted on batches of size 64 using Adam optimizer for 50 epochs with learning rate set to which was reduced to after 25 epochs. DnCNN was trained using SURE loss, without any ground truth clean data. We perform experiments for three different additive Gaussian noise levels having standard deviations of , and ; see Figure 2 for SURE training loss curve for each noise level. The network easily converges for low noise while higher noise levels make convergence harder. For a benchmark, we also trained DnCNN using MSE loss of (2) and compare its performance with the SURE approach.

Figure 2: Figure shows SURE loss as a function of number of steps of training at different noise levels. Note that higher levels of noise make it hard for the network to converge.

For evaluation, we randomly select images from the test set of Indiana University X-Ray dataset. To quantitatively evaluate the performance of proposed SURE based approach, we use two widely used performance metrics, Peak Signal to Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM) (wang2004image, ). PSNR of reconstructed image from true image is defined as for image pixels in the range of 0 and 255. On the other hand, SSIM measure perceived similarity between reconstructed and true image.In addition to Indiana University dataset, we also use images from famous Chest X-Rays dataset (wang2017chestx, ) for testing as well. Table 1 shows quantitative results for both datasets at different noise levels. Figure 3 shows visual results for both datasets for Gaussian noise having a standard deviation of 25. Quantitative and qualitative results show that model trained on Indiana University dataset also has very compelling results on Chest X-Ray data. This demonstrates the generalizability of SURE based approach to other datasets of similar modalities. Table 2 shows quantitative comparison between DnCNN trained using SURE loss and MSE loss. Note that although SURE does not require any clean ground truth images, it’s performance is still comparable to DnCNN trained via supervised MSE loss, which requires ground truth images during training.

(a) Original
(b) Noisy
(c) SURE
(d) MSE
Figure 3: Denoising results for Indiana University (IndianaXrays, ) (first row) and Chest X-Ray Dataset (wang2017chestx, ) (second row) for Gaussian noise with standard deviation of . DnCNN trained on SURE loss is able to remove noise (c) from noisy images (b), without requiring any ground truth training data. Results are comparable against supervised MSE as shown in (d), which uses ground truth images for training DnCNN.
Indiana University
10 37.50 0.959 9.12 0.06
25 32.00 0.881 13.15 0.06
50 27.78 0.763 21.50 0.06
Chest X-Rays 10 37.55 0.962 13.57 0.19
25 31.85 0.877 26.14 0.19
50 28.88 0.774 45.43 0.19
Table 1: Table shows average quantitative results in terms of PSNR, SSIM, and MSE of SURE based DnCNN denoiser for Indiana University X-Rays dataset (IndianaXrays, ) and Chest X-Ray dataset (wang2017chestx, ). The SURE based model shows promising results, even though it does not require any clean ground truth images for training DnCNN. Interestingly DnCNN model trained only on Indiana University X-Rays dataset perform well on Chest X-Ray dataset.
DnCNN-SURE 25 32.00 0.88 13.15
50 27.78 0.76 21.50
DnCNN-MSE 25 35.39 0.95 08.96
50 29.61 0.82 17.32
Table 2: Quantitative denoising results for DnCNN trained using SURE loss and MSE loss for Gaussian noise with standard deviation of 25 and 50. Average values of PSNR, SSIM, and MSE are calculated for 10 images from test set of Indiana University X-ray dataset (IndianaXrays, ) .

4 Discussion and Future Direction

The main contribution of this work is to demonstrate the effectiveness of SURE for unsupervised X-ray denoising. Not only are we able to remove additive noise from X-ray images but also preserve the fine structure in X-ray scans. Our work assumes that true image is corrupted by Gaussian noise with known variance. In our future work, we will extend this SURE based approach to Poisson noise that is more relevant for X-ray imaging, especially in low dose regime. For this, we can use transforms to first convert Poisson noise to Gaussian noise and then use Gaussian noise removal methods. This is because algorithms proposed for Gaussian noise fails to give plausible results on Poisson noise.


We gratefully acknowledge the support of the NVIDIA Corporation for the donation of NVIDIA TITAN Xp GPU for our research.


  • [1] Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. Tensorflow: a system for large-scale machine learning. In OSDI, volume 16, pages 265–283, 2016.
  • [2] Hu Chen, Yi Zhang, Weihua Zhang, Peixi Liao, Ke Li, Jiliu Zhou, and Ge Wang. Low-dose ct denoising with convolutional neural network. In Biomedical Imaging (ISBI 2017), 2017 IEEE 14th International Symposium on, pages 143–146. IEEE, 2017.
  • [3] Lovedeep Gondara. Medical image denoising using convolutional denoising autoencoders. In Data Mining Workshops (ICDMW), 2016 IEEE 16th International Conference on, pages 241–246. IEEE, 2016.
  • [4] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521(7553):436, 2015.
  • [5] Sathish Ramani, Thierry Blu, and Michael Unser. Monte-carlo sure: A black-box optimization of regularization parameters for general denoising algorithms. IEEE Transactions on Image Processing, 17(9):1540–1554, 2008.
  • [6] Shakarim Soltanayev and Se Young Chun. Indiana University Chest X-ray Collection. https://openi.nlm.nih.gov/gridquery.php?q=Indiana%20University%20Chest%20X-ray%20Collection, 2013. [Online; accessed 10-Oct-2018].
  • [7] Shakarim Soltanayev and Se Young Chun. Training deep learning based denoisers without ground truth data. arXiv preprint arXiv:1803.01314, 2018.
  • [8] Charles M Stein. Estimation of the mean of a multivariate normal distribution. The annals of Statistics, pages 1135–1151, 1981.
  • [9] Dang NH Thanh and VB Surya Prasath. A review on ct and x-ray images denoising methods. 2018.
  • [10] Xiaosong Wang, Yifan Peng, Le Lu, Zhiyong Lu, Mohammadhadi Bagheri, and Ronald M Summers. Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In Computer Vision and Pattern Recognition (CVPR), 2017 IEEE Conference on, pages 3462–3471. IEEE, 2017.
  • [11] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600–612, 2004.
  • [12] Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE Transactions on Image Processing, 26(7):3142–3155, 2017.
Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
Add comment
Loading ...
This is a comment super asjknd jkasnjk adsnkj
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test description