Automating Motion Correction in Multishot MRI Using Generative Adversarial Networks

Automating Motion Correction in Multishot MRI Using Generative Adversarial Networks

Siddique Latif Muhammad Asim Information Technology University (ITU)-Punjab, Pakistan Muhammad Usman Information Technology University (ITU)-Punjab, Pakistan Junaid Qadir Information Technology University (ITU)-Punjab, Pakistan Rajib Rana University of Southern Queensland, Australia

Multishot Magnetic Resonance Imaging (MRI) has recently gained popularity as it accelerates the MRI data acquisition process without compromising the quality of final MR image. However, it suffers from motion artifacts caused by patient movements which may lead to misdiagnosis. Modern state-of-the-art motion correction techniques are able to counter small degree motion, however, their adoption is hindered by their time complexity. This paper proposes a Generative Adversarial Network (GAN) for reconstructing motion free high-fidelity images while reducing the image reconstruction time by an impressive two orders of magnitude.


Automating Motion Correction in Multishot MRI Using Generative Adversarial Networks



noticebox[b]32nd Conference on Neural Information Processing Systems (NIPS 2018), Montréal, Canada.\end@float

1 Introduction

Multishot Magnetic Resonance Imaging (MRI) has been an excellent imaging modality which provides high resolution image while addressing some well-known problems of classical standard MRI techniques—such as the slowness of true single-echo sequences and the problems of inflexible contrast and limited resolution in single-shot sequences [1]. However, it suffers from a critical drawback: it is highly sensitive towards subjects’ motion; even very small movement (e.g., due to the respiration or slight head motion) can become a limiting factor for ultra-high resolution images. The common presence of motion artifacts in Multishot MR image—which results from the dislocation of the k-space that is filled fractionally in each shot—is clearly undesirable especially in clinical settings due to the risk of misdiagnosis [2] and must be addressed. Although modern state-of-the-art motion correction techniques (e.g., the iterative algorithms proposed in [3, 4]) are able to counter smaller degree of motion, their adoption is hindered by the inordinate time for motion correction. We address this void by proposing an end-to-end trained Generative Adversarial Network (GAN) for motion correction together with a popular parallel MRI reconstruction algorithm known as Conjugate Gradient Sensitivity Encoding (CG SENSE) to return superior results. Our results show that by removing the motion estimation step involved in the current schemes, and by employing an end-to-end trained GAN, motion artifacts can be faithfully removed to produce high quality images while reducing the image construction time.

Figure 1: The proposed motion correction framework for multishot MRI, where CG SENSE is used to reconstruct the motion-corrupted images, and the generator network of the GAN, in conjunction with the discriminator network, is tasked with motion correction.

2 Model Architecture and Dataset

Our choice of a GAN-based model for motion correction stems from their great success in image restoration problems and even in MRI reconstruction [5], where they have shown their capabilities in producing high-fidelity images [6]. The GAN framework includes a generator and a discriminator which play an adversarial game defined by the following optimization program.


where and denote the motion corrupted and corrected images, respectively. We implemented U-Net [7] like convolutional network architecture for the generator consisting of an encoder and decoder as shown in Figure 2. Every encoder block consists of convolutional layers (each with feature maps) except for the middle layer that has feature maps. The decoder blocks have similar structure as the encoder blocks except that we replace all the convolutional layers with transposed convolutions.

Since there is no public dataset available for motion-corrupted images, we rely on synthetically generated data for evaluating the performance of our proposed method. We have utilized images from the BRATS 2015 dataset [8]. Motion artifacts have been introduced in motion-free images of the dataset by employing the technique proposed in [9], and also utilized by [3] and [4] to evaluate the performance of motion correction techniques. This method consists of three phases: firstly, the introduction of motion in motion-free k-space data for each shot; secondly, the extraction of specific segments of k-space data; and lastly, the combination of segmented k-space data. Since the BRATS dataset [8] is in the image domain, therefore, the additional step of the creation of simulated k-space multishot MRI data, following [10], has been performed.

Figure 2: Architecture of the generator network (U-Net) consisting of encoding and decoding blocks. The discriminator has exactly the same architecture as the encoder part of our U-Net.
Figure 3: Qualitative results of our solution’s output (third row) for increasing degrees of motion , , , , and compared with on motion-corrupted MRI images (first row), groundtruth images (bottom row), and state-of-the-art algorithm [3]’s output (second row). It can be seen that the output of our algorithm is qualitatively better than the output of previous approach.

3 Experiments and Discussion

The simulated k-space motion-corrupted data has been reconstructed using standard CG SENSE algorithm [11]. For each dataset, corresponding to a certain degree of motion, we train our network using RMSProp optimizer with learning rate and batch size of , until convergence. For each update of , we updated twice. In each case, we also pre-train the generator network to allow training to converge faster. The network was trained on of data (13019 images) and all results are reported using remaining 30% of test data (5580 images).

Degree of motion
Peak Signal to Noise Ratio (PSNR) 31.82 28.27 27.64 27.20 26.82
Structural Similarity Index (SSIM) 0.95 0.92 0.91 0.90 0.90
Table 1: Average values of performance metrics using GAN for 2 shot MRI imaging on entire test data.

For the smaller degree of motions, our network successfully captures the underlying statistical properties to recover sharp and excellent images (see Figure 3). Also, high average values for Peak Signal to Noise Ratio (PSNR) and Structural Similarity Index (SSIM) on the entire test data are observed at in Table 1. We observe a smooth performance decay with increasing degrees of motion, as expected, and at , MRI scans are corrupted to an extent where they would be rendered practically useless. Our network removes most of the corruptions, but due to severe degradation, recovers overly smooth images. We choose to compare our approach with the current state of the art method [3]. Notably, this method was able to recover from up to rotations in 2-shot acquisitions, and we are achieving significantly acceptable quality of MR images up to using GANs.

Degree of Motion
Cordero et al. [3]
Our Approach
Table 2: Comparing computational time in seconds of our approach with the current state-of-art technique [3]. The reported times include both reconstruction (using CG SENSE) as well as motion correction time (using GAN) on the entire test data.

We see through our results documented in Table 2 that we are correcting motion at almost time faster then the Cordero et al. [3] method due to the outsourcing of the motion-correction task (which was previously estimated iteratively during with the significant delay) to the GAN.

4 Conclusions

The work in this paper is the first instance of employing a GAN for motion correction in MRI. It works on top of the standard CG SENSE reconstruction similar to the previous method on motion correction [3]. However, it does not estimate motion states prior to its correction, which significantly lessens the computational time that is very crucial for clinical applications. We empirically showed with our preliminary results that generative framework provided by GANs can provide significantly improved results in terms of time complexity and clinically acceptable image quality. In this work we have reported results for shot acquisition, we are aiming in our future work to apply GANs for an increased number of shots and to perform a more extensive set of experiments.


  • [1] Matt A Bernstein, Kevin F King, and Xiaohong Joe Zhou. Handbook of MRI pulse sequences. Elsevier, 2004.
  • [2] Jalal B Andre, Brian W Bresnahan, Mahmud Mossa-Basha, Michael N Hoff, C Patrick Smith, Yoshimi Anzai, and Wendy A Cohen. Toward quantifying the prevalence, severity, and cost associated with patient motion during clinical MR examinations. Journal of the American College of Radiology, 12(7):689–695, 2015.
  • [3] Lucilio Cordero-Grande, Rui Pedro AG Teixeira, Emer J Hughes, Jana Hutter, Anthony N Price, and Joseph V Hajnal. Sensitivity encoding for aligned multishot magnetic resonance reconstruction. IEEE Transactions on Computational Imaging, 2(3):266–280, 2016.
  • [4] Alexander Loktyushin, Hannes Nickisch, Rolf Pohmann, and Bernhard Schölkopf. Blind multirigid retrospective motion correction of MR images. Magnetic resonance in medicine, 73(4):1457–1468, 2015.
  • [5] Lisa Lei Morteza Mardani. Semi-supervised super-resolution gans for mri. 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA., 2017.
  • [6] Jinshan Pan, Yang Liu, Jiangxin Dong, Jiawei Zhang, Jimmy Ren, Jinhui Tang, Yu-Wing Tai, and Ming-Hsuan Yang. Physics-based generative adversarial models for image restoration and beyond. arXiv preprint arXiv:1808.00605, 2018.
  • [7] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pages 234–241. Springer, 2015.
  • [8] Menze et al. The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS). IEEE Transactions on Medical Imaging, page 33, 2014.
  • [9] Mark Bydder, David J Larkman, and Joseph V Hajnal. Detection and elimination of motion artifacts by regeneration of k-space. Magnetic resonance in medicine, 47(4):677–686, 2002.
  • [10] Michael J Allison, Sathish Ramani, and Jeffrey A Fessler. Accelerated regularized estimation of MR coil sensitivities using augmented lagrangian methods. IEEE transactions on medical imaging, 32(3):556–564, 2013.
  • [11] Klaas P Pruessmann, Markus Weiger, Markus B Scheidegger, and Peter Boesiger. Sense: sensitivity encoding for fast MRI. Magnetic resonance in medicine, 42(5):952–962, 1999.
Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
Add comment
Loading ...
This is a comment super asjknd jkasnjk adsnkj
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test description