Cognitive Subscore Trajectory Prediction in Alzheimer’s Disease

Cognitive Subscore Trajectory Prediction in Alzheimer’s Disease

Lev E. Givon The Charles Stark Draper Laboratory, Inc. Laura J. Mariano The Charles Stark Draper Laboratory, Inc. David O’Dowd The Charles Stark Draper Laboratory, Inc. John M. Irvine The Charles Stark Draper Laboratory, Inc. Abraham R. Schneider The Charles Stark Draper Laboratory, Inc. the Alzheimer Disease Neuroimaging Initiative Data used in preparation of this article were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database ( As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in analysis or writing of this report. A complete listing of ADNI investigators can be found at:
July 12, 2019

1 Introduction

Accurate diagnosis of Alzheimer’s Disease (AD) entails clinical evaluation of multiple cognition metrics and biomarkers. Metrics such as the Alzheimer’s Disease Assessment Scale-Cognitive test (ADAS-cog) [7] comprise multiple subscores that quantify different aspects of a patient’s cognitive state such as learning, memory, and language production/comprehension. Although computer-aided diagnostic techniques for classification of a patient’s current disease state exist [4], they provide little insight into the relationship between changes in brain structure and different aspects of a patient’s cognitive state that occur over time in AD.

2 Methods

We created a Convolutional Neural Network (CNN) architecture that maps an input tuple comprising a patient’s current structural MRI (sMRI) scan and a future time (in number of months) to the values of the patient’s 13 ADAS-Cog subscores predicted for that time. The architecture (Fig. 1) consists of

  • 3 convolutional layers, each comprising convolution, maximum pooling, and rectified linear activation;

  • 3 fully connected layers, each comprising linear units and rectified linear activation, where , respectively.

The first convolutional layer’s input consists of the specified sMRI scan; all outputs of the third convolutional layer and the input prediction time are fed to all units in the first fully connected layer. Weights in all convolutional layers were initialized as described in [2]. To prevent overfitting, dropout [8] with a probability of 0.5 was performed after max pooling in every convolutional layer during training. Optimization was performed using the RMSProp [3] algorithm. A smooth loss function was employed to reduce sensitivity to outliers [1]. The architecture was implemented in PyTorch [6] and trained on Ubuntu Linux 16.04 with NVIDIA Tesla M40 Graphics Processing Units (GPUs).

Figure 1: Convolutional neural network architecture for prediction of multiple cognitive subscores.

To evaluate the architecture, we performed 5-fold cross-validation with training and testing data drawn from a set of tuples of sMRI scans, cognitive scores, and time intervals between MRI and cognitive score acquisition. Intervals were multiples of 6 months between 0 and 36. Each fold was stratified to contain equal numbers of subscores for each unique interval. To ensure that each fold contained a sufficient number of tuples for each interval, the folds were drawn from a dataset comprising 1000 entries. This dataset contained multiple images for some patients, although the number of images obtained from an individual patient across the dataset was not fixed owing to the size of the parent ADNI dataset from which the images were drawn.

All sMRI and cognitive score data used for evaluation of the architecture were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database ( The primary goal of ADNI has been to test whether serial MRI, PET, other biological markers, and clinical and neuropsychological assessment can be combined to measure the progression of mild cognitive impairment (MCI) and early Alzheimer’s disease (AD). Only 1.5T MRI scans from a quality-controlled subset of the ADNI1 phase of the project were utilized [13]. sMRI scans were minimally preprocessed to remove non-brain tissue, normalize intensities, and register all scans to a single coordinate space. All subscores were normalized to the range prior to training and testing.

3 Results

We computed the root mean squared error (RMSE) for each of the 13 subscores for each interval (Fig. 2). The means and standard errors across all intervals and subscores varied from 0.009 to 0.251 and from 0.001 to 0.046, respectively. To compare the performance of our network with other approaches that only predict the aggregate score, we also computed the RMSE and Pearson’s correlation coefficient of the predicted/actual aggregate score normalized to the range (Fig. 3). We obtained the latter by rescaling the normalized subscores to their original respective ranges and normalizing the sum using the highest possible aggregate score. The means and standard errors of the RMSEs of our method’s normalized aggregate score prediction for the intervals considered varied from 0.058 to 0.087 and from 0.014 to 0.028, respectively. The means and standard errors of the correlation between predicted and actual aggregate scores across all intervals varied from 0.643 to 0.862 and from 0.040 to 0.149, respectively.

Figure 2: RMSEs of predicted normalized subscores.
Figure 3: RMSEs and Pearson’s correlation coefficients of predicted normalized aggregate scores.

4 Discussion

To our knowledge, our architecture is the first to concurrently predict multiple cognitive examination subscores from minimally preprocessed structural brain data. Variations in prediction accuracy across the subscores illustrate differences in the relationships between brain structure and specific aspects of cognition that are obscured by prediction of the aggregate score. The mean performance of our architecture when applied to aggregate score prediction is similar to that of existing techniques that variously utilize stepwise regression [10], relevance vector regression [9], multi-task learning [12, 11, 15], and support vector machines [14]. Since the quality of features extracted by CNNs is proportional to the amount of training data and number of network layers [5], we anticipate that deeper variations of our architecture trained on more extensive subsets of the ADNI database will be able to achieve state-of-the-art performance.

5 Conclusion

We have developed a CNN architecture that can predict the trajectories of the 13 subscores comprised by a subject’s ADAS-cog examination results from a current sMRI scan up to 36 months from image acquisition time without resorting to manual feature extraction. Mean performance metrics are within range of those of existing techniques that require manual feature selection and are limited to predicting aggregate scores.

6 Keywords

Alzheimer’s Disease, deep learning, convolutional neural network, cognitive assessment, machine learning, MRI

7 Acknowledgments

Data collection and sharing for this project was funded by the Alzheimer’s Disease Neuroimaging Initiative (ADNI, principal investigator: Michael Weiner) (National Institutes of Health Grant U01 AG024904) and DOD ADNI (Department of Defense award number W81XWH-12-2-0012). ADNI is funded by the National Institute on Aging, and the National Institute of Biomedical Imaging and Bioengineering. ADNI data are disseminated by the Laboratory for Neuro Imaging at the University of Southern California.


  • [1] Ross Girshick. Fast R-CNN. In 2015 IEEE International Conference on Computer Vision (ICCV), pages 1440–1448, December 2015.
  • [2] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. In Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), ICCV ’15, pages 1026–1034, Washington, DC, USA, 2015. IEEE Computer Society.
  • [3] Geoffrey Hinton, Nitish Srivastava, and Kevin Swersky. Introduction to Neural Networks and Machine Learning - Lecture 6, 2014.
  • [4] Ehsan Hosseini-Asl, Robert Keynton, and Ayman El-Baz. Alzheimer’s Disease Diagnostics by Adaptation of 3D Convolutional Network. In 2016 IEEE International Conference on Image Processing (ICIP), pages 126–130, September 2016.
  • [5] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. ImageNet Classification with Deep Convolutional Neural Networks. In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 25, pages 1097–1105. Curran Associates, Inc., 2012.
  • [6] Adam Paszke, Soumith Chintala, Ronan Collobert, Koray Kavukcuoglu, Clement Farabet, Samy Bengio, Iain Melvin, Jason Weston, and Johnny Mariethoz. PyTorch: Tensors and Dynamic Neural Networks in Python with Strong GPU Acceleration, May 2017.
  • [7] Wilma G. Rosen, Richard C. Mohs, and Kenneth L. Davis. A New Rating Scale for Alzheimer’s Disease. The American Journal of Psychiatry, 141(11):1356–1364, November 1984.
  • [8] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. Journal of Machine Learning Research, 15:1929–1958, 2014.
  • [9] Cynthia M. Stonnington, Carlton Chu, Stefan Klöppel, Clifford R. Jack Jr., John Ashburner, and Richard S. J. Frackowiak. Predicting Clinical Scores from Magnetic Resonance Scans in Alzheimer’s Disease. NeuroImage, 51(4):1405–1413, July 2010.
  • [10] K. B. Walhovd, A. M. Fjell, A. M. Dale, L. K. McEvoy, J. Brewer, D. S. Karow, D. P. Salmon, and C. Fennema-Notestine. Multi-Modal Imaging Predicts Memory Performance in Normal Aging and Cognitive Decline. Neurobiology of Aging, 31(7):1107–1121, July 2010.
  • [11] J. Wan, Z. Zhang, J. Yan, T. Li, B. D. Rao, S. Fang, S. Kim, S. L. Risacher, A. J. Saykin, and L. Shen. Sparse Bayesian Multi-Task Learning for Predicting Cognitive Outcomes from Neuroimaging Measures in Alzheimer’s Disease. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pages 940–947, June 2012.
  • [12] Hua Wang, Feiping Nie, Heng Huang, Shannon Risacher, Chris Ding, Andrew J Saykin, and Li Shen. Sparse Multi-Task Regression and Feature Selection to Identify Brain Imaging Predictors for Memory Performance. Proceedings / IEEE International Conference on Computer Vision. IEEE International Conference on Computer Vision, pages 557–562, 2011.
  • [13] Bradley T. Wyman, Danielle J. Harvey, Karen Crawford, Matt A. Bernstein, Owen Carmichael, Patricia E. Cole, Paul Crane, Charles DeCarli, Nick C. Fox, Jeffrey L. Gunter, Derek Hill, Ronald J. Killiany, Chahin Pachai, Adam J. Schwarz, Norbert Schuff, Matthew L. Senjem, Joyce Suhy, Paul M. Thompson, Michael Weiner, and Clifford R. Jack. Standardization of Analysis Sets for Reporting Results from ADNI MRI Data. Alzheimer’s & Dementia: the Journal of the Alzheimer’s Association, 9(3):332–337, May 2013.
  • [14] Daoqiang Zhang and Dinggang Shen. Multi-Modal Multi-Task Learning for Joint Prediction of Multiple Regression and Classification Variables in Alzheimer’s Disease. NeuroImage, 59(2):895–907, January 2012.
  • [15] Jiayu Zhou, Jun Liu, Vaibhav A. Narayan, and Jieping Ye. Modeling Disease Progression Via Multi-Task Learning. NeuroImage, 78:233–248, September 2013.
Comments 0
Request Comment
You are adding the first comment!
How to quickly get a good reply:
  • Give credit where it’s due by listing out the positive aspects of a paper before getting into which changes should be made.
  • Be specific in your critique, and provide supporting evidence with appropriate references to substantiate general statements.
  • Your comment should inspire ideas to flow and help the author improves the paper.

The better we are at sharing our knowledge with each other, the faster we move forward.
The feedback must be of minimum 40 characters and the title a minimum of 5 characters
Add comment
Loading ...
This is a comment super asjknd jkasnjk adsnkj
The feedback must be of minumum 40 characters
The feedback must be of minumum 40 characters

You are asking your first question!
How to quickly get a good answer:
  • Keep your question short and to the point
  • Check for grammar or spelling errors.
  • Phrase it like a question
Test description