Predicting Dementia Severity by Merging Anatomical and Diffusion MRI with Deep 3D Convolutional Neural Networks

Machine learning methods have been used for over a decade for staging and subtyping a variety of brain diseases, offering fast and objective methods to classify neurodegenerative diseases such as Alzheimers disease (AD). Deep learning models based on convolutional neural networks (CNNs) have also been used to infer dementia severity and predict future clinical decline. Most CNN-based deep learning models use T1-weighted brain MRI scans to identify predictive features for these tasks. In contrast, we examine the added value of diffusion-weighted MRI (dMRI) - a variant of MRI, sensitive to microstructural tissue properties - as an additional input in CNN-based models of dementia severity. dMRI is sensitive to microstructural brain abnormalities not evident on standard anatomical MRI. By training CNNs on combined anatomical and diffusion MRI, we hypothesize that we could boost performance when predicting widely-used clinical assessments of dementia severity, such as individuals scores on the ADAS11, ADAS13, and MMSE (mini-mental state exam) clinical scales. For benchmarking, we evaluate CNNs that use T1-weighted MRI and dMRI to estimate "brain age" - the task of predicting a persons chronological age from their neuroimaging data. To assess which dMRI-derived maps were most beneficial, we computed DWI-derived diffusion tensor imaging (DTI) maps of mean and radial diffusivity (MD/RD), axial diffusivity (AD) and fractional anisotropy (FA) for 1198 elderly subjects (age: 74.35 +/- 7.74 yrs.; 600 F/598 M, with a distribution of 636 CN/421 MCI/141 AD) from the Alzheimers Disease Neuroimaging Initiative (ADNI). We tested both 2D Slice CNN and 3D CNN neural network models for the above predictive tasks. Our results suggest that for at least some deep learning architectures, diffusion-weighted MRI may enhance performance for several AD-relevant deep learning tasks relative to using T1-weighted images alone.