Multisite Test-Retest Reliability and Compatibility of Brain Metrics derived from FreeSurfer Versions 7.1, 6.0, and 5.3

Abstract

Automatic neuroimaging segmentation and parcellation tools provide convenient and systematic methods for extracting numerous features from brain MRI scans, and are becoming standard practice for large-scale coordinated studies. One such tool, FreeSurfer, provides an easy-to-use pipeline to extract metrics describing cortical and subcortical morphometry. Over the past two decades, there have been over 25 stable releases of FreeSurfer, and different versions are used across published works. Despite this, the reliability and compatibility of metrics derived from the most recent major version releases have yet to be assessed empirically. Here, we use test-retest data from three public brain MRI datasets to assess within-version reliability and between-version compatibility across 42 regional outputs from three versions of FreeSurfer: the latest, v7.1, and two previous stable releases - v5.3, and v6.0. We find v7.1 was less compatible with older versions for measuring cortical thickness. In particular, the thickness of the cingulate gyrus had low compatibility (intraclass correlation coefficient (ICC) between 0.37 and 0.61) between versions. Temporal and frontal poles, and the medial orbitofrontal surface area metrics, also showed low to moderate compatibility with v7.1. While our work compares all three versions, our sub-comparisons between the older versions (v5.3 and v6.0) replicates earlier findings of low compatibility of pallidum and putamen volumes. Low between-version compatibility was not always indicative of low within-version reliability - all versions showed good to excellent reliability across most regional measures (ICC>0.8). Age associations, quality control metrics, and Dice coefficients in an independent sample of 106 individual scans, processed with all three versions of FreeSurfer, revealed differences in results of downstream statistical analysis. As neuroimaging studies adopt more recently released software, we provide researchers with a reference to highlight the regions and metrics that may yield findings inconsistent with published works using older FreeSurfer software. An interactive viewer for the results is provided at http://data.brainescience.org/Freesurfer_Reliability/