A Comparative Evaluation of Structural MRI Foundation Models for Age, Sex, and Body-Mass Index Predictions

Foundation models pre-trained on large neuroimaging datasets offer a promising approach to overcome the limited sample sizes typical of clinical imaging studies, yet their generalization across diverse populations remains unclear. We present the first systematic benchmark of four publicly available structural MRI foundation models: AnatCL, BrainIAC, 3D-Neuro-SimCLR, and SwinBrain. Using T1-weighted MRIs from the Parkinson's Progression Markers Initiative (PPMI), Healthy Brain Network (HBN), and Nathan Kline Institute (NKI) datasets, we evaluate these models on sex classification, brain age prediction, and body mass index prediction, comparing against models trained from FreeSurfer-derived cortical thickness and cortical surface area features. Submitted models are evaluated using a standardized frozen feature probing framework. The evaluation methods are available in BrainFMBench, a living benchmark for structural brain MRI foundation models hosted on GitHub, where new models can be added through pull requests. Although some foundation models outperformed FreeSurfer on particular tasks and datasets, 3D-Neuro-SimCLR and AnatCL outperformed the baselines overall, with 3D-Neuro-SimCLR demonstrating the most consistent performance (with the notable exception of HBN sex classification). The remaining models did not consistently outperform the baselines, indicating that the advantage of learned representations over morphometric features was not consistent across datasets and tasks for these models. In addition, cross-model feature correlation analysis reveals that foundation model representations correlate differently with traditional cortical measurements. These findings position structural MRI foundation models, particularly 3D-Neuro-SimCLR and AnatCL, as promising avenues to boost the performance of predictive models in neuroimaging.