Comparison of dimensionality reduction and feature selection for cognitive task decoding in functional magnetic resonance imaging

Abstract

BackgroundAdvances in functional magnetic resonance imaging (fMRI) have led to the ability to study the brain across many contexts. However, the large number of features generated by functional connectivity approaches may overfit the data. These problems can be overcome with either feature selection (FS) or dimensionality reduction (DR), which can be applied to less complex models. We utilize two open-source datasets to compare the performance of DR/FS methods on cognitive task decoding using a suite of ML classifiers. New MethodWhile DR and FS methods have been used previously in decoding research, no systematic comparison of their performance has been undertaken. Here, we compare available methods using commonly utilized machine learning libraries to establish which methods provide the best predictive performance. We then conduct statistical tests to examine the relative contributions of DR and FS methods and classifiers on decoding accuracy. ResultsNeither DR or FS was found to be superior. However, differences were identified across datasets and tasks. In the majority of methods and datasets, a peak in predictive performance was found using a small percentage of the total number of original features. Comparison with existing methodsSome methods perform better than the baseline method of prediction with all available features or selecting features randomly. Decoding performance utilizing the HCP datasets with certain DR/FS methods exceeds that of deep learning approaches. ConclusionsSimple machine learning models with DR/FS have competitive decoding performance. These results suggest a "sweet spot" for the tradeoff between the retention of features and predictive accuracy.