Evaluating the effects of regularization and cross-validation parameters on the performance of SVM-based decoding of EEG data

Abstract

Regularization has been extensively used in multivariate pattern classification (MVPA; decoding) of EEG data to mitigate the risk of overfitting. N-fold cross-validation is also used to mitigate this risk, and it is often combined with averaging across trials to improve the SNR. However, the impact of different regularization and cross-validation parameters on decoding performance remains unclear. This study aimed to evaluate the effects of variations in the support vector machine (SVM) regularization parameter (C) and the number of crossfolds (and the number of trials per average) on the performance of SVM-based decoding analyses. To achieve this, we examined the decoding performance in relatively simple binary classification tasks from seven commonly used event-related potential paradigms (N170, mismatch negativity, N2pc, P3b, N400, lateralized readiness potential, and error-related negativity). Additionally, we evaluated the decoding performance in more challenging multiclass tasks, including decoding face identity, facial expression, stimulus location, and stimulus orientation. The results revealed that both decoding accuracy and effect size were highest when the regularization strength was equal to or greater than 1. Furthermore, using between 3 and 5 folds with at least 10 trials per average yielded optimal decoding performance in most cases. Researchers applying SVM-based decoding to datasets similar to those examined here--in terms of population, recording systems, class numbers, and paradigms--might benefit from using the parameters that we found to be optimal here.