A multi-scale fusion CNN model based on adaptive transfer learning for multi-class MI-classification in BCI system

Abstract

Deep learning-based brain-computer interface (BCI) in motor imagery (MI) has emerged as a powerful method for establishing direct communication between the brain and external electronic devices. However, due to inter-subject variability, inherent complex properties, and low signal-to-noise ratio (SNR) in electroencephalogram (EEG) signal are major challenges that significantly hinders the accuracy of the MI classifier. To overcome this, the present work proposes an efficient transfer learning-based multi-scale feature fused CNN (MSFFCNN) which can capture the distinguishable features of various non-overlapping canonical frequency bands of EEG signals from different convolutional scales for multi-class MI classification. In order to account for inter-subject variability from different subjects, the current work presents 4 different model variants including subject-independent and subject-adaptive classification models considering different adaptation configurations to exploit the full learning capacity of the classifier. Each adaptation configuration has been fine-tuned in an extensively trained pre-trained model and the performance of the classifier has been studied for vast range of learning rates and degrees of adaptation which illustrates the advantages of using an adaptive transfer learning-based model. The model achieves an average classification accuracy of 94.06% ({+/-}2.29%) and kappa value of 0.88 outperforming several baseline and current state-of-the-art EEG-based MI classification models with fewer training samples. The present research provides an effective and efficient transfer learning-based end-to-end MI classification framework for designing a high-performance robust MI-BCI system.