Mora-ERP-based RNN-Transformer for decoding single-trial EEGs during silent Japanese speeches

We developed a method for decoding single-trial electroencephalography (EEG) during silent Japanese speeches. In order to cope with problems that there would be always noises in single-trial EEGs, a recurrent neural network (RNN) was used which could reproduce signals under noises. Each of silent-mora-related potentials and the single-trial EEG minus the event-related potential (ERP) were assigned to the signal and the noise, respectively, with reference to the averaging principle. Next, in our Transformer, dot product between the RNN output and the single-trial EEG after positional encoding then Softmax with Loss yielded probabilities of moras, each of which consists of silent Japanese words, phrases or part of sentences. The present decoding was completed by tracing the maximal probability at each block representing time. Average mora error rates (MERs) on pretrained and validated performances for the patient was as low as 1.5 % and 0 %, respectively. The performance for the testing would be refined by many single-trial EEGs during silent Japanese speeches obtained by EEG Web interfaces. This method might be applied to other "mora" languages.