Imagined speech can be decoded from low- and cross-frequency features in perceptual space

Abstract

Reconstructing intended speech from neural activity using brain-computer interfaces (BCIs) holds great promises for people with severe speech production deficits. While decoding overt speech has progressed, decoding imagined speech have met limited success, mainly because the associated neural signals are weak and variable hence difficult to decode by learning algorithms. Using three electrocorticography datasets totalizing 1444 electrodes from 13 patients who performed overt and imagined speech production tasks, and based on recent theories of speech neural processing, we extracted consistent and specific neural features usable for future BCIs, and assessed their performance to discriminate speech items in articulatory, phonetic, vocalic, and semantic representation spaces. While high-frequency activity provided the best signal for overt speech, both low- and higher-frequency power and local cross-frequency contributed to successful imagined speech decoding, in particular in phonetic and vocalic, i.e. perceptual, spaces. These findings demonstrate that low-frequency power and cross-frequency dynamics contain key information for imagined speech decoding, and that exploring perceptual spaces offers a promising avenue for future imagined speech BCIs.