Evidence for dimensional representations and anticipatory dynamics in facial expression perception

Abstract

Expression recognition relies on the ability to distinguish subtle visual differences across a range of facial expressions. Here, we examine the neural representation of dynamic expressions as reflected by electroencephalography (EEG) data in human adults. We find that a wide range of expressions (i.e., 14 emotional and 10 conversational expressions) can be decoded from neural signals, and that their representational structure evinces the classic dimensions of valence and arousal. Critically, we recover, through EEG-based video reconstruction, dynamic representations whose content succeeds in capturing even fine differences across related expressions (e.g., happy-satiated versus schadenfreude). Further, time-resolved decoding reveals anticipatory dynamics that maximize accuracy before the occurrence of an apex expression in the visual stimulus. These results are validated against behavioral data, which yield static reconstructions consistent with their neural counterparts. Thus, our results shed light on the representational basis of expression recognition and serve to recover the dynamic content of visual experience.