Behavioral and ERP Markers of Emotion Recognition: Comparing Embodied, Facial, and Emoji Stimuli

Abstract

While extensive research has been conducted on emotion recognition from facial stimuli, it remains unclear how embodied emotional cues conveyed through body postures, gestures and actions (e.g., stick figures) compare with facial (human faces) and face-like symbolic representations (emoji faces) in shaping behavioral and neural responses. We employed a multimodal approach, combining behavioral measures (accuracy and reaction time) with electroencephalography (EEG) to examine the electrophysiological correlates of emotion recognition. Overall, sixty-six (equal number of males and females; 18-30 year old) participants identified positive, negative, and neutral emotions depicted in the three formats. Behavioral results showed significant effects of both emotion and format on accuracy and reaction time. Emoji faces were recognized with the highest accuracy and fastest reaction times, followed by stick figures and then human faces (p .05). Correlation analyses revealed no significant associations between ERP and behavioral measures. Thus, abstract, minimalistic representations like stick figures elicit enhanced early emotion-related processing despite similar early and later processing across formats. HighlightsO_LIEmotion recognition across realistic (human faces), symbolic (emoji faces) and embodied (stick figures) modalities were compared. C_LIO_LIEmotions were recognized fastest and most accurately in emoji faces format. C_LIO_LIStick figures format elicited significantly higher EPN compared to other two formats across positive and negative emotions. C_LIO_LINo significant differences were observed in terms of N170 and LPP. C_LIO_LIEmbodied, abstract emotional cues modulate automatic emotional appraisal, rather than initial sensory encoding and sustained cognitive evaluation. C_LI