Real versus virtual faces: Differentiating neural responses to emotional expressions

As avatars become commonplace, understanding how the brain processes emotion expressed by virtual faces is critical. Across two experiments, we compared behavioral and neural responses to real and virtual faces expressing seven emotions (anger, disgust, fear, joy, sadness, surprise, neutral). In Experiment 1 (n=61), participants rated the similarity of paired real and virtual faces. Expressions conveying the same emotion were rated as highly similar across face types, whereas mismatched emotions yielded substantially lower similarity ratings, indicating perceived emotional meaning was preserved despite differences in face realism. In Experiment 2 (n=91), functional near-infrared spectroscopy was used to measure brain activity while participants viewed the same stimuli. Functional connectivity analyses revealed network level differences between face type and emotion across the brain. Real faces elicited stronger connectivity patterns across frontal, central-temporal, and parietal regions, whereas high-arousal emotions (fear, anger, and joy) were associated with a broader network engagement than other expressions. Complementary general-linear-model analyses revealed increased activation was limited to visual areas for virtual faces, surprise, and neutral expressions. Our results suggest face-type processing occurs in early visual areas, while emotion-type are distinguished by distinct patterns of network level connectivity across the brain for real and virtual faces.