Embracing naturalistic paradigms: substituting GPT predictions for human judgments

Abstract

Naturalistic paradigms can assure ecological validity and yield novel insights in psychology and neuroscience. However, using behavioral experiments to obtain the human ratings necessary to analyze data collected with these paradigms is usually costly and time-consuming. Large language models like GPT have great potential for predicting human-like behavioral judgments. The current study evaluates the performance of GPT as a substitute for human judgments for affective dynamics in narratives. Our results revealed that GPTs inference of hedonic valence dynamics is highly correlated with human affective perception. Moreover, the inferred neural activity based on GPT-derived valence ratings is similar to inferred neural activity based on human judgments, suggesting the potential of using GPTs prediction as a reliable substitute for human judgments.