Predicting human prediction error empowers reward learning task design

Abstract

Environmental conditions affect human reward prediction. Stable environments foster accurate prediction but constrain learning opportunities, whereas uncertain environments diminish predictability. This stability-uncertainty dilemma complicates task design. We conceptualize this challenge as a task learning paradigm termed meta-prediction - predicting human prediction itself. The meta-prediction entwines two Bellman equations: one emulating human reward learning while the other generates new tasks by predicting the prediction error arising from the first. The meta-prediction with 82 subjects data generated subject-independent tasks across four distinct scenarios. These tasks orchestrate foraging and uncertainty conditions, confirming our frameworks task design ability. Moreover, their mechanistic interpretability provides insight into human reward learning. An independent fMRI study with 49 individuals validated that these tasks effectively modulated behavior and neural activities in prediction error encoding regions, including ventral striatum and lateral prefrontal cortex. Lastly, we demonstrated its compositional capacity to generate complex tasks, uncovering intrinsic biases in human reward learning.