EEG correlates of physical effort and reward processing during reinforcement learning

Abstract

Effort-based decision making is often described by choices according to subjective value, a function of reward discounted by effort. We asked whether a neural reinforcement learning signal, the feedback related negativity (FRN), is modulated not only by reward outcomes but also physical effort. We recorded EEG from human participants while they performed a task in which they were required to accurately produce target levels of muscle activation to receive rewards. Participants performed isometric knee extensions while quadriceps muscle activation was recorded using EMG. Real-time feedback indicated muscle activation relative to a target. On a given trial, the target muscle activation required either low or high effort. The effort was determined probabilistically according to a binary choice, such that the responses were associated with 20% and 80% probability of high effort. This contingency could only be known by experience, and it reversed periodically. After each trial binary reinforcement feedback was provided to indicate whether participants were sufficiently accurate in producing the target muscle activity. Participants adaptively avoided effort by switching responses more frequently after choices that resulted in hard effort. Feedback after participants choices which revealed the resulting effort requirement for the subsequent knee extension did not elicit an FRN component. However, the neural response to reinforcement feedback after the knee extension was increased during and after the time period of the FRN by preceding physical effort. Thus, retrospective effort modulates reward processing which may underlie paradoxical behavioral findings whereby rewards requiring more effort to obtain can become more powerful reinforcers. Significance StatementWhen making decisions, we typically select more rewarding and less effortful options. Neural reinforcement learning signals reinforce rewarding actions and deter punishing actions. When participants received feedback that their choices would require easy or hard physical effort, we did not observe reinforcement learning signals that are typically observed in response to feedback predicting reward and punishment. Thus, the reinforcement learning system does not strictly treat effort as loss or punishment. However, when the effort was completed and participants received feedback indicating whether they successfully achieved a reward or not, reinforcement learning signals were amplified by preceding effort. Thus, retrospective effort can affect neural responses to reinforcement outcomes, which may explain how effort can enhance the motivational effect of reinforcers.