A progressive ratio task with costly resets reveals adaptive effort-delay tradeoffs

Abstract

The Progressive Ratio (PR) schedule is a popular test for measuring the motivational value of a reinforcer, in which subjects must exert an increasing amount of work to obtain each successive reward. Despite its popularity, the PR task hinges on a low-dimensional behavioral readout--breakpoint, or the maximum work requirement subjects are willing to complete before abandoning the task. Here, we show that with a simple modification, the PR task can be transformed into an optimization problem reminiscent of the patch-leaving foraging scenario, which has been analyzed extensively by behavioral ecologists, psychologists, and neuroscientists. In the Progressive Ratio with Reset (PRR) task, rats perform the PR task on one lever, but can press a second lever to reset the current ratio requirement back to its lowest value at the cost of enduring a reset delay, during which both levers are retracted. Rats used the reset lever adaptively on the PRR task, and their ratio reset decisions were sensitive to the cost of the reset delay. We derived an approach for computing the optimal bout length--the number of rewards to earn before pressing the reset lever that produces the greatest long-term rate of reward--and found that rats flexibly changed their behavior to approximate the optimal strategy. However, rats showed a systematic bias for bout lengths that exceeded the optimal length, an effect reminiscent of "overharvesting" in patch-leaving tasks. The PRR task thus represents a novel means of testing whether and how rats adapt their behavior to the cost-benefit structure of the environment in a way that connects deeply to the broader literature on associative learning and optimal foraging theory.