Reduced Successor Representation Potentially Interferes with Cessation of Habitual Reward-Seeking

Abstract

Difficulty in cessation of drinking, smoking, or gambling has been widely recognized. Conventional theories proposed relative dominance of habitual over goal-directed control, but human studies have not convincingly supported them. Referring to the recently suggested "successor representation" of states that enables partially goal-directed control, we propose a dopamine-related mechanism potentially underlying the difficulty in resisting habitual reward-seeking, common to substance and non-substance reward. Consider that a person has long been taking a series of actions leading to a certain reward without resisting temptation. Given the suggestions of the successor representation and the dimension reduction in the brain, we assumed that the person has acquired a dimension-reduced successor representation of states based on the goal state under the established non-resistant policy. Then, we show that if the person changes the policy to resist temptation, a large positive reward prediction error (RPE) becomes generated upon eventually reaching the goal, and it sustains given that the acquired state representation is so rigid that it does not change. Inspired by the anatomically suggested spiral striatum-midbrain circuit and the theoretically proposed spiraling accumulation of RPE bias in addiction, we further simulated the influence of RPEs generated in the goal-based representation system on another system representing individual actions. We then found that such an influence could potentially enhance the propensity of non-resistant choice. These results suggest that the inaccurate value estimation in the reduced successor representation system and its influence through the spiral striatum-midbrain circuit might contribute to the difficulty in cessation of habitual reward-seeking.