Gradual proactive regulation of body state by reinforcement learning of homeostasis

Abstract

Living systems maintain physiological variables such as temperature, blood pressure, and glucose within narrow ranges--a process known as homeostasis. Homeostasis involves not only reactive feedback but also anticipatory adjustments shaped by experience. Prior homeostatic reinforcement learning (HRL) models have provided a computational account of anticipatory regulation under homeostatic challenges. However, existing formulations lack mechanisms for gradual, trial-by-trial adjustment and for extinction learning. To address this issue, we developed a continuous HRL framework that enables trial-wise tuning of anticipatory regulation. The model incorporates biologically informed components: asymmetric reinforcement, weighting negative outcomes more than positive outcomes; and a dual-unit, context-gated inhibitory mechanism. We applied the framework to thermoregulatory conditioning with ethanol-induced hypothermia and successfully reproduced cue-triggered compensation, gradual tolerance, and rapid reacquisition after extinction. We then extended the framework to multiple physiological variables influenced by shared neural or hormonal control signals, and found that when regulatory priorities across variables were uneven, a deviation in one variable propagated through shared control to others, yielding a cascading, system-wide failure to return to ideal state (non-recovery)--a pattern reminiscent of autonomic dysregulation (e.g., dysautonomia, ME/CFS). Overall, our framework provides a computational basis to advances a systems-level understanding of multi-organ homeostatic dysregulation in vivo.