Synthesising integrated robot behaviour through reinforcement learning for homeostasis

Abstract

Homeostasis is a fundamental property for the survival of animals. Computational reinforcement learning provides a theoretically sound framework for learning autonomous agents. However, the definition of a unified motivational signal (i.e., reward) for integrated survival behaviours has been largely underexplored. Here, we present a novel neuroscience-inspired algorithm for synthesising robot survival behaviour without the need for complicated reward design and external feedback. Our agent, the Embodied Neural Homeostat, was trained solely with feedback generated by its internal physical state and optimised its behaviour to stabilise these internal states: homeostasis. To demonstrate the effectiveness of our concept, we trained the agent in a simulated mechano-thermal environment and tested it in a real robot. We observed the synthesis of integrated behaviours, including walking, navigating to food, resting to cool down the motors, and shivering to warm up the motors, through the joint optimisation for thermal and energy homeostasis. The Embodied Neural Homeostat successfully achieved homeostasis-based integrated behaviour synthesis, which has not previously been accomplished at the motor control level. This demonstrates that homeostasis can be a motivating principle for integrated behaviour generation in robots and can also elucidate the behavioural principles of living organisms.