Reinforcement Learning in Robotics

Inferring Time-Varying Internal Models of Agents Through Dynamic Structure Learning

This paper introduces Dynamic Structure Learning (DSL), a framework that models how agents adapt their internal structures over time, applied to rats navigating a maze, showing evolving mental maps and learning rules.

Composing egocentric and allocentric maps for flexible navigation

This paper develops a reinforcement learning agent that combines egocentric and allocentric successor representations to improve navigation in complex environments.

Estimating the motor exploration in reinforcement learning

The paper estimates motor exploration strategies in songbirds using a reinforcement learning framework and finds that non-ideal variability aligns with suboptimal learning trajectories.

Trial-by-trial learning of successor representations in human behavior

The paper investigates how humans learn successor representations using trial-by-trial reaction times in graph sequence learning tasks, finding that eligibility trace mechanisms play a predominant role.

Fictive Learning in Model-based Reinforcement Learning by Generalized Reward Prediction Errors

This paper introduces a fictive learning rule in model-based reinforcement learning that generalizes reward prediction errors to unvisited states and actions, explaining behavioral phenomena in the two-step task.

Recurrent Neural Network Exploration Strategies During Reinforcement Learning Depend on Network Capacity

This study examines how RNN capacity affects exploration strategies in reinforcement learning, finding that higher capacity networks exhibit more human-like strategies but still differ in perseveration and switch rates.

Even if suboptimal, novelty drives human exploration

This study shows humans persistently explore stochastic environments despite suboptimal strategies, driven by novelty rather than optimal information gain.

An inductive bias for slowly changing features in human reinforcement learning

This study shows humans learn better from slowly changing reward-predicting features in reinforcement learning tasks, suggesting an inductive bias toward slow features.

Inductive biases of neural specialization in spatial navigation

This study shows that specialized neural modules improve learning and generalization in navigation tasks, resembling primate behavior and using a Kalman filter-like state representation.

Humans adaptively select different computational strategies in different learning environments.

The paper shows humans adaptively choose between flat and hierarchical learning strategies based on environmental complexity.

Goal Reduction with Loop-Removal Accelerates RL and Models Human Brain Activity in Goal-Directed Learning

This paper introduces a goal reduction mechanism that accelerates reinforcement learning and models human brain activity in goal-directed learning by deriving subgoals through loop-removal.

Emergence of belief-like representations through reinforcement learning

This paper shows that recurrent neural networks can estimate value directly from observations without explicitly forming beliefs, generating reward prediction errors similar to experimental observations.

Having "multiple selves" helps learning agents explore and adapt in complex changing worlds

This paper shows that modular agents, composed of subagents for distinct needs, outperform monolithic agents in maintaining homeostasis in changing environments.

CoBeL-RL: A neuroscience-oriented simulation framework for complex behavior and learning

CoBeL-RL is a simulation framework for modeling complex behavior and learning in neuroscience, offering virtual environments and RL algorithms for efficient simulation and analysis.

Novelty is not Surprise: Human exploratory and adaptive behavior in sequential decision-making

This study introduces a deep sequential decision-making paradigm to explain human behavior by incorporating novelty and surprise, showing that model-free actions dominate decisions despite the availability of a world-model.

Temporal and state abstractions for efficient learning, transfer and composition in humans

This study investigates how humans use hierarchical temporal and state abstractions to transfer knowledge across tasks, showing transfer effects not explained by simpler models.

Multi-Task Reinforcement Learning in Humans

This study investigates human multi-task reinforcement learning by comparing participant behavior to algorithms in a two-step decision-making task with changing rewards.

Computational evidence for hierarchically-structured reinforcement learning in humans

This study shows human behavior in complex environments can be explained by hierarchical reinforcement learning, outperforming flat RL and hierarchical Bayesian models in capturing observed patterns.

Computational neural mechanisms of goal-directed planning and problem solving

This paper introduces a localist neural network model that autonomously learns environment structures and achieves arbitrary goals without re-learning reward values, demonstrating its ability to solve complex problems like the Tower of Hanoi.

Computational characteristics of the striatal dopamine system described by reinforcement learning with fast generalization

This paper introduces a reinforcement learning model inspired by the striatal dopamine system, showing it enables fast generalization and safe exploration but can also lead to maladaptive behaviors.

Computational mechanisms of curiosity and goal-directed exploration

The paper presents two computational mechanisms for goal-directed exploration based on different types of information-gain, showing how they induce distinct patterns of Bayes-optimal behavior.

Learning and Forgetting Using Reinforced Bayesian Change Detection

This paper introduces a Bayesian learning model with adaptive forgetting to improve reinforcement learning by dynamically adjusting to environmental changes and balancing exploration-exploitation.

Selective Maintenance of Value Information Helps Resolve the Exploration/Exploitation Dilemma

This paper shows that selectively maintaining high-value information helps resolve the exploration/exploitation dilemma in decision-making, performing as well as uncertainty-driven exploration but with lower cognitive load.

Modeling Sensory-Motor Decisions in Natural Behavior

The paper proposes a modular reinforcement learning model to estimate rewards and discount factors from human behavior data, enabling accurate prediction of navigation decisions in virtual reality.

Compositional clustering in task structure learning

This paper introduces a Bayesian agent that learns to generalize task structures by clustering transition and reward functions independently, showing that performance depends on task domain statistics.

Predictive representations can link model-based reinforcement learning to model-free mechanisms

This paper proposes a framework linking model-based reinforcement learning to model-free mechanisms using successor representations and introduces algorithms that mitigate their limitations.