Reliability of decision-making and reinforcement learning computational parameters
Anahit MkrtchianiD, Vincent ValtoniD, Jonathan Paul RoiseriD
Abstract
Computational models can offer mechanistic insight into cognition and therefore have the potential to transform our understanding of psychiatric disorders and their treatment. For translational efforts to be successful, it is imperative that computational measures capture individual characteristics reliably. Here we examine the reliability of reinforcement learning and economic models derived from two commonly used tasks. Healthy individuals (N=50) completed a restless four-armed bandit and a calibrated gambling task twice, two weeks apart. Reward and punishment learning rates from the reinforcement learning model showed good reliability and reward and punishment sensitivity from the same model had fair reliability; while risk aversion and loss aversion parameters from a prospect theory model exhibited good and excellent reliability, respectively. Both models were further able to predict future behaviour above chance within individuals. This prediction was better when based on participants own model parameters than other participants parameter estimates. These results suggest that reinforcement learning, and particularly prospect theory parameters, as derived from a restless four-armed bandit and a calibrated gambling task, can be measured reliably to assess learning and decision-making mechanisms. Overall, these findings indicate the translational potential of clinically-relevant computational parameters for precision psychiatry.
02 /Specification & Classification
Click any tag to browse matching papers- Citations
- 36
- FWCI
- 4.95
- Field percentile
- top 4%
- Organism
- Approach
Reproducibility ledger
Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.
Sign in to file a reproducibility report.
Related papers
- The Interpretation of Computational Model Parameters Depends on the Context
Maria Katharina Eckstein, Sarah L. Master, … Anne Gabrielle Eva Collins·eLifePeer-reviewed·Sep 12, 2022
- Harnessing the flexibility of neural networks to predict dynamic theoretical parameters underlying human choice behavior
Yoav Ger, Eliya Nachmani, … Nitzan Shahar·PLoS Computational BiologyPeer-reviewed·Jan 4, 2024
- Joint Modeling of Reaction Times and Choice Improves Parameter Identifiability in Reinforcement Learning Models
Ian C. Ballard, Samuel M. McClure·Journal of Neuroscience MethodsPeer-reviewed·Jan 18, 2019
- Computational modeling of reinforcement learning and functional neuroimaging of probabilistic reversal dissociates compulsive behaviors in Gambling and Cocaine Use Disorders
Katharina Zühlsdorff, Juan Verdejo‐Román, … Jonathan W. Kanen·BJPsych OpenPeer-reviewed·Dec 11, 2023
- Better than maximum likelihood estimation of model-based and model-free learning style
Sadjad Yazdani, Abdol‐Hossein Vahabie, … Majid Nili Ahmadabadi·bioRxiv·Apr 6, 2018
- Deformed Probability Estimation in Goal-Directed reinforcement learning model explains anxious-depression dimensions of psychiatric disorders
Sadjad Yazdani, Majid Nili Ahmadabadi, … Abdol‐Hossein Vahabie·Brain Research BulletinPeer-reviewed·Sep 1, 2026