Inferring the causes of noise from binary outcomes: A normative theory of learning under uncertainty

Abstract

Inferring the true cause of noise--distinguishing between volatility (environmental change) and stochasticity (outcome randomness)--is essential for learning in noisy environments. While most studies rely on binary outcomes, previous models are designed for continuous outcome and use ad hoc approximations to handle binary data, introducing theoretical inconsistencies and interpretational issues. Here, we develop a normative framework for inferring the causes of noise from binary feedback that remains faithful to the discrete nature of the generative process and underlying statistical structure. First, we establish a generative model using a state space approach tailored for binary outcomes and derive the corresponding hidden Markov model inference procedure. Second, we introduce a computational model combining the hidden Markov model with particle filtering to simultaneously infer volatility and stochasticity from binary outcomes. Third, we validate predictions through a 2x2 probabilistic reversal learning task with human participants, systematically manipulating both noise parameters. Results show that participants adjust their learning rates consistent with model predictions, increasing learning rates under volatile conditions and decreasing them under high stochasticity. Our theoretical and experimental results offer a principled approach for dissociating volatility and stochasticity from binary outcomes, providing insights into learning processes relevant to typical cognition and psychiatric conditions.