Contributions of attention to learning in multidimensional reward environments

Abstract

Real-world choice options have many features or attributes, whereas the reward outcome from those options only depends on a few features/attributes. It has been shown that humans learn and combine feature-based with more complex conjunction-based learning to tackle challenges of learning in complex reward environments. However, it is unclear how different learning strategies interact to determine what features should be attended and control choice behavior, and how ensuing attention modulates future learning and/or choice. To address these questions, we examined human behavior during a three-dimensional learning task in which reward outcomes for different stimuli could be predicted based on a combination of an informative feature and conjunction. Using multiple approaches, we first confirmed that choice behavior and reward probabilities estimated by participants were best described by a model that learned the predictive values of both the informative feature and the informative conjunction. In this model, attention was controlled by the difference in these values in a cooperative manner such that attention depended on the integrated feature and conjunction values, and the resulting attention weights modulated learning by increasing the learning rate on attended features and conjunctions. However, there was little effect of attention on decision making. These results suggest that in multidimensional environments, humans direct their attention not only to selectively process reward-predictive attributes, but also to find parsimonious representations of the reward contingencies for more efficient learning. Significance StatementFrom trying exotic recipes to befriending new social groups, outcomes of real-life actions depend on many factors, but how do we learn the predictive values of those factors based on feedback we receive? It has been shown that humans simplify this problem by focusing on individual factors that are most predictive of the outcomes but can extend their learning strategy to include combinations of factors when necessary. Here, we examined interaction between attention and learning in a multidimensional reward environment that requires learning about individual features and their conjunctions. Using multiple approaches, we found that learning about features and conjunctions control attention in a cooperative manner and that the ensuing attention mainly modulates future learning and not decision making.