Reinforcement learning in simple instrumental tasks is usually modeled as a monolithic process in which reward prediction errors are used to update expected values of choice options. This modeling ignores the different contributions of different memory and decision-making systems thought to contribute even to simple learning. In an fMRI experiment, we asked how working memory and incremental reinforcement learning processes interact to guide human learning. Working memory load was manipulated by varying the number of stimuli to be learned across blocks. Behavioral results and computational modeling confirmed that learning was best explained as a mixture of two mechanisms: a fast, capacity-limited, and delay-sensitive working memory process together with slower reinforcement learning. Model-based analysis of fMRI data showed that striatum and lateral prefrontal cortex were sensitive to reward prediction error, as shown previously, but critically, these signals were reduced when the learning problem was within capacity of working memory. The degree of this neural interaction related to individual differences in the use of working memory to guide behavioral learning. These results indicate that the two systems do not process information independently, but rather interact during learning.
Reinforcement learning theory has been remarkably productive at improving our understanding of instrumental learning as well as dopaminergic and striatal network function across many mammalian species. However, this neural network is only one contributor to human learning, and other mechanisms such as prefrontal cortex working memory, also play a key role. Our results show in addition that these other players interact with the dopaminergic RL system, interfering with its key computation of reward predictions errors.
The authors declare no competing financial interests.
We thank Christopher R Gagne for his role in data collection. This research was supported by NIH grants NS065046 & MH099078 to D.B and MH080066-01 to MJF, a James S. McDonnell foundation award do D.B., and NSF1460604 to MJF and AGEC.