Abstract
Hippocampal maps and ventral prefrontal cortex (vPFC) value and goal representations support foraging in continuous spaces. How might hippocampal-vPFC interactions control the balance between behavioral exploration and exploitation? Using fMRI and reinforcement learning modeling, we investigated vPFC and hippocampal responses as humans (38 female, 34 male), explored and exploited a continuous one-dimensional space, with out-of-session and out-of-sample replication (23 female, 20 male). The spatial distribution of rewards, or value landscape, modulated activity in the hippocampus and default network vPFC subregions, but not in ventrolateral prefrontal control subregions or medial orbitofrontal limbic subregions. While prefrontal default network and hippocampus displayed higher activity in less complex, easy-to-exploit value landscapes, vPFC-hippocampal connectivity increased in uncertain landscapes requiring exploration. Further, synchronization between prefrontal default network and posterior hippocampus scaled with behavioral exploration. Considered alongside electrophysiological studies, our findings suggest that exploration targets are identified through coordinated activity binding prefrontal default network value representations to posterior hippocampal maps.
Significance Statement The ventral prefrontal cortex represents goals and values, while the hippocampus contains maps of physical and abstract spaces. In recent years, neuroscientists have sought to understand how hippocampal-prefrontal interactions help us to solve complex problems such as deciding whether to exploit known rewards or explore in search of better alternatives. Using functional imaging and reinforcement learning modeling, we examine these interactions as humans explore and exploit an environment with a complex distribution of rewards or value landscape. We describe how spatial information of the hippocampus enriches value representations in the prefrontal default mode subnetwork, controlling the balance between exploration and exploitation.
Footnotes
The authors declare no competing financial interests.
Author Contributions: Conceptualization: AYD and MNH. Investigation: MNH and BL. Resources: MNH, BL and AYD. Formal analysis: AYD, MNH, AMI, BEL, and AEP. Writing – original draft: AEP, VMB, AYD. Writing – editing: All authors. Project administration: AYD.
This work was funded by K01 MH097091, R01 MH10095, K23 MH122626 and R01 MH067924 from the National Institute of Mental Health. This research was supported in part by the University of Pittsburgh Center for Research Computing through the resources provided. Specifically, this work used the H2P cluster, which is supported by NSF award number OAC-2117681 and the HTC cluster, which is supported by NIH award number S10OD028483. This research was also performed, in part, using resources and the computing assistance of the Pennsylvania State University Institute for CyberScience Advanced CyberInfrastructure (ICS-ACI). We would like to thank Jiazhou Chen, Morgan Buerke, Mandy Collier, Michelle Perry, Laura Taglioni, Shreya Sheth, Tanya Shah, Kaylee Stewart, and Bea Langer for collecting the fMRI data in Experiment 2. We would like to thank Kai Hwang and Rajpreet Chahal for collecting the fMRI data in Experiment 1. We would like to thank Katalin Szanto for administrating the longitudinal study of which Experiment 2 was the scanning arm. Thanks to Nathaniel J. Powell for comments on this manuscript.
