Abstract
Although it is widely known that brain regions such as the prefrontal cortex, the amygdala, and the ventral striatum play large roles in decision making, their precise contributions remain unclear. Here, we used functional magnetic resonance imaging and principles of reinforcement learning theory to investigate the relationship between current reinforcements and future decisions. In the experiment, subjects chose between high-risk (i.e., low probability of a large monetary reward) and low-risk (high probability of a small reward) decisions. For each subject, we estimated value functions that represented the degree to which reinforcements affected the value of decision options on the subsequent trial. Individual differences in value functions predicted not only trial-to-trial behavioral strategies, such as choosing high-risk decisions following high-risk rewards, but also the relationship between activity in prefrontal and subcortical regions during one trial and the decision made in the subsequent trial. These findings provide a novel link between behavior and neural activity by demonstrating that value functions are manifested both in adjustments in behavioral strategies and in the neural activity that accompanies those adjustments.
Article PDF
Similar content being viewed by others
References
Aguirre, G. K., Zarahn, E., & D’Esposito, M. (1998). The variability of human, BOLD hemodynamic responses. NeuroImage, 8, 360–369.
Arana, F. S., Parkinson, J. A., Hinton, E., Holland, A. J., Owen, A. M., & Roberts, A. C. (2003). Dissociable contributions of the human amygdala and orbitofrontal cortex to incentive motivation and goal selection. Journal of Neuroscience, 23, 9632–9638.
Baleydier, C., & Mauguiere, F. (1980). The duality of the cingulate gyrus in monkey: Neuroanatomical study and functional hypothesis. Brain, 103, 525–554.
Barraclough, D. J., Conroy, M. L., & Lee, D. (2004). Prefrontal cortex and decision making in a mixed-strategy game. Nature Neuroscience, 7, 404–410.
Barto, A. G. (1995). Reinforcement learning. In M. A. Arbib (Ed.), Handbook of brain theory and neural networks (pp. 804–809). Cambridge, MA: MIT Press.
Bechara, A., Damasio, H., & Damasio, A. R. (2000). Emotion, decision making and the orbitofrontal cortex. Cerebral Cortex, 10, 295–307.
Bechara, A., Damasio, H., & Damasio, A. R. (2003). Risky business: Emotion, decision-making, and addiction. Journal of Gambling Studies, 19, 23–51.
Bechara, A., Damasio, H., Damasio, A. R., & Lee, G. P. (1999). Different contributions of the human amygdala and ventromedial prefrontal cortex to decision-making. Journal of Neuroscience, 19, 5473–5481.
Bechara, A., Damasio, H., Tranel, D., & Damasio, A. R. (1997). Deciding advantageously before knowing the advantageous strategy. Science, 275, 1293–1295.
Berridge, K. C., & Robinson, T. E. (2003). Parsing reward. Trends in Neurosciences, 26, 507–513.
Blackwood, N., Ffytche, D., Simmons, A., Bentall, R., Murray, R., & Howard, R. (2004). The cerebellum and decision making under uncertainty. Cognitive Brain Research, 20, 46–53.
Botvinick, M. M., Braver, T. S., Barch, D. M., Carter, C. S., & Cohen, J. D. (2001). Conflict monitoring and cognitive control. Psychological Review, 108, 624–652.
Brett, M., Johnsrude, I. S., & Owen, A. M. (2002). The problem of functional localization in the human brain. Nature Reviews Neuroscience, 3, 243–249.
Bush, G., Vogt, B. A., Holmes, J., Dale, A. M., Greve, D., Jenike, M. A., & Rosen, B. R. (2002). Dorsal anterior cingulate cortex: A role in reward-based decision making. Proceedings of the National Academy of Sciences, 99, 523–528.
Cavada, C., Company, T., Tejedor, J., Cruz-Rizzolo, R. J., & Reinoso-Suarez, F. (2000). The anatomical connections of the macaque monkey orbitofrontal cortex: A review. Cerebral Cortex, 10, 220–242.
Cohen, M. X., Heller, A. S., & Ranganath, C. (2005). Functional connectivity with anterior cingulate and orbitofrontal cortices during decision-making. Cognitive Brain Research, 23, 61–70.
Cools, R., Clark, L., Owen, A. M., & Robbins, T. W. (2002). Defining the neural mechanisms of probabilistic reversal learning using event-related functional magnetic resonance imaging. Journal of Neuroscience, 22, 4563–4567.
Critchley, H. D., Corfield, D. R., Chandler, M. P., Mathias, C. J., & Dolan, R. J. (2000). Cerebral correlates of autonomic cardiovascular arousal: A functional neuroimaging investigation in humans. Journal of Physiology, 523, 259–270.
Depue, R. A., & Collins, P. F. (1999). Neurobiology of the structure of personality: Dopamine, facilitation of incentive motivation, and extraversion. Behavioral & Brain Sciences, 22, 491–569.
Desjardins, A. E., Kiehl, K. A., & Liddle, P. F. (2001). Removal of confounding effects of global signal in functional MRI analyses. NeuroImage, 13, 751–758.
Devinsky, O., Morrell, M. J., & Vogt, B. A. (1995). Contributions of anterior cingulate cortex to behaviour. Brain, 118, 279–306.
Egelman, D. M., Person, C., & Montague, P. R. (1998). A computational role for dopamine delivery in human decision-making. Journal of Cognitive Neuroscience, 10, 623–630.
Erev, I., & Roth, A. E. (1998). Predicting how people play games: Reinforcement learning in experimental games with unique, mixed strategy equalibria. American Economic Review, 88, 848–881.
Fell, J., Klaver, P., Elger, C. E., & Fernandez, G. (2002). The interaction of rhinal cortex and hippocampus in human declarative memory formation. Review of Neuroscience, 13, 299–312.
Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003). Discrete coding of reward probability and uncertainty by dopamine neurons. Science, 299, 1898–1902.
Friston, K. J., Tononi, G., Reeke, G. N., Sporns, O., & Edelman, G. M. (1994). Value-dependent selection in the brain: Simulation in a synthetic neural model. Neuroscience, 59, 229–243.
Gehring, W. J., & Willoughby, A. R. (2002). The medial frontal cortex and the rapid processing of monetary gains and losses. Science, 295, 2279–2282.
Goeree, J. K., & Holt, C. A. (1999). Stochastic game theory: For playing games, not just for doing theory. Proceedings of the National Academy of Sciences, 96, 10564–10567.
Hikosaka, K., & Watanabe, M. (2000). Delay activity of orbital and lateral prefrontal neurons of the monkey varying with different rewards. Cerebral Cortex, 10, 263–271.
Kahn, I., Yeshurun, Y., Rothstein, P., Fried, I., Ben-Bashat, D., & Hendler, T. (2002). The role of the amygdala in signaling prospective outcome of choice. Neuron, 33, 983–994.
Kerns, J. G., Cohen, J. D., MacDonald, A. W., III, Cho, R. Y., Stenger, V. A., & Carter, C. S. (2004). Anterior cingulate conflict monitoring and adjustments in control. Science, 303, 1023–1026.
Krawczyk, D. C. (2002). Contributions of the prefrontal cortex to the neural basis of human decision making. Neuroscience and Biobehavioral Reviews, 26, 631–664.
Lagarias, J. C., Reeds, J. A., Wright, M. H., & Wright, P. E. (1998). Convergence properties of the Nelder-Mead simplex method in low dimensions. SIAM Journal of Optimization, 9, 112–147.
Liu, Z., Murray, E. A., & Richmond, B. J. (2000). Learning motivational significance of visual cues for reward schedules requires rhinal cortex. Nature Neuroscience, 3, 1307–1315.
Luce, D. P. (1999). Individual choice behavior. New York: Wiley.
Manes, F., Sahakian, B., Clark, L., Rogers, R., Antoun, N., Aitken, M., & Robbins, T. (2002). Decision-making processes following damage to the prefrontal cortex. Brain, 125, 624–639.
McCullough, L. D., & Salamone, J. D. (1992). Involvement of nucleus accumbens dopamine in the motor activity induced by periodic food presentation: A microdialysis and behavioral study. Brain Research, 592, 29–36.
Miyashita, Y., Okuno, H., Tokuyama, W., Ihara, T., & Nakajima, K. (1996). Feedback signal from medial temporal lobe mediates visual associative mnemonic codes of inferotemporal neurons. Cognitive Brain Research, 5, 81–86.
Montague, P. R., & Berns, G. S. (2002). Neural economics and the biological substrates of valuation. Neuron, 36, 265–284.
Mookherjee, D., & Sopher, B. (1994). Learning behavior in an experimental matching pennies game. Games & Economic Behavior, 7, 62–91.
O’Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003). Temporal difference models and reward-related learning in the human brain. Neuron, 38, 329–337.
Paulus, M. P., Feinstein, J. S., Tapert, S. F., & Liu, T. T. (2004). Trend detection via temporal difference model predicts inferior prefrontal cortex activation during acquisition of advantageous action selection. NeuroImage, 21, 733–743.
Ramnani, N., & Miall, R. C. (2003). Instructed delay activity in the human prefrontal cortex is modulated by monetary reward expectation. Cerebral Cortex, 13, 318–327.
Ranganath, C., Cohen, M. X., & Brozinsky, C. J. (in press). Working memory maintenance contributes to long-term memory formation: Neural and behavioral evidence. Journal of Cognitive Neuroscience.
Ridderinkhof, K. R., Nieuwenhuis, S., & Bashore, T. R. (2003). Errors are foreshadowed in brain potentials associated with action monitoring in cingulate cortex in humans. Neuroscience Letters, 348, 1–4.
Roesch, M. R., & Olson, C. R. (2003). Impact of expected reward on neuronal activity in prefrontal cortex, frontal and supplementary eye fields and premotor cortex. Journal of Neurophysiology, 90, 1766–1789.
Rogers, R., Ramnani, N., Mackay, C., Wilson, J. L., Jezzard, P., Carter, C. S., & Smith, S. M. (2004). Distinct portions of anterior cingulate cortex and medial prefrontal cortex are activated by reward processing in separable phases of decision-making cognition. Biological Psychiatry, 55, 594–602.
Rolls, E. T. (2000). The orbitofrontal cortex and reward. Cerebral Cortex, 10, 284–294.
Rolls, E. T. (2004). The functions of the orbitofrontal cortex. Brain & Cognition, 55, 11–29.
Rorden, C., & Brett, M. (2000). Stereotaxic display of brain lesions. Behavioral Neurology, 12, 191–200.
Sanfey, A. G., Hastie, R., Colvin, M. K., & Grafman, J. (2003). Phineas gauged: Decision-making and the human prefrontal cortex. Neuropsychologia, 41, 1218–1229.
Sanfey, A. G., Rilling, J. K., Aronson, J. A., Nystrom, L. E., & Cohen, J. D. (2003). The neural basis of economic decision-making in the Ultimatum Game. Science, 300, 1755–1758.
Sarin, R., & Vahid, F. (2001). Predicting how people play games: A simple dynamic model of choice. Games & Economic Behavior, 34, 104–122.
Schultz, W. (2004). Neural coding of basic reward terms of animal learning theory, game theory, microeconomics and behavioural ecology. Current Opinion in Neurobiology, 14, 139–147.
Shidara, M., & Richmond, B. J. (2002). Anterior cingulate: Single neuronal signals related to degree of reward expectancy. Science, 296, 1709–1711.
Stern, C. E., & Passingham, R. E. (1994). The nucleus accumbens in monkeys (Macaca fascicularis): I. The organization of behaviour. Behavioural Brain Research, 61, 9–21.
Sutton, R. S. (Ed.) (1992). Special issue on reinforcement learning. Machine Learning, 8.
Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning. Cambridge, MA: MIT Press.
Tanaka, S. C., Doya, K., Okada, G., Ueda, K., Okamoto, Y., & Yamawaki, S. (2004). Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nature Neuroscience, 7, 887–893.
Tversky, A., & Kahneman, D. (1981). The framing of decisions and the psychology of choice. Science, 211, 453–458.
van Veen, V., & Carter, C. S. (2002). The anterior cingulate as a conflict monitor: fMRI and ERP studies. Physiology & Behavior, 77, 477–482.
Volkow, N. D., Fowler, J. S., Wang, G. J., & Swanson, J. M. (2004). Dopamine in drug abuse and addiction: Results from imaging studies and treatment implications. Molecular Psychiatry, 9, 557–569.
Whishaw, I. Q., & Kornelsen, R. A. (1993). Two types of motivation revealed by ibotenic acid nucleus accumbens lesions: Dissociation of food carrying and hoarding and the role of primary and incentive motivation. Behavioural Brain Research, 55, 283–295.
Worsley, K. J., & Friston, K. J. (1995). Analysis of fMRI time-series revisited—again. NeuroImage, 2, 173–181.
Zahm, D. S. (1999). Functional-anatomical implications of the nucleus accumbens core and shell subterritories. In J. F. McGinty (Ed.), Advanciing from the ventral striatum to the extended amygdala: Implications for neuropsychiatry and drug abuse (Annals of the New York Academy of Sciences, Vol. 877, pp. 113–128). New York: New York Academy of Sciences.
Zald, D. H., Boileau, I., El-Dearedy, W., Gunn, R., McGlone, F., Dichter, G. S., & Dagher, A. (2004). Dopamine transmission in the human striatum during monetary reward tasks. Journal of Neuroscience, 24, 4105–4112.
Zarahn, E., Aguirre, G. K., & D’Esposito, M. (1997). Empirical analyses of BOLD fMRI statistics: I. Spatially unsmoothed data collected under null-hypothesis conditions. NeuroImage, 5, 179–197.
Author information
Authors and Affiliations
Corresponding author
Additional information
This work was supported by an extramural research grant from the Institute for Research on Pathological Gambling and Related Disorders.
Rights and permissions
About this article
Cite this article
Cohen, M.X., Ranganath, C. Behavioral and neural predictors of upcoming decisions. Cognitive, Affective, & Behavioral Neuroscience 5, 117–126 (2005). https://doi.org/10.3758/CABN.5.2.117
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/CABN.5.2.117