2006 Special IssueHold your horses: A dynamic computational role for the subthalamic nucleus in decision making
Introduction
Deciphering the mechanisms by which the brain supports response selection, a central process in decision making, is an important challenge for both the artificial intelligence and cognitive neuroscience communities. Based on a wealth of data, the basal ganglia (BG) are thought to play a principal role in these processes. In the context of motor control, various authors have suggested that the role of the BG is to selectively facilitate the execution of a single adaptive motor command, while suppressing all others (Basso and Wurtz, 2002, Brown et al., 2004, Frank, 2005a, Gurney et al., 2001, Hikosaka, 1994, Jiang et al., 2003, Mink, 1996, Redgrave et al., 1999). Interestingly, circuits linking the BG with more cognitive areas of frontal cortex (e.g., prefrontal) are strikingly similar to those observed in the motor domain (Alexander, DeLong, & Strick, 1986), raising the possibility that the BG participate in cognitive decision making in an analogous fashion to their role in motor control (Beiser and Houk, 1998, Frank, 2005a, Frank and Claus, 2006, Frank et al., 2001, Middleton and Strick, 2000, Middleton and Strick, 2002). Studies with Parkinson’s patients, who have severely depleted levels of dopamine (DA) in the BG (Kish, Shannak, & Hornykiewicz, 1988), have provided insights into the functional roles of the BG/DA system in both motor and higher level cognitive processes (Cools, 2005, Frank, 2005a, Shohamy et al., 2005). Of particular recent interest is the finding that deep brain stimulation in the subthalamic nucleus (STN) dramatically improves Parkinson motor symptoms, with both reported enhancements and impairments in cognition (Karachi et al., 2004, Witt et al., 2004). Because the BG consists of a complex network of dynamically interacting brain areas, a mechanistic understanding of exactly how the STN participates in response selection and decision making is difficult to develop with traditional box and arrow models. Computational models that explore the dynamics of BG network activity are therefore useful tools for providing insight into these issues, and in turn, how they affect individuals with Parkinson’s disease and related disorders.
In this paper, I review converging evidence for a mechanistic, functional account of how interacting areas within the BG-frontal system learn to select adaptive responses and participate in cognitive decision making, as informed by prior computational simulations. I then present a neural network model that explores the unique contribution of the STN within the overall BG circuitry. The simulations reveal that the STN can dynamically control the threshold for executing a response, and that this function is adaptively modulated by the degree to which multiple competing responses are activated, as in difficult decisions. It is concluded that the STN may be essential to allow all information to be integrated before making decisions, and thereby prevents impulsive or premature responding during high-conflict decision trials. Furthermore, analysis of the dynamics of activity within various BG areas during response selection in intact and simulated Parkinson states demonstrates a striking relationship to the same patterns observed physiologically, providing support for the model’s biological plausibility and further insight into the neural processes underlying response selection.
Section snippets
Overall BG network functionality
The “standard model” proposes that two BG pathways independently act to selectively facilitate the execution of the most appropriate cortical motor command, while suppressing competing commands (Albin et al., 1989, Mink, 1996). Two main projection pathways from the striatum go through different BG nuclei on the way to thalamus and up to cortex (Fig. 1(a)). Activity in the direct pathway sends a “Go” signal to facilitate the execution of a response considered in cortex, whereas activity in the
Integrating contributions of the subthalamic nucleus in the model
Despite its success in capturing dopamine-driven individual differences in learning and attentional processes, the above model falls short in its ability to provide insight into BG dynamics that depend on the subthalamic nucleus (STN). The model was designed to simulate how the BG can learn to selectively facilitate (Go) one response while selectively suppressing (NoGo) another. Because the projections from the STN to BG nuclei (GPe and GPi) are diffuse (Mink, 1996, Parent and Hazrati, 1995),
Discussion
This work presents a novel computational exploration of the subthalamic nucleus within the overall basal ganglia circuitry. The model integrates various neural and behavioral findings and provides insight into the STN role in response selection and decision making. Consistent with other BG models (Brown et al., 2004, Gurney et al., 2001), the STN provides a “Global NoGo” signal that suppresses all responses. But the current simulations revealed that this signal is dynamic, such that it is
Conclusion
How do the present simulations provide insight into the problem of when the subthalamic nucleus is beneficial for cognition, compared with situations in which too much STN activity may impair cognitive function? A preliminary answer to this question may be that the STN is useful in situations that would otherwise lead to “jumping the gun” on decision making processes, by preventing premature choices. However, when excessive hesitancy is experienced, the present model would suggest turning off
Acknowledgements
I thank Randy O’Reilly, Adam Aron, Patrick Simen, Todd Braver and Jonathan Cohen for helpful discussion.
References (144)
- et al.
The functional anatomy of basal ganglia disorders
Trends in Neurosciences
(1989) - et al.
Functional architecture of basal ganglia circuits: Neural substrates of parallel processing
Trends in Neuroscience
(1990) - et al.
Information processing, dimensionality reduction and reinforcement learning in the basal ganglia
Progress in Neurobiology
(2003) - et al.
Midbrain dopamine neurons encode a quantitative reward prediction error signal
Neuron
(2005) - et al.
Physiological aspects of information processing in the basal ganglia of normal and Parkinsonian primates
Trends in Neurosciences
(1998) - et al.
Move to the rhythm: Oscillations in the subthalamic nucleus–external globus pallidus network
Trends in Neurosciences
(2002) - et al.
Event-related potentials for response inhibition in Parkinson’s disease
Neuropsychologia
(2005) - et al.
From single extracellular unit recording in experimental and human Parkinsonism to the development of a functional concept of the role played by the basal ganglia in motor control
Progress in Neurobiology
(2002) - et al.
How laminar frontal cortex and basal ganglia circuits interact to control planned and reactive saccades
Neural Networks
(2004) - et al.
L-Dopa medication remediates cognitive inflexibility, but increases impulsivity in patients with Parkinson’s disease
Neuropsychologia
(2003)
Motivation, reward, and Parkinson’s disease: Influence of dopatherapy
Neuropsychologia
Opponent interactions between serotonin and dopamine
Neural Networks
An fMRI study of reward-related probability learning
Neuroimage
Primate models of movement disorders of basal ganglia origin
Trends in Neurosciences
Premature responding following bilateral stimulation of the rat subthalamic nucleus is amplitude and frequency dependent
Brain Research
Error-related negativity predicts reinforcement learning and conflict biases
Neuron
Molecular effects of dopamine on striatal projection pathways
Trends in Neurosciences
The basal ganglia
Banburismus and the brain: Decoding the relationship between sensory stimuli, decisions, and reward
Neuron
Banishing the homunculus: Making working memory work
Neuroscience
Serial reaction time learning and Parkinson’s disease: Evidence for a procedural learning deficit
Neuropsychologia
Dopamine regulates the impact of the cerebral cortex on the subthalamic nucleus–globus pallidus network
Neuroscience
The physiological basis of clinical deficits in Parkinson’s disease
Progress in Neurobiology
Basal ganglia output and cognition: Evidence from anatomical, behavioral, and clinical studies
Brain and Cognition
The basal ganglia: Focused selection and inhibition of competing motor programs
Progress in Neurobiology
Localization of 5-HT(2A) receptors on dopamine cells in subnuclei of the midbrain A10 cell group
Neuroscience
Preparation for movement: Neural representations of intended direction in three motor areas of the monkey
Journal of Neurophysiology
Parallel organization of functionally segregated circuits linking basal ganglia and cortex
Annual Review of Neuroscience
Visually guided movements suppress subthalamic oscillations in Parkinson’s disease patients
Journal of Neuroscience
Cortical and subcortical contributions to stop signal response inhibition: Role of the subthalamic nucleus
Journal of Neuroscience
Category learning deficits in Parkinson’s disease
Neuropsychology
An integrative theory of locus coeruleus–norepinephrine function: Adaptive gain and optimal performance
Annual Review of Neuroscience
Phenotypical characterization of the neurons expressing the D1 and D2 dopamine receptors in the monkey striatum
Journal of Comparative Neurology
Neuronal activity in substantia nigra pars reticulata during target selection
Journal of Neuroscience
Effects of STN lesions on simple vs choice reaction time tasks in the rat: Preserved motor readiness, but impaired response selection
European Journal of Neuroscience
Bilateral lesions of the subthalamic nucleus induce multiple deficits in an attentional task in rats
European Journal of Neuroscience
Model of cortical-basal ganglionic processing: Encoding the serial order of sensory events
Journal of Neurophysiology
Mechanism of action of deep brain stimulation
Neurology
Reversal of experimental Parkinsonism by lesions of the subthalamic nucleus
Science
The primate subthalamic nucleus. II. Neuronal activity in the MPTP model of Parkinsonism
Journal of Neurophysiology
Conflict monitoring and cognitive control
Psychological Review
Simple neural networks that optimize decisions
International Journal of Bifurcation and Chaos
How the basal ganglia use parallel excitatory and inhibitory learning pathways to selectively respond to unexpected rewarding cues
Journal of Neuroscience
Internal versus external cues and the control of attention in Parkinson’s disease
Brain
Neurochemical modulation of response inhibition and probabilistic learning in humans
Science
Impaired incentive learning in treated Parkinson’s disease
Canadian Journal of Neurological Sciences
Extended habit training reduces dopamine mediation of appetitive response expression
Journal of Neuroscience
Cognitive inflexibility after prefrontal serotonin depletion
Science
Dopaminergic modulation of cognitive function — implications for L-DOPA treatment in Parkinson’s disease
Neuroscience and Biobehavioral Reviews
Cited by (539)
Reinforcement learning and meta-decision-making
2024, Current Opinion in Behavioral SciencesReduced sleep quality defines a subtype of obsessive-compulsive disorder with lower Glx levels in the resting thalamus and worse response inhibition
2024, Journal of Psychiatric ResearchThe role of reinforcement learning in shaping the decision policy in methamphetamine use disorders
2024, Journal of Choice Modelling
- 1
Portions of this paper were previously presented in conference format at the International Workshop on Models of Natural Action Selection (Frank, 2005b).