Elsevier

Neural Networks

Volume 19, Issue 8, October 2006, Pages 1120-1136
Neural Networks

2006 Special Issue
Hold your horses: A dynamic computational role for the subthalamic nucleus in decision making

https://doi.org/10.1016/j.neunet.2006.03.006Get rights and content

Abstract

The basal ganglia (BG) coordinate decision making processes by facilitating adaptive frontal motor commands while suppressing others. In previous work, neural network simulations accounted for response selection deficits associated with BG dopamine depletion in Parkinson’s disease. Novel predictions from this model have been subsequently confirmed in Parkinson patients and in healthy participants under pharmacological challenge. Nevertheless, one clear limitation of that model is in its omission of the subthalamic nucleus (STN), a key BG structure that participates in both motor and cognitive processes. The present model incorporates the STN and shows that by modulating when a response is executed, the STN reduces premature responding and therefore has substantial effects on which response is ultimately selected, particularly when there are multiple competing responses. Increased cortical response conflict leads to dynamic adjustments in response thresholds via cortico-subthalamic-pallidal pathways. The model accurately captures the dynamics of activity in various BG areas during response selection. Simulated dopamine depletion results in emergent oscillatory activity in BG structures, which has been linked with Parkinson’s tremor. Finally, the model accounts for the beneficial effects of STN lesions on these oscillations, but suggests that this benefit may come at the expense of impaired decision making.

Introduction

Deciphering the mechanisms by which the brain supports response selection, a central process in decision making, is an important challenge for both the artificial intelligence and cognitive neuroscience communities. Based on a wealth of data, the basal ganglia (BG) are thought to play a principal role in these processes. In the context of motor control, various authors have suggested that the role of the BG is to selectively facilitate the execution of a single adaptive motor command, while suppressing all others (Basso and Wurtz, 2002, Brown et al., 2004, Frank, 2005a, Gurney et al., 2001, Hikosaka, 1994, Jiang et al., 2003, Mink, 1996, Redgrave et al., 1999). Interestingly, circuits linking the BG with more cognitive areas of frontal cortex (e.g., prefrontal) are strikingly similar to those observed in the motor domain (Alexander, DeLong, & Strick, 1986), raising the possibility that the BG participate in cognitive decision making in an analogous fashion to their role in motor control (Beiser and Houk, 1998, Frank, 2005a, Frank and Claus, 2006, Frank et al., 2001, Middleton and Strick, 2000, Middleton and Strick, 2002). Studies with Parkinson’s patients, who have severely depleted levels of dopamine (DA) in the BG (Kish, Shannak, & Hornykiewicz, 1988), have provided insights into the functional roles of the BG/DA system in both motor and higher level cognitive processes (Cools, 2005, Frank, 2005a, Shohamy et al., 2005). Of particular recent interest is the finding that deep brain stimulation in the subthalamic nucleus (STN) dramatically improves Parkinson motor symptoms, with both reported enhancements and impairments in cognition (Karachi et al., 2004, Witt et al., 2004). Because the BG consists of a complex network of dynamically interacting brain areas, a mechanistic understanding of exactly how the STN participates in response selection and decision making is difficult to develop with traditional box and arrow models. Computational models that explore the dynamics of BG network activity are therefore useful tools for providing insight into these issues, and in turn, how they affect individuals with Parkinson’s disease and related disorders.

In this paper, I review converging evidence for a mechanistic, functional account of how interacting areas within the BG-frontal system learn to select adaptive responses and participate in cognitive decision making, as informed by prior computational simulations. I then present a neural network model that explores the unique contribution of the STN within the overall BG circuitry. The simulations reveal that the STN can dynamically control the threshold for executing a response, and that this function is adaptively modulated by the degree to which multiple competing responses are activated, as in difficult decisions. It is concluded that the STN may be essential to allow all information to be integrated before making decisions, and thereby prevents impulsive or premature responding during high-conflict decision trials. Furthermore, analysis of the dynamics of activity within various BG areas during response selection in intact and simulated Parkinson states demonstrates a striking relationship to the same patterns observed physiologically, providing support for the model’s biological plausibility and further insight into the neural processes underlying response selection.

Section snippets

Overall BG network functionality

The “standard model” proposes that two BG pathways independently act to selectively facilitate the execution of the most appropriate cortical motor command, while suppressing competing commands (Albin et al., 1989, Mink, 1996). Two main projection pathways from the striatum go through different BG nuclei on the way to thalamus and up to cortex (Fig. 1(a)). Activity in the direct pathway sends a “Go” signal to facilitate the execution of a response considered in cortex, whereas activity in the

Integrating contributions of the subthalamic nucleus in the model

Despite its success in capturing dopamine-driven individual differences in learning and attentional processes, the above model falls short in its ability to provide insight into BG dynamics that depend on the subthalamic nucleus (STN). The model was designed to simulate how the BG can learn to selectively facilitate (Go) one response while selectively suppressing (NoGo) another. Because the projections from the STN to BG nuclei (GPe and GPi) are diffuse (Mink, 1996, Parent and Hazrati, 1995),

Discussion

This work presents a novel computational exploration of the subthalamic nucleus within the overall basal ganglia circuitry. The model integrates various neural and behavioral findings and provides insight into the STN role in response selection and decision making. Consistent with other BG models (Brown et al., 2004, Gurney et al., 2001), the STN provides a “Global NoGo” signal that suppresses all responses. But the current simulations revealed that this signal is dynamic, such that it is

Conclusion

How do the present simulations provide insight into the problem of when the subthalamic nucleus is beneficial for cognition, compared with situations in which too much STN activity may impair cognitive function? A preliminary answer to this question may be that the STN is useful in situations that would otherwise lead to “jumping the gun” on decision making processes, by preventing premature choices. However, when excessive hesitancy is experienced, the present model would suggest turning off

Acknowledgements

I thank Randy O’Reilly, Adam Aron, Patrick Simen, Todd Braver and Jonathan Cohen for helpful discussion.

References (144)

  • V. Czernecki et al.

    Motivation, reward, and Parkinson’s disease: Influence of dopatherapy

    Neuropsychologia

    (2002)
  • N.D. Daw et al.

    Opponent interactions between serotonin and dopamine

    Neural Networks

    (2002)
  • M.R. Delgado et al.

    An fMRI study of reward-related probability learning

    Neuroimage

    (2005)
  • M.R. DeLong

    Primate models of movement disorders of basal ganglia origin

    Trends in Neurosciences

    (1990)
  • L. Desbonnet et al.

    Premature responding following bilateral stimulation of the rat subthalamic nucleus is amplitude and frequency dependent

    Brain Research

    (2004)
  • M.J. Frank et al.

    Error-related negativity predicts reinforcement learning and conflict biases

    Neuron

    (2005)
  • C.R. Gerfen

    Molecular effects of dopamine on striatal projection pathways

    Trends in Neurosciences

    (2000)
  • C.R. Gerfen et al.

    The basal ganglia

  • J.I. Gold et al.

    Banburismus and the brain: Decoding the relationship between sensory stimuli, decisions, and reward

    Neuron

    (2002)
  • T.E. Hazy et al.

    Banishing the homunculus: Making working memory work

    Neuroscience

    (2006)
  • G.M. Jackson et al.

    Serial reaction time learning and Parkinson’s disease: Evidence for a procedural learning deficit

    Neuropsychologia

    (1995)
  • P.J. Magill et al.

    Dopamine regulates the impact of the cerebral cortex on the subthalamic nucleus–globus pallidus network

    Neuroscience

    (2001)
  • J.H. McAuley

    The physiological basis of clinical deficits in Parkinson’s disease

    Progress in Neurobiology

    (2003)
  • F.A. Middleton et al.

    Basal ganglia output and cognition: Evidence from anatomical, behavioral, and clinical studies

    Brain and Cognition

    (2000)
  • J.W. Mink

    The basal ganglia: Focused selection and inhibition of competing motor programs

    Progress in Neurobiology

    (1996)
  • C. Nocjar et al.

    Localization of 5-HT(2A) receptors on dopamine cells in subnuclei of the midbrain A10 cell group

    Neuroscience

    (2002)
  • G.E. Alexander et al.

    Preparation for movement: Neural representations of intended direction in three motor areas of the monkey

    Journal of Neurophysiology

    (1990)
  • G.E. Alexander et al.

    Parallel organization of functionally segregated circuits linking basal ganglia and cortex

    Annual Review of Neuroscience

    (1986)
  • R. Amirnovin et al.

    Visually guided movements suppress subthalamic oscillations in Parkinson’s disease patients

    Journal of Neuroscience

    (2004)
  • A.R. Aron et al.

    Cortical and subcortical contributions to stop signal response inhibition: Role of the subthalamic nucleus

    Journal of Neuroscience

    (2006)
  • F.G. Ashby et al.

    Category learning deficits in Parkinson’s disease

    Neuropsychology

    (2003)
  • G. Aston-Jones et al.

    An integrative theory of locus coeruleus–norepinephrine function: Adaptive gain and optimal performance

    Annual Review of Neuroscience

    (2005)
  • I. Aubert et al.

    Phenotypical characterization of the neurons expressing the D1 and D2 dopamine receptors in the monkey striatum

    Journal of Comparative Neurology

    (2000)
  • M.A. Basso et al.

    Neuronal activity in substantia nigra pars reticulata during target selection

    Journal of Neuroscience

    (2002)
  • C. Baunez et al.

    Effects of STN lesions on simple vs choice reaction time tasks in the rat: Preserved motor readiness, but impaired response selection

    European Journal of Neuroscience

    (2001)
  • C. Baunez et al.

    Bilateral lesions of the subthalamic nucleus induce multiple deficits in an attentional task in rats

    European Journal of Neuroscience

    (1997)
  • D.G. Beiser et al.

    Model of cortical-basal ganglionic processing: Encoding the serial order of sensory events

    Journal of Neurophysiology

    (1998)
  • A. Benazzouz et al.

    Mechanism of action of deep brain stimulation

    Neurology

    (2000)
  • H. Bergman et al.

    Reversal of experimental Parkinsonism by lesions of the subthalamic nucleus

    Science

    (1990)
  • H. Bergman et al.

    The primate subthalamic nucleus. II. Neuronal activity in the MPTP model of Parkinsonism

    Journal of Neurophysiology

    (1994)
  • Bogacz, R., Brown, E., Moehlis, J., Holmes, P., & Cohen, J. D. The physics of optimal decision making: A formal...
  • M.M. Botvinick et al.

    Conflict monitoring and cognitive control

    Psychological Review

    (2001)
  • E. Brown et al.

    Simple neural networks that optimize decisions

    International Journal of Bifurcation and Chaos

    (2005)
  • J. Brown et al.

    How the basal ganglia use parallel excitatory and inhibitory learning pathways to selectively respond to unexpected rewarding cues

    Journal of Neuroscience

    (1999)
  • R.G. Brown et al.

    Internal versus external cues and the control of attention in Parkinson’s disease

    Brain

    (1988)
  • S.R. Chamberlain et al.

    Neurochemical modulation of response inhibition and probabilistic learning in humans

    Science

    (2006)
  • D. Charbonneau et al.

    Impaired incentive learning in treated Parkinson’s disease

    Canadian Journal of Neurological Sciences

    (1996)
  • W.Y. Choi et al.

    Extended habit training reduces dopamine mediation of appetitive response expression

    Journal of Neuroscience

    (2005)
  • H.F. Clarke et al.

    Cognitive inflexibility after prefrontal serotonin depletion

    Science

    (2004)
  • R. Cools

    Dopaminergic modulation of cognitive function — implications for L-DOPA treatment in Parkinson’s disease

    Neuroscience and Biobehavioral Reviews

    (2005)
  • Cited by (539)

    • Reinforcement learning and meta-decision-making

      2024, Current Opinion in Behavioral Sciences
    View all citing articles on Scopus
    1

    Portions of this paper were previously presented in conference format at the International Workshop on Models of Natural Action Selection (Frank, 2005b).

    View full text