Review
Behavioral dopamine signals

https://doi.org/10.1016/j.tins.2007.03.007Get rights and content

Lesioning and psychopharmacological studies suggest a wide range of behavioral functions for ascending midbrain dopaminergic systems. However, electrophysiological and neurochemical studies during specific behavioral tasks demonstrate a more restricted spectrum of dopamine-mediated changes. Substantial increases in dopamine-mediated activity, as measured by electrophysiology or voltammetry, are related to rewards and reward-predicting stimuli. A somewhat slower, distinct electrophysiological response encodes the uncertainty associated with rewards. Aversive events produce different, mostly slower, electrophysiological dopamine responses that consist predominantly of depressions. Additionally, more modest dopamine concentration fluctuations, related to punishment and movement, are seen at 200–18 000 times longer time courses using voltammetry and microdialysis in vivo. Using these responses, dopamine neurotransmission provides differential and heterogeneous information to subcortical and cortical brain structures about essential outcome components for approach behavior, learning and economic decision-making.

Introduction

The description of the function of dopamine has been dominated for many decades by two basic observations: the severe movement deficits after dopamine-depleting lesions in patients with Parkinson's disease, and the reduced behavioral responses to motivating stimuli after interference with dopamine neurotransmission in experimental rats. Because relating all brain activity to the functioning of single neurons (as the basic unitary elements for neural processing) is preferred, the question arises as to which of the many functions deficient after dopamine-depleting lesions could actually be attributed to the impulse activity of single dopaminergic neurons projecting to postsynaptic targets, such as the striatum, nucleus accumbens and frontal cortex. A review of the past 50 years of dopamine research might be the appropriate time for reviewing the current data concerning the relation of dopamine-impulse activity to specific behavioral functions. Here, we review not only the electrophysiological response of dopaminergic neurons to reward- and uncertainty-related events, but we also set them in a larger context by describing data from other techniques measuring dopamine-mediated changes, such as voltammetry and microdialysis. These physiological dopamine-mediated changes are compared with the effects of dopamine-depleting lesions to obtain a more coherent view of the function of dopamine.

Section snippets

Reward

Although it is essential for survival of both individuals and genes, reward information does not affect the brain through specific, dedicated, sensory receptors. By contrast, the function of rewards is defined by their action on behavior. Neural decision-making systems, dealing with the pursuit of essential objects for survival, would benefit from explicit neuronal signals for reward, just as visual perception is derived from specific information provided by retinal responses to visual stimuli.

Reward-prediction error

Rewards occur after reward-predicting stimuli and behavioral actions. They generate approach and consummatory behavior, constitute outcomes of the preceding stimuli and actions, serve as positively reinforcing feedbacks (‘come back for more’) and produce reward predictions through associative conditioning. The reward-prediction error reflects the difference between predicted and obtained rewards and constitutes the essential term for reward-driven learning, according to the Rescorla–Wagner

Reward-predicting stimuli

Reward-predicting stimuli occur before rewards, predict the outcomes of actions, provide essential advance information for decision-making in choice behavior, generate overt approach behavior and serve as positive, conditioned reinforcers for earlier stimuli and actions in higher-order associative learning.

Most midbrain dopaminergic neurons show phasic activation following conditioned visual, auditory and somatosensory reward-predicting stimuli [16] (Figure 1a, e–g). The activity of the neurons

Reward uncertainty

In the natural world, rewards usually occur with some degree of uncertainty. Uncertainty is conceptualized by probability theory, which defines the degree of uncertainty within a given distribution of probable outcomes. Conventional measures of uncertainty include the statistical term ‘variance’ and the information-theoretic term ‘entropy’. Uncertainty has a substantial influence on the subjective reward value, reducing it in risk-averse individuals and increasing it in risk seekers, as

Aversive events

Punishers have motivationally opposite effects to rewards, produce withdrawal behavior, constitute negative outcomes, serve as negative reinforcers in aversive conditioning by reducing the behavior leading to punishment and increasing the behavior leading to its avoidance, and produce aversive predictions for decision-making in choice behavior.

Dopaminergic neurons from monkeys, rabbits and rats respond, mostly with depressions, to electrical stimulation of peripheral nerves, air puffs, painful

Voltammetry

Rapid changes in dopamine concentrations, with time courses of a few seconds, are detected in the nucleus accumbens, striatum and frontal cortex by electrochemical methods using microelectrodes that measure the currents related to oxidation and reduction of dopamine. The changes occur in relation to various behaviorally relevant events, including exposure to novel environments, unpredicted primary liquid rewards, sexual stimuli, conditioned visual and olfactory stimuli predicting food or drug

Microdialysis in vivo

Insertion of tubes of submillimeter thickness, with semipermeable membranes, into specific brain structures can be used to collect molecules diffusing from the surrounding tissue into the perfusate. The dialyzate is subsequently analyzed for dopamine using highly sensitive biochemical and electrochemical methods.

A considerable number of behavioral events lead to 20–100% increases in the dopamine concentration in the nucleus accumbens, striatum and frontal cortex. These increases last up to tens

Processes deficient in parkinsonism

Dopamine depletion in patients with Parkinson's disease and experimental animals produces severe, well-documented deficits in movement, motivation and cognition. Externally administered dopamine precursors and receptor-stimulating agents encourage restitution of many motor, motivational and cognitive functions, although some deficits in discrimination, learning and appetitive behavior remain 62, 63, 64, 65, 66. Although treatment using dopamine precursors might restore dopamine concentrations

Concluding remarks

The reviewed data demonstrate that a neurotransmitter system can have two distinct properties that, amazingly, are operational at the same time. It can be involved in the transmission of time-specific information about a restricted spectrum of external events. Concurrently, it can have a crucial, apparently sustained, influence on postsynaptic neural processes without temporal modulation. Although other neurotransmitter systems are also known to contribute to the chemical mix of the

Acknowledgements

The author's work is supported by the Wellcome Trust, Swiss National Science Foundation, Human Frontiers Science Program and several other grant and fellowship agencies.

References (69)

  • R.W. Keller

    Environmental stimuli but not homeostatic challenges produce apparent increases in dopaminergic activity in the striatum: An analysis by in vivo voltammetry

    Brain Res.

    (1983)
  • A. Louilot

    Differential reactivity of dopaminergic neurons in the nucleus accumbens in response to different behavioral situations. An in vivo voltammetric study in free moving rats

    Brain Res.

    (1986)
  • M.D. Doherty et al.

    High-speed chronoamperometric measurements of mesolimbic and nigrostriatal dopamine release associated with repeated daily stress

    Brain Res.

    (1992)
  • L. Hernandez et al.

    Food reward and cocaine increase extracellular dopamine in the nucleus accumbens as measured by microdialysis

    Life Sci.

    (1988)
  • A. Imperato

    Repeated stressful experiences differently affect limbic dopamine release during and following stress

    Brain Res.

    (1992)
  • J.D. Salamone

    The involvement of nucleus accumbens dopamine in appetitive and aversive motivation

    Behav. Brain Res.

    (1994)
  • P.W. Kalivas et al.

    Selective activation of dopamine transmission in the shell of the nucleus accumbens by stress

    Brain Res.

    (1995)
  • V. Bassareo et al.

    Differential responsiveness of dopamine transmission to food-stimuli in nucleus accumbens shell/core compartments

    Neuroscience

    (1999)
  • M.G. Feenstra

    Dopamine and noradrenaline efflux in the prefrontal cortex in the light and dark period: effects of novelty and handling and comparison to the nucleus accumbens

    Neuroscience

    (2000)
  • M.H. Joseph

    The interpretation of the measurement of nucleus accumbens dopamine by in vivo dialysis: the kick, the craving or the cognition?

    Neurosci. Biobehav. Rev.

    (2003)
  • A.M.J. Young

    Increased extracellular dopamine in nucleus accumbens in response to unconditioned and conditioned aversive stimuli: studies using 1 min microdialysis in rats

    J. Neurosci. Methods

    (2004)
  • A.M.J. Young

    The role of dopamine in conditioning and latent inhibition: what, when, where and how?

    Neurosci. Biobehav. Rev.

    (2005)
  • M.F. Chesselet

    Presynaptic regulation of neurotransmitter release in the brain: facts and hypothesis

    Neuroscience

    (1984)
  • N. Saulskaya et al.

    Conditioned dopamine release: dependence upon N-methyl-d-aspartate receptors

    Neuroscience

    (1995)
  • A.G.M. Canavan

    The performance on learning tasks of patients in the early stages of Parkinson's disease

    Neuropsychologia

    (1989)
  • E.R. Vriezen et al.

    Memory for temporal order and conditional associative-learning in patients with Parkinson's disease

    Neuropsychologia

    (1990)
  • A.A. Grace

    Phasic versus tonic dopamine release and the modulation of dopamine system responsivity: a hypothesis for the etiology of schizophrenia

    Neuroscience

    (1991)
  • W. Schultz

    A neural substrate of prediction and reward

    Science

    (1997)
  • L.J. Kamin

    Selective association and conditioning

  • P. Waelti

    Dopamine responses comply with basic assumptions of formal learning theory

    Nature

    (2001)
  • P.N. Tobler

    Coding of predicted reward omission by dopamine neurons in a conditioned inhibition paradigm

    J. Neurosci.

    (2003)
  • P.R. Montague

    Bee foraging in uncertain environments using predictive hebbian learning

    Nature

    (1995)
  • P.R. Montague

    A framework for mesencephalic dopamine systems based on predictive Hebbian learning

    J. Neurosci.

    (1996)
  • Cited by (807)

    View all citing articles on Scopus
    View full text