Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens

Abstract

The ability to predict favorable outcomes using environmental cues is an essential part of learned behavior. Dopamine neurons in the midbrain encode such stimulus-reward relationships in a manner consistent with contemporary learning models, but it is unclear how encoding this translates into actual dopamine release in target regions. Here, we sampled dopamine levels in the rat nucleus accumbens on a rapid (100 ms) timescale using electrochemical technology during a classical conditioning procedure. Early in conditioning, transient dopamine-release events signaled a primary reward, but not predictive cues. After repeated cue-reward pairings, dopamine signals shifted in time to predictive cue onset and were no longer observed at reward delivery. In the absence of stimulus-reward conditioning, there was no shift in the dopamine signal. Consistent with proposed roles in reward prediction and incentive salience, these results indicate that rapid dopamine release provides a reward signal that is dynamically modified by associative learning.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Early in associative learning, rapid elevations in NAc [DA] were time-locked to receipt of reward, but not to conditioned stimuli.
Figure 2: Rapid increase in NAc dopamine relative to reward retrieval during initial conditioning block.
Figure 3: Dopamine signaling in response to conditioned stimuli during the initial conditioning block.
Figure 4: After extended conditioning, rapid dopamine release events in the NAc shift to conditioned stimuli and no longer signal primary rewards.
Figure 5: For another group of animals, phasic dopamine signals remained time-locked to reward delivery in the absence of a predictor.
Figure 6: Comparison of dopamine changes relative to cue and reward stimuli using S:B transformation.
Figure 7: Anatomical distribution of carbon-fiber electrode placements in the NAc core.

Similar content being viewed by others

References

  1. Schultz, W., Dayan, P. & Montague, P.R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).

    Article  CAS  Google Scholar 

  2. Bayer, H.M. & Glimcher, P.W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129–141 (2005).

    Article  CAS  Google Scholar 

  3. Pan, W.X., Schmidt, R., Wickens, J.R. & Hyland, B.I. Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J. Neurosci. 25, 6235–6242 (2005).

    Article  CAS  Google Scholar 

  4. Sutton, R.S. & Barto, A.G. Reinforcement Learning (MIT Press, Cambridge, Massachusetts, 1998).

    Google Scholar 

  5. Montague, P.R., Hyman, S.E. & Cohen, J.D. Computational roles for dopamine in behavioural control. Nature 431, 760–767 (2004).

    Article  CAS  Google Scholar 

  6. Mirenowicz, J. & Schultz, W. Importance of unpredictability for reward responses in primate dopamine neurons. J. Neurophysiol. 72, 1024–1027 (1994).

    Article  CAS  Google Scholar 

  7. Fiorillo, C.D., Tobler, P.N. & Schultz, W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299, 1898–1902 (2003).

    Article  CAS  Google Scholar 

  8. Morris, G., Nevet, A., Arkadir, D., Vaadia, E. & Bergman, H. Midbrain dopamine neurons encode decisions for future action. Nat. Neurosci. 9, 1057–1063 (2006).

    Article  CAS  Google Scholar 

  9. Mogenson, G.J., Jones, D.L. & Yim, C.Y. From motivation to action: functional interface between the limbic system and the motor system. Prog. Neurobiol. 14, 69–97 (1980).

    Article  CAS  Google Scholar 

  10. Day, J.J., Wheeler, R.A., Roitman, M.F. & Carelli, R.M. Nucleus accumbens neurons encode Pavlovian approach behaviors: evidence from an autoshaping paradigm. Eur. J. Neurosci. 23, 1341–1351 (2006).

    Article  Google Scholar 

  11. Roitman, M.F., Wheeler, R.A. & Carelli, R.M. Nucleus accumbens neurons are innately tuned for rewarding and aversive taste stimuli, encode their predictors, and are linked to motor output. Neuron 45, 587–597 (2005).

    Article  CAS  Google Scholar 

  12. Yun, I.A., Wakabayashi, K.T., Fields, H.L. & Nicola, S.M. The ventral tegmental area is required for the behavioral and nucleus accumbens neuronal firing responses to incentive cues. J. Neurosci. 24, 2923–2933 (2004).

    Article  CAS  Google Scholar 

  13. Di Ciano, P., Cardinal, R.N., Cowell, R.A., Little, S.J. & Everitt, B.J. Differential involvement of NMDA, AMPA/kainate, and dopamine receptors in the nucleus accumbens core in the acquisition and performance of pavlovian approach behavior. J. Neurosci. 21, 9471–9477 (2001).

    Article  CAS  Google Scholar 

  14. Di Chiara, G. Nucleus accumbens shell and core dopamine: differential role in behavior and addiction. Behav. Brain Res. 137, 75–114 (2002).

    Article  CAS  Google Scholar 

  15. Dalley, J.W. et al. Time-limited modulation of appetitive Pavlovian memory by D1 and NMDA receptors in the nucleus accumbens. Proc. Natl. Acad. Sci. USA 102, 6189–6194 (2005).

    Article  CAS  Google Scholar 

  16. Zhang, H. & Sulzer, D. Frequency-dependent modulation of dopamine release by nicotine. Nat. Neurosci. 7, 581–582 (2004).

    Article  CAS  Google Scholar 

  17. Cragg, S.J. Variable dopamine release probability and short-term plasticity between functional domains of the primate striatum. J. Neurosci. 23, 4378–4385 (2003).

    Article  CAS  Google Scholar 

  18. Cragg, S.J. Meaningful silences: how dopamine listens to the ACh pause. Trends Neurosci. 29, 125–131 (2006).

    Article  CAS  Google Scholar 

  19. Montague, P.R. et al. Dynamic gain control of dopamine delivery in freely moving animals. J. Neurosci. 24, 1754–1759 (2004).

    Article  CAS  Google Scholar 

  20. Garris, P.A. et al. Dissociation of dopamine release in the nucleus accumbens from intracranial self-stimulation. Nature 398, 67–69 (1999).

    Article  CAS  Google Scholar 

  21. Bassareo, V. & Di Chiara, G. Differential influence of associative and nonassociative learning mechanisms on the responsiveness of prefrontal and accumbal dopamine transmission to food stimuli in rats fed ad libitum. J. Neurosci. 17, 851–861 (1997).

    Article  CAS  Google Scholar 

  22. Waelti, P., Dickinson, A. & Schultz, W. Dopamine responses comply with basic assumptions of formal learning theory. Nature 412, 43–48 (2001).

    Article  CAS  Google Scholar 

  23. Ungless, M.A., Magill, P.J. & Bolam, J.P. Uniform inhibition of dopamine neurons in the ventral tegmental area by aversive stimuli. Science 303, 2040–2042 (2004).

    Article  CAS  Google Scholar 

  24. Margolis, E.B., Lock, H., Hjelmstad, G.O. & Fields, H.L. The ventral tegmental area revisited: is there an electrophysiological marker for dopaminergic neurons? J Physiol 15, 907–924 (2006).

    Article  Google Scholar 

  25. Phillips, P.E., Stuber, G.D., Heien, M.L., Wightman, R.M. & Carelli, R.M. Subsecond dopamine release promotes cocaine seeking. Nature 422, 614–618 (2003).

    Article  CAS  Google Scholar 

  26. Heien, M.L., Johnson, M.A. & Wightman, R.M. Resolving neurotransmitters detected by fast-scan cyclic voltammetry. Anal. Chem. 76, 5697–5704 (2004).

    Article  CAS  Google Scholar 

  27. Heien, M.L. et al. Real-time measurement of dopamine fluctuations after cocaine in the brain of behaving rats. Proc. Natl. Acad. Sci. USA 102, 10023–10028 (2005).

    Article  CAS  Google Scholar 

  28. Everitt, B.J. & Robbins, T.W. Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nat. Neurosci. 8, 1481–1489 (2005).

    Article  CAS  Google Scholar 

  29. Robbins, T.W. & Everitt, B.J. Limbic-striatal memory systems and drug addiction. Neurobiol. Learn. Mem. 78, 625–636 (2002).

    Article  CAS  Google Scholar 

  30. Uslaner, J.M., Acerbo, M.J., Jones, S.A. & Robinson, T.E. The attribution of incentive salience to a stimulus that signals an intravenous injection of cocaine. Behav. Brain Res. 169, 320–324 (2006).

    Article  CAS  Google Scholar 

  31. Cardinal, R.N. et al. Effects of selective excitotoxic lesions of the nucleus accumbens core, anterior cingulate cortex, and central nucleus of the amygdala on autoshaping performance in rats. Behav. Neurosci. 116, 553–567 (2002).

    Article  Google Scholar 

  32. Parkinson, J.A., Olmstead, M.C., Burns, L.H., Robbins, T.W. & Everitt, B.J. Dissociation in effects of lesions of the nucleus accumbens core and shell on appetitive pavlovian approach behavior and the potentiation of conditioned reinforcement and locomotor activity by d-amphetamine. J. Neurosci. 19, 2401–2411 (1999).

    Article  CAS  Google Scholar 

  33. Pan, W.X. & Hyland, B.I. Pedunculopontine tegmental nucleus controls conditioned responses of midbrain dopamine neurons in behaving rats. J. Neurosci. 25, 4725–4732 (2005).

    Article  CAS  Google Scholar 

  34. Dommett, E. et al. How visual stimuli activate dopaminergic neurons at short latency. Science 307, 1476–1479 (2005).

    Article  CAS  Google Scholar 

  35. Kakade, S. & Dayan, P. Dopamine: generalization and bonuses. Neural Netw. 15, 549–559 (2002).

    Article  Google Scholar 

  36. Parkinson, J.A. et al. Nucleus accumbens dopamine depletion impairs both acquisition and performance of appetitive Pavlovian approach behaviour: implications for mesoaccumbens dopamine function. Behav. Brain Res. 137, 149–163 (2002).

    Article  CAS  Google Scholar 

  37. Ikemoto, S. & Panksepp, J. The role of nucleus accumbens dopamine in motivated behavior: a unifying interpretation with special reference to reward-seeking. Brain Res. Brain Res. Rev. 31, 6–41 (1999).

    Article  CAS  Google Scholar 

  38. Roitman, M.F., Stuber, G.D., Phillips, P.E., Wightman, R.M. & Carelli, R.M. Dopamine operates as a subsecond modulator of food seeking. J. Neurosci. 24, 1265–1271 (2004).

    Article  CAS  Google Scholar 

  39. Wise, R.A. Dopamine, learning and motivation. Nat. Rev. Neurosci. 5, 483–494 (2004).

    Article  CAS  Google Scholar 

  40. Berridge, K.C. The debate over dopamine's role in reward: the case for incentive salience. Psychopharmacology (Berl.) 191, 391–431 (2006).

    Google Scholar 

  41. Berridge, K.C. & Robinson, T.E. What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res. Brain Res. Rev. 28, 309–369 (1998).

    Article  CAS  Google Scholar 

  42. Setlow, B., Schoenbaum, G. & Gallagher, M. Neural encoding in ventral striatum during olfactory discrimination learning. Neuron 38, 625–636 (2003).

    Article  CAS  Google Scholar 

  43. Cepeda, C. & Levine, M.S. Dopamine and N-methyl-D-aspartate receptor interactions in the neostriatum. Dev. Neurosci. 20, 1–18 (1998).

    Article  CAS  Google Scholar 

  44. Kerr, J.N. & Wickens, J.R. Dopamine D-1/D-5 receptor activation is required for long-term potentiation in the rat neostriatum in vitro. J. Neurophysiol. 85, 117–124 (2001).

    Article  CAS  Google Scholar 

  45. Eyny, Y.S. & Horvitz, J.C. Opposing roles of D1 and D2 receptors in appetitive conditioning. J. Neurosci. 23, 1584–1587 (2003).

    Article  CAS  Google Scholar 

  46. Watson, C.J., Venton, B.J. & Kennedy, R.T. In vivo measurements of neurotransmitters by microdialysis sampling. Anal. Chem. 78, 1391–1399 (2006).

    Article  Google Scholar 

  47. Richfield, E.K., Penney, J.B. & Young, A.B. Anatomical and affinity state comparisons between dopamine D1 and D2 receptors in the rat central nervous system. Neuroscience 30, 767–777 (1989).

    Article  CAS  Google Scholar 

  48. Greengard, P. The neurobiology of slow synaptic transmission. Science 294, 1024–1030 (2001).

    Article  CAS  Google Scholar 

  49. Phillips, P.E., Robinson, D.L., Stuber, G.D., Carelli, R.M. & Wightman, R.M. Real-time measurements of phasic changes in extracellular dopamine concentration in freely moving rats by fast-scan cyclic voltammetry. Methods Mol. Med. 79, 443–464 (2003).

    CAS  PubMed  Google Scholar 

  50. Paxinos, G. & Watson, C. The Rat Brain in Stereotaxic Coordinates (Elsevier, New York, 2005).

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank R.A. Wheeler, B.J. Aragona, P.E.M. Phillips and J.L. Jones for helpful comments on this manuscript, and the University of North Carolina Department of Chemistry Electronics Facility for technical expertise. This research was supported by the US National Institute on Drug Abuse (DA 017318 to R.M.C., DA to 10900 R.M.W., DA 021979 to J.J.D. and DA 018298 to M.F.R.).

Author information

Authors and Affiliations

Authors

Contributions

J.J.D. and M.F.R. conducted the behavioral and electrochemical experiments. J.J.D. wrote the manuscript and conducted data and graphical analyses. M.F.R. contributed to the writing of the manuscript. R.M.C. and R.M.W. supervised the project and the writing of the manuscript.

Corresponding author

Correspondence to Regina M Carelli.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary Fig. 1

Individual differences in development of cue-related dopamine signals early in learning. (PDF 1400 kb)

Supplementary Fig. 2

Relationship between approach responses and dopamine signal. (PDF 366 kb)

Supplementary Fig. 3

-evoked dopamine release after extended conditioning. (PDF 345 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Day, J., Roitman, M., Wightman, R. et al. Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens. Nat Neurosci 10, 1020–1028 (2007). https://doi.org/10.1038/nn1923

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn1923

This article is cited by

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing