QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

HOME  |   SEARCH  |   ARCHIVE  |   SUBSCRIBE  |   CONTACT  |   HELP

The Journal of Neuroscience, June 29, 2005, 25(26):6235-6242; doi:10.1523/JNEUROSCI.1478-05.2005

This Article Full Text Full Text (PDF) Submit an eLetter Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in Web of Science Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Web of Science (63) Citing Articles via Google Scholar Google Scholar Articles by Pan, W.-X. Articles by Hyland, B. I. Search for Related Content PubMed PubMed Citation Articles by Pan, W.-X. Articles by Hyland, B. I.

 Previous Article  |  Next Article 

Behavioral/Systems/Cognitive
Dopamine Cells Respond to Predicted Events during Classical Conditioning: Evidence for Eligibility Traces in the Reward-Learning Network

Wei-Xing Pan,¹ Robert Schmidt,² Jeffery R. Wickens,² and Brian I. Hyland¹

Departments of ¹Physiology and ²Anatomy and Structural Biology, School of Medical Sciences, University of Otago, Dunedin 9001, New Zealand

Behavioral conditioning of cue-reward pairing results in a shift of midbrain dopamine (DA) cell activity from responding to the reward to responding to the predictive cue. However, the precise time course and mechanism underlying this shift remain unclear. Here, we report a combined single-unit recording and temporal difference (TD) modeling approach to this question. The data from recordings in conscious rats showed that DA cells retain responses to predicted reward after responses to conditioned cues have developed, at least early in training. This contrasts with previous TD models that predict a gradual stepwise shift in latency with responses to rewards lost before responses develop to the conditioned cue. By exploring the TD parameter space, we demonstrate that the persistent reward responses of DA cells during conditioning are only accurately replicated by a TD model with long-lasting eligibility traces (nonzero values for the parameter ) and low learning rate (). These physiological constraints for TD parameters suggest that eligibility traces and low per-trial rates of plastic modification may be essential features of neural circuits for reward learning in the brain. Such properties enable rapid but stable initiation of learning when the number of stimulus-reward pairings is limited, conferring significant adaptive advantages in real-world environments.

Key words: ventral tegmental area; temporal difference algorithm; dopaminergic; extracellular recordings; reward; associative learning

---

Received April 15, 2005; revised May 13, 2005; accepted May 14, 2005.

This article has been cited by other articles:

				A. Fontanini, S. E. Grossman, J. A. Figueroa, and D. B. Katz Distinct Subtypes of Basolateral Amygdala Taste Neurons Reflect Palatability and Reward J. Neurosci., February 25, 2009; 29(8): 2486 - 2495. [Abstract] [Full Text] [PDF]

				C. B. Puryear and S. J.Y. Mizumori Reward prediction error signals by reticular formation neurons Learn. Mem., December 2, 2008; 15(12): 895 - 898. [Abstract] [Full Text] [PDF]

				C. O. Tan and D. Bullock A Local Circuit Model of Learned Striatal and Dopamine Cell Responses under Probabilistic Schedules of Reward J. Neurosci., October 1, 2008; 28(40): 10062 - 10074. [Abstract] [Full Text] [PDF]

				W.-X. Pan, R. Schmidt, J. R. Wickens, and B. I. Hyland Tripartite Mechanism of Extinction Suggested by Dopamine Neuron Activity and Temporal Difference Model J. Neurosci., September 24, 2008; 28(39): 9619 - 9631. [Abstract] [Full Text] [PDF]

				A. Ishikawa, F. Ambroggi, S. M. Nicola, and H. L. Fields Dorsomedial Prefrontal Cortex Contribution to Behavioral and Nucleus Accumbens Neuronal Responses to Incentive Cues J. Neurosci., May 7, 2008; 28(19): 5088 - 5098. [Abstract] [Full Text] [PDF]

				Y. B. Kim, N. Huh, H. Lee, E. H. Baeg, D. Lee, and M. W. Jung Encoding of Action History in the Rat Ventral Striatum J Neurophysiol, December 1, 2007; 98(6): 3548 - 3556. [Abstract] [Full Text] [PDF]

				E. M. Izhikevich Solving the Distal Reward Problem through Linkage of STDP and Dopamine Signaling Cereb Cortex, October 1, 2007; 17(10): 2443 - 2452. [Abstract] [Full Text] [PDF]

				H. Ji and P. D. Shepard Lateral Habenula Stimulation Inhibits Rat Midbrain Dopamine Neurons through a GABAA Receptor-Mediated Mechanism J. Neurosci., June 27, 2007; 27(26): 6923 - 6930. [Abstract] [Full Text] [PDF]

				J. A. J. van der Meulen, R. N. J. M. A. Joosten, J. P. C. de Bruin, and M. G. P. Feenstra Dopamine and Noradrenaline Efflux in the Medial Prefrontal Cortex During Serial Reversals and Extinction of Instrumental Goal-Directed Behavior Cereb Cortex, June 1, 2007; 17(6): 1444 - 1453. [Abstract] [Full Text] [PDF]

				J. Cheng and M. G.P. Feenstra Individual differences in dopamine efflux in nucleus accumbens shell and core during instrumental learning. Learn. Mem., March 1, 2006; 13(2): 168 - 177. [Abstract] [Full Text] [PDF]

Home  |   Search  |   Archive  |   Subscribe  |   Contact  |   Help