QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

HOME  |   SEARCH  |   ARCHIVE  |   SUBSCRIBE  |   CONTACT  |   HELP

The Journal of Neuroscience, August 1, 2007, 27(31):8161-8165; doi:10.1523/JNEUROSCI.1554-07.2007

This Article Abstract Full Text (PDF) Submit an eLetter Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in Web of Science Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Web of Science (51) Citing Articles via Google Scholar Google Scholar Articles by Balleine, B. W. Articles by Hikosaka, O. Search for Related Content PubMed PubMed Citation Articles by Balleine, B. W. Articles by Hikosaka, O.

 Previous Article  |  Next Article 

Mini-Review
The Role of the Dorsal Striatum in Reward and Decision-Making

Bernard W. Balleine,¹ Mauricio R. Delgado,² and Okihide Hikosaka³

¹Department of Psychology and the Brain Research Institute, University of California Los Angeles, Los Angeles, California 90095-1563, ²Department of Psychology, Rutgers University, Newark, New Jersey 07102, and ³Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, Bethesda, Maryland 20892-4435

	Abstract

Top Abstract Instrumental conditioning in... Striatal-based learning... Action-contingency learning in... Summary and future directions References

 
Although the involvement in the striatum in the refinement and control of motor movement has long been recognized, recent description of discrete frontal corticobasal ganglia networks in a range of species has focused attention on the role particularly of the dorsal striatum in executive functions. Current evidence suggests that the dorsal striatum contributes directly to decision-making, especially to action selection and initiation, through the integration of sensorimotor, cognitive, and motivational/emotional information within specific corticostriatal circuits involving discrete regions of striatum. We review key evidence from recent studies in rodent, nonhuman primate, and human subjects.

Key words: choice; utility; frontal cortex; executive; reward; striatum

To choose appropriately between distinct courses of action requires the ability to integrate an estimate of the causal relationship between an action and its consequences, or outcome, with the value, or utility, of the outcome. Any attempt to base decision-making solely on cognition fails fully to determine action selection because any information, such as "action A leads to outcome O," can be used both to perform A and to avoid performing A. It is interesting to note in this context that, although there is an extensive literature linking the cognitive control of executive functions specifically to the prefrontal cortex (Goldman-Rakic, 1995; Fuster, 2000), more recent studies suggest that these functions depend on reward-related circuitry linking prefrontal, premotor, and sensorimotor cortices with the striatum (Chang et al., 2002; Lauwereyns et al., 2002; Tanaka et al., 2006).

Importantly, evidence from a range of species suggests that this corticostriatal network controls functionally heterogeneous decision processes involving (1) actions that are more flexible or goal directed, sensitive to rewarding feedback, and mediated by discrete regions of association cortices particularly medial, orbitomedial, premotor, and anterior cingulate cortices together with their targets in caudate/dorsomedial striatum (Haruno and Kawato, 2006; Levy and Dubois, 2006); and (2) actions that are stimulus bound, relatively automatic or habitual, and mediated by sensorimotor cortices and dorsolateral striatum/putamen (Jog et al., 1999; Poldrack et al., 2001). These processes have been argued to depend on distinct learning rules (Dickinson, 1994) and, correspondingly, distinct forms of plasticity (Partridge et al., 2000; Smith et al., 2001). Furthermore, degeneration in these corticostriatal circuits has been linked to distinct forms of psychopathology, e.g., in Huntington's, obsessive compulsive disorder, and Tourette's syndrome on the one hand (Robinson et al., 1995; Bloch et al., 2005; Hodges et al., 2006) and in Parkinson's and multiple system atrophy on the other (Antonini et al., 2001; Seppi et al., 2006).

Here, we review recent evidence implicating the dorsal striatum in decision-making and point to the considerable commonalities in the functionality of this region in rodent, nonhuman primate, and human subjects.

	Instrumental conditioning in rats

Top Abstract Instrumental conditioning in... Striatal-based learning... Action-contingency learning in... Summary and future directions References

 
Behavioral research over the last two decades has identified forms of learning in rodents homologous to goal-directed and habitual learning in humans. This suggestion is based on extensive evidence that choice between different actions, e.g., pressing a lever or pulling a chain when these actions earn different food rewards, is determined by the animals' encoding the association between a specific action and outcome and the current value of the outcome; choice is sensitive both to degradation of the action–outcome contingency and to outcome revaluation treatments (Dickinson and Balleine, 1994; Balleine and Dickinson, 1998). In contrast, when actions are overtrained, decision processes become more rigid or habitual; performance is no longer sensitive to degradation and devaluation treatments but rather is controlled by a process of sensorimotor association (Dickinson, 1994; Dayan and Balleine, 2002). As such, whereas action–outcome encoding appears to be mediated by a form of error-correction learning rule, the development of habits is not (Dickinson, 1994); indeed, traditionally this form of learning has been argued to be sensitive to contiguity rather than contingency (Dickinson et al., 1995).

Recent experiments have started to reveal differences in the circuitry associated with these distinct forms of decision process in rodents (see Fig. 1). Cell body lesions of prefrontal cortex, particularly the dorsal prelimbic (Balleine and Dickinson, 1998; Corbit and Balleine, 2003), but not infralimbic (Killcross and Coutureau, 2003), region abolishes the acquisition of goal-directed actions, and performance is acquired by sensorimotor association alone. Prelimbic involvement in goal-directed learning is, however, limited to acquisition (Ostlund and Balleine, 2005; Hernandez et al., 2006), suggesting that this cortical region is involved in learning-related plasticity localized to an efferent structure. Prelimbic cortex projects densely to both dorsomedial striatum and accumbens core (Gabbott et al., 2005) and, although well controlled studies assessing specifically goal-directed learning suggests that the latter region is not involved in this learning process (Corbit et al., 2001), recent evidence has implicated dorsomedial striatum (Balleine, 2005). Thus, both pretraining and posttraining lesions (Yin et al., 2005c), muscimol-induced inactivation (Yin et al., 2005c), and the infusion of the NMDA-antagonist AP5 (Yin et al., 2005b) within a posterior region of dorsomedial striatum all abolish goal-directed learning and render choice performance insensitive to both contingency degradation and outcome devaluation treatments, i.e., choice becomes rigid and habitual (Yin et al., 2005a).

View larger version (25K): [in this window] [in a new window]   Figure 1. Corticostriatal circuits involved in decision-making. a, b, The learning processes controlling the acquisition of reward-related actions are mediated by converging projections from regions of anteromedial prefrontal cortex (MPC) to the rodent dorsomedial striatum or primate dorsoanterior striatum (DM), whereas the processes mediating the acquisition of stimulus-bound actions, or habits, are thought to be mediated by projections from sensorimotor cortex (SM) to the rodent dorsolateral–primate dorsoposterior striatum (DL) (b). These corticostriatal connections are parts of distinct feedback loops that project back to their cortical origins via substantia nigra pars reticulata (SNr)/globus pallidus internal segment (GPi) and the mediodorsal (MD)/posterior (PO) nuclei of the thalamus. c, Reward and predictors of reward are the major motivational influences on the performance of goal-directed and habitual actions that are thought to be mediated by corticostriatal circuits involving, particularly, ventral striatum (VS) and regions of the amygdala. Dopamine is an important modulator of plasticity in the dorsal striatum, whereas its tonic release has long been associated with the motivational processes mediated by the ventral circuit. VTA, Ventral tegmental area; SNc, substantia nigra pars compacta.

 
Interestingly, evidence suggests that a parallel corticostriatal circuit involving infralimbic and sensorimotor cortices together with the dorsolateral striatum in rodents may mediate the transition to habitual decision processes associated with sensorimotor learning (Jog et al., 1999; Killcross and Coutureau, 2003; Barnes et al., 2005). Whereas the infralimbic region has been argued to mediate aspects of the reinforcement signal controlling sensorimotor association (Balleine and Killcross, 2006), changes in motor and dorsolateral striatum appear to be training related (Costa et al., 2004; Hernandez et al., 2006; Tang et al., 2007) and to be coupled to changes in plasticity as behavioral processes become less flexible (Costa et al., 2006; Tang et al., 2007). Correspondingly, whereas overtraining causes performance to become insensitive to outcome devaluation, lesions of dorsolateral striatum reverse this effect rendering performance goal-directed and once again sensitive to outcome devaluation treatments (Yin et al., 2004). Likewise, muscimol inactivation of dorsolateral striatum renders otherwise habitual performance sensitive to changes in the action–outcome contingency (Yin et al., 2005a). Current evidence suggests, therefore, that, whereas stimulus-mediated action selection is mediated by this lateral corticostriatal circuit, the more flexible, outcome-mediated action selection subserving goal-directed action is mediated by a more medial corticostriatal circuit consistent with the general claim that distinct corticostriatal networks control different forms of decision process (Daw et al., 2005).

	Striatal-based learning processes in nonhuman primates

Top Abstract Instrumental conditioning in... Striatal-based learning... Action-contingency learning in... Summary and future directions References

 
Similarly, recent studies using nonhuman primates have suggested that the striatum may be an important brain area in decision-making. A clue to this hypothesis came from single-unit recording studies using trained animals. Neurons responding to task-related sensory events that become active before task-related motor behaviors and that are tonically active until expected rewards are delivered have been described in a circumscribed region of dorsal striatum (Hikosaka et al., 1989; Hollerman et al., 1998). Importantly, the activity of these neurons has been found to be modulated by the expected presence, amount, or probability of reward or by the magnitude of attention or memory required to execute the task (Kawagoe et al., 1998; Shidara et al., 1998; Cromwell and Schultz, 2003). The coexpression of the sensorimotor, cognitive, and motivational/emotional signals in single neurons provides conditions favorable for learning. Some of the recent studies addressed this question more directly by examining neuronal activity while the animals were learning new sensorimotor associations, new motor sequences, or adapting to new reward outcomes. They found that, indeed, many striatal neurons decrease or increase their activity as learning progressed (Tremblay et al., 1998; Blazquez et al., 2002; Miyachi et al., 2002; Hadj-Bouziane and Boussaoud, 2003; Brasted and Wise, 2004). Notably, visuomotor (saccadic) activity appropriate for the correct saccadic response appeared earlier in the associative striatum (caudate nucleus) than in the dorsolateral prefrontal cortex during the course of learning (Pasupathy and Miller, 2005).

The results of single-unit recordings were further supported by experimentally manipulating learning-related neuronal activity. An orthodox method was to suppress activity of neurons in a small functional area in the striatum by injecting GABA agonists such as muscimol. Consistent with dissociations observed in rodents, the suppression of the anterior, associative striatum disrupted learning of new sequential motor procedures, whereas the suppression of the putamen disrupted execution of well learned motor sequences (Miyachi et al., 1997). A second method has sought to promote learning by electrically stimulating the striatum (Nakamura and Hikosaka, 2006b; Williams and Eskandar, 2006). Importantly, this was effective only when the stimulation was applied just after the animal executed a motor response correctly. The timing specificity raises the possibility that sensorimotor, cognitive, and motivational/emotional signals reach single striatal neurons concurrently at the time around the motor execution to cause plastic changes in synaptic mechanisms.

These experiments on learning have been conducted mostly in the dorsal striatum in which motivational/emotional signals from cortex are thought to be sparse. Rather, it appears more likely that motivational signals are supplied by dopaminergic inputs originating mainly from the substantia nigra pars compacta. It has been hypothesized that dopaminergic neurons encode a mismatch between the expected and the actual reward value (Schultz, 1998). This so-called reward prediction error signal is ideal for guiding learning until the gain of reward is maximized (Houk et al., 1995).

Indeed, studies on the synaptic mechanisms in the striatum have shown that long-term potentiation (LTP) or long-term depression (LTD) can occur in the corticostriatal synapses depending on the combination of cortical inputs, striatal outputs, and D1 and D2 dopaminergic inputs (Reynolds and Wickens, 2002). A study using behaving animals showed that these mechanisms are necessary for behavioral adaptation of motor behavior to changing reward-position contingencies (Nakamura and Hikosaka, 2006a). Local injections of a D1 antagonist into the caudate nucleus lengthened the motor (saccadic) reaction times when large rewards were expected, whereas injections of a D2 antagonist lengthened the reaction times when small rewards were expected.

	Action–contingency learning in the human dorsal striatum

Top Abstract Instrumental conditioning in... Striatal-based learning... Action-contingency learning in... Summary and future directions References

 
Studies in humans corroborate the research in animals suggesting that the dorsal striatum is an integral part of a circuit involved in decision-making. Accumulating evidence, primarily from neuroimaging but also neuropsychological investigations, has implicated the dorsal striatum in different aspects of motivational and learning processes that support goal-directed action. For instance, positron emission tomography (PET) studies report increases in dopamine release in the dorsal striatum (as measured by displacement of endogenous dopamine by radioligands) when participants are presented with potential rewards, such as the opportunity to gain money (Koepp et al., 1998; Zald et al., 2004) or even when presented with food stimuli while in a state of hunger (Volkow et al., 2002). Similarly, fMRI studies typically report increases in blood oxygenation level dependent (BOLD) responses in the dorsal striatum during anticipation of either primary (O'Doherty et al., 2002) or secondary (Knutson et al., 2001) rewards, much like the ventral striatum.

What distinguishes the human dorsal striatum from the rest of the basal ganglia is its involvement in action-contingent learning (Delgado et al., 2000, 2005a; Knutson et al., 2001; Haruno et al., 2004; O'Doherty, 2004; Tricomi et al., 2004). Similar to the animal literature (Ito et al., 2002; Yin et al., 2005c), learning about actions and their reward consequences involves the dorsal striatum (O'Doherty et al., 2004; Tricomi et al., 2004), as opposed to more passive forms of appetitive learning found to depend on the ventral striatum (O'Doherty, 2004). These results mirror neuropsychological studies conducted on patients afflicted with Parkinson's disease, who are impaired in their learning of probabilistic stimuli when action contingencies are present (Poldrack et al., 2001) but are unimpaired when no contingency between action and outcome exists (Shohamy et al., 2004). Within the human dorsal striatum, learning of action–reward associations has been found in both putamen and caudate nucleus with potentially different roles based on their sensorimotor or associative connectivity, respectively (Alexander and Crutcher, 1990). Some studies argue, for example, that the putamen is important for stimulus–action coding (Haruno and Kawato, 2006). In contrast, a number of studies suggest that the head of the caudate nucleus is involved in coding reward-prediction errors during goal-directed behavior (Davidson et al., 2004; O'Doherty et al., 2004; Delgado et al., 2005a; Haruno and Kawato, 2006).

More recently, neuroimaging studies have extended these general ideas on the function of the human dorsal striatum to more complex social issues. Increases in BOLD responses in the dorsal striatum, for example, have been reported when an interactive social component exists, such as the occurrence of cooperation (Rilling et al., 2002) or revenge (de Quervain et al., 2004). The caudate nucleus has also been implicated in the acquisition of social reputations (via reciprocity in an economic exchange game, the "trust game") through trial and error (King-Casas et al., 2005). However, existing social biases (e.g., knowledge about moral characteristics) can also hinder corticostriatal learning mechanisms and influence subsequent decisions (Delgado et al., 2005b). A future challenge for researchers is, therefore, to understand the role of the dorsal striatum in goal-directed behaviors with respect to the vast array of existing social complexities.

	Summary and future directions

Top Abstract Instrumental conditioning in... Striatal-based learning... Action-contingency learning in... Summary and future directions References

 
Together, this recent evidence suggests that the dorsal striatum mediates important aspects of decision-making, particularly those related to encoding specific action–outcome associations in goal-directed action and the selection of actions on the basis of their currently expected reward value. We have summarized these findings and the major trends in research that they imply in Figure 1. These findings are consistent with computational theories of adaptive behavior, notably forms of reinforcement learning, and when considered in the context of current views of the broader corticobasal ganglia system, appear likely to provide the basis for an integrated approach to striatal function.

Important questions still remain, such as the anatomical and functional similarities across species, particularly with respect to the cognitive control of actions, whether the striatum is the only site at which goal-directed learning occurs (Hikosaka et al., 2006) and whether dopamine is the sole teacher of this learning (Aosaki et al., 1994; Seymour et al., 2005). Is the striatum involved only in reward-based learning, or does it contribute to the inhibition of responses associated with aversive consequences (Hikosaka et al., 2006)? Finally, is the product of learning (e.g., the motor memory) stored in the striatum? The differential effects of muscimol injections suggests one of two things: either the associative striatum guides acquisition and the motor memory is stored in the putamen (perhaps in addition to other motor areas), or these regions are involved in quite distinct motor processes; the associative striatum encoding a more abstract relationship between actions and their consequences and the putamen, which is connected with motor cortical areas, encoding actions in the muscle–joint domain that, after extensive practice, become resistant to changes in outcome: the hallmark of habitual behavior.

	Footnotes

 
Received April 6, 2007; revised June 4, 2007; accepted June 4, 2007.

This work was supported by National Institute of Mental Health Grant 56446 (B.W.B.) and by the National Eye Institute Intramural Research Program (O.H.).

Correspondence should be addressed to Bernard W. Balleine, Department of Psychology, Box 951563, Los Angeles, CA 90095-1563. Email: balleine{at}psych.ucla.edu

Copyright © 2007 Society for Neuroscience 0270-6474/07/278161-05$15.00/0

	References

Top Abstract Instrumental conditioning in... Striatal-based learning... Action-contingency learning in... Summary and future directions References

 

Alexander GE, Crutcher MD (1990) Functional architecture of basal ganglia circuits: neural substrates of parallel processing. Trends Neurosci 13:266–271.[CrossRef][Web of Science][Medline]

Antonini A, Moresco RM, Gobbo C, De Notaris R, Panzacchi A, Barone P, Calzetti S, Negrotti A, Pezzoli G, Fazio F (2001) The status of dopamine nerve terminals in Parkinson's disease and essential tremor: a PET study with the tracer [11-C]FE-CIT. Neurol Sci 22:47–48.[CrossRef][Web of Science][Medline]

Aosaki T, Tsubokawa H, Ishida A, Watanabe K, Graybiel AM, Kimura M (1994) Responses of tonically active neurons in the primate's striatum undergo systematic changes during behavioral sensorimotor conditioning. J Neurosci 14:3969–3984.[Abstract]

Balleine BW (2005) Neural bases of food seeking: affect, arousal and reward in corticostriatolimbic circuits. Physiol Behav 86:717–730.[CrossRef][Medline]

Balleine BW, Dickinson A (1998) Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology 37:407–419.[CrossRef][Web of Science][Medline]

Balleine BW, Killcross S (2006) Parallel incentive processing: an integrated view of amygdala function. Trends Neurosci 29:272–279.[CrossRef][Web of Science][Medline]

Barnes TD, Kubota Y, Hu D, Jin DZ, Graybiel AM (2005) Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories. Nature 437:1158–1161.[CrossRef][Medline]

Blazquez PM, Fujii N, Kojima J, Graybiel AM (2002) A network representation of response probability in the striatum. Neuron 33:973–982.[CrossRef][Web of Science][Medline]

Bloch MH, Leckman JF, Zhu H, Peterson BS (2005) Caudate volumes in childhood predict symptom severity in adults with Tourette syndrome. Neurology 65:1253–1258.[Abstract/Free Full Text]

Brasted PJ, Wise SP (2004) Comparison of learning-related neuronal activity in the dorsal premotor cortex and striatum. Eur J Neurosci 19:721–740.[CrossRef][Web of Science][Medline]

Chang JY, Chen L, Luo F, Shi LH, Woodward DJ (2002) Neuronal responses in the frontal cortico-basal ganglia system during delayed matching-to-sample task: ensemble recording in freely moving rats. Exp Brain Res 142:67–80.[CrossRef][Web of Science][Medline]

Corbit LH, Balleine BW (2003) The role of prelimbic cortex in instrumental conditioning. Behav Brain Res 146:145–157.[CrossRef][Web of Science][Medline]

Corbit LH, Muir JL, Balleine BW (2001) The role of the nucleus accumbens in instrumental conditioning: evidence of a functional dissociation between accumbens core and shell. J Neurosci 21:3251–3260.[Abstract/Free Full Text]

Costa RM, Cohen D, Nicolelis MA (2004) Differential corticostriatal plasticity during fast and slow motor skill learning in mice. Curr Biol 14:1124–1134.[CrossRef][Web of Science][Medline]

Costa RM, Lin SC, Sotnikova TD, Cyr M, Gainetdinov RR, Caron MG, Nicolelis MA (2006) Rapid alterations in corticostriatal ensemble coordination during acute dopamine-dependent motor dysfunction. Neuron 52:359–369.[CrossRef][Web of Science][Medline]

Cromwell HC, Schultz W (2003) Effects of expectations for different reward magnitudes on neuronal activity in primate striatum. J Neurophysiol 89:2823–2838.[Abstract/Free Full Text]

Davidson MC, Horvitz JC, Tottenham N, Fossella JA, Watts R, Ulug AM, Casey BJ (2004) Differential cingulate and caudate activation following unexpected nonrewarding stimuli. NeuroImage 23:1039–1045.[CrossRef][Web of Science][Medline]

Daw ND, Niv Y, Dayan P (2005) Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat Neurosci 8:1704–1711.[CrossRef][Web of Science][Medline]

Dayan P, Balleine BW (2002) Reward, motivation, and reinforcement learning. Neuron 36:285–298.[CrossRef][Web of Science][Medline]

Delgado MR, Nystrom LE, Fissell C, Noll DC, Fiez JA (2000) Tracking the hemodynamic responses to reward and punishment in the striatum. J Neurophysiol 84:3072–3077.[Abstract/Free Full Text]

Delgado MR, Miller MM, Inati S, Phelps EA (2005a) An fMRI study of reward-related probability learning. NeuroImage 24:862–873.[CrossRef][Web of Science][Medline]

Delgado MR, Frank RH, Phelps EA (2005b) Perceptions of moral character modulate the neural systems of reward during the trust game. Nat Neurosci 8:1611–1618.[CrossRef][Web of Science][Medline]

de Quervain DJ, Fischbacher U, Treyer V, Schellhammer M, Schnyder U, Buck A, Fehr E (2004) The neural basis of altruistic punishment. Science 305:1254–1258.[Abstract/Free Full Text]

Dickinson A (1994) Instrumental conditioning. In: Animal cognition and learning (Mackintosh NJ, ed), pp 4–79. London: Academic.

Dickinson A, Balleine BW (1994) Motivational control of goal-directed action. Anim Learn Behav 22:1–18.[Web of Science]

Dickinson A, Balleine BW, Watt A, Gonzales F, Boakes RA (1995) Overtraining and the motivational control of instrumental action. Anim Learn Behav 22:197–206.

Fuster JM (2000) Executive frontal functions. Exp Brain Res 133:66–70.[CrossRef][Web of Science][Medline]

Gabbott PL, Warner TA, Jays PR, Salway P, Busby SJ (2005) Prefrontal cortex in the rat: projections to subcortical autonomic, motor, and limbic centers. J Comp Neurol 492:145–177.[CrossRef][Web of Science][Medline]

Goldman-Rakic PS (1995) Architecture of the prefrontal cortex and the central executive. Ann NY Acad Sci 769:71–83.[Web of Science][Medline]

Hadj-Bouziane F, Boussaoud D (2003) Neuronal activity in the monkey striatum during conditional visuomotor learning. Exp Brain Res 153:190–196.[CrossRef][Web of Science][Medline]

Haruno M, Kawato M (2006) Different neural correlates of reward expectation and reward expectation error in the putamen and caudate nucleus during stimulus-action-reward association learning. J Neurophysiol 95:948–959.[Abstract/Free Full Text]

Haruno M, Kuroda T, Doya K, Toyama K, Kimura M, Samejima K, Imamizu H, Kawato M (2004) A neural correlate of reward-based behavioral learning in caudate nucleus: a functional magnetic resonance imaging study of a stochastic decision task. J Neurosci 24:1660–1665.[Abstract/Free Full Text]

Hernandez PJ, Schiltz CA, Kelley AE (2006) Dynamic shifts in corticostriatal expression patterns of the immediate early genes Homer 1a and Zif268 during early and late phases of instrumental training. Learn Mem 13:599–608.[Abstract/Free Full Text]

Hikosaka O, Sakamoto M, Usui S (1989) Functional properties of monkey caudate neurons. III. Activities related to expectation of target and reward. J Neurophysiol 61:814–832.[Abstract/Free Full Text]

Hikosaka O, Nakamura K, Nakahara H (2006) Basal ganglia orient eyes to reward. J Neurophysiol 95:567–584.[Abstract/Free Full Text]

Hodges A, Strand AD, Aragaki AK, Kuhn A, Sengstag T, Hughes G, Elliston LA, Hartog C, Goldstein DR, Thu D, Hollingsworth ZR, Collin F, Synek B, Holmans PA, Young AB, Wexler NS, Delorenzi M, Kooperberg C, Augood SJ, Faull RL, et al (2006) Regional and cellular gene expression changes in human Huntington's disease brain. Hum Mol Genet 15:965–977.[Abstract/Free Full Text]

Hollerman J, Tremblay L, Schultz W (1998) Influence of reward expectation on behavior-related neuronal activity in primate striatum. J Neurophysiol 80:947–963.[Abstract/Free Full Text]

Houk JC, Adams JL, Barto A (1995) A model of how the basal ganglia generate and use neural signals that predict reinforcement. In: Models of information processing in the basal ganglia (Houk JC, Davis JL, Beiser DG, eds), pp 249–270. Cambridge, MA: MIT.

Ito R, Dalley JW, Robbins TW, Everitt BJ (2002) Dopamine release in the dorsal striatum during cocaine-seeking behavior under the control of a drug-associated cue. J Neurosci 22:6247–6253.[Abstract/Free Full Text]

Jog MS, Kubota Y, Connolly CI, Hillegaart V, Graybiel AM (1999) Building neural representations of habits. Science 286:1745–1749.[Abstract/Free Full Text]

Kawagoe R, Takikawa Y, Hikosaka O (1998) Expectation of reward modulates cognitive signals in the basal ganglia. Nat Neurosci 1:411–416.[CrossRef][Web of Science][Medline]

Killcross S, Coutureau E (2003) Coordination of actions and habits in the medial prefrontal cortex of rats. Cereb Cortex 13:400–408.[Abstract/Free Full Text]

King-Casas B, Tomlin D, Anen C, Camerer CF, Quartz SR, Montague PR (2005) Getting to know you: reputation and trust in a two-person economic exchange. Science 308:78–83.[Abstract/Free Full Text]

Knutson B, Adams CM, Fong GW, Hommer D (2001) Anticipation of increasing monetary reward selectively recruits nucleus accumbens. J Neurosci 21(RC159):1–5.[Web of Science][Medline]

Koepp MJ, Gunn RN, Lawrence AD, Cunningham VJ, Dagher A, Jones T, Brooks DJ, Bench CJ, Grasby PM (1998) Evidence for striatal dopamine release during a video game. Nature 393:266–268.[CrossRef][Medline]

Lauwereyns J, Watanabe K, Coe B, Hikosaka O (2002) A neural correlate of response bias in monkey caudate nucleus. Nature 418:413–417.[CrossRef][Medline]

Levy R, Dubois B (2006) Apathy and the functional anatomy of the prefrontal cortex-basal ganglia circuits. Cereb Cortex 16:916–928.[Abstract/Free Full Text]

Miyachi S, Hikosaka O, Miyashita K, Karádi Z, Rand MK (1997) Differential roles of monkey striatum in learning of sequential hand movement. Exp Brain Res 115:1–5.[CrossRef][Web of Science][Medline]

Miyachi S, Hikosaka O, Lu X (2002) Differential activation of monkey striatal neurons in the early and late stages of procedural learning. Exp Brain Res 146:122–126.[CrossRef][Web of Science][Medline]

Nakamura K, Hikosaka O (2006a) Role of dopamine in the primate caudate nucleus in reward modulation of saccades. J Neurosci 26:5360–5369.[Abstract/Free Full Text]

Nakamura K, Hikosaka O (2006b) Facilitation of saccadic eye movements by postsaccadic electrical stimulation in the primate caudate. J Neurosci 26:12885–12895.[Abstract/Free Full Text]

O'Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ (2004) Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304:452–454.[Abstract/Free Full Text]

O'Doherty JP (2004) Reward representations and reward-related learning in the human brain: insights from neuroimaging. Curr Opin Neurobiol 14:769–776.[CrossRef][Web of Science][Medline]

O'Doherty JP, Deichmann R, Critchley HD, Dolan RJ (2002) Neural responses during anticipation of a primary taste reward. Neuron 33:815–826.[CrossRef][Web of Science][Medline]

Ostlund SB, Balleine BW (2005) Lesions of medial prefrontal cortex disrupt the acquisition but not the expression of goal-directed learning. J Neurosci 25:7763–7770.[Abstract/Free Full Text]

Partridge JG, Tang KC, Lovinger DM (2000) Regional and postnatal heterogeneity of activity-dependent long-term changes in synaptic efficacy in the dorsal striatum. J Neurophysiol 84:1422–1429.[Abstract/Free Full Text]

Pasupathy A, Miller EK (2005) Different time courses of learning-related activity in the prefrontal cortex and striatum. Nature 433:873–876.[CrossRef][Medline]

Poldrack RA, Clark J, Pare-Blagoev EJ, Shohamy D, Creso Moyano J, Myers C, Gluck MA (2001) Interactive memory systems in the human brain. Nature 414:546–550.[CrossRef][Medline]

Reynolds JN, Wickens JR (2002) Dopamine-dependent plasticity of corticostriatal synapses. Neural Netw 15:507–521.[CrossRef][Web of Science][Medline]

Rilling J, Gutman D, Zeh T, Pagnoni G, Berns G, Kilts C (2002) A neural basis for social cooperation. Neuron 35:395–405.[CrossRef][Web of Science][Medline]

Robinson D, Wu H, Munne RA, Ashtari M, Alvir JM, Lerner G, Koreen A, Cole K, Bogerts B (1995) Reduced caudate nucleus volume in obsessive-compulsive disorder. Arch Gen Psychiatry 52:393–398.[Abstract/Free Full Text]

Schultz W (1998) Predictive reward signal of dopamine neurons. J Neurophysiol 80:1–27.[Abstract/Free Full Text]

Seppi K, Schocke MF, Prennschuetz-Schuetzenau K, Mair KJ, Esterhammer R, Kremser C, Muigg A, Scherfler C, Jaschke W, Wenning GK, Poewe W (2006) Topography of putaminal degeneration in multiple system atrophy: a diffusion magnetic resonance study. Mov Disord 21:847–852.[CrossRef][Web of Science][Medline]

Seymour B, O'Doherty JP, Koltzenburg M, Wiech K, Frackowiak R, Friston K, Dolan R (2005) Opponent appetitive-aversive neural processes underlie predictive learning of pain relief. Nat Neurosci 8:1234–1240.[CrossRef][Web of Science][Medline]

Shidara M, Aigner TG, Richmond BJ (1998) Neuronal signals in the monkey ventral striatum related to progress through a predictable series of trials. J Neurosci 18:2613–2625.[Abstract/Free Full Text]

Shohamy D, Myers CE, Grossman S, Sage J, Gluck MA, Poldrack RA (2004) Cortico-striatal contributions to feedback-based learning: converging data from neuroimaging and neuropsychology. Brain 127:851–859.[Abstract/Free Full Text]

Smith R, Musleh W, Akopian G, Buckwalter G, Walsh JP (2001) Regional differences in the expression of corticostriatal synaptic plasticity. Neuroscience 106:95–101.[CrossRef][Web of Science][Medline]

Tanaka SC, Samejima K, Okada G, Ueda K, Okamoto Y, Yamawaki S, Doya K (2006) Brain mechanism of reward prediction under predictable and unpredictable environmental dynamics. Neural Netw 19:1233–1241.[CrossRef][Web of Science][Medline]

Tang C, Pawlak AP, Prokopenko V, West MO (2007) Changes in activity of the striatum during formation of a motor habit. Eur J Neurosci 25:1212–1227.[CrossRef][Web of Science][Medline]

Tremblay L, Hollerman JR, Schultz W (1998) Modifications of reward expectation-related neuronal activity during learning in primate striatum. J Neurophysiol 80:964–977.[Abstract/Free Full Text]

Tricomi EM, Delgado MR, Fiez JA (2004) Modulation of caudate activity by action contingency. Neuron 41:281–292.[CrossRef][Web of Science][Medline]

Volkow ND, Wang GJ, Fowler JS, Logan J, Jayne M, Franceschi D, Wong C, Gatley SJ, Gifford AN, Ding YS, Pappas N (2002) "Nonhedonic" food motivation in humans involves dopamine in the dorsal striatum and methylphenidate amplifies this effect. Synapse 44:175–180.[CrossRef][Web of Science][Medline]

Williams ZM, Eskandar EN (2006) Selective enhancement of associative learning by microstimulation of the anterior caudate. Nat Neurosci 9:562–568.[CrossRef][Web of Science][Medline]

Yin HH, Knowlton BJ, Balleine BW (2004) Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur J Neurosci 19:181–189.[CrossRef][Web of Science][Medline]

Yin HH, Knowlton BJ, Balleine BW (2005a) Inactivation of dorsolateral striatum enhances sensitivity to changes in the action-outcome contingency in instrumental conditioning. Behav Brain Res 166:189–196.[CrossRef][Web of Science][Medline]

Yin HH, Knowlton BJ, Balleine BW (2005b) Blockade of NMDA receptors in the dorsomedial striatum prevents action-outcome learning in instrumental conditioning. Eur J Neurosci 22:505–512.[CrossRef][Web of Science][Medline]

Yin HH, Ostlund SB, Knowlton BJ, Balleine BW (2005c) The role of the dorsomedial striatum in instrumental conditioning. Eur J Neurosci 22:513–523.[CrossRef][Web of Science][Medline]

Zald DH, Boileau I, El-Dearedy W, Gunn R, McGlone F, Dichter GS, Dagher A (2004) Dopamine transmission in the human striatum during monetary reward tasks. J Neurosci 24:4105–4112.[Abstract/Free Full Text]

This article has been cited by other articles:

				S. B. R. E. Brown and K. R. Ridderinkhof Aging and the neuroeconomics of decision making: A review Cogn Affect Behav Neurosci, December 1, 2009; 9(4): 365 - 379. [Abstract] [PDF]

				Y. Kubota, J. Liu, D. Hu, W. E. DeCoteau, U. T. Eden, A. C. Smith, and A. M. Graybiel Stable Encoding of Task Structure Coexists With Flexible Coding of Task Events in Sensorimotor Striatum J Neurophysiol, October 1, 2009; 102(4): 2142 - 2160. [Abstract] [Full Text] [PDF]

				N. Bodi, S. Keri, H. Nagy, A. Moustafa, C. E. Myers, N. Daw, G. Dibo, A. Takats, D. Bereczki, and M. A. Gluck Reward-learning and the novelty-seeking personality: a between- and within-subjects study of the effects of dopamine agonists on young Parkinson's patients Brain, September 1, 2009; 132(9): 2385 - 2395. [Abstract] [Full Text] [PDF]

				E. Dias-Ferreira, J. C. Sousa, I. Melo, P. Morgado, A. R. Mesquita, J. J. Cerqueira, R. M. Costa, and N. Sousa Chronic Stress Causes Frontostriatal Reorganization and Affects Decision-Making Science, July 31, 2009; 325(5940): 621 - 625. [Abstract] [Full Text] [PDF]

				B. De Martino, D. Kumaran, B. Holt, and R. J. Dolan The Neurobiology of Reference-Dependent Value Computation J. Neurosci., March 25, 2009; 29(12): 3833 - 3842. [Abstract] [Full Text] [PDF]

				E. Y. Kimchi and M. Laubach Dynamic Encoding of Action Selection by the Medial Striatum J. Neurosci., March 11, 2009; 29(10): 3148 - 3159. [Abstract] [Full Text] [PDF]

				L. Adermark and D. M Lovinger Frequency-Dependent Inversion of Net Striatal Output by Endocannabinoid-Dependent Plasticity at Different Synaptic Inputs J. Neurosci., February 4, 2009; 29(5): 1375 - 1380. [Abstract] [Full Text] [PDF]

				J. Glascher, A. N. Hampton, and J. P. O'Doherty Determining a Role for Ventromedial Prefrontal Cortex in Encoding Action-Based Value Signals During Reward-Related Decision Making Cereb Cortex, February 1, 2009; 19(2): 483 - 495. [Abstract] [Full Text] [PDF]

				E. E. Forbes, A. R. Hariri, S. L. Martin, J. S. Silk, D. L. Moyles, P. M. Fisher, S. M. Brown, N. D. Ryan, B. Birmaher, D. A. Axelson, et al. Altered Striatal Activation Predicting Real-World Positive Affect in Adolescent Major Depressive Disorder Am J Psychiatry, January 1, 2009; 166(1): 64 - 73. [Abstract] [Full Text] [PDF]

				M. R Delgado, J. Li, D. Schiller, and E. A Phelps The role of the striatum in aversive learning and aversive prediction errors Phil Trans R Soc B, December 12, 2008; 363(1511): 3787 - 3800. [Abstract] [Full Text] [PDF]

				J. P. Christianson, A. M. Benison, J. Jennings, E. K. Sandsmark, J. Amat, R. D. Kaufman, M. V. Baratta, E. D. Paul, S. Campeau, L. R. Watkins, et al. The Sensory Insular Cortex Mediates the Stress-Buffering Effects of Safety Signals But Not Behavioral Control J. Neurosci., December 10, 2008; 28(50): 13703 - 13711. [Abstract] [Full Text] [PDF]

				P. Dayan and N. D. Daw Decision theory, reinforcement learning, and the brain Cogn Affect Behav Neurosci, December 1, 2008; 8(4): 429 - 453. [Abstract] [PDF]

				K. Kirmse, A. Dvorzhak, S. Kirischuk, and R. Grantyn GABA transporter 1 tunes GABAergic synaptic transmission at output neurons of the mouse neostriatum J. Physiol., December 1, 2008; 586(23): 5665 - 5678. [Abstract] [Full Text] [PDF]

				D. Schiller, I. Levy, Y. Niv, J. E. LeDoux, and E. A. Phelps From Fear to Safety and Back: Reversal of Fear in the Human Brain J. Neurosci., November 5, 2008; 28(45): 11517 - 11525. [Abstract] [Full Text] [PDF]

				T. S. Gertler, C. S. Chan, and D. J. Surmeier Dichotomous Anatomical Properties of Adult Striatal Medium Spiny Neurons J. Neurosci., October 22, 2008; 28(43): 10814 - 10824. [Abstract] [Full Text] [PDF]

				S. Rossi, V. De Chiara, A. Musella, H. Kusayanagi, G. Mataluni, G. Bernardi, A. Usiello, and D. Centonze Chronic Psychoemotional Stress Impairs Cannabinoid-Receptor-Mediated Control of GABA Transmission in the Striatum J. Neurosci., July 16, 2008; 28(29): 7284 - 7292. [Abstract] [Full Text] [PDF]

				C. F. Doeller, J. A. King, and N. Burgess Parallel striatal and hippocampal systems for landmarks and boundaries in spatial memory PNAS, April 15, 2008; 105(15): 5915 - 5920. [Abstract] [Full Text] [PDF]

				B. W. Balleine The Neural Basis of Choice and Decision Making J. Neurosci., August 1, 2007; 27(31): 8159 - 8160. [Full Text] [PDF]

This Article Abstract Full Text (PDF) Submit an eLetter Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in Web of Science Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Web of Science (51) Citing Articles via Google Scholar Google Scholar Articles by Balleine, B. W. Articles by Hikosaka, O. Search for Related Content PubMed PubMed Citation Articles by Balleine, B. W. Articles by Hikosaka, O.

Home  |   Search  |   Archive  |   Subscribe  |   Contact  |   Help