Corticostriatal Interactions during Learning, Memory Processing, and Decision Making

Introduction

The striatum receives input from a vast expanse of areas distributed across the cortical mantle, yet it is still poorly understood how all of this input is transformed into outputs that regulate behavior and cognition. Cortical projections to the striatum are topographically ordered in a series of parallel anatomical “loops” running from neocortex to the striatum, pallidum, thalamus, and back to neocortex. These parallel macrocircuits have been linked to different global functions: whereas a “limbic,” ventromedial prefrontal–ventral striatal loop has been delineated to mediate motivational and reward processing, other loops engage in sensorimotor or cognitive processing (Alexander et al., 1990; Graybiel et al., 1994; Voorn et al., 2004; Yin and Knowlton, 2006). Information processed along these pathways is under the modulatory control of dopamine released from fibers originating in the ventral tegmental area (VTA) and substantia nigra pars compacta (SNpc). Given the parallel organization of corticostriatal circuits, the question arises how coherent behavior, requiring integration of sensorimotor, cognitive, and motivational information, is achieved. In this mini-symposium, we will first address the question of how parallel corticobasal ganglia circuits may communicate. Next we will ask how corticostriatal circuits are involved in learning and memory formation. One approach to investigate this topic is to zoom in on one corticostriatal subsystem in particular. Therefore we will examine the functioning of the hippocampal–ventral striatal subsystem in learning to associate places and contexts to outcomes, such as reward. We will subsequently focus on the “sensorimotor” subsystem, including the dorsolateral striatum (DLS), in mediating habit formation. Recent advances in associative learning and plasticity in corticostriatal circuits have been guided by theories of reinforcement learning (Sutton and Barto, 1998). Hence we will compare theoretical predictions about dorsal and ventral striatum with results from multineuron recordings in freely behaving rats. In addition to differences in information processing between loops, we will consider commonalities found across striatal subregions. One such common element is the GABAergic fast-spiking interneuron (FSI), which makes up only a tiny fraction of all striatal cells, but powerfully modulates activity of striatal projection neurons, called medium-sized spiny neurons (MSNs). We will review how these interneurons contribute to transforming cortical input into striatal output. Thus, by bringing together findings from neuroanatomy, in vivo electrophysiology, behavioral neuroscience, and computational modeling, this symposium will highlight key advances in our understanding of corticostriatal functions in decision making and memory formation.

Communication between corticostriatal systems via the ventral mesencephalon

For integrating sensorimotor, cognitive, and motivational information that is processed by different corticostriatal circuits, some form of communication between parallel circuits appears to be required. Such communication may take place via intrastriatal collaterals of projection neurons and striatal interneurons (van Dongen et al., 2005). Another route allowing information flow from limbic to sensorimotor-related corticostriatal pathways is via the projections from nucleus accumbens (NAc), which constitutes the main portion of the ventral striatum, to the SNpc and VTA and, subsequently, to dorsal, sensorimotor-related striatal regions via dopaminergic fibers originating from the SNpc/VTA complex (Nauta et al., 1978). Much attention has focused on communication via a cascade-like spiraling circuitry connecting ventromedial regions of striatum with ever more dorsolaterally located striatal areas by way of an open loop with the substantia nigra. This arrangement, described in the monkey (Haber et al., 2000), may hold true also for the rat and allows for a serial rather than parallel flow of information processing. The cellular population in rat SNpc/VTA projecting to the striatum has been shown to comprise at least two types of neurons. The first type shows a mediolateral topographical organization in its projection pattern, with medially located cells projecting to medial parts of the striatum and laterally located cells reaching lateral striatal regions, like a mirror image of the mediolateral topography in the striatonigral axonal projections. The second type does not obey such topography: (dorso)lateral striatal territories are reached from a medial and dorsal position in SNpc and VTA (Maurin et al., 1999).

The former population is probably involved in striatonigro striatal feedback loops (cf. Haber et al., 2000); the latter has nonreciprocal relationships with the striatum and may establish direct contacts between ventral and dorsal corticostriatal circuits. Whether such is indeed the case was studied using neuroanatomical tract tracing and neurophysiological techniques. Rats were simultaneously injected with an anterograde tracer (BDA MW 10,000) into the medial shell region of NAc and a retrograde tracer (Fluorogold) into DLS. A sensorimotor identity of the latter striatal area was confirmed by the presence of retrogradely labeled neurons in somatosensory and motor cortex. Importantly, the retrograde tracer labeled neurons along the entire medial to lateral extent of SNpc and dorsolaterally in VTA. Laterally in SNpc, anterograde fiber labeling from the shell remained separate from retrograde cell labeling. However, in the dorsal parts of medial SNpc and laterally in VTA, the distributional patterns of retrogradely labeled cells and the anterogradely filled fibers from the shell showed extensive overlap (Fig. 1). In these areas, fibers and cell bodies in close proximity to one another suggested the presence of synaptic contacts. Indeed, electrical stimulation of the shell induced an inhibitory response in electrophysiologically identified dopaminergic neurons that were antidromically driven from DLS and located in the very region of distributional overlap. These findings provide for a short and direct route via which limbic information can reach sensorimotor circuits.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Schematic representation of frontal hemisections through the rat forebrain and ventral mesencephalon. Axonal projections from the shell target cells in VTA and SNpc that project either (back) to the shell, to the core, or to sensorimotor-related caudate–putamen (blue-green). Fibers originating from the core reach areas in SNpr associated with axonal projections from SNpc to striatal sectors with inputs from anterior cingulate (blue) or prelimbic (purple) cortex. The blue-green region in medial SNpc and dorsal VTA receives inputs from the shell and contains retrogradely labeled cells after tracer injection in sensorimotor striatum. ac, Anterior commissure; SNpr, substantia nigra pars reticulata.

Hippocampal–striatal communication in contextual conditioning

Traditionally, corticostriatal feedback loops have been associated with executing cognitive and motor functions in the mammalian brain (Alexander and Crutcher, 1990; Chudasama and Robbins, 2006). A potentially important role that is not so readily attributed to the corticostriatal system is that in learning and memory. Extensive evidence suggests that certain subregions of the neocortex and striatum receive convergent inputs from key component structures of learning and memory systems such as the hippocampus and amygdala, and are thus ideally placed to play a regulatory role. However, the functional significance of such a pattern of limbic innervations in the corticostriatal system is yet to be demonstrated. Data from lesion and drug infusion studies selectively targeting corticostriatal areas that receive inputs from the hippocampus and basolateral amygdala support the existence of separate streams of associative information processing in the ventral striatum. Indeed, the integrity of the NAc shell subregion and the activation of dopamine D₂ receptors (Setlow and McGaugh, 1999; Ito et al., 2008) have been shown to be necessary for the processing of spatial information that is also dependent on the hippocampus. In contrast, the NAc core has been shown to be critical for the processing of associations between emotionally significant events and discrete pavlovian stimuli, which is also dependent on the basolateral amygdala (Cador et al., 1989; Parkinson et al., 2000; Ito et al., 2006, 2008).

There is some evidence for different subregions of the prefrontal cortex (PFC) supporting different domains of learning and memory processes, with a number of studies reporting marked impairment in spatial working memory (Ragozzino et al., 1998; Kesner and Ragozzino, 2003) as a consequence of selective lesions of the prelimbic cortex. However, somewhat surprisingly, this selective effect on spatial memory often fails to manifest itself when the lesions are not just restricted to the prelimbic cortex, but extend to other parts of medial PFC (de Bruin et al., 2001; Lacroix et al., 2002; Deacon et al., 2003), highlighting the difficulty of separating mnemonic deficits from concomitant non-mnemonic deficits such as attentional and behavioral flexibility. Nevertheless, studies using disconnection lesions of brain structures within the corticolimbic–striatal system to investigate the functional connectivity between two brain areas have established the importance of serial transfer of spatial information between the prelimbic cortex and hippocampal CA1 region (Floresco et al., 1997) and ventral hippocampus (Wang and Cai, 2006). For the first time, recent work has now demonstrated a functional pathway between the rat hippocampus and NAc shell in the control of conditioned appetitive behavior by spatial contextual information (Fig. 2) (Ito et al., 2008) that is clearly distinct from another circuit between the NAc core and basolateral amygdala, which is concerned with discrete stimulus control over Pavlovian behavior (Everitt et al., 1991). Importantly, interruption of neural processing between the hippocampus and NAc shell was found to cause disruption to the acquisition of context-dependent retrieval of discrete cue information, providing functional evidence that the shell subregion may provide a site at which contextual and discrete cue information may be integrated. In addition, new evidence has been obtained to indicate that the dopaminergic innervation of the rat NAc plays a key role in modulating learning and memory processes. Using a novel paradigm in which two types of sucrose-associated cues (spatial vs nonspatial) are concurrently made available to compete for control over conditioned appetitive behavior, direct infusions of d-amphetamine (which induces elevation in extracellular dopamine levels) into the NAc core and shell were found to exert differential and selective effects upon their conditioning rates. These results point to the important role of the dopaminergic innervations of the NAc shell and core in regulating limbic information flow, and illustrate differential, and even opposing, roles of the NAc core and shell regions in conditioned appetitive behavior under the control of spatial cues.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2.

Effect of disconnection lesions of the hippocampus (HPC) and nucleus accumbens shell on cue and spatial conditioning. A, Schematic representation of asymmetric, unilateral excitotoxic lesions of the hippocampus and shell. B, Schematic diagram of one chamber of the Y-maze apparatus (only one of the three trays and CS lights are represented here). C, Acquisition of place-cue retrieval following core, shell, disconnection lesions, or disconnection sham operations. The difference score is the number of approaches to CS+ minus the number of approaches to CS−. D, Conditioned place preference performance expressed as the percentage of time spent in each chamber. Data are from Ito et al. (2008).

Cross-structural replay of contextual and reward information

Although it is unclear how the consolidation of composite place–reward memories is achieved, the strengthening of synapses involved in coding spatial and motivational information is thought to benefit from the spontaneous replay of recently acquired neural activity patterns that occurs in periods of quiet wakefulness and slow-wave sleep following a behavioral event (Marr, 1971; Buzsáki, 1989; McClelland et al., 1995; McNaughton et al., 2003). Theories on memory consolidation propose that the hippocampus initiates a state of information retrieval during postexperiential, “off-line” periods such as sleep, and transmits this state to its target structures (Marr, 1971; Buzsáki, 1989; McClelland et al., 1995; Pennartz et al., 2002; McNaughton et al., 2003). Neuronal reactivation, as observed during “off-line” periods in rats and monkeys, occurs in several brain areas including the hippocampus (Pavlides and Winson, 1989; Wilson and McNaughton, 1994; Lee and Wilson, 2002), neocortex (Qin et al., 1997; Hoffman and McNaughton, 2002; Euston et al., 2007; Ji and Wilson, 2007), and ventral striatum (Pennartz et al., 2004). These areas appear to replay a specific aspect of a composite memory trace corresponding to the information processed during the awake experience. Spatial–contextual information may be reactivated in the hippocampus as reactivation pertains to cells with neighboring place fields in a given environment (Wilson and McNaughton, 1994; Kudrimoti et al., 1999; Diba and Buzsáki, 2007). In contrast, the ventral striatum predominantly reactivates motivational information (Lansink et al., 2008). Neurons that first expressed a firing-rate change associated with expecting or consuming a liquid reward showed stronger reactivation than neurons not showing a modulation.

If replay across hippocampus and ventral striatum contributes to consolidation of place–reward associations, the different pieces of information belonging to the same event should be reprocessed coherently to prevent formation of erroneous associations with other events. Simultaneous recordings from hippocampal and ventral striatal neuronal ensembles in freely moving rats indeed demonstrated cross-structural reactivation of combined spatial and motivational information (Lansink et al., 2009). Spatial information carried by hippocampal cells is reactivated preferentially in advance of motivational information processed by ventral striatal neurons during postbehavioral rest (Fig. 3). This preferential order (“hippocampus first, ventral striatum second”) could not be explained from a similar preferred order in the behavioral phase. This is the only direct evidence to date of a temporally leading role of the hippocampus in cross-structural replay. Cross-structural replay appears compressed by a factor 10 compared with the behavioral time scale of neuronal activation. Hence, several seconds of “real-time” behavioral segments containing joint place–reward information are brought together in a time frame of hundreds of milliseconds during sleep. Thus, in joint replay activity of hippocampal cells that are directly or indirectly presynaptic to the ventral striatum advances that in striatal cells in a time window allowing long-term synaptic modifications to occur (Markram et al., 1997; Abbott and Nelson, 2000). This capture of spatial and motivational information in narrow time windows may offer a suitable mechanism for storage of place–reward associations.

• Download figure
• Open in new tab
• Download powerpoint

Figure 3.

Schematic representation of spontaneous cross-structural replay of place and reward information. A, As a rat runs along a track, hippocampal neurons (HC1–HC4) are activated at specific locations (blue ellipses), whereas a ventral striatal neuron (VS1) is firing before and after reward (R) reception (red ellipse, yellow dot). B, Spike patterns during three task episodes plotted along with hippocampal local field potentials (HC LFP). During posttrack rest periods, replay of firing patterns takes place on an ∼10 times accelerated time scale. Hippocampal neurons are reactivated shortly before the ventral striatal neuron. During pretrack rest, firing patterns were dissimilar to those during track running. See the study by Lansink et al. (2009) for further details.

Testing computational models of reward learning against corticostriatal physiology

Theories of motivated behavior have drawn much inspiration from the field of reinforcement learning (Sutton and Barto, 1998; Schultz, 2000; Daw and Doya, 2006). Temporal-difference reinforcement learning (TDRL) models, which learn the values of available actions in specific situations (states) to maximize reward, need to track actual reward receipt, state value, and prediction error. An influential view is that cortical areas provide a state signal, DLS associates states with actions, prediction errors are signaled by midbrain dopamine neurons, and ventral striatum encodes state value (Mishkin et al., 1984; Hikosaka et al., 1989; Houk et al., 1995; Schultz et al., 1997; O'Doherty et al., 2004). In such an “actor/critic” arrangement, the ventral striatum is not involved in on-line action selection, but rather serves to train action values located elsewhere, a suggestion supported by a recent lesion study in the rat (Atallah et al., 2007) and consistent with value-related signals found in human fMRI studies (Knutson and Cooper, 2005; Preuschoff et al., 2006) (but see Hare et al., 2008) as well as monkey recording work (Schultz et al., 1992; Cromwell and Schultz, 2003). In the rat literature, reports of reward-responsive neurons in ventral striatum are abundant, but these have generally not yet been related to value [but see Taha and Fields (2005) and Ito and Doya (2009)].

While a “critic” role for the ventral striatum is theoretically attractive, work in rats has long established that the ventral striatum is more directly involved in action selection (Mogenson et al., 1980). A wide range of pharmacological, lesion, and recording studies broadly supports the view that the ventral striatum plays a key role in mediating certain motivated actions, such as responding to cues in autoshaping and Pavlovian-instrumental transfer (Ikemoto and Panksepp, 1999; Cardinal et al., 2002; Nicola, 2007). These and other behaviors, such as outcome-dependent, deliberative decision making (Balleine and Dickinson, 1998) cannot be accommodated in the basic actor/critic TDRL model (Dayan, 2002). More powerful “model-based” approaches rely instead on the representation and evaluation of action outcomes before they are chosen (Daw et al., 2005; Niv et al., 2006; Johnson et al., 2007). Signals related to action outcomes have been identified in orbitofrontal cortex and hippocampus (Schoenbaum et al., 1998; Johnson and Redish, 2007;Ramus et al., 2007), and lesions of these areas impair flexible decision making requiring such nonlocal information (Morris, 2007; Murray et al., 2007). Both orbitofrontal cortex and hippocampus project to ventral striatum, suggesting that the latter may be involved in on-the-fly, internal evaluation (Dehaene and Changeux, 2000; Niv et al., 2006; Johnson et al., 2007) of afferent outcome representations. Such a scheme could reconcile experimental evidence for value representations in the ventral striatum with its direct involvement in action selection via a role for evaluation in such settings.

Is there any evidence for the activation of ventral striatal reward signals during decision making? Building on earlier work (Peoples et al., 2004; Roitman et al., 2004; German and Fields, 2007), van der Meer and Redish (2009a) showed that neurons in rat ventral striatum that responded to reward tended to fire during deliberative decision making as well. Because this effect disappeared with repeated experience and was also apparent during “reversals” at different points on the maze, it cannot be straightforwardly explained by a cue-based account. A similarly dynamic reward signal was present in ventral striatal local field potentials: 80 Hz oscillations not only ramped up to reward receipt, but were also increased at the high-cost choice point (van der Meer and Redish, 2009b). Further work is needed to determine whether these representations of reward during decision making are related to value, and how their timing relates to cortical and hippocampal outcome signals. Thus, ventral striatal gamma oscillations may provide a mechanism through which integration of action outcome and reward information could take place.

Corticostriatal interactions in habit learning

Components of the neocortical–basal ganglia loops are essential for learned actions to become habitual, and abnormal activity within these loops is implicated in a range of clinical disorders related to action compulsion (as in obsessive-compulsive spectrum disorders and drug addiction) and action disability (as in Parkinson's disease and Huntington's disease) (Graybiel, 2008).

Corticobasal ganglia pathways associated with the sensorimotor striatum are thought to be necessary for the transition of instrumental behavior into habits, a process suggested to involve the “chunking” of complex action sequences into units that are rapidly executable, fluid, and robust to changes in outcome contingency (i.e., stimulus–response based) (Dickinson, 1985; Kimura, 1995; Voorn et al., 2004; Poldrack et al., 2005; Yin and Knowlton, 2006; Graybiel, 2008; Balleine et al., 2009; Belin et al., 2009). Graybiel and colleagues have identified neuronal firing patterns in the rat sensorimotor striatum that may be a signature of activity related to this learning and chunking process (Jog et al., 1999; Barnes et al., 2005). Tetrode recordings of MSNs were made in the DLS as rats were trained and then overtrained on a conditional T-maze task. Initially, the average activity of task-related DLS neurons was distributed throughout the maze-running period as rats explored and learned task contingencies. As performance improved and reached asymptote, DLS ensemble spike activity was restructured and continued to evolve during overtraining: the firing of task-related neurons became predominant at the beginning and end of the maze runs and lessened during midrun (relating perhaps to a decrease in firing to the action sequence itself) (Tang et al., 2007), as though the behavior were becoming chunked. In parallel, there was a decline in the average firing of neurons not exhibiting phasic task-related responses. These patterns have been suggested as neural correlates of the explore–exploit behavior of the animals undergoing procedural learning and habit formation. Subsequent extinction of the overtrained behavior was slow, and there was a striking return of firing to resemble patterns recorded during initial learning. Reacquisition rapidly reinstated the task- and non-task-related activity patterns recorded during overtraining. These results indicate that habit formation and modification do not involve turning on and off a striatal “habit system,” but rather a dynamic repatterning of neural activity (Barnes et al., 2005).

Other regions of the brain have also been implicated in mediating action chunking or habit formation, including cerebellar circuits, sensorimotor and limbic cortices, and the substantia nigra (Hikosaka et al., 2002; Aldridge et al., 2004; Faure et al., 2005; Poldrack et al., 2005; Graybiel, 2008; Doyon et al., 2009). The ventral part of medial PFC (termed infralimbic in the rat; IL) has been shown in lesion/microinjection studies to be required for stimulus–response habit formation and maintenance, and neurons there to fire in relation to learned actions or associative signals (Coutureau and Killcross, 2003; Killcross and Coutureau, 2003; Mulder et al., 2003; Peters et al., 2005; Hitchcott et al., 2007). It is not clear how this IL region interacts with the DLS in promoting habitual behavior, as they are thought not to be directly connected, nor whether the IL and DLS serve redundant, parallel, or complementary functions during habit learning. One possibility currently being examined is that limbic–cognitive corticostriatal pathways (involving IL) support repetition of learned behavior or override cognitive goal-directed signals, while sensorimotor pathways (involving DLS) directly represent action chunks and stimulus–response rules (Fig. 4) (Coutureau and Killcross, 2003; Graybiel, 2008; Balleine et al., 2009).

• Download figure
• Open in new tab
• Download powerpoint

Figure 4.

Schematic diagram of parallel cortical–basal ganglia loops contributing to habit formation and maintenance, involving the ventromedial prefrontal cortex (IL) within the limbic–cognitive loop and DLS within the sensorimotor loop. The pseudocolor plot depicts task-related neuronal activity recorded in DLS during training on a T-maze task (modified from Barnes et al., 2005). With overtraining and habit formation, firing patterns develop to accentuate the task start and end (red = greater normalized firing frequency).

Role of interneurons in transforming cortical input to striatal output

The striatum receives inputs from a very broad range of cortical areas, but has far fewer neurons. How are the rich patterns of cortical information transformed as they are compressed and manipulated within striatal microcircuits? While some features of cortical activity patterns are preserved during intrastriatal calculations, the resulting representations by single striatal output neurons appear to be quite different. For example, many medial/ventral striatal neurons show strong entrainment to the hippocampal theta rhythm (Berke et al., 2004). These cells are likely components of a PFC–hippocampal–striatal macrocircuit important for the use of contextual information in decision making; participation in theta rhythms may regulate communication within this macrocircuit. However, even theta-entrained medial striatal cells do not show the spatially linked firing rate change of hippocampal “place cells”; rather, their firing rate is modulated by nonspatial behavioral sequences (Berke et al., 2009).

One approach to deconstructing information processing within striatal microcircuits involves examining the activity patterns of distinct neuronal components. One such component, the FSIs, comprises only ∼1% of striatal neurons (Luk and Sadikot, 2001). However, these cells have been argued to have a critical role filtering out unwanted actions, and postmortem studies have found a deficit in striatal FSIs in Tourette syndrome (Kalanithi et al., 2005). Several groups studied putative FSIs in awake, behaving animals, identifying them on the basis of characteristic brief waveforms, tonic activity, and high-frequency discharges (Berke et al., 2004; Berke, 2008; Gage et al., 2008; Schmitzer-Torbert and Redish, 2008). In rats, FSIs show complex temporal sequences of activity during choice tasks, and do not normally act as a coordinated cell population with closely linked firing rates, even within striatal microregions (Berke, 2008). Thus FSIs are likely involved in the details of intrastriatal calculations, rather than providing a relatively uniform control signal (Fig. 5). This was surprising, not least because adjoining FSIs are electrically coupled together by dendritic gap junctions (e.g., Koós and Tepper, 1999; Fukuda, 2009)—although recent simulations suggest that these gap junctions do not normally synchronize the striatal FSI network (Hjorth et al., 2009). While individual FSIs can show phase locking to a range of cortical–striatal rhythms, including ∼80 Hz LFP oscillations provoked by rewards (Berke, 2005, 2009), such entrainment also appears to reflect particular combinations of cortical inputs rather than a general feature of striatal organization.

• Download figure
• Open in new tab
• Download powerpoint

Figure 5.

Striatal microcircuitry in the context of cortical–basal ganglia macrocircuits. Within striatum (STR), FSIs (red) receive complex combinations of cortical inputs and participate in local information processing by influencing nearby MSNs (blue). FSIs also receive a specific feedback input from globus pallidus (GP) that may serve as a more broadly distributed control signal (Gage et al., 2008).

Despite such idiosyncratic firing, there are some situations in which FSIs do show more coordinated activity. Gage et al. (2008) recorded from multiple basal ganglia and cortical regions simultaneously in a striatum-dependent choice task. Rats had to maintain their position in a nose-poke hole, listen for an instruction tone, hold on for a further “go” signal, then make a rapid left or right movement to receive reward. As they initiated their chosen movement, there was a brief pulse of enhanced FSI activity, that was not seen in other neuronal populations or in conjunction with other movements. This FSI pulse may occur when there is a transient need to suppress alternative highly practiced movements, an interesting possibility in light of the Tourette syndrome findings. FSIs receive a specific but poorly understood backprojection from the globus pallidus (Fig. 5) (Bevan et al., 1998), and the population firing rates of FSIs and pallidal cells were markedly anticorrelated. Thus it appears that FSI are receiving unique combinations of inputs from cortex that may influence the fine timing of MSN spiking, while receiving and broadcasting a simpler, more coordinated feedback signal from globus pallidus.

Summary

Altogether, this mini-symposium will highlight a few of the key advances in research on corticostriatal circuits. Whereas these circuits were previously viewed as primarily operating in parallel, recent findings emphasize the interactions between corticobasal ganglia pathways via several modes of cross talk. The striatum does not appear to operate merely as an “active filter” that converts cortical cognitive, motor, or emotional information into actions, but is deeply involved in learning and memory processes itself, as illustrated by contextual conditioning paradigms, joint hippocampal–ventral striatal replay during sleep and repatterning of neural activity in DLS during overtraining. Activity of fast-spiking interneurons supports the generation of the striatum's own, differentiated dynamics, which is nonetheless coordinated with that in the cortex. Computationally, the dorsal and ventral striatum have been cast into an actor–critic architecture, but current systems neurophysiology suggests a more complicated scheme that emphasizes action value representations.