Abstract
The basal ganglia, especially the circuits originating from the putamen, are essential for controlling normal body movements. Notably, the putamen receives inputs not only from motor cortical areas but also from multiple sensory cortices. However, how these sensory signals are processed in the putamen remains unclear. We recorded the activity of tentative medium spiny neurons in the caudal part of the putamen when the monkey viewed many fractal objects. We found many neurons that responded to these objects, mostly in the ventral region. We called this region “putamen tail” (PUTt), as it is dorsally adjacent to “caudate tail” (CDt). Although PUTt and CDt are mostly separated by a thin layer of white matter, their neurons shared several features. Almost all of them had receptive fields in the contralateral hemifield. Moreover, their responses were object selective (i.e., variable across objects). The object selectivity was higher in the ventral region (i.e., CDt > PUTt). Some neurons above PUTt, which we called the caudal–dorsal putamen (cdPUT), also responded to objects, but less selectively than PUTt. Next, we examined whether these visual neurons changed their responses based on the reward outcome. We found that many neurons encoded the values of many objects based on long-term memory, but not based on short-term memory. Such stable value responses were stronger in PUTt and CDt than in cdPUT. These results suggest that PUTt, together with CDt, controls saccade/attention among objects with different historical values, and may control other motor actions as well.
SIGNIFICANCE STATEMENT Although the putamen receives inputs not only from motor cortical areas but also from sensory cortical areas, how these sensory signals are processed remains unclear. Here we found that neurons in the caudal–ventral part of the putamen (putamen tail) process visual information including spatial and object features. These neurons discriminate many objects, first by their visual features and later by their reward values as well. Importantly, the value discrimination was based on long-term memory, but not on short-term memory. These results suggest that the putamen tail controls saccade/attention among objects with different historical values and might control other motor actions as well.
Introduction
The entrance region of the basal ganglia in primates is composed of the putamen and caudate nucleus whose functions are largely different (Yoshida, 1991; Haruno and Kawato, 2006; Grahn et al., 2008; Brovelli et al., 2011). The putamen, not the caudate nucleus, is involved mainly in the control of body movement behaviors (Alexander et al., 1990). Anatomically, sensorimotor cortical areas project to the putamen (Takada et al., 1985; Nambu et al., 2002; Lehéricy et al., 2004). Physiologically, many neurons in the putamen change their activity before and after particular body movements occur (DeLong, 1973; Alexander, 1987; Schultz and Romo, 1988; Kimura, 1990). Lesion or inactivation of putamen neurons disrupts normal body movements (Denny-Brown and Yanagisawa, 1976; Kato and Kimura, 1992; Kendall et al., 1998). Activation of putamen neurons often causes involuntary movements (Alexander and DeLong, 1985). Clinically, lesions including the putamen lead to various kinds of movement disorders (e.g., dystonia and parkinsonism; Bansil et al., 2012). Such changes in body movements are rarely caused by the caudate nucleus (Bhatia and Marsden, 1994; Park, 2016). These and other data suggest that the neural circuits originating from the putamen and caudate nucleus may operate independently (Alexander et al., 1986).
However, the distinction of the putamen and caudate nucleus may not be complete. As a prominent example, the caudal–ventral part of the putamen is close to the rostral part of the caudate tail (CDt), while these regions are largely separated by a white matter (Seger, 2013). Notably, both of these caudate–putamen regions receive inputs from the temporal visual cortical areas (Yeterian and Van Hoesen, 1978; Van Hoesen et al., 1981; Saint-Cyr et al., 1990; Webster et al., 1995). Moreover, both of them project to local areas in the substantia nigra pars reticulate (SNr) and globus pallidus externus (GPe; Saint-Cyr et al., 1990; Amita et al., 2018), suggesting that they contribute to the same behavioral function.
This is highly relevant to our recent study on the direct and indirect pathways of CDt (Kim et al., 2017; Hikosaka et al., 2018). We first found that a majority of CDt neurons are visually responsive (Yamamoto et al., 2012), as expected by the anatomical studies (above). Our main finding was that CDt neurons change the visual response based on the association of the visual stimuli (or objects) with different amounts of reward (Yamamoto et al., 2013). This occurred in many neurons in the downstream circuits of CDt, as follows: (1) direct pathway: caudal–dorsal–lateral part of SNr (cdlSNr); and (2) indirect pathway: caudal–ventral part of GPe (cvGPe), which then projects to cdlSNr (Yasuda et al., 2012; Kim et al., 2017).
Interestingly, the visual responses of these neurons (CDt, cdlSNr, and cvGPe) almost never changed if the reward amount changed frequently across blocks of trials (e.g., 30 trials). Instead, the visual response to an object changed gradually (e.g., across 5 d) if the object continued to be associated with a fixed amount of reward (i.e., large or small; Yasuda et al., 2012; Kim and Hikosaka, 2013; Yamamoto et al., 2013; Kim et al., 2017). Once the value-based change has occurred, the response to the object remained for a long time (e.g., >100 d) even when the object was never shown (Yasuda et al., 2012). This occurred to so many objects (e.g., >500). Correspondingly, the subject (macaque monkey) acquired and retained long-term memories of many objects by their historical values.
Such stable value coding in the CDt circuit was completely different from that in the rostral part of the caudate nucleus [i.e., caudate head (CDh)] where visual neurons change their responses very rapidly based on the recent reward value of the object (Kawagoe et al., 1998), but not on historical value (Kim and Hikosaka, 2013). Thus, the subject also acquired and flexibly changed short-term memories of a few objects by their predicted values (Yasuda et al., 2012; Kim and Hikosaka, 2013; Yamamoto et al., 2013).
These results raised questions. Do neurons in the caudal–ventral parts of the putamen respond to visual stimuli? If so, do they also encode reward values? If so, do they encode long-term memories or short-term memories? We addressed these questions in this research.
Materials and Methods
Two rhesus monkeys (Macaca mulatta, male, 12 and 7 kg, monkeys W and S) were used. All animal care and experimental procedures were approved by the National Eye Institute Animal Care and Use Committee and complied with the Public Health Service Policy on the humane care and use of laboratory animals.
Animal preparation
Using sterile procedures and general isoflurane anesthesia, the animals were implanted a head holder, a recording cylinder, and an eye coil. Analgesics were administered during each surgery and for several days afterward. Upon full recovery from the surgery, the monkeys were trained to perform oculomotor tasks. During training and the subsequent experimental sessions, the head of each monkey was secured to a primate chair located in a darkened booth, and horizontal and vertical eye position signals were recorded using the search coil technique (CNC Engineering). Daily recording sessions began after full recovery from the surgery. Water intake was controlled regularly so that monkeys were motivated to perform the behavioral tasks.
Behavioral procedure
Behavioral procedures were controlled by custom-made C++-based experimentation data acquisition system (Blip; available at http://www.robilis.com/blip/). The monkey sat in a primate chair, facing a front parallel screen in a darkened room. Visual stimuli generated by an active matrix liquid crystal display projector (catalog #PJL3211, ViewSonic) were rear projected onto the screen. We created the visual stimuli (∼10° × 10°) using fractal geometry (Yamamoto et al., 2012).
In this study, five tasks were used for checking the property of visual neurons in the putamen. To examine the object and spatial selectivity of each neuron, we used one set of eight fractal objects while the monkey performed the passive-viewing task. For the object selectivity, the eight objects were presented at the receptive field of the neuron (see Fig. 2). For the spatial selectivity, we chose one object (from among the eight objects) that the neuron responded strongly to and presented it at various positions randomly (see Fig. 3). To examine the short-term object reward value, we used the flexible value task using two fractal objects (see Fig. 4). To examine the long-term object reward value, we used the stable value procedure that contained the following three tasks: the stable value task for learning; the free-viewing task for the behavioral test; and the passive-viewing task for the neuronal test (see Fig. 5). In this study, five sets of eight fractal objects were used for the long-term object reward value procedure (see Fig. 5B). The details are described below.
Passive-viewing task (to test object selectivity).
The purpose of this task was to examine the object selectivity of the visual responses of the striatal neurons (see Fig. 2A). In this task, some of the fractal objects (n = 2–4) were chosen pseudorandomly and presented sequentially in a receptive field (presentation time, 400 ms; interval, 400 ms), while the monkey kept fixating on a central white dot. Liquid reward (0.2 ml) was delivered 300 ms after the last object was presented. The reward was thus not contingently associated with any object. Each object was presented at least seven times in one session. For each neuron, we usually used one set of objects (eight objects, no value associated) to test its object selectivity.
Spatial mapping task.
We tested the spatial preferences of neurons using the spatial mapping task in which an object was presented at 33 different positions (see Fig. 3A). The task procedure was the same as the passive-viewing task except for the various object positions and the interobject interval (200 ms). First, the most responsive object was chosen for each neuron using the passive-viewing task and used in the spatial mapping task. In each trial, the selected object was presented at one position among 33 combinations of five eccentricities (0°, 5°, 10°, 15°, 20°) and eight directions (45° apart starting with the right direction).
Flexible value task.
This procedure allowed us to examine the effects of short-term object reward association on saccadic behavior and neuronal activity (Yasuda et al., 2012; Kim and Hikosaka, 2013; Yamamoto et al., 2013). Each trial began with the appearance of a fixation point (1° square spot) at the center of the screen. In each trial, one of two fractal objects was pseudorandomly presented at a receptive field or the opposite position. When the receptive field of the neuron was at the center, the object was presented at the horizontal axis. The fixation point and target are overlapped for 400 ms. The monkey was required to make a saccade to the target after fixation point offset. In a block of 20–35 trials, one of the objects was associated with a large reward (0.3 ml, good objects), and the other with small reward (0.1 ml, bad objects). In the next block, the object reward contingency was reversed.
Stable object value association procedure.
This procedure allowed us to examine the effects of long-term object reward association on saccadic behavior and neuronal activity, while excluding any short-term reward effect (Yasuda et al., 2012; Kim and Hikosaka, 2013; Yamamoto et al., 2013). The learning was long term because the values of visual objects were fixed across daily training sessions. The learning of object values, behavioral testing, and neuronal testing was done separately (see Fig. 5A). For object–reward association learning, we used the stable value task (see Fig. 5C). For behavioral testing, we used the free-viewing task (see Fig. 6A). For neuronal testing, we used the passive-viewing task (see Fig. 7A). This separation of the learning and testing procedures precluded the possible influences of short-term reward effects. Importantly, the testing procedure was performed in a neutral condition, as follows: the monkey obtained no reward when learned objects were presented (in the case of behavioral testing) or the monkey did obtain a reward but not in association with particular objects (in the case of neuronal testing).
Stable value task.
The sequence of the stable value task was the same as the flexible value task, but the reward outcome remained unchanged for each object. In each session, a set of eight fractal objects was used as the target. On each trial, one of the fractal objects was chosen pseudorandomly as the target and was presented at one of four positions (right, up, left, and bottom). The half of the fractal objects was always associated with a large reward (0.3 ml; i.e., good objects), whereas the other half were always associated with small reward (0.1 ml; i.e., bad objects). One learning session consisted of 80 trials. Each set of objects was trained in one learning session in 1 d. The same sets of fractals were used repeatedly for learning for >5 d, throughout which time each object remained either a good object or a bad object (see Fig. 5B,C).
Free-viewing task.
This task was used as the behavioral test (see Fig. 5A), in which we examined how the monkey responded to the value-learned objects, but without any reward outcome (Yasuda et al., 2012; Kim and Hikosaka, 2013; Yamamoto et al., 2013). After the monkey fixated on a central white dot for 300 ms, four objects were chosen pseudorandomly and presented simultaneously in four symmetric positions (15° from center Fig. 6A). The monkey was free to look at them for 2 s without any reward outcome. After a blank period (500 ms), a white dot was presented at one of eight positions. If the monkey made a saccade to it, a liquid reward was delivered (0.2 ml). Each object was presented at least 10 times in one session.
Passive-viewing task (to test stable value coding).
If the monkey showed a significant behavioral bias in the free-viewing task (see Fig. 6), we started the neuronal test by using the passive-viewing task (see Fig. 7A;Yasuda et al., 2012; Kim and Hikosaka, 2013; Yamamoto et al., 2013). This is the same as the passive-viewing task used for object selectivity testing (see Fig. 2A), except that the objects had previously been associated with either a big or small reward. Since the objects were chosen pseudorandomly and the random number of them were presented sequentially before the reward, each object was no longer associated with a big or small reward. If the neuron still responded to these objects differently in relation to the previous object–reward association, the neuron was regarded to encode stable values (i.e., historical, not predicted, values). We usually used two to three sets of well learned objects (see Fig. 5B).
The behavioral test (the free-viewing task) and the neuronal test (the passive-viewing task) were performed in separate experiments, typically on separate days. To retain the object value memory, we let the monkey perform the stable value task at least once per 60 d after initial learning (see Fig. 5D).
Recording procedure
Based on a stereotaxic atlas (Saleem and Logothetis, 2007), a recording chamber was placed over the parietal cortex, tilted laterally by 25° (monkey W) or 0° (monkey S), and was aimed at the caudal part of putamen and CDt. MR images (4.7 T; Bruker) were then obtained along the direction of the recording chamber that was visualized with gadolinium that filled grid holes.
To record from single neurons, a tungsten electrode (Alpha Omega Engineering/FHC) was lowered into the striatum through a guide tube using a micromanipulator (MO-97S, Narishige). The recording site was determined using a grid system, which allowed electrode penetrations at every 1 mm. We amplified and filtered (0.3–10 kHz; A-M Systems; BAK) signals obtained from the electrodes and collected them at 1 kHz via a custom-made window discriminator. Single neurons were isolated on-line using the custom voltage–time window discrimination software (BLIP). Based on the grid hole position and the reading of the electrode depth, we estimated the three-dimensional position of the electrode.
We searched for visual neurons when monkeys performed the spatial-mapping task. Some of the fractals were chosen pseudorandomly and presented sequentially (33 positions × 8 fractals). If a single visual neuron is isolated, we identified the receptive field with firing rate changes by monitoring sounds from the audio device. Then, we pseudorandomly presented the one of eight fractals (not value-associated objects) in the receptive fields to examine the object selectivity. We identified the preferred object that the neuron showed the most response to and present it at the 33 positions to examine the spatial selectivity. After that, we tested the long-term value coding of neurons by using the passive-viewing task with value-associated fractal sets. Next, we tested the effect of the short-term reward memory by using the flexible value task. In the initial experiments, we did not test the object and spatial selectivity for neurons.
Presumed medium spiny neurons were identified by their low baseline activity (usually <3 spikes/s) and broad action potential. The baseline firing rate is the mean firing rate during the 1 s before the onset of the fixation point. We did not include the data from presumably tonically active neurons, which exhibited a characteristic tonic firing pattern and wider action potentials (Aosaki et al., 1995). Presumed fast-spiking interneurons, which exhibited high baseline activity and a sharp spike shape (Yamada et al., 2016), were also excluded.
Data analysis
We defined the putamen tail (PUTt) as the region 0–3.5 mm from the ventral edge of putamen, and the caudal–dorsal putamen (cdPUT) as the region above it (Fig. 1A). Because the border between CDt and PUTt is not clear in the anterior part [anterior commissure (AC)-5 and AC-6], we discriminated between them based on the location of the border in the more posterior part (AC-7 to AC-9). We defined visual neurons as neurons that showed a significant difference in activity between the baseline period (250 ms before object onset) and the object presentation period (0–400 ms after object onset; paired t test, p < 0.05). The time course of neuronal activity for each condition is shown by the spike density function using a Gaussian kernel (σ = 15 ms). We used only the data in correct trials for behavioral and neuronal analysis.
To quantify object selectivity and spatial selectivity for a neuron, we calculated the selectivity index, as follows: 1 − FRall/(FRmax × n), where FRall is the total of firing rate in all objects or positions, FRmax is the highest firing rate among tested objects or positions, and n is the number of tested objects or positions. To check whether the neuron showed spatial selectivity, we compared its responses to objects in the contralateral field and to objects in the ipsilateral field by using the Wilcoxon rank-sum test.
To examine the neuronal discrimination, we measured the magnitude of the response to each fractal object by counting the numbers of spikes within a test window in individual trials. The value modulation index was defined as the area under the receiver operating characteristic (ROC) based on the response magnitudes of the neuron to good objects versus bad objects. The value modulation index was calculated for each neuron. The statistical significance of the value modulation index for each group of neurons (PUTt, cdPUT, CDt) was tested using the Wilcoxon rank-sum test (see Figs. 4, 7).
In the stable value condition (see Fig. 7), some neurons responded more strongly to good objects (ROC > 0.5, positive neurons), while others responded more to bad objects (ROC < 0.5, negative neurons). We then averaged the responses of positive and negative neurons in each region separately. This was done by using a cross-validation method. Specifically, trials in one recording session were divided into the odd- and even-numbered trials. Either odd- or even-numbered trials were randomly chosen for determining the preferred value of the neuron (using the ROC analysis), and the other was used for computing the average response. The cross-validation method precluded any artificial result of neuronal discrimination due to an arbitrary choice of the preferred value.
The value modulation index may also be influenced by the object selectivity. It is difficult to address this question for each neuron. Instead, we analyzed data for the whole population of neurons in each area using a permutation test (see Fig. 7F–H). For each neuron, we divided the tested objects (good and bad) into two groups randomly and calculated the value modulation index based on this random division. Applying this analysis for all neurons, we combined the random data of individual neurons. We repeated the above processes 1000 times. The result will be called “modeled data,” which is based only on the object selectivity (not object values). If the “experimental data” are more widely distributed than the modeled data, it would indicate that the whole population of neurons together encode object values.
To calculate the value modulation index for individual objects (see Fig. 7I–K), we first compared the response of one neuron to one object (e.g., good) with the averaged responses of the same neuron to four objects with the opposite value (e.g., bad) within the same set of eight objects. For this value bias score, we calculated the ROC based on the number of spikes within a response window (0–400 ms after object presentation). We then averaged the modulation index separately for two groups of neurons (positive coding and negative coding) in each area (cdPUT, PUTt, CDt).
Results
Subregions in the caudal striatum
We studied the activity of neurons in the caudal part of putamen, which is adjacent to the caudal part of the caudate nucleus called the CDt (Fig. 1A). A recent anatomical research in our laboratory showed that many neurons in a restricted area in the caudal–ventral part of the putamen (we called this region the PUTt), together with the CDt, project to cdlSNr and cvGPe (Fig. 1B; Amita et al., 2018). We thus divided the caudal putamen dorsoventrally to the cdPUT and PUTt, and studied the neuronal activity in cdPUT, PUTt, and CDt.
Locations of the PUTt and CDt. A, Top, An image of putamen (yellow, PUTt shown by dark yellow) and caudate nucleus (gray). Below are coronal sections showing the locations of visual neurons (filled dots, monkey W; unfilled dots, monkey S). These sections are 5–10 mm posterior to the AC and are reconstructed by MR images (T1-weight image, last panel). Note that the most caudal section (AC-10) is shown by an MR image. A scale bar below each section indicates the distance from midline (12–15 mm). The PUTt was defined as the region 0–3.5 mm from the ventral edge of putamen. Dashed lines indicate the borders between cdPUT (caudal–dorsal putamen), PUTt, and CDt. B, Anatomy of PUTt and CDt. Many neurons in PUTt and CDt project to the cdlSNr, which selectively encodes stable values of visual objects. CTB488 was injected in cdlSNr, and retrogradely labeled neurons (green dots) were localized in CDt and PUTt. Modified from Amita et al. (2018) with permission. C, Baseline activity of the visual neurons in each region (mean and 95% confidence interval). The across-region difference is also shown (Scheffé test, *p < 0.05).
We recorded the activity of 241 putative medium spiny neurons (156 from monkey W, 85 from monkey S) in the caudal striatum (cdPUT, PUTt, CDt), while monkeys viewed and made saccades to fractal objects. Among them, 81% of neurons (196 of 241 neurons) showed visual responses to the fractal objects. Among the visual neurons, the firing rate during fixation (baseline activity) tended to be lower in the ventral part, especially CDt (Fig. 1C; F(2,195) = 10.36, p = 5.35E-05, ANOVA; p < 0.05, Scheffé test).
The property of visual response
We first studied whether these caudal striatal neurons respond differently to different fractal objects by using the passive-viewing task (Fig. 2A; see Materials and Methods). In this task, the object was chosen randomly from a set of eight fractals (no value associated), which were presented one at a time at the preferred position for the recorded neuron (Fig. 3) while the monkey fixed its gaze at the center. Figure 2B shows the activity of a representative neuron in PUTt. This neuron responded to many objects, while the response magnitude varied across the objects (F(7,51) = 14.7, p = 1.08E-09, ANOVA).
Object selectivity of visual neurons in the caudal putamen and caudate tail. A, In the passive-viewing task, fractal objects (no value associated) were presented sequentially in the preferred location of the neuron, while the monkey was fixating at the center. The reward was not associated with particular objects. B, Object-selective responses of a PUTt neuron from monkey S to fractal objects during the passive-viewing task. C–E, Distributions of the object selectivity index among neurons in cdPUT (C), PUTt (D), and CDt (E). The neurons showing significant object selectivity are indicated by black (ANOVA, p < 0.05). F, Object selectivity in each region (mean and 95% confidence interval). The across-region difference is also shown (Scheffé test, *p < 0.05).
Spatial selectivity of visual neurons in the caudal putamen and caudate tail. A, The response of a PUTt neuron in monkey W to a preferred object that was presented at 33 positions. B, Preferred positions of PUTt neurons. Each circle indicated the position where the neuron responded most strongly and the number of neurons. C, Spatial selectivity in each region (mean and 95% confidence interval).
To evaluate the object selectivity for each neuron, we calculated the object selectivity index (Materials and Methods). If a neuron shows strong object selectivity, the index is higher, close to 1 (Fig. 2C–E). The object selectivity of the neuron shown in Figure 2B was 0.78. Among all 124 neurons studied, 66 neurons (53%) showed significant object selectivity. Many of them were distributed in the ventral region (Fig. 2C–E), as follows: cdPUT (5 of 29, 17%), PUTt (26 of 48, 54%), and CDt (35 of 47, 74%). Similar results were found in both monkeys [monkey W: cdPUT (2 of 9, 22%), PUTt (11 of 18, 61%), and CDt (29 of 39, 74%); monkey S: cdPUT (3 of 20, 15%), PUTt (15 of 30, 50%), CDt (6 of 8, 75%)]. On average, the object selectivity index was statistically different across the three regions (Fig. 2F; F(2,77) = 5.20; p = 0.01, ANOVA; p < 0.05, Scheffé test): cdPUT (0.45 ± 0.06), PUTt (0.58 ± 0.06), and CDt (0.69 ± 0.05).
To examine the spatial selectivity, we presented the preferred object for the recorded neuron at 33 positions while the monkey fixed its gaze at the center. Figure 3A shows the activity of a representative neuron in PUTt. Its response was largely confined to the hemifield contralateral to the recording site. A majority of PUTt neurons (90%) responded most strongly to a position in the contrarateral hemifield or along the vertical midline (Fig. 3B). Similar selectivities were found in cdPUT (88%) and CDt (93%). On average, the spatial selectivity index was not statistically different across the three regions (Fig. 3C; F(2,88) = 0.59; p = 0.56, ANOVA; p > 0.05, Scheffé test).
Flexible object value information
To examine the effects of short-term reward experiences, we used a flexible value task (Fig. 4A) in which the object reward contingency was reversed in a blockwise manner. In the first block of trials, one object was associated with a large reward (good objects) and the other was associated with a small reward (bad objects). In the second block, the relationship was reversed. Figure 4B shows the target acquisition time during all recording sessions in two monkeys. It was shorter for the saccades to the good object than those to the bad object (z = −8.56, p = 1.09E-17, Wilcoxon rank-sum test: monkey W, 44 sessions, z = −7.99, p = 1.30E-15; monkey S, 25 sessions, z = −3.91, p = 9.37E-5). This result indicated that monkeys learned the object value quickly and changed their behavior based on short-term memory.
Responses of caudal putamen neurons to objects with flexibly changing values. A, Sequence of events in the flexible value task. Each of two objects changed its value (good, with large reward; bad, with small reward) across blocks of trials (as shown in C). B, Target acquisition time for good or bad objects, defined as the time from the offset of the fixation spot to the time when the eye position entered the object area (mean ± SEM; **p < 0.01 by Wilcoxon rank-sum test). C, A representative PUTt neuron from monkey S responded to the two objects differently, but was insensitive to their flexibly changing values. D, Population activity of neurons in cdPUT during the flexible value task (right) and the distribution of value modulation indices of the individual neurons (left). The population activity (right) is shown separately for two conditions: when the objects were good vs bad. The shaded areas indicate ±SEM. For value modulation indices (left), neurons that responded differently depending on the flexible value (good vs bad) are shown by black (Wilcoxon rank-sum test, p < 0.05). A value modulation index score of 1.0 would mean that the neuronal activity was always higher by good objects than bad objects; a score of 0.0 would mean the opposite. The triangle with horizontal lines indicates the mean ± 95% confidence interval. E, Same data for neurons in PUTt.
We recorded activity of 20 and 36 visual neurons in cdPUT and PUTt, respectively, using the flexible value task. Figure 4C shows the activity of a representative PUTt neuron. While the neuron responded to the two objects differently, the response to each object did not change when the object changed its value (z = 0.84, p = 0.40, Wilcoxon rank-sum test). As a population, neurons in both cdPUT (Fig. 4D, left) and PUTt (Fig. 4E, left) showed no significant change in visual response based on the predicted reward value (t(19) = −0.03, p = 0.98; t(35) = 1.66, p = 0.11; paired t test). To evaluate the value-dependent bias for each neuron, we calculated value modulation index (Materials and Methods). A ROC area of >0.5 indicates that the neuron responded more strongly to good objects than to bad objects. Their indices were distributed ∼0.5 and were not significantly different from 0.5, except for two PUTt and two PUTt neurons (p < 0.05, paired t test).
These results confirmed that both cdPUT and PUTt neurons were virtually uninfluenced by predicted reward values. Previous studies reported that CDt neurons also showed no effect of predicted reward values (Kim and Hikosaka, 2013; Yamamoto et al., 2013). These results, together, suggest that the caudal part of striatum does not encode object value information based on short-term memory.
Stable object value information
Next, we examined the effects of long-term reward experiences on caudal putamen neurons by using the stable value procedure (Yasuda et al., 2012; Kim and Hikosaka, 2013; Yamamoto et al., 2013; Ghazizadeh et al., 2016a; Griggs et al., 2018). This procedure consisted of the following two steps: (1) stable object value learning and (2) testing of saccadic behavior and neuronal activity (Fig. 5A). These procedures were performed on separate days so that any effect of recently updated object values (i.e., short-term reward experiences) was excluded. In each trial of the stable value task, one fractal object was chosen randomly from a set of eight objects (Fig. 5B) and was presented as the target, the monkey made a saccade to it, and a reward was delivered (Fig. 5C). The reward amount was fixed (either small or large) for each object across trials and across the entire experience of the monkey. Among a set of eight fractal objects, four were always associated with a large reward (good objects); the other four were always associated with a small reward (bad objects).
Procedure for testing the effect of the stable object value learning. A, After across-day learning (stable value task; C), its effect is tested for the subject's behavior (free viewing; Fig. 6A) and for the neuronal response (passive viewing; Fig. 7A). The free-viewing task and the passive-viewing task were performed as separate sets of experiments. B, Many fractal objects (n = 40) were divided into the following two groups: (1) consistently associated with a large reward (Good) and (2) consistently associated with a small reward (Bad). C, The stable value task was similar to the flexible value task (Fig. 4A), but the reward outcome remained unchanged for each object. D, Time line of training and testing: behavioral and neuronal tests were started after the across-day value learning session (>5 d). Later, the value-learning session was performed only occasionally, once per <60 d.
Figure 5D shows the time line of the stable value procedure. Each monkey experienced multiple sets of objects (Fig. 5B), with each set experienced across many days (usually >5 d). Within 1 d, one object set was presented as one learning session. After this initial learning, we started the behavioral and neuronal test. Other object sets were added for learning on subsequent days. Our previous studies reported that monkeys remembered the value of many objects for a long time (Yasuda et al., 2012; Griggs et al., 2018), even >1 year after the last learning experience (Ghazizadeh et al., 2018). In the current study, the learning was repeated occasionally (e.g., once per 60 d) to make the object value memories sustained.
To test the behavioral effect of object–reward association, we examined the gaze and saccadic eye movement of the monkey in the free-viewing task (Fig. 6A). In this task, four objects were chosen randomly from a set of eight objects and were presented simultaneously. The monkey was free to look at them for 2 s. Because there was no behavioral outcome related to the presented objects, any bias of the gaze of the monkey would be based on the long-term value memories of individual objects that were created in the stable value task (Fig. 5C), which usually had been performed many days before. Example data during the free-viewing task are shown in Figure 6B for the mean gaze duration for each object in a single session of the free-viewing task. Before experiencing the stable value task (Fig. 5C) for this set of objects, the gaze duration was randomized across the objects (Fig. 6B, top). This changed completely after the object value training (Fig. 6B, bottom): the monkey often looked at good objects while mostly avoiding bad objects. Notably, the free-viewing task was performed 47 d after the last training (stable value task), indicating that object value memories were retained for a long time, as described above.
Gaze bias after stable object value learning. A, During the free-viewing task, four fractal objects were chosen pseudorandomly from one set (eight objects; see B) and presented simultaneously. Monkeys were free to look at these objects (or look elsewhere) for 2 s without reward feedback. B, Example data (monkey W): gaze durations on a set of eight fractal objects (Fig. 5B, first set) during one session of the free-viewing task before (top) and after learning (bottom). The monkey had learned stable values of these objects eight times during the stable value task (Fig. 5C), but the last learning session occurred 47 d before the free-viewing task. The data indicate the mean and SD of gaze duration in each trial. C, The percentage of the gaze duration on good or bad objects (left) and the first saccade to a good or bad object (right) during a total of 139 sessions (from two monkeys) of the free-viewing task (mean ± SEM; **p < 0.01 by Wilcoxon rank-sum test).
We then collected the behavioral data in all sessions for all fractal sets (Fig. 5B) after the value learning (Fig. 5D). As a whole, two monkeys preferred to look at good objects for a long time (Fig. 6C, left; t(138) = 16.44, p = 9.44E-34, paired t test; monkey W: 93 sessions, t(92) = 15.63, p = 1.67E-27; monkey S: 45 sessions, t(44) = 12.86, p = 7.77E-17) and tended to make the first saccade to good objects (Fig. 6C, right; t(138) = 20.01, p = 1.25E-42, paired t test; monkey W: 93 sessions, t(92) = 16.40, p = 2.32E-28; monkey S: 45 sessions, t(44) = 7.40, p = 2.96E-9). These results suggest that the monkeys had learned and retained the historical (not predicted) values of many objects and thereby chose good objects automatically.
To test the neuronal effect of object–reward association, we examined the activity of caudal striatal neurons during the passive-viewing task (Fig. 7A). Because the reward outcome was not associated with the presented objects, the neuronal response should be affected by the previous learning (Fig. 5C), not by any predicted reward value (Fig. 4A). Figure 7B shows the activity of a representative neuron in PUTt. The neuron responded to good objects more strongly than to bad objects, although the responses to the good objects were variable. On average, the responses to good objects were statistically larger than those to bad objects (Fig. 7B, right; z = 6.70, p = 2.15E-11, Wilcoxon rank-sum test).
Neuronal bias after stable object value learning. A, In the passive-viewing task, fractal objects were presented sequentially in the preferred location of the neuron (same as in Fig. 2A). B, A representative PUTt neuron from monkey S responded to stably good objects more strongly than to stably bad objects (Fig. 5B, first set) during the passive-viewing task. The monkey had learned stable values of these objects 12 times during the stable value task (Fig. 5C), but the last learning was 44 d before the passive-viewing task. Right, The averaged responses to good and bad objects. C–E, Distribution of value modulation indices of individual neurons in cdPUT (C), PUTt (D), and CDt (E). Neurons that showed a significant difference in response between good and bad objects are shown in black (Wilcoxon rank-sum test, p < 0.05). A value modulation index score of 1.0 would mean that the neuronal activity was always higher for good objects than for bad objects; a score of 0.0 would mean the opposite. The filled and unfilled triangles indicate the mean and mean±2SD, respectively. According to a permutation test (see Materials and Methods), Gray solid lines indicate the estimated mean (= 0.5), and dashed lines indicate the estimated mean+2SD and mean−2SD in cdPUT (C), PUTt (D), and CDt (E). F–H, Population activity of neurons, which is shown separately for neurons coding negative values (negative neurons, left) and neurons coding positive values (positive neurons, right) using a cross-validation method (see Experimental procedures). Red and blue lines indicate the averaged neuronal responses for good and bad object, respectively. The yellow line indicates the difference between the preferred and nonpreferred responses (mean ± SEM). I–K, Neuronal discrimination of individual objects by their stable values in cdPUT (I), PUTt (J), and CDt (K). For each of the good objects (red) and bad objects (blue), the discrimination index values were averaged across all negative neurons (left) and all positive neurons (right).
We then quantified the strength of value coding for each neuron (value modulation index; see Materials and Methods) and showed their distributions among all visual neurons in each region (Fig. 7C–E). In cdPUT (Fig. 7C), the value modulation index was mostly close to 0.5, indicating weak value coding in general. In PUTt (Fig. 7D) and CDt (Fig. 7E), the index was often away from 0.5, indicating that many neurons encoded stable values for objects. Indeed, neurons showing statistically significant value biases (black in Fig. 7C–E) were more common in PUTt and CDt than in cdPUT, as follows: cdPUT, 15%, 6 of 40 neurons; PUTt, 37%, 34 of 93 neurons; CDt, 29%, 18 of 62 neurons. Similar results were found in both monkeys: monkey W: cdPUT, 20%, 5 of 25 neurons; PUTt, 43%, 24 of 56 neurons; CDt, 30%, 14 of 48 neurons; monkey S: cdPUT, 7%, 1 of 15 neurons; PUTt, 26%, 10 of 38 neurons; CDt, 29%, 4 of 14 neurons.
One might still wonder whether these neurons were encoding object selectivity and not value selectivity. To examine this possibility, we estimated the distribution of the modulation index assuming that there is no correlation between neural activity and object value using a permutation test (1000 times for each neuron) in each region (see Materials and Methods). Note that the modeled data are based on object selectivity, not object values. In each case, we calculated the following two indices: mean−2SD and mean+2SD. The mean of each index is shown by a gray dashed line in Figure 7C–E. For the experimental data, the same indices (mean−2SD and mean+2SD) are shown by unfilled triangles in Figure 7C–E. These results indicate that the experimental data were more widely distributed than the modeled data (Fig. 7C–E; cdPUT: mean−2SD, p = 0.003, mean+2SD, p ≤ 0.001; PUTt: mean−2SD, p ≤ 0.001, mean+2SD, p ≤ 0.001; CDt: mean+2SD, p ≤ 0.001, one-sided permutation test), except for mean−2SD in CDt (p = 0.627). These data suggest that the whole population of neurons in each area encoded stable value information, especially in the positive direction.
Some neurons responded more strongly to bad objects, suggesting that there are two groups of neurons: positive value neurons and negative value neurons. However, neurons in both PUTt and CDt responded to good objects more strongly relative to the mean of estimation data as a population (Fig. 7D,E, gray solid line; p = 0.002 and p = 0.002; two-sided permutation test). Figure 7F–H show the population activity of the two groups (positive and negative) in each region, including cdPUT. These data provide another aspect of stable value coding: time course. The value coding (both positive and negative) started ∼100 ms after the appearance of the object in PUTt and CDt, but later ∼150 ms in cdPUT. Interestingly, the peak of value coding (both positive and negative) occurred earlier in CDt (∼120 ms) than in PUTt (∼150 ms).
So far, we have shown the following two features of neurons in the caudal part of the striatum (PUTt, CDt, cdPUT): (1) object selectivity (Fig. 2) and (2) stable value coding (Fig. 7). This raised a question: if individual neurons show strong object selectivities (especially in PUTt and CDt; Fig. 2), how can many objects be discriminated by their stable values? To address this question, we analyzed the combined responses of neurons to each object based on its value (Fig. 7I–K). The data suggest that all objects were discriminated in each area (especially in PUTt and CDt) by their stable values. It is likely that the strong object selectivity was created randomly across neurons, and therefore their combined responses encode stable values of all objects.
Discussion
Although many previous studies reported the neuronal activity in the putamen during motor or cognitive task (Kimura, 1990; Lee and Assad, 2003; Hori et al., 2009; Phillips and Everling, 2012), the caudal–ventral part of the putamen (PUTt) was not included in these studies. In this study, we found that PUTt as well as CDt contains many visual neurons. Their locations approximately correspond to the area that receives inputs from the temporal visual cortical areas (Yeterian and Van Hoesen, 1978; Van Hoesen et al., 1981; Saint-Cyr et al., 1990; Webster et al., 1995). These neurons, overall, were sensitive to fractal objects with complex visual features (e.g., color, shape, size). This may be based on a large number of inputs from many neurons in the temporal visual cortex to single putamen–caudate neurons. Notably, the properties of these visual neurons varied in the dorsal–ventral dimension. While the background firing rate was higher in the dorsal part (Fig. 1C), object selectivity was higher in the ventral part (Fig. 2). The greater object-selective area (i.e., ventral part), including putamen and caudate, would be appropriate for identifying particular objects, which would be useful for object perception and attention (Kourtzi and Connor, 2011). The smaller object-selective area (i.e., dorsal part), which we called cdPUT, would still be useful in motor actions, which may not require a clear recognition of individual objects. On the other hand, motor action requires spatial information, and indeed cdPUT neurons were as spatially selective as neurons in PUTt and CDt (Fig. 3C). In fact, some neurons in this part of the putamen are active when visual stimuli were presented near their tactile receptive field (Graziano and Gross, 1993).
Dorsal visual neurons (in cdPUT) were different from ventral visual neurons (in PUTt and CDt) in an additional aspect: many neurons in PUTt and CDt encoded stable reward values of individual objects, but most neurons in cdPUT did not (Fig. 7). This may be related to the difference in object selectivity (Fig. 2). If a neuron cannot visually discriminate objects, the neuron by itself would be unable to discriminate them by their values. This may be what occurs in cdPUT neurons. In contrast, neurons in PUTt and CDt are eligible to encode object values, because their object selectivity was generally high (Fig. 2).
Recent studies in our laboratory have shown that there are at least two types of object value coding (Hikosaka et al., 2014; Kim and Hikosaka, 2015): flexible value coding, based on short-term memory (Fig. 4); and stable value coding, based on long-term memory (Fig. 5). Flexible value coding is often important because the value of an object may change based on, for example, time, environment, and context, and the subject (or brain) needs to update the value quickly based on recent experiences (or short-term memories; Dempster, 1981; Gathercole, 1999). In this case, old (long-term) memories are useless. On the other hand, stable value coding is also important because there are so many objects in real life, many of which the subject experienced a long time ago but may appear unexpectedly. The subject (or brain) then needs to keep the value memories of all of these objects for a long time (Hikosaka et al., 2014). In this case, recent (short-term) memories are often disrupting.
Logically, short-term memory is created by removing long-term memory. Therefore, it is likely that short-term memory and long-term memory are created by different neural mechanisms (Jonides et al., 2008; Baddeley, 2010). In fact, these two memory functions are controlled by parallel circuits in the basal ganglia (Kim and Hikosaka, 2013; Yasuda and Hikosaka, 2015): CDh circuits for flexible value coding based on short-term memory; and CDt circuits for stable value coding based on long-term memory. These data suggest that both short-term and long-term memories are important and need to be processed separately.
In the current study, we found that a local region of the putamen (PUTt) is also involved in the object value coding. Neurons in PUTt encoded stable object values (Fig. 7), but not flexible object values (Fig. 4). Many neurons responded to good objects (i.e., objects previously, not currently, associated with a large reward) more strongly than to bad objects (i.e., objects previously associated with a small reward); some of them responded to bad objects more strongly than to good objects (Fig. 7D,G). Overall, PUTt neurons behaved similarly in response to CDt neurons (Fig. 7E,H).
Several studies in our laboratory have revealed the behavioral function of the neuronal circuits originating from CDt. CDt projects to two local regions in the basal ganglia (cdlSNr and cvGPe; Kim et al., 2017; Amita et al., 2018). The inhibitory connection to cdlSNr, which acts as the direct pathway, conveys the good object-dominant signal and inhibits cdlSNr neurons (Yasuda and Hikosaka, 2015), which leads to the disinhibition of neurons in the superior colliculus (SC). The final effect is to facilitate saccades to good objects. The inhibitory connection to cvGPe, which acts as the indirect pathway, conveys the bad object-dominant signal and inhibits cvGPe neurons (Kim et al., 2017). Since cvGPe neurons have inhibitory connections to cdlSNr neurons (Smith and Bolam, 1991; Bolam and Smith, 1992; Sato et al., 2000), the output of CDt then disinhibits cdlSNr neurons, which leads to enhanced inhibition of SC neurons (Kim et al., 2017). The final effect is to inhibit saccades to bad objects.
A more recent study suggests that PUTt shares the function as CDt (Amita et al., 2018). After the injection of a retrograde tracer (CTB488) into cdlSNr, retrogradely labeled neurons were massively but selectively located in both PUTt and CDt (Fig. 1B). Almost the same result was found after the tracer injection into cvGPe. These results suggest that PUTt shares the same direct and indirect pathways with CDt and therefore controls saccades based on long-term memories of object values.
The downstream circuits of PUTt would then facilitate saccades to good objects and inhibit saccades to bad objects. In fact, two monkey subjects in this study showed such value-biased behavior (Fig. 6). When multiple fractal objects (typically, n = 4) were presented simultaneously, the monkey looked at them with many saccades, even though reward was delivered randomly (free-viewing task; Fig. 6A). After experiencing the same objects with consistent rewards (large or small amount) repeatedly (Fig. 5B,C), the free viewing of the monkey became biased toward good objects (i.e., objects that had been associated with a large reward; Fig. 6). There were tendencies: the first saccade to a good object (Fig. 6C, right) and the longer duration of gaze on good objects (Fig. 6C, left). Importantly, such free-viewing biases occurred even when the monkey had not viewed the objects for a long time (49 d; Fig. 6B), although the monkey had been seeing many other objects during the retention period. Such a behavior may be related to automatic attention (Ghazizadeh et al., 2016b).
These neuronal and behavioral results suggest that PUTt, together with CDt, acquires the long-term memories of the values of individual objects and biases gaze/attention toward good objects (by its direct pathway) while avoiding bad objects (by its indirect pathway; Kim et al., 2017). The neuronal bias in the PUTt/CDt circuits is likely to cause the gaze/attention bias. Importantly, these neuronal/behavioral biases are based on the historical (old) experience of reward (not recent/predicted reward). Indeed, the inactivation of CDt impaired the behavioral bias based on the long-term memory, but not short-term memory (Kim and Hikosaka, 2013). Our preliminary experiment showed that the similar effect occurred by the inactivation of PUTt (Maeda and Hikosaka, 2018).
However, this hypothesis might not be complete. Saccade to and gaze on a particular object is important for subsequent behaviors, especially reaching a hand to the object (Prablanc et al., 1979; Biguer et al., 1982; Johansson et al., 2001; Mooshagian et al., 2014). In this sense, the selective visual signal would be useful for selective hand reach movement (Land et al., 1999) in addition to the gaze/attention bias. This raises the possibility that PUTt sends outputs to the globus pallidus internus, in addition to SC, thereby controlling hand movements as well as gaze/attention. This will be a new question for our research.
Footnotes
This research was supported by the Intramural Research Program at the National Institutes of Health, National Eye Institute. We thank Shinya Yamamoto for discussions; and D. Parker, V. McLean, M.K. Smith, G. Tansey, A.M. Nichols, T.W. Ruffner, J.W. McClurkin, and A.V. Hays for technical assistance.
The authors declare no competing financial interests.
- Correspondence should be addressed to Jun Kunimatsu at kunimatsu.jun{at}gmail.com
