Abstract
Dopamine neurons of the ventral midbrain have been found to signal a reward prediction error that can mediate positive reinforcement. Despite the demonstration of modest diversity at the cellular and molecular levels, there has been little analysis of response diversity in behaving animals. Here we examine response diversity in rhesus macaques to appetitive, aversive, and neutral stimuli having relative motivational values that were measured and controlled through a choice task. First, consistent with previous studies, we observed a continuum of response variability and an apparent absence of distinct clusters in scatter plots, suggesting a lack of statistically discrete subpopulations of neurons. Second, we found that a group of “sensitive” neurons tend to be more strongly suppressed by a variety of stimuli and to be more strongly activated by juice. Third, neurons in the “ventral tier” of substantia nigra were found to have greater suppression, and a subset of these had higher baseline firing rates and late “rebound” activation after suppression. These neurons could belong to a previously identified subgroup of dopamine neurons that express high levels of H-type cation channels but lack calbindin. Fourth, neurons further rostral exhibited greater suppression. Fifth, although we observed weak activation of some neurons by aversive stimuli, this was not associated with their aversiveness. In conclusion, we find a diversity of response properties, distributed along a continuum, within what may be a single functional population of neurons signaling reward prediction error.
Introduction
By numerous measures, midbrain dopamine neurons are relatively homogeneous, sharing much more in common than merely their use of dopamine as a neurotransmitter. Relative to the diversity of cell types observed within and across many brain regions, dopamine neurons appear generally similar to one another in gene expression, electrophysiological properties, and response profiles in behaving animals. Nonetheless, careful examination has revealed differences in each of these. Subgroups of dopamine neurons have been distinguished based on gene expression, electrophysiological properties, and afferent inputs (Haber et al., 1995; Neuhoff et al., 2002; Ford et al., 2006; Lammel et al., 2008; Margolis et al., 2008; Brown et al., 2009; Lammel et al., 2012).
Despite this diversity, the responses of dopamine neurons to reward stimuli in behaving animals appear relatively homogeneous, signaling a reward prediction error (RPE) in which reward events that are worse than expected suppress firing rate (e.g., Schultz, 1998; Fiorillo et al., 2003; Joshua et al., 2009). If all dopamine neurons signal this same sort of RPE, it is expected that they should be inhibited by aversive stimuli. However, although aversive stimuli have been found to suppress firing in many dopamine neurons, they activate at least some (Mirenowicz and Schultz, 1996; Guarraci and Kapp, 1999; Coizet et al., 2006; Joshua et al., 2008; Brischoux et al., 2009; Matsumoto and Hikosaka, 2009; Mileykovskiy and Morales, 2011; Wang and Tsien, 2011; Cohen et al., 2012). Matsumoto and Hikosaka (2009) have proposed two subclasses of dopamine neurons, distinguished by whether they are activated or suppressed by aversive stimuli.
The present study examines this issue and the diversity of responses in general. It is distinguished from previous studies by examining responses to two very different types of aversive stimuli (air puff vs oral saline or bitter solutions), as well as omission of expected juice. Of greater significance, we quantified the aversiveness of stimuli relative to the appetitiveness of juice, and we then compared neuronal responses to appetitive and aversive stimuli of similar absolute motivational value.
In an accompanying article, we characterized the multiphasic temporal dynamics of neuronal responses (Fiorillo et al., 2013). We demonstrated that activation at short latencies (<150 ms after stimulus onset) is related to the sensory intensity of stimuli, not their motivational value, demonstrating that activation by aversive stimuli is not necessarily related to their aversiveness. Although the majority of neurons displayed suppression of firing rate (at longer latencies), here we ask whether a minority of neurons might be activated by aversiveness, as proposed by Matsumoto and Hikosaka (2009). In addition, we search for subgroups of dopamine neurons and correlations in response properties across neurons. Although we find a single continuum of response variability, we also find evidence consistent with the distinction of dorsal and ventral tier dopamine neurons, which have previously been shown to differ in their expression of calbindin and H-type cation (HCN) channels (Haber et al., 1995; Neuhoff et al., 2002). We discuss the variability of neuronal responses in relation to models of how dopamine neurons may generate RPE.
Materials and Methods
Animals.
Two rhesus macaques (Macaca mulatta) were studied. Monkey O (10.5 kg) was female, and Monkey F (5.5 kg) was male. These same two monkeys contributed data on dopamine neurons to two other studies (Fiorillo, 2011; Fiorillo et al., 2013), and Monkey O contributed to a third (Fiorillo et al., 2008). Procedures complied with guidelines established by the National Institutes of Health and were overseen locally by the Stanford University Animal Care and Use Committee. Liquid intake was restricted to ensure motivation to participate in experiments.
Eye tracking and data acquisition.
A monkey's head was fixed in place in front of a computer monitor in a sound-insulated room. Eye position was monitored with an infrared eye tracking system (Eyelink II; SR Research), except for initial experiments performed in Monkey O, in which a scleral search coil was surgically implanted and eye position was monitored within a magnetic field. Expo software (written by Peter Lennie and modified by Julian Brown) was used to deliver stimuli and to collect all data.
Stimuli.
A detailed description of the stimuli is provided by Fiorillo et al. (2013). Air puff of 200 ms duration was delivered from a spout directed at the left nostril (parallel to the ground) from a distance of 1 cm. Juice, saline, and bitter solutions were delivered by gravity from reservoirs directly into the mouth. The standard volume of juice delivered during neuronal recordings was 130 μl, and it flowed for 150 ms. A larger volume of 240 μl was used in experiments with juice omission (Fiorillo, 2011). Saline and bitter solutions were 10–40 μl delivered over 30–80 ms. The durations of “unconditioned” stimuli were the same on each trial, and thus highly predictable, and neuronal responses appeared to be related exclusively to the time of stimulus onset, not offset. Conditioned stimuli were visual images of 4 degrees presented in the center of the monitor and were accompanied by a sound of 72 dB.
Choice tasks to measure aversiveness.
Choice tasks were used to quantify the aversiveness of stimuli, as described in detail by Fiorillo et al. (2013). The aversiveness of air, saline, bitter, and loud sound was estimated by measuring how much juice a monkey would sacrifice to avoid the stimulus. Several weeks of choice experiments were performed before each set of neuronal recordings, and a few days of choice behavior were performed during and after the period of neuronal recordings to ensure that aversiveness had not changed. In the initial choice experiments, aversiveness was measured repeatedly after adjusting stimulus intensity (air pressure and location, or saline or bitter concentration, or volume) to find an intensity that was equal and opposite in value to a small volume of juice (∼70 μl; 50 μl has been shown to be appetitive to monkeys and to modulate dopamine neurons) (Tobler et al., 2005). Once this target level of aversiveness was reached and proven to be stable over at least several days of choice behavior, neuronal recordings were performed and responses to that same stimulus intensity were measured. Neuronal recordings were not performed during the choice task. The aversiveness of the air puff tested during neuronal recordings was much greater than required to elicit conditioned eye blink (Fiorillo et al., 2013).
Experimental design used during neuronal recordings.
The data were collected in 4 sets of experiments in Monkey O followed by 1 set in Monkey F, each set lasting ∼2 months (Fiorillo et al. 2013). Experiments with bitter solution and loud sound were only performed in Monkey O, whereas other types of stimuli were tested in both monkeys.
Most of the data analyzed here were from recordings made while unconditioned stimuli were delivered in the absence of any conditioned stimulus (CS). A block of 30–75 trials consisted of just one stimulus (juice, air, saline, bitter solution, sound) delivered with a randomly distributed intertrial interval of 2–16 s. The delivery of just one stimulus in a block of trials would be expected to minimize “generalization” (Mirenowicz and Schultz, 1996) and therefore improve the ability of the neurons to discriminate stimulus identity at short latencies. However, the relatively long intertrial intervals make the stimuli less predictable, which is likely to reduce discrimination at short latencies and to promote short-latency activation.
Experiments were also performed with Pavlovian conditioned stimuli. Conditioning with visual or audiovisual conditioned stimuli was performed for several hundred trials over 2–4 d before the start of neuronal recordings. The unconditioned stimulus (US) onset occurred 1.0 s after CS onset, and CS offset was synchronous with US offset. Intertrial intervals were 2–6 s. Each CS was only followed by its US on a pseudo-randomly selected 50% of trials, although this fact is unlikely to be of any consequence to the issues discussed here. We also present data on neuronal responses to juice omission from a task in which a visual CS of 1.0 s in duration predicted that juice (240 μl) may or may not be delivered, and juice omission was signaled by offset of the visual stimulus (Fiorillo, 2011).
Recording and identification of dopamine neurons.
Glass-insulated tungsten electrodes (Alpha-Omega) were electrochemically plated with gold and then platinum, which reduced electrode impedances from ∼2 MΩ to ∼1 MΩ (Fiorillo, 2011). The preferred electrodes had exposed metal tips that were blunt, ∼5 μm in length and 15 μm in diameter.
Voltage recordings were bandpass-filtered between 0.2 and 5.0 kHz, which may have caused waveforms to appear to be of slightly shorter duration. Dopamine neurons were distinguished from other neurons in the region by the characteristics of their extracellularly recorded impulses, including long, multiphasic waveforms (see Fig. 1A) and low basal firing rates (0.1–15.0 Hz) (Schultz and Romo, 1987). Early work on neurons in primates having these properties demonstrated that, as expected of dopamine neurons, their firing could be suppressed by systemic administration of a D2 dopamine receptor agonist, and those in substantia nigra (SN) could be antidromically activated by electrical stimulation of striatum (Schultz and Romo, 1987). The use of optogenetic methods with electrical recordings in behaving mice has recently demonstrated that identified dopamine neurons have responses to rewards and aversive stimuli that are similar to those reported here and in previous studies of primates (Cohen et al., 2012). We also observed neurons with low average firing rates and moderately long-lasting waveforms, but with highly irregular interspike intervals: a mixture of very long (>2 s) and very short (<10 ms) intervals. These were not recorded. However, precise quantitative criteria were not applied in selecting neurons to record.
Estimating locations of neurons.
The locations of recorded neurons in the ventral midbrain were estimated with the aid of physiologically identified landmarks (particularly the somatosensory representation of the face in ventroposteromedial thalamus, and the oculomotor nucleus near the midline of ventral midbrain), and the location of recorded neurons was then estimated relative to the atlas of Paxinos et al. (2000) (see Fig. 1C), as described previously (Fiorillo, 2011). There are multiple sources of error in the estimates that may account for the appearance in Figure 1C that some neurons were recorded outside of dopaminergic nuclei.
Despite the sources of error in our estimates and the lack of distinct boundaries between anatomical divisions, we classified neurons as being in either ventral tegmental area (VTA) (A10) or retrorubral field (RRF) (A8) if they were more medial, and in SN (A9) if they were more lateral. Neurons at 4.5–6.0 mm rostral of the interaural line were classified as being in VTA or RRF if they were ≤4.0 mm from the midline (electrode penetrations were all made at increments of 0.5 mm in rostrocaudal and mediolateral dimensions). Neurons at 6.5 and 7.0 mm rostral were classified as VTA or RRF if they were not >3.5 or 3.0 mm, respectively, from the midline. Neurons observed at 7.5–10.0 mm rostral and ≤2.0 mm from the midline were classified as VTA. Cells meeting the above criteria were classified as being in RRF if they were ≤6.5 mm rostral of the interaural line, >2.0 mm dorsal (in the atlas coordinates of Fig. 1C), and >2.0 mm lateral of the midline. All neurons that were not classified as being in VTA or RRF were classified as being in SN. In following these criteria, each electrode penetration (51 total) was in VTA, RRF, or SN, and a single penetration never recorded neurons in more than one of these areas.
We wished to classify dopamine neurons as dorsal or ventral tier because this distinction is not only anatomical but also has a biochemical basis in primates (Haber et al., 1995). We classified all VTA and RRF neurons as “dorsal tier” (Lynd-Balta and Haber, 1994; Haber et al., 1995). Distinguishing dorsal and ventral tiers in the SN was more challenging. The somata of dopamine neurons in SN occupy a region of ∼1–3 mm in the dorsoventral (DV) dimension (at virtually all locations in the horizontal plane), with dorsal and ventral tiers being adjacent to one another. Each day the electrode penetrated vertically through the full DV extent of dopamine cell bodies, usually recording several neurons. The DV position of SN pars compacta (SNc) in atlas coordinates varies by up to ∼8 mm depending on position in the horizontal plane (see Fig. 1C), and we doubt our ability to estimate DV position precisely in atlas coordinates, especially across multiple electrode penetrations. However, we could estimate relative DV position of neurons recorded during a single penetration very precisely using the microdrive. We calculated DV position of each neuron relative to the point midway between the most dorsal and most ventral neuron (assigning the midpoint a relative DV position of “0”). For purposes of classifying SN neurons as dorsal or ventral tier, data from 1 neuron were discarded because it was the only dopamine neuron recorded for that electrode penetration. Among a total of 36 other days (electrode penetrations), 3 or more neurons were recorded each day, except for 6 d on which only 2 neurons were recorded per day. The DV range varied from a minimum of 0.34 mm to a maximum of 3.6 mm, with a mean of 1.6 mm. For all days on which the DV range was ≤2 mm, all neurons dorsal of the midpoint (midway between the most dorsal and most ventral neuron) were classified as being in the dorsal tier (our intention was not to have high confidence in the correct classification of each neuron, but rather to make use of as much information as possible for estimating the mean response in dorsal and ventral tiers). For days on which the DV range was >2 mm, all neurons within 1 mm of the most dorsal neuron were classified as dorsal tier, and all others as ventral tier. We applied this asymmetric rule to account for the observation that, whereas the DV thickness of the dorsal tier is relatively uniform across the entire horizontal plane, the ventral tier is of similar thickness (∼1 mm) to the dorsal tier in some locations in the horizontal plane but extends ventrally into the SN reticulata (SNr) at other locations (Fig. 1 of Lynd-Balta and Haber, 1994; Haber et al., 1995). Thus, for an electrode penetration in SN for which dopamine neurons were recorded over a relatively large range of 3 mm, we estimated that the more ventral 2 mm was in the ventral tier, with the most ventral neurons likely being in SNr. We further classified neurons as being in SNr if they met the criteria for being in SN, and they were >2.0 mm ventral of the most dorsal neuron; other SN neurons were classified as being in SNc. Thus, altogether we distinguished five classes of neurons according to their location, three classes in dorsal tier (VTA, RRF, and dorsal SNc) and two classes in ventral tier (ventral SNc and SNr). The number of neurons in each of the five categories for each monkey is shown in Figure 6.
Neuronal sampling bias.
Although an effort was made to record from a representative sample of dopamine neurons, at least two sources of potential bias should be noted. First, like most previous studies in primates, most recorded neurons were in SNc, and most likely project to dorsal and ventral striatum (Lynd-Balta and Haber, 1994). SNc has the highest density of dopamine neurons, and it is far enough lateral that vertical electrode penetrations do not run the risk of hitting the sagittal sinus and thereby causing hemorrhaging. In mice, it has been found that neurons projecting to cortical and limbic regions have smaller and slower action potentials than do neurons projecting to dorsal striatum and parts of ventral striatum (Lammel et al., 2008). It is therefore expected that these mesocorticolimbic neurons would have substantially smaller extracellularly recorded waveforms than “conventional” dopamine neurons (Fig. 4 of Lammel et al., 2008). The dopamine neurons recorded here and in previous work on primates tend to have small amplitude waveforms compared with background “noise” and to neighboring neurons that are judged not to be dopamine neurons, and particular electrode properties are needed to record from them as described above. The neurons recorded here presumably correspond to the “conventional” phenotype, and it seems likely that mesocorticolimbic neurons were underrepresented, or neglected entirely, in our recorded population of neurons. However, we did record from 26 neurons in VTA (13% of all recorded neurons). These neurons were within 2 mm from the midline and might therefore be expected to be mesocorticolimbic based on their location. Any distinctions in their waveforms or responses appeared to have been subtle (e.g., see Fig. 6).
Some evidence presented here suggests that neurons with lower baseline firing rates may tend to be generally less responsive. Although anecdotal, years of casual observation also suggest that neurons with very low firing rates (some as low as ≤0.1 Hz) tend to be less responsive and to have larger amplitude waveforms. Waveform amplitude is highly variable and depends on many factors, and we have not analyzed it here. We have recorded a moderate number of neurons with baseline rates of <1 Hz (see Fig. 5 of Fiorillo et al., 2013). Although an attempt was made to record from all dopamine neurons regardless of responsiveness, it is very difficult to collect enough data from neurons with very low firing rates, and such neurons may be underrepresented in our sample. It is likely that some neurons with very low rates were entirely undetected.
Data analyses.
Data were analyzed using Matlab. Firing rates were measured during the time periods indicated in Table 1. For all types of stimuli, the baseline firing rate was measured in the 1.0 s period before stimulus onset. To quantify the statistical significance of firing rates relative to baseline firing rate in a single neuron, firing rates were calculated for all individual trials and compared with baseline rates with an unpaired t test; p < 0.05 was taken to be significant, without any correction for the fact that the same test was performed separately on multiple neurons. For comparisons across a population of neurons, the mean firing rate across trials was calculated for each condition in each neuron, and these single neuron mean firing rates were then compared between conditions across the population of neurons using paired or unpaired t tests.
The time periods in which firing rates were measureda
The Matlab function “regstats” was used for simple and multiple linear regression analyses designed to characterize how neuronal responses varied with rostrocaudal and mediolateral position (see Fig. 8), and in an effort to distinguish the influence of DV position (see Fig. 7) from chronological order of recording. Independent variables were normalized to have equal ranges. Neuronal data were pooled across all electrode penetrations within one or both monkeys, except that analysis related to DV position and order of recording were restricted to neurons in SN.
Methodological comparison with Matsumoto and Hikosaka (2009).
Among studies in behaving animals, only Matsumoto and Hikosaka (2009) have argued strongly for more than one functionally distinct population of dopamine neurons. Although our results appear similar to theirs, we argue that the data may be explained by a single population signaling RPE. Here we compare our methodology to theirs. To summarize, we suggest that task design and data analysis may have been critical differences. It appears likely that our air puff may have been substantially more aversive than theirs. There is little reason to believe that we recorded from a different population of dopamine neurons.
Matsumoto and Hikosaka (2009) used conditioned eye blink as their measure of the aversiveness of air puff (as did Joshua et al., 2008), whereas we have used a choice task as our primary measure (Fiorillo et al., 2013). It is therefore not possible to compare the aversiveness of their air puff to the appetitiveness of their juice, an issue of substantial importance in the case that the two are present within the same context (because one will overshadow the other if the difference in value is large). It is also difficult to directly compare the aversiveness of air puff across studies. However, our air puff had an absolute value just slightly below that of juice, and it is unlikely that any studies in primates would use aversive stimuli with absolute values greater than that of reward stimuli (because doing so would likely interfere with the cooperation of the monkeys). Furthermore, we found that our air puff was at least an order of magnitude stronger than that needed to induce eye blink. Air puff of much greater intensity than required to evoke eye blink was neutral in our choice task (Fiorillo et al., 2013). Thus, our air puff was likely to have been much more aversive than that used by Matsumoto and Hikosaka (2009). Their air puff could have been essentially neutral, especially in the case that it was predicted by a CS and thus avoidable by eye blink—ours was directed at the nose and thus unavoidable. We found that neuronal responses to neutral loud sound were similar to air puff (Fiorillo et al., 2013). Further below it is suggested that, even if their air puff was aversive, if it had much lower absolute motivational value than juice, its low value relative to juice could undermine their key data suggesting neuronal activation by aversiveness.
It is possible but unlikely that Matsumoto and Hikosaka (2009) recorded from a different population of dopamine neurons. They reported that neurons activated by aversive stimuli were found predominately in the dorsolateral SN, but it appears that we recorded from many neurons in this region (see Fig. 1C). Furthermore, as discussed in Results and previously (Fiorillo et al., 2013), the depth of an electrode may be confounded with the time that the monkey has spent in the task each day because experimenters typically record neurons in sequence as they advance the electrode deeper each day. Our electrode penetrations were dorsal to ventral, whereas theirs were dorsolateral to ventromedial. Because the gradient of responses that they reported was in the same direction as their electrode penetrations, it could conceivably be related to order of recording rather than neuronal position.
In Results and as described previously (Fiorillo et al., 2013), we have suggested that the activation they saw to aversive stimuli was related to sensory intensity rather than aversiveness. Their key evidence that we have not explained previously is that a CS predicting air puff caused greater activation than a CS predicting no air puff. They compared responses to three conditioned stimuli that predicted probabilities of air puff of 0, 0.5, and 1. They began their analysis by selecting cells that displayed significant activations above baseline to the CS predicting certain air puff (“100% air puff CS”) and labeled these as “ACS-excited.” Because these neurons were selected for large activation to this CS, their subsequent finding that responses to this CS were larger than the other two conditioned stimuli could have been biased by the initial selection process. Likewise, their “ACS-inhibited” neurons were selected for significant suppression to “100% air puff CS” and that they were subsequently found to have greater suppression than other conditioned stimuli could be artifactual.
Perhaps a greater interpretational difficulty arises from the design of their task. Neurons were recorded during alternating blocks of ∼5 min each in which all stimuli were either appetitive or aversive, except one “neutral” CS−, which predicted no outcome. The interpretational challenge arises because an identical CS− was repeatedly presented in both contexts, and the claim that some neurons were preferentially activated by an aversive CS is based on comparison with the neutral CS−. The responses of dopamine neurons are highly sensitive to the reward prediction associated with a context, and it is the relative value of the stimuli that matters if they are presented in the same context (Tobler et al., 2005). In the context of the juice block, the CS− had the most negative value and was worse than expected, whereas in the air puff block its value should have been the most positive of any CS because it alone predicted absence of air puff. However, it is likely that the juice had a very high absolute motivational value relative to the value of air puff, which may have been almost negligible by comparison (and possibly not even aversive; see above). The relative value of the CS− would then be much worse than the other conditioned stimuli in the juice block and only slightly better than the other conditioned stimuli in the air puff block. However, that is assuming a “steady state” in which the two contexts are well known to the animal and do not influence one another. Because there were only 12 trials of each CS in each block, the relative value of the neutral CS− was frequently changing with the context of the block. Although it is likely that there was some change in subjective value of the CS− within a few trials, there is no way to know how it compared in value to the CS predicting air puff, even near the end of the 12 trials. It is entirely plausible that the CS− had substantially more negative value than the CS predicting air puff throughout most or all of the air puff block. In other words, omission of juice may have been much worse than delivery of air puff, and the association of the CS− with omission of juice may have overshadowed its association with omission of air puff. This could explain their key result that a CS predicting air puff causes greater activation of dopamine neurons than a neutral CS−. Our interpretation of their result is that both conditioned stimuli caused early excitation because of their sensory intensity (Fiorillo et al., 2013), but that the CS− had the more negative value and thus caused greater inhibition, which counteracted the early activation and caused greater subsequent suppression. It is worth noting in this regard that inhibition of dopamine neurons appears to be caused by stimuli predicting absence of reward, rather than by aversiveness (Fiorillo et al., 2013).
A reexamination of the data of Matsumoto and Hikosaka (2009) could help to distinguish among the two competing interpretations of their data. The three key points of the reanalysis would be to entirely exclude responses to the CS− that predicted no outcome, to measure firing rates at longer latencies so as to omit the initial, sensory related activation, and finally, to perform the same analysis on the entire population of neurons (rather than first categorizing neurons based on significance tests). We would predict that only an insignificant fraction of neurons are more strongly activated by the CS predicting air with high probability relative to the CS predicting air with intermediate probability.
Results
Choice behavior to establish motivational value of aversive stimuli
Before neuronal recordings in two rhesus macaques, behavioral experiments were performed to choose an appropriate intensity for aversive stimuli (air puff, saline, and bitter solutions) (Fig. 1 of Fiorillo et al. (2013)). We quantified the aversiveness of stimuli through a choice task in which we measured how much juice the monkeys would sacrifice to avoid the aversive stimuli. The intensity of the stimuli was then adjusted until average aversiveness fell within a range that was equal and opposite to 70–110 μl of juice (except some experiments with a higher concentration of bitter solution that had an estimated value of 200 μl). The same stimulus intensity was then maintained during neuronal recordings; thus, the aversiveness of our stimuli was comparable to the appetitiveness of our standard juice reward (130 μl). Primate dopamine neurons show robust activations and suppressions of firing rate in response to juice volumes as low as 50 μl, depending on whether the reward value is more or less than expected (Tobler et al., 2005), making it very likely that the aversiveness of our stimuli should be high enough to engage dopamine neurons.
As described in Materials and Methods, our stimuli were much more aversive than necessary to evoke avoidance responses (such as eye blink), which were the measure of aversiveness in some previous studies (Joshua et al., 2008; Matsumoto and Hikosaka, 2009). We found that air puff near the eye, despite being 10-fold more intense than that required to elicit eye blink, was motivationally neutral in our choice task (Fiorillo et al., 2013). We also studied neuronal responses to high and moderate intensity sounds (90 and 72 dB) in Monkey O but not Monkey F. The high-intensity sound was found to be neutral in the choice task (Fiorillo et al., 2013).
Multiphasic responses to appetitive and aversive stimuli
Extracellularly recorded waveforms of neurons in the ventral midbrain were judged to be from dopamine neurons based on their long durations (Fig. 1A,B) as well as their low and moderately regular rates. These were the only criteria used in the present study to identify putative dopamine neurons. Neither responsiveness to reward nor any other stimulus was used as a criterion. Figure 1C shows the estimated position of each neuron relative to an atlas (Paxinos et al., 2000). The present analysis is based on recordings from 195 neurons (134 in Monkey O, 61 in Monkey F) recorded in SN (A9), VTA (A10), and the retrorubral field (A8) (Fig. 1C). See Materials and Methods for further description of the localization of these regions and the potential for bias in our sampling of dopamine neurons. Although our ability to definitively categorize neurons into these three groups was limited, we estimated that 81%, 13%, and 6% of 195 neurons were in SN, VTA, and RRF, respectively (see Materials and Methods). We further categorized neurons into dorsal and ventral tiers, with ventral tier corresponding to dopamine neurons in the more ventral part of SN and dorsal tier corresponding to all others (Lynd-Balta and Haber, 1994; Haber et al., 1995). The waveforms of ventral tier neurons (45% of all recorded neurons) appeared to be of slightly longer duration on average (2.19 ± 0.04 vs 2.05 ± 0.04 ms; p = 0.01, unpaired t test), although with substantial overlap between the two populations (Fig. 1B). Because we found only modest dependence of responses on neuronal location, we analyze responses across all recorded neurons before considering the influence of location further below.
Localization and identification of dopamine neurons. A, Dopamine neurons were identified by their broad waveforms. The thick black waveform represents the average of all recorded neurons. Two waveforms from individual neurons are shown to represent extreme examples of narrow (blue) and broad (red) waveforms. To aid comparison, all waveforms have been positioned and scaled according to the time and amplitude of the largest negative peak. B, A scatter plot in which each point represents the duration (time of peak 3 minus peak 1) and the ratio of the amplitude from baseline of peak 2 to peak 1. Black circles, Dorsal tier neurons; red triangles, ventral tier neurons. Data points for the two neurons for which raw waveforms are shown in A are marked with an “X.” C, Neuronal locations (black, Monkey O; red, Monkey F) in the midbrain were estimated using physiologically identified landmarks and then translated onto drawings from an atlas (Paxinos et al., 2000), as previously described (Fiorillo, 2011). The first number in each panel indicates the coronal section in the atlas (distance rostral of the interaural line), and the numbers in parentheses indicate the range of neuronal locations shown. There were several sources of error in the estimation of cell location and its translation to the atlas, which probably accounts for the appearance that neurons were recorded in nuclei that are not thought to contain dopamine neurons.
We recorded responses to appetitive juice, juice omission, aversive saline and bitter solutions, aversive air puff, neutral loud sound, and conditioned stimuli predicting these. Neuronal responses to saline and bitter solutions were virtually identical, and thus we group these together as “saline-bitter” (Fiorillo et al., 2013). Stimuli were found to cause multiphasic responses (Fig. 2A), the temporal aspects of which were analyzed in detail in the accompanying article (Fiorillo et al., 2013). The population average response to aversive stimuli consisted of activation followed by suppression followed by activation. Firing rates were analyzed in three periods denoted as “early,” “middle,” and “late,” which were chosen to capture the average peaks of these changes in firing rate (the precise measurement periods varied somewhat depending on the stimulus; Table 1 and Fig. 2A). We summarize the responses of all neurons with scatter plots. Whereas the accompanying article shows firing rates of each neuron during each period in relation to its baseline firing rate (Fig. 5 of Fiorillo et al., 2013), Figure 2B–F provides additional information about how responses evolved over time in single neurons by showing the rates of each neuron in both the early and middle periods (with baseline rate subtracted).
Summary of multiphasic neuronal responses. A, Population average responses (PSTHs) to delivery of appetitive juice (black), omission of expected juice (magenta), aversive air puff (blue), aversive saline or bitter solutions (red), and neutral loud sound (90 dB) (green). Firing rates were measured using a moving average with 50 ms bins and 1 ms increments. The horizontal bars indicate the time periods during which firing rates were measured to construct panels B–F (see Table 1). This summary panel shows the same data as Figure 4E of Fiorillo et al. (2013). B, Firing rates measured in the early and middle periods after delivery of juice. C, Saline or bitter solution. D, Air to the nose. E, Loud sound. F, Omission of expected juice delivery (the event at time 0 was offset of a visual CS). In all cases, data points representing single cells are colored according to the results of significance tests: green squares, activation or suppression in the middle period; blue triangles, in the early period; red diamonds, both; black circles, neither.
The typical neuron displayed activation in the early period but suppression in the middle period in response to all stimuli (Fig. 2), the only exception being juice omission, for which early suppression was somewhat more common than early activation; unlike all other stimuli, it was signaled by the offset of a visual stimulus rather than onset of an auditory stimulus. The activation occurring before ∼150 ms is primarily dependent on the sensory intensity of the stimulus (Fiorillo et al., 2013). Sensory intensity was high for aversive air and neutral loud sound (82 and 90 dB, respectively, with air also causing strong tactile stimulation of the nose), each of which evoked a very brief and short latency activation (Fig. 2A,D,E). Saline, bitter, and juice had moderate and nearly identical sensory intensities (72 dB) that caused longer latency activations (Fig. 2A). We previously concluded that reward value is best discriminated between ∼150 and 250 ms under our test conditions, a time during which excitation and inhibition are probably both strong and substantially cancel one another (Fiorillo et al., 2013). Excitation predominates in the case of juice, whereas inhibition predominates in the case of aversive stimuli.
Lack of evidence for discrete subpopulations
A simple means to examine the diversity of neuronal responses is to make scatter plots in which each point represents the firing rate (or other property) of a single neuron. Casual inspection of all 2D scatter plots that we have examined has revealed a single “cloud” in which there is substantial variability across neurons, but a lack of clear and distinct clusters (Figs. 1B and 2A; Fig. 4). With respect to firing rates, the lack of clusters is particularly clear in plots of raw firing rates, such as “phasic” firing rates versus “tonic” baseline firing rates [Fig. 5 of Fiorillo et al. (2013)]. Certain types of scatter plots, such as those with subtracted firing rates in neighboring time periods (e.g., Fig. 2), can give rise to apparent outliers and the appearance of clusters that may not be meaningful. A lack of clusters was also found in previous studies in behaving monkeys (Fiorillo et al., 2003; Joshua et al., 2008; Matsumoto and Hikosaka, 2009). One implication of the continuum of response variability is that the classification of neurons based on statistical tests can be misleading because these impose arbitrary thresholds to force neurons into categories for which there is no statistical evidence.
Even in the absence of discrete clusters of neuronal firing rates, it would be natural to consider there to be two types of neurons if their responses are qualitatively distinct, as in the case of increases versus decreases in firing rate to the same stimulus attribute. Matsumoto and Hikosaka (2009) proposed two types of dopamine neurons distinguished by opposite responses to the aversiveness of air puff. However, we have shown that aversive air puff causes activation as a result of its high sensory intensity but suppression as a result of its association with absence of reward, even in the same neurons (Fiorillo et al., 2013). Although the suppression occurs later than the activation, the two effects overlap in time. It is expected that the firing rate depends on the amplitude and duration of both excitation and inhibition and that these vary slightly from neuron to neuron. It is therefore not surprising that averaging over a relatively long period of time identifies some neurons that show a net activation (resulting from sensory intensity) and others with net suppression (resulting from aversiveness), as found by Matsumoto and Hikosaka (2009). As described further below, we search for, but fail to find, evidence of a subgroup of neurons that are activated by aversiveness.
Discrete clusters could emerge when considering neuronal responses in relation to anatomical location. However, we have also failed to observe clusters of neurons in 2D scatter plots of firing rates versus neuronal position, whether neuronal position was plotted in rostral-caudal, medial-lateral, or DV dimensions (data not shown). Further below we do note correlations between neuronal responses and positions. We have not conducted multidimensional cluster analyses, which could potentially identify discrete clusters in our data, even in the apparent absence of clusters in two dimensions.
Suppression to aversive stimuli is correlated across neurons
Dopamine neurons as a group are known to be modulated by stimuli of multiple sensory modalities, as well as by stimuli associated with a variety of different types of appetitive and aversive outcomes (e.g., food, water). Although the entire population displays sensitivity to a broad spectrum of stimuli, there could be discrete subgroups of dopamine neurons (or a continuum) that respond preferentially to certain types of stimuli (for example, certain sensory modalities). Alternatively, all dopamine neurons could be “broadly tuned,” showing no discrimination across stimulus types but being sensitive (to varying degrees) only to overall appetitiveness or aversiveness. If they are broadly tuned, then individual neurons that are activated or suppressed by one type of appetitive or aversive stimulus ought to show similar responses to another stimulus if its motivational value is similar.
It has previously been observed that those dopamine neurons that are more strongly activated by appetitive juice (onset of which is auditory and tactile) tend to also be more strongly activated by visual stimuli that predict juice (for example, see supplemental Fig. 4 of Fiorillo et al., 2003, and supplemental Fig. 13 of Matsumoto and Hikosaka, 2009). The same phenomenon was also observed in the present study (data not shown). We asked the related question of whether neurons that display suppression of firing rate to one stimulus are similarly suppressed by others. Approximately one-third to one-half of neurons were significantly inhibited by saline-bitter, air, and juice omission (Fig. 2). Of 82 neurons tested with all three stimuli, the firing rates of 17% (14) were suppressed by all three, greater than the 7% expected if suppression were independently distributed across the population (42% × 34% × 51%, based on significant suppression by saline-bitter, air, and juice omission, respectively).
To further examine whether suppression to one stimulus was associated with suppression to another, we bisected the entire population of neurons based on the amplitude of suppression, which we measured as firing rate in the middle period divided by baseline firing rate (rather than subtracting baseline rates, to avoid indirectly selecting neurons with higher baseline rates that are capable of having greater decreases in firing rate). Given three types of stimuli (air, saline-bitter, and juice omission) and two ways of sorting neurons for each pair (e.g., response to air sorted by response to juice omission, and vice versa), we made six plots of peristimulus time histograms (PSTHs) (Fig. 3). In all cases, stronger suppression to one stimulus was clearly associated with stronger suppression to the other (Fig. 3), a phenomenon also observed when data from each monkey were analyzed separately (data not shown). The strongest relationship was between air and saline-bitter, whereas the relationship between juice omission and saline-bitter appeared weaker (Fig. 3, compare A,C with B,E).
Suppression of firing rate to multiple types of stimuli is correlated across neurons. For all neurons in which responses to two or more aversive stimuli were recorded, the population was bisected depending on the amplitude of suppression in response to one type of stimulus, and then PSTHs were constructed for each subgroup in response to a second type of stimulus. Black represents the subgroup of neurons with stronger suppression to the stimulus that was used for sorting; and gray represents the subgroup with weak or no suppression. For sorting, firing rates were measured in the middle period (Fig. 2A; Table 1) and divided by baseline firing rates. A, Population PSTHs of responses to saline or bitter solutions sorted by amplitude of suppression to air puff. Bin size, 20 ms. B, Responses to saline or bitter sorted by amplitude of suppression to juice omission. C, Responses to air sorted by amplitude of suppression to saline or bitter. D, Responses to air sorted by amplitude of suppression to juice omission. E, Responses to juice omission sorted by amplitude of suppression to saline or bitter. F, Responses to juice omission sorted by amplitude of suppression to air.
These observations were further supported through correlation analyses in which changes in firing rates from baseline were measured during the middle period and plotted against one another for each of the three pairs of stimulus conditions (data not shown, but analogous to scatter plots of Fig. 4). All correlation coefficients were highly significant (p < 0.0004 in each case, air vs saline-bitter, r = 0.63, n = 83; air vs juice omission, r = 0.6, n = 82; saline-bitter vs juice omission, r = 0.38, n = 83), although the correlation between saline-bitter and juice omission was significantly weaker than the others (p = 0.03 based on Fisher r to z transformation), as inferred based on the PSTHs (Fig. 3). Significant but weaker correlations were observed when firing rates were divided by baseline rates (rather than subtracting baseline rates; division produced more outliers).
Neurons with greater suppression have greater activation to reward. A, Left, PSTHs (mean ± SEM) during juice delivery after bisecting the population of neurons depending on the amplitude of suppression to air (firing rate in middle period divided by baseline firing rate). Neurons with stronger suppression to air (or saline-bitter, as shown in B) also have stronger suppression to juice (at ∼300 ms). Bin size, 20 ms. Right, scatter plots of firing rates (with baseline rates subtracted) after juice (y-axis) and air (x-axis). Data points representing single cells are colored according to the results of significance tests: green squares, activation or suppression after aversive stimuli only; blue triangles, juice only; red diamonds, both; black circles, neither. Inset, Numbers indicate the number of neurons (n), Pearson's correlation coefficient (r), and its significance (p). B, Analogous to A, but showing the relationship between suppression to saline-bitter and activation to juice. C, Analogous to A, but showing the relationship between suppression to juice omission and activation to juice.
The amplitude of suppression could be correlated with baseline firing rate, which would complicate the interpretation of the analysis above. As expected, cells with higher baseline firing rates did tend to have greater suppression when suppression was measured as a difference in firing rate from baseline (Fig. 5). However, we found no evidence for any correlation with baseline firing rates when suppression was measured as a ratio of baseline firing rate (r values ranged from −0.14 to +0.04, with p > 0.20 and n > 85 in all 3 cases). Because cells with very low rates could contribute outliers that might result in an artifactual bias toward a negative correlation, we repeated the analysis after excluding all cells with baseline rates of <2 Hz, but the result was nearly identical (r values of −0.13 to 0.08, p > 0.33 and n > 55 in all 3 cases). By contrast, we did observe a modest but significant positive correlation between baseline firing rates and the increase in firing rate caused by juice during the early period (r = 0.20, p = 0.005, n = 194) (changes measured by subtracting baseline rates) (Fig. 5A of Fiorillo et al., 2013).
Neurons with late “rebound” activation have higher baseline firing rates. A, Left, PSTHs showing responses to air (mean ± SEM) in neurons without significant late activation (black) and with both significant late activation and significant suppression during the middle period (red). The middle and late periods in which rates were measured are shaded. Bin size, 50 ms. Right, scatter plot of firing rates in all neurons during the late period (y-axis) and during the prestimulus baseline (1.0 s period before stimulus onset) (x-axis). Red triangles represent firing rates that were significantly different from baseline (p < 0.05). The blue line represents a linear fit of the data. Inset, Number of neurons (n), Pearson's correlation coefficient (r), and its significance (p). B, Analogous to A, but for saline-bitter rather than air.
Suppression to aversive stimuli is correlated with activation to juice
We also examined the relationship between suppression and activation across neurons. Juice appeared to evoke both excitation and inhibition, which substantially overlap in time and probably counteract one another (Fiorillo et al., 2013). It might therefore be expected that neurons with strong suppression may also have less net activation to juice delivery. However, both theory and experimental evidence (primarily from neurons other than dopamine neurons) suggest that inhibition may predict and counteract excitation, the two tending to be balanced and thereby generating prediction error (Fiorillo et al. (2013)). According to this view, neurons with strong activation may tend to have strong suppression as well. In support of this view, we found that neurons with stronger suppression to juice (in the middle period) also had stronger activation (in the early period) (Fiorillo et al., 2013). Here we extend this observation to aversive stimuli.
When we bisected the population of neurons based on the amplitude of suppression during the middle period, we found that neurons with stronger suppression to aversive stimuli also had stronger activation to juice (Fig. 4). Scatter plots revealed a strong correlation of activation to juice with suppression to air, saline-bitter, and juice omission (Fig. 4) (p < 0.0001 in all 3 cases; Fig. 4 legend). The same relationship was seen when data from each monkey were analyzed separately (data not shown).
The time course of the augmented response in cells with strong suppression (Fig. 4, left panels) provides clues concerning the nature of the inhibition and suggests subtle differences between aversive stimuli and juice omission. In the case that responses to juice were sorted based on suppression to air or saline-bitter, the augmented activation was in the latter part of the response, not during the rising phase (Fig. 4A,B). In addition, the augmented activation was followed by augmented suppression at ∼300 ms. By contrast, when cells were sorted based on suppression to juice omission, there was no augmentation in suppression to juice (Fig. 4C). Whereas onset of appetitive and aversive stimuli is presumably accompanied by a phasic inhibition, the suppression that is seen in response to juice omission may result, at least in part, from a relatively tonic inhibition that becomes strong during the delay period after a CS (Cohen et al., 2012) (and which may cause suppression as a result of a loss of excitation when juice is omitted at the time of CS offset). Cells with stronger phasic inhibition to air and saline-bitter may also have stronger phasic inhibition to juice, resulting in a small “rebound” suppression (Fig. 4A,B). By contrast, selecting cells with suppression to juice omission may identify cells with stronger tonic suppression, and tonic suppression may not substantially alter the time course of the response to juice or contribute to the rebound suppression (Fig. 4C).
Late activation is correlated with higher baseline firing rates
A late activation was observed between ∼400 and 600 ms after aversive stimuli, neutral loud sound, and juice omission in ∼15–30% of neurons (Fig. 5), as previously described (Fiorillo et al., 2013). The late activation usually occurred only after a period of suppression and could therefore be a “rebound” that is causally dependent on preceding suppression. Neurons with late activation appear to constitute a subgroup of neurons with inhibitory responses because other neurons with suppression returned to baseline during the late period or continued to display suppression (Fig. 5). Among 32 and 54 neurons tested with both air and saline-bitter, changes in firing rate during the late period were positively correlated across stimulus types (r > 0.44 and p < 0.001 in each monkey), suggesting that the same neurons displayed late activation to both types of stimuli.
It is known that many dopamine neurons express high levels of HCN channels, which are activated by hyperpolarization and can cause a subsequent depolarization. We therefore suspected that the late rebound activation may be caused by activation of H-type channels. In brain slices of mouse SN, dopamine neurons that do not express calbindin have larger H-type currents and higher baseline firing rates relative to those that do express calbindin (Neuhoff et al., 2002). Likewise, SN dopamine neurons lacking calbindin also have higher baseline firing rates in anesthetized rats (Brown et al., 2009). The higher expression of HCN channels contributes to the higher baseline firing rates in SN calbindin-negative neurons (Neuhoff et al., 2002). Indeed, we found that neurons with late activation had elevated baseline firing rates (∼3–5 Hz) compared with neurons that did not display late activation (Fig. 5) (p < 0.05 in each monkey after pooling across stimuli, and for each stimulus after pooling across both monkeys; unpaired t tests). In a minority of these neurons, there was no clear suppression during the middle period; thus, we could not be sure whether the late activation could have been a “rebound.” We therefore repeated the analysis by selecting only those cells with significant late activation and either firing rates below baseline during the middle period, or significantly below baseline in the middle period (in the latter case, n = 12 for air puff, n = 11 for saline-bitter). In each case, we found the same result: that neurons with rebound activation had significantly higher baseline firing rates.
Because neurons with late rebound activation represent a subgroup of those with suppression, it could be that neurons with suppression have higher baseline firing rates. However, as noted above, we found no relationship between suppression amplitude and baseline firing rate (in the case that suppression is measured as a percentage of baseline rate, rather than though subtraction of baseline rate or through tests of statistical significance because the latter methods naturally associate higher baseline rates with greater potential suppression). Furthermore, when we examined only the subset of neurons with significant suppression in the middle period, we also observed that those with significant late activation had higher baseline firing rates than those without late activation.
Like the larger group of neurons with suppression during the middle period (Fig. 4) (of which this subgroup is a part), neurons with rebound activation also had larger increases in firing rate after juice (data not shown).
Greater suppression and late activation in ventral tier neurons
We categorized all neurons as belonging to one of five groups based on the estimates of neuronal locations shown in Figure 1: VTA, RRF, dorsal SNc, ventral SNc, and SNr. It should be noted that the classification was prone to error because of uncertainty in estimating neuronal location together with a lack of precise and discrete boundaries between these regions. Differences were observed between regions in neuronal activation to juice, but the differences did not appear to be consistent between monkeys (Fig. 6). Activation in VTA was less than SN in Monkey O, but such a difference was less clear in Monkey F.
Activation to juice in five anatomically defined groups of dopamine neurons. A, Monkey O, with the number of neurons in each group. Bin size, 20 ms. B, Analogous to A, but showing data from Monkey F.
Functional diversity of dopamine neurons may be closely related to distinct neurochemical phenotypes, and these phenotypes do not necessarily have simple relationships to spatial location or standard anatomical categories (e.g., Haber et al., 1995; Lammel et al., 2008). Of particular interest here is the distinction between dopamine neurons that express calbindin and have low levels of HCN channels and those that do not express calbindin and have high levels of HCN channels (Neuhoff et al., 2002). In primates, calbindin-positive dopamine neurons occupy the “dorsal tier” of SNc, as well as all of VTA and RRF, and they project to ventral striatum as well as many limbic and cortical regions, whereas the “ventral tier” neurons are in SN only, they are calbindin-negative, and they project to all of striatum, except the shell of the nucleus accumbens (Lynd-Balta and Haber, 1994; Haber et al., 1995). According to the hypothesis described in the preceding section of Results, the neurons we found with high baseline firing rates, strong suppression, and late activation (Fig. 5) may be calbindin-negative and have high expression of HCN channels (based on work in rodents; e.g., Neuhoff et al., 2002). We therefore divided neurons into dorsal tier (VTA, RRF, and SNc dorsal) and ventral tier (SNc ventral, SNr). Consistent with the hypothesis, suppression and late activation were stronger in ventral tier neurons relative to dorsal tier. This was observed in both monkeys for responses to air, saline-bitter, and juice omission, although differences in suppression were small in Monkey O (Fig. 7). There was a slight and insignificant trend toward higher baseline firing rates in ventral tier neurons. No difference was observed in the early, sensory-related activation to air (Fig. 7B).
Greater suppression and late activation to aversive stimuli in ventral tier neurons. Neurons in VTA, RRF, and dorsal SNc were classified as dorsal tier, and neurons of ventral SNc and SNr were classified as ventral tier (see Fig. 6 for numbers of neurons in each of the 5 groups). Left, Monkey O; Right, Monkey F. Gray represents dorsal tier; black, ventral tier. A, Response of neurons to saline or bitter solution. Because of the confounding effects of a shift in motivational value of saline and a low concentration of bitter in Monkey O over the course of the day (see Results and Fiorillo et al., 2013), here we show only data obtained with a high concentration of bitter in Monkey O. B, Response of neurons to air. C, Response of neurons to juice omission (signaled by visual CS offset at time “0”).
Our efforts to distinguish the responses of dorsal and ventral tier neurons could be confounded with changes that might occur over the course of each day's recording. Estimates of relative DV position were based on recordings of two or more neurons during single electrode penetrations (see Materials and Methods). The electrode was lowered vertically from dorsal to ventral; and for most but not all penetrations, dorsal neurons were recorded before more ventral neurons. Thus, any observed differences between dorsal and ventral neurons could instead be the result of the order of recording (which could be correlated with factors such as satiation, or predictability of stimuli). Saline and a low concentration of bitter were found to become more aversive over the course of the day in Monkey O (and were sometimes appetitive at the start of the day); likewise, early activation to saline diminished as a nearly linear function of the order of neurons recorded (Fiorillo et al., 2013). The decline in early activation to these stimuli appeared to have been better explained by order of recording than by DV position (Fiorillo et al., 2013). The most likely explanation would seem to be that the aversiveness of salt was offset and sometimes exceeded by the appetitive value of the water in which it was dissolved, and that as water was devalued with increasing satiation each day, saline shifted toward increased aversiveness. However, even after excluding these data and considering only a high concentration of bitter solution in Monkey O, ventral neurons still appeared to have greater suppression and late activation (Fig. 7A, left). The same was true for saline in Monkey F (Fig. 7A, right), in which no shift in value was apparent (Fiorillo et al., 2013). In the case of air puff, we have no particular reason to expect a shift in value, nor was a shift observed in either monkey (Fiorillo et al., 2013); yet we also observed greater suppression and late activation in ventral tier neurons (Fig. 7B). The same pattern was observed with juice omission (Fig. 7C), despite the fact that any devaluation of juice could only be expected to cause smaller suppression late in the day.
As one means to distinguish the potential contributions of order of recording from DV position, we identified 17 SN neurons recorded over 6 d for which the first neuron recorded was the farthest ventral (opposite of the typical case) (2 and 4 d in Monkeys O and F, respectively). Data for aversive stimuli were not available for these neurons in Monkey O. Among 10 neurons from Monkey F recorded with air, saline, and juice omission, there was significantly greater suppression in ventral neurons, or a trend in that direction, in all three cases (p = 0.006, p = 0.11, and p = 0.10, respectively; unpaired t tests: n = 5 dorsal, n = 3 or 4 ventral).
As a further means to distinguish their contributions, we performed simple regression analyses on responses of all neurons in SN, with either DV position or chronological order of recording as the independent variable (we excluded neurons in VTA and RRF because these should all be “dorsal tier” and thus DV position in those areas would not be an important factor according to our hypothesis). Both factors appeared able to explain late activation to aversive stimuli in Monkey O, and both factors showed a similar but insignificant tendency in the case of suppression. Given the high degree of positive correlation between DV position and order of recording in Monkey O, we were not able to distinguish the contribution of each factor. However, there were a greater number of electrode penetrations in Monkey F for which DV position had the inverse of the usual relation to order of recording, as described above. In the case that responses in Monkey F to the three types of aversive stimuli were pooled, DV position had a significant effect on both suppression (β = −1.8, p = 0.01) and late activation (β = 1.2, p = 0.02). By contrast, chronological order of recording did not have a significant influence on suppression (β = 0.3, p = 0.71) or late activation (β = −0.6, p = 0.23), and the tendency in each case was the opposite of that needed to explain the greater responses in ventral neurons. Analogous results were obtained when DV position and order of recording were included as two independent variables in a multiple regression analysis. These results in Monkey F, like those in the preceding paragraph, suggest that ventral neurons had greater suppression and late activation and that this effect was not explained by chronological order of recording.
Regression analyses indicated that either DV position or order of recording could explain the larger activation to juice in dorsal SN neurons in Monkey O (Fig. 6A). For two electrode penetrations that recorded the most ventral neuron first, ventral tier neurons displayed significantly larger activations to juice in Monkey O (p = 0.047, n = 5 dorsal, n = 2 ventral, unpaired t test). The same trend was present in Monkey F (n = 5 dorsal, n = 4 ventral); thus, the effect was significant after pooling neurons across both monkeys (p = 0.028). Because this is the opposite of the result observed across all neurons in Monkey O (Fig. 6A), it suggests that order of recording could have causally influenced neuronal activation in Monkey O, making it difficult for us to distinguish whether DV position was also a factor that influenced activation to reward. In regression analyses in Monkey F, neither factor had a significant influence. However, if there were any influence of DV position in Monkey F, it appeared that neurons in ventral SNc may have had larger activations than those in dorsal SNc or SNr, an inverted “V” function of DV position that would not be identified through a linear regression analysis (Fig. 6B).
Greater suppression in more rostral neurons
Whereas distinctions have often been made between dorsal and ventral tiers of dopamine neurons, and between the more medial VTA and more lateral SN, we are unaware of previous evidence for differences in dopamine neurons along the rostrocaudal axis. We sorted neurons into rostral and caudal groups by choosing a midpoint that gave us approximately equal numbers of neurons in each group. We found little evidence for differences in baseline firing rates or in responses to juice, although activation and subsequent suppression to juice both may have been greater in rostral neurons in Monkey F (Fig. 8A). However, suppression to air and saline-bitter were greater in the rostral group in both monkeys, and late activation also appeared greater in rostral neurons in Monkey O (Fig. 8C,D). Differences were not readily apparent in suppression to juice omission, although rostral neurons may have had greater suppression in Monkey F (Fig. 8B).
Comparison of neuronal responses in rostral versus caudal neurons To create two groups of approximately equal numbers of neurons, the population of neurons was bisected into those rostral or caudal of 8.25 mm rostral of the interaural line in each monkey. A, PSTHs showing responses to juice (20 ms bins); Insets, baseline firing rates (mean ± SEM) in rostral and caudal neurons during blocks of juice delivery. B, Juice omission (50 ms bins). C, Saline or bitter. D, Air.
To further examine the dependence of responses on location in the horizontal plane, we performed multiple regression analyses with rostrocaudal and mediolateral position (absolute distance from the midline) as two independent variables. Separate analyses were performed in each monkey for baseline firing rates, early activation to juice, and suppression in the middle period after juice omission, saline-bitter, and air. For baseline firing rates and activation to juice, no significant relations were found, although there was a trend toward higher baseline firing rates in more caudal neurons in each monkey (p = 0.09 for both monkeys combined, for baseline rates during blocks of juice delivery). In Monkey F, but not O, there was a trend toward greater activation to juice in more rostral neurons (p = 0.08).
Rostrocaudal but not mediolateral position appeared to consistently have an influence on suppression, as suggested by PSTHs (Fig. 8C,D). Of 6 regression analyses on suppression in the middle period (saline-bitter, air, and juice omission in each monkey), all six had negative coefficients indicative of more suppression in rostral neurons, and in three cases (air in Monkey F and saline-bitter in each monkey), the effect of rostrocaudal position was significant (p < 0.05). Significant effects of rostrocaudal position were found for both air and saline-bitter when neurons from both monkeys were included in a single analysis, or in each monkey when data from both types of stimuli were pooled together. The same trend was present for juice omission (p = 0.08 across both monkeys). When we included data for all three stimulus conditions and both monkeys in one analysis, the regression model explained a significant amount of the variance in responses (p = 0.00003), and neurons further rostral displayed greater suppression below baseline (β = −2.2, p = 0.00002). We did not observe a significant effect of mediolateral position in any one of these 11 regression analyses, including the case that responses to all stimuli in both monkeys were included in one analysis (β = 0.14, p = 0.78). This regression analysis indicates that rostrocaudal position had a significant influence, and a substantially stronger influence than mediolateral position. It does not rule out the possibility that mediolateral position may have a small but significant influence when considered in isolation.
Are some dopamine neurons activated by aversiveness?
Matsumoto and Hikosaka (2009) recorded responses of dopamine neurons in macaque SN to juice, air puff, and their omission, as well as CS predicting these. They found evidence that some neurons in SN were excited by both juice and air puff, or by a CS that predicted juice or air puff. They proposed that these neurons may be excited by absolute motivational value, or “salience,” being activated by both appetitiveness and aversiveness (see Bromberg-Martin et al. (2010) for additional discussion of these neurons).
We found that early activation (∼50–150 ms) reflected stimulus intensity, whereas sensitivity to reward value emerged later (∼150–250 ms) (Fiorillo et al., 2013). Matsumoto and Hikosaka (2009) measured firing rates in a relatively long time period that was chosen to capture most of the modulation in firing rate, including the initial activation and subsequent suppression to aversive stimuli (see Materials and Methods for a detailed comparison of our study with theirs). It is therefore possible that the activations they observed to aversive stimuli were related to sensory intensity rather than motivational value. Whereas our analysis above and in the accompanying article examined firing rates in temporal periods that were chosen to capture strong activation or suppression of firing rates, here we focus on the reward-sensitive period (150–250 ms) in which firing rates appear to most accurately reflect the subjective valuation of the stimuli. During this time, it is likely that dopamine neurons receive both strong excitation and inhibition, with excitation dominating in the case of appetitive but not aversive stimuli (Fiorillo et al., 2013). We first ask whether there is a subgroup of dopamine neurons that is activated by aversive stimuli during the reward-sensitive period (150–250 ms), and we then present an analysis that examined firing rates over a longer period (80–250 ms) in an effort to partially replicate the results of Matsumoto and Hikosaka (2009).
During the reward period, which corresponds to the falling phase of the activation to juice, 43% (83 of 194) of neurons were significantly activated by juice, whereas 7% (14) were suppressed. By contrast, ∼9% of neurons were significantly activated by aversive and neutral stimuli (Fig. 6A) (saline-bitter: 8%, 9 of 106; air: 9%, 8 of 86; loud sound: 10%, 3 of 31; juice omission: 9%, 8 of 88). No correction has been made in any of the present analysis for repeatedly performing t tests, and thus we would expect 5% of neurons to reach our significance threshold by chance. Indeed, even measurement of the response to juice in a period intended to capture maximal activation (Fig. 2A; early period) found significant suppression in 2.6% of neurons (5 of 194).
A large fraction of dopamine neurons are activated by a broad spectrum of appetitive stimuli; and as discussed above, those individual dopamine neurons that are sensitive to one type of stimulus tend to be sensitive to others (they are “broadly tuned” to reward). If there is a subset of dopamine neurons that is activated by aversiveness or motivational salience in general, then we should find neurons with activations to different types of aversive stimuli. Among 76 neurons tested with air, saline-bitter, and juice omission, none was activated by all three. We therefore searched for neurons activated by any two of these. Among 78 neurons tested with both air and saline-bitter, none was significantly activated by both (Fig. 9A). Of 82 neurons tested with air and juice omission, one was activated by both, but it was not activated by either saline or juice. Of 77 neurons tested with juice omission and saline, one was activated by both, but it did not respond to air and it was significantly suppressed by juice. These 2 neurons, which were activated by two of the three aversive stimuli, showed mixed responses to conditioned stimuli predicting air or saline, being activated in some cases but not others.
Insubstantial evidence for a subgroup of dopamine neurons that is activated by aversiveness. A, A scatter plot of responses (with baseline rates subtracted) to air and saline-bitter in 84 neurons (24 neurons tested with air and bitter in Monkey O; 60 tested with air and saline in Monkey F). Firing rates were measured during the “reward period” (150–250 ms, as illustrated in C and D). Data points are labeled according to the results of significance tests: blue triangles, activation or suppression after air only; green squares, saline-bitter only; red diamonds, both; black circles, neither. No neurons had a significant activation to both air and saline-bitter. B, Same as in A, but showing responses to conditioned stimuli predicting air and saline-bitter. No neurons were significantly activated by both. Each CS was presented in a block of trials with no other trial types. C, PSTHs were made for the 24 neurons with responses shown in the upper right quadrant of A, which displayed a lack of suppression to both air and saline-bitter, and for the other 56 neurons shown in A. Bin size, 50 ms. D, PSTHs showing the response to juice in the 22 neurons lacking suppression to aversive stimuli versus the 56 others. These two groups are the same as distinguished in C.
We also searched for neurons having activations to conditioned stimuli predicting aversive stimuli. Each audiovisual CS consisted of a distinct visual image but an identical auditory component, and each CS-US pair was presented in a block of trials in which no other stimuli were presented. Conditioned stimuli typically elicited a small early activation followed by suppression (Fig. 6 of Fiorillo et al., 2013). Of 48 neurons tested with a CS predicting air and another predicting saline or bitter solutions (in separate blocks of trials), none was activated by both. For each type of CS considered alone, <10% of neurons were activated (Fig. 9B). Thus, our results with CS responses appear to match our results with US responses. In both cases, we failed to find evidence of neurons activated in a general, “broadly tuned” manner by aversiveness or absolute motivational value.
It should be noted that Matsumoto and Hikosaka (2009) also did not find neurons that were activated by aversive events in general. They looked but did not find a correlation between those neurons that were activated by air puff and those that were activated by a CS predicting air puff (and they therefore classified these as separate groups). Although they tested juice omission, they did not report that any neurons were activated by it.
To further compare our results with those of Matsumoto and Hikosaka (2009), we performed an analysis similar to theirs. Because it is clear in both their study and ours that activation to air occurred at very short latencies and was brief, and we already have shown this to depend on sensory intensity rather than motivational value (Fiorillo et al., 2013), we focus here on responses to conditioned stimuli predicting aversive stimuli. Although, like us, they found what appeared to be a single cluster of responses in scatter plots (supplemental Fig. 2 of Matsumoto and Hikosaka, 2009), they began their analysis of CS responses by sorting their population of neurons into three categories based upon tests of statistical significance (which naturally impose arbitrary thresholds for the purpose of creating categories). Following their example, we categorized our neurons as ACS-excited (n = 14), ACS-inhibited (n = 9), or unresponsive (n = 54) depending on whether they showed significant (p < 0.05) activation or suppression between 80 and 250 ms after onset of a CS that predicted air puff (the time period was chosen to capture most of the activation and subsequent suppression). Although we found a smaller proportion of responsive neurons than Matsumoto and Hikosaka (2009) (perhaps in part because they selected neurons to record based on significant activation to unpredicted juice), in both studies the largest group of neurons were unresponsive to the CS (70% vs 40%), and ACS-excited neurons outnumbered ACS-inhibited neurons by nearly the same ratio in both studies (1.56 vs 1.58).
After a CS predicting air puff, our ACS-excited neurons (n = 14) displayed a population average activation that started at ∼60 ms and ended at ∼140 ms (Fig. 10) (onset latency of 87 ± 12 ms, mean ± SEM; and median of 65 ms, with latencies measured as described by Fiorillo et al., 2013). The early and brief activation suggests that these neurons were responding to sensory stimulation rather than motivational value. If ACS-excited neurons correspond to a group of neurons selected for early, sensory-related activation, they might also display earlier activation to other stimuli as well. We therefore examined responses to visual conditioned stimuli predicting juice and found that ACS-excited neurons were indeed activated earlier than ACS-inhibited neurons, with latencies of 101 ± 14 versus 145 ± 7 ms (p = 0.02 overall; p < 0.10 in each monkey). This replicates the observations of Matsumoto and Hikosaka (2009), who found latencies of 114 and 144 ms for responses to conditioned stimuli predicting juice in ACS-excited and ACS-inhibited neurons, respectively (Matsumoto and Hikosaka, 2009; compare their Fig. 2b with their Fig. 2f, and see pages 10 and 11 of their supplemental information). We also found nearly identical results after performing the same analysis with cues predicting saline-bitter (111 ± 4 ms latency for a CS predicting juice in ACS-excited neurons, 138 ± 5 in ACS-inhibited neurons, p = 0.005). Thus, we have replicated some of the key observations of Matsumoto and Hikosaka (2009), but we do not find evidence in support of their conclusion that some dopamine neurons are activated by aversiveness. Other evidence that they provided for neurons activated by aversiveness may be explained by their statistical analyses and experimental design (see Methodological comparison with Matsumoto and Hikosaka (2009)).
Selection of neurons with activation to a CS predicting air puff reveals short-latency activations that are likely to be related to sensory intensity rather than motivational value. Neurons were sorted into three subgroups based on their responses to a CS predicting air puff, in an effort to replicate the analysis of Matsumoto and Hikosaka (2009). Of 77 neurons tested (Fig. 6 of Fiorillo et al., 2013), 14 had significant activation (“ACS-excited” in black), 9 had significant suppression (“ACS-inhibited” in red), and 54 “unresponsive” neurons did not have significant responses during the period of 80–250 ms (shaded region). Bin size, 20 ms. Analogous results were obtained with cues predicting saline-bitter (data not shown).
Neurons lacking suppression to aversive stimuli
Although we did not find evidence of neurons that were activated by aversiveness, we did observe neurons that clearly were not suppressed by our aversive stimuli. To examine these neurons more closely, we divided the 78 neurons in Figure 9A, tested with both air and saline-bitter, into two groups based on firing rates during the “reward period”: 22 “nonsuppressed” neurons with even the slightest increase in firing rate to both air and saline-bitter (Fig. 9A, upper right quadrant; 5 and 17 neurons from Monkeys O and F) versus the 56 other neurons (other 3 quadrants; 19 and 37 neurons from Monkeys O and F). It is noteworthy that only 8 of the 22 nonsuppressed neurons had even a slight increase in firing rate above baseline after juice omission (during the reward period).
We observed three quantitatively distinct properties of the neurons lacking suppression. First, although these 22 neurons displayed an increase in average firing rate to aversive stimuli (for which they were selected), their average activation in response to juice was at least twice as large (Fig. 9 compare C with D; see also scatter plots of Fig. 4, in which most neurons with small increases in firing rate to aversive stimuli in the middle period display larger increases in response to juice). The much larger activation to juice compared with aversive stimuli would not be expected if these neurons are activated by absolute motivational value because these stimuli all had nearly equal absolute values and because, even if juice were to have substantially greater value, adaptation of dopamine neurons should yield similar responses under these test conditions (Tobler et al., 2005).
A second feature of these nonsuppressed neurons is that they had longer-lasting activation by juice (Fig. 9D). The initial phase of their activation by juice was small compared with other neurons (Fig. 9D), as expected given the results above for a subset of neurons with weak or no suppression (Fig. 4). However, whereas the average response of the other neurons went slightly below baseline after ∼250 ms, these neurons continued to display elevated firing rates. This suggests that these neurons have relatively weak homeostatic inhibition in response to juice (Fiorillo et al., 2013). This is consistent with the possibility that their activations to aversive stimuli may have been caused by sensory intensity, but having weak homeostatic inhibition, the activation in some neurons persisted into the reward period.
Third, the 22 neurons lacking suppression to aversive stimuli appeared to have slightly elevated baseline firing rates relative to the 56 other neurons (4.8 ± 0.7 Hz vs 3.7 ± 0.3, p = 0.08). This is despite the fact that the subset of neurons with late rebound activation and high baseline rates (Fig. 5) were included among the 56 neurons because they had strong suppression to aversive stimuli. Furthermore, among the 22 nonsuppressed neurons, the increase in firing rate (150–250 ms, averaged across responses to air and saline-bitter, with baseline rates subtracted) was positively correlated with baseline firing rate (r = 0.65, p = 0.0006). However, this correlation might entirely result from spike count variance increasing with mean firing rate (Bayer et al., 2007), and because we selected this subgroup by excluding all neurons with firing rates below baseline, we may have created an artifactual positive correlation.
Discussion
Despite examining multiple phases of the responses of dopamine neurons to a variety of appetitive and aversive stimuli, we observed only a single continuum, with no evidence of statistically discrete subpopulations. However, we did find evidence of a distinction between dorsal and ventral tier neurons despite our limited ability to correctly classify neurons. Thus, it seems likely that discrete subpopulations might be revealed by application of recording techniques that can distinguish biochemical phenotypes (e.g., Lammel et al., 2008; Brown et al., 2009; Cohen et al., 2012). Nonetheless, it appears that differences between potential subpopulations may be quantitative rather than qualitative, with variation across neurons being greater within than across subpopulations. It is important to note that there may be a group of “mesoprefrontal” dopamine neurons that have yet to be recorded in behaving animals because of their small extracellular waveforms (Lammel et al., 2008) (see Materials and Methods).
Within a single continuum, we found several factors that could distinguish rather loosely defined “subtypes” of dopamine neurons. The most obvious distinction would be between responsive (∼75% activated by juice with p < 0.05) and unresponsive (∼25%) neurons. The simplest hypothesis is that less responsive neurons differ only in their sensitivity to reward, and they do not have substantial responses in studies such as this because all of the rewards tested are very small and highly predictable (because of short intervals between rewards) (Fiorillo et al., 2008). The less sensitive neurons could represent a reserve pool that enables the dopamine signal of the entire population to grow larger as the magnitude of RPE increases over a large range. This would be complementary to the rapid adaptation that serves a similar purpose in the more sensitive neurons (Tobler et al., 2005).
We can further distinguish three types of responsiveness: early, sensory-related activation (40–150 ms), reward-related activation and suppression (150–350 ms), and late “rebound” activation (350–600 ms). We found that it is substantially the same group of neurons that exhibits both reward-related activation as well as suppression to a wide variety of stimuli (appetitive, aversive, and neutral) (Figs. 3 and 4). A subset of these showed late “rebound” activation after a period of suppression, as well as higher baseline firing rates (Fig. 5). We did not find any relationship between early, sensory-related activation, and subsequent sensitivity to motivational value, consistent with previous observations (Bromberg-Martin et al., 2010).
Lack of evidence for dopamine neurons activated by aversiveness
Although aversive stimuli caused both activation and suppression, we found that neutral stimuli did as well, and that the short-latency activation (∼<150 ms) is related to sensory intensity rather than motivational value (Fiorillo et al., 2013). When we examined firing rates at longer latencies (150–250 ms), we found no clear evidence of a subgroup of neurons activated by aversiveness (Fig. 9).
Our conclusion contradicts that of Matsumoto and Hikosaka (2009), who interpreted their data to suggest that a subgroup of dopamine neurons are activated by aversiveness as well as appetitiveness. It is possible but unlikely that we failed to record from the same neuronal population in dorsolateral SN (compare our Fig. 1C with their Fig. 4). We have reason to suspect that our air puff was substantially more aversive than theirs (see Materials and Methods). In addition, we were able to replicate several of their results, and we suggest that their data may be fully consistent with our interpretation. Their key evidence that aversiveness (rather than sensory intensity) caused activation is that a CS+ predicting air puff caused greater activation than a CS− predicting no air puff. However, we think that this might have been an artifact of their experimental design (which used the same CS− in both appetitive and aversive contexts), possibly compounded by their statistical analysis (which began by sorting cells based on significance tests). In Materials and Methods, we suggest how this might be, and we compare our methodology with theirs. We also propose a simple reanalysis of their data to test whether they did indeed identify a subgroup of neurons preferentially activated by aversiveness.
Relationship of neuronal diversity to RPE
There are likely to be two types of inhibition in dopamine neurons: “homeostatic” and “opponent” (Fiorillo et al., 2013). Homeostatic inhibition contributes evidence of reward, and it thereby functions to predict and counteract excitation associated with reward. It can be seen in our results after the activation to juice and probably also after juice omission. That it is strong in the same cells that are strongly excited by reward supports models of how prediction error is generated through a balance of excitation and inhibition (Fig. 4) (e.g., Brown et al., 1999; Fiorillo, 2008). According to such models, homeostatic inhibition sometimes causes hyperpolarization (negative prediction error), but more often, predictions are accurate and homeostatic inhibition counteracts excitation without causing hyperpolarization. In contrast, opponent inhibition would be caused by stimuli that contribute evidence against reward (including aversive stimuli), which naturally tend to occur in the absence of evidence for reward and its associated excitation. Opponent inhibition would therefore tend to drive strong hyperpolarization. That opponent inhibition is observed predominately in the same neurons that display reward excitation and homeostatic inhibition suggests that neuronal sensitivity to evidence for and against reward is matched. Our appetitive and aversive stimuli had nearly equal (and low) absolute values and may therefore have engaged the same set of relatively sensitive neurons. Both types of inhibition are fully consistent with RPE.
The concept of prediction and prediction error may help to explain the diversity across neurons in expression of voltage-regulated ion channels as well as synaptic inputs (Fiorillo, 2008). Some of the predictions in “prediction error” probably correspond to forms of negative feedback mediated by voltage-regulated ion channels, which are homeostatic and keep changes in firing rate brief (“phasic”). The density of ion channels varies across neurons, and it is to be expected that, for a given stimulus, some neurons have either slightly excessive or inadequate negative feedback. This may explain small and late modulations in firing rate. For example, the small number of neurons that were weakly activated at longer latencies (>150 ms) by aversive stimuli (Fig. 9) may have had inadequate negative feedback after early activation by sensory intensity. Similarly, “rebound” activations and suppressions may have been caused by excessive negative feedback in some cells (Figs. 2, 4, and 5). HCN channels depolarize dopamine neurons after hyperpolarization (e.g., Neuhoff et al., 2002), and they could mediate the late rebound activation. It has been proposed that the function of H-type channels is to restore homeostasis by predicting and counteracting the latter part of inhibitory synaptic events to keep hyperpolarization brief (Fiorillo, 2008).
Neurons with rebound activation also tended to have higher baseline firing rates and to be localized ventrally (Figs. 5 and 7). All of these features suggest that they could correspond to a group of calbindin-negative dopamine neurons with high expression of HCN channels (Neuhoff et al., 2002; Brown et al., 2009), found in the ventral tier of SN dopamine neurons in primates (Haber et al., 1995). The concept of prediction error could help to explain the observed variability in expression of HCN channels. As suggested above, there appear to be two “types” of dopamine neurons (or a continuum) that differ in their sensitivity to motivational value. The more sensitive neurons would more often be hyperpolarized by nonreward stimuli via synaptic inhibition (because they respond to virtually all such events), whereas the less sensitive neurons would only rarely be hyperpolarized (because they respond only to strong evidence against reward). Because strong hyperpolarization is a relatively common pattern in the sensitive neurons, they could deploy HCN channels to counteract hyperpolarization and keep it brief. By keeping the suppression brief, the higher expression of HCN channels in the sensitive neurons would actually serve to minimize differences in the RPE output across the entire population of neurons. Some of our sensitive neurons could have expressed a slight excess of HCN channels, resulting in the rebound activation that we observed (Fig. 5). Other sensitive neurons may have lacked a sufficient density of HCN channels, and they therefore exhibited long-lasting suppression of firing rate (Fig. 5). In the population average, late activation and late suppression substantially average out (Fig. 2A). Although we found that the more sensitive neurons may be found primarily (or exclusively) in the ventral tier (Fig. 7), Brown et al. (2009) did not find a difference in suppression to aversive stimuli between calbindin-positive and -negative SN dopamine neurons in anesthetized rats (although the number of neurons they recorded was relatively small, and most were unresponsive).
The scenario presented above remains speculative, but it illustrates how neuronal diversity could be explained within the framework of prediction error. If we consider that quantitative aspects of “reward” and “prediction” will vary from one neuron to another, the concept of RPE appears to be capable of accounting for the observed diversity of responses in dopamine neurons.
Footnotes
This work was supported by the World Class University program through the National Research Foundation of Korea, funded by the Ministry of Education, Science and Technology Grant R32-2008-000-10218-0 to C.D.F. and a grant to William T. Newsome from the Howard Hughes Medical Institute. We thank Bill Newsome for advice and helpful discussions and Julian Brown for excellent technical assistance.
The authors declare no competing financial interests.
- Correspondence should be addressed to Dr. Christopher D. Fiorillo, Department of Bio and Brain Engineering, Korea Advanced Institute of Science and Technology, 291 Daehak-ro, Building 611, Yuseong-gu, Daejeon 305-701, Republic of Korea. fiorillo{at}kaist.ac.kr
References
In this issue
