Abstract
The amygdala is critical for connecting emotional reactions with environmental events. We recorded neurons from the basolateral complex of two monkeys while they performed visually cued schedules of sequential color discrimination trials, with both valid and random cues. When the cues were valid, the visual cue, which was present throughout each trial, indicated how many trials remained to be successfully completed before a reward. Seventy-six percent of recorded neurons showed response selectivity, with the selectivity depending on some aspects of the current schedule. After a reward, when the monkeys knew that the upcoming cue would be valid, 88 of 246 (36%) neurons responded between schedules, seemingly anticipating the receiving information about the upcoming schedule length. When the cue appeared, 102 of 246 (41%) neurons became selective, at this point encoding information about whether the current trial was the only trial required or how many more trials are needed to obtain a reward. These cue-related responses had a median latency of 120 ms (just between the latencies in inferior temporal visual area TE and perirhinal cortex). When the monkey was releasing a touch bar to complete the trial correctly, 71 of 246 (29%) neurons responded, with responses in the rewarded trials being similar no matter which schedule was ending, thus being sensitive to the reward contingency. Finally, 39 of 246 (16%) neurons responded around the reward. We suggest that basolateral amygdala, by anticipating and then delineating the schedule and representing reward contingency, provide contextual information that is important for adjusting motivational level as a function of immediate behavior goals.
Introduction
The amygdala is important in generating emotional context for situations and events (LeDoux, 2000; Everitt et al., 2003; Phillips et al., 2003). For example, functional neuroimaging studies in humans show that the amygdala is activated during recognition of fearful and happy facial expressions (Morris et al., 2002; Hare et al., 2005). The amygdala also appears to be involved in evaluating the reward value of a stimulus (Malkova et al., 1997; Parkinson et al., 2001). All of these findings are consistent with the view that the amygdala is important for assessing the affective (or emotional) valence of a stimulus and/or production of a specific affective state (Everitt et al., 2003; Phillips et al., 2003).
We have been studying how the brain uses external stimuli, particularly visual cues, to predict how much work (compare with number of trials) is needed to get a reward (visually cued reward schedule task) (Bowman et al., 1996). The visual cue, which predicts the work load remaining in the current schedule, affects the monkey's motivational and probably emotional states; because the visual cue progressively indicates that more work remains, the error rates and reaction times increase. The increases in error rates and reaction times indicate the monkeys are less motivated. This decrease in motivation in turn shows that the cues indicating that more work remains negatively affect the monkey's emotional state in that the monkeys become less inclined to complete the current trial correctly. Single neuron recording studies have identified robust signals related to this type of reward expectancy in the ventral striatum (Shidara et al., 1998), the perirhinal cortex (Liu and Richmond, 2000), and the anterior cingulate cortex (Shidara and Richmond, 2002).
Here we report on single neuronal recordings from monkeys performing this reward schedule task in the basolateral complex of the amygdala, a region that receives input from the anterior temporal lobe, including area TE (Cheng et al., 1997) and rhinal cortex (Stefanacci et al., 1996), and is high in dopamine (Sadikot and Parent, 1990). The signals carried by amygdala neurons, by coding for schedule and the reward contingency, could be used to resolve ambiguity about the upcoming reward. The anatomical connections and timing relations of the signals suggest that the amygdala may be part of a temporal lobe network that takes information about stimulus identity from temporal lobe sensory areas and generates signals that interpret the external stimuli or events by providing an emotional context, allowing the animal to organize its behavior.
Materials and Methods
Animals, behavioral task, and visual stimuli
Two adult rhesus monkeys (Macaca mulatta, 9 and 5 kg, one male and one female, respectively) were used. During our experiments, the monkey squatted in a primate chair facing a 21 inch computer monitor.
The monkeys were first trained to perform a sequential red-to-green color discrimination task (Fig. 1 A). The monkeys were trained without a fixation requirement. A display was 43.5 cm in front of the monkey. A black-and-white random dot background covered the whole screen of the display.
When the monkey made contact with a touch lever to initiate a trial, a visual cue (rectangle 38 × 2° in visual angle) appeared in the center of the screen. Four hundred milliseconds later, a small red spot (0.3 × 0.3°) appeared on the center of the visual cue. After a random period between 1 and 2 s, the color changed to green. If the monkeys released the touch lever within a 0.2-1.3 s period after the appearance of the green spot, the spot became blue, indicating that a correct response had occurred. After another 100-200 ms, the visual cue and the blue spot disappeared, and a drop of liquid was delivered. The intertrial interval was 1.1 s.
If the monkey did not release the bar within the designated period (i.e., within a 0.2-1.3 s period after the green spot onset), an error was registered, the stimuli disappeared, and the intertrial interval before the next trial started.
After the monkeys learned to perform the color discrimination task (with 90% correct trials per session), we introduced the visually cued reward schedules. In the visually cued reward schedule task (Fig. 1 A), the monkey had to complete one, two, or three color discrimination trials correctly to obtain a reward, i.e., schedules of one, two, or three trials. We identify the trials in the schedules as schedule states. For labeling of the schedule states (Fig. 1), the numerator indicates a progress in the schedule, and the denominator indicates the schedule length (in number of trials). The schedule states are 1/1 for the one-trial schedule, 1/2 and 2/2 for the two-trial schedule, and 1/3, 2/3, and 3/3 for the three-trial schedule. A reward was delivered only after the successful completion of the final trial of a schedule. A new schedule of one, two, or three trials was picked randomly after a reward.
If the monkeys released the touch lever too early, i.e., earlier than 200 ms after the green spot appeared, or too late, i.e., later than 1.3 s after the green spot appeared, the stimuli all disappeared, an error was registered, and the normal intertrial interval began. There was no explicit punishment for errors, but the schedule state advanced only after a correct trial. After an error trial, the same trial of the same schedule state was repeated.
We tested valid and random cue conditions. In the valid cue condition, the cue became brighter as the rewarded trial approached and was brightest in the rewarded trial (Fig. 1). The brightness was proportionally related to the schedule state [pixel intensities in eight bit; 85 (33%) for the 1/3 state, 127 (50%) for the 1/2, 170 (67%) for the 2/3, and 255 (100%) for all three rewarded trials, namely the 1/1, 2/2, and 3/3 states] (Fig. 1 B). In the random cue condition, the reward schedule sequence remained, but the brightness of the cue was chosen randomly from the four brightness levels trial by trial so that there was no relationship between the cue brightness and the schedule state.
In another no-work, no-cue control condition, designated as free-reward condition, a reward was delivered at a regularly irregular rate of between 5 and 8 s (chosen pseudorandomly) with no work required, nor was any related stimulus shown.
Each task was run as a block of trials.
Surgical preparation
After the monkeys were trained to perform the reward schedule task, a cylinder for microelectrode recording and a head holder (Crist Inc., Hagerstown, MD) were affixed to the skull above the dorsal surface of the brain during an aseptic surgical procedure with the animal under isoflurane anesthesia. A scleral magnetic search coil for measuring eye movement was implanted during the same surgery (Robinson, 1963; Judge et al., 1980). The monkeys were given a 2 week postoperative recovery period. After recovery, the monkeys were given practice in the tasks with a loose fixation requirement.
Recording of single neuronal activity
Single-neuron data and behavioral data were collected while the monkeys performed the reward schedule task in both the valid and random cue conditions. A hydraulic microdrive (Narishige, Tokyo, Japan) or an electromechanical microdrive (Alpha Omega, Nazareth, Israel) was mounted on the recording cylinder, and tungsten microelectrodes [impedance, 1.5-1.7 MΩ; Frederick Haer Company (Bowdoinham, ME) or Roboz-Microprobe (Rockville, MD)] were inserted vertically through a stainless steel guide tube that was placed in a hole of a grid within a recording chamber (Crist et al., 1988) (Crist Inc.). Experimental control and data collection were performed by a personal computer, using the REX real-time data-acquisition program adapted for the QNX operating system (Hays et al., 1982). Single-neuron activity was isolated by discrimination using time-voltage window discriminator (Frederick Haer Company). All event and spike times were stored at 1 ms time resolution. All of the experimental procedures were in accordance with the National Institutes of Health Guide for the Care and Use of Laboratory Animals and were approved by the Animal Care and Use Committee of the National Institute of Mental Health.
Recording sites localization
In each recording session, the depth of the recording area was measured from the upper tip of a guide tube. Before we changed the location of a guide tube in the grid, a magnetic resonance (MR) image was acquired to document the recording location (Saunders et al., 1990). A tungsten microelectrode was inserted into the monkey's brain to use as a landmark to indicate the recording location on the MR images (susceptibility artifact).
For histological examination of our recording sites, electrolytic microlesions (30 μA, 40 s) were made in the brain of one monkey (monkey A). One week later, the monkey was deeply anesthetized with sodium pentobarbital and perfused through the heart with saline and 10% Formalin fixative. The brain was sectioned into 50 μm slices in the coronal plane and stained with cresyl violet. The sites of guide tubes and microlesions were identified on the sections under microscopic examination. The histological reconstructions matched the locations determined using the MR, so the MR images alone were used for localization of recording sites in the second monkey.
Data analysis
Behavioral measure. Behavioral performances of the monkeys were measured using reaction times and error rates. Reaction times were defined to be the time interval from the appearance of the green spot (go signal) to the time of bar release.
Neuronal response measure. Neuronal responses were analyzed with respect to several discrete events (Fig. 1 A): touching a bar by the monkeys, appearance of the cue (cue), appearance of the red spot (wait), change of the red spot to a green spot (go), release of the bar by the monkey (bar release), change of the green spot to a blue spot (correct), the time when both the cue and the spot disappeared, and the time of reward apparatus activation, i.e., reward delivery, in the final trial of the schedule (reward). We counted number of spikes within different perievent periods as neuronal response measure.
Precue activity was measured by counting the number of spikes during a 300 ms period just before the cue onset. Control activity for the precue activity was measured during a 300 ms period starting 600 ms before the cue onset. Go/prebar-release activity was measured by counting the number of spikes during a 300 ms period just before the bar release. Reward-related response was measured by counting the number of spikes during a 400 ms period after the reward. Control activity for the reward-related response was measured during a 400 ms period just before the reward.
Conventional statistical procedures (mentioned in Results) were used to assess significance. All data analyses were done in the R statistical computing environment (Ihaka and Gentleman, 1996).
Response latency. The latency of the cue-related responses was determined by comparing spike counts before cue presentation (background activity) and spike counts in a test window that moved starting from the time of the cue onset. To determine response latency, we used two time windows with different durations. We first used a wider test window (100 ms) and then used a narrow test window (50 ms). Background activity was set as spike counts in a 100 ms period just before the cue onset for the 100 ms test window and as spike counts in a 50 ms period just before the cue onset for the 50 ms test window. The 100 ms sliding time window was moved in 5 ms steps, and the 50 ms window was moved in 3 ms steps. Using the 100 ms time window, we found first four consecutive windows that showed significantly different activity compared with the background activity (i.e., activity in a 100 ms period before the cue onset; paired t test, p < 0.05). The first of the four consecutive windows was picked. Starting at the beginning of the first window, we then moved a 50 ms window to find first four consecutive windows that showed significantly different activity compared with the background activity (i.e., activity in a 50 ms period before the cue onset). The first window of the consecutive four windows was picked again, and the middle of the window was defined as the latency for the neuronal response. Values that are measured with this procedure are consistent with values we would have chosen by eye.
Results
Behavioral and electrophysiological data were collected while two adult rhesus monkeys performed the visually cued reward schedule task (Fig. 1).
Behavioral analysis
As seen previously (Bowman et al., 1996; Shidara et al., 1998; Liu and Richmond, 2000; Shidara and Richmond, 2002), the monkeys were sensitive to the cue and the work schedule in the valid cue condition (Fig. 2A). The error rates and reaction times were indistinguishable across the three rewarded trials, 1/1, 2/2, and 3/3 (single-factor ANOVA to the two monkeys together; mean error rates, F(2,735) = 1.79, p > 0.05; mean reaction times, F(2,735) = 0.0022, p > 0.05). The error rates and reaction times were significantly different across the first trials, 1/1, 1/2, and 1/3 (single factor ANOVA; mean error rates, F(2,735) = 46.72, p < 0.0001; mean reaction times, F(2,735) = 73.7, p < 0.0001), showing that the monkeys were sensitive to the information provided by the cue about the schedule.
Both monkeys reacted progressively more quickly as the rewarded trial approached in the two- and three-trial schedules (single-factor ANOVA with repeated measures for effect of schedule progress, two-trial schedule, F(1,245) = 314.3, p < 0.0001; three-trial schedule, F(2,490) = 324.8, p < 0.0001). Error rates became smaller as the schedule progressed through the three-trial schedule (single-factor ANOVA with repeated measures for effect of schedule progress, F(2,490) = 79.4, p < 0.0001). In the two-trial schedule, error rates changed significantly for monkey B only (F(1,130) = 1.3, p > 0.05 for monkey A; F(1,114) = 109.7, p < 0.0001 for monkey B).
As expected in the random cue condition, the error rates and reaction times were the same in all three first trials (1/1, 1/2, and 1/3; single-factor ANOVA; mean error rates, F(2,390) = 0.81, p > 0.05; mean reaction times, F(2,390) = 0.18, p > 0.05). In the random cue condition, some information about the reward contingency is available because the longest possible sequence of unrewarded trials was fixed (two in the three-trial schedule); the probability of a reward is 1/3 in the first trial after a preceding reward, 1/2 if one unrewarded trial had passed, and 1, i.e., the reward is certain, if two unrewarded trials have passed. In previous studies, the monkeys had not shown any sensitivity to the number of trials that had passed (Bowman et al., 1996; Shidara et al., 1998; Liu and Richmond, 2000; Shidara and Richmond, 2002). Here, however, both monkeys were sensitive to number of trials that had passed, with a stronger trend in the three-trial schedule. For monkey A, the error rates increased as the schedule progressed (single-factor ANOVA with repeated measures for effect of schedule progress, F(2,146) = 5.8, p < 0.01) whereas the reaction times were not modulated by the schedule state (F(2,146) = 0.63, p > 0.05). For monkey B, the reaction times were related to the schedule states, becoming shorter as the schedule state progressed (F(2,112) = 46.6, p < 0.0001), whereas the error rates were not modulated by the schedule state (F(2,112) = 1.5, p > 0.05). In the states of the two-trial schedule, neither reaction times nor error rates were significantly different for monkey A. Reaction times decreased for monkey B (F(1,56) = 13.6, p < 0.001).
Both monkeys were also sensitive to the cue brightness in the random cue condition (Fig. 2B). The brightest cue was treated differently from other cues in both reaction times and error rates (p < 0.01, t test). Thus, the monkeys were sensitive to two features of the random cue condition, the brightness of the cue, and the number of unrewarded trials that had passed.
To examine behavioral variability, we took data from 84 of 246 recording sessions (46 for monkey A, 38 for monkey B) for the valid cue condition in which at least 20 trials had been performed for each schedule state. Data were available from the random cue condition in 40 of these sessions. In the valid cue condition, the reaction times were significantly different across three first trials (1/1, 1/2, and 1/3) in 43 of 84 (51%) sessions (single-factor ANOVA, p < 0.01). The reaction times were indistinguishable across the three rewarded trials, i.e., 1/1, 2/2, and 3/3, for 79 of 84 (94%) sessions (single-factor ANOVA, p > 0.05). In the two-trial schedule, the reaction times were significantly different along progress for 43 of 84 (51%), and in the three-trial schedule for 41 of 84 (49%; single-factor ANOVA with repeated measures for effect of schedule progress, p < 0.01). All 84 recording sessions showed at least one of the following four reaction time characteristics: significantly different across the three first trials, not significantly different across the three rewarded trials (i.e., not influenced by the post-reward number as described above for the random cue condition), significantly different along the progress in the two-trial schedule, or significantly different along the progress in the three-trial schedule. Twenty-three sessions showed one, 18 sessions showed two, another 25 showed three, and 18 showed a significant effect of all four reaction time characteristics. Thus, the valid cue seemed to affect the animals in every recording session, although the effect changed its pattern from session to session. For 18 of the 23 sessions showing only one criterion, the criterion met was not significantly different across the three rewarded trials. The neurons recorded in these 18 sessions did not seem to react differently from neurons recorded in the other sessions, suggesting that there was no substantial difference between the data collected when the behavior was affected by the schedule in a large way compared with the data when the behavior was less affected. In the random cue condition, the reaction times of three first trials were not significantly different for any sessions (single-factor ANOVA, p < 0.01), and the reaction times decreased significantly as the schedule state progressed in 2 of 15 sessions of monkey B (single-factor ANOVA with repeated measures for effect of schedule progress, p < 0.01, a result indistinguishable from a random effect).
We could not find clear relationships between neuronal responses that are presented in the following and these session-to-session variability regarding the reaction times.
Electrophysiology
Activity was recorded from 246 (131 from monkey A, 115 from monkey B) single neurons located in the basolateral complex [including lateral (L), basal (B), and accessory basal (AB) nuclei] of the right amygdala (Fig. 3) of two monkeys during the visually cued reward schedule task (Fig. 1). Responses were collected while single neurons were well isolated, which gave from 6 to 80 repetitions (median 21) of each schedule state (1/3, 2/3, 3/3, 1/2, 2/2, and 1/1) in valid cue condition, and 5-43 repetitions (median 22) in random cue condition. We were unable to identify systematic response differences related to recording site location, so we present the neurons as one set.
We examined neuronal activity over different task events, namely before and after presentation of visual cue (precue activity and cue-related response, respectively), before monkey's bar release (go/prebar-release activity), and around reward apparatus activation (reward related response). We found that 188 of 246 neurons (76%) showed a response in one (59 of 188) or more (129 of 188) of these time periods (Table 1). Thus, for more than one-half of neurons (129 of 246, 52%), task-related activity occurred at more than one point in the behavioral task.
Signals related to reward schedules
Precue activity
In the valid cue condition, 119 of 246 (48%) of the neurons in the basolateral complex of the amygdala showed precue activity that is selective for differentiating interschedule from intraschedule periods (t test between activity in the 300 ms precue period before the first trials and activity in the precue period before the non-first trials, p < 0.01) (as in the example in Fig. 4A). A total of 88 of 119 neurons (74%) showed this type of preschedule activity building toward the end of the interschedule period, i.e., appearance of the first cue (one-tailed t test between activity in the precue period before the first trials and before the non-first trials, p < 0.005). There was virtually no precue activity preceding non-first trials (2/2, 2/3, and 3/3).
The time onset for the preschedule activity varied from trial to trial (compare with Fig. 4A), usually starting well after the reward (≥700 ms here) and long before (≥300 ms here) the cue signaling the beginning of the next schedule. The preschedule activity generally continued until shortly after cue onset (the ending defined as the time when a cue-triggered response began). Only 2 of 88 neurons showed a significant increase in activity just after the cue onset compared with just before the cue onset (paired t test between activity just before and after the cue onset; activity was measured in either a 100 ms period or, if the latency was shorter than 100 ms, a period before latency of the after cue related response, p < 0.01).
The other 31 of 119 neurons (26%) showed a complementary response pattern, i.e., the precue activity in the non-first trials (2/2, 2/3, and 3/3) was larger than precue activity in the first trials (one-tailed t test, p < 0.005). No single neuron had precue activity before every cue (t test between an activity in the precue period and an activity in the control period, p < 0.01).
When we divided the 300 ms precue period into two 150 ms periods, the activity of 33 of 119 neurons (all 33 neurons for preschedule activity) was significantly different in the two periods (one-tailed paired t test on increase, p < 0.005), showing that the precue activity of many neurons was higher just before the appearance of the cue than it had been earlier.
A total of 82 neurons were tested in the random cue condition. The difference in activity between preschedule and intraschedule periods disappeared (49 neurons) (Fig. 4B) or became significantly smaller (11 neurons; t test, p < 0.005). Thus, the precue activity was, in general, not simply related to the reward in the preceding trial. The preschedule activity in the population was small in the random cue condition (Fig. 5). Thus, the preschedule activity is related to receiving the information about the next reward schedule that will be indicated by upcoming valid visual cue, possibly attributable to increased arousal, attentiveness, or some other mechanisms.
Because amygdala lesions interfere with reinforcer devaluations (Malkova et al., 1997), we looked for neuronal firing changes related to the progress through a recording session (presumably the monkeys become satiated as the session progresses, although each recording session might be too short to detect effects related to satiation). We did not identify any trend in the precue activity.
Cue-related response
From inspection, it appeared that many neurons had responses after the appearance of the cues. To identify the period with most cue-selective response, a 350-ms-wide analysis window was moved across the responses in 10 ms steps (Fig. 6). In the range starting with the time window from -285 to 75 ms relative to cue appearance and ending with the time window from 115 to 465 ms relative to cue appearance, 102 of 246 neurons showed at least three consecutive time windows with significant selectivity across the three first cues or the three non-first cues (single-factor ANOVAs, p < 0.01). Of these 102 neurons, 81 showed significant differential activity across the responses elicited by the three first cues (1/1, 1/2, and 1/3) (Fig. 7), six across the three non-first cues (2/2, 2/3, and 3/3), and 15 to both (Fig. 8). The time window from 100 to 450 ms after cue onset, which was initially chosen by eye, was used to approximate the time period with greatest effect (Fig. 6).
In the interschedule or intraschedule precue period, only 3 of 102 neurons (3%, i.e., <5% level) showed a significant differential response related to the upcoming schedule during the anticipatory precue period (starting with the time window from -575 to -225 ms relative to cue appearance ending with the time window from -355 to -5 ms relative to cue appearance) (Fig. 6A). Thus, the precue activity had no systematic effect on the postcue selectivity. The effect of the three first or non-first cues on neuronal activity increased starting at ∼100 ms after the cue onset (Fig. 6). The activity in a 100 ms period just after the cue onset, i.e., the continuation of the precue activity after the cue onset, was indistinguishable across the three first cues or the three non-first cues for all 102 neurons (single-factor ANOVA, p > 0.01).
To determine whether the responses to the first cues might be related to the predicted schedule length, pairwise comparisons (1/1 vs 1/2, 1/2 vs 1/3, and 1/1 vs 1/3; Tukey's test to adjust for multiple comparisons, p < 0.01) were performed. In this analysis, 84 of 102 (82%) neurons showed differential activity across the first trials (compatible with, but not identical to, the ANOVA above) (Table 2). For 43 of 84, the responses elicited by 1/1 cue were different from the responses to the 1/2 or 1/3, whereas the responses to these latter cues were indistinguishable (Fig. 7A); for 23 of 43 neurons, the lack of difference in the responses between the 1/2 and 1/3 cues occurred because there was no response to the 1/2 and 1/3 cues. Thus, there was a significant emphasis of the 1/1 schedule in the population. Nonetheless, the selectivity of the neurons within the population provides information about which schedule is starting (Table 2).
None of the comparisons across first trials was significant for the random cue condition (Figs. 7B,8B), as expected if the monkey was unable to predict the schedule length. The 1/1 cue often induces a significantly different response from the other two cues in the valid cue condition, perhaps attributable to the monkey anticipating an immediate reward. However, it is not quite straightforward to attribute this effect to a simple difference in arousal level as inferred from the behavioral measure. The behavioral results (compare with Fig. 2) suggest that the monkeys might be more aroused (or at least trying harder, i.e., more motivated) in the 1/2 state than in the 1/3 state, yet there does not seem to be any systematic difference between the cue-related responses in the 1/2 and 1/3 states for many neurons. This makes it difficult to invoke a simple connection between probable state of arousal, motivation, or effort and the first-cue elicited responses.
Pairwise comparisons (2/3 vs 2/2, 2/3 vs 3/3, and 2/3 vs 3/3; Tukey's test to adjust for multiple comparisons, p < 0.01) showed that 14 of 102 (14%) neurons exhibited differential activity across the non-first trials (Table 3, Fig. 8), revealing that the responses in non-first trials can on occasion be related to the reward contingency. Of the 43 neurons that showed a significantly different response to the 1/1 cue than to the other two first cues (see above), only five showed differential activity across the non-first trials, suggesting that the neurons responding to the 1/1 cue were not simply responding because the cue signaled a rewarded trial (i.e., the response to the 2/3 cue was generally not different from the response to the 2/2 or 3/3 cues). It may be that the responses to the 1/1 cue are special because the cue indicates the conjunction of a first and rewarded trial, the most favorable message possible in this task.
Sixty-seven of the 102 neurons were tested in the random cue condition. The cue-related responses of 13 neurons were influenced according to two-level first vs non-first trial discrimination, with 8% of response variance being explained [single-factor ANOVA, p < 0.01; f test with 65-190 and 1 degree of freedom; n = 13; interquantile range (iqr) of 6-11%].
The responses of 28 of 67 (42%) neurons were related to the cue brightness (single-factor ANOVA, p < 0.01; f test with 54-213 and 3 degrees of freedom) (Fig. 8C), with 18% (median, iqr of 11-23%) of response variance being explained. Of the 28 neurons, the response was strongest to the brightest cue for 11 neurons (Fig. 8C) or to the darkest cue for 17. For 18 of 28 neurons, the strength of the cue-related responses in the valid cue condition were significantly different from the responses to the same brightness cue in the random cue condition (t test, p < 0.01) (an example shown in Fig. 8), indicating that the cue-related responses in the valid cue condition was not simply related to the cue brightness for majority of neurons (the 18 plus 39 neurons without the brightness effect, i.e., 57 of 67 neurons).
For another 38 of 246 neurons, the activity changed significantly after the cue appearance, but the activity was indistinguishable across 1/1, 1/2, and 1/3 states (single-factor ANOVA, NS; t test between rate in a 100-450 ms window after cue appearance and a 150 ms window before the appearance, p < 0.01). These responses might be simply related to arousal induced by the appearance of the cue.
The latency of the cue-triggered response was 119 ms (median, iqr of 98-158 ms; n = 312; measured for the 113 neurons that showed a clear phasic increase or decrease in activity after the cue onset). The median firing rate was 13 spikes/s (iqr of 8-23 spikes/s; n = 312). The latency was slightly but significantly shorter when precue activity preceded the response (median of 110 ms; iqr of 93-153 ms; n = 103) compared with the response latency when there was no precue activity before the response (median of 123 ms; iqr of 103-163 ms; n = 209; Wilcoxon's test, p < 0.05). These latencies are longer than those reported for area TE and shorter than those reported for perirhinal cortex (Liu and Richmond, 2000).
Signals related to reward
Go/prebar-release activity
In the valid cue condition, we suspected from inspection that the activity after the go signal presentation and just preceding bar release was the same in the final trials of all schedules (Fig. 9A). A total of 71 of 246 neurons showed significant modulation according to two-level rewarded versus nonrewarded trial discrimination (single-factor ANOVA, p < 0.01; f test with 45-440 and 1 degree of freedom). Fourteen percent of the response variance was explained (median, n = 71; iqr of 9-25%). When the responses of these 71 neurons were analyzed using four levels in which the levels were the three nonrewarded states (1/3, 2/3, and 1/2) plus a level for any rewarded trial (thus grouping 1/1, 2/2, and 3/3 together; reward proximity in the schedule), the increased degrees of freedom significantly improved the explanatory power for 10 neurons (an f test comparing the amount of variability accounted for by the two models, p < 0.01) (Venables and Ripley, 2002), and 33% of the response variance was explained (median, n = 10; iqr of 30-40%). When the responses of these 71 neurons were analyzed using the six schedule states, the increased degrees of freedom significantly improved the explanatory power for only two neurons, confirming our suspicion that the responses were related to the reward contingency or the number of trials remaining before the rewarded trial.
The responses were time locked to either the bar release (45 of 71 neurons) or the go signal (26 of 71 neurons; responses classified according to the alignment giving the biggest peak response in the time period between the go signal appearance and bar release). The activity was also modulated by progress in both the two- and three-trial schedules for 25 of 71 neurons (p < 0.01, single-factor ANOVA with repeated measures for effect of schedule progress; f test with 11-37 and 1 degree of freedom for the two-trial schedule; f test with 16-66 and 2 degrees of freedom for the three-trial schedule). The activity increased as the monkey approached the rewarded trial in the schedules (19 of 25 neurons; one-tailed t test, p < 0.005) (Fig. 9A), suggesting that the go/prebar-release activity is related to the expectation of forthcoming reward and/or proximity of reward, possibly via increased arousal.
Forty-five of these 71 neurons were tested in the random cue condition. All but one neuron (44 of 45) lost the differential activity depending on the reward contingency (single-factor ANOVA, p > 0.01; f test with 67-220 and 1 degree of freedom). Go/prebar-release activity in the population was present in every trial (Fig. 10) (example of one neuron Fig. 9B). For 33 neurons, the activity was independent of the cue brightness (single-factor ANOVA, p > 0.01; f test with 65-218 and 3 degrees of freedom) (Fig. 9C), suggesting that the go/prebar-release activity in the valid cue condition did not encode the cue brightness.
Trial-by-trial reaction time and go/prebar-release activity was significantly related only for 9 of 71 neurons (linear regression, 10 of 426 cases, i.e., 6 states by 71 neurons; p < 0.01), showing that the strength of the go/prebar-release activity was only occasionally related to the reaction time. In addition, monkey B showed shorter reaction times as number of trials that had passed increased in the random cue condition, but no neuron (0 of 13 neurons) of the monkey showed significant modulation of the go/prebar-release activity related to reward proximity in the random cue condition (single-factor ANOVA, p > 0.01; f test with 80-218 and 3 degrees of freedom). Thus, the go/prebar-release activity generally did not appear related to motor aspects of the monkey's bar release.
Force measurements taken from the drinking spout (Ravel et al., 2001) for one monkey (monkey A) revealed that the monkey started to lick ∼300 ms after onset of the wait signal in the rewarded trials, long before (by at least 700 ms) the onset of the go signal. Thus, it is unlikely that the go/prebar-release activity was related to the monkey's mouth movements.
Reward-related response
Thirty-nine of 246 neurons responded at the time the reward apparatus was activated (which only occurred in rewarded trials). For 32 neurons, there was an excitatory response (Fig. 11A), and, for the other seven, there was an inhibitory response (one-tailed t test against strength of activity in a 400 ms period before the reward, p < 0.005). For seven neurons, the response started before reward apparatus activation. For 12 of 39 neurons (including one inhibitory neuron) that had responses in the rewarded trial, we also found responses in nonrewarded trials at the time when the reward would have been delivered if this had been a rewarded trial. For these 12 neurons, the strength of responses in the rewarded and nonrewarded trials were not consistently different (t test, p < 0.005; larger to the rewarded trials, 4 of 12; smaller to the rewarded trials, 4 of 12; not significantly different, 4 of 12).
Of 21 of 39 neurons tested with the random cue condition, 19, including four that showed anticipatory activity before the reward, responded to the reward (one-tailed t test against strength of activity in the control period, p < 0.005) (Fig. 11B). Thus, most of the reward-related signals were found in both the valid and random cue conditions (Fig. 12A), suggesting that they are related to reward delivery itself in some way. Seven neurons responded to the reward only in the random cue condition.
The free-reward condition provided evidence that some reward-related responses might become associated with other task events during the reward schedule task. Among 113 neurons tested with both the valid cue and free-reward conditions, 20 responded in the free-reward condition only and 16 responded in both conditions (Fig. 12B) (an example in Fig. 11). Of the 20 neurons that showed the reward-related response only in the free-reward condition, 14 responded either to the cue presentation (Fig. 13) or around the bar release in the valid cue condition. If the neuron shown in Figure 13 had been tested only in the free-reward condition (Fig. 13A), we would have concluded that the response was a “reward-”related response. However, in the valid cue condition, the response occurred to the first cue (Fig. 13B), suggesting that the neuron has a type of signal similar to the prediction signal described for dopamine neurons (Mirenowicz and Schultz, 1994; Schultz et al., 1997). The response transfer to the cue presentation or to the bar release in the valid cue condition might be related to dopaminergic inputs to basolateral complex (Sadikot and Parent, 1990). A similar type of response transfer has been found in other brain areas that receive dopaminergic inputs, such as the orbitofrontal cortex (Matsumoto et al., 1995; Tremblay and Schultz, 2000) and the striatum (Apicella et al., 1997; Ravel et al., 2001).
Discussion
More than 70% of recorded neurons showed response modulation for one or more aspects of this visually cued reward schedule task (Table 1). The selectivity of this population changed dynamically across the schedules and trials. The population begins firing before the cue presentation; for individual neurons, this activity occurs during either the interschedule or the intraschedule periods but not both, signaling whether the next trial begins a new schedule or not. When the first cue appears, the population encodes which schedule is starting, with the greatest proportion of neurons indicating whether or not the one-trial (1/1) schedule has begun. Starting just before the bar release, i.e., from the go/prebar-release period, the responses become selective for the reward contingency, i.e., they indicate whether a reward is forthcoming or not. A few neurons responded to reward delivery. The dynamics, going from uncertainty to more and more focus on the reward, may be related to resolution of an emotionally laden (or arousing) reward situation.
Possible origins of the signals
The basolateral complex of amygdala receives inputs from a late stage of ventral visual processing, having reciprocal connections with the area TE (Aggleton et al., 1980; Amaral and Price, 1984; Cheng et al., 1997; Stefanacci and Amaral, 2002). Area TE has reciprocal connections with the perirhinal cortex (Suzuki and Amaral, 1994; Saleem and Tanaka, 1996), and perirhinal cortex has reciprocal connections with the amygdala (Amaral and Price, 1984; Saunders and Rosene, 1988; Stefanacci et al., 1996). Presumably visual information that is first processed in area TE is sent to both perirhinal cortex and amygdala, and these three areas influence each other.
During the visually cued reward schedule task, only cue-related activity has been observed in area TE and perirhinal cortex. In contrast, amygdala neurons show responses before the cue presentation, before the bar release, and to the reward delivery as well as to the cue. Thus, visual, reward-related, and other signals merge in the amygdala.
In the amygdala, in the valid cue condition, the population is biased toward the 1/1 schedule. In the random condition, there is small but significant response related to brightness of the cue, somewhat like what has be seen in area TE (Liu and Richmond, 2000). In perirhinal cortex, individual neurons show responses that are idiosyncratically related to subsets of the cues, with the cue-related responses encoding information about the schedule states in the population (Liu and Richmond, 2000).
The latency of the cue-triggered responses in the amygdala neurons is longer than those in area TE and shorter than those in perirhinal cortex. These results imply that information processing in the amygdala lies between TE and perirhinal cortex and might contribute to form the associative signals in perirhinal cortex in which the D2 receptor is crucial for learning the relation of the cue to the schedule states (Liu et al., 2000, 2004). There is, however, a difficulty. At approximately the time of bar release and/or a reward delivery, perirhinal cortex neurons did not respond, so either the cue-triggered amygdala responses do not contribute to perirhinal neuronal responses or cue-triggered activation of perirhinal neurons requires other input in conjunction with the amygdala inputs.
Another possible source for some of the signals in the amygdala are dopamine neurons in substantia nigra pars compacta/ventral tegmental area and/or neurons in orbitofrontal cortex, both of which project strongly to the basolateral complex of amygdala (Sadikot and Parent, 1990; Ghashghaei and Barbas, 2002). Dopamine neurons are known to respond to visual stimuli that precede and predict reward delivery and to motor acts such as bar release that lead to a forthcoming reward (Schultz and Romo, 1987). Some amygdala neurons respond to the cue or around bar release in the valid cue condition but to reward in the free-reward condition, a situation related to reward anticipation or prediction. In the amygdala, the response elicited by the 1/1 cue is often different from the response to the other two first cues (Table 2), a condition that is different from what is found for the dopamine neurons, in which the responses to the first cue are indistinguishable (S. Ravel and B. J. Richmond, unpublished observation). This suggests that the dopamine neurons might play a role in eliciting these cue-related responses but are probably not adequate to explain why the cue in the 1/1 condition elicits a different response than the 1/2 or 1/3 cues.
Recent lesion studies have shown that reinforcer devaluation depends on the amygdala and orbitofrontal cortex (Malkova et al., 1997; Baxter et al., 2000; Izquierdo and Murray, 2004), suggesting that both the amygdala and orbitofrontal cortex play a role in association of the conditioned stimulus with the current value of a reward. Orbitofrontal neurons respond to a visual stimulus that is associated with a reward and the reward itself (Tremblay and Schultz, 2000). Orbitofrontal cortex has a heavy projection to presumed inhibitory neurons (intercalated masses located between lateral and basal nucleus) in the amygdala (Ghashghaei and Barbas, 2002), and signals in the orbitofrontal cortex are thought to modify signal processing within the amygdala. Inhibition after the preschedule activity, such as is shown in Figure 8 for the 1/2 and 1/3 cues, might have been influenced by such processing. The basolateral amygdala is important for the orbitofrontal cortex to represent the significance of a reward-predicting cue (Schoenbaum et al., 2003); thus, the signals in the amygdala could influence signals in orbitofrontal cortex, too.
Other inputs such as those from the posterior thalamus might also play a role in the cue-related response (Komura et al., 2001; Shi and Davis, 2001).
Functional roles of the neuronal responses in the amygdala
The basolateral complex of amygdala receives sensory inputs from sensory association cortices and reward-related information from substantia nigra pars compacta/ventral tegmental area and has reciprocal connections with areas that are important for motor selection depending on reward, e.g., the anterior cingulate cortices (Amaral and Price, 1984; Stefanacci and Amaral, 2002), and the medial and orbitofrontal cortex (Amaral and Price, 1984; Ghashghaei and Barbas, 2002). It also has projections to the ventral striatum (Friedman et al., 2002; Fudge et al., 2002). The responses we recorded show integration of information from these modalities in a time-multiplexed manner (Table 1). Perhaps the amygdala plays a role when association among sensory signals, motor outputs, and reward is required for task performance.
It is known that amygdala neurons (including the basolateral complex) are responsive to sensory events associated with reward and/or to reward itself in monkeys (food, juice, or intracranial self-stimulation) (Nishijo et al., 1986, 1988a,b; Nakamura et al., 1992) and that neurons in basolateral nucleus show anticipatory activity before getting a reward in rats (Pratt and Mizumori, 1998; Schoenbaum et al., 1998). Responses of amygdala neurons to an unconditioned stimulus change depending on satiety (Yan and Scott, 1996) or taste (Nishijo et al., 1988b). Based on these previous studies, we had expected responses related to reward and possibly sensory events predicting the reward (e.g., cue). The precue activity and the modulations around the bar release were unexpected. Comparing our results in other brain regions with those presented here, the amygdala is the only area in which this strong and somewhat prolonged precue activity has been seen thus far (Bowman et al., 1996; Shidara et al., 1998; Liu and Richmond, 2000; Shidara and Richmond, 2002). The amygdala is the only area studied thus far for which the population responses emphasizes the shortest schedule.
A number of previous results suggest that the amygdala plays an important role in evaluating the value or valence of objects. The anatomical connections and timing relations of the signals suggest that the amygdala is in a temporal lobe network that takes information about stimulus identity from temporal lobe sensory areas and generates signals predicting the behavioral path to be taken for obtaining a reward. Given the anatomical connections with the amygdala and a suggested role of the amygdala in emotions, it seems likely that emotions aroused by salient stimuli participate in determining the behavioral path. We have seen a preschedule signal that grows as the time for cue appearance approaches, possibly because the monkey becomes progressively more aroused while awaiting the appearance of the first cue. When the first cue appears, the signal becomes specific, encoding information about which schedule started with emphasis on the shortest schedule, presumably the best (for the monkey) situation. This is compatible with the role of the amygdala in assessing the valence associated with a stimulus (Everitt et al., 2003), in which the most desirable information, that this is the one-trial schedule, is associated with the most selectivity in the population. Later in each trial, the responses are better explained in terms of the reward contingency in the current schedule. These signals seem to provide information that interprets external events and stimuli so that the motivational level can be adjusted according to current behavioral goals. Such signals might be important for forming associative signals related to the reward schedules in perirhinal cortex and/or ventral striatum or forming long-term reward expectation signals in anterior cingulate cortex.
Footnotes
This work was supported by the Intramural Research Program of the National Institute of Mental Health. We thank Dr. R. C. Saunders, Dr. H. Kosaki, Dr. M. Shidara, Dr. Z. Liu, Dr. S. Ravel, Dr. M. C. Weiner, Dr. I. Mizuhiki, Dr. K. Kawano, Dr. R. P. Hasegawa, Dr. Y. T. Hasegawa, Dr. S. Yamane, S. Steenrod, and D. Pritchett for their help. We gratefully acknowledge support in Japan from Japan Society for the Promotion of Science (B.J.R.).
Correspondence should be addressed to Dr. Barry J. Richmond, National Institute of Mental Health/National Institutes of Health, Building 49, Room 1B80, Bethesda, MD 20892-4415. E-mail: bjr{at}ln.nimh.nih.gov.
Copyright © 2005 Society for Neuroscience 0270-6474/05/2511071-13$15.00/0