Abstract
Inferotemporal cortex (IT) is believed to be directly involved in object processing and necessary for accurate and efficient object recognition. The frontal eye field (FEF) is an area in the primate prefrontal cortex that is involved in visual spatial selection and is thought to guide spatial attention and eye movements. We show that object-selective responses of IT neurons and behavioral performance are affected by changes in frontal eye field activity. This was found in monkeys performing a search classification task by temporarily inactivating subregions of FEF while simultaneously recording the activity from single neurons in IT. The effect on object selectivity and performance was specific, occurring in a predictable spatially dependent manner and was strongest when the IT neuron's preferred target was presented in the presence of distractors. FEF inactivation did not affect IT responses on trials in which the nonpreferred target was presented in the search array.
Introduction
The frontal eye field (FEF) is a part of an attentional network (Thompson and Schall, 1999; Coe et al., 2002; Juan et al., 2004; Armstrong and Moore, 2007; Buschman and Miller, 2007; Monosov et al., 2008; Gregoriou et al., 2009; Noudoost et al., 2010; Wardak et al., 2010; Zhou and Desimone, 2011) that includes many brain regions, such as the lateral intraparietal sulcus (Gottlieb et al., 2009; Bisley and Goldberg, 2010) and the superior colliculus (Cavanaugh and Wurtz, 2004; Ignashchenkova et al., 2004; Lovejoy and Krauzlis, 2010). FEF plays a role in spatial selection of behaviorally relevant locations for shifts of gaze (Schiller and Chou, 1998; Schall and Thompson, 1999) and willful shifts of spatial attention (Awh et al., 2006; Monosov and Thompson, 2009; Schafer and Moore, 2011). We demonstrated that the magnitude of FEF activity is correlated with the speed and accuracy of object identification, and that inactivation of FEF produced deficits in object identification (Monosov and Thompson, 2009); additionally, spatial signals in FEF emerge earlier than object-specific signals in IT (Monosov et al., 2010). Consistent with this, an earlier elegant examination of effects of FEF microstimulation on ventral visual processing showed that V4 neurons' responses were affected by artificial activation of FEF in a manner of spatial attention (Moore and Armstrong, 2003). In addition, inactivation of FEF reduces V4 stimulus selectivity (Noudoost and Moore, 2011). Spatial attention-related signals in the brain may be important for perception when many stimuli compete for neural representation (Moran and Desimone, 1985; Chelazzi et al., 1998; Reynolds et al., 1999).
IT is thought to be the end stage of the ventral “what” visual processing stream (Mishkin et al., 1983). Human lesion studies and computational modeling suggest that IT plays a critical role in object recognition (Logothetis and Sheinberg, 1996; Riesenhuber and Poggio, 2002). We reasoned that we should be able to read out the influence of FEF activity on visual processing from IT because it represents the culmination of feature processing by brain regions in the ventral visual stream.
We tested the hypothesis that FEF spatial signals modulate neuronal representations of objects by temporarily inactivating FEF with microinjections of muscimol, a GABA type A (GABAA) agonist, while simultaneously recording IT neurons as monkeys performed a covert visual search classification task (CVST).
Materials and Methods
General procedures.
Data were collected from two male monkeys (Macaca mulatta). All procedures were performed in accordance with the U.S. Public Health Service policy on the humane care and use of laboratory animals, and all protocols were approved by the National Eye Institute Animal Care and Use Committee. Visual stimulation and behavioral control were done by a computer running REX (Hays et al., 1982). General experimental procedures were previously described (Monosov and Thompson, 2009).
Spike density functions were calculated for each trial by convolving spike times with a Gaussian filter (σ = 10 ms). All statistical tests used to assess the differences across sessions were two-tailed paired sign-rank tests.
We identified FEF locations at which low current microstimulation (<50 μA) elicited ∼8–12° eye movements to the horizontal contralateral position (relative to the electrode). These FEF sites were the targets of our muscimol injections. The search array was arranged so that one stimulus was in the horizontal contralateral position (relative to the recording electrode); eccentricities ranged between 9° and 10° of visual angle. Recording locations in IT were verified with structural MRI in both monkeys and spanned +11 to +20 anterior of the interaural position (Fig. 1E).
Target objects, task timeline, and activity from two example sessions. During each trial, any one of 20 target stimuli (A) could appear along with three distractors, each located randomly at the four search array locations (B). While maintaining fixation on the central spot, the monkeys classified the target with a left or a right lever turn based on a learned arbitrary association. C, An illustration of spatial relationships during a hypothetical experimental session. Black spots represent the search array locations. The region of the saccadic deficit was measured using a memory-guided saccade task. IT neurons have large receptive fields that typically encompass multiple array locations (hypothetical IT receptive field is indicated by a circle). IT neuron activity and behavior performance before and during FEF inactivation were compared for trials in which the preferred target was inside the saccadic deficit region and for trials in which it was outside the saccade deficit region. D, Average activity (±SEM) during correct trials from two example IT neurons before (black trace) and during (red trace) FEF inactivation when the preferred target was presented at array locations inside the saccade deficit region (left) and outside the saccade deficit region (right). Scatters show fixation positions for correct trials from the two example sessions, before (black) and during (red) inactivation. The neuronal data for neutral and cued trials was combined. E, A structural MRI showing an electrode in the inferotemporal cortex of Monkey 2. This coronal image was sliced at an angle that followed the electrode's trajectory. F, The timeline of a single experimental session.
FEF was inactivated and IT was recorded on the same hemisphere. Our targeted injections affected the contralateral search array location, at which IT responses were on average the strongest.
Task.
The stimulus library was the same as in our previous studies (Mruczek and Sheinberg, 2007a). Initially, the monkeys were trained to identify 20 target objects presented at the center of the screen with a lever turn. Ten objects were associated with a left lever turn and ten were rewarded following a right lever turn (Fig. 1A). A set of 50 other objects served as distractors. After the object–lever turn association training, the monkeys learned the cued covert search task (Fig. 1B). In this task, after grasping the lever and positioning it within 10° of vertical, a small (0.6°) central yellow fixation cross appeared on a black background. The monkeys were required to maintain eye position within 2° of the central fixation cross until the reward. After the monkeys fixated the central cross, a cue array of four rings appeared. The cue array rings were isoeccentric and were 3.5° in diameter. In about half of the trials, one of the four rings was red and the rest were green (cued trials). In the other half of the trials, all the rings were green (neutral cue trials). After a 235 ms cue–target onset asynchrony, an object appeared inside each ring. One of the objects was sampled from the target object pool (Fig. 1A) and the remaining objects were sampled from the distractor pool. In cued trials, the target object appeared in the red ring. While maintaining fixation on the central cross, the monkeys reported the identity of the target with the associated left or right lever turn for a juice reward. Performance accuracy did not differ during cued and uncued trials (p > 0.2). Reaction time (RT) across sessions was slightly faster on cued trials than on uncued trials (uncued trials: 335 ± 57 ms; cued trials: 320 ± 65 ms; p < 0.05).
Reaction times were measured from the time of the presentation of the search array objects to the beginning of the lever turn, defined as a turn >15° from vertical.
At the beginning of each recording session, a passive viewing task was used to identify two target objects associated with opposite lever turns such that the IT cell was consistently activated by one target (the preferred target) more than by the other target (the nonpreferred target) (Mruczek and Sheinberg, 2007a). Four distractor objects were also chosen that evoked activity from the IT neuron similar to that evoked by the nonpreferred target object.
On each trial, a target was chosen from the target pool. There was a 37.5% chance that the target was the IT neuron's preferred object, a 37.5% chance that it was the nonpreferred object, and a 25% chance that it was randomly chosen from the 18 remaining objects in the target pool (control). The three distractors on each trial were chosen at random from the four previously identified distractor objects. The locations of the target and each of the distractors were randomized from trial to trial. The control trials were used to discourage the monkeys from adapting search strategies specific to the preferred and nonpreferred target objects chosen for that session. In this study, and in similar tasks (Mruczek and Sheinberg, 2007b; Monosov et al., 2010), performance accuracy and reaction times from the control trials with random target images did not differ significantly from those obtained from the preferred and nonpreferred target trials during each of the 13 experimental sessions (performance accuracy: χ2 test, p > 0.2; reaction time: p > 0.1).
During eight sessions, we added a single-target condition during which the targets appeared without distractors and the animal classified them with a lever turn (without breaking fixation just as during the search trials). Spatial cues were not used for single-target trials on four of eight recording sessions. In these sessions, the single target appeared 235 ms after the monkey fixated the fixation point. During the other four of eight sessions, a single ring predicted the location of the upcoming target-stimulus. The timing of the events during single-target trials in all eight sessions in which they were included was the same as the previously described search trials, as illustrated in Figure 1B. Target probabilities on single-target trials were the same as in the search trials (a 37.5% chance that the target was the preferred object, a 37.5% chance that it was the nonpreferred object, and a 25% chance that it was randomly chosen from the 18 remaining objects in the target pool). On sessions during which the single-target trials were included (8/13), ∼25% of all the trials performed by the animals were single-target trials (which were interspersed throughout the experimental session).
Muscimol injection procedure.
Muscimol injections were made in the left FEF in Monkey 1 and in the right FEF in Monkey 2. Injections were made using a custom-made injectrode. On a muscimol injection day, we inserted the injectrode into the brain at a depth previously identified to contain spatially selective FEF neurons. We verified online that each injection site exhibited strong multiunit visual and saccade-related activity in the memory-guided saccade task in the horizontal contralateral direction (relative to the electrode). The timeline of each experimental session is outlined in Figure 1F. We injected muscimol dissolved in saline (5 μg/μl) in boluses of 0.2 μl separated by at least 30 s. The total volume of solution injected ranged between 3 and 6 μl over an ∼15–20 min period. Details about our methods of injection were previously outlined in (Monosov and Thompson, 2009).
Results
The objective of this report was to ascertain the effect of FEF inactivation on object processing in IT. Therefore, in the analysis of neuronal data, we included only those sessions during which we (1) stably recorded an object-selective IT neuron for the duration of the experimental session, (2) observed an eye movement deficit after the muscimol injection in the memory-guided saccade task (MGS), and (3) were able to gather neuronal and behavioral data during CVST before and after the muscimol injection. Sufficient data were obtained from two monkeys during 13 sessions (M1: 5, M2: 8). Neuronal modulations during cued and neutral trials after the time of the search array presentation were similar (see Fig. 4) and were therefore combined.
The locations tested in MGS matched the spatial locations of the search arrays. If after collecting data in the covert visual search task the IT neuron was still well isolated, we injected muscimol into FEF. After the injection, MGS contralesional saccadic latencies increased during all sessions (Mann–Whitney U test, p < 0.001; average increase: ∼150 ms). Saccades to the ipsilesional target and the two targets on the vertical axis were either not affected or had slight target overshooting (Dias and Segraves, 1999; Monosov and Thompson, 2009). Another very sensitive marker of the effectiveness of the FEF inactivation is a small but consistent shift in fixation accuracy away from spatial region effected by the inactivation (Dias and Segraves, 1999). We observed a small but significant shift in average fixation accuracy (average shift: 0.24 ± 0.1° SD; significant fixation accuracy shifts were observed on 12/13 sessions (Wilcoxon rank sum test, p < 0.05) to the ipsilateral hemifield.
We analyzed behavior and IT activity before and during FEF inactivation on trials in which the target to be classified appeared in the region of the saccadic deficit, which was always contralateral to the site of muscimol injection. These trials are referred to as the IN-trial group (n = 13). Performance accuracy (PA) in IN trials was 91% before inactivation and 84.7% during inactivation (p < 0.05; Fig. 2A). PA during trials in which the target appeared in the three search array positions that were outside of the region of saccadic deficit was 87% before the inactivation and 90% during the inactivation (n = 13; p = 0.1; Fig. 2B). Because in search tasks monkeys tend to be less accurate on trials in which the target appears in the vertical search array locations, in Figure 2C we also show PA before and during FEF inactivation in the horizontal ipsilesional search array location. PA was significantly modulated only in search trials in which the target appeared in the region of saccadic deficit (Fig. 2). We found no significant PA effect on single-target classification trials (p > 0.05; PA > 95% before and during inactivation).
Performance accuracy for trials during which the target to be identified appeared IN (A) and OUT (B) of the region of the saccadic deficit before and during FEF inactivation. Because the IN trials were trials in which the target appeared in the contralesional horizontal position, for comparison, we also show performance accuracy for the ipsilesional horizontal position separately (C). Performance accuracy before inactivation in the IN trials (A) was almost identical to the performance accuracy before and during FEF inactivation in the ipsilesional horizontal location (p > 0.4) shown in C. Error bars indicate SEM. The statistical comparisons are indicated above the bar plots. Asterisk indicates significant difference in performance accuracy before and during FEF inactivation when the target appeared within the region of saccadic deficit.
The median RT across sessions before FEF inactivation was 327 ± 57 ms. To characterize the reaction times before and during FEF inactivation, for each experimental session we performed a two-way ANOVA with epoch relative to inactivation (before/during) and spatial location of the target to be classified in the search array as factors. During 2/13 sessions, we found significant differences in the speed of object identification between the four search array locations (p < 0.05). During 7/13 sessions, we found significant differences in the speed of object identification dependent on inactivation epoch. Importantly, there were no significant interactions between target location and before/during injection during 11/13 sessions (2/13 sessions had significant increases in Position 1, the inactivated region).
We compared object-related IT neural responses in a time window of 150–300 ms after the search array presentation, before and during FEF inactivation. During trials in which the preferred target was the target to be classified, FEF inactivation resulted in decreased object-related IT responses during IN trials (p < 0.01), and no observed changes in object-related responses during OUT trials (p > 0.15). During trials in which the nonpreferred target was the target to be classified, IT activity was not affected by FEF inactivation (see Fig. 4).
During eight recording sessions (M1: 4, M2: 4), we interleaved single-target classification trials in which a single target stimulus was presented alone at one of the four locations. During these trials, we did not observe a significant FEF inactivation effect on object-related IT responses (Fig. 3B,D). We interpret this pattern of results to mean that the effects of FEF inactivation are stronger during search trials than during single-target trials; higher neuron number is needed to test conclusively whether FEF inactivation modulates object selectivity during single-target trials.
Pooled average IT activity from correct trials during which the preferred target was presented aligned on the time of the search array presentation. A, Activity from search trials in which the target appeared inside the saccade deficit region (IN trials), before and during FEF inactivation. B, Activity from trials in which a single preferred target appeared inside the saccade deficit region before and during FEF inactivation. C, Activity from search trials in which the target appeared outside of the saccade deficit region (OUT trials). D, Activity from trials in which a single preferred target appeared outside the saccade deficit region before and during FEF inactivation. E, Neuronal discrimination index (ROC: preferred vs nonpreferred responses) for search and single-target trials before and during FEF inactivation for the IN and OUT trials. AUC stands for area under ROC curve. Error bars indicate SEM. p values for paired two-tailed sign rank tests comparing before and during conditions are indicated above the error bars. The dotted-line indicates chance (50%) neuronal discriminability (0.5 ROC). Asterisk indicates significant difference in neuronal discrimination of the target before and during FEF inactivation in search trials in which the target appeared within the region of saccadic deficit.
To summarize the effects of FEF inactivation on object selectivity in IT during visual search and single-target trials, we computed an index of neuronal discrimination of the preferred and nonpreferred object using receiver operating characteristic (ROC) analysis. This analysis is critical because of its resistance to small changes in recording conditions as it takes into account both the preferred and nonpreferred responses (which should both change in a similar way if recording conditions change during the session); this analysis can therefore be useful for comparisons of the magnitude of the effects of FEF perturbation on neuronal activity in the visual cortex with similar studies. We computed the area under the ROC curve (Thompson et al., 1996) for all correct trials in a time window of 150–300 ms after the presentation of the search array. The ROC analysis compared all trials in which the preferred target was the target to be classified with all trials in which the nonpreferred target was the target to be classified. In this analysis, values >0.5 indicate greater activity for the preferred target, values <0.5 indicate greater activity for the nonpreferred target, and 0.5 means no difference in activity for the preferred and nonpreferred targets. The results of the ROC analysis (Fig. 3E) show that neuronal selectivity was most affected by FEF inactivation within the region of saccadic deficit during search trials (n = 13; p < 0.01). The small but consistent change in object selectivity during search-IN trials (0.717 before inactivation vs 0.656 during inactivation; difference = 0.06 ROC) seemed to approximately match the change in performance accuracy shown in Figure 2 (PA: 91% before inactivation and 84.7% during inactivation; difference = 6.3%). Such a relationship between changes in IT neuronal discriminability and behavioral performance may suggest that changes in spatial bias affected performance accuracy and IT activity in a similar manner, but this needs to be tested by future studies.
Weak deficits in neuronal discrimination were observed during single-target trials, but they did not reach statistical significance (n = 8; see above discussion regarding interpretation of single-target trials). Neuronal discrimination was significantly modulated in the search trials in the region of saccadic deficit during the eight sessions in which we included single-target trials (n = 8; p < 0.01).
Discussion
When the preferred target was within the region of saccadic deficit, IT responses decreased during FEF inactivation. When the preferred target was outside the region of the saccadic deficit, IT responses on average were not affected by FEF-inactivation. IT neuronal discrimination was affected within the region of saccadic deficit. These effects were strongest when the visual display was cluttered. On some trials, we included a spatial cue, but did not modulate task difficulty to encourage its usefulness. Regardless of whether the animals used this cue, the main result of our findings stands: IT object responses for the preferred stimulus decreased during visual search when the preferred target appeared within the region of saccadic deficit and did not change during the nonpreferred target trials.
An earlier study showed that temporary inactivation of dorsolateral prefrontal cortex (via cooling) decreases color selectivity of some IT neurons and that this change is primarily driven by decreases in the responses for the preferred color (Fuster et al., 1985). In that study, the effects of dorsolateral prefrontal cortex cooling were assessed in a population of IT neurons that were not preselected for color selectivity, while in our study, we preselected IT neurons based on their object preference (see Materials and Methods) because we wanted to test the effects of FEF inactivation on IT neurons that are involved in processing the target stimuli. Nonetheless, the effects of dorsolateral prefrontal cortex cooling on IT seem to be similar to those observed in this inactivation of FEF (Fig. 4).
A–D show activity during correct trials in which the preferred target was the target to be classified; E–H show activity during correct trials in which the nonpreferred target was the target to be classified. A, Cued IN trials before (black trace) and during (red trace) FEF inactivation. B–D, Cued OUT trials (B), neutral cued IN trials (C), and neutral cued OUT trials (D). E–H, All conventions are the same as in A–D. Time windows for the statistical comparisons for the cue- and target-array responses are indicated by gray bars below the activity traces. The results of paired two-tailed sign-rank tests are indicated above the gray bars. All activity is aligned on the time of the target array presentation—time 0.
IT activity and the monkeys' performance accuracy were not modulated in a consistent manner by FEF inactivation when the target was presented without distractors. A recent study in V4 showed that FEF inactivation decreases stimulus selectivity of V4 neurons during passive presentation of the preferred and nonpreferred single stimuli (Noudoost and Moore, 2011). We observed similar trends (but these trends did not reach statistical significance). We hypothesize that differences in task demands (our task was an active classification task) and our relatively low neuron number in the single-target condition may explain the difference between the results in V4 and IT. Our results suggest that FEF may be more important for visual perception of stimuli in cluttered scenes than for perception of single stimuli, a notion that is consistent with the role of FEF in “top-down spatial attention” (Juan et al., 2004; Hamker, 2005; Armstrong et al., 2006; Gregoriou et al., 2009; Monosov and Thompson, 2009; Pouget et al., 2009; Zhou and Thompson, 2009; Schafer and Moore, 2011).
Overall, our data fit well with a recent fMRI study that suggests that spatial signals from FEF are required for accurate visual search (Wardak et al., 2010), and with a recent study that showed that spatial attention improves decoding of object identity from IT neurons in cluttered scenes (Zhang et al., 2011). We hypothesize that the FEF inactivation-related effects we observed on IT responses may be due to changes in the monkeys' spatial bias and are not the result of direct influences of FEF on IT. In fact, FEF strongly projects to V4 and TEO rather than IT, and these projections may be inhibitory (Pouget et al., 2009). An attractive speculation is that by acting on “earlier” visual areas (such as V4 and TEO) (Bertini et al., 2004), spatial signals from FEF [and other regions (Bisley and Goldberg, 2010; Lovejoy and Krauzlis, 2010)] facilitate selection processes necessary for the binding of features into object representations that guide behavior.
Footnotes
This work was supported by the Intramural Research Program of the National Eye Institute–National Institutes of Health as well as NEI Grant R01-EY14681 (D.L.S.). We thank Drs. Jeffrey Schall, David Leopold, Okihide Hikosaka, Bruce Cumming, Michael Schmid, Ethan Bromberg-Martin, Christian Quaia, Robert Wurtz, Susheel Vijayraghavan, Shinya Yamamoto, and Masaharu Yasuda for valuable comments; and Charles Zhu and Neal Phipps for excellent MRI services.
- Correspondence should be addressed to Ilya E. Monosov, Laboratory of Sensorimotor Research, National Eye Institute, NIH, Building 49, Room 2A50, Bethesda, MD 20892. ilya.monosov{at}gmail.com
