Abstract
Repetition suppression, which refers to reduced neural activity for repeated stimuli, is typically explained by bottom-up or local adaptation mechanisms. However, recent theories have emphasized the role of top-down processes, suggesting that this response reduction reflects the fulfillment of perceptual expectations. To support this, an influential human fMRI study showed that the magnitude of suppression is modulated by the probability of a repetition. No such repetition probability effect was found in macaque inferior temporal (IT) cortex for spiking activity despite the presence of repetition suppression. Contrary to the human fMRI studies that showed an effect of repetition probability, the macaque single-unit study used a large variety of unfamiliar stimuli and the monkeys were not required to attend the stimuli. Here, as in the human fMRI studies, we used faces as stimuli and made the monkeys attend to the stimulus content. We simultaneously recorded spiking activity and local field potentials (LFPs) in the middle lateral face patch (ML) of one monkey (male) and a face-responsive region of another (female). Although we observed significant repetition suppression of spiking activity and high gamma-band LFPs in both animals, there were no effects of repetition probability even when repetitions were task relevant and repetition probability affected behavioral decisions. In conclusion, despite the use of face stimuli and a stimulus-related task, no neural signature of repetition probability was present for faces in a face responsive patch of macaque IT. This further challenges a general perceptual expectation account of repetition suppression.
SIGNIFICANCE STATEMENT Repetition suppression is a reduced brain activity for repeated stimuli commonly observed across species. In the predictive coding framework, such suppression is thought to reflect fulfilled perceptual expectations. Although this hypothesis is supported by several human fMRI studies reporting an effect of repetition probability on repetition suppression, this could not be replicated in single-cell recordings in monkey inferior temporal (IT) cortex. Subsequent studies narrowed down the conditions for the effect to requiring attention and being limited to particular stimulus categories such as faces. Here, we show that, even under these conditions, repetition suppression in monkey IT neurons is still unaffected by repetition probability, even in a task with a behavioral effect, challenging the perceptual expectation account of repetition suppression.
Introduction
Sensory processing in the brain does not only depend on the current input from the senses, but is also affected by previous sensory experience. A well known example is the reduced neural activity when stimuli are repeated, called repetition suppression (Desimone, 1996). Repetition suppression is abundant in inferior temporal (IT) cortex. Indeed, several studies showed that the responses of macaque single IT neurons typically decrease with stimulus repetition (Miller et al., 1991; Vogels et al., 1995; Sawamura et al., 2006; McMahon and Olson, 2007; Liu et al., 2009; De Baene and Vogels, 2010; Kaliukhovich and Vogels, 2011, 2012, 2014; Kuravi and Vogels, 2017). The repetition suppression seen in single-unit responses and local field potential (LFP) gamma power likely underlies the repetition suppression observed in fMRI activations in the ventral stream of monkeys (Sawamura et al., 2005) and humans (Grill-Spector and Malach, 2001; Grill-Spector et al., 2006; Malach, 2012; Barron et al., 2016).
Bottom-up or local adaptation mechanisms (Whitmire and Stanley, 2016) have been proposed to underlie repetition suppression in the ventral visual stream (Vogels, 2016). However, it has also been suggested that repetition suppression results from a reduction of responses that encode a prediction error through a mechanism involving top-down influences of perceptual expectation (Friston, 2005; Summerfield et al., 2008). In support of this expectation-based hypothesis, multiple fMRI studies in humans reported stronger repetition suppression in blocks of trials in which a repetition is more frequent compared with those in which a repetition is infrequent and presumably unexpected (Summerfield et al., 2008; Kovács et al., 2012, 2013; Larsson and Smith, 2012; Andics et al., 2013; Grotheer and Kovács, 2014; Grotheer et al., 2014; Mayrhauser et al., 2014; Ewbank et al., 2016). These repetition probability effects were originally reported using faces in the fusiform face area (FFA) (Summerfield et al., 2008) and later in other face selective regions such as the occipital face area (Kovács et al., 2012; Larsson and Smith, 2012; Grotheer et al., 2014; Ewbank et al., 2016) and non face selective regions such as the lateral occipital complex (Kovács et al., 2012; Larsson and Smith, 2012; Grotheer and Kovács, 2014; Grotheer et al., 2014).
In contrast to these human fMRI studies, a single-cell study (Kaliukhovich and Vogels, 2011) found no evidence of an effect of repetition probability on repetition suppression in macaque IT. Subsequent human fMRI studies narrowed down the conditions under which the repetition probability effect on repetition suppression was typically observed. First, the repetition probability effect is absent when attention is not directed to the stimuli (Larsson and Smith, 2012). Second, repetition probability effects appear to be specific for particular stimulus categories such as faces (Kovács et al., 2013) or letters (Grotheer and Kovács, 2014), not being present in the same subjects for objects (Grotheer and Kovács, 2014; but see Mayrhauser et al., 2014) and false fonts (Grotheer and Kovács, 2014). Both constraints could potentially explain the absence of an effect of repetition probability in the single-unit study of Kaliukhovich and Vogels (2011): the monkeys were passively fixating and perhaps paying little to no attention to the stimulus content and unfamiliar fractal patterns or object images were used instead of faces.
To reveal a neural correlate in monkey IT of the human fMRI findings on repetition probability, we used faces as stimuli while manipulating the probability of trials in which faces were repeated. During the experiments, we recorded spiking activity and LFPs in the macaque middle lateral face patch (ML), which might be the homolog of FFA (for discussion, see Yovel and Freiwald, 2013). In a first experiment, the monkeys performed a task that required attention to the stimulus content. Therefore, we mimicked the task and stimulus category of the human fMRI studies that observed a repetition probability effect. In a second experiment, we went a step further by making face repetitions task relevant.
Materials and Methods
Subjects
Experiments were conducted with two rhesus monkeys (Macaca mulatta; one male, Monkey G, and one female, Monkey D). Aseptic surgeries under deep isoflurane gas anesthesia were performed to implant an MRI-compatible head post and recording chamber using procedures similar to those reported previously (Kaliukhovich and Vogels, 2011). The location of the recording chamber was guided with a preoperative MRI scan and fMRI activations (see below). Recording locations were verified with postoperative MRI by extrapolation of the trajectories of tungsten wires. These were fixed within glass capillaries and inserted into the holes of the recording grid, which was positioned inside the recording chamber. Animal care and experimental procedures were approved by the KU Leuven Animal Ethics Committee and in accordance with national and European guidelines.
Face patch localization
In each monkey, face-selective patches were localized using fMRI as described previously in detail (Taubert et al., 2015). Briefly, we used the same 80 naturalistic greyscale stimuli as of Tsao et al. (2003), which included 5 categories (16 images each per category): human faces, human (headless) bodies, fruits, manmade objects, and hands. The images (size 8°) were presented on a gray background during continuous fixation with a red fixation dot superimposed. Blocks of all 16 stimuli of each of the five categories were presented in pseudorandom order. Each block duration was 16 s: 16 images presented in shuffled order for 1 s each without interstimulus interval. A block in which only the fixation target was shown (16 s) was presented after every fifth categorical block presentation. Each block was presented five times per run of 490 s with the restriction that all blocks needed to be presented once before they were repeated.
Imaging data were acquired with a 3 tesla full-body scanner (MAGNETOM Prisma; Siemens) using a custom-made 8 channel phased-array receive coil and radial transmit-only surface coil (Ekstrom et al., 2008). We used a gradient-echo T2*-weighted echo-planar imaging sequence of 34 horizontal slices (Monkey G; voxel size = 1.5 mm isotropic, TR = 2 s, TE = 15 ms, flip angle = 90°) or 40 horizontal slices (Monkey D; voxel size = 1.25 mm isotropic, TR = 2 s, TE = 18 ms, flip angle = 90°). Signal-to-noise ratio was enhanced with a MION contrast agent (monocrystalline iron oxide nanoparticles, 8–11 mg/kg; Rienso; Takeda) injected intravenously before scanning (Vanduffel et al., 2001).
The functional images were preprocessed separately per day using SPM12 for slice-time correction and spatial realignment to the first volume of the first run. Next, the mean of the realigned functional scans was used to compute transformation parameters for co-registration with a skull-stripped anatomical MRI of the subject (JIP Toolkit version 3.1). After coregistration, the images (resliced at 1 mm isotropic voxel size) were spatially smoothed with an isotropic 3D Gaussian kernel (2 mm full width at half maximum; SPM12).
For statistical analysis, we used SPM12 to fit a general linear model to the functional images, estimating regression coefficients per run. Regressors were convolved with the MION response function (Vanduffel et al., 2001) and included one for each block type (image category) as well as motion and eye movement regressors of no interest. Face patches were defined with xjView (version 9.0) using a threshold of t-value = 5 (positive activations only) with the contrast faces versus all other categories.
Repetition probability experiments
For the main experiments, we generated 50,000 images of unique human faces seen from the same frontal perspective (FaceGen Modeler, version 3.5, https://facegen.com/). They were presented in trials of two stimulus presentations of 250 ms separated by 500 ms. In a repetition trial, the same image was repeated, whereas two different faces were shown in an alternation trial (Fig. 1B). A trial was initiated by 500 ms of maintained fixation and was interrupted whenever fixation was broken. For maintained fixation, the monkey's gaze had to stay within an area (fixation window) calibrated to be 2 × 2 visual degrees centered on the fixation dot. Each face (vertical extent 5°) was practically trial unique as a result of the large number of faces and the restriction that all images had to be used once before they could be used again. The maximum number of reuses of a particular face image in our recording sessions was 8 and 4 with on average 65 (SD = 49) and 372 (SD = 217) days between repeats for Monkey G and D, respectively. To make sure that the faces presented in alternation trials were visually distinct, we predetermined face pairs as follows. First, we downsampled the 136 × 136 images to 32 × 32 pixels and unfolded the resulting image matrices into vectors of length 32 (width) *32 (height) *3 (RGB). Next, we performed principal component analysis on the full set of 50,000 image vectors and retained only the first 50 principal components (98.4% explained variance). Then, we calculated all pairwise Euclidean distances between the faces in the 50D space. Finally, we sequentially selected 25,000 face pairs by each time taking the two faces with the maximum pairwise distance (face dissimilarity) from the remaining pool of faces. As a result, the average difference between faces in alternation trials was substantially larger than it would be in the case of random pairings. See Figure 1A for example face pairs.
Stimuli and experimental procedure. A, Example face pairs that we used for alternation trials selected according to percentage rank number of face dissimilarity (Euclidean distance in 50D PC space; see Materials and Methods) from the most similar pair that we used with rank number 0 to the most dissimilar pair with rank number 100. B, Different types of trial sequences in Experiment 1. Subjects were required to fixate throughout the entire sequence and give the correct saccade response to receive a fluid reward. In each trial, new stimuli were selected until all 50,000 were used (after which the cycle restarted). For target trials, either the first or the second face could be inverted. Note that there were no target trials in Experiment 2, in which the monkey had to indicate repetition (left) versus alternation (right). C, Composition of repetition and alternation blocks in terms of repetition and alternation trials and regular and target trials.
The alternation and repetition trials were both presented in blocks of 40, 100, or 120 unaborted trials (the number was changed between sessions). Aborted trials (i.e., fixation was broken before and during the presentation of both stimuli) were not counted and a new trial was started. A block had either a high or low repetition probability: repetition blocks (75% unaborted repetition trials) and alternation blocks (25% unaborted repetition trials). The first five trials of a block were always of the same type (i.e., repetition or alternation) as the block type. Both block types were presented alternatingly and the type of the first block in a recording session was randomly determined. Between blocks, there were five trials with 300 ms presentations of a full screen color (blue, yellow, green, orange, or purple) during maintained fixation. Except for the task, these procedures were identical to the ones previously described in detail in Kaliukhovich et al. (2011).
Experiment 1: orthogonal task.
Twenty percent of the unaborted trials of each block were target trials in which either the first or second face was inverted (i.e., face presented upside down; Fig. 1B). At 100 ms after the end of the presentation of the second face of a trial, the subject was required to indicate whether it had been a regular (no face inversion) or target trial by making an eye movement to the left (regular) or right (target) dot. These dots (eccentricity 6° on the horizontal meridian) were presented 100 ms after the offset of the second face together with a removal of the fixation dot and lasted for a maximum of 2000 ms until the monkey made a saccade or broke fixation. The monkey received a fluid reward after maintaining fixation throughout the trial and giving the correct response. A higher reward for target trials was required to keep the subjects motivated to do the task because a left response on all trials would already result in 80% correct. Note that we only analyzed responses to regular trials, which all had the same reward size for a correct response regardless of being an alternation or repetition trial. See Figure 1C for an illustration of the composition of the trials in the blocks in this experiment. The average number of repetition and alternation blocks per single neuron or multiunit site was 4.9 and 2.5 for Monkey G and D, respectively. Note that the face inversion manipulation and the proportions of target, repetition, and alternation trials were the same as in the main experiment of Summerfield et al. (2008).
Before the experiment, both monkeys were trained using the same stimuli with a version of the task that also included trials that contained two inverted faces (which could have the same or a different identity). The high proportion of correct responses over the last 5 training days (Monkey G: 96.6–99%; Monkey D: 93–100%) for the trials that included 2 inverted faces suggested that the monkeys were not performing sequential same versus different face orientation judgements but were discriminating upright versus inverted faces. This task was also repeated in between daily recording sessions, with similar performance on trials with two inverted faces (Monkey G: 96.3–100%; Monkey D: 93.2–95.5%).
Experiment 2: task relevant repetitions.
Experiment 2 was identical to Experiment 1 except for the task: the monkey had to indicate whether a trial was a repetition (or an alternation) by making a saccade to the left or right dot, respectively, after the offset of the second face of a trial. This same–different task makes the face repetitions task relevant. There were no inverted face trials in this experiment and all trials received the same reward for a correct response.
Electrophysiological recordings
We recorded LFPs simultaneously with single or multiunit spiking activity using Epoxylite-insulated tungsten microelectrodes (FHC) with an impedance of ∼1 MΩ in situ. For every recording session, a single electrode was lowered into the brain with a Narishige microdrive through a stainless steel guide tube that was fixed in a Crist grid. Spikes of single neurons were isolated online using a window discriminator. In addition, when no single neuron could be isolated, spikes of multiple neurons were thresholded online to record multiunit activity. Stimuli were displayed on a CRT monitor (1024 × 768 pixels at 75 Hz; Brilliance 202P4; Philips) at an eye distance of ∼57 cm. The gaze was continuously tracked by means of a video-based eye tracker using one eye at a sampling rate of 1 kHz (EyeLink; SR Research).
Spiking activity.
While advancing the electrode in search for responsive units, we presented 16 human face images and 16 nonface images in a face category selectivity test as described previously (Taubert et al., 2015). The images were taken from the image set used in the fMRI localizer, but with the noise background removed. The set of 16 nonface images consisted of four images per category (bodies, fruits, manmade objects, and hands), which were selected to be similar to faces in their round shape (Taubert et al., 2015). To initiate a trial, the subject had to fixate (2 × 2 visual degree fixation window) for 300 ms, followed by 300 ms of stimulus presentation and an additional 300 ms fixation period before receiving a fluid reward. The lower bound of the intertrial interval was set to 500 ms, but it could be longer based on the behavior of the monkey because they were required to initiate each trial. All 32 stimuli were presented in random order with the restriction that all images had to be presented before one could be repeated. For each stimulus presentation, we computed the net response using the firing rate in the 300 ms window starting 50 ms after stimulus onset minus that in the 50 ms window before stimulus onset. For each neuron or multiunit site, spiking activity was recorded for at least three presentations of each image to assess the face category selectivity. We quantified the face category selectivity using the following index (Tsao et al., 2006; Taubert et al., 2015):
with Rface being the mean net response to the 16 faces and Rnonface the mean net response to the 16 nonface images. This face selectivity index (FSI) is > 0 for neurons or multiunit sites that respond more to faces than nonfaces (i.e., they are face category selective).
After the face category selectivity test, we ran the repetition probability experiment for at least two blocks (one repetition and one alternation) per neuron or multiunit site. We computed for each stimulus presentation the firing rate in the 250 ms window starting 50 ms after stimulus onset. The data for target trials and the first five unaborted trials of each block were excluded. Per neuron or multiunit site, the gross firing rates for the first (S1) and second (S2) face were averaged across unaborted trials for each of the four conditions (repetition and alternations trials in repetition and alternation blocks). These mean firing rates were used to compute an adaptation index (AI) as follows:
This number expresses the proportional difference in response strength R between the first (RS1) and second (RS2) stimulus and is >0 if the response to the second stimulus is lower (e.g., repetition suppression), <0 if it is higher, and 0 if responses are equal. We computed AIs separately for repetition and alternation trials in the repetition and alternation blocks. Population peristimulus time histograms (PSTHs) were computed by binning firing rates in bins of 25 ms per neuron followed by averaging the binned mean firing rates across neurons.
LFPs.
At most spiking activity recording sites, we also recorded LFPs sampled at 1 kHz. Offline, the signal was band-pass filtered between 0.2 and 170 Hz and line noise was removed using a 50 Hz notch filter (48–52 Hz). We used time–frequency Morlet wavelet decomposition for spectral analysis as described previously (Kaliukhovich and Vogels, 2011) using FieldTrip (Oostenveld et al., 2011). Frequencies below 10 Hz were excluded from the wavelet analysis to avoid wavelets overlapping adapter and test stimulus presentations. At each frequency, we normalized power by division by the average baseline power (200 ms window before stimulus onset). For LFP power responses to S1 and S2, we used the average normalized power in the 250 ms window starting 50 ms after stimulus onset for 4 frequency bands: 12–25 Hz, 26–60 Hz, 61–100 Hz (i.e., the 3 windows used by Kaliukhovich and Vogels, 2011), and 101–170 Hz. Like we did for spiking activity, these LFP power responses were then used to calculate AIs. To compute AIs for the more noisy LFP signals, we averaged the power for S1 across repetition and alternation trials per block.
Statistical analysis
SPM t-maps are based on parametric t tests and visualized in FslView (version 4.0.1) for Figure 2 (t-value threshold = 5 for faces vs bodies, fruits, manmade objects, and hands contrast and 12 for faces vs fixation contrast).
For statistical inference, we used bias-corrected accelerated bootstrap confidence intervals (Efron, 1987) and randomization tests unless indicated otherwise. The bootstrap estimates are based on random sampling with replacement (10,000 iterations) of the neurons or multiunit sites. p-values (uncorrected for multiple comparisons) were calculated using randomization tests (10,000 iterations) to estimate the distribution of the test statistic under the null hypothesis. Because the precision of these p-values is limited by the number of randomizations, we only report values to up to 3 decimal places (smaller values are indicated by p < 0.001). In addition, on several occasions we use the JZS Bayes factor with the default √2/2 scale parameter to quantify evidence for the null hypothesis of one-sample t tests (Rouder et al., 2009).
Finally, we used a hierarchical regression (i.e., a multilevel or mixed-effect model) to combine the data of both monkeys while accounting for the variability across animals, as well as across neurons or recording sites (Vinken et al., 2017). This model was fit on the average raw firing rates per neuron/site for six conditions: S1, S2 alternation trial, and S2 repetition trial in either repetition or alternation blocks. We assumed a log normal distribution of average firing rates after confirming that it describes our firing rate distributions well. We estimated fixed effects for an intercept, stimulus order (S1 or S2), trial type (alternation or repetition), block type (alternation or repetition), the interaction stimulus order × block type, and the interaction trial type × block type. The model included random effects for each regressor at the subject and neuron/site levels and full variance–covariance matrices for both levels. Regressors were coded using dummy variables, with S indicating first (value = 0) or second (1) stimulus (stimulus order), T indicating alternation (0) or repetition (1) trials (trial type), and B indicating alternation (0) or repetition (1) blocks (block type). On the population level, the log-firing rate y for the six conditions i can then be written as follows:
Therefore, the estimated coefficients capture the response to S1 (b0), the difference between S2 and S1 for alternation trials (b1), the difference between repetition trial S2 and alternation trial S2 (b2), the difference between repetition block S1 and alternation block S1 (b3), the difference between stimulus order effect for repetition block and alternation block (b4), and the difference between trial type effect for repetition block and alternation block (b5). Therefore, the latter captures the modulation of stimulus-specific adaptation by repetition probability and is expected to be negative if suppression would be stronger for expected repetitions. All parameters were estimated using the Bayesian regression modeling package rstanarm (Stan Development Team, 2018) by generating 10,000 samples from the posterior distribution using default priors. For statistical inference we report the posterior mean and the 95% interval (containing 95% of the posterior density) to express uncertainty.
Results
We recorded spiking activity and LFPs in two monkeys during trials in which either two different faces were shown (alternation trial) or the same face was repeated (repetition trial). In separate blocks, we manipulated the probability of a repetition trial: 75% repetition trials and 25% alternation trials for repetition blocks and vice versa for alternation blocks. In two different experiments, we ensured that the monkeys attended the stimulus content and recorded mainly from face-selective neurons (Monkey G: 94% of single-units and 95.4% of multiunits' FSI > 0; Monkey D: 84.8% and 53.7%). If repetition suppression for faces reflects fulfillment of perceptual expectations (Summerfield et al., 2008), then it should be stronger in repetition blocks in which a repetition is expected compared with alternation blocks in which it is unexpected.
Face category selectivity
We localized face-selective regions using an fMRI block design with images of five categories: faces, bodies, fruits, manmade objects, and hands (Tsao et al., 2003). We collected 13 runs for Monkey G and 31 for Monkey D. The results show the six prototypical face patches (Tsao et al., 2008) in IT cortex of Monkey G: posterior lateral (PL), ML, middle fundus (MF), and anterior lateral (AL) bilaterally, anterior fundus (AF) only in the right hemisphere, and anterior medial (AM) only in the left (Fig. 2A). For IT cortex of Monkey D, there was only one face patch (bilaterally), which we identified as AL based on its location. There were no other face-selective patches defined by the contrast faces versus all other categories in this monkey even at lower thresholds. However, the contrast faces versus fixation did peak around the expected location of ML, suggesting that this area responds strongly to faces (albeit not selectively on the level of voxels). We will call this region of Monkey D putative ML from here on (Fig. 2B). Although the homology of face category areas in humans and macaques is not firmly established (Yovel and Freiwald, 2013), the relative locations of face, body, place, and color patches (Caspari et al., 2014; Lafer-Sousa et al., 2016), retinotopic mapping (Janssens et al., 2014) and warping of monkey to human cortex (Tsao et al., 2003; Rajimehr et al., 2009) agree with a correspondence between ML and FFA.
fMRI localized face patches and single-cell face category selectivity. A, For Monkey G, we were able to identify six face-selective patches: PL, ML, MF, and AL bilaterally; AM and AF unilaterally (faces vs bodies, fruits, manmade objects, and hands contrast; t-value threshold = 5). The locations are indicated on 4 coronal slices (slices 1–4 selected along the posterior–anterior axis as indicated on the sagittal view). The heatmap below the images shows the face category selectivity profile of spiking activity in ML. Each row represents one image (16 faces and 16 nonfaces) and each column represents one neuron (168 cells sorted by FSI). Values are net responses normalized by the maximum. Right, Preferred face and nonface. B, For Monkey D, we were able to identify only one face-selective patch: AL. Responses to faces (face vs fixation contrast; t-value threshold = 12) did peak at the anatomically expected location of ML. We call this region putative ML. The locations are indicated on two coronal slices. The heatmap to the right of the images shows the face category selectivity profile in putative ML (34 cells, same conventions as in A).
Both fMRI localized ML of Monkey G and putative ML of Monkey D are the areas that we targeted for our recordings of spiking activity during the repetition probability experiments. First, we ran a face category selectivity test to confirm the results of the fMRI localizer. Spiking activity recorded in ML (Monkey G) showed face selectivity for both single neurons [mean FSI = 0.69, 95% confidence interval (CI) = 0.62–0.74, SD = 0.38, 168 neurons; Fig. 2A] and multiunit sites (mean FSI = 0.71, 95% CI = 0.66–0.76, SD = 0.35, 219 sites). Spiking activity recorded in putative ML (Monkey D) did show face selectivity for single neurons (mean FSI = 0.35, 95% CI = 0.19–0.49, SD = 0.44, 33 neurons; Fig. 2B) but not for multiunit sites (mean FSI = 0.06, 95% CI = −0.09 to 0.20, SD = 0.62, 67 sites). Because nonface categories were underrepresented (four stimuli each vs 16 faces), we also calculated FSIs using four randomly selected faces per unit (10,000 randomizations). This resulted in slightly lower indices for Monkey G (single neurons: mean FSI = 0.59, 2.5, and 97.5 percentiles of 10,000 randomizations = 0.53–0.64; multiunit sites: mean FSI = 0.66, 0.63–0.69) and comparable indices for Monkey D (single neurons: mean FSI = 0.33, 0.23–0.43; multiunit sites: mean FSI = 0.08, 0.00–0.17). It should be noted that our recordings were biased toward higher FSI values because we only recorded neurons that responded to faces (regardless of their response to nonfaces) as faces were the only stimuli in our main experiment.
Effect of repetition probability on spiking activity and LFP signals
In general, we expected stimulus-specific adaptation: AI repetition trials > 0 and > AI alternation trials. If there is a repetition probability effect, then the difference AI repetition trials − AI alternation trials should be larger for repetition blocks compared with alternation blocks. In a first experiment, as in human fMRI studies on repetition probability, the behavioral task was orthogonal to the manipulation of repetition probability. In a second experiment, face repetitions were task relevant. Spiking activity was recorded simultaneously with LFPs in fMRI localized regions: ML of Monkey G (Experiment 1: 97 single-units, 110 multiunits, 76 LFP sites; Experiment 2: 20 single-units, 60 multiunits, and 80 LFP sites), and putative ML for Monkey D (Experiment 1: 34 single-units, 18 multiunits, and 52 LFP sites; Experiment 2: 50 multiunits, and 56 LFP sites).
Orthogonal task
In our first experiment we implemented the task used in the main experiment of Summerfield et al. (2008): the monkey had to detect inverted faces occurring in 20% of all trials. In these target trials, either the first or the second face could be inverted. After each trial, the monkey had to indicate with saccades whether the trial included an inverted face (leftward saccade) or not (rightward saccade). This task requires the monkey to attend the stimulus content, but is unrelated to face repetitions or alternations. In the initial 58 recording sessions, we used a block length of 40 trials following previous studies (Kaliukhovich and Vogels, 2011; Kovács et al., 2013), which is twice the 20 trials used by Summerfield et al. (2008). Later, we increased the number of trials to 120, reasoning that longer blocks provide more information about the repetition probabilities and thus may increase expectation-related effects. Because there was no evidence of an effect of block length, we pooled the data of the different block lengths. The behavioral performance of both monkeys in the orthogonal task was excellent and, importantly, independent of trial type and block (Fig. 3A).
Behavioral performance in Experiments 1 and 2. A, Mean percentage correct (and 95% CI error bars) during the orthogonal task of Experiment 1, in which stimulus repetitions were irrelevant. B, Same plot for Experiment 2, in which repetitions were task relevant. RT, Repetition trial; AT, alternation trial.
Spiking activity.
For the face-responsive single-units recorded in Monkey G (n = 97), we observed stronger suppression of the activity to S2 relative to S1 for a face repetition than for an alternation without any block effect. This is clear from both the population PSTHs as well as the AIs computed for individual neurons (Fig. 4A). The response reduction for a repeated stimulus was ∼18% compared with 8% for an alternation. The presence of suppressed activity in the alternation trials, cross-adaptation, is typical in IT when S1 and S2 are similar (De Baene and Vogels, 2010), as is the case for a homogeneous stimulus category such as faces. The stimulus-specific response reduction (AI in repetition − AI in alternation trials) was 10% for each block [repetition block (RB): M = 0.10, SD = 0.24, p < 0.001; alternation block (AB): M = 0.10, SD = 0.23, p < 0.001] with no evidence of a difference between blocks (M = 0.00, SD = 0.26, p = 0.95). Assuming a normal distribution, the Bayes factor (Rouder et al., 2009) in favor of no block effect (BF0) is 8.9. Therefore, given the data, it is ∼9 times more probable that there is no effect of repetition probability in the population average relative to the alternative hypothesis of a block effect. We have previously reported a discrepancy between repetition related effects from single versus multiunit data in rodent visual cortex, perhaps as a result of a single-cell sampling bias (Vinken et al., 2017). Therefore, we examined also multiunit activity for potential repetition probability effects. The results for multiunit data recorded in Monkey G (n = 110; Fig. 4B) were, however, very similar to the single-unit data, with a stimulus specific response reduction of 9–11% (RB: M = 0.11, SD = 0.13, p < 0.001; AB: M = 0.09, SD = 0.11, p < 0.001) with no evidence of a difference between blocks (M = 0.02, SD = 0.17, p = 0.22, BF0 = 4.5).
Spiking activity recorded during the orthogonal task. A–D, First column, Population PSTHs showing the firing rate during the two stimulus presentations (S1 and S2) of repetition trials in repetition blocks (red) and alternation blocks (blue), as well as alternation trials in repetition blocks (orange) and alternation blocks (light blue). Second column, AIs for each trial × block combination (positive values mean suppression for S2), stimulus-specific effects (AI repetition trials − AI alternation trials), and block effects (difference in stimulus specific effect: repetition block − adaptation block). Third column, Scatter plot of AIs for repetition trials in repetition blocks (abscissa) and alternation blocks (ordinate). Triangles on axes indicate mean values. Fourth column, Scatter plot of AIs for alternations trials (see third column). A, Monkey G single-unit results; B, Monkey G multiunit results; C, Monkey D single-unit results; D, Monkey D multiunit results.
In Monkey D, for single-unit data (n = 34; Fig. 4C), the stimulus-specific reduction was 5–8% (RB: M = 0.05, SD = 0.22, p = 0.25; AB: M = 0.08, SD = 0.15, p = 0.004) and 8–10% for multiunit data (n = 18; RB: M = 0.10, SD = 0.13, p = 0.009; AB: M = 0.08, SD = 0.12, p = 0.009; Fig. 4D). There was no evidence of a difference between blocks for either single (M = −0.03, SD = 0.23, p = 0.43, BF0 = 4.0) or multiunit (M = 0.01, SD = 0.15, p = 0.73, BF0 = 3.9). Note that, in this monkey, several neurons responded to the fixation target as well, explaining the somewhat higher prestimulus baseline firing rates. After removal of the fixation dot (and the presentation of the two peripheral response dots), the activity decreased to below baseline levels.
LFP signals.
To exclude the possibility that electrophysiological repetition probability effects are restricted to LFPs, we analyzed LFP data recorded together with spiking activity. Figure 5A shows baseline normalized time–frequency power maps per trial type × block combination for Monkey G (n = 76 sites). Consistent with previous reports (De Baene and Vogels, 2010; Kaliukhovich and Vogels, 2011), these maps indicate that there is a stimulus-specific adaptation effect only for frequencies of ∼70 Hz and higher. This is confirmed by the AIs for the 101–170 Hz window, which show a stimulus-specific power reduction of 8% (RB: M = 0.08, SD = 0.12, p < 0.001; AB: M = 0.08, SD = 0.12, p < 0.001). As with spiking activity, there was no evidence of a difference between blocks (M = 0.00, SD = 0.13, p = 0.92, BF0 = 7.9). For lower-frequency bands, there was either a stimulus-unspecific suppression (26–60 Hz and 61–100 Hz) or enhancement (10–25 Hz) for S2 without evidence of a block effect in all bands (10–25 Hz: BF0 = 7.9, 26–60 Hz: BF0 = 7.2, 61–100 Hz: BF0 = 6.0). The results for Monkey D (n = 52 sites; Fig. 5B) indicate a stimulus-specific reduction for both the 61–100 Hz and 101–170 Hz band: 3–4% for the former (RB: M = 0.03, SD = 0.06, p = 0.004; AB: M = 0.04, SD = 0.07, p < 0.001) and 4% for the latter (RB: M = 0.04, SD = 0.07, p < 0.001; AB: M = 0.04, SD = 0.08, p < 0.001). Neither showed evidence of an effect of block (61–100 Hz: M = −0.01, SD = 0.09, p = 0.32, BF0 = 4.0; 101–170 Hz: M = 0.00, SD = 0.10, p = 0.89, BF0 = 6.6). In addition, the lowest-frequency band of 10–25 Hz showed a stimulus-specific enhancement of 5–6% (RB: M = −0.06, SD = 0.13, p = 0.003; AB: M = −0.05, SD = 0.13, p = 0.01) with no evidence of a difference between blocks (M = −0.01, SD = 0.19, p = 0.81, BF0 = 6.4). For 26–60 Hz, there was a stimulus unspecific suppression for S2 without evidence of a block effect (BF0 = 4.5).
Time–frequency power spectra of LFP signals recorded during the orthogonal task. A, Results for Monkey G. First row, Time–frequency maps of power relative to baseline (−200 to 0 ms). We used a base 10 logarithmic color scale to give power suppression (values <1) equal contrast as enhancement (values >1). Second row, AIs calculated for separate frequency bands (same conventions as in Fig. 4). B, Results for Monkey D (same conventions as in A).
Together, these data suggest that neither the use of face stimuli nor an orthogonal stimulus-related task are sufficient conditions for an effect of repetition probability on the adaptation of spiking activity or LFPs. Therefore, in the next experiment, we made repetitions task relevant.
Making repetition task relevant
In this second experiment, there were no inverted target trials. Instead, at the end of each trial, the monkey had to indicate whether a face was repeated or not. As in Experiment 1, we manipulated repetition probability across blocks. In this way, the manipulated probability of a repetition became directly relevant for the task. We conducted this experiment with Monkey G using block lengths of 40 and 100 trials and in Monkey D using block lengths of 40 trials. An important advantage of the task is that we can now assess the effect of repetition probability on behavior in addition to neural activity. There was a clear interaction between block type and trial type: percentage correct for repetition trials was higher in repetition blocks compared with alternation blocks, whereas the reverse was true for alternation trials (Fig. 3B). Therefore, repetition probability has a marked behavioral effect, demonstrating that the animal was sensitive to repetition probability. Furthermore, the behavioral effects were consistent with those predicted by the assumption that decreasing repetition probability reduces the expectation of a repetition.
Spiking activity.
Despite this behavioral effect, repetition probability still did not affect repetition suppression of neural activity in ML. In Monkey G, spiking activity showed a stimulus-specific response reduction of 20–21% for single-unit data (n = 20; RB: M = 0.21, SD = 0.19, p < 0.001; AB: M = 0.20, SD = 0.11, p < 0.001; Fig. 6A) and 11–13% for multiunit data (n = 60; RB: M = 0.13, SD = 0.11, p < 0.001; AB: M = 0.11, SD = 0.09, p < 0.001; Fig. 6B). There was no evidence of a difference between blocks for either single (M = 0.01, SD = 0.20, p = 0.78, BF0 = 4.1) or multiunit (M = 0.02, SD = 0.11, p = 0.15, BF0 = 2.6). In Monkey D, there was a stimulus-specific reduction of 6% for multiunit activity (n = 50; RB: M = 0.06, SD = 0.13, p = 0.003; AB: M = 0.06, SD = 0.10, p < 0.001; Fig. 6C) with no evidence of a block effect on repetition suppression (M = 0.00, SD = 0.16, p = 0.91, BF0 = 6.5). Although there was no block effect on repetition suppression, the overall stimulus-driven response in Monkey D tended to be lower in repetition compared with alternation blocks (M = −4 Hz, SD = 11, p = 0.014). In addition, there was no positive correlation across sessions between the behavioral interaction and block effect in either Monkey G (Spearman's R; single-units: r = −0.34, p = 0.15; multiunits: r = −0.07, p = 0.62) or Monkey D (multiunits: r = −0.23, p = 0.10).
Spiking activity recorded during the same–different task (same conventions as in Figure 4).
LFP signals.
In Monkey G, LFP power (n = 80; Fig. 7A) showed a clear stimulus-specific reduction of 11% for the 101–170 Hz band (RB: M = 0.11, SD = 0.07, p < 0.001; AB: M = 0.11, SD = 0.06, p < 0.001) with no evidence of a difference between blocks (101–170 Hz: M = 0.00, SD = 0.08, p = 0.94, BF0 = 8.0). For lower-frequency bands, there was either a stimulus-unspecific suppression (26–60 Hz and 61–100 Hz) or enhancement (10–25 Hz) for S2 without evidence of a block effect in all bands (10–25 Hz: BF0 = 8.1, 26–60 Hz: BF0 = 4.6, 61–100 Hz: BF0 = 3.2). In Monkey D (n = 56; Fig. 7B), there was a stimulus-specific reduction of 4–6% for the 101–170 Hz band (RB: M = 0.06, SD = 0.08, p < 0.001; AB: M = 0.04, SD = 0.07, p < 0.001) with no evidence of a difference between blocks (M = 0.01, SD = 0.10, p = 0.30, BF0 = 4.1). For 10–25 Hz and 61–100 Hz, there were stimulus-specific effects of 5–8% for the former (RB: M = −0.08, SD = 0.17, p < 0.001; AB: M = −0.05, SD = 0.19, p = 0.04) and 4–6% for the latter (RB: M = 0.06, SD = 0.09, p < 0.001; AB: M = 0.04, SD = 0.09, p = 0.002). Neither showed evidence of an effect of block (10–25 Hz: M = −0.03, SD = 0.26, p = 0.46, BF0 = 5.2; 101–170 Hz: M = 0.03, SD = 0.12, p = 0.13, BF0 = 2.3). For 26–60 Hz, there was a stimulus-unspecific suppression for S2 with no evidence of a block effect in (BF0 = 6.6).
Time–frequency power spectra of LFP signals recorded during the same–different task (same conventions as Figure 5).
In conclusion, even when repetition was task relevant and modulated task performance, it did not affect the repetition suppression for spiking activity or LFP signals.
Estimating population effects
Finally, we used a multilevel model to combine spiking activity data of both monkeys while still accounting for the variability across animals. Multilevel models are the recommended approach for combining nested data when multiple neurons are collected per animal to make a valid statistical inference about the population (Aarts et al., 2014). The resulting population-level estimates (Fig. 8) indicate clear evidence of stimulus-specific repetition suppression (coefficient b2 for regressor T) regardless of the task (Experiments 1 and 2) and in both single-unit and multiunit responses. In addition, for the orthogonal task (Experiment 1), but not for the same–different task (Experiment 2), there was a significant suppression for the second stimulus in alternation trials as well (coefficient b1 for regressor S). Importantly, there was no evidence of increased stimulus-specific adaptation in repetition blocks in which stimulus repetitions would be expected. This would be indicated by a negative value for coefficient b5 for the interaction regressor T × B. On the contrary, at best, there was a nonsignificant trend toward positive values for coefficient b5 for single-unit data (Experiment 1), which would indicate less suppression in repetition blocks. Finally, there was no evidence of an effect of block on S1 (coefficient b3 for regressor B) or for an interaction between block and suppression for S2 in alternation trials (coefficient b1 for regressor S x B).
Multilevel model effect estimates at population level (across subjects) for spiking activity. Black markers indicate point estimates of posterior means and 95% intervals (error bars) on a log-firing rate scale. Stimulus-specific adaptation [i.e., the difference between repetition trial (RT) and alternation trial (AT) for S2] is captured by b2. Any block effect on stimulus-specific adaptation is captured by b5 (negative values indicate more suppression in repetition blocks). The intercept b0 captures the response to S1, b1 the difference between S2 and S1 for alternation trials (i.e., stimulus order effect), b3 the difference between repetition block and alternation block for S1, and b4 any block effect on the difference between S2 and S1 for alternation trials. Corresponding subject averages across cells/sites are indicated for Monkey G (red) and Monkey D (blue).
Discussion
We investigated the effect of face repetition probability on repetition suppression of spiking activity and LFPs in (putative) ML of macaque IT cortex. The monkeys were required to attend to stimulus content while face repetition probability was manipulated. Despite the use of faces and attention to stimulus content, we found no evidence in any electrophysiological signal in favor of an effect of repetition probability on repetition suppression and this was consistent in the two monkeys. Furthermore, even when repetition was made task relevant and repetition probability affected the behavior of the animal, no neural signature of repetition probability was present in (putative) ML. In all experiments, repetition suppression was present but its size was unaffected by repetition probability.
The results of our electrophysiological data are consistent with Kaliukhovich and Vogels (2011). Following recent human fMRI studies, here, we improved the procedure by including two supposedly crucial conditions for observing the repetition probability effect: a task directing attention toward the stimuli (and in Experiment 2 the repetition) (Larsson and Smith, 2012) and the use of faces, which might form a special stimulus category based on prior experience (Grotheer and Kovács, 2014). Therefore, neither attention nor face stimuli were sufficient conditions for observing a perceptual expectation effect in electrophysiological responses in face patch ML.
Expectation effects in macaque IT
Kaliukhovich and Vogels (2014) demonstrated that macaque IT neurons show no expectation-related surprise response to deviants in visual oddball sequences during passive fixation. In contrast, a recent study did report effects of perceptual expectations on responses of mostly face-selective macaque IT neurons (Bell et al., 2016). The authors claimed that their effects were distinct from low-level sensory adaptation based on a multivariate regression analysis in which they attempted to control for repetition suppression. They argued that expectation effects might have been absent in previous studies of spiking activity because of a lack of attentional requirements of the task. However, the present series of experiments excludes this possibility because our tasks required that the monkeys attend the face stimuli. Recently (Vinken and Vogels, 2017), we have shown that the analysis of Bell et al. (2016) did not properly control for repetition suppression. Indeed, we obtained the same putative expectation effects in simulated neurons that only included mechanisms of adaptation that, unlike the conclusion by Bell et al. (2017), can accumulate across stimulus presentations in the course of several seconds (Sawamura et al., 2006; Kaliukhovich and Vogels, 2014). Therefore, we argue that the results of Bell et al. (2017) do not require a perceptual expectation account, but merely reflect repetition suppression. Although repetition suppression and expectation effects are viewed as unitary phenomena under a predictive coding framework, we suggest a cautious approach by distinguishing expectation from repetition suppression or adaptation. The latter depends on the similarity between successive stimuli (Jiang et al., 2006; Verhoef et al., 2008; De Baene and Vogels, 2010; Natu et al., 2016), whereas expectations are independent of the similarity between the predictive and predicted stimulus and can be generated for arbitrary related stimuli (Meyer and Olson, 2011; Kumar et al., 2017), even from different sensory modalities (Kok et al., 2012, 2013, 2017). Furthermore, expectation and repetition suppression effects can be dissociated during the course of the neural response with MEG (Todorovic and de Lange, 2012) and appear to be additive in human BOLD activations (Grotheer and Kovács, 2015).
Other studies demonstrated expectation effects in macaque IT cortex (Meyer and Olson, 2011; Kumar et al., 2017), including ML (Schwiedrzik and Freiwald, 2017), which could not be explained by repetition suppression. In those studies, expectations were induced by exposing the animals to the same fixed sequence of different stimuli for multiple daily sessions instead of within-session, relatively brief manipulations of stimulus repetition as done here and by Kaliukhovich and Vogels (2014) and Bell et al. (2016). This raises the interesting possibility that expression of neural expectation effects in IT may require longer, multiple session exposure to stimulus sequences. Note that the monkey's behavior does adapt to the within-session changes in repetition probability (Bell et al., 2016; present data: Experiment 2), implying that the lack of an effect of repetition probability is not because of a failure of the monkey to detect the short-term changes in repetition probability.
Effect of repetition probability and repetition suppression
Olkkonen et al. (2017) failed to find an effect of repetition probability on repetition suppression for faces in human FFA with fMRI despite the presence of a behavioral effect of the repetition probability manipulation in a separate set of subjects. They suggested that a possible explanation for the discrepancy between their study and other studies that did report an effect of repetition probability might be the use of computer-generated faces. They argue that these stimuli might not have attracted sufficient attention compared with real faces during their orthogonal size judgment task. Whatever the cause of the absence of a repetition probability effect, they did find repetition suppression, showing that the latter can be independent of repetition probability. Their fMRI results in humans fully agree with our monkey IT data. Furthermore, our finding of a behavioral effect of repetition probability for the same stimulus presentations in the same subjects rules out their attentional explanation. Note that it is unlikely that the use of human, artificially generated faces can explain the lack of expectation on repetition suppression in our monkey study. Indeed, perturbation studies suggest that ML plays a causal role in the perception of artificially generated human faces (Afraz et al., 2015; Moeller et al., 2017) and “natural” human faces (Sadagopan et al., 2017). Furthermore, Schwiedrzik and Freiwald (2017) showed expectation-related effects in ML for artificial human faces generated with the same software as in our study.
At present it is unclear why some human fMRI and EEG studies (Summerfield et al., 2008, 2011) found a modulation of repetition suppression by repetition probability, whereas other fMRI studies did not. Given the publication bias against negative findings, we cannot know how many unpublished studies failed to replicate the modulation of repetition suppression by repetition probability. Therefore, we do not know how general this effect is. Previous human fMRI studies suggested that the modulation of repetition suppression by repetition probability may depend on factors such as attention (Larsson and Smith, 2012) and stimulus category (Kovács et al., 2013; Grotheer and Kovács, 2014). The present study demonstrates that repetition suppression of spiking activity and LFP power is not modulated by repetition probability in macaque IT even when the subject is attending the stimuli, faces are used, and the repetition probability modulates behavior. All studies that failed to observe a repetition probability effect nonetheless showed repetition suppression. If repetition suppression does result from expectations, then such expectation of repetition must be firmly engrained in the system and not easily modifiable. Such repetition prior may be implemented by bottom-up and local network fatigue-based adaptation mechanisms and top-down mechanisms might at best only have a modulatory function. Therefore, predictive coding theory does not instruct which specific neural mechanisms underlie adaptation.
Between-monkey variability in face patch network
Typically, six or more patches of face selective cortex can be defined with fMRI in the temporal cortex (Tsao et al., 2008). These patches form a system for the processing and perception of faces (Moeller et al., 2008, 2017; Freiwald and Tsao, 2010). However, in Monkey D, we could only find one anterior patch that we presume to be AL based on its location. Also, Tsao et al. (2008) reported that one of their 10 monkeys did not show evidence of face patches despite repeated scanning. It has been shown that the formation of the face patches requires exposure to faces during development (Arcaro et al., 2017). However, Monkey D was not visually deprived during development nor reared in any unusual way. It is unclear whether the face patch organization of Monkey D reflects a natural variation between monkeys of clustering of face-selective neurons and it remains to be investigated whether this has perceptual consequences. Note that the data of Monkey D recorded at the location of her putative ML were entirely consistent with those from the actual face patch ML of Monkey G, both in terms of clear stimulus-specific adaptation and the absence of an expectation effect.
Conclusion
In sum, we investigated whether repetition probability affects repetition suppression of single neurons in monkey IT under two supposedly necessary conditions: a task using face stimuli and a task that requires attention for the stimuli (or repetitions). Even under these specific circumstances, we did not find any effect of repetition probability. The single-unit results were confirmed by recordings of LFPs and multiunit spiking activity. These results further call into question the importance of repetition-induced top-down mechanisms in neural adaptation.
Footnotes
This work was supported by Fonds voor Wetenschappelijk Onderzoek (FWO) Flanders (Projects G058212N, G088916N, and G.00007.12-Odysseus, and doctoral fellowship to K.V.) and KU Leuven (Postdoctoral Mandate PDM/17/186 to K.V.). We thank C. Ulens, I. Puttemans, P. Kayenbergh, G. Meulemans, S. Verstraeten, W. Depuydt, and M. De Paep for technical support.
The authors declare no competing financial interests.
- Correspondence should be addressed to Rufin Vogels, Laboratory for Neurophysiology and Psychophysiology, Department of Neurosciences, KU Leuven, 3000 Leuven, Belgium. rufin.vogels{at}kuleuven.be
