Abstract
The dorsal pathway of the primate brain, especially the middle temporal area (MT or V5) and the superior middle temporal area (MST or V5a), is strongly involved in motion detection. The relation between neural firing rates and psychophysical performance has led to the assumption that the neural code used by these areas consists of the relative discharge rates of neuronal populations. As an additional neural code, temporal correlation of neural activity has been suggested. Our study addresses the involvement of such a code in awake monkeys performing a motion discrimination task.
We found significant temporal correlations between simultaneously recorded pairs of units in areas MT and MST and other extrastriate cortical areas. Units recorded from the same electrode were more frequently synchronized than units recorded from different electrodes placed within the same or different cortical areas. Activity synchronization was present in the expectation period before stimulus presentation and could not be induced de novo by the stimulus. Rather, we found a contrast-dependent reduction of correlation strength on stimulus onset. Correlation strength did not vary systematically with stimulus directions. We conclude that under the conditions of this study, temporal decorrelation of MT and MST neurons could be used to detect the stimulus, but synchronization does not convey specific information about its direction of motion and therefore is unlikely to contribute to performance in our direction discrimination task. Activity synchronization in the period before stimulus onset could be related to attentive expectation.
- synchronization
- MT
- MST
- extrastriate cortex
- cross-correlation
- macaque monkey
- expectation
- attention
- motion detection
The middle temporal area (MT) and the superior middle temporal area (MST) of the superior temporal sulcus are parts of the “dorsal pathway” of the primate brain, the main function of which is motion processing. Neurons in these areas are highly selective for the direction of motion of visual stimuli, both using naturalistic scenes as well as more abstract stimulus patterns, such as bars or dot patterns (Dubner and Zeki, 1971; Maunsell and van Essen, 1983; Albright, 1984; Mikami et al., 1986a,b; Britten et al., 1993; Pekel et al., 1996). Lesions of these areas produce specific deficits in motion detection (Newsome & Pare, 1988; Cowey & Marcar, 1992). Neural activity recorded in MT and MST covaries surprisingly well with the direction of motion a given subject perceives (Newsome et al., 1989; Celebrini & Newsome, 1994; Britten et al., 1992, 1996;Shadlen et al., 1996).
Manipulating the discharge rate of a relatively small amount of neurons by electrical stimulation can bias the decisions of monkeys performing a visual direction discrimination task in favor of the preferred direction of the stimulated population (Salzman et al., 1990, 1992;Salzman & Newsome, 1994; Celebrini & Newsome, 1995). These results led to the suggestion (Salzman & Newsome, 1994; Shadlen et al., 1996) that the direction of stimulus motion is determined on the basis of a comparison between average discharge rates of several neuronal populations (“winner-take-all mechanism”).
In addition to such rate-coding mechanisms, the principle of temporal coding has been introduced to account for more complex features of perception, e.g. coding of global stimulus features (Abeles, 1982; Gray et al., 1989; DeCharms & Merzenich, 1996) or the binding of different features of a stimulus in order to produce the perception of a single coherent object (von der Malsburg, 1981, 1995; Eckhorn et al., 1988;Gray et al., 1989; Engel et al., 1990, 1991b, 1992; Eckhorn & Obermueller, 1993). Activity synchronization has been observed between neurons lying close to each other as well as separated by many millimeters in many cortical areas in anesthetized and awake animals (Frostig et al., 1983; Krüger & Aiple, 1988; Gray et al., 1989;Hata et al., 1991; Bressler et al., 1993; Eggermont & Smith, 1996;Livingstone, 1996; Tamura et al., 1996; Gray & Viana Di Prisco, 1997). Concerning the dorsal pathway of the macaque, Kreiter and Singer (1992,1996) have already demonstrated that temporal coupling is also present between neurons in area MT of the awake macaque monkey. The question now arises of whether synchronization of neural activity carries specific information about the direction of stimulus motion. We assessed this question by studying temporal relations between the activities of simultaneously recorded neurons in awake behaving monkeys performing a direction discrimination task.
MATERIALS AND METHODS
Preparation of experimental animals. Two adult rhesus monkeys (one female and one male Macaca mulatta) were used in this study. All treatments of experimental animals were performed with the greatest possible care to avoid pain and distress and were in full compliance with the guidelines of the National Institutes of Health for the care and use of laboratory animals and of the European Community (EUVD 86/609/EEC). The monkeys were trained in the direction discrimination task described below until they reached a stable level of performance, which was significantly above chance level. In a single surgery session, scleral search coils, a head restraint, and two recording chambers were implanted under deep pentobartital anesthesia. Surgical procedures were performed under strictly sterile conditions. The recording chambers (one for each hemisphere) were placed stereotactically onto the skull such that their centers were situated above the central representation of area MT.
Direction discrimination paradigm. The monkeys were trained in a direction discrimination task (Fig.1). Eye movements were monitored by scleral search coils. The monkey was comfortably seated in a primate chair with its head restrained. It faced a rear projection screen (covering 90 ×90° visual angle) onto which a static, structured background pattern was projected. The pattern consisted of two-dimensional Gaussian noise formed by randomly sized black and white areas. This type of background pattern has been used previously and described by Hoffmann et al. (1980) (Fig. 1). The monkey had access to five touch bars in front of its lower chest. It started each trial by touching the central touch bar, after which a fixation point came up in the middle of the screen. Throughout the task, the monkey had to fixate this point with maximal deviations of ±1 to ±2° visual angle. The wider fixation window (2°) was used with monkey H and during the first experiments with monkey A. With prolonged practice of monkey A in the task, fixation window size could be reduced to ±1°. After a randomly chosen time interval (in steps of 600 msec, between 600 and 3000 msec after the trial began), a moving stimulus appeared on the screen consisting of evenly spaced white bars. Stimulus position and size could be deliberately chosen such that it covered all receptive fields of the neurons recorded simultaneously (e.g., if in an extreme case one receptive field covered the upper left quadrant and the other covered the lower right quadrant, the stimulus was adjusted such that it covered the whole screen). In monkey H, stimulus velocity mostly was either 14.4 or 29.6°/sec, in monkey A 18.2°/sec, which is close to the average preferred stimulus velocity in area MT. The stimulus pattern moved in one of the four cardinal directions. Various contrast levels were used: 0%, meaning that no stimulus was presented at all, 2, 3, and 4%, which are close to perception threshold, and 17, 24, and 53%, which are well above threshold.
Visual direction discrimination task. The monkey started each trial by touching the central touch bar (B). After a randomly chosen interval (which we call the expectation period), a moving white bar pattern came on, covering the receptive fields of all units recorded simultaneously (A). After a variable reaction time, the monkey indicated the direction of motion by touching the corresponding detection touch bar. Thereafter, the stimulus remained on for another 500 msec, and only then the monkey received its liquid reward (if it kept fixation during the whole trial and indicated the correct direction of motion) (D). C, Spatial distribution of local contrast in the background pattern, which covered the entire screen and onto which the stimulus was superimposed. A photograph of the type of background pattern plus additional superimposed bar stimulus can be found in an article by Hoffmann et al. (1980).
Normally, three different contrast levels were tested in each recording session: two contrast levels near perceptual threshold and one contrast level above threshold. In each recording, typically between 100 and 550 stimuli were presented. Given that there were 13 different stimulus conditions (four different stimulus directions, each of which was presented at three different contrast levels, plus one condition in which no stimulus was presented at all), this yielded 8–42 trials for each of these conditions. The monkey was trained to indicate the direction of stimulus motion as soon as possible by a hand movement to one of the four indication touch bars. After touch, the stimulus remained on for another 500 msec during which the monkey had to maintain fixation. Only after that period and after indication of the correct direction of motion, a liquid reward was delivered. Spike trains and experimental control signals were stored digitally with a temporal resolution of 1000 Hz (spike trains and stimulus onset) or 500 Hz (release and touch of touch bars).
Recording technique. Neuronal responses of multiple single units were simultaneously recorded by aid of a multielectrode recording matrix (Thomas Recordings, Marburg, Germany), using up to four electrodes, from each of which up to three different units were isolated by spike sorters (Alpha Omega, Jerusalem, Israel; and Spectrum Scientific, Dallas, TX). The distance between two neighboring electrodes was 300 μm; thus, the maximal lateral distance between two recording sites (using four electrodes) was 900 μm. Recording sites were aimed at areas MT and MST using response characteristics, receptive field sizes, penetration scheme, and recording depths as landmarks. All units that could be well isolated were recorded. By this procedure, we not only encountered “classical” MT or MST cells responding well to the stimulus but also not infrequently captured the activity of cells not responding to any stimulus direction at all. Spike isolation was performed based on spike shape and optimized considering interspike interval distributions, which were continuously displayed during recording. In the following, we use the term “unit” for the signals of cells determined by the spike separation procedure. A few multiple units were included in this study, if their separation from the simultaneously recorded single units was confirmed.
Histology. During the experiment, electrolytic lesions and tracer injections (horseradish peroxidase, fluorogold, and rhodamine-coated latex beads) were placed at interesting recording sites. After completion of the experiments, the brains were histologically processed. Areal boundaries were determined based on myeloarchitecture and SMI 32 immunohistochemistry (Hof & Morrison, 1995). Recording sites were reconstructed on the basis of penetration tracks and records of recording depth. To avoid possible subjective biases, histological processing and anatomical reconstruction of recording sites was performed by an independent person (Dr. C. Distler) not involved in the electrophysiological recordings.
Data analysis. Cross-correlations were computed off-line by aid of the Matlab software package (MathWorks Inc., Natick, MA). Each of the different stimulus conditions was analyzed separately. For analysis of the activity before stimulus presentation, all trials could be considered together. The algorithm for estimation of cross-correlations was the following: out of a given pair, one unit was taken as trigger unit (unit 1). For each of the spikes fired by this unit (within the time window under study), the time delays to each of the spikes of the other unit (unit 2) were calculated and plotted in a histogram of typically ±100 msec delay and a bin width of 1 msec. This procedure was repeated for all spikes and all trials, summing up all entries and yielding the “raw cross-coincidence histogram” (RCCH).
To allow for comparison with correlograms constructed under different conditions (especially with altered discharge rates of the two units induced by the stimulus), it was necessary to normalize the RCCHs to a score that was independent from firing rates of the two units. We chose the so-called Z score for normalization, because it has been shown that it produces estimates of correlation strength that are quite independent from firing rates and reliably reflect the real functional connectivity in a given neural architecture (Aertsen et al., 1989). It is calculated by subtracting the theoretically expected value and subsequently deviding by the expected SD. Under the assumption that both units fire independently and that their firing is random and poisson-distributed, the expected value can be determined from the firing rates of both units (e.g., if one unit fired at 5 Hz and the other at 1 Hz, one would expect five intervals in a period of 1 sec and 5/1000 for each bin of 1 msec width), and the SD is the square root of the expected value (Eggermont & Smith, 1996). We took Zscores >3 as statistically significant deviations from the null hypothesis (of two independent random poisson processes). Of course, as in any statistical process, deviations of more than three times the SD do occur from time to time randomly and would lead to false-positives in our significance measure. To avoid these false positives, we performed two tests: (1) False-positives often consist of single bins exceeding the confidence limits. We smoothed correlograms by a three-point averaging filter, and only if the resulting correlograms still had peak heights >3 (in the Z score), they were scored as significant correlations. (2) We divided the trials for a given condition in two subgroups. Only if in both groups a significant correlation occurred, the pair was scored as significantly correlated.
In addition, we calculated the shuffle predictor (arrived at by correlating subsequent trials with each other and the last trial with the first one). The shuffle predictor is usually interpreted as an estimate for correlogram features induced by an influence repeating itself identically for all trials (typically, the stimulus). Usually, the shuffle predictor in our correlograms was flat (with a certain amount of random jitter), and its mean value corresponded well with the expected value calculated from firing rates (e.g., see fig.2). Whenever the shuffle predictor was not flat, we calculated the Z score by subtracting the shuffle predictor and deviding by the (empirically measured) SD of the predictor. For quantification of correlation strength, correlograms were smoothed by a three-point averaging filter to reduced noise. Correlation strength was always determined as the peak height (maximal value) in smoothed, normalized correlograms.
Examples of positive correlations found between neuronal pairs in the expectation period before the stimulus came on. The shaded area indicates the confidence interval for statistically independent random firing. The white lineindicates the shift predictor. A, Example of a correlogram displaying oscillatory side peaks in the γ range. Units were from one electrode in area MT and shared the same preferred direction.B, Example of a correlogram with a single, centered peak. Units were from one electrode in area MT, and preferred directions differed by ∼90°. This was one of the broadest correlation peaks we observed. C, Example of relatively weak coupling with a displaced peak occurring frequently with pairs from different electrodes (in this example, units also were from different areas, namely one in MT and one in V3a).
Only those pairs were considered for further analysis which yielded RCCHs (calculated for the expectation period in all trials) with at least 1000 entries. Note that by this procedure, even cells with very low spontaneous rates could be included in our sample, as long as the number of trials was sufficiently high (e.g., when, as in the typical case, 200 trials were recorded, the spontaneous rate of both cells had to be only 0.7 spikes per trial of 600 msec duration to ensure a sufficient number of entries). For comparison between temporal coupling strength during the expectation period to the one during stimulation, we chose those cell pairs that had a minimum number of 1000 entries for both conditions. We chose the number of entries as the critical parameter, because the signal-to-noise ratio in correlograms depends clearly on this parameter more than, e.g., the number of trials or discharge rates alone. For the investigation of contrast dependence of correlation strength, we calculated the relative correlation strength for each stimulus condition compared with the spontaneous activity in the same trials (calculated by the peak height in Z score during stimulus-driven activity divided by the one obtained during the expectation phase). This procedure allowed us to exclude that trial-to-trial changes of the monkey’s state influenced the results. Of course, however, this led to a dramatic decrease of data, because per stimulus condition, typically only some tens of trials were available (see above). Only cell pairs in which at least 300 entries were available in both correlograms (before and after stimulus presentation) were evaluated for this approach.
To assess the time course of correlogram changes, we used a sliding window technique: correlograms were constructed for time windows of 500 msec length, which were moved in 50 msec steps over the data. Because in this analysis, nonflat shift predictors were sometimes encountered (because of rate increases after stimulus onset), the shuffle predictor and its empirically measured SD were used for normalization. The results were displayed as three-dimensional plots of correlograms along time. By other authors, the term “peristimulus time cross-coincidence histogram” PSCCH was coined for this kind of display (Nowak et al., 1995). Temporal resolution of this procedure of course is inferior to the joint-PSTH method introduced by Aertsen et al. (1989), but the latter requires a larger number of events to produce reliable results and therefore was not suited for our data. Direction selectivity of single units was assessed by calculating a direction selectivity index, DI = 1 − (activity in null-direction/activity in preferred direction), after subtracting spontaneous activity. Units were regarded as direction selective whenever DI exceeded 0.5 (indicating that activity in preferred direction was at least twice as big as that in null direction). The preferred directions (PDs) of units with a DI exceeding 0.5 were interpolated between the four directions tested by calculating the first trigonometric moment of the respective responses (Thiele & Hoffmann, 1996). PDs first were calculated for each level of contrast separately, and then the median, or, in case of only two values, the mean was taken.
RESULTS
After completion of training in the direction discrimination task, both monkeys had achieved a stable performance level, which was significantly above chance. Psychophysical data on reaction times and performance of the monkeys at the different contrast levels will be described in a separate, forthcoming study. For this study, 354 pairs (from 450 units, 276 of which were recorded from monkey A and 174 from monkey H) were analyzed. After histological processing, electrode tracks were reconstructed, and areal locations of recording sites were determined. Most of the units were situated in areas MT or MST (n = 258), but a considerable number of units were also located in other extrastriate areas [V2, V3, V4, the posterior part of the polysensory area of the superior temporal sulcus (STPp), the floor of the superior temporal sulcus (STPf), the lateral intrapariatal area (LIP), the ventral intraparietal area (VIP), and in the lateral sulcus (LS); n = 157; Table1; the areal location of the residual units could not be determined].
Number of units recorded and incidence of synchronization
For each cell pair, between 100 and 550 trials were recorded, yielding 8–42 trials for each of the 13 different stimulus conditions usually tested (three contrast levels times four different directions of movement, plus the situation in which no stimulus was presented). Considering median discharge rates of 8.9 Hz before stimulus onset and 19.3 Hz with the best stimulus, the number of spikes available (in the 600 msec time windows we used; see below) was between 500 and 3000 spikes for spontaneous activity and between 1000 and 6000 for stimulus-driven activity.
To get a notion about the baseline synchronization without the presence of the moving bar pattern, we analyzed cross-correlations during the period while the monkey was waiting for the stimulus to appear and only the stationary background pattern was present in the visual field (we will call this the “expectation period”). The incidence and percentage of cell pairs showing statistically significant correlation (see Materials and Methods) during the expectation period is shown in Table 1 and Fig. 3. On the whole, 130 pairs (37%) showed significant correlation. The residual 224 pairs (63%) were not significantly correlated. Examples of correlograms obtained in the waiting period are depicted in Fig. 2. In most cases, single peaks straddling the origin were observed. For pairs separated from the same electrode, it was of course only possible to detect one spike at a given time step (in our case, of 0.8 msec duration). Therefore, zero bins were inevitably underestimated in these cases. Figure3 compares the percentage of cell pairs with or without significant correlation for different subsamples. No difference was found between the respective percentages in areas MT and MST compared with the other extrastriate areas recorded. In cell pairs recorded from the same electrode and within the same cortical area, more pairs were found to be significantly correlated than in cell pairs from different electrodes and different areas. Significant coupling between different areas was only rarely found (six pairs: two cases between V3a and MT, two between areas MT and MST, one between LS and STPp, and one between MT and STPp). One could speculate whether this indicates an especially tight coupling between these areas. Because of the relatively small number of cell pairs from different areas (n = 38), however, we would be very hesitant about such an interpretation. Inhibitory interaction indicated by a negative correlation was very rarely observed (we found only one case of a clear trough in a correlogram). This is in agreement with the general finding that inhibitory interactions are much more difficult to detect by correlation analyses than excitatory ones (Aertsen & Gerstein, 1985;Melssen & Epping, 1987).
Percentage of pairs exhibiting significant (white) or no (black) correlation during the expectation period. The numbers above the bars show the absolute numbers of correlated pairs of the total numbers (inparentheses).
Correlograms with secondary peaks, which can under certain conditions indicate damped oscillations, were observed only in a minority of cases (an example of which is shown Fig. 2A). Corresponding oscillation frequencies were evaluated by displaying the power spectrum (for frequencies between 10 and 100 Hz) after Fourier transformation. Because of the low modulation present in our sample, peaks observed in the power spectra were relatively small. Out of the five cases with clear side peaks, three had frequencies in the β band (between 13 and 16 Hz), and two had frequencies in the γ band (40 and 46 Hz). From the absence of side peaks in the correlograms of the other cell pairs, however, it cannot be concluded that oscillatory coupling was not present at all. Kreiter and Singer (1992, 1996) reported for area MT and Murthy and Fetz (1996b) for the sensorimotor cortex that oscillations occur in relatively brief episodes interleaved with nonoscillatory activity. Together with fluctuations in oscillation frequency, this would smooth out satellite peaks. Because we did not sample local field potentials during our recordings, we did not have the possibility to analyze the periods of “oscillating” activity selectively, as has been done by Murthy and Fetz (1996a, 1996b)recently.
Does temporal coupling link units with similar stimulus preferences?
We tried to answer the question of whether there was any relation between the incidence of temporal correlation and the PDs of the two units in a given pair. First qualitative inspection of the data revealed that examples of positive coupling between units could be found not only for units sharing preferred directions (Fig.2A) but also for pairs differing 90° (Fig.2B) or even 180° (Fig. 2C) in preferred directions. To assess the relation between preferred directions and the incidence of synchronization more quantitatively, we interpolated the preferred direction of each unit by calculating the first trigonometric moment of the measured responses (for those units in which the direction selectivity index DI exceeded 0.5; see Materials and Methods). We plotted the number of pairs showing significant or no correlation against the differences between the preferred directions of the two units (Fig. 4). Because we hardly ever found significant correlation during stimulation, we evaluated the temporal correlation during the expectation period for this approach. While the distribution of uncorrelated pairs was nearly uniform, coupled units clustered at smaller differences. At first sight, this might suggest that cell pairs with similar preferred directions were preferentially coupled. Closer inspection of the data, however, revealed that the distributions of coupled and uncoupled cells for pairs from different electrodes were equally uniform over the whole range of 0–180° difference in preferred directions. For pairs from the same electrode, both coupled and uncoupled pairs only had differences in preferred directions of up to ∼90°. This finding could have been expected, because the representation of movement directions is known to be clustered in areas MT and MST (Albright et al., 1984; Celebrini & Newsome, 1995). Thus, the high incidence of correlated pairs with small differences between preferred directions can be explained by the higher percentage of coupled pairs recorded from the same electrode. The low percentage of coupling between pairs with diverging preferred directions can be attributed to the fact that the probability of coupling seems to decrease dramatically with increasing distance between the two units. It might be interesting to mention that we also encountered positively correlated pairs, consisting of one unit responding in a directionally tuned way to the stimulus, and another cell that did not respond at all. We conclude that the probability of synchronization (during the expectation phase) between two given units of our sample depended more on their spatial proximity than on the similarity of stimulus preferences.
Incidence of temporal coupling during the expectation period as a function of stimulus preference. Only direction-selective pairs were included in this graph. The number of pairs is plotted against the difference between the two preferred directions. Both uncoupled pairs (black bars;n = 37) and synchronized pairs recorded from two electrodes (gray bars; n = 15) had a relatively uniform distribution over the whole range of direction differences. Synchronized pairs from one electrode were preferentially found at smaller differences in preferred directions (white bars; n = 36).
Quantitative description of positive correlations
To describe the correlograms observed in our sample, we used three quantitative measures: (1) Peak height in the normalized correlogram (Fig. 5A). Peak heights of all positively correlated pairs had a median of 5.71. (2) Position of the peak. Peaks usually straddled the origin and were located close to zero time delay. The distribution of peak positions (Fig. 5B) shows that the maximal deviation from zero was 24 msec, and the median was 1 msec. In pairs from one electrode, of course no peaks at time delay zero could be encountered, because only one spike could be detected at a given time (see above). Peaks at zero delay were, however, observed in pairs from two electrodes. (3) Peak width. Peak width was assessed by measuring its half-width at half-height over the offset (given by the theoretically expected value; see Materials and Methods; Fig. 5C). Values for peak widths ranged between 0.5 and 36 msec, with a median of 7.5 msec.
Quantification of correlation parameters obtained during the expectation period. On the left, scatter plots indicate values of all pairs analyzed. On the right, the frequency of values is displayed as a histogram. Black dotsare pairs from areas MT and MST and one electrode; gray dotsare from other areas and one electrode; and white dots are from two electrodes (regardless of areas). Median values are marked byasterisks. A, Peak heights: B, peak positions; C, peak widths (half-width at half-height).
Comparing the quantitative parameters of temporal synchronization between MT–MST pairs with those from other areas, we found no significant differences in any of the parameters investigated. There were, however, differences between pairs recorded from the same and those from different electrodes: Pairs recorded from different electrodes had significantly smaller and wider peaks and showed a higher variability in peak position (median peak height for pairs from one electrode, 6.49; from two electrodes, 4.43; p = 0.0015; median peak width for one electrode: 5.75; for two electrodes, 12.00; p < 0.0001, rank sum test). The highest time delays of correlogram peaks were encountered in the group from two electrodes.
Correlation strength is reduced by the visual stimulus
How is temporal coupling between neurons affected by visual stimulation? One would perhaps expect the most drastic effect for the preferred stimulus, i.e., the stimulus eliciting the strongest response in terms of discharge rates. We compared correlograms obtained from 600-msec-long time windows immediately before stimulus onset and after onset of the preferred stimulus. Figure 6shows two examples of stimulus responses and changes in temporal correlations between pairs in area MT. Surprisingly, in all cases in which significant coupling occurred, it was already present before onset of the moving bar pattern. In most cases, correlation strength (assessed by peak height) was reduced on stimulation. In no case did we see a de novo induction or enhancement of positive correlation in the stimulus-driven activity. At one single recording site we observed an oscillatory cross-correlation in the γ frequency range during stimulation. Also in this case, however, the central peak was clearly reduced during stimulation compared with the expectation period. Whenever no correlation was present before stimulus onset, correlograms remained flat also during stimulation. To quantify the differences of coupling strength between the waiting period and stimulus-driven activity, we compared normalized peak heights from correlograms constructed under the two conditions (Fig.7, for this test, we chose only cell pairs in which both cells responded to the stimulus). Median normalized peak height for all pairs decreased from 5.72 before to 1.63 during stimulation (n = 64). The difference between these two conditions was highly significant (p < 0.0001, signed rank test). The general and significant trend of decreased synchrony within the stimulus period was found in all areas investigated, suggesting that it might be a generalized phenomenon (pairs within area MT: 6.2 before, 1.6 with stimulus; p< 0.0001; n = 28; pairs within area MST: 5.8 before, 1.6 with stimulus; p = 0.0005; n = 12; pairs with at least one unit not situated in areas MT or MST: 6.04 before, 1.7 with stimulus; p < 0.0001;n = 17).
Two examples of activity correlation before and during stimulation (A, B). The middle panel shows the responses (PSTHs with a bin width of 25 msec) of the two units aligned to stimulus onset (0); stimulus contrast is 4% (for clarity, the response of one cell is displayed in grayand the other in black). Correlograms between the activities of the two pairs are shown for spontaneous activity (600 msec before stimulus onset, all trials included. top panel) and stimulus-driven activity (0–600 msec after onset of optimal stimulus;bottom panel). A, Two MT units recorded from the same electrode with 90° difference between preferred directions. B, Two MT units with similar preferred directions (3° difference; direction selectivity indices, 1.1 and 1.0).
Comparison of correlation strength measured during 600 msec before and 600 msec after the onset of the visual stimulus evoking the strongest response in both units (“best stimulus,”n = 64). As an estimator for correlation strength, we used normalized peak amplitude (Z score). The line of unity is dashed.
In only four (of 64) cell pairs, correlation strength was reduced to a level that still exceeded the significance level. Interestingly, all these cell pairs were recorded from the same electrode and had preferred directions differing by <50°. However, we found no correlation between the amount of decrease in synchronization strength on visual stimulation and the difference of preferred directions of the two cells in a given pair (data not shown). We therefore assume that this finding merely reflects the especially strong correlations found in these pairs before stimulation.
Time course of correlation changes
As a next step we aimed at investigating the time course of correlation changes on visual stimulation. A general disadvantage of cross-correlation analyses is the absence of temporal resolution. To overcome this disadvantage at least partly, we used a sliding window technique: correlograms were constructed for sliding windows (500 msec width) moved in steps of 50 msec over the data. Only trials with identical stimulus conditions were used (best stimulus). Figure8 shows a typical result, calculated from a pair consisting of one single unit and multiunit activity (containing two or three units) recorded from the same electrode in area MST. Correlograms constructed for each time window were plotted at their centers. Before the stimulus came on (i.e., in the expectation period), there was a clear and highly significant correlation peak at zero delay, which was strongly reduced on stimulus onset and thereafter remained at a more or less constant level. For comparison, we also calculated the discharge rates in the same time windows used for the construction of sliding window correlograms (Fig. 8, top right panel). The sudden drop of correlation strength at time 0 coincides with the first rate change observed in the corresponding time window (please note that the correlogram plotted at time 0 was constructed by analyzing the time between −250 and +250 msec before and after stimulus onset, respectively. The rate change at time 0 is produced by the fact that the units analyzed here had visual latencies slightly <250 msec; this long latency was attributable to the very low contrast of the stimulus). In general, the time course of correlation revealed by this analysis was a smooth and monotonic transition between the state during the expectation period and the state during stimulation.
Left, Time course of correlation changes. This example was calculated from one single unit and one multiunit separated from the same electrode in area MST. Correlograms were constructed for sliding windows (500 msec width) moved in steps of 50 msec and plotted at the center of the time window. Zeroat the time axis represents stimulus onset. Thecolor codes the amplitudes of correlogram values. Stimulation is in the preferred direction of both units and with 3% contrast. Correlograms are normalized using the Z score. The highly significant peak observed before stimulus presentation breaks down during stimulation. Top right, Time course of discharge rates. Bottom right, Time course of the correlation peak height.
Correlation strength decreases with increasing stimulus contrast
Figure 9 shows a typical example of how correlation strength varied with increasing stimulus contrast. In trials in which no stimulus was presented at all, a clear peak was visible in the center of the correlograms, accompanied by additional side peaks. With increasing contrast levels, firing rates of the two cells gradually rose and at the same time, correlation dropped until it was virtually abolished at 4% contrast. Such a gradual reduction of correlation strength occurring concomitantly with increasing stimulus contrast was consistently found throughout the data set. To assess the relation between stimulus contrast and correlation strength quantitatively, we calculated median relative correlation strengths for different contrast levels (Fig. 10). To avoid any confounding effects caused by intertrial variability on synchronization levels, we constructed correlograms for the expectation phase and the stimulation phase in the same trials. Relative correlation strength during stimulation was then expressed as the ratio of normalized peak height during stimulation divided by the one before stimulation. A relative correlation value of 1 would indicate that correlation strength was not affected by the stimulus, values <1 indicate that correlation decreased under stimulation. For this analysis, we chose 16 cell pairs that responded well to the stimulus and had a relatively high spontaneous discharge rate (needed to ensure a sufficient number of entries in correlograms of the expectation phase).
Example of correlograms calculated for increasing contrast levels. This pair comprised single unit and multiunit activity, which were recorded from one electrode in area MST. The visual stimulus always moved in the preferred direction of both units.Top, PSTHs for different contrast levels (aligned to stimulus onset = 0). The multiunit response is shown ingray and the single unit response black.0% indicates that no stimulus was presented.Bottom, Correlograms for the conditions depicted above.
Comparison of correlation strength for stimuli presented at various contrast levels (preferred directions only;n = 10 for monkey A and 6 for monkey H). Histograms show median relative correlation strength (calculated by dividing normalized peak height during the first 600 msec of stimulus-driven activity by the one measured during the 600 msec just before stimulus onset in the same trials) for increasing stimulus contrast. Asterisks mark bars with significant deviations from 1 (signed rank test, p = 0.05).
For both monkeys, relative correlation strength was reduced gradually to about half of its original strength with highest stimulus contrasts. The same effect was observed when alternative methods of quantification were used (e.g., peak height divided by offset). The decrease in correlation strength was found to be significant (p < 0.05, signed rank test) whenever stimulus contrast was at least 4%, which corresponded to perceptual threshold in monkey H and was slightly above threshold in monkey A.
Temporal correlation is not directionally tuned
Perhaps the most important point with respect to the question of whether temporal correlation contributed to our direction discrimination task is whether it is systematically related to the direction of stimulus motion. Figure 11compares the rate responses of two units out of area MT with the correlograms obtained with the four different stimulus directions. Although discharge rates differed dramatically between stimulus directions, correlation strength was equally reduced for all directions compared with before stimulation. A common way to assess the overall output of a population of directionally tuned neurons is to construct a population tuning curve. This is done by aligning the responses of all neurons to their preferred direction and expressing response strength as the mean activity relative to the response to the preferred direction of each cell. We aimed at comparing the possible population output based on firing rates to the one based on correlation strength. To construct a “population tuning” based on correlation strength, we chose those pairs of our sample, which (1) were situated in areas MT or MST and showed directionally tuned activity (DI > 0.5), (2) were significantly correlated before stimulation, and (3) shared the same preferred direction (otherwise, the alignment to the preferred direction would not make sense). Fig.12 shows the population tuning constructed from 13 pairs fulfilling these conditions. The firing rates (Fig. 12A) transmit a well directionally tuned signal, whereas no directional tuning is present in correlation strength (Fig. 12B). Thus, correlation strength of neurons in areas MT and MST seems not to convey specific information about the direction of stimulus motion.
Example of responses in a neuronal pair (recorded from one electrode in area MT) to different stimulus directions. PSTHs are shown in A (one cell in gray, the other inblack), correlograms for the different conditions inB. In the middle display of B, correlation during spontaneous activity (all trials) is shown; at thefour sides are correlograms during the first 600 msec of stimulation with the different directions. Display of correlograms is as in Fig. 2.
Comparison of population tuning curves based on firing rates (A) and on correlation strength (peak height in Z score; B). Only MT–MST pairs in which both cells were well directionally tuned (DI > 0.5) and in which the two units had the same preferred direction were included in this plot (n = 13). Preferred directions were aligned upward, and response strengths were normalized with respect to the strongest response in each unit.
Relation between neuronal correlation and residual eye movements
Although the monkeys were trained to maintain fixation during the whole trial period, and all trials in which the monkey’s eyes left the fixation window (of 1–2° visual angle) were discarded, the possibility remained that residual, small-amplitude eye movements occurred within the fixation window. It has been shown that such small-amplitude fixational movements can affect the activity of cells in the superior temporal sulcus of the awake monkey (Bair et al., 1996). The retinal slip induced by these eye movements over the structured background pattern could represent a correlated retinal input that could perhaps account for any activity correlation observed during the expectation phase. To investigate this point, we analyzed two subsets of trials separately: (1) trials without any eye movements occurring during or 300 msec before the time window analyzed (see example in Fig. 13B), and (2) trials with an eye movement occurring in the first 200 msec of the time window analyzed (see example in Fig. 13A). Rate responses of MT and MST cells to retinal slip induced by saccadic eye movements are restricted to 300 msec after the eye movement (A. Thiele, Henning, and K.-P. Hoffmann, unpublished observations). The criterion of an eye movement occurring during the first 200 msec ensured that the time window analyzed comprised at least 400 msec after the eye movement, so that any possibly expected changes could be expected to fall within the time window analyzed. For this analysis, we chose cell pairs in which both cells were clearly visually responsive and that had similar preferred directions, because in these cases any possible influence of common retinal input can be expected to be largest. Correlograms were constructed from equal numbers of trials of these two subsets (for the last 600 msec before stimulus onset). Figure 13,C and D, show correlograms obtained from an example cell pair in trials with or without eye movements. Clearly, there was no difference in the degree of synchronization between the two conditions. Comparing correlation strengths for all 42 cell pairs tested in this way did not reveal any difference between trials with or without eye movements (signed rank test, p = 0.375; Fig. 13E). Taking into account only trials without eye movements during the expectation phase, correlation strength still highly significantly decreased during stimulation with the preferred stimulus (Fig. 13F; signed rank test, p < 0.001). Discharge rates in trials with or without eye movements failed to reveal any significant differences (signed rank test; data not shown). Therefore, the visual stimulus seems to have been either too weak or too short to induce major changes in discharge rate. The main spatial frequency of contrast modulation in the background pattern we used was about one cycle per degree. So, any eye movement of 2–4° maximal amplitude could have induced only very few black to white or white to black transitions, which perhaps were not sufficient to induce any significant rate response in the areas under study. During the stimulation phase, the incidence of residual eye movements was higher than during the waiting period, because the monkey had a tendency to look at the stimulus or even follow its movement by a nystagmic eye movement, especially after it had already indicated its decision.
Comparison of trials containing an eye movement (during the first 200 msec of the time window analyzed; A,C) with those without any residual eye movements (B, D). A, B Horizontal and vertical eye position traces of two example trials. Zero indicates stimulus onset. Correlograms constructed from the two sets of trials of an example cell pair are displayed in C and D.E, Comparison of normalized correlation strength (peak height in Z score) between the two conditions for all 42 cell pairs (with strong visual response and similar stimulus preferences) subjected to this test. F, Comparison of correlation strength (peak height in Z score) of trials without eye movements during the expectation phase with all trials during the stimulation period for the best visual stimulus (eliciting the highest discharge rate; n = 23).
The fact that during visual stimulation correlation was weaker or absent, therefore, would also speak against an induction of correlation by retinal slip (although one might argue that such an effect could, at least in theory, be overridden by the dramatic increase in firing rate induced by the stimulus). In the first recordings of this study, we have sometimes used no background stimulus at all (dark screen) or a diffusely illuminated background. One example of each of these conditions was included in our sample, both of which showed the same stimulus-dependent disruption of activity synchronization. We conclude that correlated visual input induced by residual eye movements in the fixation window cannot account for neural activity synchronization observed during stimulus expectation.
DISCUSSION
Synchronization of neural activity before the onset of the moving stimulus
The first surprising result of this study was that neural activity synchronization occurred before presentation of the moving pattern that had to be detected during the task. During this phase, the only visual stimulus present was the stationary background pattern, which, for the cortical areas of the dorsal pathway represents a rather poor and ineffective “stimulus.” Some cross-correlation studies have revealed significant correlation without stimulation, e.g., in auditory cortex (Eggermont, 1992; Eggermont & Smith, 1996) and in field potentials of various cortical areas in awake, behaving cats (Bouyer et al., 1981; Roelfsema et al., 1997). Other studies described that synchronization in form of synchronized γ oscillations was absent in spontaneous activity and could only be induced by visual stimulation (Eckhorn et al., 1988; Gray et al., 1989; Gray et al., 1989; Kreiter & Singer, 1996; Livingstone, 1996; Gray & Viana Di Prisco, 1997).
Our correlograms typically showed single peaks straddling the origin, indicating a synchronous activation of both units in a given pair. The prevalence of synchronized rather than temporally delayed activities has been reported for various cortical areas (Eckhorn et al., 1988;Gray et al., 1989; Engel et al., 1991a, b; Ahissar et al., 1992;Eggermont, 1992; Nelson et al., 1992; Vaadia & Aertsen, 1992; Bressler et al., 1993; Nowak et al., 1995; Eggermont & Smith, 1996; Kreiter & Singer, 1996) and has been confirmed recently also by intracellular measurements of postsynaptic potentials (Matsumara et al., 1996).
During the period before the presentation of the moving bar pattern, the monkeys in our paradigm had to be highly attentive, because they were expecting a behaviorally relevant stimulus. It has already been suggested by other authors that synchronization (be it of oscillatory nature or not) could be involved in attention, arousal, and anticipation (Freeman, 1975; Crick & Koch, 1990; MacKay & Mendonca, 1995; Makeig & Jung, 1996; Munk et al., 1996; Murthy & Fetz, 1996a;Steriade et al., 1996). Considerable evidence is pointing in this direction. In the motor system, synchronization is higher during movement preparation than during movement itself (Sanes & Donoghue, 1993; MacKay & Mendonca, 1995). In a recent study, coherent oscillations have been observed in motor cortex most frequently during a hold phase during which the monkeys maintained a precision grip and waited for a cue to indicate that they could release the grip (Baker et al., 1997). This is an interesting parallel to the expectation phase in our study, during which the monkeys also had to hold onto the central touch bar and waited for the stimulus to indicate that they could release it and perform the directed arm movement. In field potential recordings from awake, behaving cats, prominent synchronization was observed in the period when the cat waited for the stimulus to change, which could also be a situation comparable to our expectation period (Roelfsema et al., 1997). In EEG data, Basar & Schürmann (1996)and Basar & Bullock (1992) have reported synchronized oscillations (in the α range) locked to the moment when a stimulus is expected (so called “anticipatory α”). Neural synchronization has also been preferentially observed during demanding sensorimotor tasks such as retrieving raisins from unseen locations and much less during the execution of overtrained stereotyped movements (Murthy & Fetz, 1996a).Bouyer et al. (1981) described rhythmic activity in the γ frequency band during focused attentive behavior and immobility. By electrically stimulating the reticular formation (as a source to increase arousal),Munk et al. (1996) found an increase of synchronized oscillations, which, however, in contrast to our study were preferentially observed during stimulus-driven activity. We assume that the expectation of a behaviorally relevant stimulus constitutes a very special state of the animal, which is different from the state of a monkey not expecting a stimulus (e.g., in a pure fixation task) and even more from an anesthetized monkey. This has to be taken into account for comparison of our study with previous descriptions of time structure in areas MT and MST of the monkey (Kreiter & Singer, 1992, 1996).
Reduction of synchronization by the visual stimulus
As a second surprising result, we found that neural activity became less temporally correlated or even totally uncorrelated when the moving visual stimulus appeared. There are few accounts of similar effects in other sensory modalities.
In auditory cortex, positive coupling of neurons could be disrupted by stimulation (Frostig et al., 1983). In an EEG study, oscillatory synchronization in the γ band was found to be reduced after stimulation with expected stimuli (frequently administered acoustic stimuli) compared with unexpected stimuli (Marshall et al., 1996).
The decrease of correlation strength depended on stimulus contrast. Increasing stimulus contrast of course also can lead to an increase in neural activity, suggesting that there could be a causal relationship between these parameters. An inverse relation between mean firing rate and strength of oscillations has been described already by Ghose and Freeman (1992) during low contrast stimulation. Simulation studies (Melssen & Epping, 1987) have shown that under certain conditions, excitatory correlations are reduced with increasing activation of the cells. Our results, however, suggest that the connection between these variables is not so simple. First, we sometimes found reductions in correlation strength already at very low contrasts, which were not accompanied by rate changes. Second, correlation strength was also reduced for the null direction of directionally tuned pairs, for which no rate increase occurred.
At first glance, our results seem to be highly contradictory to the results by Kreiter and Singer (1992, 1996), claiming that activity synchronization in area MT of the monkey cortex is induced by the visual stimulus and is suitable to code for global stimulus features. Our two experimental approaches therefore deserve detailed inspection of any systematic differences. (1) The stimulus conditions differed considerably; Kreiter and Singer used single (although sometimes two) bars, moving relatively slowly (2–6.7°/sec compared with 14–29°/sec in our approach) and no background pattern. Our analysis always included the onset of the rate response, whereas Kreiter and Singer concentrated more on the later, sustained phase of activity. (2) Kreiter and Singer did not perform detailed analysis of correlations in the absence of the visual stimulus. (3) The visual stimulus they used was of no behavioral relevance to the animal, because the monkey was only required to maintain fixation and did not have to detect it. (4) For our study, all well isolated units were recorded and analyzed, as long as they could be well separated from each other and from the background activity. Kreiter and Singer optimized their recordings with respect to visual responses to the stimuli they used. Considering all these differences, we come to the conclusion that the two experimental approaches cannot be compared directly. Further studies are necessary to test the effect each of the single factors mentioned above separately.
Implications of our results for the function of temporal activity correlation
The hypothesis of binding by temporal correlation suggests that temporal relations could be used to link stimulus-specific features across receptive fields. This hypothesis would predict that units with similar stimulus preferences should be coupled to each other. Our results showed that indeed most synchronized pairs had similar preferred direction. This finding, however, could be attributed to the fact that temporal coupling occurred preferentially between neighboring neurons that are known to share stimulus preferences in area MT and MST (Albright et al., 1984; Celebrini & Newsome, 1995).
Another prediction of the binding hypothesis is that temporal synchronization could be used to separate figures from the ground and to disambiguate situations in which various objects are presented at the same time. Neurons responding to or “coding for” the same object should synchronize their activities, whereas those coding for separate objects should be temporally unrelated. The decorrelation we observed on stimulus presentation would indeed be consistent with this hypothesis, because it could be used for segregation between the stimulus and the stationary background used in our task. Because temporal correlation gradually decreased with increasing stimulus contrast, correlation strength could also be exploited to code the strength of a visual input. This hypothesis, however, would require that rate code and temporal code in neuronal activity would bear different information contents; neurons would have to code for the background (in terms of synchronization), although they increase their firing rates in response to the stimulus.
For the performance of the monkey in the task, certainly the most relevant information about the stimulus was its direction of motion. We found that correlation equally decreased for all stimulus directions, and that the population of all pairs showed no directional tuning in its correlation strength. Therefore, we assume that temporal correlation between neurons cannot be used by the system for coding of direction of motion in the areas investigated. Rather, the modulation of firing rates with different stimulus directions constitutes a much better candidate for coding this parameter. Indeed, we have found that discharge rates correlate very well with the performance of the monkey in our paradigm, both in relation to detection errors (Thiele et al., 1996) and during stimulus-independent decisions (Thiele & Hoffmann, 1996).
We conclude that activity synchronization is unlikely to contribute to direction discrimination in the task we used, but desynchronization could be used to detect the presence of the visual stimulus independent from its direction of motion. The high incidence of synchronization during the expectation period could be related to a state of attentive expectation.
Footnotes
This work was supported by the German Science Foundation Deutsche Forschungsgemeinschaft (“Neurovision,” Grant SFB 509, and Graduate Program KOGNET). We thank Ehud Ahissar and Matthias Munk for inspiring discussions in the initial phase of this work and an anonymous referee for suggesting additional tests on our data. We are indebted to Dr. C. Distler for histological processing.
Correspondence should be addressed to S. Cardoso de Oliveira at the above address.
