How neural activity produces our experience of color is controversial, because key behavioral results remain at odds with existing physiological data. One important, unexplained property of perception is selective adaptation to color contrast. Prolonged viewing of colored patterns reduces the perceived intensity of similarly colored patterns but leaves other patterns relatively unaffected. We measured the neural basis of this effect using functional magnetic resonance imaging. Subjects viewed low-contrast test gratings that were either red–green (equal and opposite long- and middle-wavelength cone contrast, L-M) or light–dark (equal, same-sign, long- and middle-wavelength cone contrast, L+M). The two types of test gratings generated approximately equal amounts of neural activity in primary visual cortex (V1) before adaptation. After exposure to high-contrast L-M stimuli, the L-M test grating generated less activity in V1 than the L+M grating. Similarly, after adaptation to a high-contrast L+M grating, the L+M test grating generated less activity than the L-M test grating. Behavioral measures of adaptation using the same stimuli showed a similar pattern of results. Our data suggest that primary visual cortex contains large populations of color-selective neurons that can independently adjust their responsiveness after adaptation. The activity of these neural populations showed effects of adaptation that closely matched perceptual experience.
Color perception results from the action of a neural pathway that extends from the retina far into cortex. Physiological and anatomical studies have revealed many distinct stages in this pathway, including retinal, thalamic, and a series of cortical components (for review, see Gegenfurtner and Sharpe, 1999). Behavioral experiments have also isolated sequential components of color processing, including an intermediate, postreceptoral stage containing three color-opponent mechanisms that signal the relative amounts of red versus green, blue versus yellow, and light versus dark in a stimulus (Hurvich and Jameson, 1957; Cole et al., 1993). These mechanisms appear to linearly combine the long (L), middle (M), and short (S) wavelength cone responses, approximately computing L-M, L-(S+M), and L+M, respectively.
One important property of perceptual color-opponent mechanisms is that they selectively adapt. Previous viewing of a high-contrast L-M pattern, for example, greatly reduces observers' sensitivity to L-M patterns but leaves perception of other patterns relatively unaffected (Krauskopf et al., 1982; Bradley et al., 1988; Shapiro and Zaidi, 1992;Webster and Mollon, 1994). (Under simple, neutral viewing conditions, L-M contrast patterns appear as red–green contrast patterns, i.e. alternate red and green stripes in a grating. Under these same viewing conditions, L+M patterns appear as light–dark patterns). Similarly, adapting to an L+M pattern selectively reduces sensitivity to L+M patterns. This adaptation has a large effect on color constancy (Webster and Mollon, 1995) and may decorrelate neural responses in cortex (Atick et al., 1993).
Precisely how the neural pathways support color perception remains controversial. Retinal ganglion cells and neurons in the lateral geniculate nucleus (LGN) compute linear combinations of cone signals that resemble those of perceptual mechanisms, suggesting that they provide the bases of perceptual color opponency (DeValois et al., 1958;Gouras, 1968; Derrington et al., 1984; Reid and Shapley, 1992). However, activity of these neurons fails to account for several important properties of perceptual mechanisms (Lennie and D'Zmura, 1988; DeValois and DeValois, 1993), including selective adaptation. Critically, neurons in primate lateral geniculate nucleus do not change their response properties after prolonged exposure to contrast (Derrington and Lennie, 1984). Furthermore, these neurons are monocular, whereas selective adaptation to contrast can transfer between eyes (Krauskopf et al., 1982; Webster and Mollon, 1994).
Because retinal and LGN neurons fail to show properties needed to explain color perception, primary visual cortex (V1) was proposed to be the source of the signals underlying perceptual color-opponent mechanisms (Lennie and D'Zmura, 1988; DeValois and DeValois, 1993). However, evidence supporting this claim is incomplete at best. Although V1 neurons do change their responsiveness after exposure to patterns, adapting to color contrast failed to produce consistent effects (Lennie et al., 1990, 1994). Additionally, it is controversial whether large numbers of red–green color-opponent neurons exist in V1 or whether most neurons respond to both L-M and L+M patterns, depending on the spatial properties of the patterns (Thorell et al., 1984; Ts'o and Gilbert, 1988; Lennie et al., 1990).
To advance understanding of the neural bases of color perception, we identified neural populations in cortex that selectively adapted to color contrast. Using functional magnetic resonance imaging (fMRI), cortical responses were measured to L-M and L+M test gratings before and after exposure to high-contrast L-M and L+M adapting gratings. Cortical regions that selectively adapt should show weaker responses to L-M patterns than to L+M patterns after L-M adaptation, and the effect should reverse after L+M adaptation.
MATERIALS AND METHODS
Four subjects participated in each experiment. Subjects viewed drifting sinusoidal gratings while cortical responses were measured with fMRI. Stimuli were either L-M (containing equal and opposite long- and middle-wavelength cone contrast, approximately matching the preferred color of the red–green perceptual mechanism) or L+M (containing equal, same-sign long- and middle-wavelength cone contrast, approximately matching the preferred stimulus of the light–dark mechanism). Two kinds of scans were performed. In no-adaptation scans, presentations of low-contrast stimuli (referred to as “test” stimuli to distinguish them from high-contrast adapters) alternated with presentations of a gray mean field (see Fig. 1). In adaptation scans, the same low-contrast test stimuli alternated with presentations of a high-contrast adapting grating that otherwise had the same spatial and temporal properties as the tests. To strengthen its effect on the test gratings, the adapting grating was presented continuously for 1 min preceding each adaptation scan.
In experiment 1, no-adaptation scans consisted of 20 sec test stimulus presentations in alternation with 20 sec presentations of a gray mean field. During adaptation scans, 20 sec tests alternated with 20 sec presentations of the adapting grating. Six test stimuli were presented in each scan; the test type alternated between L+M and L-M. The order of test presentation was counterbalanced across scans. In experiment 2, test stimulus presentations lasted 4 sec and alternated with 21 sec presentations of either mean field or the adapting grating. Twelve test stimuli were presented in each scan, and the test types were randomly ordered. In each scanning session, subjects performed two no-adaptation scans and four adaptation scans. L-M and L+M adapters were used on separate days to avoid lingering effects of adaptation. To maintain subjects' attention, adapters and tests in experiment 1 and adapters in experiment 2 briefly (250 msec) reduced their contrast at random times, averaging one contrast reduction every 4 sec. Subjects were instructed to monitor for the contrast reductions and press a response key when one was observed.
All stimuli were pairs of 5° circular patches of vertically oriented 0.5 cycle/° grating centered 3° on either side of the fovea. In experiment 1, adapting and test gratings drifted horizontally at 2 Hz and reversed their directions at random intervals whose mean was 0.5 sec. In experiment 2, adapting stimuli drifted at 8 Hz and test stimuli did not drift, but instead contrast reversed at 1 Hz. The parameters used in experiment 2 were selected based on pilot work that attempted to maximize the selectivity of adaptation. L-M tests and adapters had total cone contrasts (Euclidean sum of the three types of cone contrasts) of 0.04 and 0.11, respectively, and L+M tests and adapters had contrasts of 0.07 and 1.2 in experiment 1. L-M tests and adapters had contrasts of 0.035 and 0.09, whereas L+M tests and adapters had contrasts of 0.086 and 0.59 in experiment 2. Test stimuli contrasts were many times detection threshold contrast, which is typically below 0.005 for these types of patterns.
An additional reference scan was used to identify active pixels in visual cortex. The reference scan consisted of patches of a high-contrast (90%) black–white reversing (8 Hz) checkerboard pattern presented in alternation with a uniform gray mean field. The patches were the same size as the test stimuli. Because our pixel size (42.25 mm3) was substantially larger than early cortical organization with respect to either color tuning or temporal frequency tuning, this reference scan was unbiased with respect to our experimental conditions. In all scans, eight slices of fMRI data, taken at a pseudocoronal prescription, were acquired every 2.5 sec (repetition time) using the blood oxygenization level-dependent technique (3 tesla; echo time, 45 msec; flip angle, 80°; voxel size, 3.25 × 3.25 × 4 mm].
As a behavioral measure of adaptation, subjects performed a color-matching task. Subjects adjusted the color and contrast of a stimulus on the unadapted side of the visual field to match the appearance of a test stimulus on the other, adapted side. Stimuli were identical to those used in the fMRI experiments, except that only one of the two adapting gratings was used; a single circular adapting grating was presented on the left side of fixation. The timing of alternations between test and adapting stimuli was also identical to the fMRI studies. Subjects adjusted the color of the matching stimulus (presented to the right of fixation) during the test presentations. Adaptation did not greatly change the perceived color of the stimulus, only the apparent contrast, which was reduced. This allowed us to quantify our data using a single number, the relative reduction (percentage change) in contrast between the appearance match and the actual test stimulus.
In both behavioral and fMRI experiments, subjects viewed the stimuli in a mirror that displayed a rear-projection screen placed either at their feet (experiment 1) or in the bore of the magnet (experiment 2). The stimuli were projected onto the screen from the control room through a window and were generated using a computer controlled LCD projector. The red, green, and blue components of the projector were tested for independence, and color look-up tables that produced linear increases in intensity were computed for each component. The spectra of each component were measured using a spectral radiometer, and cone contrasts of stimuli were computed using these spectra and estimates of the human cone spectral sensitivity (Smith and Pokorny, 1975). Behavioral experiments were performed in the magnet using the same display apparatus. Experiments were conducted within the guidelines provided by the University of California, Los Angeles Human Subjects Protection Committee, which approved the protocol.
To analyze the fMRI data, we computed the average time course for pixels within each visual area that were active (correlation coefficient with a sinusoid above 0.2) in the reference scan. The overall pattern of data did not change when different thresholds were used to determine active pixels. The average fMRI time courses from the no-adaptation and adaptations scans were then divided into blocks for averaging. In the no-adaptation scans, these blocks corresponded to each test presentation and the following mean field presentation, a duration of 40 sec in experiment 1 and 25 sec in experiment 2. In the adaptation scans, the blocks corresponded to each test presentation and the following presentation of the adapter. To generate the time courses shown in Figures 2-4, these blocks were averaged for each stimulus type in each condition, first within and then across subjects, producing grand averages.
Responses were quantified by fitting functions to the average fMRI time course blocks for each stimulus presentation (after averaging the time course within the visual area). Before averaging, the time course of each pixel was converted to percentage change scores by subtracting then dividing by the mean pixel value for each scan. In experiment 1, response amplitudes were computed as the amplitudes of sinusoids that best fit the data. The phase of the sinusoid was fixed and determined by responses in the checkerboard reference scan. Therefore, peaks in the fMRI time course generated positive amplitudes, and troughs produced negative amplitudes. In experiment 2, a model hemodynamic impulse response was convolved with the stimulus time course (taken from the model response was a gamma function, taken from Boynton et al., 1996). We scaled this function to fit the data and estimated response amplitude as the scale factor that provided the best fit. In both experiments, mean response amplitudes were calculated for each stimulus type in each condition by averaging first within and then across subjects.
The fMRI data were analyzed for V1 and visual areas V2, Vp, V3a, V7, and V8. Visual areas were identified in separate sessions, using standard techniques for mapping retinotopic organization (Engel et al., 1994, 1997a; Sereno et al., 1995; DeYoe et al., 1996). Later visual areas V3a and V7 dorsally and V4 and V8 ventrally could not be distinguished in our data and so were analyzed together as two regions (V3a/V7 and V4/V8). For unknown reasons, area V3 gave unreliable responses in our experimental (nonretinotopy) protocols and so was excluded from additional analyses.
In no-adaptation scans, the test stimuli generated positive peaks in the fMRI response (Fig. 1 C,bottom). In the adaptation scans, neural activity produced by the low-contrast tests was less than the activity produced by the high-contrast adapter. Because the tests generated less activity than the adapters, they produced troughs in the fMRI time course as signal fell from the high baseline produced by the adapter (Fig.1 C, bottom). Importantly, the depth of the trough reflected the strength of the response to the test stimulus; lower levels of neural activity during test presentation resulted in deeper troughs in the fMRI time course.
The data from primary visual cortex show clear evidence of selective adaptation to color contrast. In the no-adaptation scans (Fig.2, left), responses to the L-M and L+M tests were peaks of equal amplitude. This indicates that, before adaptation, the two test stimuli produced neural activity in V1 of equal strength. In the L-M adaptation scans (Fig. 2,middle), responses to the tests were troughs, and the troughs produced by the L-M test were deeper than the troughs produced by the L+M test. Thus, the fMRI signal dropped to lower levels during the L-M test presentations than during the L+M test presentations. This pattern indicates that, after adaptation, the L-M test stimuli generated less neural activity than the L+M test stimuli, although the same stimuli produced equal responses before adaptation. When quantified, differences in response amplitude after adaptation were statistically reliable (Fig. 2 B). The pattern of data matches the results expected if V1 contained red–green color-opponent neurons whose responses were selectively reduced by exposure to L-M contrast. Later visual areas all showed similar trends of results, although only V2 and VP reached statistical reliability.
Selective adaptation effects were not seen, however, for L+M adaptation. V1 responses for this condition (Fig. 2, right) were troughs of equal size, indicating that the two test stimuli produced neural activity of equal magnitude. The L+M adapter appears to have affected the neural populations that respond to L-M and L+M tests equally. Again, later visual areas showed similar patterns of results.
Perceptual measurements made with our stimuli closely match the fMRI data (Fig. 3 A). Adapting to L-M (left) reduced the apparent contrast of L-M stimuli reliably more than it reduced the apparent contrast of L+M stimuli. Adapting to L+M (right) had a nonselective effect, reducing the apparent contrast of both stimuli equally. Previous perceptual studies have also found that adaptation to L+M can sometimes be less selective than adaptation to L-M, especially when test stimuli are well above threshold levels, as ours were (Flanagan et al., 1990; Webster and Mollon, 1994).
A second experiment attempted to maximize the selectivity of the adaptation effects. Behavioral pilot experiments indicated that 8 Hz adapters generated more selective behavioral adaptation than 2 Hz adapters. The reasons for this increase in selectivity deserve additional study and may include differences in temporal tuning between the light–dark and red–green mechanisms. Pilot work also found more selective effects for short test durations. Experiment 1 used long, 20 sec test presentations, during which adaptation may have weakened. Accordingly, in experiment 2, 4 sec presentations of test stimuli alternated with 20 sec presentations of 8 Hz adapting gratings. These stimulus parameters produced more selective behavioral effects (Fig.3 B). In addition, a uniform mean field test presentation (a zero-contrast stimulus) was added to the fMRI protocol to help estimate the absolute magnitude of adaptation effects.
In the second experiment, responses in area V1 showed clear evidence of selective adaptation to both L-M and L+M color contrast (Fig.4). In the no-adaptation condition (left), responses to the two types of test stimuli did not differ reliably. L-M adaptation, as in the first experiment, resulted in reliably weaker responses to L-M tests than to L+M tests (middle). The effect of adaptation on the L-M test was large; the fMRI signal dropped equally for L-M tests and for uniform mean field tests, indicating that adaptation effectively abolished the entire neural response to the test. The overall magnitude of responses, however, was smaller than in experiment 1. This was attributable to the 8 Hz L-M adapter, which was a weaker stimulus for V1 than was the 2 Hz adapter, and thus provided a lower base level from which neural activity could fall. Responses to the L+M test were quite strong, stronger than responses to the high-contrast L-M adapter, yielding a small increase in signal.
Critically, adaptation to L+M was also selective. After L+M adaptation, L+M tests produced reliably deeper troughs in the fMRI time course than did L-M tests (Fig. 4, right). This pattern indicates that responses to L+M were weaker than responses to L-M after L+M adaptation.
As in the first experiment, color-matching measurements agreed well with the data from primary visual cortex (Fig. 3 B). Adapting to L+M reliably reduced the apparent contrast of L+M tests more than it reduced the apparent contrast of L-M tests. Adapting to L-M reliably reduced the apparent contrast of L-M tests more than it reduced the apparent contrast of L+M tests.
Extrastriate visual areas showed similar patterns of results to V1 for L-M adaptation, although trends in area VP did not reach statistical reliability. Cortical regions outside of V1 showed trends for selective adaptation to L+M stimulation, but none reached statistical reliability. This pattern of results likely arises from reduced signal relative to noise in our measurements of extrastriate cortex. It is additionally possible that L+M adaptation effects are relatively smaller outside of V1. These effects are not likely to be completely absent, however, given that they do appear as trends in each extrastriate area.
Our results provide strong evidence of selective adaptation to color contrast in primary visual cortex. The most parsimonious explanation of our results is that V1 contains separate populations of red–green and light–dark color-opponent neurons. These neurons reduce their responsiveness after prolonged exposure to their preferred color contrast. Because of selective adaptation, the presence of these distinct neural subpopulations could be identified using fMRI without relying on spatial segregation of responses.
There are at least two alternative accounts in which the selective adaptation observed here might arise from only a single population of neurons. Both of these seem unlikely, however, given what is known about neural adaptation in primate cortex. First, selective adaptation might result from a single population of neurons reducing its overall responsiveness, if the relationship between stimulus contrast and neural response differs for L-M and L+M. For example, in some neurons, the L-M contrast response function might be steeper than the L+M function. In these neurons, as overall responsiveness is reduced by adaptation, responses to L-M would grow larger than responses to L+M. Such an explanation cannot easily account for the results of experiment 2, however. In that experiment, adaptation to one stimulus caused L-M responses to grow larger than L+M, whereas adaptation to another stimulus caused the opposite pattern. Reducing overall responsiveness of the neurons cannot produce such a pattern of results without unusually shaped contrast response functions and extremely fortuitous choices of stimulus contrast.
Our results could also be produced by a single population of neurons that changes its color tuning as a result of adaptation. Color tuning is the relative sensitivity of neurons to light of different colors. A large population of neurons in V1 might, for example, show very broad color tuning, responding well to both L-M and L+M stimulation. Adapting to L-M might selectively reduce the responses of these neurons to L-M but might leave other responses intact.
Measurements in V1 using single-unit recording find evidence for adaptation causing changes in both overall responsiveness and tuning (Maffei et al., 1973; Movshon and Lennie, 1979; Saul and Cynader, 1989;Sclar et al., 1989). Responsiveness changes are attributable to a hyperpolarization of membrane potential of neurons (Carandini and Ferster, 1997; Sanchez-Vives et al., 2000), whereas the mechanisms of tuning changes remain unclear. In general, overall changes in responsiveness are much larger in absolute terms than the selective reductions that produce tuning changes (Albrecht et al., 1984;Carandini et al., 1997). Furthermore, the shifts in tuning as measured by the location of peak responsiveness are relatively small. For example, tuning changes for orientation, which have been the most thoroughly measured, averaged <8° in cat V1 (Dragoi et al., 2000), in which orientation bandwidths are typically 30–40° (DeValois et al., 1982).
The relatively small magnitude of tuning changes produced by adaptation make the single population account of our data unlikely. We cannot rule out the potential influence of tuning changes, however, and they could certainly amplify an effect produced by changes in overall responsiveness. Intriguingly, one recent report has measured many neurons in V1 that respond to both red–green and light–dark (Johnson et al., 2001), but the effects of adaptation on these neurons is unknown. Untangling tuning changes from overall responsiveness changes remains an important issue in understanding adaptation generally.
Our data are consistent with models of V1 that contain large numbers of neurons that are more responsive to chromatic (e.g., L-M) stimuli than to luminance (L+M) stimuli. Some single-unit measurements of color selectivity have also found large, separate populations of red–green color-opponent neurons in V1 (Livingstone and Hubel, 1984; Thorell et al., 1984; Ts'o and Gilbert, 1988).
The close match between behavioral measurements and fMRI responses suggests that neurons in V1 provide an important basis for perceptual color-opponent mechanisms. This conclusion agrees with previous work comparing the color tuning of human V1 with perceptual sensitivity (Engel et al., 1997b). Although other properties of perceptual mechanisms have not yet been compared with V1 responses (for example, the effect of changing stimulus spatial frequency on color sensitivity), it appears probable that tasks that reveal color-opponent perceptual mechanisms are supported to a large extent by the responses of striate cortex.
In particular, our data support the idea that V1 plays an important role in the computation of perceived contrast. Previous work has established a close relationship between the magnitude of neural activity measured with fMRI and contrast increment detection thresholds (Boynton et al., 1998). Other measurements have also reported similarities between contrast detection performance and the fMRI signal in V1 (Furmanski and Engel, 2000). Together, these results suggest that the fMRI signal in V1 is coupled to some of the neural events that underlie contrast appearance. Suprathreshold perceived contrast is a complicated computation, however, that can be influenced by a wide variety of factors, including some that have only minimal effects on detection thresholds (Ross and Speed, 1996; Snowden and Hammett, 1996). Important components of this computation may arise beyond striate cortex.
The effects measured here are not likely to arise earlier in the visual pathway than V1. Single-unit recording failed to find effects of adaptation to contrast in the lateral geniculate nucleus of the macaque (Derrington and Lennie, 1984), and effects reported in cat are small (Ohzawa et al., 1985; Shou et al., 1996) (but see Smirnakis et al., 1997). In addition, behavioral work indicates that the perceptual adaptation transfers between the two eyes (Krauskopf et al., 1982;Webster and Mollon, 1994), pointing to a neural locus in cortex, in which information from the two eyes is first combined. Finally, behavioral work finds that adaptation to color contrast is orientation selective (Bradley et al., 1988). These data also suggest a cortical locus, because earlier parts of the visual pathway do not contain orientation selective neurons. The color-selective adaptation we observed in extrastriate cortex probably reflects input from adapted V1 neurons.
The power of our approach comes from its ability to measure changes in response that likely arise from subpopulations of neurons within a single visual area. fMRI was used to infer the presence of distinct neural subpopulations, even when they were not spatially segregated. Many psychophysical methods, such as selective adaptation, have been developed to infer distinct parts of a visual pathway from a single measure, behavior. Here, we have applied this same approach to a different univariate measure, the average response of V1. By combining psychophysical paradigms with neuroimaging, perceptual mechanisms such as color opponency can finally be linked to the action of specific neural populations in visual cortex.
This work was supported by National Institutes of Health Grant EY11862. We are grateful to Cassandra Moore and Frank Tong for comments on this manuscript and to Mark Cohen for his assistance with fMRI. We also thank John Mazziotta, the University of California, Los Angeles Brain Mapping Medical Organization, the Ahmanson Foundation, the Pierson-Lovelace Foundation, the Tamkin Foundation, and the Jennifer Jones-Simon Foundation for their support.
Correspondence should be addressed to Stephen A. Engel, University of California, Los Angeles, Department of Psychology, Franz Hall, 1282a, Los Angeles, CA 90025. E-mail:.