Abstract
During normal saccadic inspection of natural images, the receptive fields of cortical neurons are bombarded with frequent simultaneous changes in local mean luminance and contrast, yet there have been no systematic studies of how cortical neurons respond to such stimulation. The responses of single neurons in the primary visual cortex of the cat were measured for 200 ms presentations of sine-wave gratings confined to the conventional receptive field. Both local mean luminance and contrast were parametrically and randomly varied over the 1–1.5 log unit ranges that are typical of natural images. We find that responses are strongly modulated by both the local mean luminance and contrast, but in an approximately separable manner: the contrast response function is approximately invariant except for a scale factor that depends on the local mean luminance. The shape of the temporal response profiles were found to be approximately invariant with contrast, but were strongly affected by the local mean luminance. The results suggest that most, if not all, cortical neurons carry substantial local luminance information.
- visual cortex
- contrast response
- luminance response
- transient stimulation
- neural coding
- natural scene statistics
Introduction
Most neurons in the primary visual cortex have receptive fields that cover a relatively small localized region of the retinal image. Thus, when the eyes and/or head are moved, a given receptive field will sample a different part of the retinal image. Furthermore, many mammals make frequent rapid saccadic eye movements, as well as rapid head movements, and as a consequence, both the luminance and the contrast falling on a given receptive field will change abruptly and frequently.
Under natural viewing conditions, human saccadic eye movements vary in extent from a fraction of a degree to >20°, with an average of ∼5–8° (Bahill et al., 1975; Becker, 1975; Moeller et al., 2004). In the cat, saccadic eye movements occur less frequently, but tend to be even larger in magnitude (Moeller et al., 2004). Analysis of local contrast and luminance in natural images shows that there are large variations of local contrast and luminance within a given image (Laughlin, 1981; Ruderman, 1994; Brady and Field, 2000; Mante et al., 2005; Frazor and Geisler, 2006), that the variations in local luminance and contrast are approximately statistically independent (Mante et al., 2005; Frazor and Geisler, 2006), and that there is almost no correlation between the contrasts falling within a pair of receptive fields (1° in diameter) if their separation exceeds 2°, and almost no correlation in the luminance falling within receptive fields if their separation exceeds 4° (Frazor and Geisler, 2006). The upshot is that during natural viewing of static scenes it is quite common for a given cortical receptive field to receive large, statistically independent random variations in luminance and contrast, where each new combination of luminance and contrast has a duration corresponding to that of a typical fixation.
Perhaps surprisingly (given the above observations) there have been no systematic studies of how cortical neurons respond to rapid simultaneous changes in both local luminance and contrast. Previous studies of contrast responses in cat and monkey have made measurements under conditions where the local luminance is held constant at the mean value for the display (for review, see Carandini et al., 1999; Albrecht et al., 2003). The relatively fewer previous studies of luminance responses in the primary visual cortex have measured responses to large field modulations without parametric variation of contrast (Bartlett and Doty, 1974; Maguire and Baizer, 1982; Rossi et al., 1996, Rossi and Paradiso, 1999; Peng and Van Essen, 2005; Tucker and Fitzpatrick, 2006). Here, we report measurements of the responses of single neurons in the primary visual cortex of the cat to optimal sine-wave grating stimuli presented for fixation-like durations, where both local luminance and local contrast were varied randomly and independently from trial to trial over the 1–1.5 log unit ranges typical of natural images (Frazor and Geisler, 2006).
Materials and Methods
Preparation.
The procedures for the paralyzed anesthetized preparation, the electrophysiological recording, the stimulus display, and the measurement of neural responses using systems analysis were similar to those described previously (Hamilton et al., 1989; Albrecht and Geisler, 1991; Metha et al., 2001). All experimental procedures were approved by the University of Texas at Austin Institutional Animal Care and Use Committee, and conform to National Institutes of Health guidelines. In brief, young adult cats (Felis domesticus) were prepared for recording under deep isoflurane anesthesia. After the surgical procedures, isoflurane anesthesia was discontinued. Anesthesia was maintained throughout the duration of the experiment with sodium pentothal (2–6 mg/kg/h), and paralysis was maintained with gallamine triethiodide (10 mg/kg/h) as well as pancuronium bromide (0.1 mg/kg/h). The physiological state of the animal was monitored throughout the experiment by continuous measurement of body temperature, inhaled/exhaled respiratory gases, pressure in the airway, fluid input, urine output, urinary pH, caloric input, blood glucose level, electroencephalogram, and electrocardiogram. Microelectrodes were inserted into regions of the primary visual cortex such that the receptive fields of the neurons were located within 5° of the visual axis. Three different types of microelectrodes were used: varnish-insulated tungsten, glass pipette (1 m NaCl), or glass-coated platinum-iridium. The impedances of the microelectrodes ranged from 3 to 16 MΩ. Action potentials were collected with a temporal accuracy of 0.1 ms.
Preliminary measurements.
Before the main experimental protocol, the optimum orientation, spatial frequency, and temporal frequency were determined by varying the stimuli along these dimensions while listening to the firing rate of the cell. For the dimension of contrast, the minimum detectable contrast, semisaturation contrast, and saturation contrast were determined. In all of the experiments reported here, the sine-wave grating patterns were confined to the conventional receptive field, which was determined by expanding the size (the length and the width separately) of an optimal drifting sine-wave grating until the response of the neuron stopped increasing (De Valois et al., 1985; DeAngelis et al., 1994). After these qualitative determinations, the responses of each cell were quantitatively and systematically measured as a function of orientation, spatial frequency, and contrast. Cells were classified as simple cells or complex cells using the criteria described by De Valois et al. (1982) (see also Skottun et al., 1991). We report the results for all cells that produced systematic responses to orientation, spatial frequency, and contrast in the preliminary phase.
Stimuli.
The stimuli were presented on a monochrome Image Systems (Plymouth, MN) monitor at a frame rate of 100 Hz, with a global mean luminance of 27.4 cd/m2. To overcome the nonlinearities inherent in visual displays, both hardware and software methods were used to ensure a linear relationship between the requested and measured luminance. Viewing was monocular through an artificial pupil with a diameter of 3 mm.
After the preliminary measurements, windowed sine-wave grating stimuli of optimal orientation and spatial frequency were presented for 200 ms at a 5 Hz drift rate (1 cycle of drift per presentation) followed by 200 ms of mean luminance (Fig. 1). The transient change in local mean luminance occurred simultaneously with the presentation of the grating stimulus, and the starting phase of the drift was held fixed for a given neuron. The local mean luminance extended beyond the conventional receptive field and the local contrast pattern was confined to the conventional receptive field. The diameter of the region of local mean luminance averaged 12° and was at least twice the largest dimension of the conventional receptive field, which placed the contrast boundary of the local mean luminance area at the typical outer edge or beyond the suppressive surround region measured with grating stimuli (Cavanaugh et al., 2002; Walker et al., 2000). The entire display was 14.25° in width and height.
Stimuli for main experiment. In each presentation, a drifting grating stimulus and local mean luminance was displayed for 200 ms (1 cycle of 5 Hz drift), followed by 200 ms of global mean luminance. The local mean luminance and local contrast were parametrically varied and randomly interleaved.
The local mean luminance and contrast were varied parametrically in a pseudorandom manner. The mean luminance took one of four values (4.4, 8.8, 17.7, or 70.7 cd/m2) and the contrast (sine-wave amplitude divided by local mean luminance) took one of 13 values usually ranging from 0 to 70%. In a given experiment, each combination of luminance and contrast was presented 40 times, for a total of 2080 trials. For most neurons, the entire experiment was repeated more than once. We chose these ranges of local luminance and contrast based on previous measurements in natural images, where it was found that the 95% range of both local luminance and local contrast is 1–1.5 log units in the typical image (Frazor and Geisler, 2006).
Results
Contrast and luminance response functions
Full parametric data were obtained for 26 experiments on 12 neurons (eight simple cells and four complex cells). The responses for six of the neurons (four simple cells and two complex cells) are shown in Figure 2, which plots a separate contrast response function for each luminance. The responses were obtained by summing spikes over the first 200 ms of the response of a neuron to the stimulus (i.e., from 50 to 250 ms after stimulus onset). As can be seen, the responses vary strongly with both local contrast and local luminance, although there is considerable heterogeneity from cell to cell.
a–f, Responses of six neurons in primary visual cortex of cat to simultaneous transient changes in contrast and luminance. The solid curves are the average contrast response function scaled separately for each luminance. The data in c and e are from complex cells; the data in the other panels are from simple cells.
As expected from the existing literature, the responses generally increased monotonically with contrast, but with variations in shape and maximum response across neurons. We also found that for most of the neurons, the responses increased monotonically with luminance. However, this was not always the case; for example, notice that in Figure 2, b and f, the response is greatest to an intermediate luminance (note that the solid squares represent the highest luminance). The substantial variations observed in the neural responses as a function of transient luminance have potentially important implications for how local luminance is coded in the visual system (see Discussion).
A central question in our study was whether the responses to contrast and luminance are separable. In other words, does the shape of the transient contrast response function depend on the value of the transient luminance change? Or equivalently, does the shape of the luminance response function depend on the value of the transient contrast change? If the responses are separable, then the mean response as a function of contrast and luminance should be equal to the product of a single contrast function and a single luminance function: r(C,L̄) = rC(C)rL̄(L̄). To evaluate the degree of separability, we first averaged all of the contrast response functions measured in a given experiment. If the responses are separable, then this should be an accurate estimate of the shape of the contrast response component, rC(C). Next, for each luminance we scaled the average contrast response function by a single factor picked to maximize the fit to the contrast response data measured at that luminance. If the responses are separable, then these factors should be an accurate estimate of the shape of luminance response component rL̄(L̄). Finally, if the responses are approximately separable, then the percentage of variance accounted for by this procedure should be high. The solid curves in each panel of Figure 2 show the average contrast response function in that panel scaled to provide the minimum mean squared error at each luminance. Although the fits are not perfect, they are quite good. This is verified in Figure 3, which shows the percentage of variance (POV) accounted for by the separable model for each neuron, where the percentage variance was computed using the following formula: POV = {1 − [Σ(observed − predicted)2]/[Σ(observed − mean)2]} × 100. Except for one neuron, which was a weak responder, the separable model accounted for better than 90% of the variance. The separable model was also evaluated for integration durations of 100, 250, and 400 ms, and again the separable model generally accounted for better than 90% of the variance (the 400 ms integration time was from stimulus onset to the end of the blank interval). There were no differences in the degree of separability between the simple and complex cells (POV, 94.0% for the simple cells; 93.8% for the complex cells).
Histogram (across neurons) of the percentage of variance predicted by scaling the average contrast response function separately for each luminance. The good fits indicate that the responses to contrast and luminance are approximately separable.
Temporal response profiles
The approximate invariance in the shape of the contrast-response function with local mean luminance holds for the total spikes within a fixed integration interval. This result, of course, does not imply invariance in the shape of the temporal response within the integration interval. Figure 4 plots the average poststimulus time histogram (PSTH) for each combination of local contrast and local mean luminance (PSTHs for a representative simple cell and a representative complex cell are given in supplemental Figs. 1 and 2, available at www.jneurosci.org as supplemental material). For each local mean luminance (Fig. 4a–d), we find that the temporal response profiles are approximately invariant in shape as function of contrast, up to a scale factor and time shift. This finding generalizes and is consistent with our previous measurements in cat and monkey at a single mean luminance (Albrecht et al., 2002).
a–d, Temporal response profiles (PSTHs for 10 ms bins) for each combination of transient contrast and transient luminance averaged over 26 experiments on 12 neurons. In agreement with Albrecht et al. (2002), the temporal response profiles are approximately invariant with transient local contrast, up to a scale factor and time shift. However, the temporal response profiles change dramatically with transient luminance and, hence, there appears to be useful information for local mean luminance in the pattern of temporal response.
Remarkably, we find dramatic changes in the temporal response profile as a function of local mean luminance. At the lowest luminance (4.4 cd/m2), the largest response occurs very late. As luminance increases, this late component weakens and earlier components emerge and/or shift forward in time. These substantial variations in the PSTHs as a function of transient luminance also have potentially important implications for how local luminance is coded in the visual system. The average PSTHs for the simple and complex cells are qualitatively similar (see supplemental Figs. 3, 4, available at www.jneurosci.org as supplemental material).
Discussion
In summary, we measured the responses of cortical neurons in the cat to transient presentations of optimal sine-wave gratings that varied randomly and independently in luminance and contrast over the ranges that typically occur within a natural image. We found that (1) the responses for a fixed temporal integration interval were strongly modulated by both contrast and luminance, (2) the responses in a fixed temporal integration interval were approximately separable in contrast and luminance, (3) the PSTHs were relatively invariant in shape with contrast, and (4) the PSTHs changed dramatically in shape with luminance.
Separable responses in contrast and luminance imply that the effect of local luminance is to scale the transient contrast response function without changing its shape. In this regard, local luminance is similar to most other stimulus dimensions that affect the contrast responses of cortical neurons. For example, the primary effect of varying spatial frequency (Albrecht and Hamilton, 1982), orientation (Sclar and Freeman, 1982), phase (Albrecht and Geisler, 1991; Albrecht et al., 2002), and direction of motion (Geisler and Albrecht, 1997) is to scale the contrast-response function without changing its shape (for review, see Geisler and Albrecht, 1997). [Note that if contrast response functions are fitted with a Naka–Rushton/Michaelis–Menton function, f(x) = rmaxxn/(xn + x50n), then scaling the contrast response function is equivalent to changing the value of rmax.] One of the obvious but important implications of this shape invariance in the contrast-response function is that the tuning functions for spatial frequency, orientation, spatial phase, and direction of motion are approximately invariant with contrast. Our results add to this list, showing that the transient luminance tuning of cortical neurons in the cat is also approximately invariant with contrast.
We do not have enough measurements along the luminance dimension to provide a detailed description of how cat cortical neurons are tuned to transient local luminance. Nonetheless, it is clear that in each neuron the response varies substantially as a function of luminance. It is also clear that there is considerable heterogeneity across neurons: some neurons appear to saturate as a function of luminance (Fig. 2a), some neurons appear to increase smoothly in their response without evidence of saturation (Fig. 2c–e), and some neurons have a peak response to an intermediate luminance (Fig. 2b,f). This shows that there is plenty of local luminance information in the responses of most cortical neurons at least for the luminance range we tested (4.4–71 cd/m2).
How might this luminance information be used by the visual system? One of the important distinguishing features between surfaces in natural images is their luminance; thus, it would be useful to have mechanisms in the visual cortex that code local luminance. The obvious idea about how to accomplish this is to have a special set of neurons with receptive fields that effectively sum or average the receptor responses over a relatively large area, and indeed there is evidence for a subset of cortical neurons that respond well to uniform fields (Bartlett and Doty, 1974; Maguire and Baizer, 1982; Rossi et al., 1996, 1999; Peng and Van Essen, 2005).
However, sensitivity to uniform fields may often be unnecessary. There are very few natural surfaces that are actually uniform; in almost every surface there is texture. The fact that most cortical neurons are tuned for transient luminance, in much the same way that they are tuned for other dimensions such as orientation, spatial frequency, spatial phase, and direction of motion, provides an opportunity for the visual system to extract robust local luminance information even from neurons that do not respond to uniform fields. Specifically, the visual system could potentially extract local luminance information from a population of cortical cells using the same kinds of circuits it uses to extract, say, local orientation.
The finding that some of the neurons (Fig. 2b,f) prefer an intermediate luminance is consistent with previous studies (for review, see Peng and Van Essen, 2005). However, those studies were concerned with the subset of neurons that respond well to uniform fields. We did not select neurons on the basis of how well they responded to uniform fields, and in fact most did not respond significantly to large variations in the luminance of a uniform field (for responses at zero contrast, see Figs. 2, 4, and supplemental Figs. 1–4, available at www.jneurosci.org as supplemental material). Thus, our results suggest that even among cells that do not respond to uniform fields, there are cells tuned to intermediate luminance levels.
The scaling and time shifting of the temporal response profiles with contrast and the dramatic changes in shape of the temporal response profiles with luminance presumably result from a mixture of rapid luminance and contrast gain-control mechanisms that may occur in the retina (Shapley and Victor, 1979; Shapley and Enroth-Cugell, 1984; Baccus and Meister, 2002) or in the cortex (Carandini et al., 1999; Albrecht et al., 2003; Tucker and Fitzpatrick, 2006) (see also supplemental Fig. 5, available at www.jneurosci.org as supplemental material). In a previous study, Tucker and Fitzpatrick (2006) report that steps in mean luminance (both increments and decrements) produce strong transient hyperpolarization of layer 2/3 neurons in tree shrew visual cortex. Their results are qualitatively consistent with the delayed responses we observed after a luminance decrement (Fig. 4a), but not with the responses after a luminance increment (Fig. 4d). We will not try to speculate further here on the underlying mechanisms, but simply note that the effects are large and, hence, worthy of further investigation. For example, the strong nonlinear effects of transient changes in local luminance on the contrast responses of cortical neurons may help explain why standard methods of characterizing receptive fields do a poor job of predicting responses to natural stimuli (David et al., 2004).
Our finding that the responses of cortical neurons (for a fixed temporal integration interval) are approximately separable in luminance and contrast for fixation like (200 ms) presentations may be related to the statistical properties of natural images (Mante et al., 2005; Frazor and Geisler, 2006). Both local luminance and local contrast in a typical natural image vary over a 1–1.5 log unit range and, importantly, the luminance and contrast at the same location are approximately statistically independent. If luminance and contrast were statistically dependent, then there would be advantages for nonseparable responses to luminance and contrast. For example, in Gaussian noise that has the same power spectrum as natural images (so called 1/f noise), there is a strong negative correlation (−0.8) between local luminance and contrast. If this were the case in natural images, then making optimal use of the dynamic range of a neuron would require steepening the luminance response function at low contrast and steepening the contrast response function a high luminance. Given that luminance and contrast are statistically independent in natural images, it is efficient to respond in a separable manner. [We note that local luminance and contrast are approximately statistically independent in noise that has both a 1/f amplitude spectrum and the pixel luminance histogram of natural images (Frazor and Geisler, 2006). Strictly speaking, this is not 1/f noise because it is not Gaussian; we call it first-order 1/f noise.]
Although the consistency with natural scene statistics is interesting, our most important observation is that random variation in local mean luminance, over the range expected during saccadic inspection of natural images, has a dramatic effect on both the maximum of the contrast-response function and on the shape of temporal response profile. Thus, our results strongly suggest that, under normal viewing conditions, there is substantial information about local luminance in the responses of almost all neurons in the primary visual cortex.
Footnotes
-
This work was supported by National Institutes of Health Grant EY02688. We thank Eyal Seidemann for comments and Lawrence Stern and Carl Creeger for technical assistance.
- Correspondence should be addressed to Wilson S. Geisler, The University of Texas at Austin, Center for Perceptual Systems, 108 East Dean Keeton, 1 University Station A8000, Austin, TX 78712-0187. geisler{at}psy.utexas.edu
References
