 |
|||||
|
|||||
|
|||||
|
|||||
|
|||||
Previous Article | Next Article
The Journal of Neuroscience, July 15, 2002, 22(14):6129-6157
Detection and Discrimination of Relative Spatial Phase by V1 Neurons
Ferenc Mechler, Daniel S. Reich, and Jonathan D. Victor Department of Neurology and Neuroscience, Weill Medical College of Cornell University, New York, New York 10021
ABSTRACT
TOP
ABSTRACT
INTRODUCTION
Edge-like and line-like features result from spatial phase congruence, the local phase agreement between harmonic components of a spatial waveform. Psychophysical observations and models of early visual processing suggest that human visual feature detectors are specialized for edge-like and line-like phase congruence. To test whether primary visual cortex (V1) neurons account for such specificity, we made tetrode recordings in anesthetized macaque monkeys. Stimuli were drifting equal-energy compound gratings composed of four sinusoidal components. Eight congruence phases (one-dimensional features) were tested, including line-like and edge-like waveforms. Many of the 137 single V1 neurons (recorded at 45 sites) could reliably signal phase congruence by any of several response measures. Across neurons, the preferred spatial feature had only a modest bias for line-like waveforms. Information-theoretic analysis showed that congruence phase was temporally encoded in the frequency band present in the stimuli. The most sensitive neurons had feature discrimination thresholds that approached psychophysical levels, but typical neurons were substantially less sensitive. In single V1 neurons, feature discrimination exhibited various dependences on the congruence phase of the reference waveform. Simple cells were over-represented among the most sensitive neurons and on average carried twice as much feature information as complex cells. However, the distribution of the indices of optimal tuning and discrimination of relative phase was indistinguishable in simple and complex cells. Our results suggest that phase-sensitive pooling of responses is required to account for human psychophysical performance, although variation in feature selectivity among nearby neurons is considerable.
Key words: spatial feature detection; feature discrimination; phase-selective nonlinearity; congruence phase; edge; line; macaque; primary visual cortex; transinformation; simple and complex cells
INTRODUCTION
Psychophysical studies of spatial vision have demonstrated the importance of spatial phase information in shape perception (Burton and Moorhead, 1981
; Oppenheim and Lim, 1981
The prevailing view of early vision posits localized and spectrally band-limited image analysis at multiple spatial scales. The privileged role of lines and edges as features in human vision is posited to derive from phase congruence (Morrone and Burr, 1988
Theoretical work also motivates these experiments. The nonlinear feature detector model developed by Burr and Morrone (1992)
Our paper expands on earlier studies that assayed with spatial compound gratings the feature (relative phase) selectivity of single neurons in the primary visual cortex (V1) of cat (De Valois and Tootell, 1983
Parts of this paper have been published previously at the 1998 and 1999 Annual Meeting of The Society for Neuroscience (Mechler et al., 1998a
MATERIALS AND METHODS
Physiological preparation. Standard acute preparation techniques were used for electrophysiological recordings from single units in the V1 of the primate (cynomolgus monkeys, Macaca fascicularis). All procedures were in accordance with institutional and National Institutes of Health guidelines for the care and experimental use of animals. Some details of the techniques have been given earlier (Mechler et al., 1998b
Experiments were performed on 14 adult animals, weighing 3-4.5 kg. Before surgery, animals were given atropine (0.1 mg/kg, i.m.) and then anesthetized with ketamine (10 mg/kg, i.m.; Ketaset, Fort Dodge, IA). Anesthesia was maintained with sufentanil citrate (3-6 µg · kg
1 · hr
A limited unilateral craniotomy to expose the primary visual cortex was made overlying and posterior to the lunate sulcus (the Horsley-Clarke stereotaxic coordinates were typically 14-16 mm posterior and 14-16 mm lateral). A 1-2 mm durotomy was made for the recording electrode, which was stabilized after insertion by agarose gel.
Extracellular recording. Spike responses of single units were recorded extracellularly. We used either traditional glass-coated tungsten microelectrodes (single tip; typical resistance 2 M
) (Merrill and Ainsworth, 1972
The signals from the electrode or tetrode channels were passed through a unity gain (for the tetrode, multi-channel) differential head-stage amplifier (NB Labs, Denison, TX, or NeuraLynx, Tucson, AZ), and then further amplified and filtered (0.3-6 kHz pass-band, NeuraLynx eight-channel differential amplifier). Analog candidate spike waveforms, as detected by a threshold criterion, were digitized at 25 kHz within a short (~1.2 msec) temporal window containing the peak amplitude, and then recorded on computer disk (Discovery software, DataWave Technologies, Longmont, CO). Multiple single units were isolated by cluster analysis of spike waveforms initially performed on-line (Autocut, DataWave Technologies), then off-line [custom software (Reich, 2001
Histology and laminar assignment of recording sites. Experiments lasted for 4-5 d, at the end of which the animal was killed by infusion of a lethal dose of methohexital (Brevital; Eli Lilly & Co., Indianapolis, IN). After transcardiac perfusion with 4% paraformaldehyde in PBS, a block of the occipital cortex containing the penetration was saved for histological reconstruction of the electrode track. The block was cut in 40-µm-thick parasagittal sections, approximately parallel with the plane of the electrode penetration. Lesioned landmarks and fluorescent tracing aided track reconstruction. Electrolytic lesions (5 µA × 5 sec, electrode positive) were made, on withdrawal after recording was completed, at two or more points along all the tracks made with an Ainsworth single electrode, and on some tracks made with tetrodes. Fluorescent full-track tracing was made with the lipophilic dye Dil (D-282; Molecular Probes, Eugene, OR). The dye, applied in a thin coat on the tetrode tip before penetration, left a ~40- to 200-µm-wide trace from entry to the point of deepest penetration. These traces were easily identified in fluorescent micrographs prepared from sections before Nissl staining. In the same sections, the laminar boundaries were identified from the overlaid light micrographs of the Nissl density taken after Nissl staining. Lesions were also best identified on the Nissl-stained sections. Laminar positions of the recording sites were estimated relative to the pattern of Nissl density along the reconstructed electrode track after correction for tissue shrinkage. With this method we successfully identified the laminar position of two-thirds of the recording sites. Sites near a laminar boundary within the precision of reconstruction were classified as located in either lamina across the boundary. However, even with good histology, occasionally landmark positions could not be found or remained ambiguous, and laminar positions were either not assigned to recording sites or could only be classified in one of three gross divisions (granular, supragranular, or infragranular layers).
Optics. The eyes were treated with anti-inflammatory (Ocufen) and anti-bacterial (neomycin) ophthalmic solutions. Pupils were dilated with topical application of 1% atropine sulfate (Atrosulf-1; Optics Laboratories Co., Fairton, NJ) and covered with gas-permeable contact lenses (Metro Optics Inc., Houston, TX) under eyelids retracted with 6-0 chromic gut sutures. Artificial pupils (2 mm) and corrective lenses were used to focus the stimulus on the retina. Optical correction was estimated by retinoscopy and then refined by optimizing responses of isolated single units to high spatial frequency visual stimuli.
Visual stimulation. Foveae were mapped on a tangent board by back-projection with an ophthalmoscope. The receptive fields of isolated neurons were mapped on the same board with a laser. The standard simple/complex classification, based on the modulation ratio, was used: if the fundamental of the response to a drifting grating of near optimal spatial parameters was larger than the DC component (after subtraction of the maintained rate of firing), then the cell was cast as simple, and complex otherwise (Movshon et al., 1978b
Visual stimuli were generated by a special purpose stimulus generator (Milkman et al., 1978
After isolation of single units, their receptive fields were characterized in a standard way using drifting sine gratings: tuning was measured first for orientation, then for spatial frequency, and finally for temporal frequency, each parameter optimized for subsequent tuning measurements. The contrast response function was measured using the optimal sine grating. When multiple single units were simultaneously isolated with tetrodes, receptive-field characterization was always done for the most responsive unit, and often for a second unit. For many neurons, the receptive field was also characterized with pseudorandom black-and-white checkerboards modulated by long (212-1 frames) binary m-sequences at 67.58 Hz. Our implementation of m-sequence stimuli and associated analysis procedures have been described in detail previously (Victor, 1992
Compound gratings. In our experiments, 1D gratings were drifting at or near the optimal orientation and direction for the V1 neurons. With the spatial origin centered on the display, the spatiotemporal light variation
I(x,t) around a spatiotemporal mean intensity I0 in a single drifting sine grating is described, in cosine formulation for convenience, by:
where C is the Michelson contrast (defined as C = [max(
(1) min(
is spatial frequency (c/°), f is temporal frequency (in Hz), and
is relative phase (in radians). At time zero, the intensity peak is at position
(so, if
Each of our compound-grating stimuli is constructed from four of these single-grating harmonic components. We use a superposition of odd harmonics. That is, the mth component grating is chosen to have a frequency equal to 2m-1 times the fundamental. Consequently, the light variation around the mean intensity in the mth component, Sm(x,t), is given by:
(2)
Thus, the four gratings included a fundamental and its third, fifth and seventh harmonic (see Fig. 2a, boxed area), each with a contrast inversely proportional to the harmonic number, and at the same drift velocity v = Fk/
Thus,
(3)
View larger version (59K):[in this window]
[in a new window]
Figure 1. The definition of congruence phase,
View larger version (30K):[in this window]
[in a new window]
Figure 2. Construction of our compound grating stimuli. a, A square wave (edge) is a linear combination of an infinite series of spatial sine waves. This Fourier decomposition of the edge contains only the odd harmonics of the fundamental spatial frequency f, each with amplitude inversely proportional to its harmonic index. Note that the components have the same relative phase (
Note that the phase parameter
Note that although the edge-like combination of an infinite number of sine components is convergent (because it is the Fourier series of an edge; see Fig. 2a) the infinite series does not converge for any other phase congruence. Consequently, with the exception of the edge-like stimulus, the peak (Michelson) contrast of each compound waveform would grow without limit, albeit slowly, as additional odd-harmonic components were added. However, this does not lead to any practical difficulties, because we use only a finite set of gratings for all phase combinations. For a fixed set of components, the Michelson contrast in our feature space decreases monotonically (as a cosine function of congruence phase) from line to edge in either direction on the phase circle. The Michelson contrast is largest for the line-like waveform (congruence phase
, and smallest for the edge-like waveform (congruence phase . We set the contrast of the fundamental component C to 0.5 so that the modulation of the four-component line-like waveform had a Michelson contrast of 0.84. The root-mean-square contrast was 0.38 for each compound grating. Data analysis. Off-line data analysis was performed in the Matlab programming environment using custom software. In general, fast Fourier transforms were used whenever Fourier analysis is mentioned. The details of the information analysis based on Fourier metrics have been given previously (Mechler et al., 1998b
RESULTS
Data were obtained from V1 neurons with parafoveal receptive fields (centered at 2-5° eccentricity). Following convention, we used the modulation ratio (see Materials and Methods) for the classification of V1 neurons: if the modulation ratio exceeded 1.0, neurons were classified as simple cells, and complex cells otherwise. A total of 226 data sets were collected from 137 neurons (88 complex and 49 simple) from 45 recording sites. Criteria for quantitative analysis were (1) good isolation was maintained throughout the experiments described below, and (2) responses to at least one of the compound gratings were reliable (d' > 1.0 for the amplitude of any of the first six Fourier components of the response in comparison to the blank condition, or
across these first six components). Slightly more than half of the data sets met these criteria. These 121 data sets from 32 recording sites included 78 data sets from 46 complex cells and 43 data sets from 31 simple cells. (Some cells yielded two data sets from compound gratings of different drift velocity). Note that in each recorded cluster the fundamental frequency and orientation of the compound gratings were optimized for one cell only (usually the most robustly responding one). Because grating parameters were not necessarily optimal for each cell in the cluster, the fraction of cells that could yield responses that met analysis criteria (had they been stimulated with gratings of optimal orientation) may be higher than 77/137. Cells that did not meet the above selection criteria for analysis typically also responded poorly to the component gratings presented alone at the selected frequencies and orientation. Feature tuning in V1 neurons
Our aim in this study was to gain insight into how V1 neurons signal and discriminate spatial waveforms, including those that resemble salient spatial features such as edges and lines. These features are presumed salient because of spatial phase congruence. We know that although appropriate symmetry-selective filtering is necessary, linear filtering alone cannot explain the underlying feature-extraction mechanism. Subcortical visual processing involves nonlinear transformations, but these transformations are primarily related to adjustment of overall gain and dynamics, and are not orientation or feature specific. Thus, the neuronal circuitry that performs feature extraction in primates is almost certainly at a cortical level.
The neuronal implementation of feature extraction, however, is as yet unknown. Natural candidates for the pre-filters are V1 simple cells the receptive field profiles of which have the appropriate even or odd symmetry as required by a local energy model. Although the analysis of phase selectivity to spatial compound gratings is a necessary step in understanding the relationship of these neurons to feature extraction, only a few studies of single neurons evaluated this directly: De Valois and Tootell (1983)
The defining feature of simple cells is the simple, approximately linear fashion in which they appear to sum spatial stimuli within their classical receptive fields (Hubel and Wiesel, 1962
Responses of a paradigmatic simple cell are shown in Figure 3. This layer 4C
simple cell had little spontaneous activity in the absence of visual stimuli (shown as the blank condition, i.e. a uniform screen of luminance set at the mean of the grating stimuli in Fig. 3a). Single drifting gratings (those in Fig. 3a, as well as other sine gratings used for characterizing the neuron; data not shown) elicited responses that seemed close approximations to half-wave rectified sinusoids the modulation frequency of which was that of the first harmonic component of the stimuli. This behavior is characteristic of typical simple cells, both in our data and as previously reported (Movshon et al., 1978a
View larger version (28K):[in this window]
[in a new window]
Figure 3. Typical responses to compound gratings and their components recorded from a V1 simple cell, a layer 4C
Note the similarity between the response to the full edge (Fig. 3b, true edge) and the response to the stimulus that approximates an edge via its first four components (Fig. 3b, "edge"). For this cell, the response to the full edge is slightly narrower in time. This indicates that the pass-band of the linear receptive field of the cell was broad enough so that one or more stimulus components of the edge above the seventh harmonic affected the response of the cell. In most neurons, however, responses to the full edge and its truncated approximation were indistinguishable. Thus, the pass-bands of most neurons were sufficiently narrow so as to exclude the details present in those higher harmonics. This is expected given the average 2-2.5 octave spatial frequency bandwidth (full width at half-height) of macaque V1 neurons (De Valois et al., 1982
The above observations were quantified by Fourier analysis. There is a more general reason for doing the Fourier analysis: we have no a priori knowledge of which response component carries feature dependent signals. Although nonlinear interactions may act to enhance selectivity toward a particular spatial feature, this need not be consistent across all response components. First, we consider conventional scalar response measures defined on Fourier amplitudes alone and in combination, the analysis of which is relatively straightforward. Next, we present an analysis of the Fourier amplitudes and phases jointly (as vectors in the complex plane), which is perhaps more demanding, but also more interesting, because the complex measures have larger signaling capacity attributable to the extra degree of freedom in the phases.
Feature tuning in scalar response measures
Figure 4a shows the analysis of Fourier amplitudes of the responses of the simple cell from Figure 3 to the sine gratings presented alone. Selective tuning to gratings of various spatial and temporal frequencies, drifting at a constant speed, is indicated by the response amplitudes measured at the fundamental frequency of each grating (amplitudes marked with thick bars). Note that the grating contrast was scaled as in the components of an edge: the contrast of first component was three, five, and seven times larger than the contrast of the second, third, and fourth components, respectively. This means that the simple cell was even more sensitive to gratings of high frequencies than this plot indicates, i.e., the high-frequency cut-off in the pass-band of this cell fell beyond the seventh harmonic, because its response to this stimulus was unequivocal (m = 4 in Fig. 4a). Nonlinear responses to single gratings are indicated by non-zero components at multiples of the fundamental frequency for each grating. The approximately
View larger version (46K):[in this window]
[in a new window]
Figure 4. Mean Fourier amplitudes of the responses shown in Figure 3. Error bars indicate 95% confidence limits on the mean. a, Fourier components (DC and F1 through F8) of responses to simple grating stimuli. From front to back: blank screen (at the mean luminance of the edge), the first four non-zero drifting sine components of an edge (Eq. 2), and the full edge. b, Fourier components of responses to drifting compound gratings. For this simple cell and most other V1 neurons, nearly all response energy is contained at these eight frequencies. The maximum amplitude (across congruence phase) of most response harmonics predicts similar optimal waveforms for this simple cell, ( 
Nonlinearities are also seen in the response to the full edge (Fig. 4a, true edge). One manifestation of nonlinearity is the presence of responses at even harmonics, as described above. A second manifestation is that the responses measured at the odd harmonics to either of the compound gratings (Fig. 4b) or the full edge (Fig. 4a) is not equal to the responses to the corresponding gratings presented alone. For this cell, the individual grating responses would predict that the peak component of the response to each compound gratings or the full edge occurs at the third harmonic frequency (F3), but in fact it occurs at F1 or F2. Although some Fourier components above the eighth harmonic temporal frequency (F8) are still significant, the overwhelming part of the response energy is contained in the DC and the first eight components.
For this and other simple cells, examination of the Fourier amplitudes of the responses to compound gratings (Fig. 4b) reveals that F1 has both the largest response amplitude and the largest variation of amplitude across the stimulus set. At each frequency, linearity predicts identical Fourier amplitudes for all compound gratings. Note that although the approximate constancy of the DC component is consistent with the linear prediction in this simple cell (cell of Fig. 3), which thereby gives the DC component the poorest feature tuning, most other Fourier amplitudes show systematic variation (i.e., tuning) with stimulus congruence phase. Moreover, this tuning seems similar across components. Judging by the maximum amplitude of most components, the optimal waveform for this simple cell has a congruence phase
In general, the nonlinear signature of complex cell responses to the compound gratings is that even-order Fourier harmonics dominate the response. In the typical complex cell, unlike the typical simple cell, the largest response component as well as the response component with the largest phase-dependent modulation is the DC or the second harmonic component, F2. Figure 5 shows the responses of six more V1 neurons (mostly complex cells). As a group, these give a sense of the variety of phase-selective responses encountered in V1; individually, each is selected to emphasize a distinct point. Figure 5a shows the responses of a typical complex cell. For this cell, unlike for the typical simple cell, the poststimulus time histograms for drifting gratings, especially at high frequencies, are unmodulated. For compound gratings, the response histograms for this cell are characteristically bimodal, with a response transient corresponding to the passage of the stimulus transient of both contrast polarities. This contrasts with the unimodal histograms seen for the paradigmatic simple cell (Fig. 3). For each drifting waveform, there are two response peaks approximately half a period apart (in terms of the fundamental), but their size and ratio vary systematically with the congruence phase. Thus, the typical complex cell shows a strong nonlinearity (domination of the response energy by even-order harmonics), but the phase-dependent variation manifest in the size and ratio of the peaks diverges from what is expected of a phase-insensitive energy operator.
View larger version (20K):[in this window]
[in a new window]
Figure 5. Response histograms for six V1 cells that exhibit the variety of response patterns observed in our sample. Responses to compound gratings are shown ordered around the phase circle, as in Figure 3b, and the responses to the blank as well as the four component gratings (equivalent to Fig. 3a), in columnar arrangement inside the phase circle (blank on top). Vertical scale bars indicate size of the peak response. a, Typical complex cell (450213.u); vertical scale 60 spikes/sec; fundamental period 315.7 msec. b, The complex cell that was most sensitive and had highest signal-to-noise in our sample (431115.s); vertical scale 350 spikes/sec; 315.7 msec. c, The simple cell that was the most sensitive and had the highest signal-to-noise in our sample (440909.t); vertical scale 350 spikes/sec; 1263 msec. d, A complex cell that approximates a broadly tuned edge detector (490707.s); vertical scale 30 spikes/sec; 1263 msec. e, A complex cell that responds only to the full edge (shown above the response to the four-component approximation of the edge) but not to the four-component compound gratings (470320.t); vertical scale 20 spikes/sec; 1263 msec. f, A borderline simple/complex cell that approximates a broadly tuned line detector (440813.s); vertical scale 100 spikes/sec; 1263 msec.
Figure 5, b and c, respectively, shows the responses of the complex and simple cell that had the highest gain and the least noisy responses in our sample. Both follow with high fidelity the higher harmonic modulations present in the stimulus. The simple cell responses exhibit a tendency of firing to be restricted to one-half of the stimulus period, indicative of dominant odd-harmonic Fourier components in the response. The response histograms of the complex cell exhibit the opposite tendency, toward a firing pattern that is replicated in each half of the stimulus period, indicative of dominant even-harmonic Fourier components in its response. However, these descriptions are caricatures, and most cells within our sample of >100 V1 neurons showed intermediate behavior. (The ability of the even and odd response harmonics to signal congruence phase is given in a systematic population analysis below.)
Each neuron discussed so far was typical in that it had a more or less vigorous response to each congruence phase, but with a variable response waveform. On the basis of the response histograms alone, therefore, it is difficult to tell by eye for most neurons whether they are selective to one or the other spatial waveform to any significant degree, and a quantitative analysis of the responses is necessary. However, a minority of the neurons were quite selective to certain waveforms to a degree that was obvious even from a cursory examination of their response histograms. Figure 5d-f presents examples of such phase-selective neurons. Figure 5d shows a complex cell that was broadly tuned to edges. Figure 5e shows another edge-selective complex cell that was quite responsive to the full edge but barely to the four-component edge-like compound grating. For this cell, most grating components probably fell below its pass-band, but it fulfilled the criteria for analysis based on d' (see above). This behavior was rare (only 2 of 137 cells in our sample). The final example, a borderline simple/complex cell shown in Figure 5f, can be described as a (broadly tuned) line detector. This cell preferred an approximately line-like waveform (for the congruence phases tested, the largest peak of the response histogram occurs at
7
Some V1 cells (such as the simple cell in Fig. 3) signal variation of congruence phase predominantly in their odd response harmonics, and other cells (such as most complex cells in Fig. 5) signal congruence phase predominantly in their even response harmonics. Therefore, scalar measures of the even and odd response energy are also obvious candidates for further analysis. For the simple cell of Figures 3 and 4, some of these measures are examined in Figure 6.
View larger version (38K):[in this window]
[in a new window]
Figure 6. The dependence of various scalar response measures on the congruence phase, for the simple cell of Figures 3 and 4. To describe the feature tuning of the cell in each measure, the data (open symbols) were fit (thick lines) with a five-parameter second-order harmonic function (Eq. 4) independently for each response measure. The optimal congruence phase (
The four response measures shown here are the mean firing rate (Fig. 6a, DC), the even-harmonic energy (defined as the summed squared amplitudes of the DC and harmonics 2, 4, 6, and 8). (Fig. 6b), the odd-harmonic energy (summed squared amplitudes of harmonics 1, 3, 5, and 7) (Fig. 6c), and the total response energy (summed squared amplitudes of the DC and the first eight harmonic components of the response) (Fig. 6d). The linear prediction that the response is independent of congruence phase fails. Each of these response measures systematically depends on the stimulus phase, and, for the three energy measures, this dependence is substantial.
To describe the dependence of each of these response measures on spatial phase, we used the method of least squares to fit a harmonic function of the congruence phase,
This five-parameter fitting function is a natural choice for the following reason. The complex amplitudes of the response harmonics are well approximated by an ellipse parametric in twice the congruence phase, as demonstrated empirically in Figure 9 and analytically (considering contributions up to and including fourth-order nonlinear contributions) in the Appendix. Given such an elliptical dependence of the complex amplitudes of the individual harmonics on congruence phase, one can show that the dependence of an energy measure on congruence phase will be a function of the form of Equation 4. For each response measure considered, we defined the optimal congruence phase,
(4) In the circular feature space used here, the sharpness of the tuning to features of a response measure (i.e., its feature selectivity) is naturally measured by the circular variance (CV) of the response measure (Mardia, 1972
i.e., 1 minus the length of the vector-averaged response measures. To apply this measure, we take the response amplitudes Rk from the fitted curve and
For the simple cell in Figure 6, the four response measures, although not equally sharply tuned, yield very similar optimal phases (arrows). This is remarkable because one might expect that they reflect the effects of different nonlinearities. For this cell, the optimal compound waveforms had a congruence phase
The analysis shown for the simple cell in Figure 6 was also carried out for the examples of Figure 5 (mostly complex). Figure 7 summarizes quite similar results for the DC and the three energy measures. The DC (open circles) usually predicted the same optimal congruence phase, but in most cases was a less selective measure than the energy measures, as quantified by the CV. Although a greater selectivity is expected for the energy measures than for the DC merely because the energy (impulses squared/seconds squared) but not the DC (impulses/second) is a squared quantity, the full extent of the observed selectivity difference is not explained by units of measurement. In the case of the typical complex cell in Figure 7a, the even energy (squares) and odd energy (triangles) are similarly tuned, but the even energy dominates. The dominance of the response by even energy is even more pronounced in the case of the complex cell in Figure 7b. In this case, and in the case of the "edge-detector" (Figure 7d), the even and odd energy are also differently tuned. (Note that although the odd energy is very small, the measured values are highly reliable, as determined by the illustrated bootstrap confidence limits.) However, in most cases when the even and odd response energies were both substantial, such as in the cases of the simple cell (Fig. 7c) and the line detector (Fig. 7f), the two scalar measures tended to be similarly tuned. Note that Figure 7, b and c shows the cells with the highest signal-to-noise ratios in our sample of V1 neurons; the error bars of the other cells are more typical.
View larger version (39K):[in this window]
[in a new window]
Figure 7. The dependence of the same scalar response measures as in Figure 4, the DC (open circles), odd energy (open triangles), even energy (open squares), and total response energy (filled circles), on congruence phase for the six examples of Figure 5. Panels correspond to those in Figure 5. Note that the vertical scale for the energy measures (left) and the DC (right) differ. For each cell, the optimal phase (
Figure 8 shows an example of how phase tuning varies locally in V1. These four complex cells, recorded simultaneously by a tetrode, exhibit considerable difference in phase sensitivity (gain), selectivity, and preference. This is representative of the variation of these parameters in local V1 ensembles. Cell 1, the cell with the highest gain in this local cluster, and cell 2 are least selective: their tuning curves (Fig. 8c, left) approximate what would be expected from a strict (phase-insensitive) energy calculation. In comparison, cell 3 (the least sensitive in this cluster) and cell 4 (the cell comparable in sensitivity to cell 2) are both well tuned but tuned to different preferred phases (Fig. 8c, right). Cell 3 is tuned to a waveform the congruence phase of which is intermediate between that of a line and an edge. (Judged from its responses shown in Fig. 8a, cell 3 seems simple but it was classified as a complex cell on the basis of its response to the optimal single grating.) Cell 4 is tuned to a line-like waveform.
View larger version (27K):[in this window]
[in a new window]
Figure 8. Four complex cells simultaneously recorded by tetrode (infragranular layers). a, b, Response histograms. Vertical scale bar indicates 150 spikes/sec for Cell 1, and 50 spikes/sec for Cells 2-4. Horizontal scale bar indicates the 1263 msec fundamental (F1) stimulus period. a, Responses to single sine components presented alone. b, Responses to compound gratings with eight different congruence phases. Data sets corresponding to different cells are in concentric arrangement. c, The dependence of three energy measures on congruence phase plotted for the four cells as in Figure 7 (odd energy, triangles; even energy, squares; total energy, filled circles). Optimal congruence phase (
Another notable point is that responses of cell 4 to compound gratings have a single mode (Fig. 8b, innermost histograms), much like those of simple cells, but its responses to single sine gratings, except at the lowest spatial frequencies (Fig. 8a, histograms in rightmost column), consist mostly of spike rate elevation and only weak modulation, the defining characteristic of complex cells. Such apparently mixed behavior was observed in many cells of both classes (as defined by their responses to single gratings) in our sample: simple cells could have strong even harmonic components in response to compound gratings (as in Fig. 7c), whereas complex cells could have strong odd harmonics in response to compound gratings. Mixed behavior, intermediate behavior between what is expected for an "ideal" simple and ideal complex cell, was reported earlier in cat area 17 neurons studied with contrast-reversed single gratings (Spitzer and Hochstein, 1985
Feature tuning in vector response measures
The energy measures considered above are sensitive to response size but not timing. This extra degree of freedom present in the phases may also make it possible for the responses to encode the stimulus space (a circle), which is of genuinely two-dimensional (2D) topology and which the scalar measures are incapable of encoding. To determine whether this is indeed the case, we next consider a joint analysis of the amplitude and phase of response components. We begin this analysis on the simple cell of Figures 3, 4, and 6. Figure 9a shows the dominant response component, F1, plotted as a vector on the complex plane for each of the eight compound gratings. F1 is referenced to the phase of the fundamental stimulus component by subtracting the congruence phase
View larger version (43K):[in this window]
[in a new window]
Figure 9. Amplitude and phase of the first four Fourier harmonics in the response, represented by a vector quantity in the complex plane, for the simple cell shown in Figures 2-4. The center of each shaded circle represents the mean response to a compound grating. Circles indicate 95% confidence of the mean. The distance of a point from the origin indicates the magnitude of the response, and the direction represents its phase plotted with the phase correction indicated by h (see Appendix). Progression of congruence phase ( , are often <0.05. The optimal stimulus (
To get a better view of the details of the F1 responses in Figure 9a, we present an expanded version in Figure 10. One kind of third-order nonlinearity that can contribute to F1 is represented by the combination F1 + Fk
View larger version (46K):[in this window]
[in a new window]
Figure 10. An expanded view of a portion of Figure 9a. Red circle is linear prediction, and blue circle is full edge. See Results for details. A snapshot of each compound grating is shown next to the corresponding responses.
The other kind of third-order nonlinear interaction that leads to responses at the fundamental frequency consists of contributions such as F3
The data in Figure 10 approximate an ellipse rather than a circle. We show in the Appendix that the fifth-order nonlinearities further displace the center of the locus (p = 1 terms), add elliptical distortions to the circle (p =
The net result of these nonlinearities is that the locus of the F1 response depends strongly on the stimulus profile. The strong modulation of the amplitude is comparable to what was seen for the odd-harmonic energy (Fig. 6c), which was dominated by the contribution of F1. As we show in the Appendix, an elliptic approximation of the configuration of the Fourier harmonics of the response in the complex plane captures the contributions from nonlinearities up to a certain order (order 4 for F1). Thus, for descriptive purposes, we fit an ellipse to the set of eight data points, forcing an equal phase separation of corresponding points on the ellipse (Fig. 10, white dots on the blue ellipse, indicated by blue arrowheads):
(5)
Here, z(
are determined by minimizing the sum of the squares of the distances between z(
Although the fitted ellipses may deviate detectably from the data because of the presence of high-order nonlinearities, they provide a useful summary of the response to the eight stimuli. Both the statistical significance and the usefulness of this summary depend on the response variance, but in a different manner. The total variance in the data (i.e., the variance of the responses to each trial of each congruence phase) consists of two parts: first, the scatter caused by trial-by-trial variation of the responses to each congruence phase, and second, the dependence of the mean response on congruence phase. Trial-by-trial variation of the Fourier components are measured by the 95% confidence regions estimated by the T
statistic (Victor and Mast, 1991 2 statistic, may nevertheless be meaningless, because there is little systematic dependence on congruence phase to be explained. We therefore restrict our phase plane analysis of Fourier components to data sets that are not dominated by scatter within congruence phases. We chose as a criterion that the phase dependence of the mean response account for at least 40% of the total variance. This criterion eliminated one-third of the data sets originally selected for analysis (N = 121 reduced to N = 94). In slightly less than half of the data sets that met this criterion (42 of 94), the elliptical fit to F1 or F2 was a statistically acceptable fit, as measured by the
For the example cell shown in Figure 10, the fitted ellipse runs through the 95% confidence regions of each response (gray disks). However, when the eight data points (black crossed centers of gray disks) are considered together, their overall deviation from the fit (white dots on ellipse indicated by blue arrowheads) is beyond the range expected from measurement error (
= 22.49; p = 0.013). That is, the data occupy a locus that is significantly different from an ellipse, although no single point differs significantly from the prediction of the elliptical locus. As pointed out before, such deviations from the ellipse are in fact expected for the F1 response harmonic from fifth-order and higher odd-order nonlinearities. The low tolerance found for the fit for this simple cell is the direct consequence of the unusually low levels of noise in the data (in comparison with most of our cells), resulting in a high confidence in the position of the measured average responses. In spite of this, the fit in this case is a useful descriptor because it captures 99% of the phase-dependent variance of the mean responses. The amount of unexplained variance via the T analysis of the individual responses, although statistically significant, is small. The most impressive feature of this plot, and one that is typical of the other data sets, is that the responses capture the topology of the stimulus space. That is, the orderly progression of F1 in a single loop around the ellipse mimics the progression of the congruence phase around the phase circle. One can also see that the response fundamental of this cell measured for the true edge (blue circle) and its truncated approximation with only four components are indistinguishable (their error circles fully overlap in Fig. 10). Because none of the error circles corresponding to the eight compound gratings overlap with one another or with the origin, this cell can detect and identify each waveform based just on the response fundamental, provided that F1 amplitude and phase are jointly considered. Amplitude alone primarily distinguishes waveforms the congruence phase of which is in the range 0
To compare response characteristics across neurons, we define the optimal congruence phase to be the phase parameter on the ellipse that corresponds to the most distant point from the origin. That is, it is the congruence phase of the stimulus that leads to the largest F1 response, as interpolated by the fitted ellipse. With this definition, the optimal congruence phase for this cell is
122°), a stimulus that is intermediate between edge and line (see snapshots in Fig. 10). This corresponds closely to the optimal stimulus as inferred from the scalar measures, shown in Figure 6.
Figure 9, b and d, show that the higher response harmonics F2 and F4 also depend on the congruence phase of the stimulus in a similar way as F1, but F3 (Fig. 9c) has a different behavior. Its dependence on stimulus phase is much less prominent (the mean responses are indistinguishable by spatial phase as the overlapping error circles indicate) than for F1, F2, and F4. Moreover, the F3 response to every compound grating is less than the linear prediction, i.e., the F3 response to the second component grating, which contains this frequency alone (red). (The amplitude of this response was also shown in Fig. 4a.) That is, the spatial nonlinearities contributing to F3 that are elicited by the compound gratings are all antagonistic to the linear contribution to the F3 response. In sum, although nonlinear interactions may act to enhance selectivity toward a particular spatial feature, this need not be consistent across all response components. Nevertheless, in this and most other cells, most of the significant Fourier harmonics of the response tended to be maximal for the same stimulus waveform.
The results of a similar analysis of response harmonics in the complex plane is summarized in Figure 11 for the six V1 neurons shown in Figure 5 and 7. For each cell, one or more of the representative Fourier components are plotted with the conventions of Figure 9. The remarkably different levels of signal-to-noise ratio among these cells are evident from the very different sizes of the error circles in these plots (for clarity, some of the error circles were omitted). As explained above for the simple cell of Figure 10, the goodness of the elliptical fit is typically greater for neurons that have responses of lower signal-to-noise ratio. In general, the harmonics at which the response was largest generally were also the most phase selective. For example, the high-fidelity complex cell (Fig. 11b) is well tuned in F2 and F4 but poorly in F1. Conversely, the "line-detector" (Fig. 11f) is remarkably tuned in the odd harmonics F1 and F3, which are also the largest. The locus on the complex plane of the odd harmonic responses of this phase-selective neuron is well approximated by an elongated ellipse the long axis of which is aligned with the direction from the origin. This alignment, together with the narrow eccentricity, maximizes phase sensitivity and selectivity. However, this neuron is less well tuned in the even harmonics: the F2 (data not shown to prevent overlap with F1), the largest even harmonic, is comparable in amplitude to F3, but its elliptic locus is less eccentric and tilted at an angle relative to the direction of the origin. A comparison of these results with those of Figure 7 reveals that the optimal congruence phase predicted by the ellipses fitted to the various Fourier components of the responses corresponds very well with the optimal phase values deduced from the scalar response measures.
View larger version (44K):[in this window]
[in a new window]
Figure 11. Dependence of representative Fourier components of the response on congruence phase in the six examples of Figures 5 and 7. Arrangement of the cells in panels follows that in Figure 5. Notice the different radial scales in different panels. The continuous lines are the best fitting ellipses (Eq. 5). For each cell, various Fourier components are plotted in the same complex coordinates. For clarity, the 95% confidence regions (gray disks) are omitted for some responses. The response to the full edge (white disk) and the fundamental grating alone (m = 1; dark disk) are also shown when they do not overlap compound grating responses. The direction of the progression of congruence phase on the ellipse is opposite for even and odd harmonics, as in Figure 9. a, Cell 450213.u, F1 (h = 1),
Feature tuning in the V1 population
Figure 12a summarizes the comparison of the optimal congruence phase obtained from scalar response measures (as in Fig. 6) and vector response measures (as in Fig. 10) for the population of simple cells. Figure 12b is a similar population summary for the complex cells. These are summaries of the 94 data sets (36 simple, 58 complex) that both passed the original d' criterion for analysis, and the additional criterion on trial-by-trial variance for the complex plane analysis.
View larger version (46K):[in this window]
[in a new window]
Figure 12. Summary statistics of the optimal congruence phase in the population of V1 cells that met selection criteria. a, Simple cells, 36 data sets. b, Complex cells, 58 data sets. Optimal congruence phase was determined separately for seven response measures; DC; total energy; even energy; odd energy; F1 with h = 1; F2 with h = 0; and F2 with h = 2. (See Appendix for explanation of the plotting convention on the complex plane as indicated by the h values.) The wedge diagrams on the diagonal show the distribution of the optimal congruence phase (each dot plotted around the circumference corresponds to a single cell) obtained from each of the seven measures. The significance of a unimodal deviation from a uniform distribution is indicated by asterisk (*p < 0.05; **p < 0.01; ***p < 0.005), and where significant, the direction of this bias (arrows), and its 95% angular confidence range (surrounding wedges) are shown. Above the diagonal, scattergrams compare optimal congruence phases obtained from each pair of response measures via a scatter plot. Note that the domain [0,
For each response measure (indicated by labels above the top row of graphs), the wedge diagrams on the diagonal show the distribution of the optimal congruence phase. The area (not the radius) of each wedge is proportional to the frequency of cells the optimal congruence phase of which fell into the corresponding range of phases. Optimal phase of each cell is indicated by a dot at the corresponding direction on the perimeter. The wedge diagram indicates gross deviations from the uniform distribution on the circle; its details are sensitive to binning. The Rayleigh test (Mardia, 1972
The various response measures are not exactly equivalent measures of phase tuning. Comparisons of optimal congruence phases for pairs of response measures show a range of correlation, as seen in the scattergrams above the top diagonal of Figure 12. In each scattergram, each data point corresponds to a single cell, and compares the optimal phase angles for a pair of response measures, with the horizontal axis of the scattergram corresponding to the row measure (indicated by the row label of the scattergram) and the vertical axis of the scattergram corresponding to the column measure (indicated by the column label of the scattergram). Note that both axes refer to periodic variables. If two response measures predicted identical optimal phases, then all points would fall on the diagonal line of unity slope with zero phase difference between the two predictions. If the optimal phases predicted by two response measures are fully uncorrelated, then the data would be evenly dispersed within a stripe of
As indicated by the values of rc (Fig. 12, top left corner of each scattergram), some pairs of measures were highly correlated (*p < 0.05; **p < 0.01; ***p < 0.005). One such pair is total power versus total even power for both cell classes (Fig. 12a, b). Other pairs of measures were less tightly correlated, and the null-hypothesis of random association on the circle could not be rejected (p > 0.05), e.g., for both cell classes, the F2 measures versus the odd response power or the DC in Figure 12, a and b. In simple cells but not complex cells, measures based on F1 and odd response energy were significantly correlated with those based on DC, the F2, and the even response energy (Fig. 12a). Only in simple cells was the F1 component comparable to the even harmonics, or odd response energy comparable to the even response energy, so it is not surprising that tuning measures based on these quantities behaved in a more random fashion in complex cells.
In simple cells, the sensitivity (as defined by the gain) of the response measures was a good predictor of their feature selectivity (as quantified by the selectivity index, 1
Strong correlation between two circular quantities does not preclude systematic phase differences between them. A systematic phase difference obtained on the population would indicate that two response measures have a relative bias in estimating the optimal feature, which in turn would mandate caution in using one or both measures for this purpose. To examine whether optimal congruence phases predicted by different response measures had such a difference, we plotted the distribution of phase difference in histograms shown in the bottom diagonal matrix for each pair of response measures. The mean phase difference
In summary, response measures that were robust in general were quite sensitive to congruence phase, well correlated with one another, and tuned to very similar optimal phases. The caveat is that the robustness and sensitivity of a response measure to relative phase depends on the extent and ratio of phase-sensitive nonlinear contributions to the odd and even Fourier harmonics, which can vary from cell to cell.
For all measures, the optimal phase is rather evenly distributed in V1, which indicates that V1 neurons can signal the symmetry of spatial waveforms with little bias. We do see a modest but significant population bias in the optimal congruence phase predicted by many response measures toward what corresponds to the line-like (even symmetric) waveform. This raises the possibility that there is a small but significant relative abundance of cells sensitive to line (but not to edges or odd symmetric spatial waveforms) among the V1 neurons. An alternative explanation could lie in the unequal Michelson (peak) contrast of our compound gratings: smallest for the edge-like, largest for the line-like waveform (although all were equal in contrast energy). In this scenario, all other factors being equal, apparent abundance of neurons would correspond to the peak contrast, and in turn, the relative efficacy of the stimulus they are selective for. However, the argument that Michelson contrast and not contrast energy is the effective stimulus for V1 neurons does not seem compelling, because it would predict both that line-selective neurons are apparently most abundant and that edge-selective neurons are apparently least abundant. This is not supported by the wedge diagrams in Figure 12. One possibility that is consistent with our data is that there is a small relative abundance among V1 neurons for both edge and line preference over all others (i.e., an underlying moderately bimodal circular distribution centered on the odd- and even-symmetric waveforms), as might be anticipated from the psychophysics of relative phase sensitivity and discrimination, such as the results of Burr (1980)
Discrimination of spatial features in V1 neurons
The analysis so far has focused on what spatial features drive V1 neurons optimally, and addressed this question via various traditional response measures. This approach demonstrated that there is a certain degree of arbitrariness in choosing a response measure and that the optimal congruence phase may depend on this choice. Similar problems arise when the traditional response measures are used to address a related question: how accurately can V1 cells report the differences in features, i.e., how well can an observer of the spike responses of these neurons discriminate the waveforms of pairs of compound gratings? Discrimination of features depends on response selectivity (i.e., how sharply the response depends on the stimulus parameter of interest), not response size. For example, the response of an orientation-selective neuron is maximized by a stimulus of optimal orientation (tuning peak), but the orientation discrimination sensitivity of the neuron is maximal at the steepest slopes on the rising and falling edge of the orientation-tuning curve (Vogels and Orban, 1990
Shift-reduced Fourier metric
These preliminaries motivated our analysis of neuronal phase discrimination in V1. We used a metric space method (Victor and Purpura, 1996
As we saw earlier, the responses of a neuron to various compound gratings often had similar magnitude and even similar waveforms, but different phases. Thus, much of the information present in these neuronal messages about the gratings is carried by the absolute response phase. However, such phase information cannot be used by an observer to discriminate stimulus waveforms, because the absolute phase is confounded with the absolute starting time of the stimulus cycle of the drifting waveforms. This might suggest an alternative experimental design, more along the lines of psychophysical studies, in which the compound gratings are presented as stationary targets. The temporal confound would be eliminated but in its place would be a spatial confound: that of spatial features and absolute positional information. Although this confound could be eliminated by a dense sampling of stimulus position, doing so would lengthen the experiment by a substantial factor.
The alternative is to recognize that in the absence of knowledge of absolute starting time, discrimination of stimulus waveforms can only be based on intrinsic features of the response. This provides a solution to the temporal confound, as follows. We assume that the responses to each cycle of the stimulus are independent and that spikes within each part of the cycle are identically distributed across trials. That is, the absence of knowledge of absolute starting time can be modeled by allowing spike trains recorded during a single stimulus cycle (Fig. 13a) to be wrapped around the circular stimulus cycle (Fig. 13b,c). Only features of spike trains that can be distinguished even after an arbitrary wrapping are available to discriminate the spatial profiles. That is, in the absence of knowledge of absolute starting time, the intrinsic difference between two responses is the minimum distance that can be achieved after any shift of one spike train relative to the other around the stimulus cycle. We call this minimum distance a shift-reduced Fourier metric (Fig. 13d).
View larger version (44K):[in this window]
[in a new window]
Figure 13. Construction of shift-reduced Fourier metrics. a, Two spike trains, Sa and Sb, corresponding to a single cycle of two periodic stimuli. Vertical dotted line and arrow indicate zero and flow of time, respectively. b, Cyclic wrapping of spike trains, with their original temporal phases. c, The two spike trains, after a temporal phase shift (clockwise rotation) of Sb that minimizes the difference between the two trains for a Fourier metric. d, Calculation of the shift that minimizes the distance between the two spike trains for a particular Fourier metric. A vector of those Fourier components that constitute the Fourier metric, the DC and F1 components in the case illustrated, represent each spike train. These components are obtained by the Fourier transform of the spike trains in the usual manner. The DC and F1 components are shown for each spike train: solid black for Sa and solid white for Sb. The square of the difference between Sa and Sb (the quantity to be minimized) is the sum of the squares of the distances between the corresponding Fourier components of Sa and a phase-shifted version of Sb. The DC component (equal to the difference between the solid black and white vectors on the vertical axis) is unchanged by phase shifts. However, the difference between the F1 components (the solid black and white vectors in the horizontal plane) is minimized when the F1 component of Sb (solid white in horizontal plane) is rotated to have equal phase to that of the F1 component of Sa (dashed white overlapping solid dark in horizontal plane). When multiple harmonics are used in the Fourier metric, each non-zero frequency corresponds to a separate plane, and phase shifts rotate the response vectors within each plane. The phase shift that minimizes the distance between the spike trains is the one with corresponding rotations (in the plane of each harmonic) that minimize the sum of the squares of the distances between the vectors for each harmonic. An exhaustive search algorithm identified this shift.
Shift-reduced Fourier metrics can be constructed from any set of Fourier components. Enlarging the subset of Fourier components approximates a waveform increasingly well, but variations in the response carry information about the stimulus only up to a certain temporal precision (Mechler et al., 1998b
View larger version (26K):[in this window]
[in a new window]
Figure 14. Discrimination of pairs of compound gratings in a single V1 simple cell, the most sensitive to congruence phase differences in our sample. Thresholds are compared to human psychophysical thresholds obtained by Burr (1980)
Similar observations held for the discrimination of all other stimulus pairs (Fig. 14b). Most stimulus pairs, especially nearby elements of the feature space, evoked very similar numbers of spikes. Thus, the DC alone allowed very poor or no discrimination of compound gratings (an ordinate of close to 0 bits at the 0 point on the abscissa). The same held for a shift-reduced Fourier metric that included only the first harmonic in addition to the DC (abscissa = 1). This is because the metric aligns the phase of the first harmonic, leaving only the DC response and the first harmonic amplitude to discriminate among the stimuli. Because these measures often varied little across stimuli, discrimination on their basis is necessarily poor. Discrimination increased substantially when multiple Fourier components were included in a shift-reduced metric, because it was typically not possible to identify a single phase shift that simultaneously brought multiple harmonics into alignment. Thus, once multiple harmonics were included in the shift-reduced metric, relative temporal phase can play a role in discrimination of stimulus pairs. For most pairs of compound gratings, discrimination information increased with the inclusion of response components up to F8. The information curves typically cut off above F16, indicating that higher components carried no independent stimulus-related message and/or were too variable to be useful. Similar observations were made for most neurons (with a few exceptions, where information peaked at a lower harmonic, but the sharp decline in discrimination was above F8). Therefore, we used the shift-reduced Fourier metric based on the DC and the first eight harmonic components of the response for subsequent information calculations.
The fact that the responses of most neurons carried temporally encoded information about the spatial waveform of these stimuli primarily in their first eight Fourier components reflects, of course, the temporal frequency content of the stimuli. For simple cells, this might be considered a trivial observation, but for complex cells it is not, because their responses do not merely mimic the temporal modulation of the stimulus, even for standard sinusoidal gratings. Moreover, the nontrivial nature of this observation is demonstrated by changing the drift speed of the compound gratings while keeping the spatial frequencies constant. A fourfold increase in the drift speed (12°/sec instead of 3°/sec) changes the temporal fundamental of the stimuli fourfold (from 1 to 4 Hz). Correspondingly, the time represented by a constant phase shift by any particular harmonic is decreased fourfold. Nevertheless, for most neurons, the peak of the information curve obtained with the shift-reduced Fourier metrics remained at the eighth response harmonic of the stimulus fundamental. This means that the precision of the encoded temporal detail increased fourfold "in tune" with the fourfold increase in the temporal frequency content of the stimulus. For only 10 data sets (20% of high speed data), information curves peaked for Fourier metrics that used truncated series up to and including the second or fourth rather than the eighth harmonic of the fundamental. The maximum precision of temporal coding of spatial detail in these cells (defined by the half period of the fourth harmonic) is ~30 msec, and in those neurons with an information peak that remained at the eighth harmonic truncation, the precision limit would be 15 msec or better. These numbers correspond well to earlier reports of temporal coding of temporal phase (Geisler et al., 1991
Neuronal discrimination thresholds for congruence-phase
One way to summarize the information measurements obtained with the shift-reduced Fourier metric shown in Figure 14b is to construct threshold functions by analogy to psychophysical threshold functions (Fig. 14c). To measure neuronal thresholds, neurometric functions using spike counts were used by Movshon and colleagues (Newsome et al., 1989
In a similar manner, discrimination thresholds can be calculated with each of the other waveforms serving as the reference. Figure 14d shows (open symbols) discrimination threshold (difference of congruence phases) as a function of the congruence phase of the reference waveform. This threshold function summarizes waveform or 1D feature discrimination for this simple cell.
The cell shown in Figure 14 was the most sensitive to congruence phase in our sample. The information content of pairwise discrimination based on single responses was often close to the ideal maximum of 1 bit (perfect discrimination), especially for pairs of compound gratings with congruence phase that differed by
Although this cell, one of the most sensitive to differences in spatial waveform in our V1 sample, can perform phase discrimination for line-like stimuli at least as well as the normal human observer (Fig. 14d, filled symbols) (converted to our units from the Burr study), it is well outperformed by the human observer for discrimination of edge-like stimuli. The pattern of dependence of threshold on congruence phase in this cell is exactly the opposite of the human. Thus, we ask whether this discrepancy holds in V1 neurons in general. We also determine how spatial feature discrimination in V1 neurons compares with that in human observers for the typical neuron and not just for the most sensitive V1 neurons.
To answer these questions, we analyzed all data that passed our initial criterion for analysis (as stated in the first section of Results: 121 data sets from 31 simple cells and 46 complex cells). Within this group, we computed the phase-discrimination threshold functions for all neurons in the subset that met the 82%-correct threshold criterion. This turned out to be a stringent criterion: only ~10 % of this group (20% of the simple cells and 5% of the complex cells) met this criterion for at least one reference phase (Fig. 15a). Of these most sensitive V1 neurons, eight data sets from five simple cells, including the example in Figure 14, and 1 complex cell, met the 82%-correct criterion for all eight congruence phases used as reference. Connecting lines of different types indicate five of these threshold curves. Only isolated data points are plotted for the three remaining neurons, at which measurable thresholds (at the 82% criterion) were identified for a subset of the eight reference congruence phases. The human threshold curve measured by Burr (1980)
View larger version (31K):[in this window]
[in a new window]
Figure 15. a, Phase discrimination thresholds in all the 13 V1 data sets (9 from simple cells, 4 from complex) that had measurable thresholds at the 82%-correct response criterion for at least at one reference phase. The most sensitive simple cell (large open circles connected by solid line) is the same as in Figure 14. Seven more data sets from simple cells [4 of these are shown with different open symbols (small circle, triangle, square, diamond) connected by distinct line types], and one complex cell (filled triangles connected by solid line) met the criterion at all test phases, and five other data sets (from 4 neurons) had measurable thresholds at some but not all test phases (isolated data points). Stars connected with the thick line depict the median neuronal threshold. Human thresholds were replotted from Figure 14 as filled symbols connected by dotted line. b, Phase discrimination thresholds in the 46 data sets from 34 V1 neurons (17 from 12 simple cells, 29 from 22 complex cells), ~40% of all data analyzed in both cell class, that had measurable thresholds at the less stringent 68%-correct response criterion. For clarity, threshold functions of individual cells are not highlighted by connected lines. Connected stars depict the median neuronal threshold. Nonmeasurable thresholds (difference of congruence phases
The close agreement between the human thresholds and the lower boundary of the data in Figure 15a is consistent with a winner-take-all mechanism in V1 that could account for the psychophysical thresholds (Parker and Newsome, 1998
Varieties of feature discrimination in V1 cells
Figure 15 summarizes, for single neurons and for an arbitrary threshold level, the dependence of phase discrimination on the reference congruence phase. However, there is no guarantee that phase discrimination is a uniform and monotonic function of phase difference. Rather, the phase discrimination capabilities of a single neuron may be a more general function of the congruence phases of the two stimuli to be discriminated. Indeed this is the case, as the discrimination functions for six cells, representative of our V1 sample, show (Fig. 16). The x- and y-axes of these plots represent the congruence phase (i.e., position in feature space) of the two compound gratings in a discrimination pair. The height of the surface is phase discrimination, measured as information obtained with the shift-reduced Fourier metric. Perfect discrimination of two stimuli corresponds to 1 bit, the maximum possible information in paired comparison. The trough touching zero bits on the diagonal represents self-comparison (i.e., the absence of discrimination of two identical stimuli). Because the feature space has period
View larger version (60K):[in this window]
[in a new window]
Figure 16. Examples of the information available in the responses of six representative V1 neurons for discriminating pairs of compound gratings. Information estimates, obtained with the shift-reduced Fourier metric truncated at the eighth Fourier harmonic (F8) of the response, are plotted on the vertical axis (in units of bits) as a function of the congruence phases of the discrimination pair (
In these plots, feature-selective discrimination corresponds to elevated surfaces (peaks or ridges) in the narrow vicinity of the congruence phase of the selected waveform. Discrimination that is a monotonic function of phase difference would result in a surface that increases monotonically in height with increasing distance from the diagonal (and its periodic repeats at intervals of
Figure 16a shows a sensitive simple cell and Figure 16b shows a sensitive complex cell. The information surface for both cells has a prominent off-diagonal ridge (near maximum offset in feature space, i.e., the locus x
The simple cell in the middle left (Fig. 16c; this is the paradigmatic simple cell of Fig. 2-5) has a different behavior. Rather than diagonal ridges, the information surface has ridges parallel to the coordinate axes, with peaks on the lines
The last three examples (Fig. 16d-f) all have one or more prominent discrete peaks (plus their periodic replicates as required by the symmetry and periodicity of the plot) rather than ridges. The complex cell in Figure 16d has a single peak at (
The last two cells in Figure 16 show another narrowly tuned behavior, which could be called feature-selective nonopponent. Remarkably, they are tuned to discriminating pairs of waveforms that are very similar to each other (i.e., occupy nearby positions in feature space and form near-diagonal positions in these plots), but they do not discriminate stimuli that are in opponent positions in our feature space. The simple cell (Fig. 16e), with its single discrimination peak at (
Every cell in our V1 sample had, in pure form or in some combination, features displayed by one of these three examples. Most neurons had a ridge (indicating some form of nonselective behavior). In most cases, the ridge was near the maximum offset in feature space (the off-diagonal locus x
The average discrimination, taken as the information surface averaged across neurons, displays a mixture of these features. Figure 17a shows the average waveform discrimination in V1 simple cells, calculated across 43 data sets. The average information is dominated by the feature-nonselective opponent ridge (at maximum offset in feature space). Superimposed on this ridge near (
View larger version (28K):[in this window]
[in a new window]
Figure 17. The information (bits on vertical axis) available in the average V1 neuron for discriminating pairs of 1D spatial waveforms. N indicates the number of averaged data sets. The information analysis and plotting convention are the same as in Figure 16. a, Simple cell data sets (N = 43); b, complex cell data sets (N = 78); c, all V1 data sets (N = 121).
Unimpressive as these information values are in the average V1 neuron, the levels of selectivity and the variety of patterns exhibited by the V1 neurons most tuned to phase discrimination
and these include complex cells, not just simple cells
Finally, we address the relationship between the optimal congruence phase defined as the tuning peak (predicted by the fitted functions such as in Fig. 6 or the ellipses in Fig. 9) and the congruence phase of the features corresponding to the discrimination peak (identified by the maximum information on the information surfaces such as those in Fig. 16). Within the domain of orientation, the relationship between tuning and discrimination is straightforward: it is determined by the slope of the tuning function, however, for the problem at hand, the relationship is more complicated. For example, consider the simple cell in Figures 6 and 9 again. The optimal congruence phase was
Spatial phase discrimination: comparison of simple and complex cells
Finally, we analyze the relationship between sensitivity to spatial features and the traditional simple/complex classification of V1 neurons. The traditional view, based largely on conventional tests with bars and simple grating stimuli, holds that the quasilinear nature of simple cells allows them to convey precise positional and spatial phase information, whereas the nonlinear spatial integration that distinguishes complex cells markedly reduces the positional and phase information that they can transmit (Movshon et al., 1978b
We examined whether these notions extended to discrimination of spatial features, which requires both spatial phase sensitivity (considered to be characteristic of simple cells) and spatial nonlinear interactions (considered to be characteristic of complex cells). The conventionally used quantitative classifier of V1 neurons is the modulation ratio (Skottun et al., 1991
Figure 18 shows the relationship of the modulation ratio to the two measures: peak sensitivity (Fig. 18a) and lowest threshold (Fig. 18b). The top panels are scatter plots of either measure against the modulation ratio, considered as an index of cell classification. A positive correlation was expected if simple cells possessed significantly greater feature discrimination sensitivity than complex cells (Fig. 18a). Conversely, a negative correlation was expected if simple cells possessed significantly lower feature discrimination thresholds than complex cells (Fig. 18b). In fact, neither scatter plot shows any significant dependence of these measures on the index of cell class (r < 0.1). Additionally, the data show no evidence for a dichotomy in feature discrimination along the class index, either at the class boundary (a modulation ratio of 1) or anywhere else. Most V1 neurons, whether simple or complex, exhibit weak discrimination of congruence phase. The most sensitive neurons (top half of the scatter plot in a, bottom quarter in b) form a small but nondistinct minority within the continuum of the overall V1 population. Although the simple cells (left half of each scatter plot) seem slightly over-represented among the most sensitive neurons, the most sensitive neurons also include complex cells (right half of each scatter plot).
View larger version (35K):[in this window]
[in a new window]
Figure 18. a, The dependence of phase discrimination sensitivity, measured by peak information in surface plots of the type shown in Figure 16 (vertical axis) on cell class (indexed by the modulation ratio, F1/DC, for the optimal drifting sine grating). Dotted lines indicate the conventional simple/complex class boundary. Top, Scatter plot of the modulation ratio, F1/DC versus information peak. They are uncorrelated (r = 0.089). The neurons most sensitive to spatial features occupy the top part of the plot. Middle, Distribution of peak information (marginal distribution, along the vertical axis, of data in scatter plot at the top). Histograms are fraction of data sets separately from simple cells (red) (N = 59; median = 0.088) and from complex cells (blue) (N = 100; median = 0.092); the two distributions are not different (Kolmogorov-Smirnov test; d = 0.13; p = 0.533). The most sensitive cells fall to the right of the heavy arrowhead. Bottom, Conventional simple/complex classification of V1 neurons based on the modulation ratio. Our sample of neurons analyzed in a (marginal distribution along the horizontal axis of scatter plot at top) exhibits the well known bimodal distribution. b, Similar analysis to that shown in a, but sensitivity to congruence phase, quantified by the lowest congruence phase difference in each neuron that met the 68%-correct (0.1 bit equivalent) criterion, is analyzed instead of peak information. Only a subset of the data analyzed in a met this criterion. Top, Scatter plot of the modulation ratio, F1/DC versus lowest threshold; they are uncorrelated (r =
The middle panels in Figure 18 show the marginal distribution of each measure of feature sensitivity. For either one, the Kolmogorov-Smirnov test did not reject the null hypothesis that the simple and complex samples came from the same distribution. However, the similarity of the distributions in simple and complex cells comes with an apparent relatively higher abundance of simple than complex cells among the most sensitive neurons. Is this significant? For the analysis in Figure 18a, we selected 159 data sets of a total of 226 recorded (59 of 74 simple, and 100 of 152 complex). Of these, only 21 data sets (10 simple and 11 complex data sets) exhibited peak sensitivity of 0.2 bits or higher (an arbitrary threshold for high feature sensitivity, indicated by the arrow in the middle panel of Fig. 18a). This translates to a higher proportional representation of simple than complex cells, 14% (17%) of all (analyzed) simple data sets versus 7% (11%) of all (analyzed) complex cell data sets, but not to a statistically significant degree (p > 0.1 by 2 × 2
The distribution of the modulation ratio in both samples (Fig. 18, bottom panels) exhibits the well known bimodality with the usual dip near 1. Thus our sample, as conventional classification is concerned, was typical of the results of others (Skottun et al., 1991
Summary
We used compound gratings consisting of a fundamental and its first three odd harmonics the congruence phase of which was varied systematically. This stimulus set was well suited for dissociating linear and nonlinear mechanisms (only odd harmonics were present), and also for dissociating phase-specific nonlinearities from those sensitive only to stimulus power (all waveforms were of equal energy). Moreover, the use of compound gratings containing several harmonics, and the choices of relative phases, facilitated an interpretation in terms of extraction of simple features (i.e., the phase coherences typical of lines and edges). With these stimuli, we obtained a number of novel findings:
(1) Phase-specific nonlinear interactions, up to fifth and higher orders, are present with similar strength and harmonic composition in both simple and complex cell responses. In simple cells, odd harmonic responses were of slightly higher gain then even harmonics, but they were equally selective for spatial feature. In complex cells, even harmonics were of much higher gain, but the odd harmonics were more selective.
(2) The preferred congruence phase in phase-tuned V1 neurons is relatively poorly predicted by the spike count, but equally well predicted by other scalar (e.g., energy) and vector (significant Fourier harmonics) response measures.
(3) Cells were encountered that are tuned to edges, lines but also intermediate stimuli with mixed symmetry. Although a few cells that approximated a phase-insensitive energy operator were also encountered, even most complex cells were tuned to congruence phase.
(4) Although the distribution of the preferred congruence phase in V1 was broad, with all congruence phases represented, the population as a whole, regardless of cell class, displayed a slight bias toward lines and possibly edges. This may represent a genuine relative abundance of even symmetry preference among V1 neurons.
(5) Feature preference and selectivity also varied within a local cluster of V1 neurons.
(6) Information analysis using a method that was insensitive to absolute phase or position (the shift reduced Fourier metric) revealed that V1 neurons encoded relative phase in the entire frequency band present in the stimuli (as limited by the pass-band of the linear filter of the receptive field).
(7) The envelope of feature discrimination thresholds, estimated from information analysis, in the most sensitive V1 neurons (5% of complex cells and 20 % of simple cells) matched the human psychophysical thresholds, but the dependence of feature discrimination threshold on congruence phase showed many patterns in single V1 neurons, and most differed from the pattern in human observers.
(8) The responses of most cells were rather noisy and displayed low sensitivity to relative phase. A minority of V1 neurons that included both simple and complex cells were highly sensitive and selective. The peak discrimination sensitivity of the average V1 neuron does not reach the level of 70% correct. Simple cells on average were twice as sensitive at peak as complex cells. However, the distribution of peak feature sensitivity and the lowest threshold of feature discrimination were both indistinguishable between simple and complex cells and indicated a continuum in V1.
(9) The existence, among sensitive neurons, of nonlinearities tuned to feature pairs or feature differences, suggests that a subpopulation of V1 neurons does more than simply pass on the relevant information necessary for spatial feature detection to a downstream stage.
DISCUSSION
Visual perception relies on the successful identification and localization of features such as lines and edges that define the shape and boundaries of objects. These features arise from spatial phase congruence, the local agreement of spatial phase of multiple harmonic components of an image. However, despite their importance, the neural computations underlying detection and discrimination tasks based on relative spatial phase are unclear. Because feature extraction requires phase-selective nonlinear filtering, the earliest stage at which this operation is likely to occur along the visual pathway is in the primary visual cortex. Our experiments were designed to test whether V1 neurons can account for the specificity of feature extraction for lines and edges by studying their sensitivity to relative phase.
Physiology: comparison with other studies
Only a few studies have examined the sensitivity of single neurons in the primary visual cortex to relative spatial phase or phase coherence. In anesthetized cats, De Valois and Tootell (1983)
Spatial feature discrimination in simple and complex cells
We characterized discrimination of elementary features (on the basis of relative spatial phase) according to several measures and found little if any correlation between these measures and the simple/complex distinction. This result is unexpected from the traditional view of the phase sensitivity of simple and complex cells. High versus low spatial phase sensitivity in simple versus complex cells is widely held as one of the clearest and most quantitative indicators of a functional dichotomy between these neuronal classes (Movshon et al., 1978a
Correspondence with psychophysics of feature detection and discrimination
The psychophysics of absolute and relative spatial phase discrimination has been amply studied [for a review of much of the early work see De Valois and De Valois (1980)
Our results on neuronal phase discrimination thresholds obtained at the 82%-correct equivalent criterion are consistent with the notion that a winner-take-all mechanism among the V1 neurons most sensitive to relative phase can account for Burr's psychophysical thresholds, but this explanation carries several caveats: (1) we used five times higher contrast than was used for the psychophysics; (2) we optimized fundamental spatial frequency for the bandwidth of the cell, but psychophysics was done with a fixed 3 c/° fundamental, which was near optimal for the observer but presumably not for all of his V1 neurons; (3) we used four harmonic components (f + 3f + 5f + 7f), making richer and potentially more salient waveforms than the two-component (f + 3f) stimulus that was used in psychophysics. All of these differences act to favor the neuronal thresholds; without them, even the best single neuronal thresholds would be expected to be higher than human psychophysical thresholds.
Computational implications for the functional circuitry in visual cortex
Although the most sensitive simple and complex cells (e.g., those of Fig. 15a) show discrimination approaching psychophysical levels, the correspondence with human thresholds typically did not hold for all test phases for any single neuron. Thus we must consider the possibility that pooling of signals from several single neurons is necessary to account for the psychophysical thresholds. Across V1 neurons, we observed a wide variety of pattern in the dependence of threshold on test phase; most were different from the dependence observed in human thresholds. Furthermore, unlike the situation for orientation and spatial frequency, the observed variation in phase selectivity among nearby neurons is considerable. Thus, although spatial pooling likely plays a considerable role, it must be phase specific, rather than locally indiscriminate.
We encountered V1 neurons with a phase discrimination function that was tuned to feature pairs (e.g. selective feature-opponent) (Fig. 16d-f), selective presence of a feature (X-OR) (Fig. 16c), or feature differences (nonselective feature-opponent) (Fig. 16a,b). Their existence raises the possibility that V1 neurons actively participate in feature detection and discrimination and do not merely act as linear filters that pass on the relevant information to downstream detector mechanisms. On the other hand, because (as we have shown) V1 neurons do not account for the pattern of feature sensitivity, later stages must do more than merely improve thresholds.
It is unclear how these precursors of feature discriminators in V1 could be constructed, within the known functional circuitry of V1. Creating nonselective feature-opponent cells by pooling signals from many variously tuned selective feature-opponent cells encounters some difficulty, given the observation that both simple and complex cells can act as nonselective feature-opponent discriminators. On the other hand, building nonselective feature-opponent cells from selective ones with the aid of selective inhibition leaves open the question of where the selectivity of inhibition arises. One way to resolve these difficulties might be to invoke feedback from extrastriate visual areas, such as V2, rather than to attempt to build these properties solely from the intrinsic circuitry of V1, but this must be considered speculative at present. These and other questions, such as those related to the cortical processing of 2D shape information, should motivate experiments in extrastriate cortex that could expand on these efforts.
FOOTNOTES Received Nov. 20, 2001; revised Jan. 29, 2002; accepted Jan. 30, 2002.
This work was supported by National Institutes of Health Grants EY9314 (J.D.V., F.M.), EY7138 (D.S.R.), and GM7739 (D.S.R.).
Correspondence should be addressed to Ferenc Mechler, Department of Neurology and Neuroscience, Weill Medical College of Cornell University, 1300 York Avenue, New York, NY 10021. E-mail: fmechler{at}med.cornell.edu.
APPENDIX
A formalism for compound grating stimuli
We consider general compound grating stimuli, and then specialize to the particular drifting compound gratings used in these experiments. This approach adds little notational inconvenience and clarifies what is generic to compound gratings and what is specific to our choice of stimuli.
We assume that the compound gratings are composed of L sinusoidal components, each at integer multiples of a common fundamental frequency f. The set of integer multiples are denoted {k(1), k(2), ... , k(L)}, and the corresponding frequencies are denoted {Fk(1), Fk(2), ... , Fk(L)}. Thus Fk(j) is the frequency of the jth component, which is the k(j)th harmonic of a common fundamental, i.e., Fk(j) = k(j)f. We use the vector
as a compact notation for the set of components {k(1), k(2), ... , k(L)} and use similar vector notation for other analogous sets of indices below.
Each single sinusoid of a compound grating can be written as a sum of a complex exponential and its complex conjugate pair. This decomposition allows for convenient tracking of the phase and frequency of nonlinearly interacting components. Correspondingly, a sum of sinusoids W
(t) corresponding to a compound grating of components
(A1)
Here, the indexing is extended into negative frequencies to accommodate the complex notation, and all quantities with a zero index are equal to 0 by convention. The multipliers ak and their complex conjugates
k are the amplitudes of the sinusoidal components and may depend on contrast, spatial frequency, and space. Complex conjugation for negative components guarantees a real sinusoid at each of the positive frequencies. The convention that a0 = 0 indicates that only contrast, but not the mean level of light intensity, is considered a part of the stimulus. The Frequencies and phases in the linear response
The essence of a linear system is that its response contains components only at frequencies that are present in the stimulus and that the phase of each component of the response has a constant offset with respect to the phase of the corresponding component of the stimulus. Thus, a linear response, R
(t), to a compound grating W (t) is described by a linear combination (i.e., the sum of complex multiples) of the exponential terms ei2 Fk(l)t ei
k(l) and their complex conjugates present in the stimulus:
(A2)
Here, the bk complex amplitudes and their conjugate pairs are constant for fixed contrast and spatial frequency, but they differ from the multipliers ak in Equation A1 because of the filtering properties of the system.
Nonlinear interaction frequencies in the response
The R(t) response of nonlinear system to a sinusoidal sum W
(t) can typically be well approximated by an appropriately chosen sum, of the nth-order response contributions (Bedrosian and Rice, 1971 is a sum of all possible products of n frequency components, each of which is selected from the set of components present in the stimulus. This generalizes the description of the linear response (in Eq. A2), which corresponds to the sum of first order terms, n = 1, to nonlinear responses of finite order:
(A3)
Here, the summation over the vector index
is compact notation for a summation over each of its n component indices, where each component index ml runs through all negative and positive values up to L. This in turn selects all combinations of n components of ) indicates that it results from a sum of the n frequencies labeled by m1, m2, ... , mn: Fk( ) = Fk(m1) + Fk(m2) + ... + Fk(mn). Note that the response frequency Fk( ) depends on the component frequencies specified by the frequency n-tuple , and also on their signs, because each component index can reference positive or negative frequencies. A similar dependence holds for the phase of the response. Distinct frequency n-tuples in Equation A3 (i.e., n-tuples of the same order n with indices
and ' that differ by more than merely a permutation, and perhaps n-tuples of distinct orders n and n') may give rise to the same output frequency. However, as we show below, these output components will generally have different dependences on the phases of the input frequencies and thus may be separated. The compact notation of Equation A3 has another potential ambiguity. Frequency n-tuples that differ only by a permutation are separately listed in Equation A3; despite the fact that they reflect identical interactions (and necessarily have identical dependences on the phases of the input frequencies). Eliminating these multiple listings is important for normalizing nonlinear kernels (Victor and Knight, 1979 Permitted intermodulation frequencies and their phases in our stimulus design
Our compound grating stimuli are constructed from the first four odd harmonics of a fundamental f, namely,
(A4)
We look for nonlinear responses at output frequencies F(
) = Fk(m1) + Fk(m2) + ... + Fk(mn) that are the r th harmonics of the fundamental temporal frequency in the stimulus: F = rf = rF1. If a response at the frequency rf can occur as part of an n th order nonlinearity, then:
The specific integer
(A5) odd-integer index mapping in our choice of input frequencies (Eq. A4) puts constraints, via Equation A5, on the range of harmonics that the response frequency may span for a given n. In particular, since the largest harmonic is seven times the fundamental and all harmonics are odd, we must have r
{
Contributions to any given response harmonic r may include nonlinear interactions of the same parity but with different orders n, and for each n there may be many alternative n-tuples of stimulus frequencies that all sum to a given r. These several contributions to the same frequency component of the response are not equally weighted for three reasons: (1) the contrasts of the components in the stimulus, ak, are inversely proportional to frequency, (2) interactions at lower orders of nonlinearity are likely to be larger than interactions at higher orders of nonlinearity, and (3) the tuning of the neuron is likely to have different sensitivities to each of the component gratings. On the basis of only the generic considerations 1 and 2, the interactions due to n -tuples composed of a small number of low-frequency harmonics are expected to be larger than those composed of high-frequency harmonics, or a larger number of harmonics. For example, of the many contributions to the first harmonic response, F1, the third order contributions from the triplets f + f
By design, the relative phases of the components are fixed in each of our compound gratings at the congruence phase. That is,
) of the n-th order nonlinear response at the frequencies Fk( ) = rf will be an integer multiple p of the congruence phase
Thus, the range of values of the phase multiplier p for a given order of nonlinearity n is p
(A6)
View this table: [in this window]
[in a new window]
Table A1. Consequences of the rules (Eq. A4-6) that govern the pattern of nonlinear frequency combinations of intermodulating stimulus components and their phase signature in our special stimulus design. The phase signature of nonlinear responses
We now describe how, for a given response harmonic r, the realized set of phase multipliers p (organized in a column under p in Table A1 across all n orders of nonlinearity) determine the geometry of the locus of responses on the complex plane. As the congruence phase
As described in the main text, we plot responses in the complex plane, after rotating their phases backward by an amount
To see that these circles indeed sum to an ellipse, consider the (x, y) coordinates (the real and imaginary parts, respectively, on the complex plane) of the points that are in phase correspondence on the two circles. The (x, y) coordinates of the p = h + 2 trajectory, parametric in phase
+), y = r+ sin(2
Nonlinear responses with |p
F1: The response measured at the fundamental temporal frequency F1 includes all components listed in Table A1 under the column r = 1, including but not necessarily limited to a linear part (n = 1). When plotted with h = 1, the contribution from the linear part will be stationary. That is, its phase multiplier is p = 1, and |p
4. Thus, when F1 responses are plotted with the phase correction h = 1, the parameters of the best-fitting ellipse reflect the combined influence of third-order and higher-order responses, but the distortion is attributable solely to fifth and higher odd-order nonlinearities.
F3, F5, etc: Plots of these higher harmonic responses with the phase correction h = 1 will also yield, at worst, an ellipse, unless nonlinearities of order n = 5 or higher are present. For example, the response measured at the temporal frequency F3 includes all components listed in Table A1 under the column r = 3. These include a linear component (elicited by the F3 grating, in the n = 1 row) and also nonlinear responses (in the rows n = 3 and n = 5; and for higher n, data not shown). For h = 1 and n = 3, only terms with p
F2, F4, etc: The responses at the even harmonics of the fundamental temporal frequency do not contain a linear part. For this reason, plotting their responses with a phase correction h = 1 is no longer natural. Rather, one could choose to reference even harmonic responses to the phase of the second order (lowest even order) contributions. Second order contributions to all even harmonic response frequencies come in one of two intermodulation patterns: either with difference frequencies (p = 0; e.g., F5
DC: The spike count is a special case of even harmonics because it only has a real part. That is, every term in Equation A3 is accompanied by its complex conjugate. A plot of the DC response, a scalar, is equivalent to plotting the DC response as a complex value with h = 0, and projecting it on the real axis. Table A1 (column r = 0) shows that contributions to DC from second-order nonlinearities (n = 2) all have p = 0, and are thus independent of the congruence phase
REFERENCES
- Ainsworth A, Dostrovsky JO, Merrill EG, Millar J (1977) An improved method for insulating tungsten micro-electrodes with glass. J Physio (Lond) 269:4P-5P.
- Albrecht DG, Geisler WS (1991) Motion selectivity and the contrast-response function of simple cells in the visual cortex. Vis Neurosci 7:531-546[Web of Science][Medline].
- Albrecht DG, Hamilton DB (1982) Striate cortex of monkey and cat: contrast response function. J Neurophysiol 48:217-237
[Free Full Text] .- Atkinson J, Campbell FW (1974) The effect of phase on the perception of compound gratings. Vision Res 14:159-162[Web of Science][Medline].
- Badcock DR (1984a) Spatial phase or luminance profile discrimination? Vision Res 24:613-623[Web of Science][Medline].
- Badcock DR (1984b) How do we discriminate relative spatial phase? Vision Res 24:1847-1857[Web of Science][Medline].
- Bedrosian E, Rice SO (1971) The output properties of Volterra systems (nonlinear systems with memory) driven by harmonic and Gaussian inputs. Proc IEEE 59:1688-1707.
- Bonds AB (1989) Role of inhibition in the specification of orientation selectivity of cells in the cat striate cortex. Vis Neurosci 2:41-55[Web of Science][Medline].
- Burr D, Morrone C (1992) Nonlinear model of feature detection. In: Nonlinear vision: determination of neural receptive fields, function, and networks, Chap II (Pinter RB, Nabet B, eds), pp 309-327. Boca Raton, FL: CRC.
- Burr DC (1980) Sensitivity to spatial phase. Vision Res 20:391-396[Web of Science][Medline].
- Burr DC, Morrone MC, Spinelli D (1989) Evidence for edge and bar detectors in human vision. Vision Res 29:419-431[Web of Science][Medline].
- Burton GJ, Moorhead IR (1981) Visual form perception and the spatial phase transfer function. J Opt Soc Am 71:1056-1063[Medline].
- Carandini M, Heeger DJ, Movshon JA (1997a) Linearity and normalization in simple cells of the macaque primary visual cortex. J Neurosci 17:8621-8644
[Abstract/Free Full Text] .- Carandini M, Barlow HB, O'Keefe LP, Poirson AB, Movshon JA (1997b) Adaptation to contingencies in macaque primary visual cortex. Philos Trans R Soc Lond B Biol Sci 352:1149-1154
[Abstract/Free Full Text] .- Carandini M, Movshon JA, Ferster D (1998) Pattern adaptation and cross-orientation interactions in the primary visual cortex. Neuropharmacology 37:501-511[Web of Science][Medline].
- Chance FS, Nelson SB, Abbott LF (1999) Complex cells as cortically amplified simple cells. Nat Neurosci 2:277-282[Web of Science][Medline].
- Cover TM, Thomas JA (1991) In: Elements of information theory. New York: Wiley.
- Dakin SC, Hess RF (1999) Contour integration and scale combination processes in visual edge detection. Spat Vis 12:309-327[Web of Science][Medline].
- De Valois KK, Tootell RB (1983) Spatial-frequency-specific inhibition in cat striate cortex cells. J Physiol (Lond) 336:359-376
[Abstract/Free Full Text] .- De Valois RL, De Valois KK (1980) Spatial vision. Annu Rev Psychol 31:309-341[Web of Science][Medline].
- De Valois RL, Albrecht DG, Thorell LG (1982) Spatial frequency selectivity of cells in macaque visual cortex. Vision Res 22:545-559[Web of Science][Medline].
- Dean AF, Tolhurst DJ (1983) On the distinctness of simple and complex cells in the visual cortex of the cat. J Physiol (Lond) 344:305-325
[Abstract/Free Full Text] .- Dean AF, Tolhurst DJ (1986) Factors influencing the temporal phase of response to bar and grating stimuli for simple cells in the cat striate cortex. Exp Brain Res 62:143-151[Web of Science][Medline].
- Field DJ, Hayes A, Hess RF (1993) Contour integration by the human visual system: evidence for a local "association field." Vision Res 33:173-193[Web of Science][Medline]. ***
- Fisher NI (1993) In: Statistical analysis of circular data. Cambridge, UK: Cambridge UP.
- Geisler WS, Albrecht DG, Salvi RJ, Saunders SS (1991) Discrimination performance of single neurons: rate and temporal-pattern information. J Neurophysiol 66:334-362
[Abstract/Free Full Text] .- Georgeson MA, Turner RS (1985) Afterimages of sinusoidal, square-wave and compound gratings. Vision Res 25:1709-1720[Web of Science][Medline].
- Hamilton DB, Albrecht DG, Geisler WS (1989) Visual cortical receptive fields in monkey and cat: spatial and temporal phase transfer function. Vision Res 29:1285-1308[Web of Science][Medline].
- Heeger DJ (1992) Normalization of cell responses in cat striate cortex. Vis Neurosci 9:181-197[Web of Science][Medline].
- Hubel DH, Wiesel TN (1962) Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J Physiol (Lond) 160:106-154
[Free Full Text] .- Klein SA, Tyler CW (1986) Phase discrimination of compound gratings: generalized autocorrelation analysis. J Opt Soc Am [A] 3:868-879[Web of Science][Medline].
- Kovacs I, Julesz B (1993) A closed curve is much more than an incomplete one: effect of closure in figure-ground segmentation. Proc Natl Acad Sci USA 90:7495-7497
[Abstract/Free Full Text] .- Levitt JB, Sanchez RM, Smith EL, Movshon JA (1990) Spatio-temporal interactions and the spatial phase preferences of visual neurons. Exp Brain Res 80:441-445[Web of Science][Medline].
- Maffei L, Fiorentini A, Bisti S (1973) Neural correlate of perceptual adaptation to gratings. Science 182:1036-1038
[Abstract/Free Full Text] .- Mardia KV (1972) In: Statistics of directional data. London: Academic.
- Mechler F, Ringach DL (2002) On the classification of simple and complex cells. Vision Res 42:1017-1033[Web of Science][Medline].
- Mechler F, Reich DS, Victor JD (1998a) Receptive field nonlinearities in macaque primary visual cortex (V1) are not selectively activated by edges. Soc Neurosci Abstr 24:1979.
- Mechler F, Victor JD, Purpura KP, Shapley R (1998b) Robust temporal coding of contrast by V1 neurons for transient but not for steady-state stimuli. J Neurosci 18:6583-6598
[Abstract/Free Full Text] .- Mechler F, Reich DS, Victor JD (1999) In V1 complex cells, phase-sensitive receptive field nonlinearities signal a variety of spatial features and violate a pure energy-operator model. Soc Neurosci Abstr 25:1932.
- Merrill EG, Ainsworth A (1972) Glass-coated platinum-plated tungsten microelectrodes. Med Biol Eng 10:662-672[Web of Science][Medline].
- Milkman N, Shapley R, Schick G (1978) Experimental applications. A microcomputer-based visual stimulator. Behav Res Meth Instrum 10:539-545.
- Milkman N, Schick G, Rossetto M, Ratliff F, Shapley R, Victor JD (1980) A two-dimensional computer-controlled visual stimulator. Behav Res Methods Instrum 12:283-292.
- Morrone MC, Burr DC (1988) Feature detection in human vision: a phase-dependent energy model. Proc R Soc Lond B Biol Sci 235:221-245[Medline].
- Movshon JA, Thompson ID, Tolhurst DJ (1978a) Spatial summation in the receptive fields of simple cells in the cat's striate cortex. J Physiol (Lond) 283:53-77
[Abstract/Free Full Text] .- Movshon JA, Thompson ID, Tolhurst DJ (1978b) Receptive field organization of complex cells in the cat's striate cortex. J Physiol (Lond) 283:79-99
[Abstract/Free Full Text] .- Nachmias J, Weber A (1975) Discrimination of simple and complex gratings. Vision Res 15:217-223[Web of Science][Medline].
- Newsome WT, Britten KH, Movshon JA (1989) Neuronal correlates of a perceptual decision. Nature 341:52-54[Medline].
- Oppenheim AV, Lim JS (1981) The importance of phase in signals. Proc IEEE 69:529-541.
- Parker AJ, Newsome WT (1998) Sense and the single neuron: probing the physiology of perception. Annu Rev Neurosci 21:227-277[Web of Science][Medline].
- Pollen DA, Gaska JP, Jacobson LD (1988) Responses of simple and complex cells to compound sine-wave gratings. Vision Res 28:25-39[Web of Science][Medline].
- Reich DS (2001) Information encoding by individual neurons and groups of neurons in primary visual cortex. In: PhD thesis The Rockefeller University.
- Reich DS, Mechler F, Purpura KP, Victor JD (2000) Interspike intervals, receptive fields, and information encoding in primary visual cortex. J Neurosci 20:1964-1974
[Abstract/Free Full Text] .- Reid RC, Victor JD, Shapley RM (1997) The use of m-sequences in the analysis of visual neurons: linear receptive field properties. Vis Neurosci 14:1015-1027[Web of Science][Medline].
- Rentschler I, Hubner M, Caelli T (1988) On the discrimination of compound Gabor signals and textures. Vision Res 28:279-291[Web of Science][Medline].
- Shapley RM, Tolhurst DJ (1973) Edge detectors in human vision. J Physiol (Lond) 229:165-183
[Abstract/Free Full Text] .- Skottun BC, De Valois RL, Grosof DH, Movshon JA, Albrecht DG, Bonds AB (1991) Classifying simple and complex cells on the basis of response modulation. Vision Res 31:1079-1086[Web of Science][Medline].
- Spitzer H, Hochstein S (1985) Simple- and complex-cell response dependences on stimulation parameters. J Neurophysiol 53:1244-1265
[Abstract/Free Full Text] .- Tolhurst DJ (1972) On the possible existence of edge detector neurones in the human visual system. Vision Res 12:797-804[Web of Science][Medline].
- Tolhurst DJ, Dealy RS (1975) The detection and identification of lines and edges. Vision Res 15:1367-1372[Web of Science][Medline].
- Treves A, Panzeri S (1995) The upward bias in measures of information derived from limited data samples. Neural Comput 7:399-407[Web of Science].
- Victor JD (1992) Nonlinear systems analysis in vision: overview of kernel methods. In: Nonlinear vision: determination of neural receptive fields, function, and networks, Chap 1 (Pinter RB, Nabet B, eds), pp 1-37. Boca Raton, FL: CRC.
- Victor JD, Knight BW (1979) Nonlinear analysis with an arbitrary stimulus ensemble. Quart Appl Math 37:113-136.
- Victor JD, Mast J (1991) A new statistic for steady-state evoked potentials. Electroencephalogr Clin Neurophysiol 78:378-388[Web of Science][Medline].
- Victor JD, Purpura KP (1996) Nature and precision of temporal coding in visual cortex: a metric-space analysis. J Neurophysiol 76:1310-1326
[Abstract/Free Full Text] .- Victor JD, Purpura KP (1997) Metric-space analysis of spike trains: theory, algorithms and application. Network 8:127-164.
- Victor JD, Purpura KP (1998) Spatial phase and the temporal structure of the response to gratings in V1. J Neurophysiol 80:554-571
[Abstract/Free Full Text] .- Vogels R, Orban GA (1990) How well do response changes of striate neurons signal differences in orientation: a study in the discriminating monkey. J Neurosci 10:3543-3558[Abstract].
Copyright © 2002 Society for Neuroscience 0270-6474/02/22146129-29$05.00/0
This article has been cited by other articles:
L. Henriksson, A. Hyvarinen, and S. Vanni
Representation of Cross-Frequency Spatial Phase Relationships in Human Visual Cortex
J. Neurosci., November 11, 2009; 29(45): 14342 - 14351.
[Abstract] [Full Text] [PDF]
N. A. Crowder, J. van Kleef, B. Dreher, and M. R. Ibbotson
Complex Cells Increase Their Phase Sensitivity at Low Contrasts and Following Adaptation
J Neurophysiol, September 1, 2007; 98(3): 1155 - 1166.
[Abstract] [Full Text] [PDF]
L. M. Romanski
Representation and Integration of Auditory and Visual Stimuli in the Primate Ventral Lateral Prefrontal Cortex
Cereb Cortex, September 1, 2007; 17(suppl_1): i61 - i69.
[Abstract] [Full Text] [PDF]
F. Mechler, I. E. Ohiorhenuan, and J. D. Victor
Speed Dependence of Tuning to One-Dimensional Features in V1
J Neurophysiol, March 1, 2007; 97(3): 2423 - 2438.
[Abstract] [Full Text] [PDF]
S.-C. Yen, J. Baker, and C. M. Gray
Heterogeneity in the Responses of Adjacent Neurons to Natural Stimuli in Cat Striate Cortex
J Neurophysiol, February 1, 2007; 97(2): 1326 - 1341.
[Abstract] [Full Text] [PDF]
C. Bardy, J. Y. Huang, C. Wang, T. FitzGibbon, and B. Dreher
'Simplification' of responses of complex cells in cat striate cortex: suppressive surrounds and 'feedback' inactivation
J. Physiol., August 1, 2006; 574(3): 731 - 750.
[Abstract] [Full Text] [PDF]
V. Bonin, V. Mante, and M. Carandini
The statistical computation underlying contrast gain control.
J. Neurosci., June 7, 2006; 26(23): 6346 - 6353.
[Abstract] [Full Text] [PDF]
J. D. Victor, F. Mechler, M. A. Repucci, K. P. Purpura, and T. Sharpee
Responses of V1 Neurons to Two-Dimensional Hermite Functions
J Neurophysiol, January 1, 2006; 95(1): 379 - 400.
[Abstract] [Full Text] [PDF]
B. B. Averbeck and L. M. Romanski
Principal and Independent Components of Macaque Vocalizations: Constructing Stimuli to Probe High-Level Sensory Processing
J Neurophysiol, June 1, 2004; 91(6): 2897 - 2909.
[Abstract] [Full Text] [PDF]
D. Aronov, D. S. Reich, F. Mechler, and J. D. Victor
Neural Coding of Spatial Phase in V1 of the Macaque Monkey
J Neurophysiol, June 1, 2003; 89(6): 3304 - 3327.
[Abstract] [Full Text] [PDF]
This Article Abstract 
Full Text (PDF) Submit an eLetter Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in Web of Science Similar articles in PubMed Alert me to new issues of the journal Download to citation manager 
Citing Articles Citing Articles via HighWire Citing Articles via Web of Science (19) Citing Articles via Google Scholar Google Scholar Articles by Mechler, F. Articles by Victor, J. D. Search for Related Content PubMed PubMed Citation Articles by Mechler, F. Articles by Victor, J. D.
|
|
|
|
 |
|