Abstract
The visual system is thought to represent the direction of moving objects in the relative activity of large populations of cortical neurons that are broadly tuned to the direction of stimulus motion, but how changes in the direction of a moving stimulus are represented in the population response remains poorly understood. Here we take advantage of the orderly mapping of direction selectivity in ferret primary visual cortex (V1) to explore how abrupt changes in the direction of a moving stimulus are encoded in population activity using voltage-sensitive dye imaging. For stimuli moving in a constant direction, the peak of the V1 population response accurately represented the direction of stimulus motion, but following abrupt changes in motion direction, the peak transiently departed from the direction of stimulus motion in a fashion that varied with the direction offset angle and was well predicted from the response to the component directions. We conclude that cortical dynamics and population coding mechanisms combine to place constraints on the accuracy with which abrupt changes in direction of motion can be represented by cortical circuits.
Introduction
Encoding the direction of a moving stimulus that abruptly changes its direction of motion provides a particularly vivid example of the challenges inherent in using cortical circuits to represent rapidly changing stimulus features that are ubiquitous in visual scenes. For a stimulus that moves continuously in one direction, direction-selective cortical neurons would be expected to exhibit a relatively stable pattern of activity, with the peak of the active population accurately signaling the direction of stimulus motion (Maunsell and Van Essen, 1983; Albright, 1984; Shmuel and Grinvald, 1996; Weliky et al., 1996). But what happens if the moving stimulus suddenly changes its direction of motion?
Several features of single-unit behavior are likely to impact the population response to changes in motion direction. First, instantaneous changes in the properties of a visual stimulus are accompanied by alterations in the activity of cortical neurons that begin with some delay and persist beyond the stimulus event for hundreds of milliseconds (Sharon and Grinvald, 2002; Jancke et al., 2004). As a consequence, the population response to a stimulus that abruptly changes its direction of motion must involve a transition period during which the distributed pattern of activity representing the initial direction of motion is replaced by the pattern of activity representing the new direction of motion. Thus, significant departures of the population response from the actual direction of stimulus motion would be expected if one assumes that the neural responses during the transition are simply the sum of the responses to the component stimuli. Second, the initial stimulus condition may impact the responsiveness of neurons to the subsequent stimulus condition, altering response magnitude, and affecting the peak and width of single-unit tuning curves (Müller et al., 1999; Dragoi et al., 2000, 2002; Felsen et al., 2002; Priebe and Lisberger, 2002; Kohn and Movshon, 2004; Kohn, 2007). These nonlinearities, generally regarded as manifestations of cortical adaptation mechanisms, have been proposed as the basis for distortions in visual perception such as the orientation tilt effect and the direction repulsion effect (Dragoi et al., 2000; Kohn and Movshon, 2004). But, it remains unclear how these nonlinear effects combine with each other and with the response dynamics that derive from the offset and onset of the component stimuli alone to ultimately shape the population response (Gardner et al., 2004).
In this study, we use voltage-sensitive dye (VSD) imaging to quantify the dynamic patterns of V1 population activity that accompany abrupt changes in the direction of motion of a random dot stimulus. The presence of an orderly columnar map of direction preference in ferret visual cortex makes it possible to translate the spatial patterns of VSD activity into a measure of the direction tuning of the population response (Weliky et al., 1996). Our results show that the behavior of the peak of the population response varies with the offset angle of the two visual stimuli, and is largely predicted by the linear sum of the response to the component stimuli.
Materials and Methods
Animal preparation.
All experimental procedures were approved by the Duke University Institutional Animal Care and Use Committee and were performed in compliance with guidelines published by the U.S. National Institutes of Health. Ferrets (all females) were anesthetized with ketamine (50 mg/kg), shaved, and scrubbed. The femoral vein was cannulated for delivery of 5% dextrose in lactated Ringer's solution and paralytic, a tracheotomy was performed, and the ferret was placed in a stereotaxic head frame. A mixture of nitrous oxide and oxygen (2:1) with halothane or isoflurane (2–2.5%) was administered and adjusted if necessary based on EKG and expired CO2. Body temperature was maintained at 37°C, and silicone oil was used to protect the corneas. A craniotomy was performed above primary visual cortex and the dura was removed. A stainless steel chamber was cemented (Dycal, Dentsply) to the skull. Intrinsic imaging was performed before voltage-sensitive dye imaging. For intrinsic imaging, the chamber was filled with Ringer's solution and sealed with a glass coverslip to allow viewing of the cortical surface. For voltage-sensitive dye imaging experiments, the chamber was filled with agar (0.7%) and sealed with a glass coverslip to allow viewing of the cortical surface. After completing surgical procedures, incisions and pressure points were infiltrated with bupivacaine, and anesthesia (halothane or isoflurane) was lowered to 0.75–1%. The animal was paralyzed with rocuronium bromide to prevent eye movements. Nitrous oxide and oxygen ratio was reduced to 1:1. This reduction in anesthesia was necessary to allow visual stimulation to activate cortical circuits. Expired CO2 level was maintained at 3.5–4.5% throughout the experiment.
Intrinsic signal optical imaging.
The cortical surface was visualized through a tandem lens macroscope attached to a low-noise CCD camera. The timing of stimulus presentation and collection of reference and stimulus frames were all controlled by software from Optical Imaging. For intrinsic signal imaging, we illuminated with 705 nm light. In general, we used an acquisition time of 5 s following stimulus onset. The data were averaged across 25 stimulus presentations to yield the final activity map for each stimulus condition. Visual stimuli were presented on an Apple Macintosh PowerMac G4 and a GDMF520 CRT (Sony, refresh rate 120 Hz), gamma-corrected using ColorVision Spyder (Pantone). Stimulation procedures were designed using Matlab (MathWorks) and the Psychophysics Toolbox. Visual stimuli were random dot patterns moving in 16 directions between 0° and 337.5° sampled at 22.5° increments. The dot diameter was 1° and contrast was 100% (white dots on a black background). The dot density was 0.137 dots/deg2. The dot pattern moved at 10°/s with 100% coherence. We used full field dot stimuli (74° horizontal × 55° vertical). Stimuli were 5 s in duration with a 5 s interstimulus interval.
Analysis of optical images was performed using custom designed software written in MATLAB (MathWorks). Briefly, a first frame subtraction (Bonhoeffer and Grinvald, 1996) was performed followed by a circularly symmetric spatial filter (passband between 1.0 and 6.7 cycles/mm, implemented as a low-pass mean filter 5 pixels in diameter, followed by a high-pass mean filter 35 pixels in diameter). Because most neurons in ferret primary visual cortex exhibit a reduced but significant response to motion in the nonpreferred direction (Moore et al., 2005), the single-pixel direction tuning curve was fitted to a double Gaussian function: Here M is the response of a pixel as a function of the direction of motion θ, A0 is the untuned response component, A1 and A2 are the strength of the response to the preferred and nonpreferred direction, respectively, ϕ1 is the corresponding preferred and ϕ2 is the corresponding nonpreferred direction, and σ is the tuning width. There were five parameters for 31 degrees of freedom, because ϕ1 and ϕ2 were constrained to be 180° apart from each other. When the fitting procedure led to a solution with A2 > A1, we exchanged A1 and ϕ1 with A2 and ϕ2 so that ϕ1 always corresponded to highest peak in the direction tuning curve, i.e., the neuron's preferred direction. The direction map is obtained by plotting for each pixel its preferred direction. The fitting procedure is described below in the subsection on the fits for the population response profile (PRP).
VSD imaging.
We followed the protocol for VSD imaging described by Grinvald and collaborators (Shoham et al., 1999; Sharon and Grinvald, 2002; Grinvald and Hildesheim, 2004). Briefly, the cortex was stained with the VSD RH-1691 by circulating the dye in a chamber over the cortex for 2 h and washing it out with Ringer's solution. Images were acquired with a CCD digital camera at a frame rate of 340 Hz and a spatial resolution of 30 μm per pixel. Frame acquisition was synchronized with the heartbeat. Respiration was stopped during the acquisition. The cortex was illuminated by a 100 W halogen light. The filter settings were as described previously (Shoham et al., 1999): the excitation filter was bandpass at 630 ± 10 nm, and the emission filter was high-pass, with a cutoff at 665 nm.
In the motion transition experiments, the visual stimuli started with 150 ms gray screen presentation followed by a 300 ms flickering dot pattern, which was designed to activate the cortex uniformly, without preference for any direction domain. Subsequently, two stimulus components each lasting 1 s were presented, with the direction of the second component different by an angle that varied between 22.5° to 180° with 22.5° angular increments. The parameters of the random dot pattern were the same as those in the intrinsic imaging experiments. The average luminance was the same for the two motion components as well as for the flicker stimulus. A difference map was determined to obtain a higher signal-to-noise ratio, which meant that each stimulus condition was paired with another one with 90° angular offset in the direction of both its motion components. To test for linearity, the response was also obtained to two control conditions for which either the first or second motion component was replaced by a flickering dot pattern, respectively. For the visual stimuli designed to test linearity, the duration of the first motion component was reduced to 500 ms to shorten the imaging time.
VSD data analysis.
The measured responses during the experiment are represented as a stack of images R(x, y, t); here, R denotes the response, x and y are the position of the pixel, and t is time. Out of this movie, the responses to a particular stimulus across the Ntrials trials were extracted: Here, i is the trial index (i = 1, … , Ntrials), ti is the start time of trial, and τ is the time relative to trial onset.
Subsequently, the response was averaged across trials. Typically, there are 40–50 trials for each experimental condition in our VSD data. Next, the baseline, calculated as the mean across image frames <150 ms before the onset of the stimulus, was subtracted to remove slow stimulus-independent fluctuations in illumination and background fluorescence levels, such as artifacts due to heartbeat and respiration (Grinvald et al., 1984; Sharon et al., 2007): Here, Nf is the number of frames in the time range −150 to 0 ms and R̃ indicates the response after baseline correction. Response relative to the blank stimulus (“stim”) was obtained by subtracting the blank response (“blank”) and dividing by it: The relative response to the orthogonal stimulus was subtracted to enhance the signal-to-noise ratio and remove the nonselective response component, which occupies the majority of the VSD signal (Sharon and Grinvald, 2002): This method is referred to as the difference imaging method. Please refer to supplemental Figure S2 (available at www.jneurosci.org as supplemental material) for detailed explanations.
The resulting difference images were then processed using a spatial bandpass filter F. The low-pass mean filter had a radius of 5 pixels; i.e., all pixels in the filter satisfying x2 + y2 < 5 are equal to 1/N, and all others are zero, where the normalization N is the number of nonzero pixels in the filter [matlab function fspecial(‘disk’,5)]; the high-pass mean filter of 35 pixels is constructed in exactly the same way, but is subtracted of the identity filter (equal to 1 when x = y = 0). The two filters were convolved to obtain F. Convolutions were performed using the matlab function imfilter. Hence, Here, * stands for convolution and F for filter.
To interpret the patterns of activity evoked by various stimulus configurations, we calculated a population response profile (PRP), which captured the relative activation of each pixel in the region of interest (ROI) in terms of the pixel's preferred direction (Basole et al., 2003). The pixel's preferred direction θx,y is determined by aligning the VSD imaging to the direction preference map acquired by intrinsic imaging using the blood vessel patterns. The pixels inside the ROI were sorted into 36 bins according to the preferred direction (0–10°, 10–20°, etc). The PRP was obtained by calculating the average response for each direction bin. Here, θ represents different direction bins and θx,y stands for the preferred direction of each pixel. 1θx,y is an indicator function. 1θx,y = 1 if θx,y ∈ θ and 1θx,y = 0 if θx,y ∉ θ.
The resulting PRPs were fitted with double Gaussian functions to determine the peak direction ϕ1 and modulation depth A1. Here, PRP is the response, θ is the preferred direction of the neurons, A0 is the remaining untuned response component, A1 and A2 are the responses of pixels for which the stimulus corresponds to the preferred and nonpreferred directions, respectively, ϕ1, ϕ2, ψ1, and ψ2 are the corresponding peak directions, and σ is the tuning width. In total, there were 5 parameters for 31 degrees of freedom, because ϕ1 and ϕ2 were constrained to be 180° apart from each other and ψ1 and ψ2 were constrained to be 180° apart from each other and to be 90° apart from ϕ1 and ϕ2, respectively. When the fitting procedure led to a solution with A2 > A1, we exchanged A1 and ϕ1 with A2 and ϕ2 so that ϕ1 always corresponded to highest peak in the direction tuning curve, i.e., the neuron's preferred direction. For each time frame, the squared difference between the measured response PRP(θ) and the fitting function PRP(θ|A0, A1, A2, ϕ1, ϕ2, ψ1, ψ2, σ) was minimized as a function of the parameters by least-squares curve fitting using Optimization Toolbox in Matlab (lsqcurvefit). To avoid suboptimal solutions corresponding to local minima, the fitting procedure was repeated 800 times and the solution with the lowest difference was retained. The average R2 value of the fits was ∼0.85. We relaxed the constraints of the preferred directions in Equation 9. Using four parameters instead of one parameter to represent the four peaks individually did not significantly improve the adjusted R2 value.
To determine the robustness of the estimated fitting parameters, we first estimated the error in the binned PRP using a bootstrap procedure (Davison and Hinkley, 1997). The original PRP for the example dataset was obtained using 48–50 trials per stimulus condition. We created 10 datasets by randomly drawing, with replacement, 40 trials from the original set, and determined the PRP for each of them. The SD in the PRP across these sets was used as an estimate for the error in the PRP. It did not exceed 20% of the peak response. The effect of this variability on the fitting procedure was estimated by using 400 PRPs generated by adding a random noise to each bin with an SD as much as (20%) or larger than (30%) the estimated PRP variability. The fits to PRPs at steady state (before the motion transition and after the motion transition) were not significantly affected. For instance, at a noise level of 30%, the peak direction and modulation depth averaged across the 400 sample PRPs matched those determined from the fit to the data to within 1%, with SD of 2.7° and 9% of the mean, respectively. The fits remained good during the motion transition for small deviations such as 45°, but the fits were affected by the added noise when the angle of deviation was 90° due to the reduction in the modulation of the VSD signal. For instance, for the latter we obtained, at 200 ms after the change of the stimulus motion, a SD of 22° and 25% of the mean, for the peak direction and modulation depth, respectively.
To confirm the population response dynamics were not affected by the difference imaging strategy, we also did the PRP analysis based on single-condition images, in which the data were high-pass filtered (low-frequency cutoff 1.0 cycles/mm, implemented as a high-pass mean filter 35 pixels in diameter) to remove spatial gradients and isolate the local modulation patches. The PRP computed from single-condition data was fitted by Equation 1. Note that for the PRP analysis, θ corresponds to the preferred direction of the pixel, whereas in the original application of Equation 1, it was the stimulus direction.
The dynamics of peak direction and modulation depth were normalized to allow comparison between different direction deviation angles. For each time series, (1) the mean peak direction 0–100 ms before the change of stimulus was normalized to 0; (2) the mean peak direction 900–1000 ms after the change of stimulus was normalized to 1; and (3) the mean modulation depth 0–100 ms before the change of stimulus was normalized to 1. The normalized dynamics were then aligned to the midpoint of change in peak direction and averaged across animals.
Electrophysiology.
Single-unit activity was recorded extracellularly from V1 with MPI microelectrodes (impedance = 1.0 MΩ, Microprobe). Action potentials were recorded from the supragranular layers (<600 μm) and discriminated using Spike2 software (Cambridge Electronic Design). Spikes were sorted based on PCA sorting method. The visual stimulus was the same as the one used in VSD experiments. However, the direction deviation angle between the two components was held constant, and the direction of the first stimulus was varied across 16 angles with 22.5° angular increments. We displayed the activation relative to the spontaneous activity, which was obtained by subtracting the response to a blank stimulus. The reconstructed PRP was then fitted by Equation 1 to identify the peak direction and modulation depth. To be consistent with the VSD experiments, only the modulated component is shown (the nonselective component has been removed).
Linear combination model.
The dynamics of cortical activation was studied by determining the PRP as a function of time Mexp(θ, t). To evaluate to what extent the cortical response to a motion transition could be understood in terms of a linear combination of responses to component stimuli, it was compared to the following linear combination model: Here, Mlin is the model response to the motion transition, θ is the preferred direction of the neurons, and t is time. N1 and N2 are the steady-state PRPs corresponding to the direction of the first and second components, respectively. α and β are the weighting factors for the two motion components. The functions N1 and N2 were given by the functional form of Equation 1, which was interpreted as the population activity for a given stimulus direction. The parameter values (A0 = 0.039, A1 = 0.698, A2 = 0.441, and σ = 38.3) were obtained from a fit to a typical VSD dataset (from Ferret 612) and ϕ1 was set to the direction of the first and second components for N1 and N2, respectively. For the first 50 ms after the motion transition, α(t) = 1 and β(t) = 0, whereas during the 500-ms-long transition period, α(t) linearly decayed to zero and β(t) increased to one. After reaching these values, α(t) and β(t) remained constant. To determine the population dynamics, we fitted, for each t value, Mlin(θ, t) to Equation 1. This yielded the peak direction (ϕ1) and modulation depth (A1) as a function of time.
Results
The dynamics of the population response to motion stimuli in primary visual cortex as visualized with VSD imaging
Voltage-sensitive dye imaging of ferret visual cortex revealed strong and stable columnar activity patterns in response to random dot patterns that moved in a constant direction. Figure 1a illustrates the pattern of VSD response induced by presentation of a random dot stimulus moving in a constant direction (135°). We used high-pass filters to remove the nonselective component of the VSD response and isolate the columnar modulation (Sharon et al., 2007). Columnar signal emerged ∼100 ms after stimulus onset and gradually strengthened, without obvious changes in the overall pattern of modulation. To understand the relation between these columnar response patterns and the tuning of the population response to motion direction, the stimulus evoked VSD maps were convolved with the direction preference map for the same region of cortex (Basole et al., 2003). The resulting PRP represents the average activity evoked by a given stimulus across all cortical sites expressed as a function of each site's preferred stimulus direction (Fig. 1b). These PRPs exhibit a major peak and minor peak offset by 180°, consistent with the fact that tuning for stimulus direction is not absolute: most neurons respond preferentially to one direction of motion and significantly, but less strongly, to motion in the anti-preferred direction (Finlay et al., 1976; Berman et al., 1987; Moore et al., 2005). To visualize how the population response evolves over time, we aligned the PRPs derived from the VSD images acquired at different times before and following stimulus presentation (Fig. 1c). These plots illustrate the emergence of both the major and minor peaks and the rapid increase in amplitude of the tuned component of the signal, which plateaus at ∼400 ms after stimulus onset. Following stimulus onset, the peak of the population response rapidly stabilizes and remains constant throughout the duration of stimulus presentation.
The impact of abrupt changes in direction of motion on the stable patterns of ongoing population activity is illustrated in Figure 2a. In these experiments, the final direction of stimulus motion for each condition was identical—horizontal motion to the left. The onset of this horizontal stimulus occurred simultaneously with the offset of a preceding stimulus that differed in its motion direction by 45°, 90°, and 135° from leftward motion. As expected, at the start of the trial interval, the columnar VSD response patterns for these stimulus conditions were significantly different; but by the end of the trial interval, the patterns were quite similar (Fig. 2a). For a direction deviation angle of 45°, the cortical activity shifted smoothly from the initial pattern to the final pattern without an obvious change in modulation strength (supplemental Movie 1, available at www.jneurosci.org as supplemental material). For a direction deviation of 90°, however, the onset of the second stimulus was accompanied by a prominent reduction in the modulation of the VSD signal, which was then followed by the emergence of a new modulation pattern (supplemental Movie 2, available at www.jneurosci.org as supplemental material). It is worth emphasizing that the decline in the VSD signal reflects a reduction in the tuned component of the response (modulation depth, defined as A1 in Eq. 9) rather than a reduction in absolute response. The cortex is never silent during the transition; rather, it is responding strongly and nonselectively during this phase of the transition (supplemental Fig. S1, available at www.jneurosci.org as supplemental material). For a direction deviation angle of 135°, there was also a noticeable reduction in the modulation of the activation pattern during the transition period (supplemental Movie 3, available at www.jneurosci.org as supplemental material).
Comparison of the PRP tuning dynamics for the three direction deviation angles illustrates the prominent features of the population response that differed as a function of direction deviation angle. For small direction deviation angles (45°), the peak of the population response swept smoothly and gradually from the initial direction (135°) to the final direction (180°) taking the shortest route across the intermediate direction space between the major peaks associated with the first and second stimulus (Fig. 2b). For somewhat larger direction deviation angles (90°), the peak of the population response also followed the shortest route across the intermediate direction space (i.e., from 90° to 180°). The peak direction of the population response starts to shift away from the initial direction later and arrives at the final direction earlier than that found for the 45° case (as shown in Fig. 3a,d). As a result, the rate of change for larger direction deviation angles was larger, exhibiting a sharp jump in the peak of the population response that coincided with a reduction in the modulation of the VSD signal. For larger direction deviation angles (135°), the population response exhibited a distinctive triphasic pattern: an initial phase in which the peak direction turned toward the minor peak of the response to the second stimulus direction, a second phase in which the peak rapidly jumped by 180° in concert with a reduction in magnitude, and a third phase in which the peak direction gradually settled on the direction of the second stimulus. The combined changes in the peak direction and the magnitude of the modulation depth for these three direction deviation angles are summarized in vector format in Figure 2c (supplemental Movies 4–6, available at www.jneurosci.org as supplemental material).
To confirm the systematic relation between the dynamics of the peak population response and direction deviation angle, we quantified the average population response for the full range of direction deviation angles in experiments from 10 animals. The properties of the population dynamics varied continuously as a function of direction deviation angle with an inflection point that is evident at direction deviation angles of 112.5°. For direction deviation angles of 22.5–90°, the peak direction during the transition swept across the intermediate direction space between the initial and final direction (Fig. 3a). Within this group, the slope of the change in peak direction (in degrees per millisecond) became larger as the direction deviation angle approached 90° (Fig. 3d). For direction deviation angles ≥112.5°, the peak direction during the transition exhibited the “triphasic” pattern (Fig. 3a). Within this group, the slope of the change in peak direction became smaller as the direction deviation angle approached 180° (Fig. 3d). (Note that to calculate the slope of the peak direction for direction deviation angles >90°, it was necessary to exclude the instantaneous transient; the values reflect the average slope of the peak direction before and after the transient.) The average magnitude of the modulation depth also varied systematically with direction deviation angle. For most direction deviation angles, there was a transient reduction in modulation depth that reached its maximum at the midpoint of the change in peak direction (Fig. 3b,c). This reduction in modulation depth became greater with increasing direction deviation angle, reaching its maximum at 112.5° and then becoming smaller with larger direction deviation angles (Fig. 3e). For those angles where there was a prominent reduction in modulation depth, there was an equally prominent recovery that could exceed the modulation depth for the initial direction of motion (Fig. 3b,c,f). The modulation depth at the end of the data acquisition period was lowest for the smallest angles in our sample (22.5°, 45°) and reached its highest value for the 112.5° direction deviation angle. For a direction deviation angle of 180° (i.e., a stimulus that reverses direction), the dynamics of the population response became a simple step function with a 180° jump during the middle of the transition process and little change in modulation depth (Fig. 3a,e). We confirmed that the dynamics of the population response were not affected by the difference imaging strategy (supplemental Fig. S2, available at www.jneurosci.org as supplemental material) that we used to enhance the VSD signal-to-noise ratio: analysis of the population response based on single-condition images yielded similar results (supplemental Fig. S3, available at www.jneurosci.org as supplemental material).
The spike discharge population response constructed from unit recording data is consistent with VSD measurements
Voltage-sensitive dye imaging is a powerful technique for measuring the responses of large populations of neurons with fine temporal resolution. However, VSD responses are dominated by subthreshold voltage signals, raising the possibility that the dynamics associated with changes in motion direction that we have described may not be evident in the spiking activity of V1 neurons. To address this issue, we used extracellular recordings from layer 2/3 neurons to reconstruct the dynamics of spiking activity that would be expected for the population response (Fig. 3). We assume that the direction tuning function of a single neuron (i.e., the relative response of a single neuron to different directions of motion) can be used to infer the relative activity of a large population of neurons with similar tuning functions, but with different preferred directions of motion (Treue et al., 2000). Assuming that the direction tuning curve of a single neuron is representative of a large population of single neurons, the spike discharge PRP for a stimulus moving in a single direction will be identical to the direction tuning curve of single neurons whose preferred direction of motion corresponds to the direction of stimulus motion (Fig. 4a).
By similar reasoning, the spike discharge PRP for a stimulus that changes its direction of motion can be extrapolated from a single neuron's response using a series of motion transition stimuli (1) that cover the full range of motion directions and (2) where each stimulus is composed of two successive motion stimuli with the same direction offset angle (see Materials and Methods). Figure 4b illustrates the spike discharge PRP estimated from a single unit responding to motion transition stimuli with a direction deviation angle of 90°. The pattern of activity estimated for the population response exhibits all the basic features that would be expected based on the VSD imaging (e.g., Fig. 2b), notably strongly tuned responses to the first and second stimuli separated by a rapid transition in which there is a strong reduction in the tuned response. The PRPs generated with VSD imaging reflect the average activity of large numbers of neurons that have somewhat different direction tuning functions [for an analysis of the variation of single-neuron direction tuning in ferret V1, see Li et al. (2006), their supplemental Table 1]. To better approximate the neuronal population response, we examined the response of multiple single units (N = 18) to three direction deviation angles (45°, 90°, and 135°) and then averaged the PRPs after proper alignment based on each neuron's preferred direction (Fig. 4c). The dynamics of peak direction and modulation depth of the average spike discharge PRP varied consistently with direction deviation angle, matching the characteristic features computed from the VSD data (Fig. 4d,e). For example, the peak of the population response accompanying a 45° deviation advanced smoothly and slowly through the intervening directions. In contrast, the peak of the population response associated with a 135° deviation exhibited the triphasic pattern found with VSD imaging (movement away from the intervening directions, a rapid 180° jump, and smooth decay to the second direction of motion). Differences in the magnitude of the modulation depth were also consistent with the results of VSD imaging; i.e., small reductions in magnitude for small angular offsets and large reductions for larger angular offsets.
Thus the dynamics of peak direction and modulation depth computed for spike data are consistent with those computed from the VSD data (Fig. 4d,e), demonstrating that the population response patterns are preserved at the spike level.
Modeling the population response to rapid changes in motion direction as the linear sum of the response to the component stimuli
At least some of the complex population response dynamics to motion change could be explained by the linear sum of the response to the component directions of motion, i.e., the gradual reduction in response to the first direction of motion combined with the gradual increase in response to the second direction of motion. Smooth shifts in the peak of the population response with little decline in modulation magnitude might be expected when the populations of neurons most activated by the two individual stimuli (“the major peaks”) are partially overlapping. Significant reductions in modulation magnitude accompanied by abrupt jumps in the population peak would be expected when the major peaks for the two stimuli are largely complementary (∼90° offset), and the triphasic transition pattern might result when the angular offsets are so great that the major peak associated with the first stimulus partially overlaps the minor peak of the second stimulus.
To evaluate whether the behavior of the population response could be explained in this way, we developed a model in which the cortical response to motion transition was assumed to result from a weighted sum of the responses to the components (Fig. 5a). The PRPs of two motion components were obtained from a fit to the VSD data shown in Figure 1 (Fig. 5a), while the dynamics of the weighting factors were assumed to be simple linear processes (Fig. 5b). The dynamics of the population response to motion transitions with different direction deviation angles predicted by the model are shown in Figure 5c–f (also see supplemental Movies 7, 8, available at www.jneurosci.org as supplemental material). We found that the linear combination model was sufficient to account qualitatively for many of the features associated with the dynamics of the population response during the transition. For example, the linear model predicts the smooth progression through intermediate directions for small direction deviations and the triphasic pattern for larger direction deviation angles. It also accounts for the systematic relation between direction deviation angle and the transient reduction in modulation during the transition. Note, however, that the linear summation model failed to account for the changes in modulation magnitude ∼200 ms after the motion transition. The modulation depth at the final steady state also depends on the size of the direction deviation angles (Fig. 3f). These aspects of the population response are likely to reflect longer-lasting adaptation mechanisms that have been the subject of numerous single-unit studies (Vautin and Berkley, 1977; Carandini et al., 1998; Dragoi et al., 2002; Priebe and Lisberger, 2002).
Additional nonlinear mechanisms shape the dynamics of the population response
Although this simple linear combination model accounts for many of the features associated with the dynamics of the population response during the transition, the model is based on assumptions about the dynamics of the underlying component responses that are untested. A stronger test of linearity is to directly compare the population response to the change in motion direction with that expected from the linear sum of the responses to the component directions of motion. For technical reasons (dye fading), we were unable to acquire the data for both component and motion transition stimuli using the experimental time course that was used for the data in Figures 2 and 3. However, by reducing the duration of the first stimulus from 1 s to 500 ms, we were able to acquire enough trials to directly compare the response to the component stimuli with the response to the motion transition stimulus. The results of these experiments (n = 3 animals) are shown in Figure 6.
The population response to the motion transition was similar to that found with the previous paradigm (compare Figs. 2, 3 and Fig. 6). The peak direction exhibited smooth transitions for small angular offsets and the triphasic pattern for large angular offsets. Small differences in the magnitude of the modulation depth between this set of experiments and those in Figure 3 for the 45° offset condition are likely due to differences in the stimulus timing parameters noted above.
Consistent with the model depicted in Figure 5, the dynamics of the peak direction of the population response were well matched with the linear sum of the component responses. However, significant departures from linearity were evident in the dynamics of modulation depth for direction deviation angles of 45° and 90°: the linear sum of the component responses predicted modulation depths that were significantly lower than those found with the motion transition stimuli (Fig. 6a,b). As shown in the vector plots (Fig. 6c), for 45° and 90° direction deviation angles, the transition between the initial state and final state for population response to the motion transition stimuli was smoother than predicted by the linear sum of the component responses. A linear regression was used to compare the PRP of the motion transition response with the PRP derived from the linear sum of the component responses. We found that linear summation of the component responses accurately predicted the modulation depth in the experimental conditions before the transition (slope a = 0.98 ± 0.01, 0.99 ± 0.01, and 1.01 ± 0.01 for 45°, 90°, and 135°, respectively). Linear summation also predicted the modulation depth during the motion transition (100 to 300 ms) for the 135° direction deviation angle, but it failed to do so for the 45° and 90° conditions (slope a = 1.97 ± 0.05, 1.26 ± 0.03, and 0.99 ± 0.02 for 45°, 90°, and 135° respectively). Thus, the direct test of linearity confirms that the peak direction of the population response to motion transitions is well predicted by linear sum of the component responses, but it also reveals significant nonlinearities that shape the modulation depth of the population response.
Discussion
These results demonstrate that the peak of the V1 population response accurately encodes the direction of a stimulus that moves in a constant direction, but fails to do so when the stimulus abruptly changes motion direction. A departure of the peak from the direction of stimulus motion under these conditions is not unexpected; but the actual behavior of the population peak that we observed—a pattern that is well predicted by the linear sum of the responses to the component directions—is surprising in light of studies that have explored the responses of single units to changes in stimulus orientation and motion direction. The inference from most single-unit studies is that abrupt changes in motion direction would yield shifts in the peak direction of the population response that are not predictable from the linear sum of the component responses (Müller et al., 1999; Dragoi et al., 2000, 2002; Felsen et al., 2002; Kohn and Movshon, 2004; Kohn, 2007). Although we cannot rule out small nonlinear shifts in the peak of the population response that fall within the noise level of the VSD measurements, our results suggest that the behavior of the peak of the population response is dominated by the linear sum of the responses to the component stimuli. The linear behavior of the peak of the population response demonstrated here is consistent with a recent VSD imaging analysis of the cortical response to rapid shifts in grating orientation (Benucci et al., 2009).
How can the linear behavior of the peak direction of the VSD population response be reconciled with the nonlinear behavior reported for the responses of single units? Many of the single-unit studies of adaptation employ paradigms in which the initial “adapting” stimulus is presented for a much longer period of time than what was used here; nonlinear effects on the tuning curves of single units and on the peak of the population response might be more evident under these conditions (Dragoi et al., 2000). But the nonlinear behavior of single units is also evident in experiments where the adapting stimulus is presented for short periods of time, comparable to those in the present experiments (Müller et al., 1999; Dragoi et al., 2002), indicating that this cannot be the sole explanation.
Another contributing factor may be the approach used to assess the impact of stimulus change on single-unit responses, and how the data derived from these experiments have been used to predict the behavior of the population response. Most efforts to probe adaptation effects have examined the impact of stimulus change on single-unit tuning curves, focusing on whether the effects of an adapting stimulus lead to shifts in the peak of the tuning curve (Müller et al., 1999; Dragoi et al., 2000, 2002; Felsen et al., 2002; Kohn and Movshon, 2004; Kohn, 2007). However, the effects of a given adapting stimulus on single-unit tuning curves are heterogeneous, involving attractive or repulsive shifts in the peak of the tuning curve as well as variations in response magnitude (Dragoi et al., 2000, 2002; Kohn and Movshon, 2004; Kohn, 2007; Ghisovan et al., 2008). How these different effects combine to shape the peak of the population response is difficult to assess. Moreover, the combined impact of the single-unit tuning curve shifts on the population response may be small compared to the impact of the response dynamics that derive from the offset and onset of the component stimuli alone; these dynamics are not typically captured in the analysis of single-unit tuning curves. In support of this possibility, we note that our analysis of single-unit responses to stimulus change—which did not rely on measurements of single-unit tuning curves—matched the behavior of the population response assessed by VSD imaging (Fig. 4). This result also makes it unlikely that the results derived from VSD imaging of the population response fail to reveal nonlinearities that may be present in spike discharge responses.
Assuming the peak of the population response represents the perceived direction of stimulus motion, then the deviation of the peak from the actual direction of stimulus motion predicts distortions in perception that accompany abrupt changes in motion direction. For relatively small direction deviation angles such as 45°, the peak of the population response moves smoothly from the initial direction, through the full range of intermediate directions (that are not present in either stimulus), before arriving at the direction of the second stimulus. If the intermediate directions signified by the peak of the population response contribute to perception, this would be expected to induce a smoothing of perceived motion, such that a bilinear motion stimulus would appear to follow the trajectory of an arc. Consistent with this interpretation, observers report that a moving dot stimulus that undergoes an abrupt bilinear change in motion direction appears to follow a smooth trajectory provided the direction deviation angle is relatively small (<70°) (Tripathy and Barrett, 2003, 2006). The data presented here suggest that smoothing should not be evident for larger direction deviation angles (>90°), and we have confirmed this prediction for a moving dot stimulus (our unpublished data).
Because the primary factors that account for these population effects—the noninstantaneous nature of circuit dynamics and the broad tuning of individual neurons—are fundamental properties of population coding mechanisms, similar distortions involving the perception of intermediate values not present in the stimulus should be evident for instantaneous changes of other stimulus features that rely on population codes. In this regard, a population code for position could explain the line motion illusion (Wertheimer, 1912; Kenkel, 1913), where the presentation of two separate stimuli gives rise to the perception of a single stimulus growing in length. Likewise, a population code for orientation could explain the illusory perception of the rotation of a single line that is induced by the rapid presentation of two lines that differ in orientation (Shepard and Judd, 1976; Robins and Shepard, 1977). Population coding mechanisms for position on the body surface could also account for the cutaneous rabbit illusion, the illusory perception of an intermediate stimulus that is induced by the rapid presentation of two discrete somatosensory stimuli (Geldard and Sherrick, 1972).
While the peak direction of the population response was well predicted by the sum of the responses to the component stimuli, the magnitude of the response modulation following a change in motion direction exhibited significant departures from linearity. Deviations from linearity in modulation depth were especially evident in the population response to the second direction of motion; for small direction deviation angles, the modulation depth for the second direction of motion was significantly smaller than that for the initial direction, while for larger direction deviation angles, it was significantly greater (Fig. 3f). Similar nonlinear effects on response magnitude have been described for single units following changes in stimulus orientation (Vautin and Berkley, 1977; Carandini et al., 1998; Dragoi et al., 2002). In fact, the angular dependence of the effects on modulation depth described here is consistent with that observed for single-unit responses to changes in orientation: for small angular offsets, changes in stimulus orientation are accompanied by a suppression of response to the unit's preferred orientation, and this is replaced by facilitation at 90° offsets (Dragoi et al., 2002). Nonlinearities were also evident in modulation depth during the motion transition: for small angular offsets, the magnitude of modulation was significantly greater than that predicted from the component responses (Fig. 6b). The contribution of these nonlinearities in modulation depth to perception of motion direction and the cellular mechanisms that underlie them remain unclear.
We conclude that cortical dynamics interact with population coding mechanisms to place constraints on the accuracy with which abrupt changes in direction of motion can be represented by cortical circuits. The perceptual distortions that we have suggested arise from this interaction are likely to represent an acceptable tradeoff that balances the need for accuracy with the innumerable advantages that are afforded by distributed coding mechanisms (Rieke, 1997; Pouget et al., 2000; Dayan and Abbott, 2005).
Footnotes
This work was supported by National Eye Institute Grant EY11488 to D.F. and American Heart Association Pre-doctoral Fellowship 0615341U to W.W. We thank Julie Heiner for technical assistance and Wyeth Bair, Haluk Ogmen, Srimant Tripathy, Ozgur Yilmaz, and the members of the Fitzpatrick laboratory for comments on the manuscript.
- Correspondence should be addressed to David Fitzpatrick, Max Planck Florida Institute, 5353 Parkside Drive, MC19-RE, Jupiter, FL 33468. david.fitzpatrick{at}mpfi.org