Abstract
Motion perception is a vital part of our sensory repertoire in that it contributes to navigation, awareness of moving objects, and communication. Motion sense in carnivores and primates originates with primary visual cortical neurons selective for motion direction. More than 60 years after the discovery of these neurons, there is still no consensus on the mechanism underlying direction selectivity. This paper describes a model of the cat's visual system in which direction selectivity results from the well-documented orientation selectivity of inhibitory neurons: inhomogeneities in the orientation preference map for inhibitory neurons leads to spatially asymmetric inhibition, and thus to direction selectivity. Stimulation of the model with a drifting grating shows that direction selectivity results from the relative timing of excitatory and inhibitory inputs to a neuron. Using a stationary contrast-reversing grating reveals that the inhibitory input is spatially displaced in the preferred direction relative to the excitatory input, and that this asymmetry leads to the timing difference. More generally, the model yields physiologically realistic estimates of the direction selectivity index, and it reproduces the critical finding with contrast-reversing gratings that response phase advances with grating spatial phase. It is concluded that a model based on intracortical inhibition can account well for the known properties of direction selectivity in carnivores and primates.
SIGNIFICANCE STATEMENT Motion perception is vital for navigation, communication, and the awareness of moving objects. Motion sense depends on cortical neurons that are selective for motion direction, and this paper describes a model for the physiological mechanism underlying cortical direction selectivity. The essence of the model is that intracortical inhibition of a direction-selective cell is spatially inhomogeneous and therefore depends on whether a stimulus generates inhibition before or after reaching the cell's receptive field: the response is weaker in the former than in the latter case. If the model is correct, it will contribute to the understanding of motion processing in carnivores and primates.
Introduction
The sense of motion is important for survival in that it has evolved independently in many species, from flies (Mauss et al., 2017) to humans (Nishida et al., 2018). Motion perception, correspondingly, is an essential component of our sensory repertoire. We use motion percepts to navigate the environment, to sense object motion, and to communicate with others. This percept, which results from signals passing through the dorsal pathway of visual cortex (Mishkin et al., 1983), originates in motion-sensitive neurons in primary visual cortex (Movshon and Newsome, 1996).
Motion-sensitive neurons have a property, direction selectivity, which means that a cell responds more to stimulus motion in one direction, say upwards, than to motion in other directions. When tested with near-optimal spatial frequencies, direction selectivity in the lateral geniculate nucleus is weak for both cats (Thompson et al., 1994) and primates (Xu et al., 2002). Instead, the direction selectivity found in carnivore and primate cortex originates in primary visual cortex. Such selectivity was first described >60 years ago (Hubel and Wiesel, 1959), but there is still no consensus on the underlying mechanism.
There are two common threads in cortical models of direction selectivity: (1) the visual stimulus is sampled across visual field locations; and (2) there are timing asymmetries between the signals at differing locations (Adelson and Bergen, 1985; Reid et al., 1987; McLean and Palmer, 1989; Livingstone, 1998). The main difference between models is in the source of the asymmetries. One influential set of models assumes that there are parallel subcortical pathways to cortex and that signals in one pathway are delayed relative to other signals. There are lagged and nonlagged cells in the lateral geniculate nucleus, for example, and it has been proposed that the relative delay of lagged cells could provide the basis for direction selectivity (Saul and Humphrey, 1990). This delay, however, is long (at least 40 ms) relative to the duration of cortical responses to flashed stimuli, and it is unclear whether this proposal can lead to a workable model (Peterson et al., 2004).
A more promising source for direction selectivity is intracortical inhibition. Priebe and Ferster (2005) recorded intracellular potentials in cat primary visual cortex while cells were stimulated with moving gratings. They found that excitatory and inhibitory conductances varied roughly in antiphase but that there were subtle shifts in their relative phases as the grating changed from the preferred to the opposite direction. They therefore concluded that direction selectivity could be because of the relative timing of excitatory and inhibitory inputs to the recorded cells. More recently, Wilson et al. (2018) optogenetically inactivated GABAergic neurons in ferret primary visual cortex and found that this inactivation reduces direction selectivity.
While these empirical studies implicate inhibition in direction selectivity, they do not tell us the source of the timing asymmetries necessary for such selectivity. A clue comes from Cardin et al. (2007) who showed that putative inhibitory neurons in cat primary visual cortex are orientation-selective. All excitatory cells are embedded in an orientation preference map such that excitatory cells around them have a variety of orientation preferences. We can expect the inhibitory cells surrounding an excitatory cell to have a similar variety of preferences. Perhaps the inhibitory input from columns on one side of an excitatory cell differs from that of columns on the opposite side because of differing orientation preferences. A very recent study of ferret primary visual cortex supports this proposal: Scholl et al. (2019) found that inhibitory synaptic inputs have preferred orientations that can differ significantly from that of the column in which the synapses reside.
I tested these ideas with a computational model (Nguyen and Freeman, 2019) and two hypotheses: direction selectivity is because of intracortical inhibition; and the asymmetries of timing responsible for direction selectivity originate from inhomogeneities in the orientation preference map. The cortex modeled was the cat's because there is such a rich literature available on visual signal processing in this species. Parts of this work have been published in abstract form (Freeman and Zheng, 2019).
Materials and Methods
The model used here is the same as in a previous publication (Nguyen and Freeman, 2019), with the exception of three model parameters, which were given more realistic values as described below. The model was made analytically more tractable by restricting it: stimuli were monochromatic; the only retinal ganglion cells considered were those with the highest spatial resolution, X cells; cortical input was monocular. Signal flow in the model is illustrated in Figure 1, and the main ingredients of the model are now summarized.
Equations
The defining equations for the model consist of one differential equation for the generator potential in each neuron type as follows: (1) (2) (3) (4) (5) (6) (7) where t is time, is location in the visual field, is the visual stimulus, j is subcortical channel number, and k is cortical cell number. The generator potential, p, is the difference between membrane potential and its resting value for photoreceptors and bipolar cells, and the difference between membrane potential and action potential threshold for the remaining cells; the second subscript indicates processing stage. The functions are as follows: (8) (9) (10)
The function , where s stands for subcortex, includes optical blur and neural convergence in the retina. The other convergence functions and are calculated by subscript substitution into Equation 8 and represent the convergence of, respectively, subcortical signals onto cortical cells (c indicates cortex) and inhibitory signals onto excitatory cells. Potential is converted to impulse rate as follows: (11)
Equations 1–11 together define the model.
Parameters
Model parameters are defined in Table 1. They take the same values as before (Nguyen and Freeman, 2019), except for three time constants: the difference between the on- and off-center time constants is reduced to 1 ms to better match the lower response latency in off-dominated cortical neurons (Komban et al., 2014), and the inhibitory axon time constant is reduced to 100 ms to be more consistent with the timing of rebound responses (DeAngelis et al., 1993) and the delay of inhibitory conductance relative to excitatory conductance during cyclic stimulation (Priebe and Ferster, 2005). The derivation of one other parameter is discussed here because it determines the amplitude of the generator potential shown throughout the paper. Carandini and Ferster (2000) measured membrane potential in simple cells stimulated with gratings of optimal orientation and spatial frequency; the mean contrast sensitivity for the cells in their Figure 13 was 70 mV/contrast-unit. The geniculocortical gain, , was set so that the model's contrast sensitivity approximated this value.
Neuronal location
A neuron's location was defined as the center of its convergence function ( for subcortical and cortical cells, respectively). Off-center subcortical channels were placed on a square grid centered on the middle of the visual field. On-center channels were placed on a second square grid offset from the first so that the four innermost channels were equidistant from the innermost off-center channel. Locations of all channels were then perturbed in both the horizontal and vertical directions by a Gaussian deviate. The spacing of nearest-neighbor on- and off-channels was and , respectively, where the perturbation is an SD. These choices produce arrays with statistics matching well with retinal measurements (Wässle et al., 1981) as shown previously (Nguyen and Freeman, 2019). Cortical cells were located on a square grid with 0.1° spacing; one excitatory and one inhibitory cell were placed at each grid node. The simulated area of visual field had a width and height of 8°, and its middle was located 11° laterally from the central area (for reasons described by Nguyen and Freeman, 2019). Only the central of the simulated area is displayed, to avoid edge effects.
Development
Synaptic weights between geniculate and cortical cells developed with a Hebbian process during visual system maturation. All weights were initially set to 1. On each development cycle, a random geniculate cell was selected, with all cells equally likely to be chosen, and the weight of this cell's synapse with all cortical cells was increased by 0.2. The model was then stimulated with drifting gratings whose directions took 16 values evenly distributed across the whole range. If the maximum response of an excitatory cortical cell exceeded its value on the previous cycle, the weight change was retained by both excitatory and inhibitory cells at that location. Otherwise, the weight was reduced by 0.2 relative to its value before the cycle. All weights were limited to lie between 0 and 2. Inhibitory connections have also been shown to change during visual development (Huang, 2009). The synaptic weight of the synapse between each inhibitory and excitatory cell therefore increased linearly with cycle number from 1 to . The number of development cycles was set so that each geniculocortical synapse could reach its minimum or maximum. It took five cycles for a weight to reach this point, so the number of cycles was set at 5 times the number of subcortical channels (3281) rounded to the nearest thousand (16,000).
Stimuli
The model is image-computable in that it can calculate neuronal responses for any monochromatic, monocular visual stimulus. Three types of stimulus were used as follows:
Drifting grating: (12)
Contrast-reversing grating: (13) where is the distance in the grating's direction of motion, θ, and
Sparse noise: (14)
The sparse noise stimulus used contrasts of for light and dark, respectively, and a duration of 0.2 s. Stimulus squares were 1° on a side and presented at 0.25° intervals in both spatial dimensions across the visual field.
Analysis
Direction selectivity index (DSI)
Direction tuning curves, such as those in Figure 3A, have two peaks. The directions corresponding to these peaks are here termed preferred and suboptimal, where the latter refers to the lesser peak. The difference between these two directions typically differed from 180°. The fitted curve summed two von Mises functions as follows: (15) where θ is stimulus direction, is resting impulse rate, are impulse rate amplitudes in the preferred and suboptimal directions, , respectively, and is tuning bandwidth (half-width at half-height). The index is then: (16) where (17) is the impulse rate in the antipreferred direction, that is, the direction opposite to preferred.
Spatial phase
For the polar plots in Figure 11, the summed excitatory and summed inhibitory inputs to an excitatory cell have an essentially constant response phase to a contrast-reversing grating (because they are direction-nonselective). This is illustrated in Figure 11B (left) for one excitatory cell. To estimate the amplitudes and phases in this graph, I used the fundamental Fourier components for the excitatory input, , where ϕ is spatial phase. The input is shown bold because it is complex. This component varies sinusoidally with spatial phase, and it can therefore be simulated as a sum of cosine and sine parts as follows: (18)
This equation was solved for and by least-squares regression. The full-strength summed excitatory input is then given by the following: (19) (a mean is used in the second equation because the two addends are nearly equal). The summed inhibitory input, , is calculated in the same way. The ellipse in Figure 11A (right) was calculated by subtracting the inhibitory from the excitatory input and applying the excitatory cell's first-order delay as follows: (20) where .
Sparse noise
The excitatory response plotted in the receptive field (see Fig. 12B) is the maximum potential over the period of presentation (0.2 s). For inhibitory responses, the maximum in the first 0.1 s was used, to avoid the inhibitory rebound.
Alternative cortical circuitry
A connection from excitatory to inhibitory cortical neurons was added to the model, as described at the end of Results, by changing Equation 5 to the following: (21) where , replicating the convergence function from inhibitory to excitatory cells.
Code accessibility
Computer code and data for running the model are available at https://github.com/AlanFreeman/Vis.
Results
The model
The aim of this paper is to describe a model for the neural mechanisms underlying direction selectivity in visual cortex. I tackled this issue with the signal-processing model illustrated in Figure 1A. This model contains multiple subcortical channels, each of which comprises either on-center or off-center neurons. Both signs of channel converge on excitatory and inhibitory cells in layer 4 of primary visual cortex. The direct inhibitory synapses are consistent with evidence of monosynaptic thalamocortical connections onto putative inhibitory cells (Swadlow et al., 1998; Zhuang et al., 2013; Sedigh-Sarvestani et al., 2017). Inhibitory cells have two compartments. The first represents the dendrites and soma and has the fast dynamics seen in fast-spiking cells (McCormick et al., 1985; Azouz et al., 1997). The second compartment, representing the axon, has a long time constant providing the delayed inhibition seen in cortical cells presented with flashed stimuli (DeAngelis et al., 1993) and the delayed inhibitory conductance observed during stimulation with gratings (Priebe and Ferster, 2005). The model assumes that these inhibitory cells then synapse onto excitatory cells in the same layer.
Each small box in the circuit represents a neuron type, and each neuron was represented by a nonlinear first-order differential equation; all equations were solved simultaneously to obtain response time courses. Figure 1B illustrates signal processing in a typical neuron: synaptic inputs were weighted and summed, the sum was integrated over time to obtain the generator potential, and this potential was thresholded to obtain action potential rate. The generator potential is so called because it is the difference between membrane potential and action potential threshold. Model equations and parameters are provided in Materials and Methods.
The model is the same, apart from minor adjustments to three parameters, as one used in a previous study (Nguyen and Freeman, 2019) of the origin and development of orientation selectivity. Figure 2 illustrates the development process. Figure 2A (left) shows a map of retinal ganglion cells with on- and off-center cells shown, respectively, in red and blue. The spatial statistics of this array reproduce those found by Wässle et al. (1981) in the cat's retina (Nguyen and Freeman, 2019). Development proceeds by selecting a geniculocortical synapse driven by one retinal ganglion cell and increasing its strength as described under Development. If the strength change improves the response amplitude of the cortical neuron driven by the synapse, the change is retained. Otherwise, the synaptic strength is reduced. Figure 2A (right) shows the result for the cortical cell located at the black spot, where red and blue spot size indicates synaptic strength: development reduces destructive interference between neighboring on- and off-center channels, thereby segregating the inputs into oriented bands. This map indicates (along with the receptive fields in Fig. 12) that the modeled excitatory neurons are simple cells.
Figure 2B further illustrates the development process with direction tuning curves for the cortical neuron illustrated in Figure 2A (right). Four cycles in the development process are shown. Response amplitudes are very weak at the start of development and amplitudes improve as destructive interference between on- and off-inputs decreases. The stimulus here is a drifting grating whose direction is shown on the horizontal axis, and the twin peaks indicate that the neuron is orientation-selective. Further, the peak for upward motion improves while the other peak declines as development progresses, indicating increasing direction selectivity. Finally, Figure 2C shows the preferred orientation for all neurons in the area of visual field examined (an area of visual field was analyzed but only the central is displayed, to avoid edge effects). This orientation map matches several characteristics of empirical maps, as described previously (Nguyen and Freeman, 2019).
Model neurons are selective for motion direction
I stimulated neurons in the model with a drifting grating, as shown in Figure 3A (inset), to test for motion direction selectivity. Grating direction varied over its full range, and responses were calculated for excitatory cortical cells. The vertical axes show the fundamental Fourier amplitude of impulse rate. The graph at left represents the responses of a direction-selective cell, the cell whose synaptic strengths are given on the right of Figure 2A. This cell, whose location is shown in black in the visual field inset, prefers horizontal gratings, and its response therefore peaks for upward and downward movement. The peak for upward movement is bigger, meaning that this cell is selective for upward motion. The other cell represented is nonselective in that its two peaks are of much the same height.
The curve fitted to these data sums two von Mises functions whose separation was unconstrained, as shown in Equation 15. The curve was a good fit in that it accounted for 97% of variance across the 3721 responses fitted. The degree of direction selectivity was measured conventionally with the index defined in Equation 16; this index ranges from 0 (nonselective) to 1 (fully selective). Figure 3B shows the index for every excitatory cortical cell in the visual field. There are regions in which cells are maximally selective, but they are sparsely distributed relative to the orientation domains shown in Figure 2C. This sparseness is consistent with imaging of direction-selective responses in cat cortex (Shmuel and Grinvald, 1996; Nishiyama et al., 2019). Figure 3C shows a map of preferred direction superimposed on a desaturated copy of the orientation preference map displayed in Figure 2C. It can be seen that direction preference generally varies slowly within an iso-orientation domain, but can also abruptly reverse, as described in published accounts (Shmuel and Grinvald, 1996; Swindale et al., 2003; Ohki et al., 2005).
Selectivity results from inhibition timing
Priebe and Ferster(2005) found that the relative timing of excitatory and inhibitory responses was critical in establishing direction selectivity. Figure 4A shows the corresponding result for the direction-selective neuron in Figure 3A. The stimulus was a (2 Hz) drifting grating, and the response is therefore cyclic. The red and blue curves show, respectively, the summed excitatory and summed inhibitory inputs to the neuron. The resulting output, shown in black, is the difference of these inputs as filtered by the neuron's first-order delay. The peak of the response in the preferred direction is well above spike threshold (shown dashed), and therefore produces the larger response shown in Figure 3A. The peak of the response in the antipreferred direction (the direction opposite to preferred) is much closer to threshold because the excitatory and inhibitory inputs are more closely aligned. It therefore produces a small peak in the tuning curve.
To see whether this was a general result across direction-selective cells, I calculated the correlation coefficient between the excitatory and inhibitory inputs for each neuron in an area containing many direction-selective cells. Inputs that are aligned in time will produce high correlations. Figure 4B provides a histogram of the correlations and shows that inputs are closely aligned for the antipreferred direction and less so for the preferred direction. This result, according to a two-sample Kolmogorov-Smirnov test, is highly significant (). It can therefore be concluded that direction selectivity in the model results from the relative timing of excitatory and inhibitory inputs.
Selectivity requires dynamic inhibition
Figure 4 provides evidence that directon selectivity depends on the timing of signals from inhibitory cells. This leaves the possibility that the dynamics of the geniculocortical inputs are responsible for changing the relative timing of excitatory and inhibitory inputs to a cell, and that cortical input is therefore ultimately responsible for direction selectivity. I addressed this possibility by removing dynamic inhibition without changing cortical input. Dynamic inhibition can be turned into static inhibition by setting the time constant of inhibitory cells infinite. Figure 5A shows the result of setting all inhibitory time constants to a very large value (1000 s), resulting in essentially static inhibition. The illustrated neuron is the direction-selective cell in Figure 3A, and the color code is the same as in Figure 4A; the flat blue line therefore indicates static inhibition. Figure 5A (right), which shows the tuning curve for the same cell, indicates that direction selectivity has been abolished. This result is general in that static inhibition reduced the DSI of all excitatory cells in the visual field to <0.05. This result, along with that in the previous figure, shows that dynamic inhibition in the model is both necessary and sufficient for direction selectivity.
The removal of direction selectivity by increasing the inhibitory time constant suggests that, conversely, direction selectivity can be improved by reducing the time constant. Figure 5B (left), which illustrates the sensitivity of the model to changes in the inhibitory time constant, shows that this is the case. A reduced time constant increases the fraction of excitatory neurons with a DSI > 0.5 (black line). This increased selectivity, however, comes at a cost. The green line shows the mean of peak amplitude in response to a drifting grating. The response drops substantially as inhibitory time constant is reduced. The time courses in Figure 5B (right) show why this is so: with the reduced time constant used here (20 ms), the inhibitory input to the excitatory cell has higher amplitude and mean and is advanced in time. Increased inhibition, in turn, reduces the excitatory cell output almost to threshold. The inhibitory time constant (100 ms) used in the remainder of the paper provides a physiologically plausible balance between direction selectivity and response amplitude.
Inhibitory cells are orientation- but not direction-selective
Perhaps inhibitory cells are themselves direction-selective, and excitatory cells simply inherit the property from inhibitory cells. Figure 6A shows that this is not the case. The inhibitory cell illustrated there is at the same location as the direction-selective cell shown in Figure 3A. There are two peaks with essentially the same amplitude in the tuning curve, indicating that direction selectivity is absent. This result held across the whole population of inhibitory cells in that all such cells in the visual field have a DSI < 0.01. There are varying reports of inhibitory cell direction selectivity in the literature; these are described in the Discussion.
The tuning curve in Figure 6A indicates, however, that the illustrated cell is orientation-selective because of the presence of response peaks ∼180° apart. These peaks are considerably wider than those seen in excitatory cells (Fig. 3A) because of the absence of the iceberg effect (inhibitory cells in the model are not themselves inhibited). This broadness of orientation tuning is shown to be a general result on the right of Figure 6A. This graph is a histogram of orientation bandwidth (half-width at half-height) for all excitatory and inhibitory cells in the visual field. Inhibitory cells are clearly broadly tuned: their bandwidth is substantially larger than that of their excitatory counterparts. This result, which is consistent with empirical observations (Cardin et al., 2007), will prove to be important later in the paper.
Inhibition is spatially asymmetric
There would be no direction selectivity if an excitatory neuron received identical inhibitory signals from all points in its neighborhood. Figure 6B shows that inhibition differs between the visual field side from which a preferred stimulus approaches, and recedes from, the receptive field. The graphs show the tuning curve for the direction-selective cell in Figure 3A; its location is shown by the spot in the visual field inset. Inhibitory cell time constants were set very large, yielding static inhibition, in the clear areas of the visual field but kept their standard values in the shaded areas. The tuning curve is essentially normal when the dynamic inhibition is restricted to the side from which the antipreferred stimulus approaches, but the direction preference disappears or reverses when dynamic inhibition is on the other side of the visual field. Inhibition is therefore spatially asymmetric for this neuron.
Inhibition can be spatially mapped
Figure 6B provides only a rough guide as to where effective inhibition originates in the receptive field. We can obtain a much more detailed map of effective inhibition by the analysis shown in Figure 7. Figure 7A shows time courses for the excitatory cell represented by the red dot in the visual field inset: this is the same direction-selective cell as in Figure 3A. The red curve gives the summed excitatory input to the cell and the blue curve shows input from a single inhibitory cell indicated by the blue dot in the visual field inset. These two curves are in antiphase when the stimulus is in the preferred direction. Given that the excitatory cell's output depends on the difference between these two signals, the inhibitory input is facilitatory in this case. The two curves are more in-phase, and the inhibition is therefore suppressive, when the stimulus moves in the antipreferred direction.
Figure 7B shows the effect of all inhibitory cells around the same excitatory cell. The color code gives the covariance between excitatory and inhibitory signals for each location in the visual field. Yellow indicates positive covariance, and therefore response suppression. Blue, indicating negative covariance, indicates response facilitation. Preferred direction of motion is shown on the left, antipreferred on the right. It can be seen that the two patterns differ most in the upper field, a result that reflects the finding in Figure 6B that the inhibition effective in setting preferred direction is in the upper field. The curious spatial patterns of effective inhibition seen in Figure 7B derive from the orientation preference map, as we will see below.
Mean suppression increases with direction selectivity
It is efficient to condense the two maps in Figure 7B into a single map by taking their difference. Figure 8A shows the map for the antipreferred direction less the map for the preferred direction; two excitatory cells with relatively high direction selectivity are shown, and their locations marked by circles. It is clear that yellow areas prevail over blue areas, suggesting that suppression (as indicated by yellow) increases in tandem with direction selectivity. This idea is borne out by the map at the left of Figure 8B. Each point in this map gives the (normalized) mean covariance difference for the excitatory cell located at that point. Comparison of this map with the map of DSI obtained from tuning curves (see Fig. 3B) shows a strong correspondence. Figure 8B (right) compares mean covariance difference directly with DSI. The correlation coefficient in this figure is 0.75, which is highly significant (). We can conclude that the covariance between excitatory and inhibitory signals provides a guide to the strength of direction selectivity.
Responses to stationary stimuli predict preferred direction
We cannot claim to understand the mechanism underlying direction selectivity until we can predict both the preferred direction and strength of direction selectivity in any given neuron. The prediction, to be rigorous, must be made from neuronal responses not involving moving stimuli. I therefore now turn to a stimulus, the contrast-reversing grating, that has proven very useful in previous studies of motion direction selectivity (Reid et al., 1987, 1991; Jagadeesh et al., 1997). Contrast at any point in this stimulus varies sinusoidally in time, but the pattern as a whole does not move. The attraction of this stimulus is that it can be placed at a variety of locations, or spatial phases, each of which represents a snapshot of a moving grating. If the underlying mechanism is linear, these responses can be compiled into a replica of the response to movement.
Figure 9A shows the response to a contrast-reversing grating for the direction-selective neuron illustrated in Figure 3A. The grating used here and in the following analysis was optimally oriented. The response is cyclic, and the amplitude and phase of its fundamental Fourier component are shown in the graphs at left and right, respectively. The response used is the generator potential because impulse rate is zero at several spatial phases, and response phase is undefined when response amplitude is zero. The horizontal axes give the spatial phase of the stimulus, which varied across its full 360° range; representative phases are illustrated by the diagrams below the axes. As seen in the left graph, response amplitude is minimal at two spatial phases separated by 180°. These positions of the grating maximize destructive interference between responses from opposite sides of the receptive field.
Figure 9A (right) represents response phase versus spatial phase. The ordering of both of these axes is important for understanding the result. The spatial phase, from left to right, represents successive placements of the stimulus in the neuron's preferred direction. Response phase, from bottom to top, indicates increasing phase advance, that is, earlier times. The origin of both axes is arbitrary (spatial phase, e.g., was defined relative to the center of the visual field), and the plot is therefore vertically shifted so that it starts at the origin. The graph shows that response phase advances as the stimulus is displaced in the preferred direction. Reid et al. (1987) pointed out that this makes intuitive sense: as the grating is successively displaced from left to right, responses obtained at later times will be advanced relative to previous responses, resulting in addition and larger responses. By the same logic, displacement in the antipreferred direction will disperse successive responses, resulting in a loss of addition and weaker responses.
Figure 9B shows phase plots for a variety of neurons, with one line per neuron. Plots are shifted laterally so that the highest gradient is at the origin. The 10 cells at left were located in visual field areas in which direction selectivity was highest (Fig. 3B), whereas the 10 cells at right had some of the lowest direction selectivity indices. Cells with high direction selectivity have an ascending phase plot, as expected. For the cells at right, by contrast, plots are essentially flat with interspersed jumps of ∼180°. Every cell in the visual field fell into one of these two patterns. In other words, if a cell were direction-selective, its response phase advanced with spatial phase. Conversely, preferred direction can be predicted by finding the direction of stimulus displacement that advances response phase.
Stationary stimuli reveal spatial asymmetries in inhibition
We have seen with moving stimuli that the inhibitory input to an excitatory cell is asymmetrically distributed around the cell. Spatial asymmetry can also be demonstrated with a contrast-reversing grating, as illustrated in Figure 10A. The horizontal axis shows the spatial phase of a contrast-reversing stimulus, and the vertical axis gives response amplitude for the direction-selective cell in Figure 3A. The red line, which shows the summed excitatory input to the cell, goes to zero at phases 180° apart. This minimum indicates that response increments from part of the receptive field are nulled by response decrements from the remainder of the receptive field. To this extent, the phase at which amplitude goes to zero indicates the location of the excitatory input. The blue line, which shows summed inhibitory input to the cell, goes to zero at a different phase, which indicates that the location of the inhibitory input is spatially offset from that of the excitatory input. This spatial offset, shown in the figure, potentially provides the spatial asymmetry of inhibition during a moving stimulus.
This is a result that applies across all cells in the visual field. Figure 10B, a histogram of spatial offset, shows a strong bias toward positive values. A one-sided sign test () indicates that this result is highly significant. In other words, the inhibitory input is reliably displaced in the preferred direction relative to the excitatory input. Figure 10C maps spatial offset onto the visual field; the map as a whole has similarities to the map of the DSI obtained with tuning curves from moving stimuli (Fig. 3B). Indeed, the correlation between the two maps was positive () and highly significant (). The correlation between spatial offset and DSI is shown in the right graph.
These results make it clear that the inhibitory input to direction-selective cells arises from a visual field area differing from that of the excitatory input. How does that lead to direction selectivity? To answer that question, I plotted responses in polar coordinates: the length and angle of the vectors in Figure 11 give response amplitude and phase, respectively, as indicated in Figure 11A (left). The response of the cell in Figures 9A and 10A is shown by the circles on the right of Figure 11A. The elliptical form of the data is consistent with neuronal recordings (Reid et al., 1987, 1991).
The ellipse fitted to the data in Figure 11A was calculated as follows. Neither excitatory nor inhibitory inputs are direction-selective, so the vectors that represent them have an almost constant angle, as shown in Figure 11B. The inhibitory vector is retarded (rotated clockwise) relative to the excitatory vector because the inhibitory axon has a long time constant. The amplitudes of the excitatory and inhibitory vectors depend on the spatial phase of the stimulus, and the output response of the cell is closely approximated by their weighted difference (for the details of the calculation, see Materials and Methods).
The contribution of the vectors to this ellipse is conveniently shown by the two spatial phases marked by the dots in Figure 10A. For each dot, one vector has zero amplitude, so the other (weighted) vector lies on the ellipse, as shown in Figure 11C. The important point here is that the inhibitory vector has non-zero amplitude at a more advanced phase than the excitatory vector because of the spatial phase offset, and advancing spatial phase therefore leads to advancing response phase. The direction selectivity of the cell, in turn, is because of the spatial offset between the excitatory and inhibitory inputs to the cell. The same argument, made for one cell, can be applied to all direction-selective cells. For comparison, the responses of a direction-nonselective cell are shown in Figure 11D. There is no spatial offset between the excitatory and inhibitory inputs here, so their vectors vary in synchrony.
Inhibitory asymmetries result from the orientation preference map
According to the model, direction selectivity in a cell arises from spatial asymmetries in the cell's inhibitory inputs. These asymmetries result, in turn, from the orientation preference map as illustrated in Figure 12A. This shows, at left, the preference map of all excitatory cells in the visual field. The black dot on the map gives the location of the direction-selective cell used as example in Figure 3A. The blue dots show the location of two nearby inhibitory cells, one each in the preferred (above) and opposite (below) directions. At right are shown the receptive fields of each of these three cells. The receptive fields were obtained by flashing squares of light or dark at a variety of visual field locations, as described in Materials and Methods. The response used to map the receptive fields is the generator potential to facilitate comparison with Figure 12B.
There is a clear difference in the receptive fields of the two inhibitory cells. The cell in the lower field has much the same orientation preference as the excitatory cell and, indeed, occupies similar visual field territory. (This is because the synaptic weights of the subcortical inputs are similar for the two cells.) The cell in the upper visual field, by contrast, prefers an orientation differing by ∼45° from that preferred by the excitatory cell. It still responds well to the preferred orientation of the excitatory cell because of its relatively broad orientation tuning (Fig. 6A). Further, the inhibitory receptive field has a different distribution of subfields. These differences contribute to the spatial inhomogeneity of the input to the excitatory cell. They also explain the spatial asymmetry in covariance seen in Figure 7B.
The analysis in Figure 10 showed that the summed inhibitory inputs to a direction-selective cell are spatially offset from the summed excitatory inputs. Figure 12B illustrates this finding for the excitatory cell shown in Figure 12A. The graph at left represents the receptive field of the summed excitatory inputs obtained by sparse noise stimulation. The receptive field has nearly even symmetry about the cell's location, shown by the circle. The graph on the right, which gives the receptive field of the summed inhibitory input, differs in that the receptive field is displaced in the preferred direction (upwards). The spatial offset obtained by the analysis in Figure 10 is shown.
Alternative cortical circuitry
Primary visual cortex comprises a network of richly interconnected neurons, and many of these connections have been omitted from the model in the interests of simplicity. In particular, geniculo-recipient neurons receive only a small fraction of their input from the thalamus (Ahmed et al., 1994), which means that the model ignores excitatory-to-inhibitory, excitatory-to-excitatory, and inhibitory-to-inhibitory connections. Given that excitatory-to-inhibitory synapses are likely to influence direction selectivity, I added a monosynaptic connection from excitatory-to-inhibitory neurons. A major difficulty with an addition like this is that little is known about the spatial pool of neurons from which the connection arises, nor its gain (Douglas and Martin, 2004). The excitatory-to-inhibitory connection was therefore added with the simplest possible assumption, namely, that it replicated the convergence function of the pathway from inhibitory to excitatory cells. Details are provided in Materials and Methods.
The result is shown in Figure 13. The circuit at the top shows the added connection, in green. The left graph provides the motion direction tuning curve for the direction-selective neuron in Figure 3A. The added circuitry reduces its impulse rate, as expected of the extra inhibitory drive, but enhances the motion selectivity (because of the iceberg effect). This is a general result, as shown in the graph on the right: the histogram of the DSI for all neurons shifts toward full direction selectivity. This result suggests that the model's properties are robust enough to withstand added connections.
Discussion
Putative inhibitory neurons in cat primary visual cortex are orientation-selective (Cardin et al., 2007). The model analyzed here shows that this empirical observation leads to motion direction selectivity in the following steps.
• Inhibitory neurons lie in the orientation preference map and therefore have differing orientation preferences on one side of an excitatory cell from those on the other side.
• The overall map of inhibition around an excitatory cell is thus inhomogeneous.
• This inhomogeneity leads to a summed inhibitory signal that is spatially offset from the summed excitatory signal to the excitatory cell.
• The spatial offset results in an advance in temporal phase as a grating moves in one direction across the receptive field relative to movement in the opposite direction.
• The preferred direction of motion of the excitatory cell matches that of the advancing temporal phase.
While it has been known for some time that advancing stimulus spatial phase leads to advancing response phase in direction-selective cells (Reid et al., 1987), the mechanism underlying the advance has been disputed. The results shown here, from a physiologically plausible model, provide such a mechanism.
The model predicts that, if the summed excitation and inhibition to an excitatory cell are spatially offset, the cell's preferred direction will be the same as that of the inhibitory offset (Fig. 10), and that the magnitude of the cell's DSI will increase with the size of the offset. This is a difficult prediction to test without access to intracellular recordings. Given that neighboring excitatory and inhibitory cells are assumed to have similar preferred orientations, it may still be possible to test with extracellular recordings by averaging the receptive fields of excitatory cells around a cell of interest and cross-correlating the mean with the cell's receptive field. I now discuss some of the issues arising from the results.
Previous models
Peterson et al. (2004) used a sparse noise stimulus to measure spatiotemporal receptive fields of direction-selective simple cells in cat primary visual cortex. They modeled their result by assuming that each simple cell received input from two direction-nonselective sources separated in both space and time. Inputs could be from direction-nonselective simple cells or lagged and nonlagged cells in the lateral geniculate nucleus. Data fitting showed that the timing difference between inputs varied substantially across the recorded cells. Small timing differences, in particular, were inconsistent with lagged cell input. There are interesting parallels between the Peterson et al. (2004) model and the one described in the present paper. The two direction-nonselective inputs assumed by Peterson et al. (2004) could correspond with the summed excitatory and inhibitory inputs used here. Further, the spread of timing differences seen in the empirical data could correspond with the spread of spatial offset between excitatory and inhibitory inputs seen here.
Another relevant model comes from primate work. Livingstone (1998) recorded responses from direction-selective cells in primary visual cortex of awake macaques. The stimuli were optimally oriented bars flashed briefly at a variety of locations across the receptive field. The resulting space-time plots were slanted in a direction that indicated faster responses when bars were displaced in the preferred direction, and areas of reduced firing rate (interpreted as inhibition) were displaced in the preferred direction from areas of excitation. Livingstone (1998) successfully modeled these plots by assuming an inhibitory receptive field subregion that was offset in the preferred direction from an excitatory subregion. My model matches well with this description. Livingstone (1998) ascribed the displacement of the inhibitory subregion to the Meynert cell, which has dendrites located asymmetrically relative to the cell body. The main problem with this proposal is that Meynert cells reside in cortical layers 5 and 6, whereas direction-selective cells are prevalent in layer 4. Instead, I attribute asymmetric inhibition to anisotropies in the orientation preference map.
More recently, in mouse cortex, it has been shown that direction-selective neurons receive both transient and sustained inputs from the lateral geniculate nucleus, and that the timing difference between these inputs contributes to the selectivity (Lien and Scanziani, 2018). There are, however, substantial differences between mouse visual pathways and those in carnivores and primates. There is robust direction selectivity in the mouse geniculate (Marshel et al., 2012; Piscopo et al., 2013) and subcortical direction-selective neurons project to direction-selective cortical cells (Cruz-Martin et al., 2014; Sun et al., 2016). In carnivores and primates, by contrast, direction selectivity in the lateral geniculate nucleus is weak relative to cortex (Thompson et al., 1994; Xu et al., 2002). Further, and importantly for the model described here, mouse cortex does not have iso-orientation domains. It is unclear, therefore, to what extent the results in Lien and Scanziani (2018) are relevant here.
Prevalence of direction-selective neurons
The model described here aims for physiological plausibility. A key statistic which the model should match is the fraction of primary visual cortical neurons that are direction-selective. The problem here is that estimates in the literature are somewhat variable. Imaging studies describe patchy occurrence across the cortical surface (Shmuel and Grinvald, 1996; Nishiyama et al., 2019). The model reproduces this patchiness (Fig. 3B). Single-neuron electrophysiology, on the other hand, typically finds more than half of the sampled cells have a DSI ≥ 0.5 (Peterson et al., 2004). There are caveats with both types of measurement: imaging records from only the superficial cortical layers, and electrophysiological recordings are subject to sampling biases.
If the fraction of direction-selective cells in the model turns out to be an underestimate, there is a potential remedy: neuronal plasticity. It has been shown that direction selectivity can increase substantially in just 9 h of training (Ritter et al., 2017), but the model described here assumes that intracortical synaptic strengths are fixed. There is ample evidence for plasticity of inhibitory synapses (Allene et al., 2015; Guan et al., 2017). It may therefore be possible to increase the prevalence of direction-selective cells in the model by assuming plasticity in the synapse of inhibitory cells onto excitatory cells, and then training the model with moving stimuli. A second potential remedy is a connection from excitatory to inhibitory cells: Figure 13 shows that this connection improves direction selectivity.
Another factor in the prevalence of direction selectivity is species. Using single-neuron electrophysiology, Peterson et al.(2004) found that 81% of neurons in cat primary visual cortex had a DSI ≥ 0.5. By contrast, that number drops to 47% in the primate (Gur and Snodderly, 2007). High prevalence of direction selectivity in primates is found, not in primary cortex, but in area MT where the fraction of neurons with a DSI > 0.5 is essentially 100% (Albright, 1984). It seems, therefore, that whereas there is a high prevalence of direction-selective neurons in cat primary visual cortex, strong direction selectivity in the primate is postponed to a downstream stage.
Properties of inhibitory neurons
Inhibitory neurons in the model are orientation-selective, consistent with empirical findings (Cardin et al., 2007). It is assumed in the model that inhibitory neurons have an orientation preference map matching that of the excitatory cells. That is, each inhibitory cell is assumed to have the same orientation preference as its nearest excitatory cell. This assumption is consistent with the columnar organization of visual cortex: orientation preference varies little in any cortical penetration perpendicular to the surface (Hubel and Wiesel, 1962; Jin et al., 2011).
There is a second property of model inhibitory neurons for which there is less experimental evidence. Figure 6A shows that the inhibitory cells are unselective for motion direction. Azouz et al. (1997) recorded from physiologically and morphologically identified inhibitory cells in cat primary visual cortex. None of the eight cells they tested was clearly direction-selective. Imaging of ferret primary visual cortex (Wilson et al., 2017) showed strong direction selectivity among the inhibitory neurons. The imaging, however, was limited to layer 2 or 3 of cortex, whereas the cortical region modeled here is layer 4. One way in which inhibitory cells could acquire direction selectivity in the model is through an added connection from excitatory to inhibitory cells, as illustrated in Figure 13. Inhibitory cells would then inherit direction selectivity from excitatory cells. The difficulty with exploring this idea in the model is that there appears to be little or no published data on the strength of synaptic connections between excitatory and inhibitory cells.
Random variation
The map of preferred direction (Fig. 3C) is irregular, and it is of interest to know how this irregularity arises. There are two sources of random variation in the model, both structural. The first is the random perturbation that displaces retinal ganglion cells from a regular grid: each retinal ganglion cell array generates its own orientation preference map (Nguyen and Freeman, 2019). The second source of random variation is the sequence in which geniculocortical synaptic strengths were changed during the development process. Changing this sequence only slightly altered the orientation preference map: on average, the orientation difference between the two maps had an SD of 23.5 deg. This difference, however, produced a marked change in the resulting direction preference map. In particular, 34% of direction domains switched to the opposite preferred direction. Evidently, the direction preference map is sensitive to small changes in the orientation preference map.
Mechanism underlying direction selectivity
This paper proposes a mechanism leading to direction selectivity in visual cortex, as summarized in Figure 12. Inhibitory neurons are orientation-selective and lie on an orientation preference map identical to that of excitatory neurons (Fig. 12A, left). Orientation preferences of inhibitory neurons surrounding any excitatory neuron are therefore variable, leading to spatially inhomogeneous inhibition (Fig. 12A, right). This leads to a spatial offset between the summed excitatory and inhibitory inputs to the excitatory cell, as shown in Figure 12B. The summed inhibitory input is delayed relative to the excitatory input because of a long time constant in the inhibitory cell axon (Fig. 12C, copied from Fig. 4). A stimulus moving in the antipreferred direction encounters the inhibitory input before the excitatory input, shifting the inhibitory input closer in time to the excitatory input (Fig. 12C, right). This results in destructive interference and a weak response in the excitatory cell. Stimulus motion in the preferred direction, by contrast, activates the excitatory input before the inhibitory input, delaying inhibition and reducing interference.
This mechanism leads to two speculations. First, Li et al.(2006) showed that direction selectivity in ferret visual cortex develops later than orientation selectivity. They did not suggest a causal relationship between these two properties, but it is tempting to suggest that orientation selectivity is required before direction selectivity can develop. Second, Nguyen and Freeman(2019), who described the model used here, showed that the model's orientation preference map originates in retinal ganglion cell disarray. The analysis here proposes that direction selectivity in turn depends on the orientation preference map. According to the model, therefore, direction selectivity ultimately originates in retinal disarray. Whether this chain of reasoning is correct awaits testing in the laboratory.
Footnotes
I thank Jose Manuel Alonso for providing valuable comments on an earlier version of this paper; and Artemis, the University of Sydney's high-performance computing center, for the resources required to compute the geniculocortical weights and receptive fields.
The author declares no competing financial interests.
- Correspondence should be addressed to Alan W. Freeman at Alan.Freeman{at}sydney.edu.au