A Model of Binocular Motion Integration in MT Neurons

A framework for image-computable binocular MT models

We constructed a binocular multistage modeling framework (Fig. 1) to unify concepts from past studies of pattern motion selectivity and to account for the experimental results of recent studies of MT that presented different motion stimuli to each eye. Key elements from past models include the following: front-end DS filters, response normalization, motion opponency, and weighted integration across multiple direction channels. We introduced a parallel left- and right-eye dual stream structure and allowed the flexibility to vary the mixing of left and right eye signals at various points along the streams and the ability to vary the order of operations for opponency and binocular integration.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Schematic of the binocular MT model. For complete details, see Materials and Methods. A, The front end of the model is composed of 24 spatiotemporal motion energy filters (12 in each eye) in 30° steps of preferred direction. The visual stimulus is convolved with these filters to generate input to the model. B, Tuned and untuned normalization is applied to scale the raw V1 filter outputs. C, Variable strength opponent motion suppression may be computed by subtracting a proportion (c_opp) of the signal from the antipreferred direction V1 channel. This step can occur before or after binocular integration. D, Variable strength V1 binocular integration may be implemented by combining a weighted sum of signals from the left and right streams within a direction channel. For example, b = 0.7 produces left stream signals composed of 70% monocular left eye signal and 30% monocular right eye signal, and vice versa for the corresponding right stream. E, Weighting of V1 inputs to MT. Pattern cell weights (red traces) are broadly tuned and have significant inhibitory weights, whereas component cells typically have only one strong positive weight and negligible inhibitory weights (blue traces). Weights are shifted 180° between FP motion tuned and 3DT models (see Fig. 7). F, After weighting, signals are summed across both eyes and all direction channels. There is an optional additional weighting by a scale factor A_R that reduces the strength of all right streams equally before summation. G, An output nonlinearity is applied to the summed signal, before optional scaling and offsetting, to produce spiking MT output. The nonlinearity can be an exponential (solid trace, after Rust et al., 2006) or simple rectification (dashed trace).

Our observations and insights from the model are organized below as follows: (1) We first develop image-computable pattern and component cell models that are consistent with past monocular studies, and we extend these in the simplest way to operate on binocular stimuli. (2) We then test and refine the binocular models to account for the loss of pattern motion sensitivity when moving plaid gratings are presented dichoptically. (3) We further test and refine the models to account for 3D motion sensitivity in area MT, making predictions to guide future experimental studies to examine whether there is a relationship between pattern motion and 3D motion sensitivity. (4) Finally, we show our results hold when a canonical binocular disparity computation is included in the models.

Monocular characterization of pattern motion sensitivity in the binocular MT model

Our first goal was to reproduce within our framework the widely studied selectivity of MT cells for pattern motion. MT cells have traditionally been ranked along the continuum from component-DS to pattern-DS based on their responses to plaid stimuli: component cells respond best when either constituent grating of the plaid matches the preferred direction of the cell (measured with single gratings), whereas pattern cells respond best when the overall pattern moves in the cell's preferred direction (Movshon et al., 1985; Rodman and Albright, 1989; Stoner and Albright, 1992).

A thorough characterization of this behavior was performed by Rust et al. (2006) who used a full range of directions for two-component and multicomponent sinusoidal stimuli to monocularly drive MT units that spanned the spectrum from component to pattern cell behavior. They fit the resulting responses with a monocular cascade model (Fig. 3A–D). In the first stage of the cascade model, the responses of 12 DS V1 channels were modeled as direction tuning curves spaced evenly over the full 360° range (Fig. 3A). The responses were then divisively normalized by the sum of two components: a direction-tuned self-normalization term and an untuned term that summed signals across all directions. The model MT unit computed a weighted sum of these normalized V1 outputs (weights for component and pattern example cells are shown in Fig. 3B,C), and the summed signal was passed through an output nonlinearity to produce a spiking response (Fig. 3D). These last three steps correspond to the steps shown in Figure 1B, E, G in our framework, respectively. Rust et al. (2006) found that a primary distinction between fits to component and pattern MT cells was the distribution of weights from V1 to MT: pattern cell weights were broadly direction tuned, with strong, oppositely tuned inhibitory weights (Fig. 3C), whereas component cells had narrowly tuned excitatory weights and sparse, low-strength inhibitory weights (Fig. 3B).

• Download figure
• Open in new tab
• Download powerpoint

Figure 2.

Overview of the circuitry for the binocular disparity computation. A, Normalized even and odd linear filter signals from the left and right eyes are duplicated and negated before motion opponency to allow the disparity computation. B, The motion opponent signals are subtracted from each subchannel and the resulting signals rectified (Eq. 10). C, The rectified signals from left and right eyes are combined in a weighted sum and the result squared (Eq. 11). D, The summed, squared outputs are summed in a final step to produce binocular disparity energy signal (Eq. 10).

• Download figure
• Open in new tab
• Download powerpoint

Figure 3.

Monocular fits to experimental data on component and pattern cells. Example component and pattern cells in our binocular model using the fits of Rust et al. (2006). Parameters for left and right eyes are identical. Responses shown are for monocular left eye stimulation only. A, The monocular model of Rust et al. (2006) used 12 V1 channels modeled as tuning curves, with the output from each channel normalized. These 12 signals were weighted to fit a component cell (B, Rust et al., 2006, their Fig. 4A) and pattern cell (C, Rust et al., 2006, their Fig. 4E). The summed output was then passed through a nonlinearity and the signal used to produce spikes (D). E, Tuning curves for the component cell reproduced from Rust et al. (2006): their Fig. 4A for single gratings (blue trace) and 120° plaids (red). F, Tuning curves for our model component cell using the weights in B. G, Tuning curves reproduced from the pattern cell in Rust et al. (2006, their Fig. 4E). H, Tuning curves for our model pattern cell using weights shown in C. I, 2D map of responses to plaids of all directions (in 30° steps) for the component cell in E. J, Responses to plaids from our model component cell in F. K, Plaid matrix responses for the pattern cell in G. L, Responses to plaids from our model pattern cell shown in H.

To build representative binocular pattern and component MT units in our framework, we fit our model to example cells for which Rust et al. (2006) presented extensive data. Taking a minimalist approach to building binocular models, we set identical parameter values in both the left and right streams and simply summed the signals from both streams (Fig. 1F) before applying the MT output nonlinearity to generate spikes. We extracted the key parameters (V1 tuning curve bandwidth, normalization strengths, MT weight distributions, and MT nonlinearity) from the data presented in Rust et al. (2006) by inspection of their Figure 6A, E (see Materials and Methods); we chose as representative units of their most component-like and most pattern-like example cells (their Fig. 4A and Fig. 4E, respectively). We tested our binocular models using monocularly presented stimuli to be consistent with experimental conditions. A side-by-side comparison of the recorded MT data (plots reproduced from Rust et al., 2006) and our fits are shown in Figure 3E–L, where we have adopted their plotting conventions to facilitate comparison. The classical single grating and single plaid tuning curves for the model component cell (Fig. 3F) and pattern cell (Fig. 3H) closely match the recorded data (Fig. 3E,G). When tested with a full matrix of plaid combinations (Fig. 3J,L), we also obtained a good qualitative match to the experimental results (Fig. 3I,K). We have also implemented models reproducing data from the other three example cells of Rust et al. (2006, their Fig. 4B–D; see supplemental material).

• Download figure
• Open in new tab
• Download powerpoint

Figure 4.

Reduced pattern motion sensitivity with dichoptic plaids in the binocular models. Direction tuning for monocular and dichoptic plaids as in the protocol of Tailby et al. (2010). Pattern index values for the monocular and dichoptic conditions are shown as insets on all plots. A, Direction tuning curves for the component cell from Figure 3F, J for monocular 120° plaids (black trace) and dichoptic plaids (green trace). B, Monocular and dichoptic plaid direction tuning curves for the pattern cell in Figure 3H, L. C, Component cell tuning curves where inhibition is reduced to 40% of monocular strength when the component gratings of the plaid are presented dichoptically, following the method of Tailby et al. (2010). D, Tuning curves for the pattern cell model with inhibition reduced to 40% of monocular strength with dichoptic presentation.

Reduction of pattern motion sensitivity with dichoptic plaids

Having created image-computable, binocular models that accounted for the responses of typical pattern-DS and component-DS cells to monocularly presented stimuli, we next examined whether these models could account for the robust loss of pattern-DS behavior when the plaid stimuli were presented dichoptically rather than monocularly (Tailby et al., 2010). This experimental observation provides an important constraint on the integration of motion information from V1 to MT, indicating that monocular mechanisms are critical for pattern motion sensitivity. To explain their dichoptic plaid results, Tailby et al. (2010) proposed an adjustment to the monocular cascade model of Rust et al. (2006) but did not provide a unified model to account for both monocular and binocular responses. Our goal in this section is to verify that the adjustment proposed by Tailby et al. (2010) works within our binocular image-computable framework. Thus, we first test our simple binocular models with the dichoptic stimuli of Tailby et al. (2010) to observe the effect of dichoptic presentation on pattern sensitivity in our component and pattern model units, and we then implement and test the adjustment by Tailby et al. (2010). In the next section, we apply the intuition gained here to create a unified model to account for pattern selectivity under both monocular and dichoptic conditions.

The first experiment of Tailby et al. (2010) was to measure the pattern indices of MT neurons monocularly in both eyes, from cells spanning the continuum from strongly pattern-DS to strongly component-DS. They found that the measured pattern indices (PIs) were similar across the two eyes. This holds by construction in our model units because the parameters for the left and right streams were identical: the PIs for left and right eye presentation were both −2.9 for the component cell (Fig. 4A, black trace, tuning curves are identical) and both 6.0 for the pattern cell (Fig. 4B, black trace). We consider PI < −1.28 to be indicative of a component cell and PI > 1.28 to be indicative of a pattern cell with values in between unclassified, as in previous studies (Smith et al., 2005; Tailby et al., 2010).

The main finding of Tailby et al. (2010) was that PI values decreased for essentially all MT cells when plaids were presented dichoptically, with one component grating presented to the left eye while the other component grating was simultaneously presented to the right eye. In our model units, the PIs changed very little with dichoptic presentation, with PIs of −2.8 for the component unit (Fig. 4A, green trace) and 6.7 for the pattern unit (Fig. 4B, green trace). The similar PI values for monocular and dichoptic presentation are largely a consequence of the linear combination of the signals in the left and right streams (Fig. 1F), which occurs before the MT output nonlinearity and the identical MT weight distributions across the eyes. Intuitively, splitting the plaid across eyes should produce about the same amount of drive as a monocular plaid because the channels in the left and right eyes have the same weights and respond equally strongly to the component gratings. It is important to note that the results in Figure 4A, B indicate that normalization, raised as a possible key element to the monocular dependence of pattern selectivity by Tailby et al. (2010), do not play a critical role in our models, which have tuned and untuned normalization in accordance with the findings and specific fits of Rust et al. (2006). Specifically, because the normalizations (tuned and untuned) act monocularly, splitting the grating across the two eyes alters the normalization signals within each eye; however, this does not end up having any substantial effect on the pattern index. However, the slight differences in monocular and dichoptic firing rates observed in the pattern model (Fig. 4B, black vs green curves) are a result of the normalization, which is applied to the V1 channels at a monocular stage (Fig. 1B), and this difference in the tuning curves is eliminated if normalization is removed (data not shown). Thus, the simplest binocular extension of the monocular pattern and component models does not explain the experimental observations.

In their study, Tailby et al. (2010) proposed a potential modification to the monocular cascade model by which their dichoptic plaid results could be achieved. They reasoned that the reduction in PI under dichoptic viewing conditions implied that at least some of the mechanisms underlying pattern selectivity must act monocularly. The mechanisms previously identified as essential for pattern selectivity (Rust et al., 2006) were contrast-dependent normalization, integration of V1 inputs over a wide range of preferred directions, and strong opponent motion suppression. Focusing on motion opponency, Tailby et al. (2010) compared plaid direction tuning curves from the monocular model of Rust et al. (2006) with curves from an altered model, used only for dichoptic stimulation, in which the MT inhibitory weights (i.e., the negative weights in Fig. 3B,C) were decreased to 40% of their original strength. This reduction was intended to simulate a decrease in the inhibitory input, relative to that generated by the monocular plaid, when one component of the plaid is shifted to the other eye. They found that they were able to reproduce the reduction in PI from monocular to dichoptic presentation in simulated pattern-DS tuning curves using these two models. We refer to this modified model as the “Tailby adjustment,” noting that they compared the monocular stimulus presented to the standard cascade model, against the dichoptic stimulus presented to the modified model.

We verified the validity of the Tailby adjustment by implementing it in both our component and pattern model units. The component unit (Fig. 4C) produced PIs of −2.9 for the standard model and −5.0 for the reduced-inhibition model, whereas the pattern unit (Fig. 4D) generated PIs of 5.8 and 5.1 for the standard and modified models, respectively. Both of these reductions in PI fall within the range of reductions observed by Tailby et al. (2010, their Fig. 2C). These results validate the insight of Tailby et al. (2010), that reducing the inhibition provided by the negative MT weights for the dichoptic plaid, but not for the monocular plaid, could lead to appropriate models for pattern and component cells.

A single circuit model for representing monocular and dichoptic plaid responses

The Tailby adjustment provides a proof of principle that reducing the negative MT weights during dichoptic stimulation can reduce the PI, but it does not provide a unified, plausible circuit model that simultaneously works for both the monocular and dichoptic stimuli. In addition, Tailby et al. (2010) also reported that pattern cells commonly showed larger decreases in PI than we saw in our pattern cell using the Tailby adjustment, with 11 of their 18 pattern cells recorded no longer being classified as pattern selective with dichoptic presentation (Tailby et al. (2010, their Fig. 2C; under their criterion, a cell is classified as pattern if the pattern correlation coefficient significantly exceeds either 0 or the component correlation, whichever is largest). The key mechanism identified by Tailby et al. (2010) was a reduction, but not elimination, in inhibitory strength with dichoptic presentation, which suggests that both monocular and interocular sources of inhibition are at play. We implemented this in our model by: (1) introducing motion-opponent suppression between V1 channels with 180° differences in direction preference, and (2) selectively reducing the strength of the inhibitory weights from V1 to MT. The strength of the motion-opponent suppression between V1 channels (Fig. 1C) was set by a parameter c_opp (e.g., c_opp = 0.5 means the normalized V1 signal from the opponent motion channel is scaled by 0.5 before being subtracted). To vary the strength of MT inhibitory weights, we introduced a scaling factor k_inh that was applied to only the negative values in the MT weight distribution (Eq. 14). With dichoptic presentation, the V1 motion-opponent suppression between signals driven by the two component gratings is eliminated, reducing the total suppression generated by the plaid stimulus relative to monocular presentation, whereas the suppressive component generated by the MT inhibition is the same in both cases because the MT weight distribution is identical in both left and right eye streams.

For simplicity and generality in these tests, we built canonical pattern and component cell models that have idealized weight distributions. The component cell model MT weights consisted of a δ function at 180° (Fig. 1E, solid blue lines) with scalable opponent MT inhibitory weights across three antipreferred channels (−30°, 0°, and 30°; data not shown in Fig. 1E; see Materials and Methods). The pattern cell model had cosine weights (Fig. 1E, solid red lines), also with adjustable scaling for the inhibitory weights. We used simple rectification for the MT output nonlinearity (Fig. 1G, dashed line). We retained the V1 filtering and normalization parameters from our Rust et al. (2006) example component and pattern units. Introducing the canonical models is important because they embody the key factors underlying the differences between pattern and component cells, as shown by Rust et al. (2006): those being (1) a broad weight distribution for pattern versus a narrow distribution for component cells, and (2) strong opponent inhibition (negative weights) for pattern versus negligible inhibition for component cells. The canonical models retain strong pattern and component tuning while ensuring that our results do not depend on specific details of the weight distributions of the two example cells (Fig. 3B,C). Nevertheless, the following results also hold for the particular Rust et al. (2006) example cells we presented in Figures 3 and 4. The canonical models are used in all subsequent sections for testing plausible configurations of the circuitry for dichoptic and 3D motion sensitivity.

Figure 5A shows the plaid direction tuning curves for the canonical component-DS model that has both V1 opponent motion suppression and no MT inhibitory weights (c_opp = 0.5 and k_inh = 0.0). When tested with monocular plaids (black curve), the model component cell has a PI value of −2.9. When tested with dichoptic plaids (green curve), the PI dropped to −6.0. A similar trend holds for the representative model pattern unit with V1 opponency and reduced inhibitory weights (Fig. 5B; c_opp = 1.0 and k_inh = 0.25), where the PI dropped from 2.9 to −1.4 (compare black and green curves). These decreases in PI, −3.1 and −4.3 for the component and pattern models, respectively, are both within one-half SD of the mean values found in the population data (−2.0, SD 2.2 for component cells, −5.2, SD 3.4 for pattern cells) of Tailby et al. (2010). In contrast to the results above for the Tailby adjustment (Fig. 4C,D), in these simulations the model parameters are the same for both monocular and dichoptic stimulation: for the component cell, 50% V1 opponency and no MT inhibitory weights; and for the pattern cell, 100% V1 opponency and 25% strength MT inhibitory weights. The change in PI with dichoptic stimulation reflects a loss of monocular suppression when one component of the plaid is moved to the other eye, which is not fully recovered by the additional reduced dichoptic MT inhibition.

• Download figure
• Open in new tab
• Download powerpoint

Figure 5.

Effect of monocular and binocular opponent motion suppression on loss of pattern sensitivity with dichoptic plaids. A, Plaid direction tuning curves for a model component cell with 50% V1 opponent motion suppression strength and no MT inhibitory weights. Inset, Pattern index values for the two curves. B, Plaid tuning curves for a pattern cell with 100% V1 opponent motion suppression strength and inhibitory weights reduced to 25% of the full-strength weights. Inset, PI values for the two curves. C, Dependence of the monocular PI on amount of V1 opponency and MT inhibitory weight strength in the component cell model. Blue areas of the plot represent negative PI values. D, Monocular PI values in the pattern cell model as V1 opponency and MT inhibitory weight strength are varied. Blue regions represent negative PI values. Red regions of the plot represent positive PI values. E, Dichoptic PI values as opponency and inhibitory strength vary in the component cell model. F, Dichoptic PI values as opponency and inhibitory strength vary in the pattern model. G, The difference between dichoptic and monocular PI as V1 opponency and MT inhibitory strength are varied in the component model. Blue regions represent a drop in PI. White represents no change. Red regions represent an increase in PI from monocular to dichoptic presentation. *Parameters used to generate the tuning curves in A. H, Difference between dichoptic and monocular PI for the pattern model. *Parameter values chosen to generate the tuning curves shown in B.

To explore how the decrease in PI values depended on the amounts of monocular and dichoptic suppression, we varied the strength of V1 opponency (c_opp) and the strength of the MT inhibitory weights (k_inh) in the models (Fig. 5C–H). Full MT inhibitory weight strength (k_inh = 1.0) is defined for both models as shown in Table 1. For both the component (Fig. 5C) and pattern models (Fig. 5D), monocular PI tended to increase with both the strength of V1 opponency and MT inhibition. For the component model (Fig. 5C), the PI values ranged from strongly negative (consistent with a component cell) for no suppression to near 0 (consistent with an unclassified cell) for full monocular and dichoptic suppression. In contrast, the pattern model (Fig. 5D) generated PIs that ranged from near 0 with no suppression to large positive values associated with strong pattern cells for full monocular (V1) and dichoptic (MT-level) suppression. This extends the observation of Rust et al. (2006) that antipreferred direction suppression is key to generating responses to pattern motion, by showing that this suppression can be implemented both monocularly in V1 and binocularly at later stages. In contrast, with dichoptic plaid presentation, we found that in both component (Fig. 5E) and pattern (Fig. 5F) models, PI no longer varies with strength of V1 opponency (along the vertical axis) because this opponency is not engaged by the dichoptic stimulus.

Using these data, we plotted the change in PI from monocular to dichoptic presentation as a function of monocular and dichoptic suppression in both models (Fig. 5G,H). The parameter values used to generate the plots in Figure 5A, B are indicated by asterisks in Figure 5G, H. Both models showed significant regions of the parameter space that produced the decrease in PI observed by Tailby et al. (2010), whereas increases in PI with dichoptic presentation were negligible. Critically, in both models, there was no decrease in PI when V1 opponency strength was 0, indicating that V1 opponent suppression is a key factor in producing the loss of pattern sensitivity. For the component model (Fig. 5G), eliminating dichoptic MT inhibition resulted in a strongly component-like response and the largest drops in PI with the dichoptic stimulus. However, the strongest level of V1 opponency in the component model led to monocular PI values (Fig. 5C, top row) that were too high for component cells. Thus, for the representative component model (Fig. 5A), we chose a V1 opponency strength of 50%, where there is still a large drop in PI between dichoptic and monocular presentation, and the monocular PI = −2.9 is typical for component cells (Fig. 5G, asterisk). For the pattern model, the largest difference in monocular and dichoptic PI also occurred with 100% V1 opponency (Fig. 5H). PI values also decreased from monocular to dichoptic presentation across a large range of MT inhibitory weight strengths, but unlike the component model, the maximum decrease in PI occurred for non-0 inhibitory weights. Thus, for the representative pattern model (Fig. 5B), we chose a V1 opponency of 100% and MT inhibitory weight strength of 25% (Fig. 5H, asterisk). For both component and pattern cells, the full MT inhibitory weight strength produced the smallest drop in PI with dichoptic presentation.

These results demonstrate the critical role of V1 opponency in our models for explaining responses to dichoptic plaids because both pattern and component units showed no difference between monocular and dichoptic PIs when opponency strength was 0 (Fig. 5G,H). They also predict that there should be a difference in the strength of opponency in the V1 inputs to component and pattern cells: strongly component cells must have V1 inputs that show weaker opponent suppression because stronger suppression results in more pattern sensitivity (Fig. 5C). Furthermore, these results predict that MT pattern-DS cells with higher PIs will tend to have V1 inputs with stronger motion opponency.

In summary, we have applied insights from previous studies (Rust et al., 2006; Tailby et al., 2010) to build binocular component- and pattern-DS cell models that account for MT responses to monocular and dichoptic plaids observed in vivo. Within our model framework, we found that the presence of monocular V1 motion opponent suppression plays the key role in explaining the observations of Tailby et al. (2010), whereas MT inhibitory weights will promote pattern motion sensitivity when stimulated both monocularly and dichoptically. A summary of the values of c_opp and k_inh that we tested for the plots in Figure 5 can be found in Table 2. A strong prediction of our models is that V1 inputs to MT are moderately to strongly motion-opponent. It is important to note that a large range of the parameter space (V1 opponency strength and strength of MT inhibitory weights) is consistent with the results of Tailby et al. (2010). That their result held true for nearly all of the MT cells that they tested may reflect how widespread motion opponent suppression may be in the V1 inputs to MT. We consider independent evidence for V1 opponency in the Discussion.

Responses to dichoptic plaids in MT models with binocular V1 input

In the models considered above, binocular mixing occurs in MT and the relevant V1 to MT afferents are strictly monocular. This is the simplest circuitry for building binocular MT responses. However, current electrophysiological evidence, although limited, suggests that MT neurons receive input from V1 cells that are themselves primarily binocular, being well driven by each eye. In particular, Movshon and Newsome (1996) found that 11 of 12 neurons projecting from V1 to MT were in OD Groups 3–5 (range 1–7, where 4 has no obvious imbalance) (Hubel and Wiesel, 1962). It is therefore important to determine whether our results are robust across the range of possible ocular dominance strengths of V1 inputs. The models presented in Figures 3⇑–5 show results with monocular V1 inputs, and here we demonstrate these results hold for simple binocular motion integration in V1. In a later section, we also consider models where V1 binocular integration includes a disparity computation, which adds substantially more complexity to the circuitry.

We first implemented binocular integration at the V1 level in our model by mixing signals via a weighted sum within a single direction channel across the left and right streams (Fig. 1D). The balance of input from each eye is set by the parameter b; for 50% mixing (b = 0.5), both streams of V1 channels in the binocular model are equally well driven by left and right eye channels (i.e., all channels are OD 4, in which case having two sets of channels is redundant), whereas b = 0.7 indicates that the left V1 stream is 70% left eye and 30% right eye driven, and vice versa for the right stream. In Figures 3⇑–5, b was set to 1.0 for purely monocular V1 channels. We tested two possible sequences for binocular combination and opponency in the model (Fig. 6A): either binocular combination came first, the V1B model (data not shown in Fig. 1), or motion opponency came first, which we refer to as the V1OB model (as shown in Fig. 1C,D).

• Download figure
• Open in new tab
• Download powerpoint

Figure 6.

The pattern model with binocular V1 inputs. A, Diagram showing the two alternative models we tested that differ in the location of binocular integration: the V1B model where binocular integration occurs before motion opponency; and the V1OB model, where binocular integration occurs after monocular motion opponent suppression. B, Dependence of the monocular PI on amount of V1 opponency and MT inhibitory weight strength in the V1B model. C, Dependence of monocular PI on V1 opponency strength and MT inhibitory weight strength in the V1OB model. D, E, Dichoptic PI values as opponency and inhibitory strength vary in the V1B (D) and V1OB (E) models. F, G, The difference between dichoptic and monocular PI as V1 opponency and MT inhibitory strength are varied in the V1B (F) and V1OB (G) models.

We first tested the FP pattern cell model with 50/50 left/right eye (b = 0.5) binocular V1 inputs using the monocular and dichoptic plaid protocol of Tailby et al. (2010) and varied the strength of V1 opponency and dichoptic inhibition as we did for the model with monocular V1 channels (Fig. 5D,F,H). For monocular plaids, the change in PI as V1 opponency and MT inhibitory strength are varied in the V1B (Fig. 6B) and V1OB (Fig. 6C) models is very similar and also matches the results from the model without binocular V1 mixing (Fig. 5D). For dichoptic plaids, however, there is a stark difference between the two versions of the model. The V1B model (Fig. 6D) shows a similar dependence on the suppression parameters as seen for the monocular plaids (Fig. 6B), whereas the V1OB model (Fig. 6E) lacks the dependence of PI on V1 opponency seen in the monocular V1 model (Fig. 5F). Thus, the V1B model produces negligible changes, even slight increases, in PI with dichoptic presentation (Fig. 6F). Because binocular combination precedes opponency in the V1B model, the V1 opponent motion signal is now binocular, allowing the two grating components to suppress each other, even when they are presented to separate eyes (dichoptically), and pushing the PI toward pattern cell values. Thus, the monocular V1 opponency that is important to differentiating monocular and dichoptic PIs is absent, and the decrease in PI with dichoptic presentation cannot be achieved. In contrast, the V1OB model generates the decreases in PI with dichoptic stimulation that is stipulated by the neuronal data of Tailby et al. (2010) (Fig. 6G) and with similar dependencies as in the monocular V1 model (Fig. 5H). The V1OB model retains the same dependency on opponency strength as the original model with monocular V1 inputs because opponency still occurs at a stage with only monocular signals. Similarly, in the component cell model, we found that the PI value decreased for dichoptic plaids when placing binocular integration after opponency but not before (data not shown). Thus, the model strongly predicts that opponent suppression occurs before binocular integration in V1. Table 2 shows a summary of the key parameter values we tested in this section.

Tuning for FP and 3D motion in pattern and component units

We now examine whether our binocular pattern and component models, which were developed to explain MT pattern motion selectivity under monocular and dichoptic conditions, can be extended to account for the recently reported sensitivity of MT to 3D motion (Czuba et al., 2014; Sanada and DeAngelis, 2014). Both recent studies found evidence that IOVD cues are the primary drivers of 3D motion sensitivity in MT. We focus here on the results of Czuba et al. (2014), whose stimulus protocol consisted of dichoptic presentation of single sinusoidal gratings to both eyes with variations in drift direction and speed across trials. Specifically, the gratings either drifted in the same direction, which we refer to as B_Same, or in opposite directions, which we refer to as B_Opp. Czuba et al. (2014) also varied the speed independently in the two eyes in both of these conditions. The B_Same stimulus with equal speed in the two eyes corresponds to classical FP motion. The B_Opp stimulus with equal speed and opposite directions in the two eyes corresponds to 3D motion that is purely toward or away from the observer. When the speed varies across eyes, both the B_Same and B_Opp conditions simulate oblique motion that contains both FP and 3D motion components (Cynader and Regan, 1978).

Using this protocol, Czuba et al. (2014) found that some MT cells, which we will call FP cells, responded best to FP motion, whereas others, which we will call 3DT cells, preferred motion toward or away from the observer. The behavior of FP and 3DT cells is demonstrated with idealized tuning curves for monocular and binocular grating stimuli in Figure 7. In particular, the FP cells (exemplified by Czuba et al., 2014, their Fig. 2A–D) showed strong direction-tuning, quantified by a DSI, for the B_Same stimulus (Fig. 7C), with the same direction preference in each eye when tested monocularly (Fig. 7A,B). When tested with the B_Opp stimulus, the FP cell tuning curve had a low DSI, typically with two peaks (Fig. 7D) corresponding to when the stimulus component in one or the other eye matched the cell's preferred direction. Thus, the FP cell response had no bias for motion toward versus away from the observer. Our binocular model units characterized above have the same direction tuning in each eye; we thus refer to them as FP models. On the other hand, the 3DT cells of Czuba et al. (2014, exemplified by their Fig. 2E–H) preferred opposite directions of motion in each eye (Fig. 7E,F). These cells had a direction tuned response with low firing rate for the B_Same stimulus (Fig. 7G) but had direction tuning with a substantially higher firing rate for the B_Opp stimulus (Fig. 7H). These 3DT cells were thus associated with 3DT because they were strongly direction tuned when tested with the B_Opp protocol, as measured by the DSI being >0.5.

• Download figure
• Open in new tab
• Download powerpoint

Figure 7.

Typical responses for FP and 3DT MT cells from the study of Czuba et al. (2014). Typical plaid direction tuning curves as found by Czuba et al. (2014) using their dichoptic motion protocol with FP cells (left column) and 3DT cells. A, Left eye direction tuned response in the FP cell. B, Right eye direction tuned response in FP cell. Note the shared tuning with the left eye shown in A. C, Direction tuned response to B_Same in the FP cell. D, Orientation tuned response to B_Opp found in most FP cells. E, Left eye direction tuned response in the 3DT cell. F, Right eye direction tuned response in 3DT cell. Right eye tuning curve is shifted 180°, as they found for many 3DT cells. G, A direction tuned response to B_Same was still typical for 3DT cells despite the disparate monocular tuning. H, Strong direction tuned response to B_Opp characteristic of 3DT cells.

The discovery of 3D versus FP motion tuning in MT raises the question as to whether there is any relationship between 3D tuning and pattern motion sensitivity in MT. Czuba et al. (2014) did not test whether the cells they characterized with binocular motion stimuli were component or pattern-DS for classical monocular stimuli, and so we addressed this from a computational perspective by testing FP and 3D tuned versions of both our component and pattern models. We examined whether there were fundamental differences in the ability of the component and pattern models to generate the typical FP versus 3DT binocular motion tuning shown in Figure 7. As with the dichoptic plaid simulations, we began by testing models where the V1 channels were monocular (b = 1.0) as this is the simplest configuration of the models for gaining intuition on how interocular comparisons can give rise to 3D motion tuning. A full summary of the parameter values we varied for the simulations in this section can be found in Table 2.

Our FP component and pattern models (developed above) were tuned for the same preferred direction in the left and right eyes, as can be seen in their monocular single grating tuning curves (Fig. 8A,B and Fig. 8E,F for component and pattern cells, respectively). When tested with the B_Same stimulus, the FP models for both component (Fig. 8C) and pattern (Fig. 8G) cells responded strongly with a single peak in the direction tuning curve and produced DSIs of 0.8 and 0.7, respectively (see Materials and Methods; DSI defined as in Czuba et al., 2014). When tested with the B_Opp stimulus, the component cell produced an orientation-tuned curve with two lower amplitude peaks (Fig. 8D), corresponding to when the stimulus in one or the other eye moved in the preferred direction (DSI = 0.0). This matches the responses of many FP cells found experimentally (Czuba et al., 2014, their Fig. 2C,D), as idealized in our Figure 7A–D. In contrast, the FP pattern cell showed almost no tuning to the B_Opp stimulus (Fig. 8H, DSI = 0.0). Thus, our representative FP pattern unit, which was fit to the constraints of the dichoptic plaid protocol (Fig. 5B), was a poor fit for the typical response pattern of FP neurons in MT.

• Download figure
• Open in new tab
• Download powerpoint

Figure 8.

FP and 3DT models. Direction tuning curves obtained with the protocol of Czuba et al. (2014) for monocular gratings, B_Same and B_Opp. Only one of the speeds we tested is plotted for clarity. Direction varied is that shown to the left eye, with right eye direction either the same or shifted 180°. All tuning curves for a given model are normalized to the maximum firing rate obtained for that model across both the B_Same and B_Opp protocols. A–D, Direction tuning for the FP component cell. Left eye (A) and right eye (B) tuning is similar. C, D, Tuning for B_Same and B_Opp protocols, respectively. E–H, Direction tuning curves for the FP pattern cell. I–L, Direction tuning curves for the 3DT component cell. M–P, Direction tuning curves for the 3DT pattern cell. Q–T, Direction tuning for the 3DT component cell (red) and pattern cell (black) models with right eye MT weights at 50% the strength of the left eye weights (A_R = 0.5) and MT inhibitory weights at 100% (component; k_inh = 1.0) and 75% (pattern; k_inh = 0.75) strength.

We then considered what modifications to the canonical FP pattern model would satisfy both the dichoptic plaid and FP motion tuning constraints. The moderate-amplitude, flat tuning curve produced by the FP pattern model with the B_Opp stimulus depends on two factors: (1) the broad distribution of excitatory weights, characteristic of pattern cells (Simoncelli and Heeger, 1998; Rust et al., 2006), which drives a spiking response at all directions with the B_Opp stimulus; and (2) the weaker MT inhibitory weights required in this model for the dichoptic plaid constraint, which is insufficient to cancel the excitatory drive completely. We tested whether modifying these factors could achieve a pattern model that matched the FP cell data. Varying the second factor, inhibitory strength, alters the amplitude of the firing rate response as direction is varied but does not affect the shape of the tuning curve. Modifying the first factor by narrowing the distribution of MT weights could produce an orientation-tuned response to the B_Opp protocol (data not shown); however, this had the concurrent effect of lowering the PI produced by the model when tested with monocular plaids, making the resulting pattern unit more component-like.

To build 3DT models, we simply shifted the MT weight distribution in the right stream by 180° relative to that in the left stream (Fig. 1E, dotted lines in right stream). This resulted in a shift between the peaks for monocular left and right eye tuning curves (Fig. 8I,J,M,N). The direction tuning of the 3DT models for the dichoptic stimuli was reversed relative to that of their FP counterparts. Both component and pattern 3DT models were strongly direction tuned to the B_Opp stimulus (Fig. 8L,P) with DSI values of 0.8 and 0.7, respectively. The 3DT component unit showed two robust peaks in tuning for the B_Same stimulus (Fig. 8K, DSI = 0.0) where the stimulus in one or the other eye matched the unit's preferred direction for that eye. This deviated from the behavior of the recorded 3DT cells, which were typically direction tuned for the B_Same stimulus, despite being driven by both eyes about equally well when tested monocularly (Czuba et al., 2014, their Fig. 2E–G), as idealized in our Figure 7E–G. The 3DT pattern model also differs from the typical 3DT cell of Czuba et al. (2014). In particular, the model is not direction tuned for the B_Same stimulus (Fig. 8O, DSI = 0.1), whereas the typical 3DT cell shows clear tuning (Fig. 7G), with the preferred direction matching the left eye in this example.

The direction tuning shown by the 3DT cells for both dichoptic stimuli in the protocol of Czuba et al. (2014), despite each eye differing in preferred direction, suggests an ocular imbalance. Consistent with this, Czuba et al. (2014) found that their 3DT cells were more ocularly imbalanced than their FP population. Thus, we introduced a left-right stream imbalance such that the MT weights from the right stream were weaker than those from the left stream, allowing the left-stream tuning preference to dominate when both eyes were stimulated. We also increased the strength of MT inhibitory weights, to reduce the amplitude of the untuned response seen with the B_Same stimulus and thus increase the DSI. Using a fixed scaling factor, A_R, (Fig. 1F), of 0.5 applied to all weights in the right stream and increasing the scaling factor on MT inhibitory weights, k_inh, from 0.25 to 0.75, we achieved a DS tuning curve for the B_Same stimulus (Fig. 8S, black line, DSI = 0.7) while maintaining DS tuning for the B_Opp stimulus (Fig. 8T, DSI = 0.7), thus obtaining a better match to the observed tuning of typical 3DT cells.

We then tested the component 3DT model with an ocular imbalance, also increasing MT inhibitory weights to full strength (k_inh = 1.0), to determine whether we could achieve a DS tuning curve to the B_Same stimulus. This version of the 3DT component model still produced a double-peaked response, although the peak corresponding to the weaker eye had lower amplitude (Fig. 8S, red line, DSI = 0.5). This component model still lacks the necessary strength of antipreferred direction MT inhibition required to completely nullify the excitation generated by the weaker eye with the B_Same stimulus, so the resulting tuning curve has two peaks. Increasing the MT inhibitory weights in the model to values that are above those found by Rust et al. (2006) for component cells could increase the DSI for the B_Same stimulus even further. However, we have shown that, even within the range we tested, as inhibitory strength increases, the model produces PI values for plaids that are too high to be classified as a component unit (see Fig. 5C).

Having established the factors that allow 3D motion tuning in the models, we explored what range of these parameters could support DS tuning for the B_Same stimulus in the 3DT models. We varied the level of MT ocular imbalance, A_R, and MT inhibitory weight strength, k_inh, in the component (Fig. 9A) and pattern (Fig. 9B) 3DT models and plotted the resulting DSI values. Figure 8Q–T (asterisks) indicates the parameter values used for the tuning curves. The highest level of MT imbalance we tested, A_R = 0.5, results in monocularity indices of 0.26 and 0.23 in the component and pattern models, respectively, both close to the median reported for 3DT cells in Czuba et al. (2014) (0.24). In the component model (Fig. 9A), only an MT weight imbalance with the right stream at 50% of left stream strength and full-strength MT inhibitory weights produced a DSI of 0.5. With the pattern model (Fig. 9B), a much larger range of the parameter space produced DS responses, and with DSI values >0.7. The highest DSI values occur with the strongest MT inhibitory weights. With strong dichoptic inhibitory inputs, the inhibition generated by one stream, in response to an antipreferred stimulus, can suppress the excitation when the other stream is presented as a preferred stimulus. With the ocular imbalance, there will be responses to the dominant eye stream's preferred stimulus but not the weaker stream, resulting in a DS tuning curve and a high DSI. The component model, lacking strong dichoptic inhibition, will always generate tuning curves with two peaks, corresponding to each eye's preferred direction, and produce lower DSI values. Although 75% of the cells in Czuba et al. (2014) had a DSI >0.5 for the B_Same stimulus, not all 3D tuned cells did (their Fig. 3B). Thus, some subset of 3D tuned units may be consistent with our component model. Alternatively, if there are many 3DT component cells with a DSI > 0.5 for the B_Same protocol, this implies that there is an additional source of dichoptic, opponent inhibition in MT component cells that is not captured in the model fits to the monocular stimulus protocols.

• Download figure
• Open in new tab
• Download powerpoint

Figure 9.

Direction selectivity indices for binocular motion in the 3DT models. A, Plot of DSI for B_Same as MT inhibitory weight strength and MT ocular imbalance level are varied in the 3DT component model. Blue regions represent DSI values <0.5. White areas represent DSI = 0.5. *Parameter values used for the component model tuning curves plotted in Figure 8Q–T. B, DSI for binocular matched motion in the pattern model as inhibitory strength and MT ocular imbalance are varied. Red regions represent DSI values >0.5. *Parameters chosen to generate the pattern model tuning curves in Figure 8Q–T. C, Plot showing how DSI for the B_Opp stimulus varies as MT inhibitory strength and V1 ocular dominance are varied in the 3DT V1OB model. D, Plot of DSI obtained with the B_Same stimulus as MT inhibition strength and MT ocular imbalance level are varied in the 3DT V1OB model.

Having achieved 3D motion tuning in the model with monocular V1 inputs, we then tested whether this tuning was robust to varying V1 ocular dominance. We built a 3DT pattern model by shifting the MT weight distributions between left and right eye streams (Fig. 1E), with V1 motion opponency preceding binocular integration as in the FP version of the V1OB model (Fig. 6A). We use the V1OB model circuitry here, with opponency preceding V1 binocular integration, as we have shown that this ordering of the V1 stages is needed to explain dichoptic plaid results (Fig. 6). The strength and relative ordering of V1 opponent suppression do not affect 3DT model output using the protocol of Czuba et al. (2014; data not shown) because only one motion component is shown to each eye for the dichoptic Czuba et al. (2014) stimuli; thus, the V1OB and V1B configurations of the model (Fig. 6A) give similar results.

In the 3DT model with monocular V1 inputs (b = 1.0), 3D motion tuning is achieved by computing sums and differences of V1 signals across the two eye streams. If the V1 channels have perfect binocular balance, this computation would no longer be possible because it requires the comparison of monocular motion signals, and in the case with exactly balanced left and right eye V1 inputs, the “left” and “right” eye streams are identically driven. We examined this by varying the V1 binocular balance and MT inhibitory weight strength in the 3DT V1OB pattern model and measuring the resulting DSI values for the B_Opp stimulus (Fig. 9C). We found that the 3DT model with 50/50 V1 binocular mixing produced flat direction tuning curves, resulting in uniformly low DSIs, with the B_Opp protocol of Czuba et al. (2014) for all MT inhibitory strengths (Fig. 9C, b = 0.5). As we increased the ocular imbalance in the V1 channels (increasing b), strong direction tuning could be produced in the V1OB model with appropriate MT inhibitory strengths, indicating that a sufficient IOVD signal remained to generate 3DT. Setting the V1 binocular mixing to 70% dominant eye/30% nondominant eye (b = 0.7), we then tested how MT inhibitory weight strength and the MT level ocular imbalance, A_R, affected DSI values for the B_Same protocol (Fig. 9D), as we had done for the 3DT pattern model with monocular V1 inputs (Fig. 9B). We found a very similar dependence of direction tuning strength on these parameters in the model with binocular V1 inputs. Thus, the results we described for our model with monocular V1 channels can be reproduced with binocular V1 inputs, as long as a V1-level ocular imbalance is included.

In summary, our models predict that tuning for FP and 3D motion in MT is coincident with the spectrum of component: pattern motion sensitivity. The binocular FP component model that we built according to the constraints of Tailby et al. (2010) fits the response pattern of the FP neurons described by Czuba et al. (2014) with no modifications. The best fit to the data for the FP pattern model involves a modification that decreases the pattern sensitivity of the model. Conversely, the responses of 3DT neurons are best fit by a pattern model with an ocular imbalance, whereas the component model is unable to fit the data without the addition of dichoptic inhibition that increases the pattern sensitivity of the component model. This is a striking and untested prediction of our binocular models: that 3DT cells in MT may be associated with pattern motion sensitivity, whereas FP motion tuned cells will be associated with component selectivity.

Dichoptic plaids with 3DT cells

We have presented models of MT cells that are able to reproduce the results of two key studies on dichoptic motion processing in MT. Another test we performed was to examine the tuning of the 3DT model to the dichoptic plaid protocol of Tailby et al. (2010), which we used to characterize the FP models in Figures 5 and 6. This is interesting to consider because Tailby et al. (2010) may have encountered such cells during their study, and exploring the stimulus space more thoroughly may reveal predictions that can be used to validate or dismiss the models.

Figure 10A shows tuning curves for monocular (blue trace) and dichoptic plaids with 120° (red) and −120° (black) differences in component grating direction. With the dichoptic plaids, the peak direction in the 3DT pattern unit shifts by 90°. The direction of this peak shift reverses when the component gratings are swapped between the two eyes. We found that the magnitude of the shift was determined by the interocular difference in direction preference. Figure 10B shows the effect of shifting the weight distribution in the right eye, with successively lighter traces for each 30° shift in the preferred direction of the right eye. The tuning curve shifts 15° for each 30° shift in interocular direction preference. The strength of the MT inhibitory weights had no effect on the magnitude of the shift: even in the absence of inhibitory weights, the peak was still shifted 90° (data not shown). We found identical results for the 3DT component unit as well (data not shown). Thus, our model predicts that 3DT cells with 180° differences in preferred direction between the two eyes will show a shift in preferred direction of 90° when tested with dichoptic plaids.

• Download figure
• Open in new tab
• Download powerpoint

Figure 10.

Dichoptic plaids in the 3DT pattern model. For all panels, response plotted is the MT output signal after the rectification nonlinearity is applied. A, Direction tuning for monocular (blue trace) and −120° (black trace) and 120° (red trace) dichoptic plaids in the 3DT pattern model. B, Dichoptic plaid direction tuning in the 3DT pattern cell as the shift in the weight distribution to the right eye is varied from 180° (black trace) to 90° (lightest gray). C, Matrix of responses to dichoptic plaids in all direction combinations (as in Fig. 3L) for the 3DT pattern model. Red lines indicate the stimulus space sampled by the protocol of Tailby et al. (2010). Green lines indicate the B_Opp protocol of Czuba et al. (2014). D, Response matrix to all monocularly (left eye) presented plaids in the 3DT pattern cell model. E, Response matrix to all dichoptic plaids from the FP pattern cell model for comparison. F, Response matrix for a monocular version of the pattern cell model with no V1 opponency or reduced inhibitory weights (as in Rust et al., 2006).

To fully explore the response patterns of the binocular models to all plaids, we characterized the representative FP and 3DT pattern units (Figs. 5B, 8Q–T) using the protocol of Rust et al. (2006) with plaids of all direction combinations in 30° steps. We tested both dichoptic (Fig. 10C,E) and monocular stimulation (Fig. 10D). Figure 10C (red line) shows the slice through stimulus space of the plaids that form the dichoptic 120° plaid protocol of Tailby et al. (2010), whereas the green line shows the dichoptic opponent motion “plaids” that make up the B_Opp protocol of Czuba et al. (2014). The activated region of the 3DT model's plot (Fig. 10C) is shifted by 180° in the full stimulus space along the axis of grating direction in the right eye compared with the FP model's plot (Fig. 10E). This reflects the 180° shift in preferred direction of the right eye in the 3DT model. The preferred stimulus of the 3DT pattern cell was a “counterphase” dichoptic plaid (components at 0° and 180°), matching the B_Opp stimulus of Czuba et al. (2014) (Fig. 10C). The binocular FP pattern unit showed a preference for stimuli centered around 180° motion in each eye when tested dichoptically (Fig. 9E), as expected by construction of its tuning curves, but without the trough in the plaid matrix seen when stimulating the binocular model monocularly (Fig. 10D). This is because of the loss of monocularly driven tuned normalization when the plaid is presented dichoptically. When tested monocularly (Fig. 10D), the 3DT unit showed a similar preference as a purely monocular version of the model (Fig. 10F), but the region of activation was elongated in the stimulus space because of the reduced inhibition in the binocular model.

It is unclear whether the experiments of Tailby et al. (2010) would have uncovered this response pattern for 3DT units. They did report that most of their cells had very similar direction tuning in both eyes, which is consistent with the report by Czuba et al. (2014) that a small proportion of MT cells had monocularly opposed direction tuning (their Fig. 5). Tailby et al. (2010) did not report the preferred directions measured with monocular and dichoptic plaids, so shifts in the tuning curve peaks may have been overlooked. Given that the PI is the difference between the pattern and component partial correlation coefficients, a shift in preferred direction with dichoptic presentation should result in a reduction in PI. This follows because the resulting tuning curve for the dichoptic plaid should be poorly correlated with the tuning curve prediction from the monocular gratings, which would have a different preferred direction. This is what we found for our pattern 3DT unit: the PI values were 3.1 and −1.3 for monocular and dichoptic plaids, respectively, with the pattern correlation dropping from 4.3 to −0.7 and component correlation also decreasing from 1.2 to 0.6.

In summary, it is possible to reconcile the results of Tailby et al. (2010) and Czuba et al. (2014) for 3DT neurons consistent with our model. A full stimulus protocol that characterizes tuning for plaids both monocularly and dichoptically would expose the shift in tuning predicted by our models and reveal any connection between pattern motion selectivity and 3D motion sensitivity, as predicted above.

3D motion biased tuning in MT cells

The FP and 3DT cells described by Czuba et al. (2014) are two ends of a spectrum of sensitivity to 3D motion observed in their MT population. They described most of their cells as 3D biased, indicating that these cells did not show strong direction selectivity (as defined by a DSI > 0.5) for the B_Opp protocol but did have a statistically significant preference for receding or approaching directions of motion for oblique 3D directions. Figure 11A (red points on the circle) shows the oblique 3D directions generated when speed (1, 2, and 7.5 deg/s) and direction (0° or 180°) are varied between the two eyes, as in Czuba et al. (2014) (see Materials and Methods). Points in the top half of the plot correspond to motion away from the observer, whereas those in the bottom half correspond to motion toward the observer. Cells that are 3D biased responded more strongly overall to the set of stimuli with 3D directions that fall in one-half of the stimulus space (receding or approaching) than to those that fall in the other half, as measured by a matched Wilcoxon signed rank test (matched stimuli are shown connected by dashed red lines in Fig. 11A). Czuba et al. (2014) also found that 3D biased cells tended to have similar direction tuning in both eyes, similar to the FP cells, and >50% of 3D biased cells had a monocularity index < 0.125, indicating they tended to be ocularly balanced.

• Download figure
• Open in new tab
• Download powerpoint

Figure 11.

A model for a 3D motion biased cell using an MT ocular imbalance. A, Diagram showing the 3D motion direction space generated by varying interocular stimulus direction and speed (Cynader and Regan, 1978; Czuba et al., 2014). Position of the eyes is indicated by icons at 225° and 315°. The speeds we used to generate the 3D motion directions were 1, 2, and 7.5 deg/s. All speeds drove the MT neuron well monocularly. Red dots around the circumference of the circle represent the 3D directions sampled by our protocol. Dashed red lines connect the stimuli, which were matched in the statistical test for 3D bias as used by Czuba et al. (2014). Blue shaded half-circle represents the stimulus space where the left eye in our model FP component cell sees its preferred direction of motion. Yellow shaded half-circle represents the same for the right eye. B, 3D direction tuning for the FP component cell with an ocular imbalance such that the right eye weights are 70% the strength of the left eye weights. From 0° to 180° are all receding motions, whereas from 180° to 360° are approaching motions.

Czuba et al. (2014) and others have noted that 3D biased responses may arise from differences in speed tuning between the two eyes, but here we explore an alternative circuitry for generating a 3D biased tuning curve. We exploited a difference in the monocular distribution of motions generating the 3D directions to build a model unit that would be classified as 3D motion biased. Consider an FP unit that prefers leftward motion. The left eye for this unit would see its preferred direction motion in Figure 11A (semicircle shaded in blue), whereas the right eye would see its preferred direction in the yellow semicircle. Noting that the top quadrant (blue region) corresponds to receding motion where only the left eye sees its preferred direction while the bottom quadrant (yellow region) corresponds to approaching motion where only the right eye sees its preferred direction, we reasoned that an FP unit with both eyes tuned to leftward motion and an ocular imbalance favoring input from the left eye should show a tuning bias for motion directed away from the observer.

Figure 11B shows a direction tuning curve for such an FP component unit stimulated with the oblique directions represented by Figure 11A (red points). The 3D direction tuning curve is skewed toward receding motions (3D directions between 0° and 180°). This result is highly statistically significant (p < 0.002; Wilcoxon signed rank test). The unit is tuned to 170°, slightly away from pure FP leftward motion. In addition, even though an ocular imbalance in the component unit was necessary to achieve these results (A_R = 0.7), the monocularity index value for this cell is 0.12, falling within the range of ocular imbalances that Czuba et al. (2014) found for their 3D biased cells. The DSI for this unit tested with the B_Opp protocol is 0.1; thus, this unit is 3D biased, not 3D tuned, by the criteria of Czuba et al. (2014).

Importantly, 3D biased cells constructed in this way must conform to the following pattern. If an MT unit is left eye-dominant, it will be biased for receding motion if both eyes prefer leftward FP motion and biased for approaching motion if both eyes are tuned for rightward motion, and vice versa if the right eye is the dominant eye. This is a specific and readily testable prediction of this model for producing MT units with 3D motion biased responses.

MT models with motion and binocular disparity selectivity

We have presented the simplest models that could capture observations from recent binocular motion studies, but a complete binocular model of MT requires disparity tuning, which is expressed by the majority of MT cells (Maunsell and Van Essen, 1983; DeAngelis and Uka, 2003). Because V1 inputs to MT are thought to be primarily binocular (Movshon and Newsome, 1996), and disparity is computed when binocular integration occurs, it is reasonable and parsimonious to build a model where the V1 channels are jointly motion and disparity tuned, conferring both motion and disparity selectivity to the model MT unit. This leaves the key question of whether our results from the motion-only models still hold when we incorporate the constraints on the circuitry necessary for generating jointly tuned responses.

The jointly tuned model we built incorporates cascaded computations for both motion and binocular disparity energy to create complex (i.e., phase-invariant) DS and disparity-tuned V1 channels. The two main changes to the framework for the motion models shown in Figure 1 were as follows: (1) the initial motion energy stage (Fig. 1A) was opened up to allow the even and odd linear filter channels to remain separate through the normalization and opponency stages; and (2) the opponency and V1 integration stages (Fig. 1C,D) were replaced by the circuitry shown in Figure 2, which implements the binocular disparity energy model (Ohzawa et al., 1990). Because the models require monocular opponency with subsequent rectification preceding binocular integration, the disparity circuitry has a form that is very similar to that proposed by Read et al. (2002) for tuned-excitatory cells. All the V1 channels share the same disparity preference, in this case for 0 disparity (tuned excitatory). These jointly tuned V1 inputs are then subject to the same MT weights and output nonlinearity as used in the motion-only models to produce spiking MT responses. We tested our jointly tuned MT units, built with the same parameters as our canonical FP models (Fig. 5A,B), with correlated and anticorrelated dynamic random dot stimuli used to characterize binocular disparity tuning. Both the component (Fig. 12A) and pattern (Fig. 12B) models showed strong tuning for disparity, with the tuning curve for anticorrelated dots inverted and reduced in amplitude as is typical of V1 and MT disparity-tuned cells (Cumming and Parker, 1997; Krug et al., 2004). This attenuation is not seen in the classical disparity energy model but has been shown to arise when the subunits of the disparity energy computation are rectified (Read et al., 2002), as occurs in our model at the opponency stage (Fig. 2B), which includes rectification (see Materials and Methods, Eq. 10).

• Download figure
• Open in new tab
• Download powerpoint

Figure 12.

MT models with binocular disparity tuning. A, Plots of disparity tuning in the FP component model with binocular disparity for correlated (blue trace), anticorrelated (red trace), and uncorrelated (black trace) dynamic random dot stimuli. B, Disparity tuning in the FP pattern model with binocular disparity with dynamic random dots as in A. C, D, Dependence of monocular PI on V1 opponency strength and MT inhibitory weight strength in the FP component (C) and pattern (D) models with disparity. V1 opponency precedes disparity computation in these models (see Materials and Methods). E, F, Dichoptic PI values as opponency and inhibitory strength vary in the component (E) and pattern (F) models. G, H, The difference between dichoptic and monocular PI as V1 opponency and MT inhibitory strength are varied in the component (G) and pattern (H) models. I, Plot showing how DSI for the B_Opp stimulus varies as MT inhibitory strength and V1 ocular dominance are varied in the 3DT pattern model with disparity selectivity. J, Plot of DSI obtained with the B_Same stimulus as MT inhibition strength and MT ocular imbalance level are varied in the 3DT pattern model.

In the jointly tuned models, we repeated our simulations testing the dependence of pattern motion sensitivity on monocular and dichoptic opponent suppression, as presented above for the motion-only models (Figs. 5C–H, 6C,E,G). Both the component (Fig. 12C,E,G) and pattern (Fig. 12D,F,H) models with disparity tuning share similar trends with the nondisparity models, most critically the requirement for monocular opponency to see the drop in PI observed with dichoptic presentation (Fig. 12G,H). The main difference is that the monocular V1 opponency has an even stronger effect on PI values than in the motion models, leading to large increases in PI in both the component (Fig. 12C) and pattern (Fig. 12D) models for any value of opponency >0 compared with monocular plaids in the motion-only models. This reflects the fact that the rectification, which occurs after opponency, precedes the squaring in the disparity energy computation, thus resulting in a stronger suppressive effect.

We then tested a 3DT version of the disparity-tuned pattern model with the dichoptic protocols of Czuba et al. (2014), comparing the dependence of DSI on MT inhibitory weight strength and V1 (Fig. 12I) or MT (Fig. 12J) level ocular imbalances, as we did for the motion-only model with binocular V1 inputs (Fig. 9C and Fig. 9D, respectively). The results are very similar between the motion-only and joint motion-disparity tuned models, showing that our results are robust and apply to circuits having a wide variety of mechanisms of binocular integration at the V1 level, including no integration, simple additive combination, or plausible disparity energy computations.