Pattern Motion Direction Is Encoded in the Population Activity of Macaque Area MT

Tuning curves for five cells representative of the diversity of responses in our dataset. Normalized average spike counts in a time window from 50 to 550 ms after stimulus onset are computed for each stimulus and direction of motion. The tuning curves are rotated so that, for all cells, 0° on the abscissa is associated with the preferred direction of the cell. Responses to single components (1-D or 2-D noise), Type I, unikinetic, and flicker plaids are shown in each column. Dashed line indicates the spontaneous firing rate of the cell. In the two top rows, we show examples of component cells: one for which the stimuli were moving at an appropriate speed and one for which the stimulus was too fast (note bimodal tuning curve to 2-D noise). The bottom two rows show examples of cells that could be classified as pattern cells. The speed of the stimulus was appropriate for one, but too slow for the other (note bimodal tuning curve to 1-D noise). The mid row shows an intermediate cell, which cannot be unambiguously classified as either component or pattern.

The bPI, based on responses to Type I plaids, and the uPI, based on responses to unikinetic plaids, can be used to infer to what extent a cell behaves as a pattern or component cell. Note the wide scatter along both axes, indicative of the large diversity in the population. The two indices are, however, significantly correlated (green line, r = 0.61). A gPI can be computed by projecting the data points on the main diagonal (i.e., computing the average of bPI and uPI), and cells can then be classified as component (blue), mixed (gray), or pattern (orange) depending on their gPI value. Asterisk indicates the values associated with the pattern population response (see Fig. 6).

The fPI, analogous to the uPI, can be used to assess how cells respond to flicker plaids. Left, The indices are computed based on the spike count during the entire stimulus presentation. Right, Only the initial 50 ms of the response is considered. Cells are color-coded as in Figure 3 (i.e., according to their gPI value). The correlation between fPI and uPI (green lines) is considerably stronger early on (r = 0.341 vs r = 0.239), suggesting that a pattern signal is extracted from the flicker plaid initially, but it wanes with time. Asterisks indicate the values associated with the pattern population response (Fig. 6).

Population activity for single-component stimuli. We separated our 386 neurons into three groups (component, mixed, and pattern) according to their gPI value. Tuning curves for each neuron were rotated to align the preferred direction of each neuron with rightward motion (0°). A continuous firing rate signal was obtained by convolving each spike with a Gaussian. The square root-transform was then applied to the individual firing rates, before averaging and squaring the result. Icons are used to indicate the configuration of the stimuli at directions of interest. Black number in each panel indicates the firing rate value associated with the deepest red in the panel (because of the great difference in maximum value across panels, it was not possible to use the same color scale for all panels). Responses to 2-D noise are most consistent across groups, whereas those to 1-D noise stimuli (drifting, static, and flickering) exhibit considerable differences, and a drastic reduction in strength in going from component to pattern cells. The direction of motion (blue arrows) or orientation (blue lines) for particularly relevant stimulus directions is schematically indicated to highlight which stimuli give rise to which responses.

Same as in Figure 5, but for plaid stimuli. There are large differences across groups for all plaids, affecting more the distribution of activity across directions than the overall strength of the response. Component cells mostly signal the direction of motion of the drifting component(s), whereas pattern cells mostly signal the direction of rigid translation of the plaid. There are, however, also significant changes over time, especially for flicker plaids. The direction of motion (blue arrows) or orientation (blue lines) of the components making up each plaid for particularly relevant stimulus directions is schematically indicated.

Time slices through the population response to plaids of the pattern group (see Fig. 6, right column). The two unikinetic and flicker plaids were combined and averaged. The population response peaks at the pattern motion direction (dashed vertical lines) throughout the duration of stimulation for Type I and unikinetic plaids. For flicker plaids, responses in this direction are initially strongest, but the signal quickly becomes ambiguous.

Linear readout on individual pseudo-trials from our population of 140 pattern neurons (as identified by having gPI > 0.25); 360 readout neurons are simulated, each sampling differently from the population (but in a principled and predetermined manner, not learned from or fitted to the data), and the readout neuron that, at any given time, responds most strongly to a simulated trial is selected as the winner. For each stimulus type and direction of motion, we simulated 1000 trials (96,000 trials total), and plot here the mean (solid line) and 68% CI (shade) of the deviation from the actual direction of motion associated with that stimulus.

Decoding performance as a function of the gPI cutoff value for including cells in the population. Each row refers to a different stimulus; mean decoded direction, as a function of the gPI cutoff, is plotted in the left column (for two decoding periods: one early and one late); SD of the decoded direction is plotted in the right column. Dashed black line indicates the expected output of a perfect decoder of pattern motion direction. Dashed red line indicates the gPI cutoff value used in Figure 8.

A, There is a significant negative relationship between the strength of the response to flicker plaids (which is inversely related to the strength of opponency) and the gPI, indicating that pattern cells are characterized by stronger opponency than component cells. The strength of the response to flicker plaids was normalized by dividing it by the response to 1-D noise, and the base 2 logarithm was then computed to convert the ratio into octaves. Values were clipped in the ±4 range to prevent the correlation from being dominated by outliers. B, There is a significant positive relationship between the strength of the response to UP₉₀ plaids (obtained by adding to a drifting 1-D pattern an orthogonal static 1-D pattern) and the gPI, indicating that pattern cells are characterized by stronger enhancement from orthogonal static stimuli than component cells. The strength of the response to UP₉₀ plaids was normalized by dividing it by the response to 1-D noise, and the base 2 logarithm was then computed to convert the ratio into octaves. Values were clipped in the ±4 range to prevent the correlation from being dominated by outliers.

In pattern cells, adding a static 1-D pattern to a drifting one causes the direction tuning curve to rotate (despite the static pattern barely activating the cell in isolation). Top row, Mean direction tuning curve to 1-D noise (black), UP₄₅ plaids (cyan), and UP_-45 plaids (green) of our population of pattern cells (thick lines: 48 cells with unimodal response to 1-D stimuli; thin lines: all 140 pattern cells). The tuning curves of each pattern cell were first rotated to align the cell's preferred direction with 0°. Bottom row, Difference between the two curves, highlighting the enhancement (positive values) and suppression (negative values) that are associated with the addition of the static pattern. Maximum enhancement is seen when the static component is orthogonal to the preferred orientation of the cell, whereas maximum suppression arises when the static component is parallel to the preferred orientation of the cell. The static components are only effective if the cell is activated by the drifting component, suggesting nonlinear interactions.