Diversity of Ganglion Cell Responses to Saccade-Like Image Shifts in the Primate Retina

Stimulus and analysis

Saccades form rapid transitions between fixated images, and elicited responses in neurons of the visual system may be influenced by the presaccadic image, the postsaccadic image, and the transition in between. To investigate the coding properties of ganglion cells in the primate retina under saccade-like image transitions, we recorded ganglion cell spiking activity in isolated marmoset retinas with multielectrode arrays while projecting a saccade-like stimulus onto the photoreceptors. To systematically probe transitions between different illumination patterns inside the receptive fields of different ganglion cells, we chose a square-wave luminance grating with a bar width of 90 µm as the spatial layout of the stimulus. Taking into account the size of the marmoset eye (Troilo et al., 1993), this corresponds to ∼0.9° visual angle or, for example, a 10-cm-thick tree branch at a distance of ∼6 m.

To mimic the sequential order of fixations and saccades, the grating remained stationary for a fixation period of 533 ms at one of four equally spaced positions, which we call Position 1-4, before being shifted rapidly within 67 ms to a new position to start the next cycle of fixation and transition (Fig. 1A). In half of the transitions, the shift itself was masked by a gray screen at the mean light intensity of the grating to probe for effects of visual stimulation during the transition. The order of fixation positions and the occurrence of the gray-screen mask were randomized. Altogether, there were 16 possible combinations of grating positions before (“starting position”) and after (“target position”) a transition, and each combination was presented with transitions by grating motion as well as by homogeneous illumination.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Sample ganglion cell responses to saccade-like grating shifts. A, Schematic representation of the stimulus, consisting of a sequence of 533 ms fixations of a square-wave grating at one of four possible positions (spatial phases) and brief transitions of 67 ms. The transition either occurred via a rapid motion of the grating or via a gray-screen mask at mean luminance. The sequence of fixation positions and the type of transition were randomized. B, Responses of three sample ganglion cells to different combinations of grating position before and after the transition. Top row, Raster plots for both gray (red) and motion (blue) transitions. Bottom row, Corresponding PSTHs. Shaded areas mark the transition periods. C, PSTHs of the sample cells for all 16 possible combinations of grating positions before the transition (starting position) and after the transition (target position). Here, responses to gray-screen and motion transitions were pooled. D, Similarity of responses to gray and motion transitions for all cells included in the analysis. The similarity measure Rnorm2 was calculated as the modified coefficient of determination between responses to gray and to motion, normalized by the corresponding coefficient for odd versus even trials (independent of the transition type). Nearly all Rnorm2 values are >0.5, indicating high similarity between the responses to gray and motion transition.

We observed that ganglion cell responses could depend on the grating position both before and after the transition. Some of the different response patterns and their dependencies on starting and target positions are exemplified by the three sample cells displayed in Figure 1B. Cell 1 responded strongly after transitions from Position 3 to Position 2 as well as from Position 1 to Position 2, but not for transitions from Position 3 to Position 4, suggesting a preference for a specific target position, namely here Position 2. When comparing firing rate profiles for transitions via motion and via a gray screen, on the other hand, nearly identical responses were found for each individual combination of starting and target image. This was also the case for the two other sample cells of Figure 1 as well as for the entire population of ganglion cells, except for a small latency effect in some cells (e.g., Cells 2 and 3).

For a quantitative comparison of the responses under transitions via motion and via a gray screen, we computed a normalized coefficient of determination R² between the two responses (Fig. 1D). Most values lay near unity, corresponding to nearly identical response profiles under the two conditions, and values smaller than unity could generally be traced back to relative temporal shifts of the responses. These small differences in response kinetics, however, did not play a role for our analysis of response strength and sensitivity to the presaccadic and postsaccadic images. We therefore generally pooled responses from motion and gray-screen transitions for further analyses, but also confirmed that the analysis of response types held independently for each of the two transition types (see below).

To visualize the response characteristics more systematically, we computed the PSTHs for all 16 combinations of starting and target positions and displayed them in a matrix-like fashion (Fig. 1C). In this depiction, it becomes immediately apparent that Cell 1 responded strongly when the target grating position was Position 2, but not for other targets, and that the response was only slightly modulated by the starting position. Thus, this cell is sensitive mostly to the postsaccadic image, in this particular case the image that corresponds to the grating at Position 2. Note that the relevant observation here is that there is a preferred target position, but not which particular position it is. For the same cell but with a translated receptive field position or for phase-shifted versions of the displayed gratings, the preferred target position might be different, but responses should still primarily depend on which target position was reached.

Other ganglion cells, like Cell 2, could display two distinct response peaks, one during the transition itself and one after the onset of the new fixation. For this cell, both peaks occurred when the grating switched from Position 3 to Position 1, but other sample transitions with a different starting or target position elicited only one or the other (Fig. 1B, center). The matrix-like display of all 16 firing rate profiles (Fig. 1C, center) reveals that the first peak was sensitive to the starting position, occurring systematically for Positions 2 and 3, whereas the second peak depended on the target position and was elicited by Positions 1 and 4. Note that because of the cyclical nature of the grating, Positions 1 and 4 are neighboring, just like Positions 2 and 3 are.

For some ganglion cells, responses to the saccadic stimulus depended more intricately on the combination of presaccadic and postsaccadic images. Cell 3, for example, exhibited increased activity when the grating changed from Position 4 to Position 2, but neither the starting nor the target position alone were sufficient to evoke a strong response if there was no change in the grating position across the transition (Fig. 1B, right). Indeed, none of the starting or target positions by itself were sufficient to evoke a response of Cell 3, because starting and target position had to differ to trigger the cell (Fig. 1C, right). Thus, this cell was sensitive to a change in the grating position, but invariant to the specific starting and target positions of the transition.

To systematically compare these response patterns for different types of ganglion cells, we sought a reduced, quantitative description that still captured the dependencies of the responses on the starting position, on the target position, and on whether there was a change of the position. To take the distinct early and late responses of some cells into account (Fig. 1, Cell 2), we analyzed the peak firing rates in two response windows, the first from 30 ms after transition onset to 10–30 ms (depending on the experiment) after the onset of the new fixation, and the second from 10 to 30 ms until 200 ms after the onset of the new fixation (Fig. 2A).

• Download figure
• Open in new tab
• Download powerpoint

Figure 2.

Analysis of ganglion cell sensitivity. A, PSTH of a sample cell (Fig. 1, Cell 2) for the transition between Position 3 and Position 1, smoothed by a Gaussian filter. (In the quantitative analysis, the two response windows were smoothed separately.) Dashed vertical lines mark the boundaries of the first and second response windows. For each response window, the peak response is identified as depicted by the red crosses and arrows. B, The response matrices contain the peak response during the first window (left) and second window (right) for all combinations of starting and target position, depicted here in a color-coded fashion with brighter colors denoting stronger responses. The dashed squares indicate the entries that correspond to the sample PSTH shown in A. C, Fourier transformations of the response matrices. The entries of the transformed matrices quantify patterns in the response matrices. Entries that are highlighted by dashed squares correspond to relevant patterns in the response matrix, which are depicted schematically below the Fourier transformed matrices. For the sample cell, the yellow (0, 1) entry of the first and the yellow (1, 0) entry of the second Fourier transformed response matrix reflect the start and target sensitivity of the cell. D, Scatter plot of the start, target, and change sensitivity of the first response window for each cell. The large yellow data point marks the sample cell from A–C. E, Same for the sensitivity of the second response window. F, Elements of the final sensitivity vector for each cell, obtained by combining the start sensitivity of the first response window and the target and change sensitivity of the second response window.

To systematically analyze the dependence of the responses in these two temporal windows on the combination of starting and target position of the grating, we collected the peak firing rates of each window in a 4 × 4 matrix, corresponding to the 4 × 4 transitions from starting to target position (Fig. 2B). The structure of these two response matrices contains information about the sensitivity of the cell to specific grating positions. For example, the first response matrix of Cell 2 from Figure 1, displayed in Figure 2B (left), contains horizontal stripes, demonstrating its sensitivity to specific starting positions (here Positions 2 and 3). The vertical stripes in the response matrix for the second time window (Fig. 2B, right), on the other hand, correspond to sensitivity to specific target positions.

The occurrence of stripes in the response matrices thus denotes the stimulus sensitivity during the selected response window. The specific position of the stripes, however, merely depends on the location of the receptive field of the cell relative to the bars of the grating. For example, Position 2 of the grating presumably brought an increase in preferred contrast to the receptive field of Cell 1 in Figure 1, but if the receptive field of that cell had been displaced by a quarter grating period in the right direction, the same response peak would have occurred for Position 3 instead.

Therefore, to make the analysis invariant to the receptive field position, we applied a two-dimensional Fourier transformation to the response matrices and only considered the amplitudes of the Fourier components. This yielded a 4 × 4 matrix for each of the two time windows (Fig. 2C). The Fourier component amplitudes in these matrices capture how strongly different wave-like (or stripe-like) patterns are represented in the response matrices, such as stripes along the horizontal, vertical, or diagonal directions. Information about the exact positions of these stripes, on the other hand, is contained in the phases of the Fourier components, which were discarded in our analysis to assess the structure of the response matrices independent of relative position (or phase).

Three entries of the Fourier transformed response matrices are of particular interest as they correspond to the aforementioned sensitivities to starting position, target position, and change of position (Fig. 2C). Simple patterns with single horizontal and vertical stripes in the response matrices, for example, corresponding to sensitivity to the starting and target position, are captured by the (0, 1) and the (1, 0) entry, respectively, of the Fourier transform regardless of the exact preferred position; patterns with diagonal stripes, on the other hand, are typically reflected in the (3, 1) component. These relevant entries are pointed out by arrows in Figure 2C, and corresponding insets show stereotypical response matrix patterns represented by these components.

The entries (0, 2), (2, 0), and (2, 2) would correspond to higher harmonics in the sensitivity matrix (e.g., high activity for Positions 1 and 3, but low activity for Positions 2 and 4), and the (1, 1) component to activity patterns along the other diagonal in the response matrix, from top left to bottom right. As expected, such patterns are generally not observed in our data, and the corresponding Fourier components are always near zero. The remaining components either capture the mean activity, namely the (0, 0) component, or are redundant copies of components already discussed. Thus, for each response window, there are three entries of the Fourier transformed response matrices that capture the basic sensitivity pattern of the cell, as follows: the (0, 1), the (1, 0), and the (3, 1) components, which correspond to the aforementioned sensitivities to starting position, target position, and change of position.

As an example, the cell of Figure 2 exhibited a large (0, 1) Fourier component for the first time window and a large (1, 0) component for the second time window, reflecting its sensitivity to starting and target position in the first and second response window, respectively. Cell 3 from Figure 1, on the other hand, had a diagonal stripe in its second response matrix, reflected by a large (3, 1) entry (data not shown). Note that, in the case of this Cell 3, the large (3, 1) Fourier component was caused by decreased activity along the diagonal of equal starting and target position. On the other hand, since our analysis disregards phase information of the Fourier transform, a large (3, 1) entry could also signify increased activity along this diagonal, corresponding to sensitivity to recurrence of the same grating position across the transition. Such sensitivity to image recurrence has indeed been described for certain ganglion cells of the mouse retina (Krishnamoorthy et al., 2017). For the present datasets from the marmoset, however, we did not find any image-recurrence-sensitive cells. A strong (3, 1) entry in our data always corresponded to a decrease of responses along the diagonal and thus a response sensitivity to a change of the grating position. Finally, for some cells, none of the three entries described above contained a large value. In such a case, the cell either did not respond at all during the corresponding time window or responded indifferently to all transitions with no dependence on the starting or target position or any combination of the two.

To compare the sensitivities to presaccadic and postsaccadic images and combinations thereof across cells, we combined the three relevant entries of each Fourier transformed response matrix [(0, 1), (1, 0), and (3, 1)] into a three-component vector and normalized it (see Materials and Methods). For each response time window, its elements thus characterize the sensitivity of a given cell to the starting position, to the target position, and to change in the fixated stimulus pattern across the saccade. Examining this vector for the first response window for each ganglion cell (Fig. 2D) shows that this response window was generally only sensitive to the starting position, as the other two Fourier components for target position and change sensitivity were always near zero. This was expected, as the first response occurs too early to be affected by the new fixation and is thus mostly elicited by the offset of the starting position grating. The second response, on the other hand, was dominated by the components corresponding to sensitivity to the target position and to change, with considerable differences in the magnitude of these two components between individual cells, but with generally little sensitivity to the starting position alone (Fig. 2E). Thus, this response component is typically affected by the target position of the grating and by combinations of the starting position with the target position, consistent with its occurrence several tens of milliseconds after the onset of the new fixation.

To jointly analyze the most relevant patterns of stimulus sensitivities during the first and second response window, we thus combined the sensitivity for the starting position of the first response with the target and change sensitivities of the second response to obtain a final three-component vector, which we call the sensitivity vector. The sensitivity vector describes the most pronounced response properties of a ganglion cell under our saccade stimulus (Fig. 2F). For example, the sensitivity vector for the sample cell of Figure 2A–C (Fig. 2D–F, large yellow dot) has large sensitivity values for the starting and the target position, but not for the change of position, reflecting the horizontal and vertical stripes in the response matrices of Figure 2B and the lack of a diagonal structure.

As the sensitivity vector provides a general characterization of the sensitivity of a cell to pre- and post-transition image information, we used it to reconfirm that there was no significant difference between motion and gray-screen transitions. For each cell, we calculated the sensitivity vector separately for each transition type and assessed their difference as the Euclidean distance between the two sensitivity vectors, normalized by the Euclidean norm of the standard sensitivity vector for the cell. This can be viewed as a relative deviation of the sensitivity profile under motion versus gray-screen transitions, and the small average (mean ± SD across all analyzed cells) value of 0.16 ± 0.13 indicated that sensitivity profiles varied little with transition type. In fact, the relative deviations are consistent with noise, as a corresponding analysis that split the data in half (regardless of the transition type) yielded comparable relative deviations of the sensitivity vectors of 0.12 ± 0.11 (mean ± SD), corroborating that the transition type has little influence on the sensitivity profile with respect to pre- and post-transition images.

Responses of different cell classes

We next asked whether the observed differences in response sensitivities were connected to the different types of ganglion cells. To investigate this question, we first classified cells according to standard response characteristics measured under spatiotemporal white noise stimulation. Specifically, we measured the spatial receptive fields and temporal filters of the cells via the spatiotemporal spike-triggered average as well as the output nonlinearities and spike autocorrelations of the cells (Fig. 3; see Materials and Methods). Five distinct classes could be readily identified, including the standard types of On and Off parasol cells as well as midget cells. On and Off parasol cells displayed fast, biphasic temporal filters and extensive receptive field tiling. On and Off midget cells had slower filters and smaller receptive fields. Here, however, tiling could only partially be observed, owing to the limited number of recorded cells. The reason for the unequal sampling of midget and parasol cells likely lies in a recording bias of the multielectrode arrays. The fairly wide spacing of electrodes and potential differences in signal-to-noise ratio of recorded spikes may result in more misses and rejections during spike sorting for the smaller midget cells. In addition, some midget cells may not have responded well to the applied stimuli, because of their relatively strong receptive field surround. In addition to these four major primate ganglion cell types, we also identified a fifth type, an Off cell with slow temporal filters and large receptive fields. We here refer to this type as Large Off cells.

• Download figure
• Open in new tab
• Download powerpoint

Figure 3.

Classification of retinal ganglion cells of a sample experiment. A, Scatter plot of the effective receptive field diameter versus the projection onto the first principal component of the temporal filter for all classified cells (magenta, On parasol cells; green, On midget cells; red, Off parasol cells; blue, Off midget cells; orange, Large Off cells). B, Same as A, but for second principal component versus first principal component of the temporal filter. C, Autocorrelation functions, receptive field layouts (1.5-σ ellipses of receptive field Gaussians; distant cells excluded), temporal filters, nonlinearities (scaled to the same maximum) of all classified cells, and a sample spatial filter, grouped by cell type.

We found that the identified major cell types exhibited distinct characteristic responses to the saccadic stimulus. Figure 4A shows representative response profiles, which illustrate the differences in response patterns between the cell types. Many On parasol cells had two separate response components, a first response sensitive to the starting position and a second response sensitive to the target position. For example, the first On parasol cell in Figure 4A (left column, top) displayed an early response peak if the starting position was 4 (weaker if it was 1 or 3) and a later response peak if the target position was 2. Other On parasol cells did not show a clear preference for specific starting or target positions and instead responded rather indifferently (Fig. 4A, left column, bottom example). The sensitivity vectors of all On parasol cells show that there was a continuum between these two response types with indifferent cells lying closer to the origin (Fig. 4B, left column).

• Download figure
• Open in new tab
• Download powerpoint

Figure 4.

Coding properties of ganglion cell types. A, PSTHs of different sample cells, showing the responses of the cells to all 16 combinations of starting and target position. Shaded regions denote the transition period and dashed vertical lines mark the borders of the response windows used for analysis. B, Sensitivity vectors of all cells of the five distinguished cell types. C, Boxplots of the distributions of sensitivity measures (entries of the sensitivity vector) for each of the five cell types. Boxes denote the central 50% of data points (i.e., from 25% to 75%), whiskers the central 90% (i.e., from 5% to 95%), and horizontal lines inside the boxes the medians.

Figure 4C displays boxplots of the distributions over cells of the different sensitivity components. For On parasol cells (left column), this confirmed that the cells were mostly sensitive to the starting position and (somewhat less) to the target position. Change sensitivity only played a subordinate role. In one experiment, though, we also found a few On parasol cells whose second response did not seem to be sensitive to the target position but rather to the change of position.

Similar to On parasol cells, On midget cells (Fig. 4A, second column) also showed a first response sensitive to the starting position and a second response sensitive to the target position. In contrast to their parasol counterparts, however, the responses of On midget cells were dominated by the second response peak, which was more pronounced and sustained than the first. This shifts the sensitivity balance toward the target position (Fig. 4B,C).

For Off parasol cells, the most striking response feature was that many cells were sensitive to a change of the grating position across the transition. This is evident from the reduced responses during the second response window on the diagonal of equal starting and target position in the matrix representation of the PSTHs (Fig. 4A, third column) as well as from the large change sensitivity component of the sensitivity vectors (Fig. 4B,C). In addition, however, there often was also considerable sensitivity to the specific starting and target positions in the first and second response, respectively.

Off midget cells were mainly sensitive to the target position. Both examples in Figure 4A (fourth column) show cells that responded only to the occurrence of one or two specific target positions with only some modulation by the starting position. The moderate amount of modulation by the starting position was such that responses to a recurrence of the same grating position were reduced. This mild change sensitivity is also revealed by the sensitivity vectors (Fig. 4B,C). Overall, however, Off midget cell responses were dominated by their target sensitivity, which they displayed more strongly than any of the other cell types.

For the Large Off cells, the striking feature was their pronounced sensitivity to the change of the grating position (Fig. 4A, right column) with essentially no sensitivity to the specific starting or target position (Fig. 4B,C). These cells generally showed no activity during the first response window and transient responses during the second whenever starting and target position differed.

There are thus systematic differences in the sensitivity profiles between different cell types as well as some variability within a single type. Given that the spatial structure of the gratings experienced by the cells depend on their receptive fields and that receptive field sizes can differ substantially between cells, we analyzed to what degree this influenced the sensitivity variations within and across cell types (Fig. 5). Indeed, for both On parasol cells (Fig. 5A,B) and Off parasol cells (Fig. 5D,E), we found that the observed variability in starting and target sensitivity depended systematically on receptive field size across our recordings: smaller cells had stronger sensitivity values and larger cells usually responded more indifferently. This makes intuitive sense, as larger receptive fields are more likely to contain both dark and bright bars of the grating for each position. For Off parasol cells, this means that cells with larger receptive fields were typically dominated by their strong sensitivity to the change of the grating position while the specific starting and target positions did not significantly influence the responses (Fig. 4A, top example). Smaller Off parasol cells, however, were also sensitive to the actual starting and target positions, which obscured the change sensitivity to some degree (Fig. 4A, bottom example).

• Download figure
• Open in new tab
• Download powerpoint

Figure 5.

Dependence of stimulus sensitivities on receptive field size. A, Sensitivity to the starting position versus width of the receptive field for all On cells. Receptive field width was defined as the extent in x-direction (perpendicular to the grating) of the 1.5-σ ellipse of the receptive field Gaussian and given relative to the size of a grating bar. Circles represent individual cells, and solid lines are robust linear regressions for each cell type using the Theil-Sen estimator. B, Same as in A, but for sensitivity to the target position. C, Same as in A, but for sensitivity to change. D–F, Same as in A–C, but for Off cells.

Other aspects of the observed cell type-specific sensitivity profiles could not be explained by receptive field size. The relatively weak sensitivity to the starting position of On midget cells (Fig. 5A) as well as Off midget cells (Fig. 5D) compared with the target sensitivities of the cells (Fig. 5B,E), for example, was largely independent of receptive field size and deviated systematically from the start sensitivity of On and Off parasol cells even when receptive field sizes were similar (Fig. 5A,D). Moreover, the general weakness of change sensitivity in On cells was independent of receptive field size (Fig. 5C), and the three studied types of Off cells each displayed their respective levels of change sensitivity also largely independent of receptive field size (Fig. 5F). Note here, that the largest cells, Large Off cells, have the highest change sensitivity, and the smallest ones, Off midget cells, have the lowest change sensitivity. Yet, if larger receptive fields directly led to change sensitivity, one would expect substantial positive correlation within the individual cell types. Instead, receptive field size and change sensitivity showed either no or only a weak relationship that is not sufficient to account for cell type differences (Off midget and Large Off cells, p > 0.05; Off parasol cells, R = 0.15, p = 0.046). We thus conclude that change sensitivity is not simply a consequence of smaller or larger receptive fields but appears to be an intrinsic cell type-specific feature. Together, these analyses show that different types of ganglion cells systematically differ in how the combination of presaccadic and postsaccadic images affect the spiking activity during and after a saccade-like image transition.

The cell type-specific differences in sensitivity to starting position versus target position raise the question whether this follows from differences in response strength under stimulus onset versus stimulus offset. For example, the relatively stronger sensitivity of a midget cell to the target position might come from stronger activation by the onset of the newly fixated target grating, whereas the offset of the previous grating (starting position) triggers little activity. We therefore compared the responses under the saccadic stimulus with responses to onsets and offsets of gratings. Since our data did not contain responses to gratings flashed individually with the same bar width and positions as in the saccadic stimulus, we instead analyzed the responses to the onset and offset of stimulation with contrast-reversing gratings. These also contained onsets and offsets at the very start and end of each reversing sequence, preceded and followed by homogeneous mean intensity illumination. Thus, for each cell, we averaged responses to the onset and offset of the reversing-gratings stimulus over bar widths of 60 and 120 µm, near the 90 µm used for the saccade stimulus, and over different spatial phases. We then compared the relative response strengths under onset versus offset, assessed as the average firing rate in the 200 ms following onset or offset, to the relative sensitivity to the target versus starting position.

From the sample cells in Figure 6, A and B, only the On parasol cell responded more strongly to the offset than the onset of the grating, which is consistent with its stronger sensitivity to the starting than to the target position. We confirmed this observation by comparing the relative strength of the onset versus offset response with the relative sensitivity for the target versus the starting position during the saccadic stimulus for all cells (Fig. 6C). On parasol cells generally had stronger offset than onset responses and, consistent with this, had greater sensitivity for the starting compared with the target position. By contrast, responses of all Off-type cells and of the On midget cells were dominated by grating onset as well as by the sensitivity for the target position compared with sensitivity for the starting position.

• Download figure
• Open in new tab
• Download powerpoint

Figure 6.

Comparison of saccade responses with responses to flashed gratings. A, Exemplary saccade responses depicted as in Figure 4A. Large Off cell is Cell 3 from Figure 1C. B, PSTHs of the cells in A in response to offset and onset of the grating stimulus. Shown is the average of the 60 and 120 µm bar width responses. The relevant part of the grating stimulus is schematically depicted beneath each PSTH. Dashed box marks the part of the response used to determine the response strength. Scale bars of A apply here, too. C, Relative strength of the response to the onset of the grating versus its offset (y-axis) and relative sensitivity to the target versus starting position during the saccadic stimulus (x-axis). Values close to 1 represent a dominant onset response/target sensitivity, values close to −1 a dominant offset response/start sensitivity. Colors denote cell types, stars represent means of the cell types. D, Average response strength of all On parasol/midget cells in response to the onset/offset of the grating, depending on the bar width of the grating (logarithmic axis). Response strengths are normalized to the same maximum before averaging over the cells. E, Same as D but for Off cells.

We also investigated how the onset and offset response strengths of the cells depended on the grating size. The findings suggest qualitative differences between On cells (Fig. 6D) and Off cells (Fig. 6E). While Off cells responded more strongly to a grating onset than offset almost regardless of the grating size, both On parasol and On midget cells responded more strongly to the offset if the grating was sufficiently fine, but not for coarser gratings with bar widths beyond a few hundred micrometers. Thus, it seems that the relative strength of offset versus onset responses depends on the spatial stimulus structure inside the receptive field for On cells.

Note, however, that there are some caveats to these analyses of grating onsets and offsets. In particular, the small number of trials from which onset and offset responses could be gathered and the different adaptation states at onset versus offset of the reversing gratings (Appleby and Manookin, 2019; Yu et al., 2022) contribute to variability. Furthermore, the comparison between responses to the saccade-like grating shifts and to grating onset and offset are complicated by differences in grating size and contrast.

Modeling responses to the saccadic stimulus

Given the relationship between the responses to the saccadic stimulus and the onset and offset responses to gratings, we next asked whether or to what degree the different response patterns under the saccadic stimulus followed directly from how the cells responded to simple steps in light intensity. If this was the case, the saccadic stimulus might simply be considered as a sequence of two step stimuli whose responses superimpose. Alternatively, the temporal vicinity of fixation offset and onset across a saccade could modify or add response characteristics beyond what is triggered by isolated light flashes. If so, this might hint at circuit mechanisms that are triggered specifically by saccades.

As the saccade-like motion transition of the stimulus evoked essentially the same responses as the gray transition (Fig. 1), the saccadic stimulus can be understood as a combination of an offset of bright and dark regions followed by an onset of a new bright/dark pattern. We therefore compared the responses to the saccadic stimulus with responses to full-field brightness steps from mean light intensity (gray) to high intensity (white) or to low intensity (black) and back to gray. For this stimulus, all brightness changes were temporally separated by several hundred milliseconds so that they can be considered as individual stimulation events with little influence on each other.

Under the full-field brightness steps, as expected, On cells responded to an increase in brightness, whereas Off cells responded to a decrease, and parasol cells responded more transiently than midget cells (Fig. 7A). The Large Off cells also responded transiently to decreases of the brightness, but with a longer latency than Off parasol cells.

• Download figure
• Open in new tab
• Download powerpoint

Figure 7.

Modeling ganglion cell responses to saccades based on responses to brightness steps. A, Exemplary PSTHs for all five retinal ganglion cell types to full-field brightness steps. The stimulus is schematically depicted beneath each PSTH. B, Schematic depiction of obtaining a response prediction for the ON parasol cell in A. The response of the cell to the full-field brightness steps (top) was split into responses to the onsets and offsets of white and black (zoomed-in insets below, small circles and coloring of the PSTHs denote the brightness change). For the linear model (left), the response to an offset of the first grating was estimated by scaling the response of the cell to the offset of the appropriate brightness step, here one-third of the offset of white, corresponding to the relative decrease in mean luminance inside the receptive field. Analogously, the response to the onset of the new grating was estimated here as one-third of the onset of white, corresponding to the increase in mean brightness. Unlike what is depicted here, these two response components overlapped strongly, because of the briefness of the transition. They were then summed to form the final response prediction. For the nonlinear model (right), each pixel directly contributed response components to the final response, thereby omitting the averaging of the brightness inside the receptive field. At the offset of the first grating in this example, one-third of the receptive field turned from black to gray, and two-thirds from white to gray, yielding a one-third contribution of the black offset response and a two-thirds contribution of the white offset response. The onset of the second grating position was treated analogously and the four response components were summed to generate the final response prediction.

To assess the relation between the responses to brightness steps and responses to the saccadic stimulus, we aimed at modeling the latter based on the former (Fig. 7B). Since the saccadic stimulus contains spatially structured images and given that ganglion cells can pool signals over space either linearly or nonlinearly (Enroth-Cugell and Robson, 1966; Hochstein and Shapley, 1976; Schwartz and Rieke, 2011; Gollisch, 2013; Turner and Rieke, 2016; Karamanlis and Gollisch, 2021; Zapp et al., 2022), we correspondingly set up two models with either linear or nonlinear spatial integration. The two models only differed in the order of two simple operations: averaging signals across the receptive field and turning light intensity steps into firing rate via the measured responses to full-field brightness steps. In the spatially linear model, light intensity values of the gratings are first averaged over the receptive field, and this averaged signal is used to select and scale the appropriate responses to full-field brightness steps to obtain response predictions at fixation offset and at the subsequent new onset.

The spatially nonlinear model, on the other hand, first treats every stimulus pixel independently to select the appropriate responses to full-field brightness steps according to the brightness changes of the pixel in the saccade stimulus. These pixel contributions are then averaged across the receptive field. This averaging of the brightness-step responses incorporates local nonlinearities because—unlike in the spatially linear model—the contributions of pixels with opposite contrast do not cancel each other out. For example, the spatially nonlinear model responds to gratings even when brightening and darkening inside the receptive field are perfectly balanced because the brightness step responses to preferred and nonpreferred contrast changes are typically nonsymmetrical and therefore do not cancel. Thus, while the linear model predicts responses only according to the average brightness changes in the receptive field, the nonlinear model also includes the spatial structure of a stimulus in its response.

Note that computing the firing rate contributions on the level of individual pixels (<10 µm for our display) may seem counterintuitive, as this is far below the scale of bipolar cell receptive fields (Dacey et al., 2000), which are thought to correspond to the nonlinear subunits of ganglion cell receptive fields and thereby give rise to spatial nonlinearities (Demb et al., 2001; Borghuis et al., 2013; Turner and Rieke, 2016; Yu et al., 2022). Yet, because the stripes of the applied grating with a width of 90 µm are much broader than the individual pixels, and also are similar in size or larger than expected subunits, the use of individual pixels is essentially identical to using larger subunits. For most subunits, most of the pixels experience the same sequence of light intensities and therefore contribute identical response profiles to the firing rate of a ganglion cell. For the applied grating stimulus, treating each pixel individually is thus a good approximation while avoiding the need to specify a particular subunit layout. Note also that the nonlinear model does not contain a second nonlinear step that would account for overall response transformations after spatial integration. Limiting the model to a single nonlinear stage was done for simplicity, so as to evade any parameter fits, and for keeping the complexity of the linear and the nonlinear model identical.

For both the linear and the nonlinear model, the receptive field was obtained by separating the spike-triggered average from spatiotemporal white noise stimulation (Chichilnisky, 2001) into a spatial and a temporal component and fitting a two-dimensional Gaussian to the latter. For each of the 16 transitions in the saccade stimulus, predictions were first assembled separately for the offset of the previously fixated grating and the onset of the new grating before summing the two contributions with a relative delay that corresponded to the transition time. The only two free parameters in the models were an overall scaling of response amplitude and an overall temporal shift of the predictions, which accounted for amplitude and latency differences because of differences in applied contrast. The amplitude scaling, however, did not affect our Fourier-based analysis.

For On cells, we found that the responses to the saccadic stimulus could generally be predicted well by at least one of the two models. The responses of On parasol cells were captured well by the nonlinear model, but not by the linear model. The linear model failed, for example, to recreate the strong responses of the sample On parasol cell of Figure 8A (left) to transitions between Positions 2 and 4 (Fig. 8B). These grating positions yielded approximately equal bright and dark contrast in the receptive field of a cell (Fig. 8D), and the linear model therefore predicted no response, because these regions could cancel each other out. By contrast, the nonlinear model correctly captured the response patterns (Fig. 8C). It also succeeded in predicting a first response sensitive to the starting position and a second response sensitive to the target position, although it underestimated the strength of modulation of the second response caused by the target position.

• Download figure
• Open in new tab
• Download powerpoint

Figure 8.

Model evaluation for On parasol (left) and On midget (right) cells. A, PSTHs of sample cells to the full-field brightness steps. The On midget cell is the same cell as in Figure 6A. B, Experimental responses to the saccadic stimulus (thin black line with gray filling) and predictions by the linear model (thick blue line) of the same sample cells to the saccadic stimulus. Layout of the plot is the same as in Figure 1C. C, Same as B but for the nonlinear model (red line). D, Illustrations of the content of the receptive fields of cells at the four grating positions. E, Modified coefficient of determination R² for the response matrix of the data versus the linear model (x-axis) and versus the nonlinear model (y-axis). R² values are shown separately for the first and second response matrix. Colored dots represent all ganglion cells of the cell type of the column, light gray dots all other cell types. Cells with a model R² value below −0.05 have been plotted on the axis. F, Distributions of the distances between modeled and experimental sensitivity vectors for the linear and the nonlinear model.

To quantify the accuracy of the model predictions, we computed modified coefficients of determination R² between the response matrices of the experimental data of a cell and the response matrices as calculated from the modeled PSTHs (see Materials and Methods). For each response window, this yielded one value per cell and model, which usually lay in the range of zero (no correlation between model and data in that response window) to unity (perfect correlation between model and data). For the On parasol cells, the nonlinear model, but not the linear model, generally achieved R² values close to unity, especially for the first response window and only slightly less so for the second response window (Fig. 8E), corroborating the spatial nonlinearity of On parasol receptive fields under these stimulus conditions.

While the computed coefficients of determination quantify how well the amplitudes of the response peaks in the PSTHs are captured, they do not directly assess whether the models capture the sensitivity characteristics of a cell with respect to starting position, target position, and change of the grating. As an alternative measure of model accuracy, we therefore computed the distance between the measured sensitivity vector of a cell and the sensitivity vector calculated from the modeled responses. A small distance indicates that the response sensitivities have been reproduced, while a large distance represents discrepant sensitivities. For the On parasol cells, the distance of the sensitivity vectors of the linear model to the experimental sensitivity vectors was generally large, while the nonlinear model produced small distances (Fig. 8F). This confirms that On parasol responses to the saccadic stimulus could be modeled well by using the responses to a full-field stimulus and assuming a nonlinear receptive field. Only in one experiment, a subset of On parasol cells with unusually slow and weak responses to the full-field brightness step from black to gray were not modeled well by the nonlinear model.

Midget cells, like the example in Figure 8A–C (right), were modeled decently by both the linear as well as the nonlinear model. Here, the linear and the nonlinear model yielded similar response predictions because the small receptive fields of these cells contained mostly only a single bar of the grating (Fig. 8D). Because of the lack of spatial structure within the receptive field, the spatial nonlinearity played hardly any role. While the strength of the second response of the sample cell was partially overestimated, the main response properties (i.e., the transient, start-sensitive first response and the sustained, target-sensitive second response) were successfully predicted. For both response windows, the models achieved relatively high R² measures of the response matrices, with a tendency toward better predictions by the nonlinear model (Fig. 8E; first window, p = 2.1 × 10⁻⁵; second window, p = 3.3 × 10⁻⁶; Wilcoxon signed-rank test). The distance of sensitivity vectors, on the other hand, did not show a clear trend (Fig. 8F; p = 0.21, Wilcoxon signed-rank test).

In contrast to On cells, the responses of Off parasol cells as well as Large Off cells could only partially be explained by the linear or nonlinear model. The linear model displayed similar problems for Off parasol cells and Large Off cells as it did for On parasol cells, often strongly underestimating the responses (Fig. 9B) because of cancelation that does not occur in the nonlinear receptive fields of these cells. By contrast, the nonlinear model could mostly reproduce the sensitivities to the starting and target position (Fig. 9C). However, the responses on the diagonal (i.e., to a recurrence of the grating position) were consistently overestimated. For example, the Off parasol cell of Figure 9 lacked a strong second response to the transition when starting and target position were both Position 4. Both models predicted such a response, since the receptive field returned to being mostly filled with black after the transition (Fig. 9D). Accordingly, both the similarity between response matrices (Fig. 9E) and the sensitivity vector distance (Fig. 9F) show that the models did not capture the response characteristics of Off parasol and Large Off cells as successfully as for other cell types.

• Download figure
• Open in new tab
• Download powerpoint

Figure 9.

Model evaluation for Off cells. Same layout and subfigures as in Figure 8, but with Off parasol cells in the left column, Off midget cells in the middle column, and Large Off cells in the right column.

For Off midget cells, akin to On midget cells, the small size of their receptive fields led to similar predictions by the linear and nonlinear models (Fig. 9A–D, middle column). For the sample Off midget cell, the general target sensitivity was reproduced, although responses to Position 4 as target position were overestimated, possibly a result of noise in the receptive field measurement. Furthermore, the slight modulation of the responses by the starting position, hinting at some change sensitivity, was not captured by the models. For the first response window, the models achieved comparatively low R² values (Fig. 9E), largely because Off midget cells responded only weakly and unreliably during this window. For the second response window, however, which included the bulk of the Off midget responses, both models achieved decent R² values, but were likely suffering somewhat from the mild change sensitivity in the responses that was not captured by the models. The distance between the modeled and measured sensitivity vectors of the Off midget population was rather small (Fig. 9F), indicating that the general sensitivity profiles of Off midget cells could mostly be explained by their responses to the full-field brightness steps.

The analysis of Figure 5 had shown us that the receptive field size appears to directly influence some of the assessed sensitivities under our saccadic stimulus. We therefore used the obtained models to evaluate in simulations how the receptive field size affected the distinct response characteristics. To do so, we evaluated the response predictions by the models for each cell, with receptive fields scaled to a range of different sizes, and analyzed the resulting sensitivities, averaged over all cells of a particular type.

The sensitivities of the predictions of the linear model were independent of the receptive field size (Fig. 10, dashed lines). While the model produced weaker responses for larger receptive fields, owing to increasing cancelation from bright and dark regions inside the receptive field, the relative differences in average brightness levels of different grating positions remained and thus the preference for certain starting and target positions. In a more realistic setting, of course, the increasingly weak responses would be overshadowed by noise, thereby reducing all sensitivities.

• Download figure
• Open in new tab
• Download powerpoint

Figure 10.

Influence of receptive field size on model predictions. A, Mean start sensitivity of each On cell type as modeled by the linear (dashed line) and nonlinear (solid line) model, depending on the receptive field width each cell was scaled to. Receptive field width was defined as in Figure 5. Measured cells are plotted in the background for reference. B, Same as A, but for target sensitivity. C, Same as A, but for change sensitivity. D–F, Same as A–C, but for Off cells.

The nonlinear model, on the other hand, directly predicted decreasing sensitivities for starting and target positions with increasing receptive field size (Fig. 10, solid lines), as the responses became more indifferent. This characteristic qualitatively matched our observations for different cell types, most notably for On and Off parasol cells. Yet, the predicted curves do not always quantitatively align with the data, and the general declining shape of all curves does not capture the fact that parasol cells, despite their larger receptive fields, were more sensitive to the starting position than midget cells. Thus, although receptive field size clearly shapes the sensitivities of the cells to starting and target position, it alone cannot explain the cell-specific preferences for starting or target position. It seems likely that, instead, the faster response kinetics of parasol cells and the strong preference of On parasol cells for offsets compared with onsets of gratings (Fig. 6) play the larger role here. Moreover, the unexplained sensitivity to change across the transition, as observed mostly for Off cells, was not predicted by any of the tested receptive field sizes (Fig. 10C,F).

Figure 11 summarizes the sensitivity measures extracted from the two models as well as from the data. Evidently, while the sensitivity to starting and target position could generally be explained by the preference of a cell for light increments and decrements (Fig. 11A,B), in particular when using the nonlinear model, the change sensitivity could not (Fig. 11C). Accordingly, the sensitivity vectors that were calculated from the model responses were largely restricted to the plane spanned by starting and target sensitivity; the predicted change sensitivity was always close to zero (Fig. 11D,E). Therefore, the measured change sensitivity of Off parasol and Large Off cells and potentially also of Off midget cells appears to be the result of additional mechanisms. These mechanisms have the effect of spreading out the sensitivity vectors in the analyzed three-dimensional sensitivity space (Fig. 11F), thus diversifying the response characteristics of the different cell types under saccade-like image shifts.

• Download figure
• Open in new tab
• Download powerpoint

Figure 11.

Sensitivity vectors from models and experiments. A, Boxplots for the distribution of start-sensitivity values for each cell type, obtained via each of the two models and from the experimental data. The box marks the central 50%, whiskers the central 90%, and the horizontal line inside the box the median. B, Same as A, but for target sensitivity. C, Same as A but for sensitivity to grating position change. D, Scatter plot of sensitivity vectors calculated from the linear model for all cells of the five types. E, Same as D but for nonlinear models. F, Same as D but for experimental data.