Nonlinear spatial integration underlies the diversity of retinal ganglion cell responses to natural stimuli

How neurons encode natural stimuli is a fundamental question for sensory neuroscience. In the early visual system, standard encoding models assume that neurons linearly filter incoming stimuli through their receptive fields, but artificial stimuli, such as reversing gratings, often reveal nonlinear spatial processing. We investigated whether such nonlinear processing is relevant for the encoding of natural images in ganglion cells of the mouse retina. We found that standard linear receptive field models fail to capture the spiking activity for a large proportion of cells. These cells displayed pronounced sensitivity to fine spatial contrast, and local signal rectification was identified as the dominant nonlinearity. In addition, we also observed a class of nonlinear ganglion cells with opposite tuning for spatial contrast and a particular sensitivity for spatially homogeneous stimuli. Our work highlights receptive field nonlinearities as a crucial component for understanding early sensory encoding in the context of natural stimuli.


Introduction 21
The natural visual world is communicated to the brain through an array of functionally distinct, parallel channels that originate in the retina (Baden et al., 2016;Roska and Meister, 2014). A classical 23 view of retinal function advocates that the retinal output channels, represented by types of retinal 24 ganglion cells, serve as linear filters for natural visual inputs (Atick and Redlich, 1990;Shapley, 25 2009). This linear picture entails that ganglion cells signal the (weighted) average of the light intensity 26 in their center-surround spatiotemporal receptive fields (RFs) and that different ganglion cell types are 27 discerned through differences in linear properties, such as their temporal summation profile or RF 28 size. Support for the linear picture comes from quantitative models with linear spatio-temporal RFs 29 that have been successful in predicting ganglion cell responses to artificial stimuli, at least when To quantify the impact of spatial contrast on the spike output for a given cell, we grouped the images 162 into pairs of similar linear predictions by the cell's LN model. This allowed us to relate differences in 163 spike count within a pair to differences in spatial contrast, while minimizing confounding effects of 164 mean light-level changes inside the RF. The analysis revealed that spatial contrast was systematically 165 related to spike count for many cells, with more spikes elicited when spatial contrast was larger 166 ( Figure 2C). Indeed, for the majority of cells (72%, n = 685/957 recorded cells), differences in spatial 167 contrast and spike count were positively correlated, indicating that spatial contrast had a response-168 boosting effect beyond mean light level and that spatial integration was nonlinear.

183
Other cells (24%, 225/957) appeared insensitive to spatial contrast, as indicated by an approximately 184 flat relationship between differences in spatial contrast and spike count and no significant correlation 185 ( Figure 2D). This was expected as the LN model, which is based solely on mean light level in the RF, 186 did provide an accurate description of spike counts for some retinal ganglion cells. Unexpectedly, 187 however, we also found a small subset of cells (5%, 47/957) that showed smaller spike counts for 188 images with higher spatial contrast ( Figure 2E). Such inverse sensitivity to spatial contrast represents 189 a different form of nonlinear spatial integration than the response-boosting effect of spatial contrast in 190 the majority of cells and may be described as sensitivity to spatially homogeneous stimulation. 191 In order to assess whether sensitivity to spatial contrast was systematically related to LN model 192 performance, we quantified the "spatial contrast sensitivity" of a given cell by the slope of the 193 regression line between spatial contrast and response differences, normalized by the cell's maximum 194 response. We found that spatial contrast sensitivity was indeed negatively correlated with LN model 195 maximum F2 (across spatial frequencies) over the maximum F1 (across spatial frequencies) amplitude. Cells 226 from (A-C) are highlighted.
Interestingly, sensitivity to contrast reversals of gratings was often unrelated to LN model 228 performance for natural images. One of the sample cells with clear responses to fine-scale gratings 229 ( Figure 3A) had poor LN model performance for natural images ( Figure 3D), whereas the other 230 ( Figure 3B) showed good model performance ( Figure 3E). For the third sample cell ( Figure 3C), 231 model performance was good (Figure 3F), consistent with the observed insensitivity to grating 232 reversals, which suggests linear spatial integration. In order to systematically compare the sensitivity 233 to reversals of fine gratings with the LN model performance across multiple ganglion cells, we first 234 estimated a cell's spatial scale for detecting grating reversals by fitting a logistic curve to the cell's 235 peak firing rates across grating bar widths and extracting the curve's midpoint ( Figure 3G). However, 236 there was rarely a correlation between this spatial scale and LN model performance in individual 237 experiments (median Spearman's ρ = 0.16, 2/12 experiments had p < 0.01; example in Figure 3H, 238 left), and for the entire dataset, this correlation was weak, albeit significant (Spearman's ρ = 0.12, p = 239 0.001, n = 746; Figure 3H, right). Thus, sensitivity to reversals of high spatial frequency gratings, 240 typically taken as a sign for nonlinear spatial integration, does not generally imply failure of the LN 241 model. In fact, many cells with small spatial scales showed remarkably good model performance as 242 illustrated by the example in Figure 3E. 243 Examining such cells with good LN model performance but small spatial scale, we noticed 244 differences in the firing rate profiles between different gratings. Though initial response peaks might 245 be similar, responses could be more sustained with overall higher spike count when net-coverage of 246 the RF with preferred contrast was larger (see arrows in Figure 3B). This response difference may be 247 explained by spatial integration that has both a linear and a nonlinear component (e.g. resulting from 248 partial rectification of spatially local inputs amplitudes, as compared to F1, are indicative of nonlinear spatial-integration effects (Hochstein and 256 Shapley, 1976) at the level of spike counts. For the two spatially nonlinear cells of Figure 3A-B, for 257 example, the F2 peak at 30 μm was similar, but the cell with low LN model performance had an performance better than the spatial scale. To show this, we computed a nonlinearity index as the ratio 262 of the maximal F2 amplitude (over grating widths and phases) and the maximal F1 amplitude (see 263 Methods). The nonlinearity index was negatively correlated to LN model performance under natural 264 images both in single experiments (median Spearman's ρ = -0.40, 9/12 experiments had p < 0.01) as 265 well as in the whole population (Spearman's ρ = -0.35, p < 10 -22 , n = 746). Thus, the relative degree of 266 nonlinear spatial integration appears to play a role in determining ganglion cell responses to natural 267

images. 268
Despite the correlation between the nonlinearity index with LN model performance, the classical 269 analysis with contrast-reversing gratings had some drawbacks. Firstly, for ON-OFF cells, the 270 approach cannot distinguish between nonlinear integration over space or over ON-type versus OFF-271 type inputs, as both phenomena can lead to large F2 components. Secondly, the analysis only detects 272 that some rectification of non-preferred contrasts exists (as effects of preferred and non-preferred 273 contrast do not cancel out), but it does not offer information about how preferred contrast signals at 274 different locations inside the RF are combined. 275

Responses to contrast combinations inside the RF reveal the components of natural spatial 276 contrast sensitivity 277
To overcome the shortcomings of classical contrast-reversing grating stimulation and explore the 278 relationship between spatial contrast sensitivity and LN model performance more systematically, we 279 designed a checkerboard flash stimulus that tests a range of contrast combinations. This stimulus was 280 based on the idea of independently stimulating two separate sets of spatial subunits within a cell's RF 281 with different inputs (Bölinger and Gollisch, 2012;Takeshita and Gollisch, 2014). Concretely, we 282 flashed a batch of varied checkerboards onto the retina ( Figure 4A, top). The contrasts of the two sets 283 of tiles, or spatial inputs, were sampled from the stimulus space of pairs of contrast values ( Figure 4A, 284 bottom-right) to systematically explore a wide range of contrast combinations. To directly compare 285 responses between artificial and natural stimuli, we flashed the contrast pairs for 200 ms each (the 286 same duration as for the natural images) in a pseudorandom sequence, collecting four to five trials per 287 pair. The subfields of the checkerboard spanned 105 μm to the side, approximately half of the average 288 mouse RF center, to provide a strong, yet spatially structured stimulus inside the RF. 289 The checkerboard flashes revealed a variety of spatial integration profiles among different retinal 290 ganglion cells. To extract these profiles, we visualized the responses (defined as the average spike 291 count over 250 ms after stimulus onset, equivalent to the response measure under natural images) as 292 color maps over the stimulus space of contrast pairs ( Figure 4B, middle row). We then calculated iso-293 response contour lines ( Figure 4B, bottom row), which trace out those contrast pairs that led to the curved ones reflect the existence of a nonlinearity and indicate the type of nonlinearity in the shape of 298 the contour. 299 We found both ON and OFF varieties of nonlinear cells ( Figure 4B, Cells 1-2), with contour lines 300 curving away from the origin. This nonlinear signature may result from an expansive transformation 301 of local signals, such as by a threshold-quadratic functions (Bölinger and Gollisch, 2012), or by a 302 sigmoid with high threshold (Maheswaranathan et al., 2018). We also found linear ON and OFF 303 ganglion cells, with straight contour lines ( Figure 4B, Cells 4-5). Furthermore, our approach allowed 304 us to visualize the spatial integration profiles of ON-OFF cells, and distinguish between spatially 305 nonlinear and linear ON-OFF cells ( Figure 4B, Cells 3 and 6). Linear ON-OFF cells responded mostly 306 to net-increases or decreases of light intensity, but not when the two contrast signals cancelled each 307 other, leading to parallel contour lines (Cell 6). On the other hand, nonlinear ON-OFF cells often had 308 closed or nearly closed contour lines, corresponding to strong responses also for contrast 309 combinations with opposing signs (Cell 3). Finally, we identified a unique nonlinear spatial 310 integration profile in some cells, characterized by contour lines curving towards the origin ( Figure 4B, 311 Cell 7). Such a profile indicates a particular preference to a spatially homogeneous change in light 312 level, as has been previously observed in the salamander retina (Bölinger and Gollisch, 2012). We 313 mainly found such profiles for OFF-type ganglion cells, but occasionally in ON-type cells as well 314 (6/27 cells were ON-type). 315  For further analysis, we selected for each cell the patch location closest to the RF center. This 341 generally lay not further away than one RF radius ( Figure 4C, bottom right), indicating good overlap 342 of the analyzed patch location with the RF center. We found that spatial integration profiles, as 343 captured by the shape of the contour lines in stimulus space, were qualitatively similar under local 344 stimulation as compared to full-field stimulation ( Figure 4D). there is also nonlinear integration of preferred contrast, which is visible in a nonlinear shape of the 351 contour line inside the quadrant that corresponds to preferred contrast of both stimulus components. 352 To quantify these nonlinear contributions, we devised two corresponding indices ( Figure 4E). We 353 comparing responses from using just one spatial input at a specific contrast level with responses from 359 using both inputs at half that contrast level. A convexity index of zero corresponds to linearity (equal 360 responses for a single component at full contrast and for two components at half contrast), whereas 361 values smaller or larger than zero correspond to increased or decreased preference for homogeneous 362 stimuli, respectively (see Figure S2 for sample responses used to compute the indices). Over the 363 population of all recorded cells, the two indices were correlated ( Figure 4F for the full-field indices; and convexity indices across the two conditions. Although both indices displayed a significant change 368 between local and full-field stimulation (Wilcoxon signed-rank test, p < 10 -10 for the rectification and 369 p < 10 -6 for the convexity index), the values were correlated between the two conditions (Spearman's 370 ρ = 0.80, p < 10 -73 for the rectification and Spearman's ρ = 0.39, p < 10 -12 for the convexity index), 371 indicating that cells retained their relative characteristics of nonlinear spatial integration, in particular 372 regarding rectification ( Figure 4G). One subtle change was that for many cells with convexity index 373 >0 in the full-field stimulus, the index became ~0 for the local stimulus ( Figure

386
How are the extracted components of nonlinear spatial integration related to responses under natural 387 images? We found that spatial contrast sensitivity, as determined from the responses to natural images 388 (cf. Figure 2D), was correlated with both the rectification (Spearman's ρ = 0.71, p < 10 -112 ) and the 389 convexity index (Spearman's ρ = 0.58, p < 10 -67 ), as obtained from full-field stimulation ( Figure 5A  390 and 5C, top). Similar results were also found for the indices obtained from local stimulation (Figure 391 5B and 5D, top; Spearman's ρ = 0.70, p < 10 -53 for rectification and ρ = 0.33, p < 10 -9 for convexity). 392 At the same time, the rectification indices from both full-field (Spearman's ρ = -0.71, p < 10 -113 ) and from contrast-reversing gratings. The convexity indices from full-field (Spearman's ρ = -0.56, p < 10 -396 60 ) and local stimulation (Spearman's ρ = -0.40, p < 10 -14 ) were also predictors of LN model 397 performance ( Figure 5C-D, bottom), but to a smaller extent than rectification indices. We thus 398 concluded that the degree of rectification of spatial inputs in the RF center is a primary factor that 399 shapes ganglion cell responses to natural images and determines whether responses can be captured 400 by the LN model. 401 The spatial scale of contrast sensitivity for natural images 402

442
When analyzing responses across different blurring scales, we observed that cells sensitive to spatial 443 contrast reduced their spike counts already at scales smaller that their RF center ( Figure 6A, bottom). 444 To quantify the spatial scale of blurring sensitivity for each cell, we measured the similarity between 445 responses to original and blurred images by calculating the corresponding coefficient of determination 446 (R 2 ), which is unity when responses with and without blurring are identical and falls off towards zero 447 as responses to blurred images deviate more and more from the original responses. Analogous to the 448 analysis of contrast-reversing gratings, we fitted logistic functions to the decay of R 2 with blurring scale and defined the spatial scale as the midpoint of the logistic function. The obtained spatial scales 450 ranged from 100 to 500 μm ( Figure 6E) and were only weakly correlated with the spatial scales 451 measured with contrast-reversing gratings (Spearman's ρ = 0.12, p = 0.006). And unlike the spatial 452 scale obtained from reversing gratings, the spatial scale from blurred images (normalized by the RF 453 center diameter) was strongly related to LN model performance ( Figure 3F) in both individual 454 experiments (median Spearman's ρ = 0.59, 9/9 experiments had p < 0.05) and in the pooled 455 population (Spearman's ρ = 0.60, p < 10 -64 ). 456

Spatial contrast sensitivity differs among retinal ganglion cell classes 457
The analyses so far have shown that the characteristics of spatial integration are consistent for 458 individual ganglion cells across different stimulus conditions, including natural and artificial stimuli. 459 We thus hypothesized that they reflect cell-type specific properties. To test this hypothesis, we looked 460 at three readily identifiable cell classes, detected through a standard set of artificial stimuli.  Wilcoxon sign-rank test, p = 0.002). This was also reflected in the OS cells' responses to the 510 checkerboard flashes, with slightly lower full-field rectification (median = 0.57, n = 53) and convexity 511 indices (median = 0.20, n = 53) as compared to DS cells, yet with both indices significantly larger 512 than zero (Wilcoxon sign-rank test, p < 10 -9 for rectification and p = 0.002 for convexity indices). 513 Examining the distributions of these measures for OS cells more closely, we observed that they 514 actually appeared to be bimodal ( Figure S4A), which may indicate that different types of OS cells 515 differ in how nonlinear their spatial integration is. Indeed, we found that ON-type OS cells showed 516 fairly linear spatial integration characteristics ( Figure S4A), while OFF-type OS cells could be nicely 517 clustered into two separate groups, one with linear spatial integration and good LN model 518 performance and another with nonlinear spatial integration and poorer LN model performance 519 ( Figure S4A-B). Interestingly, the linear and nonlinear OFF OS cells also differed systematically in 520 their preferred orientations ( Figure S4C). We may thus speculate that the previously described classes 521 of OS cells with different contrast preference or preference for horizontal or vertical orientations natural stimuli are consistent with a linear RF. We found that this is the case only for a subset of cells 527 in the mouse retina under natural images. With targeted artificial stimulation, we were able to link 528 these differences in natural image encoding to different degrees of nonlinear spatial integration inside 529 the RF center in different cell classes.

A cell class with particular sensitivity to spatial homogeneity of natural images 560
We identified cells in the mouse retina with particular sensitivity to spatially homogeneous image 561 parts. Specifically, these cells were inversely sensitive to spatial contrast: though well described by an 562 LN model, they respond more strongly to homogeneous stimuli than to structured stimuli of equal 563 light level. This feature is not to be confused with the characteristics of suppressed-by-contrast cells 564 (Jacoby and Schwartz, 2018), which are suppressed below baseline activity by (temporal) contrast, 565 whereas the homogeneity-preferring cells here are particularly strongly activated by homogenous 566 spatial stimuli. This is reminiscent of the homogeneity detectors that have been described in the 567 salamander retina (Bölinger and Gollisch, 2012), although the latter showed rectification of non-568 preferred contrasts, unlike the homogeneity-sensitive cells described here. Note also that these cells, 569 through their particular sensitivity to homogeneous stimuli, could provide information about image 570 focus; blurring through de-focusing will increase activity for this cell type and simultaneously 571 decrease activity for spatial-contrast-sensitive cells, such as ONdelayed cells (

Mechanisms of nonlinear spatial integration in natural image responses 584
The central nonlinear operation governing spatial integration appears to be signal rectification, with 585 We used flashed image presentations because our study was focused on spatial integration. Yet, this 626 approach is insensitive to nonlinearities that become relevant when the temporal structure of stimuli is 627 considered. For example, IRS cells, which we here reported as being rather linear for the encoding of 628 images flashed in isolation, can reveal nonlinearities when rapid image transitions are considered, for 629 which disinhibitory interactions mediate a sensitivity to recurring spatial patterns (Krishnamoorthy et 630 al., 2017). Additionally, we focused on a single light level, but spatial nonlinearities may change with 631 light level: sustained ON-alpha ganglion cells in the mouse retina, for example, become more linear 632 with decreasing light intensity (Grimes et al., 2014). 633 Since we presented natural images in a full-field fashion, we furthermore need to consider RF 634 surround effects that likely also contribute to differences between cells in natural stimulus encoding. 635 A study in the primate retina, for example, showed that surround activation can alter spatial 636 integration in the RF center (Turner et al., 2018). We cannot exclude that disregarded surround effects 637 We used 13 retina pieces from 9 adult wild-type mice of either sex (6 C57BL/6J and 3 C57BL/6N, 7 684 male and 2 female), mostly between 8-12 weeks old (except for one 18-and one 26-week-old). All 685 mice were housed in a 12-hour light/dark cycle. Experimental procedures were in accordance with 686 national and institutional guidelines and approved by the institutional animal care committee of the 687 University Medical Center Göttingen, Germany. 688

Tissue preparation and electrophysiology 689
Mice were dark-adapted for at least an hour before eye enucleation. After the animal had been 690 sacrificed, both eyes were removed and immersed in oxygenated (95% O2-5% CO2) Ames' medium 691

Visual stimulation 712
Visual stimuli were generated and controlled through custom-made software, based on Visual C++ 713 and OpenGL. Different stimuli were presented sequentially to the retina through a gamma-corrected a background of low photopic light levels (2.5 or 3.5 mW/m 2 , corresponding to 6.3x10 3 or 7.9x10 3 718 R*/rod/s), and their mean intensity was always equal to the background. We fine-tuned the focus of 719 stimuli on the photoreceptor layer before the start of each experiment by visual monitoring through a 720 light microscope and by inspection of spiking responses to contrast-reversing gratings with a bar 721 width of 30 μm.Linear receptive field identification 722 To estimate the RF of each cell, we used a spatiotemporal binary white-noise stimulus (100% 723 contrast) consisting of a checkerboard layout with flickering squares (60 μm side). The update rate 724 was either 30 or 60 Hz in different experiments. We measured the spatiotemporal RF by calculating 725 the spike-triggered average (STA) over a 500-ms time window (Chichilnisky, 2001), and fitted a 726 parametric model to the RF (Chichilnisky and Kalmar, 2002). The model was spatiotemporally 727 separable and comprised a product of a spatial ( ( )) and a temporal component ( ( )). 728 The spatial component was modelled as a difference of Gaussians: surround strength relative to the RF center, and ≥ 1 is a scaling factor for the surround's extent. 733 The temporal component was modelled as a difference of two low-pass filters: 734 smaller than 25% of the largest deflection. The diameter of the RF center was defined as the diameter 748 of a circle with the same area as the 2σ (elliptical) boundary of the Gaussian center profile (Baden et 749 al., 2016). We also used the 2σ boundary for all RF center visualizations. 750

Natural image response predictions with a linear-nonlinear model 751
We selected natural images as stimuli from three sources: the van Hateren Natural Image Dataset (van 752 values so that the mean pixel intensity was equal the background and the standard deviation was 40% 760 of the mean intensity. Pixel values that then deviated from the mean by more than 100% in either 761 direction were clipped to ensure that the maximal pixel values were within the physically available 762 range of the display. Finally, all images were encoded at 8-bit color depth, to match the range of our 763 OLED monitor. The images were presented on top of a uniform grey background and centered on the 764 multielectrode array, covering a region of 3.84x3.84 mm 2 on the retina (3x3 mm 2 for 400x400 pixels). 765 In every experiment, we used 300 natural images (100 from each database), except for one (200 766 images in total). Images were presented individually for 200 ms each, with an 800-ms inter-stimulus-767 interval of homogeneous background illumination. We collected ten trials for each image, by 768 consecutively presenting ten different pseudo-randomly permuted sequences of all images. For each 769 cell, we measured the response as the trial-averaged number of spikes over a 250-ms window 770 following stimulus onset. We estimated LN model performance through ten-fold cross-validation. Briefly, the collection of 805 average responses for all images was randomly split into ten equally-sized sets. Every set was used 806 once as a test set for the full LN model, whose nonlinearity was fitted to the other 90% of image 807 responses. For each cell, LN model performance was defined as the average CC norm over all cross-808 validation sets. For all nonlinearity visualizations in the plots, we used the nonlinearity corresponding 809 to the cross-validation set with the CC norm value closest to the average. for identical images were highly variable. For each cell, we therefore calculated the coefficient of 812 determination (R 2 ) between responses averaged over even ( ) and over odd trials ( ), where m = 813 1,…,M enumerates the images. Concretely, we used a symmetrized R 2 , defined as 814 where and are the average odd and even trial responses over images. We excluded cells with 816 R 2 <0.5 from further analysis. This criterion included 959 out of 1209 recorded cells in the analysis. 817

Calculation of spatial contrast sensitivity for natural images 818
We measured the spatial contrast (SC) of an image in the RF center of a given ganglion cell as the 819 weighted standard deviation of pixel contrast values inside the 2σ contour of the Gaussian center fit: 820 where the sums run over all pixels within the 2σ contour, is the pixel value, is the pixel weight 822 as given by the value at the pixel center of the fitted RF center part, and is the weighted mean of 823 the pixel values. 824 To obtain the spatial contrast sensitivity, we sorted the images according to their linear predictions in 825 the cell's LN model and then grouped neighboring images into pairs, with each image belonging to a 826 single pair, yielding 150 pairs per cell when 300 images had been applied. For each image pair, we 827 calculated the spatial contrast difference and the trial-averaged response difference. To compare 828 across cells, we normalized the response differences by the maximum response (over images) of the 829 cell. We defined the spatial contrast sensitivity as the slope of the linear regression between the spatial 830 contrast differences and the normalized response differences. Cells were defined as contrast sensitive 831 if they had a significant regression slope at the 5% significance level. 832

Assessment of spatial nonlinearity with contrast-reversing gratings 833
To compare our findings to classical analyses of spatial integration, we stimulated the retina with full-834 field square-wave gratings of 100% contrast. The contrast of the gratings was reversed every 1 s. The 835 reversing gratings were presented sequentially from higher to lower spatial frequencies for 20-30 836 reversals each, and the whole sequence was repeated two times. Depending on the experiment, we 837 sampled five to eight spatial frequencies, with bar width ranging from 15 to 240 μm. For each spatial 838 frequency, we applied one to four equidistant spatial phases, with more phases for lower spatial 839 frequencies (e.g. one for 15, two for 30, two for 60, four for 120, four for 240 μm bar width). In some of the recordings, we also included contrast reversals of homogeneous illumination (corresponding to 841 a bar width of 6 mm or larger). Between presentations of the different gratings, there was a grey 842 screen at background intensity for 2 s. We constructed PSTHs over one reversal period by binning 843 ganglion cell spikes with 10-ms bins and averaging across reversals and repeats, leaving out the first 844 reversal after a gray period. In order to exclude cells with unreliable responses, we calculated R 2 845 values between average response vectors of even and odd trials, similar to the analysis of natural-846 image responses. We created the response vector of a single trial by concatenating single-trial PSTHs 847 from all different spatial frequencies and phases. We only considered cells with R 2 >0.1 for our 848 population analyses. The criterion was satisfied by 890 out of 1126 cells recorded for this stimulus. 849 To estimate the grating spatial scale for each cell, we extracted the peak firing rate in the PSTH 850

Assessment of spatial input nonlinearities with checkerboard flashes 863
To assess how local visual signals are transformed in nonlinear cells, we used a stimulus that had a 864 checkerboard layout with square tiles of either 105 or 120 µm to the side. The tiles were alternatingly 865 assigned to two sets (A and B) so that neighboring tiles were in different sets. For each individual 866 stimulus presentation, each set of tiles was assigned an intensity or , respectively, expressed as 867 the Weber contrast from background illumination. Similar to our presentation of natural images, these 868 checkerboard stimuli were flashed for 200 ms with an inter-stimulus interval of 800 ms, during which 869 background illumination was presented. The contrast pairs ( , ) were selected from a two-870 dimensional stimulus space organized in polar coordinates, by using 24 equidistant angles, each with 871 10 equidistant radial contrast values (√ 2 + 2 ) between 3-100% (see Figure 4A sensitive cells as cells with RI < 0 and CI < 0, corresponding to iso-response contour lines curving 907 towards the origin. 908 To probe spatial integration in the RF center with minimal surround influence, we used local 909 checkerboard flashes. The stimulus was similar to the one above, but with small patches of 2x2 tiles. For analysis, we selected for each ganglion cell the patch closest to its RF center and extracted the 924 responses to flashes when this particular patch was used. We counted the number of spikes over a 925 250-ms window following presentation onset and again subtracted the background activity, which was 926 here obtained by interpolation to the (0, 0) contrast pair. Response contour lines in stimulus space as 927 well as rectification and homogeneity indices were calculated in the same way as for the full-field 928 version of the checkerboard flashes. Similarly to the full-field stimulus, we calculated R 2 values odd-and even-trial averages and assigned this value to a blurring scale of 0 μm. We then fitted 944 logistic functions to the R 2 values with respect to the blurring scales. We defined the natural spatial 945 scale for each ganglion cell as the midpoint of the fitted logistic function. Again, by requiring R 2 >0.1 946 for odd-versus even-trial averages of the original images, we included 747 cells out of 850 for which 947 we had recorded the stimulus. 948

Detection of image-recurrence-sensitive cells 949
We detected image-recurrence-sensitive (IRS) cells as described previously (Krishnamoorthy et al., We also used drifting square-wave gratings of 100% contrast, 225 µm spatial period, and a temporal 978 frequency of 4 Hz to identify orientation-selective ganglion cells (Nath and Schwartz, 2016, 2017). 979 The gratings were shown in a sequence of eight equidistant directions with twelve periods per 980 direction, separated by 2 s of background illumination. The sequence was repeated four to five times. 981 We calculated an orientation selectivity index (OSI) as the magnitude of the complex 982 sum ∑ 2 ∑ ⁄ . The preferred orientation was obtained as the line perpendicular to half the 983 argument of the same sum. 984 To calculate the statistical significance for both indices, we used a Monte Carlo permutation approach 985     linear vs nonlinear OFF OS cells systematically differed, we compared the distributions of differences in 1208 preferred orientation for pairs of OS cells that belonged to either the same group ("within-group") or to different