Abstract
Saccadic eye movements direct the high-resolution foveae of our retinas toward objects of interest. With each saccade, the image jumps on the retina, causing a discontinuity in visual input. Our visual perception, however, remains stable. Philosophers and scientists over centuries have proposed that visual stability depends upon an internal neuronal signal that is a copy of the neuronal signal driving the eye movement, now referred to as a corollary discharge (CD) or efference copy. In the old world monkey, such a CD circuit for saccades has been identified extending from superior colliculus through MD thalamus to frontal cortex, but there is little evidence that this circuit actually contributes to visual perception. We tested the influence of this CD circuit on visual perception by first training macaque monkeys to report their perceived eye direction, and then reversibly inactivating the CD as it passes through the thalamus. We found that the monkey's perception changed; during CD inactivation, there was a difference between where the monkey perceived its eyes to be directed and where they were actually directed. Perception and saccade were decoupled. We established that the perceived eye direction at the end of the saccade was not derived from proprioceptive input from eye muscles, and was not altered by contextual visual information. We conclude that the CD provides internal information contributing to the brain's creation of perceived visual stability. More specifically, the CD might provide the internal saccade vector used to unite separate retinal images into a stable visual scene.
SIGNIFICANCE STATEMENT Visual stability is one of the most remarkable aspects of human vision. The eyes move rapidly several times per second, displacing the retinal image each time. The brain compensates for this disruption, keeping our visual perception stable. A major hypothesis explaining this stability invokes a signal within the brain, a corollary discharge, that informs visual regions of the brain when and where the eyes are about to move. Such a corollary discharge circuit for eye movements has been identified in macaque monkey. We now show that selectively inactivating this brain circuit alters the monkey's visual perception. We conclude that this corollary discharge provides a critical signal that can be used to unite jumping retinal images into a consistent visual scene.
Introduction
We benefit from rapid saccadic eye movements that direct the high-resolution foveae of our retinas toward objects of interest (Fig. 1A). Unfortunately, such rapid eye movements, which occur several times per second, displace the image on the retina, and so should produce a perceived jump of the visual scene like the jerks so frequently seen in home videos. Our visual perception, however, remains stable because our brain compensates for these disruptions.
This perceptual stability almost certainly results from not one but a combination of compensations executed by extensive circuits in the brain (Wurtz, 2008). Whatever the final perceptual stages of such compensation may be, an initial step must be deriving the perceived amplitude and direction of each saccade. For example, the saccades illustrated in Figure 1A can be represented by a vector (white arrows) where each vector produces a new image centered on the fovea (Fig. 1B). Thus, vision is a continuing repetition of the perceived saccadic vector and a new visual image falling on the fovea. If both the saccade vector and the resulting image centered on the fovea are known, the visual scene can be reconstructed (Fig. 1C). This is true whether the mechanism for visual stability is a remapping of a retinotopic system or a conversion to a spatiotopic map (Wurtz, 2008).
Much is known about the image processing underlying perception, but little is known about the source of the vectors connecting successive retinal images. One possibility proposed by philosophers and scientists from Descartes to Helmholtz (Grüsser, 1995) is that signals within the brain provide the information needed to monitor ongoing movements. This internal information has come to be known as corollary discharge (CD) or efference copy (Sperry, 1950; Von Holst and Mittelstaedt, 1950). Each time a saccade occurs, a CD copy of the actual saccade vector driving the eye is sent to other brain regions related to visual perception to inform them of the impending saccade (Fig. 1D, right). Recently, a CD for saccades has been identified in the Rhesus monkey, an animal with visual brain anatomy and function remarkably similar to that in humans (Orban et al., 2004). This CD copy of the actual saccade vector travels in a circuit (Fig. 1D, left) from superior colliculus (SC) to the medial dorsal (MD) region of thalamus, and then to the frontal eye field (FEF) in frontal cortex (Sommer and Wurtz, 2002, 2004a, 2008). A role for this CD in controlling movement has been established by showing that disruption of the CD circuit degrades a monkey's ability to guide rapid sequences of saccades when visual input is not fast enough to guide them (Hallett and Lightstone, 1976; Sommer and Wurtz, 2004b). The relationship of the CD to motor control is compelling enough that several commentaries have concluded that the CD is probably used for motor control (Bays and Husain, 2007) or for the selection of saccade targets (Zirnsak and Moore, 2014). So far no direct evidence has been presented that CD contributes to perception (for review, see Higgins and Rayner, 2015).
We address whether the established CD circuit from superior colliculus to frontal cortex actually contributes to visual perception. Our first task was to get the monkey to report where it perceived its eye to be directed at the end of a saccade (Fig. 1B, arrows). We did this using a method developed in human psychophysical experiments (Deubel et al., 1996). We then located and reversibly inactivated the CD at its relay in the lateral edge of MD, and found that this altered the monkey's perceived saccade vector. Finally, we established the dominance of the CD signal over both proprioceptive and visual influences. We conclude that the vector provided by the CD of each saccade provides the critical internal vector used to eventually unite the jumping retinal images into a stable visual scene.
Materials and Methods
Two adult male monkeys (Macaca mulatta, Monkeys C and W), weighing between 9 and 10 kg, were implanted with scleral search coils for measuring eye position, a post for immobilizing the head during experiments, and a recording cylinder to record from and inactivate regions of the thalamus. Details of these procedures, the task, and the training procedures have been described previously (Sommer and Wurtz, 2000; Joiner et al., 2013). All procedures were approved by the Institute Animal Care and Use Committee and complied with Public Health Service Policy on the humane care and use of laboratory animals.
Psychophysical task.
We used a task that allowed us to infer differences between the monkey's perceived saccade vector and its actual saccade vector. We did this by measuring the monkey's postsaccadic judgments of the locations of presaccadic targets. The task we used was adapted from that developed for humans by Deubel et al. (1996, 1998, 2002), and then used in other human experiments (Collins et al., 2009; Ostendorf et al., 2010), and finally adapted for the monkey (Joiner et al., 2013).
On each trial the monkey looked at a red fixation spot generated by a laser (∼0.1° diameter) at the center of a tangent screen 58 cm in front of it. The monkey fixated for 500 ms until a peripheral target (another laser spot) appeared 8° either to the left or right of the fixation spot (Fig. 2A). The fixation spot went off after a random interval of 500–1000 ms, cueing the monkey to make a saccade to the target. During the saccade, we displaced the saccade target up to 2° to either the left or the right of the target's original position. We refer to these left and right target displacements as either being forward (in the direction of the saccade) or backward (in the opposite direction). The displacement (up to 2° maximum in either direction) varied randomly across trials, and consisted of the target disappearing after saccade initiation, and then reappearing in the new location before the end of the saccade. We displaced the target during the saccade because thresholds for detecting displacement are elevated during saccades; transients associated with target displacement (ie, apparent motion) are not detected (Bridgeman et al., 1975).
Targets were displaced up to 2° forward or backward in either 0.2° or 0.3° increments. On any given trial, the target displacement was chosen by first generating a normally distributed random number with a mean of 0° and SD of 1.3°, then selecting the displacement that was closest to that random number. Displacement locations chosen from this distribution were truncated at 2°, the maximum target displacement allowed in either direction. This insured that smaller target displacements would be more frequent, quickly providing data to produce a significant psychometric curve. During all trials, the monkey was in the dark so there were no visual spatial cues available, and between trials a background light was turned on to minimize dark adaptation during the experiment.
At the end of the saccade, the monkey indicated the direction of the target displacement (left or right) by moving a bar in the same direction (left or right) to obtain a liquid reward. Over a series of trials, we plotted the frequency of “forward” or “backward” reports for each displacement. From the resulting psychometric curve (shown schematically in Fig. 2B) we determined the point at which the monkey moved the bar with equal frequency (50%) forward and backward, or put another way, the point at which the monkey perceived no target displacement during the saccade. This is the perceptual null location, indicating the monkey's perception of the presaccadic target location made after the end saccade of the saccade. We also measured the perceptual threshold: the distance in degrees on the psychometric curve from the perceptual null point to the point where the monkey reports 75% forward displacements.
It should be noted that monkeys received significant training on the task before the collection of experimental data. Initial training was complete when the monkey could dissociate saccade direction and target displacement. For example, when the monkey reliably made correct perceptual reports (>75%) for an obvious leftward target displacement (2°) for a rightward saccade, it was considered trained on the task. Overtraining was then done to reduce the variability in the psychometric curve from day to day. Monkeys did ∼300–600 correct trials in a given experimental session.
A variation used in the human task was that the displaced target came on either during the saccade or after a 200 ms gap (Deubel et al., 1996), long after the saccade had ended. In our previous behavioral experiments, monkeys performed no differently whether we presented the displaced target during the saccade or after a delay (Joiner et al., 2013), which we assume is a consequence of the substantially greater training received by monkeys. In the present experiments, the displaced target always reappeared during the saccade, not afterward.
The overall timeline for each monkey was as follows: initial surgery for the eye coil and head implant, preliminary training on simple fixation and saccade tasks, training on the psychophysical task implementing manual bar responses (4–9 months), second surgery for the recording cylinder and craniotomy, initial MRI for location of the thalamic target, verification of location by neuronal recording, and finally muscimol inactivations while performing the psychophysical task.
Significance tests.
For determining the significance of perceptual offsets (POs) and changes in thresholds during MD inactivation, we first fit the data to a simple psychometric curve (a cumulative Gaussian) by minimizing the (negative) log-likelihood of the curve parameters (Wichmann and Hill, 2001). We then used a likelihood-ratio test to test for significant differences between the curves. Because the perceptual offset is the difference between the perceptual null locations during control and inactivations, the significance of the difference between perceptual null locations due to MD inactivation is equivalent to the significance of the perceptual offset. We first fit the control and inactivation data from each hemifield to an unconstrained pair of cumulative Gaussian curves. In each hemifield, for both the control and the inactivation we allowed the perceptual null location and threshold to change, obtaining the log-likelihood for the fitted perceptual null location and threshold parameters (the mean and SD of the cumulative Gaussian). We then fit the same control/inactivation data to a pair of curves that shared the same perceptual null location parameter, obtaining a second log-likelihood for the new fit with no change in perceptual null location. We calculated the difference between the two log-likelihoods, which ends up being χ2 distributed with one degree of freedom (because of the difference in the added perceptual null location parameter between fits). (The ratio of the likelihoods is also χ2 distributed, but obtaining the log-likelihoods during fitting is more straightforward). The χ2 difference in log-likelihoods represents the significance of the difference between the two perceptual null locations (PNLs): the perceptual offset. Therefore, the p value derived from this χ2 difference represents the significance of the perceptual offset from MD inactivation.
For changes in threshold due to MD inactivation, we performed a similar analysis, this time using the threshold values between control and inactivation curves.
We also used an extended version of the psychometric function in which the lower and upper asymptotes were not fixed at zero and one, but were permitted to vary. To compare the relative contributions of varying perceptual null locations and varying asymptotes, we again used the likelihood-ratio test. We fit the control/inactivation data to a pair of curves in which only the asymptotes were permitted to change. We compared the log-likelihood from this fit to that when the PNLs were permitted to vary between curves as well. A significant difference in the log-likelihoods meant that varying the PNLs significantly improved the fit. The significance of varying the asymptotes was computed in a manner similar to that of varying the perceptual null locations.
To measure the change in the monkey's behavior in a parameter-independent manner, we calculated a performance asymmetry index that quantified the difference in the monkey's error rates for forward and backward target displacements (see Fig. 4D). We used unconstrained fits to the extended psychometric function above to characterize the data, and calculated the integral below the psychometric function (from −2° to 0° for backward displacements) and above the psychometric function (from 0° to 2° for forward displacements). Each integral could range from 0 to 2, so we divided by 2 resulting in integrals that ranged from 0 to 1. The asymmetry index was simply the normalized integral above the curve (for forward displacements) minus the integral below the curve (for negative displacements). This yielded an index from −1 to 1 that represented the asymmetry in the monkey's error rates between forward and negative displacements, in effect quantifying the change in the psychometric function. Negative values of the asymmetry index suggest backward shifts in the psychometric function (regardless of its actual parameterization), whereas positive indices suggest forward shifts.
Differences in saccade endpoints between control and inactivation days were calculated using a two-dimensional Kolmogorov–Smirnov test.
To examine correlations between saccadic eye movements and perceptual judgments, we first classified trials based on the amplitude of the saccade. Taking trials only with target displacements >0.5° (Joiner et al., 2013; see Fig. 6 for explanation of the 0.5° threshold), we binned saccade amplitudes into 10 bins with equal numbers of trials, and then calculated the proportion of forward responses in each bin. If the monkey accounts for its eye position using the CD, there should be little correlation between saccade amplitude and perceptual judgment. In each hemifield, we calculated the change in the strength of correlation (the absolute value of the correlation coefficient) between controls and inactivation.
Saccadic eye movements.
Eye position was sampled at 1000 Hz. At the start of each trial, monkeys were required to be within a 1.5° square around the fixation point. After the saccade to the target, only trials where the saccade ended within 5° of the original target location were analyzed. Saccade initiation was identified offline as the time that eye velocity and acceleration both exceed 100°/s and 5000°/s2, respectively.
The procedure for calibrating eye position was minimized to maximize task trials. Calibration concentrated on the location of fixation. We qualitatively deemed the peripheral eye positions to be sufficient if the saccade endpoints before the inactivation were well within the 5° target window. Although this imperfect calibration introduced day-to-day differences in saccade endpoints, the speediness of the calibration procedure yielded substantially more experimental trials.
One monkey had occasional corrective saccades to the targets after they appeared in their displaced locations. Such saccades were even rarer in the second monkey. When they did occur, they were to the displaced target, and so were visually guided. As these corrective saccades were either absent or not related to perceptual judgments, we do not consider them further.
Grid-aligned MRI localization of inactivation sites.
One of the greatest challenges in these experiments was localizing the appropriate area of MD identified previously as a relay in the CD circuit between SC and FEF (Sommer and Wurtz, 2004a). We achieved this through a combination of structural MRI identification and single neuron recording. The first step was to review the histology comprising fiber- and cell-stained sections from Sommer and Wurtz (2004a). We located the sections with penetration tracks in a standard atlas (Paxinos et al., 2000) using multiple landmarks: the claustrum, the putamen, the tail of the caudate nucleus, the lateral geniculate nucleus, and the cortical patterns of gyri and sulci. We then located the corresponding slices in the structural MRIs that matched both the histology and the atlas using the same landmarks. We digitally overlaid both the histology slides and the atlas sections on the corresponding MRI slices to locate the lateral edge of MD thalamus.
The MRIs were made with a contrast agent filling the recording grid so that the geometry of the grid and chamber was evident. Once the area of interest was located in the MRIs, we easily determined the A-P and M-L grid locations targeting the desired area.
We recorded from the target area with standard recording techniques, and identified the relay in MD by the presence of neurons that began their activity before the onset of the saccade. We found such neurons in a very limited area extending usually only 2–3 mm anterior to posterior and 1–2 mm medial to lateral.
After establishing our location in the desired region of MD, we made injections in this area and verified the locations of the injections by recording from an electrode attached to the side of the injection syringe that protruded 0.5 mm beyond the tip of the syringe. Once we determined the injection site, we advanced the syringe to place the vertical center of the opening of the syringe needle at the desired location.
We developed a custom application, implemented as a plug-in for ImageJ (Schneider et al., 2012), to visualize injection sites in the structural MRIs. We first rotated the 3D MRI dataset in three dimensions so that the grid (which was visible in the MRI) was aligned to the rotated MRI volume. We selected the boundaries of the grid and initiated the custom plug-in, which then zeroed the image coordinate frame to the center of the grid. By entering the grid coordinates and depth of each injection, the plug-in calculated the x-, y-, and z-coordinates of each injection within the MRI, and placed a mark in the appropriate MRI slice at the location of the injection. We calibrated our method by entering the grid coordinates and depths of lesions we made by passing current through an Elgiloy electrode (Koyano et al., 2011), and confirming the alignment of the visible lesions with their marked locations.
Injection parameters.
Our primary data were derived from 22 muscimol injections in two monkeys. In Monkey W, we did eight injections in one hemisphere, and in Monkey C, we did eight injections in one hemisphere and six in the other. In addition, in Monkey C, we did two saline injections: one in each hemisphere at locations that had previously yielded significant results. This comprised a total of 24 injections.
We performed an additional 12 muscimol injections (7 in Monkey W, 5 in Monkey C) to examine the effect of interleaving trials performed in the light and in the dark. These were not included in the main analysis because of the halved number of trials the monkey performed in the dark.
We made muscimol injections of 0.25–0.35 μl (5 mg/ml) at sites that met the two criteria outlined above: (1) the site was in or near the region suggested by MRI analysis, and (2) we recorded neurons with presaccadic activity at the potential injection site. Our goal was to make small injections to minimize spread of muscimol into areas adjacent to our target. Injections were made over 6–10 min. using a computer-controlled syringe driver usually injecting 0.1 μl every 12 s.
Results
Perception changes during MD inactivation
We cannot directly measure the underlying CD vectors suggested in Figure 1B. However, we can infer the internal CD vectors from measurements of the perceived saccade vectors. We did so using the task described in Materials and Methods and in a previous report (Joiner et al., 2013). In each trial, the monkey moved a bar to indicate which direction it perceived a target to be displaced during the saccade. Target displacement occurred entirely during the saccadic eye movement (Fig. 2A). Judgments over many trials in this two-alternative forced-choice task produced a psychometric function (illustrated schematically in Fig. 2B) that shows the monkey's report of displacement direction (y-axis) as a function of actual displacement (x-axis, forward displacements are in the direction of the saccade). The point on the psychometric curve where the monkey reported forward and backward displacements with equal frequency was taken as the perceptual null point. We regard this point as the monkey's perception of the original target location; if the target was not perceived to move, it must be in the same location as it was before the saccade.
A possible solution for the problems that saccades present for stable visual perception. A, Saccades (lines) and fixations (dots) from a human subject viewing a fragment of the painting by Seurat, “A Sunday Afternoon on the Island of La Grande Jatte”. The white arrows represent three hypothetical saccade vectors. B, The foveal images at the end of each of the three saccade vectors. C, Reconstruction of the visual scene using just perception of the saccade vectors and the retinal image. D, A corollary discharge that could provide the saccade vectors. Arrows on the right indicate a CD vector to cortex that represents a copy of the movement vector generating the saccade. The circuit on the left outlines an identified CD in the monkey brain from saccade-related neurons in SC through a thalamic relay in the MD to FEF. We hypothesize that the CD informs frontal cortex how to arrange successive retinal inputs into a stable visual perception.
Measuring a monkey's perceived saccade vector. A, Behavioral task. At the start of each trial, when the fixation point went out, the monkey made a saccade to a target 8° either to the right or to the left of initial fixation. During the saccade, the target was displaced randomly (up to 2°) either forward or backward during the saccade. After the saccade to the original target, the monkey received a reward for manually moving a bar in the direction of the target displacement. B, Schematic of a possible psychometric function. The function shows the proportion of forward (in the direction of the saccade) judgments (y-axis) for each target displacement (x-axis). The PNL is the postsaccadic target location at which the monkey perceived no displacement. The threshold is the difference in displacement between 50 and 75% forward judgments.
Figure 3A shows the results of an example muscimol inactivation in MD thalamus. In this experiment, inactivation predominantly affected perception in the visual field ipsilateral to the inactivation (Fig. 3A, left). The black curve shows the psychometric function from the control day before the injection The perceptual null locations for this control day and for the day after the inactivation (data not shown) were near zero, and their difference (0.16°) was not significant (p = 0.26, likelihood ratio test). The ipsilateral psychometric function during inactivation (red curve) was shifted backward from the direction of the saccade (in the negative direction on the horizontal axis) compared with controls. The perceptual null location during MD inactivation was −0.77° and was significantly different from both control days (p ≤ 0.001).
Shifts in perception during MD inactivation in 24 experiments. A, An example of psychometric curves from MD inactivation (red) compared with the day before (black). Data from which the fit curves were derived are shown as the points of the same color (±SEM). The perceptual offset, the difference in perceptual null locations between controls and inactivations, was significant for saccades into the ipsilateral field (left) but not for those into the contralateral field (right). Note that we plot the horizontal axis from negative to positive, in the direction of the saccade, for both the ipsilateral and contralateral graphs. B, Perceptual null location during inactivation (y-axis) versus perceptual null location on control days (x-axis) for all 24 inactivations. Symbol shape indicates monkey (square is Monkey C, circle is Monkey W), and symbol color denotes significance of the change in perceptual null location. Circled points represent the example in A. Perceptual null locations were typically greater during inactivations, indicated by the vertical spread of the data. Points nearer the origin are those that showed little effect, although these inactivations were in the same area as inactivations that showed greater effects. C, Relationship between perceptual offsets in ipsilateral and contralateral hemifields. The x-axis shows perceptual offsets for saccades to targets in the ipsilateral visual field, and the y-axis shows perceptual offsets for targets in the contralateral hemifields. The circled point represents the experiment illustrated in A. MD inactivation altered the perceived saccade vectors to variable degrees in both ipsilateral and contralateral visual hemifields.
Although modification of the CD produced a clear and significant perceptual offset in the ipsilateral hemifield in this case, no significant offset was evident for judgments in the contralateral visual field (Fig. 3A, right). In addition, there was no significant difference from inactivation in the slopes of the curves (ie, changes in perceptual threshold). Other inactivations did, however, show changes in threshold, but these did not correlate with changes in perceptual offset (r = −0.12, p = 0.39).
The perceptual offset represents the difference in the monkey's perceived target location before and during CD modification. This difference, we submit, is equivalent to the difference in the monkey's perceived saccade vector before and during CD modification. This is true, however, only if these differences do not correlate with variations in the actual saccade vector. Without this assumption, the difference between the perceived saccade vector and the actual saccade vector could be due to changes in either one or the other. This assumption is tested in the next section, in which we will verify that perceived changes in target locations are not correlated with changes in the actual saccades.
Figure 3B shows the perceptual null locations during all 24 MD inactivations (including 2 saline injections) during controls (x-axis) and during injections (y-axis). Each point represents a separate inactivation, and positive values represent perceptual null locations displaced forward (in the direction of the saccade). The dispersion of the data along the y-axis compared with the x-axis shows that perceptual null locations were frequently farther from zero during MD inactivation than on control days.
Previous inactivations of the CD in MD produced deficits primarily in the visual field contralateral to the injection site (Sommer and Wurtz, 2004a,b), but clearly the sample in Figure 3B show changes in both ipsilateral and contralateral visual fields. Figure 3C compares the effects in the two visual fields by plotting the perceptual offsets with each inactivation. These perceptual offsets are the differences between controls and inactivations in Figure 3B. We have converted the forward/backward perceptual null locations from Figure 3B into ipsiversive/contraversive perceptual offsets for Figure 3C so that for both sides, positive values indicate ipsiversive perceptual offsets (those toward the ipsilateral side). Inactivations with ipsiversive perceptual offsets in both hemifields fall in the upper right quadrant. Commensurate contraversive perceptual offsets fall in the lower left quadrant. Such matching was frequently the case as indicated by the correlation between perceptual offsets in both visual hemifields (least-squares regression shown, r = 0.70, p < 0.0001). Note, however, that some inactivations did not have perceptual offsets in the same direction (Fig. 3C, top left, bottom right).
This range of observations is consistent with the finding that some FEF neurons receive information from both contralateral and ipsilateral SC, and some from one or the other, and that the cross-hemifield interactions could first occur in thalamus (Crapse and Sommer, 2009). A variety of visual field deficits would therefore be expected following MD inactivation, and a variety were found both in monkeys (Sommer and Wurtz, 2004b) and humans (Gaymard et al., 1994; Bellebaum et al., 2006; Ostendorf et al., 2010, 2013).
In the example shown in Figure 3A, the responses for contraversive target displacements (negative ipsilateral displacements and positive contralateral displacements) are more variable than the responses for ipsiversive displacements. Because of this variability, the individual response points do not reach the canonical psychophysical asymptotes of zero (ipsilateral) or one (contralateral). For this reason, we next examined the possibility that our data might be better represented by a psychometric function with upper and lower asymptotes that were free to vary, rather than being fixed at zero or one. Figure 4A shows that allowing such variation changes the curves for contraversive displacements. Figure 4B shows that a consequence of adding the asymptote parameters was to reduce the differences in perceptual null locations between control and inactivation. Therefore the correlation we initially observed between ipsilateral and contralateral perceptual offsets (Fig. 3C) was not present when the asymptotes in the fits were allowed to change (Fig. 4C).
Alternate fits for the psychometric functions. A, Example of modified psychometric curves (fit to the same data as in Fig. 3A) in which the asymptotes were allowed to vary. Colors and symbols are as in Figure 3A. B, Reduced change in perceptual null locations extracted from the modified fits. Compare with Figure 3B. Circled points are from the same example experiment. C, Perceptual offsets (changes in PNLs) from the modified fits. The circled point is the same example experiment. Compare with Figure 3C. D, Asymmetry index captures changes in monkey's performance. Using the curves from A, we determined the normalized error rate by integrating the area either below the curve (for backward displacements) or above the curve (for forward). We subtracted the error rate for backward displacements (incorrect “forward reports”, the red area) from the error rate for forward displacements (incorrect “backward” reports, the blue area) to obtain the asymmetry index. Data are from the ipsilateral hemifield for the same example experiment. E, Comparison of asymmetry indices between control and inactivation. For each hemifield, asymmetry indices during controls are on the x-axis, inactivations on the y-axis. The data spread vertically, as in Figure 3B, indicating greater asymmetry during inactivations. Circled points are the same example experiment. Note that the point circled on the ipsilateral side is that represented in D. F, Comparison of perceptual offset and asymmetry index. Perceptual offsets are plotted on the x-axis, changes in asymmetry indices on the y-axis. Symbol color denotes hemifield. Our example experiment is again circled. Changes in asymmetry indices were highly and significantly correlated with perceptual offsets. Both this asymmetry index and our original measurement of perceptual offset similarly represent changes in the monkey's perceptual judgments from MD inactivation.
Using a likelihood-ratio test, we found that permitting the perceptual null points to change (modeling the perceptual offset) improved fits significantly in 16/24 experiments, and permitting the asymptotes to vary improved fits significantly in 14/24 experiments (p < = 0.01 in both cases). Qualitatively, visual inspection of the new fits confirmed that the varying asymptotes of the fit curves did not fit the region of response variability any better than the psychometric function with the asymptotes fixed at zero and one.
The monkey's behavior was definitely altered by MD inactivation, and both of our fitting procedures modeled the asymmetry in performance induced by MD inactivation. We therefore quantified the behavioral alteration in a more parameter-independent manner to take into account both the perceptual offsets and the asymptote shifts. We calculated a performance asymmetry index, which was simply the normalized ratio of errors during forward target displacements minus the ratio of errors for backward displacements (using the integrals of the fit curves, as shown in Fig. 4D for the same example data; see Materials and Methods for details). This yielded an index between −1 and 1. If all the monkey's errors were for backward target displacements, the index would be −1; errors only during forward displacements would yield an asymmetry index of 1. If the error rates were the same for forward and backward displacements, the index would be zero.
We chose to use the fit curves with the varying asymptotes (Fig. 4A) for calculation of the asymmetry index because the absolute error between the fit curves and the data were smaller than when the asymptotes were fixed (Fig. 3A), because of the greater number of parameters. However, it turns out that this choice made little difference in the asymmetry index as using the more constrained fits produced almost identical indices (mean difference −0.004 ± 0.026 SD, r = 0.99, p = 10−87).
Fig. 4E compares asymmetry indices during controls and during MD inactivation, and we once again observed the vertical spread of data as we did for the perceptual nulls in Fig. 3B. Moreover, the ipsilateral changes in the asymmetry index were highly correlated with those on the contralateral side (r = 0.69, p < 0.0002), as were the perceptual offsets in Fig. 3C. More importantly, Fig. 4F relates the changes in the asymmetry index to our original measurement of perceptual offset in Fig. 3C. This original measure is shown on the x-axis, and the change in the asymmetry index during MD inactivation is on the y-axis. The measurements were highly correlated (r = 0.98, p < 10−36). The slope of the regression line through the data was 0.40, indicating that the asymmetry index and the perceptual offset similarly represent the shift in the psychometric function (within a proportional factor).
To summarize, inactivation of the MD relay in the CD circuit from SC to FEF produces a shift of the psychometric function (the perceptual offset), resulting in a change in performance, as shown by the asymmetry index. We propose that the shift in the psychometric function results from changes in the internal CD signal representing the perceived saccade vector.
Lack of correlation between changes in perceived saccade vectors and actual saccade vectors
We have defined the perceptual offset as the change in the perceived saccade vector (based on the CD) from MD inactivation. We have explicitly assumed that the perceptual offsets result from changes in the perceived saccade vector, not changes in the actual saccade to the target, because our inactivations in MD should alter only the CD, not the actual motor command. We now test this assumption by examining whether saccade amplitudes are indeed independent of perceptual judgments.
We begin by looking at saccades to the target for our example experiment shown previously (Fig. 3A). Figure 5A shows saccade endpoints during the control periods compared with those during the inactivation, and we found that the mean endpoints (open and solid crosses ±1 SD) differed significantly between control and inactivation days. Because the control and inactivation experiments were almost always run on different days to maximize the number of trials that we could obtain, we next evaluated the variation in saccade amplitude between control days, where any difference would be due to normal variability in the eye position recording. The histograms at the top of Figure 5B were derived by comparing mean saccade amplitudes between each control day and every other control day. The resulting histogram represents the frequency of differences in saccade amplitude that result from normal day-to-day differences. The variability evident in the histograms was most likely exacerbated by our use of an abbreviated eye calibration procedure to preserve trials for the essential psychophysical judgments (see Materials and Methods).
Lack of significant correlations between perceptual offsets and changes in saccades. A, Saccade endpoints for control and inactivation from our example experiment. The means of the endpoints during controls (open gray symbols) and inactivations (solid gray) are shown by open and solid crosses, respectively (±1 SD). B, Differences in mean endpoints of saccades for all 24 inactivations (±1 SD). The top marginal histogram for each hemifield represents the distribution of differences in horizontal (x) saccade amplitude between all possible pairs of controls. The lower scatter plot shows the x and y differences in saccade endpoints between control and inactivation. Larger circles denote the example experiment in A. Colors of points denote the significance of the perceptual offset, from Figure 3B. The vertical dashed lines enclose the central 95% of each top histogram, so values outside the dashed lines indicate significant differences from controls at p < 0.05. Differences between control and inactivation days resembled the differences between pairs of control days. C, Lack of relationship between actual saccades and perceptual judgment. After binning saccade amplitudes in to 10 bins with equal numbers of saccades, we calculated the number of forward responses in each bin. We correlated saccade amplitudes with forward judgments to determine any relationship. We obtained correlations for controls and inactivations separately, and calculated the change in correlation with MD inactivation. The histograms show the distributions of these changes. Means of distributions in each hemifield are designated by the vertical dashed lines. Saccades were no more correlated with judgments during MD inactivation than they were during control days.
We next compared these normal variations with those seen between controls and inactivations. The scatterplot at the bottom of Figure 5B shows the differences in mean saccade endpoints (±1 SD) for all inactivation experiments. Many individual differences were significant, but when compared with the histograms just above, we see that the differences we observed between control and inactivation days were no different from the differences we observed between pairs of control days. The vertical dashed lines extending down encompass the central 95% of each histogram. Points falling outside these bounds indicate saccade amplitudes differing from normal saccade variability at p > 0.05 (there was one ipsilateral point). Therefore, although significant differences in saccade endpoint between control and inactivation were common, they were rarely greater than expected from the underlying variability evident from the controls.
A more important test would be to directly assess the relationship between saccades and perception. Such a test has been derived from the work of Collins et al. (2009) and Ostendorf et al. (2010) in humans, and Joiner et al. (2013) in monkeys. They observed that the subject's judgment of displaced target location was independent of eye position at the end of the saccade, suggesting that their subjects were reliably using their CD of the saccade, rather than visual information related to retinal position, for their perceptual judgments. We repeated this analysis for the current experiments.
We analyzed saccade amplitudes for all target displacements >0.5° (Joiner et al., 2013 provides a discussion and justification of the 0.5° threshold). We binned saccade amplitudes into 10 bins containing equal numbers of saccades, calculated the proportion of forward responses in each bin, and then correlated the saccade amplitudes with the proportions of forward judgments. We calculated the change in the strength of this correlation between controls and their corresponding inactivations to determine whether perceptual judgments during MD inactivation depended on the actual saccade amplitude. For ipsilateral and contralateral hemifields, Figure 5C shows the changes in correlation for all inactivations. In neither hemifield was there an overall change in the correlations between eye position and perceptual judgment during MD inactivation.
In summary, this final test confirms that MD inactivation changed perception by altering the perceived saccade vectors rather than the actual saccade vectors. Thus, the changes in the psychometric function are not due to changes in the actual saccade vectors.
Proprioception does not influence the perceived saccade vector
We must now determine whether the changes in the perceived saccade vectors during MD inactivation are due to modifying the internal CD signal or one of two other signals: proprioception, which is extra-retinal (like the CD), or vision, which might provide contextual information from the visual scene.
To influence proprioception arising from receptors in the eye muscles, the saccadic eye movements and the underlying muscle contractions must change so as to provide information about a new eye location at the end of the saccade. As just reported, changes in saccadic eye movements did not correlate with perceptual offsets. This contradicts the idea that changes in extra-retinal proprioceptive signals affect perceptual offsets. Another possibility, however, is that our injections might have spread to areas near MD that perhaps carry proprioceptive information. Neurons previously recorded near our area (Schlag et al., 1980; Tanaka, 2007) discharged before saccades, as did the ones we recorded. We also never observed neuronal firing rates changing in a step-like manner with saccades as is found in proprioception areas, such as somatosensory cortex (Wang et al., 2007; Xu et al., 2011). As we are not aware of any proprioceptive relays in the MD region we inactivated, and we never observed any neuronal activity consistent with proprioception, it is unlikely that our results are due to altering a proprioceptive signal. We conclude that our results cannot be attributed to proprioception.
Comparing the influence of CD to visual context
The stabilizing influence of large field visual images on visual perception has been emphasized by Gibson (1966) so it is possible that visual context might override the CD signal in determining the perceived saccade vector. Such an override was reported in a previous experiment, in which subjects with weakened eye muscles mislocalized targets in the dark, but localized quite well in the light (referred to as visual capture; Matin et al., 1982). We eliminated any contextual visual information in our experiments by keeping the monkey in the dark during trials, by using target spots produced by lasers to avoid incidental light from monitors, and by illuminating the screen between trials to avoid dark adaptation. To examine the effect of light and possible visual capture on changes in perception with MD inactivation, we randomly interleaved trials in total darkness with trials using an illuminated background during a second set of 12 experiments. This allowed us to determine whether the monkey automatically uses visual context to overcome any deficits in perceived saccade vectors.
We used three different illuminated backgrounds: a uniform lighted background (2 experiments, Monkey W), a stationary blurred grid pattern (2 experiments, Monkey W), and a random dot pattern (3 experiments, Monkey W; 5 experiments, Monkey C). The sparse random dot pattern extended horizontally with randomly placed 0.8° squares, the locations of which changed on each lighted trial. Our goal was to see whether the monkey would reflexively switch, during MD inactivation, to using the newly available visual information.
We consider two hypothetical results that could arise from providing visual context during 12 separate inactivations (Fig. 6A). The perceptual offsets from MD inactivation during the dark trials are on the x-axis. Perceptual offsets during light trials are on the y-axis. The open symbols on the diagonal represent no change from the illuminated background; perceptual offsets in the light are the same as those in the dark. The gray symbols on ordinate zero would be expected if visual context completely bypassed the modified CD; perceptual offsets in the light would be zero, regardless of perceptual offset in the dark. Any intermediate beneficial effect of the illuminated background would fall in the shaded gray areas.
Shifts in perceptual bias during MD inactivation did not depend on visual context. A, Two possible outcomes of adding visual context during MD inactivation. The x-axis indicates perceptual offsets in the dark, y-axis in the light. The x values are actual perceptual offsets taken from the interleaved dark trials. If there is no effect of added light, points will fall on the diagonal (white symbols). If the added light eliminates perceptual offsets, the points will fall on the horizontal (gray symbols). Intermediate benefits of visual context will fall in the gray areas. B, Actual effects of light on MD inactivations. The dashed lines are the least-squares linear fits to the data. The light backgrounds apparently only served to distract the monkeys, and provided no benefit in eliminating perceptual offsets during MD inactivation.
Figure 6B shows the actual results for the additional 12 inactivations. The dashed lines in Figure 6B are the least-squares linear regressions to the data. Note first that very few points fall in the gray regions (where visual context would reduce the offset). For the ipsilateral side, the least-squares line closely follows the unit diagonal; the trend in the ipsilateral data indicates that perceptual offsets with visual context were no different from those in the dark. In the contralateral hemifield (right), the least-squares line shows that visual context only distracted the monkeys; perceptual offsets in the light were typically worse than those in the dark.
To summarize, monkeys did not use visual context to their advantage in our task. We found no evidence of visual capture, or any indication that visual context overrides the perceived saccade vector determined by the CD.
Localization of inactivations
Locating our injection sites was critical because a key requirement was that inactivations were made at the same area of MD thalamus that had previously been demonstrated to be the relay in the CD circuit from SC to FEF (Sommer and Wurtz, 2004a,b). We located our injection sites in the desired brain area by using a custom grid-aligned MRI localization technique (see Materials and Methods). Figure 7A shows the close match between actual locations in the brain (3 Elgiloy electrode lesions) and the locations predicted by our grid-based method (3 red circles). Using this method we located all of our injections, which are shown for both monkeys (C and W) in a sample coronal section from each (Fig. 7B,D) and a horizontal summary section from each (Fig. 7C,E).
Location of injection sites in MD thalamus. A, Elgiloy electrode marks on a coronal MRI slice for Monkey C and their predicted locations (red circles) as determined by our grid-based MRI localization application. The larger panel shows a magnified version of the bounded region in the smaller panel. Scale bars, 5 mm. B, Location of injections sites in Monkey C collapsed onto one MRI coronal section showing the location of the injections at the outer edge of MD and the inner edge of the internal medullary lamina. The landmark nucleus on the MRI, the central lateral nucleus of the intralaminar nuclei, is in red. The smaller panel shows the entire coronal section, with a black box denoting the area of interest. The white dashed line in the smaller panel indicates the horizontal section used in C. The colors of the circles (representing injections) indicate the significance of the perceptual offsets for each injection; significances are based on Figure 3B. The black X symbols show recording sites where no injections were made because there was no evidence of presaccadic neuronal activity. C, Same injection sites from Monkey C on a horizontal section. The white dashed line indicates the location of the coronal section in B. D, E, Injections in Monkey W shown on coronal and horizontal sections, respectively. Organization and markings as in B and C. In both monkeys, the most effective inactivations were in lateral MD adjacent to the internal medullary lamina of the thalamus.
We found the injections to be near the lateral edge of MD adjacent to the internal medullary lamina. We verified that the current muscimol injections were in the previously studied area of MD by comparing the MRIs of the current monkeys with the histology from a monkey used in previous experiments (Sommer and Wurtz, 2004a). We also made injections only at sites where presaccadic activity was observed. We must also emphasize that our muscimol injections were deliberately small and were not expected to eliminate all the CD signals in MD, and of course, not all relevant CD signals may pass through MD.
We do not know whether this particular region of MD thalamus contains a spatial map of presaccadic activity, as is the case for the SC (Lee et al., 1988; Cavanaugh et al., 2012). However, if it does (as we surmise) then injections at different locations would affect different regions of the presaccadic activity map. We would therefore expect that target location, which was kept constant between experiments, and injection location, which varied for each experiment with grid position and depth in the brain, would often be misaligned. This misalignment might underlie the variability of injections affecting one or both hemifields, the variation of positive or negative POs, and the variety in the magnitudes of observed effects—all of which we have shown. It remains to be seen how this region of MD represents visual space, and current experiments in our laboratory are focused on this question.
Discussion
Each saccade is generated by a movement command, a vector indicating the amplitude and direction of the saccade, and this vector is represented in the activity of the saccade-related neurons in the SC. At the same time, the visual system must have a copy of that vector (a CD) to relate what is centered on the fovea to the larger visual scene (Fig. 1). Such a CD has been identified in the monkey. It originates in the same SC neurons that initiate the saccade, and is conveyed to frontal cortex by a circuit passing through MD thalamus.
We investigated whether this CD is the source of saccade vectors contributing to visual perception. Using monkeys trained to report their perceived saccade vector, and interrupting the identified CD for saccades, we found altering the CD signal changed the monkey's perception but not its actual saccade vector. To our knowledge, this is the first experimental evidence that a CD in the primate provides perceptual information.
One possibility is that these vectors provide a basis for the continuity of vision across saccades because they supply the necessary information for tying together the successive fovea-centered images. A possible neuronal mechanism for this might be anticipatory shifting receptive fields that occur at the time of a saccade, discovered in lateral intraparietal area (LIP) and FEF (Duhamel et al., 1992 in LIP; Umeno and Goldberg, 1997 in FEF). In these areas, as the activity in the receptive fields of some neurons decreases before the saccade, the activity increases at the site that the same receptive field will occupy after the saccade occurs (Kusunoki and Goldberg, 2003). This could provide a fading of an image at one location and its emergence at another, and produce continuity between the sequence of images across saccades.
Relation of perceived saccade vectors to stable visual perception
Although the contribution of the internal CD vector to perception does not alone provide a complete mechanism that explains how we achieve stable visual perception, establishing the source of the vectors does provide an ideal start for the subsequent computations that might underlie such stability. The CD is available as much as 100 ms before the saccade, it is a close copy of the actual movement command, and it is independent of proprioception and visual context. There are two prominent scenarios in which such a signal might be used to achieve visual stability (Wurtz, 2008). The first produces a supra-retinal spatial map that is updated with each saccade (Galletti and Fattori, 2002; Cicchini et al., 2013). This is essentially the scene reconstruction illustrated in Figure 1C. The evidence for this is sparse (Wurtz, 2008), but new methods of multichannel recordings of populations of neurons might well produce more compelling evidence for such a transformation from a retinotopic to a spatiotopic map. The second scenario is predictive remapping based on the observation of the shifting receptive fields that provide an anticipatory facilitation of visual activity at the future site of the receptive field after the saccade. The jump in the receptive field with the impending saccade can then be anticipated, and the perceptual consequences ameliorated in ways not yet understood. A CD is required for this anticipatory activity; the anticipatory sensitivity occurs in a limited region of the visual field that depends on the vector of the upcoming saccade. The only known source of an anticipatory vector is the CD (the perceived saccade vector) that is developing at the same time as the increased sensitivity in the future receptive field. In previous inactivation experiments, the anticipatory neuronal activity in cortex was reduced when the CD was interrupted in its passage through MD thalamus (Sommer and Wurtz, 2006). Our present experiments show that inactivating the same MD relay also changes the perceived saccade vector contributing to the perceived direction of gaze. Thus, we have anticipatory shifts in receptive fields of FEF neurons before saccades, ideas of how such shifts might lead to stability of perception, the demonstration that inactivation of a needed CD reduces the shifts, and now the demonstration that inactivation of the same CD produces a change in perception. These multiple factors are consistent with a contribution to visual stability, and several models have related these observations to possible brain circuits (Quaia et al., 1998; Hamker et al., 2011; Ziesche and Hamker, 2014), but the exact computations required to produce such stability remain to be revealed.
Interaction of CD, proprioception, and vision
Proprioception from the extraocular eye muscles could also provide extra-retinal information about the perceived saccade vector, but we found no evidence that the perceptual changes following Inactivation of MD were correlated with changes in saccades. Thus, our experiments indicate that at the end of the saccade the CD is the extra-retinal signal that represents the perceived saccade vector, not proprioception. This is consistent with the consensus that proprioception makes little contribution to information about each saccade (Wang and Pan, 2013).
Proprioception might, however, contribute to perception in a later period after the saccade ends. Previous experiments, in which the sensory fibers from the eye into the brain were cut, concluded that proprioception did not provide immediate information (Guthrie et al., 1983; Lewis et al., 2001), but could be used for longer-term adjustments (Steinbach, 1987; Lewis et al., 2001). Consistent with this longer-term role of proprioception, Poletti et al. (2013) developed a model that better predicted experimentally observed eye positions after a series of saccades when both proprioception and CD were taken into account. In our experiments, we obtained the monkey's report of eye direction soon after the saccade ended, but later reports might have provided information that included a larger contribution from the slowly developing proprioception signal (Xu et al., 2011). One possibility is that the CD provides the ends of the saccade vectors shown in Figure 1A, whereas the combination of CD and proprioception might provide the start location of the next saccade vector, thereby helping to eliminate any errors in perceived eye position that might accumulate over several saccades.
The other source of a perceived saccade vector could be vision. In previous experiments with partially paralyzed extraocular muscles in humans (Matin et al., 1982), a deficit in localizing landmarks was eliminated when switching from dark to light, a recovery referred to as visual capture. Our muscimol inactivations were done in complete darkness, with only galvanometer-controlled laser spots visible, but when we went to a lighted environment, the deficit persisted or even increased (Fig. 6). This increase might have resulted from the separation of actual and perceived gaze direction, which one might expect to be exacerbated in the light. In addition, our experiments involve a quantitative test immediately after each saccade, whereas the previous report (Matin et al., 1982) was a qualitative report over time, not related specifically to saccades. Obviously we have investigated only a tiny subset of visual contexts, but we see no indication that visual context takes over from the CD.
Implications for the function of the thalamus
The circuit from brainstem to cortex that we have studied passes through just a tiny fraction of the MD nucleus, which projects largely to frontal cortex (Kievit and Kuypers, 1977; Goldman-Rakic and Porrino, 1985). The fact that this limited region provides information about impending saccadic eye movements suggests that other larger regions of MD might carry information about impending skeletal movements or other internal functions (Sherman and Guillery, 2002). We have now shown that this information is used not only for planning actions (Sommer and Wurtz, 2008) but for perception as well. MD, as well as other thalamic regions, might well convey information about internal states, which have now been demonstrated at least for saccadic eye movements (Sommer and Wurtz, 2008; Berman and Wurtz, 2011). Thus, although the projections from cerebral cortex down to thalamus and back have been extensively considered (Sherman and Guillery, 2011), currently the major functional evidence of thalamic input to cortex is for that passing from subcortical areas via thalamus to cortex.
A CD role in schizophrenia?
Feinberg (1978) and others (Frith and Done, 1989; Ford et al., 2007) have suggested that the inability of schizophrenic patients to discriminate between their own actions and those of others might result from a defective CD. The obvious question is whether this hypothesis is consistent with the CD identified in the monkey. First, schizophrenia has been associated with deficient activity in frontal cortex (Weinberger and Berman, 1996), which is the target of the monkey CD. Second, damage to the CD in monkeys produces deficits in tests in the guidance of saccades (the double-step task; Sommer and Wurtz, 2004b), and a similar deficit has been shown in schizophrenic patients (Thakkar et al., 2015). Finally the deficit in perceptual localization we now see in monkeys has also been observed in schizophrenic patients (Rösler et al., 2015). It would be of considerable interest if a brain signal in monkeys, studied to understand vision, also provides clues about a devastating disease in humans.
Footnotes
This work was supported by the National Eye Institute Intramural Research Program at the National Institutes of Health. We thank Altah Nichols and Tom Ruffner for machine shop support, David Leopold and Frank Ye for advice on analysis of the MRIs, and two anonymous reviewers whose analysis suggestions have strengthened our findings.
The authors declare no competing financial interests.
- Correspondence should be addressed to James Cavanaugh, Laboratory of Sensorimotor Research, National Eye Institute, Building 49, Room 2A50, Bethesda, MD 20892. jrc{at}lsr.nei.nih.gov
