Abstract
Despite tremendous advances in neuroscience research, it is still unclear how neuronal representations of sensory information give rise to the contents of our perception. One of the first and also the most compelling pieces of evidence for direct involvement of cortical signals in perception comes from electrical stimulation experiments addressing the middle temporal (MT) area and the medial superior temporal (MST) area: two neighboring extrastriate cortical areas of the monkey brain housing direction-sensitive neurons. Here we have combined fMRI with electrical stimulation in a patient undergoing awake brain surgery, to separately probe the functional significance of the human homologs, i.e., area hMT and hMST, on motion perception. Both the stimulation of hMT and hMST made it impossible for the patient to perceive the global visual motion of moving random dot patterns. Although visual motion blindness was predominantly observed in the contralateral visual field, stimulation of hMST also affected the ipsilateral hemifield. These results suggest that early visual cortex up to the stage of MT is not sufficient for the perception of global visual motion. Rather, visual motion information must be mediated to higher-tier cortical areas, including hMST, to gain access to conscious perception.
Introduction
Since the discovery of direction-sensitive neurons in the middle temporal (MT) area and medial superior temporal (MST) area of the macaque brain (Allman and Kaas, 1971; Dubner and Zeki, 1971; Maunsell and van Essen, 1983; Tanaka et al., 1986; Duffy and Wurtz, 1991), these two cortical areas have served as a model to study the neural mechanisms underlying computations of motion and to examine the relationships between neural activity and perception. Electrical stimulation of the areas MT and MST is able to change visual motion discrimination of monkeys in a predictive manner (Salzman et al., 1990; Murasugi et al., 1993; Celebrini and Newsome, 1995; Britten and van Wezel, 1998). Moreover, lesioning of areas MT and MST induces profound deficits in visual motion discrimination (Newsome and Paré, 1988; Marcar and Cowey, 1992; Pasternak and Merigan, 1994; Rudolph and Pasternak, 1999); a condition being referred to as motion blindness, or akinetopsia if present in patients suffering from diseases or trauma of the visual association cortex (Zihl et al., 1983, 1991; Zeki, 1991; Barton et al., 1995; Greenlee and Smith, 1997; Vaina et al., 2001, 2005). Although areas MT and MST share important functional properties, such as direction selectivity, they also exhibit qualitative differences. First, area MT shows a proper retinotopic organization confined to the contralateral visual hemifield, whereas the receptive fields of neurons in area MST, located one synapse downstream from area MT, are much larger. Most of the neurons in area MST respond to large optic flow stimuli, which extend into the ipsilateral visual field (Saito et al., 1986; Tanaka et al., 1986; Duffy and Wurtz, 1991). Second, extraretinal signals are largely absent at the stage of cortical area MT but present in MST. Specifically, neurons in MST frequently respond to vestibular stimulation (Page and Duffy, 2003; Gu et al., 2007; Takahashi et al., 2007; Fetsch et al., 2010) and carry explicit signals of ongoing smooth-pursuit eye movements (Newsome et al., 1988; Ilg and Thier, 2003).
The human homologs of areas MT and MST (hMT and hMST, respectively) have originally been referred to and treated as one compound, i.e., the human hMT+ complex, located in immediate proximity in the posterior limb of the inferior temporal sulcus (Watson et al., 1993). More recently they have been disentangled using functional magnetic resonance imaging (fMRI) by demonstrating robust responses to ipsilateral visual stimuli in hMST not present in hMT (Dukelow et al., 2001; Huk et al., 2002; Becker et al., 2008). Here, we combined fMRI with electrical stimulation in a patient undergoing awake brain surgery to characterize the role of areas hMT and hMST in motion perception in the most direct manner. Both, the stimulation of hMT and hMST induced akinetopsia. Although visual motion blindness was primarily observed in the contralateral visual field, stimulation of hMST also affected the ipsilateral hemifield. These results suggest that visual motion information must be mediated to higher-tier cortical areas including hMST to gain access to conscious perception.
Materials and Methods
Patient
Patient S.H., a 43-year-old female at the time of testing, was admitted to our hospital after having been diagnosed with a left temporal tumor after a first episode of word-finding failures, arguably reflecting transient focal seizure activity (Fig. 1). Surgery combined with intraoperative electrical stimulation was intended in the awake patient to prevent damage to eloquent brain areas. Because the posterior parts of the tumor were located in proximity to the posterior limb of the inferior temporal sulcus, the opportunity was given to test the influence of electrical stimulation of hMT+ on visual motion perception.
Preoperative axial structural magnetic resonance images of patient S.H. showing a tumor in the left temporal cortex. T1-weighted (A) and T2-weighted/FLAIR (B) axial sections are shown from top down and are aligned to the anterior–posterior commissure line. The z values indicate the distance (in millimeters) of a given axial section from the horizontal plane passing through the anterior and posterior commissures (z = −32 to 32 mm). Slices are shown as seen from above, i.e., the left hemisphere is shown on the left side.
Standard neurological examination yielded no oculomotor disturbances, primary visual field defects, or motor or somatosensory deficits. Structural magnetic resonance imaging showed a left temporal tumor that necessitated neurosurgical treatment (Figs. 1, 2). Postoperative neuropathologic examination would reveal an anaplastic oligoastrocytoma (World Health Organization, grade III). Surgery combined with intraoperative electrical stimulation was intended in the awake patient to prevent damage to eloquent brain areas. Because the posterior parts of the tumor were located in immediate proximity to the posterior limb of the inferior temporal sulcus, the opportunity was given to test the influence of electrical stimulation of the hMT+ complex on motion perception and to further compare results from intraoperative functional mapping using focal electrical stimulation with fMRI performed before surgery.
The patient had normal acuity, no experience as a psychophysical observer and was naive concerning the scientific aims of the study. Informed written consent was obtained for both the preoperative fMRI and the intraoperative electrical stimulation examinations. Both were conducted in conformity with the Declaration of Helsinki and with the guidelines of the local ethics committee of the Medical Faculty of the University of Tübingen. Before scanning and before surgery, the patient was trained to ensure that she was able to cope with the requirements of the tasks.
Preoperative and postoperative protocols
fMRI was performed to separate area hMT from area hMST through the use of techniques described recently (Becker et al., 2008). In addition, psychophysical testing was conducted to assess the ability of the patient to discriminate the direction of global visual motion embedded in noisy random dot kinematograms (Newsome and Paré, 1988). Moreover, preoperative as well as postoperative structural magnetic resonance and computed tomography scans were performed.
fMRI: visual stimulation.
Visual stimuli were presented using a NEC GT 950 liquid crystal display (LCD) projector (800 × 600 pixel resolution, 60 Hz refresh rate) and were generated using OpenGL rendering software operating on an IBM PC-compatible Pentium class computer. During scanning, the patient laid head-first, supine, in the magnet. The patient viewed the stimuli, which were back-projected onto a screen through a mirror attached to the head coil placed ∼10 cm in front of the eyes. Viewing was binocular. The visual stimulus consisted of high-contrast random dot patterns (white dots on an otherwise dark background) and was observed by the patient during stationary fixation. The stimulus covered a square area of 8° × 8° and was presented with its center placed 7° to the left or 7° to the right of the fixation point. Each dot had a diameter of 8.6 arc min, and (in the motion conditions) moved on a straight path at a speed of 6°/s for a limited lifetime of 1000 ms before disappearing and reappearing at a new, random location. Dot density was set to 6 dots/degree2. The motion direction of the dot elements was either the same for all dots (coherent motion) or, alternatively, the dots moved independently in all possible directions (incoherent motion). For both types of motion stimuli, the motion direction changed every 2 s clockwise or counter-clockwise in steps of 60°. The control condition involved presentation of stationary dots with a limited lifetime of 1000 ms to correct for the onset of dots in the motion conditions and to keep flicker as well as luminance information constant.
During each functional scan, which in total lasted 396 s, epochs alternated as follows: stationary pattern, coherent motion, stationary pattern, incoherent motion, et cetera, or stationary pattern, incoherent motion, stationary pattern, coherent motion, et cetera. The epoch length of the individual epochs was 12 s, and the stimulus cycle was repeated eight times during one functional session. The patient participated in four functional sessions that differed with respect to the position of the random dot pattern (7° to the left and 7° to the right) and with respect to the motion conditions in the stimulus cycle as mentioned above.
The task for the patient was identical to the one described by Becker et al. (2008). Briefly, in each of the two motion epochs of a given trial, the moving dots would accelerate to 12°/s or decelerate to 3°/s. The presentation time of the faster/slower stimulus was very short (∼80 ms). By pressing a button, the patient had to indicate at the end of the second motion epoch whether the two changes of velocity were the same (right button) or opposite (left button). At the beginning of each trial, the central fixation point changed its color from red to green for 0.5 s to indicate that a new trial had started and a new comparison of the velocity changes had to be performed.
fMRI: acquisition of imaging data.
Preoperative images were collected with a 3-tesla whole-body magnetic resonance scanner (Magnetom Trio, A Tim System, Siemens AG) equipped with a 12-channel phased-array head coil. The patient participated in four functional scanning sessions, each lasting 6.5 min, interspersed with short rest periods and an additional session to obtain the high-resolution anatomical images of the brain. Functional imaging was performed using blood oxygenation level-dependent (BOLD) contrast-based echo-planar imaging. The 198 volumes of functional images were acquired with a standard two-dimensional gradient-echo sequence (repetition time, 2.0 s; echo time, 31 ms; flip angle, 90°; voxel size, 2 × 2 × 2 mm; 28 contiguous coronal slices measured with an interleaved acquisition scheme; field of view, 128 mm). Only the posterior portion of the brain was scanned, and the first six scans of each functional session were discarded to allow the signal to reach equilibrium for T1-weighted effects. At the end of the functional sessions, a high-resolution three-dimensional T1-weighted structural scan with the modified driven equilibrium Fourier transform (MDEFT) sequence (Deichmann et al., 2004) of the whole brain was acquired (repetition time, 10.55 ms; echo time, 3.14 ms; inversion time, 680 ms; flip angle, 22°; voxel size, 1.0 × 1.0 × 1.0 mm; 176 contiguous axial slices; field of view, 256 mm). The MDEFT sequence was chosen in place of standard sequences because of its improved contrast between the gray and white matter. This is beneficial for the segmentation process.
In addition to the experiment described, preoperative and postoperative routine scanning for clinical purposes was also performed. Preoperative structural images were acquired with a 1.5-tesla whole-body magnetic resonance scanner (Magnetom Sonata-Vision, Siemens AG) with a standard three-dimensional T2-weighted fluid attenuated inversion recovery (FLAIR) sequence of the whole brain (repetition time, 8800 ms; echo time, 118 ms; inversion time, 2500 ms; flip angle, 180°; voxel size, 0.9 × 0.9 × 4.4 mm; 32 contiguous transversal slices; field of view, 172 × 230 mm). Postoperative structural images of the whole head were collected with a multislice computed tomography scanner (Somatom Sensation 16, Siemens AG) with a voxel size of 0.4 × 0.4 × 4.5 mm (32 contiguous scans; matrix size, 512 × 512; convolution kernel, H70 h; x-ray tube voltage 120 kV at peak with a current of 285 mA).
fMRI: analysis of imaging data.
Functional and structural images were analyzed using BrainVoyager QX version 1.8.6 (Brain Innovation). For the functional images, the following preprocessing steps were performed: slice scan time correction (using sinc interpolation), linear trend removal, temporal high-pass filtering to remove low-frequency nonlinear drifts of three or fewer cycles per time course, and three-dimensional motion correction (using trilinear/sinc interpolation) to detect and correct for small head movements by spatial alignment of adjacent volumes to the initial volume of a session by rigid body transformations. No spatial smoothing was performed to prevent blurring of the boundary between area hMT and area hMST. The T1-weighted structural image in native brain space, i.e., not transformed to the anterior commissure/posterior commissure (ACPC) space (Talairach and Tournoux, 1988), was used as a reference to which all subsequent functional and structural images were coregistered. The coregistration parameters were provided by the coregistration function of SPM 2 (Wellcome Trust Centre for Neuroimaging, University College London, London, UK) to obtain an optimal fit. In all preprocessing steps the structural images were used in a voxel resolution of 1.0 × 1.0 × 1.0 mm and were, if necessary, transformed to this resolution by sinc interpolation (r = 4). The functional and structural images were transformed to the ACPC-space. To visualize activity patterns, the outline of the T1-weighted structural image was segmented from the head tissue and was then processed for the reconstruction of the cortical surface. The tumor region and the trepanation defect were reconstructed by manual segmentations of the preoperative T2-weighted structural images and from the postoperative computed tomography scan, respectively.
Areas hMT and hMST were delineated in both cortical hemispheres in the dorsal/posterior limb of the inferior temporal sulcus, based on a combination of anatomical and functional criteria. The regions of interest were defined as the cluster of contiguous voxels lying in immediate neighborhood of the ascending limb of the inferior temporal sulcus (Zeki et al., 1991; Watson et al., 1993; Tootell and Taylor, 1995; Tootell et al., 1995; Dumoulin et al., 2000) and showing significantly stronger responses to the given coherent and incoherent motion conditions as compared with the stationary stimuli. This comparison was based on a general linear model at a statistical threshold of T > 4.0 with an additional cluster threshold of k = 50 voxels (corresponding to p < 0.0001, uncorrected for multiple comparisons). The model regressors were defined on the basis of each motion condition and were convolved with a hemodynamic response function formed from two gamma functions (onset of curve, 0 s; time to response peak, 5 s; response dispersion, 1; undershoot ratio, 6; time to undershoot peak, 15 s; undershoot dispersion, 1). In addition, the patient's head movement parameters, derived from the three-dimensional motion correction procedure, were included as regressors of no interest to account for residual motion artifacts. No further correction for multiple comparisons was performed for the two areas under study, for which we had a priori hypotheses. Visual stimuli in a given hemifield are known to activate the entire hMT+ complex in contralateral cortex, including area hMT and area hMST. Ipsilateral activations, however, are confined to area hMST but exclude area MT (Desimone and Ungerleider, 1986; Komatsu and Wurtz, 1988; Tanaka and Saito, 1989; Duffy and Wurtz, 1991; Raiguel et al., 1997; Dukelow et al., 2001; Huk et al., 2002; Goossens et al., 2006; Smith et al., 2006; Beauchamp et al., 2007; Becker et al., 2008; Wall and Smith, 2008; Wall et al., 2008). The clusters of activations obtained from the left and right visual motion stimuli overlapped substantially. Therefore, area hMST was defined as all contiguous voxels within the hMT+ complex that were significantly active during ipsilateral motion stimulation. Area hMT on the other hand was defined as all contiguous voxels that were active during contralateral but not ipsilateral stimulation. All voxels lying anterior to the median axial coordinate of area hMST were also excluded from area hMT (Wall and Smith, 2008; Wall et al., 2008).
Psychophysical tests before surgery.
To assess the ability to discriminate the direction of global motion embedded in noise, visual motion coherence thresholds were determined in separate sessions in our psychophysical laboratory. The stimulus used was a random dot pattern in a configuration nearly identical to the one used in the fMRI experiment. Again, the random dot pattern was presented either in the right or the left visual hemifield, as in the fMRI experiment, but would appear only for 300 ms to make direction discrimination challenging. In addition, discrimination was also measured in the central visual field. Stimuli were presented on a Mitsubishi 19 inch computer monitor (1280 × 1024 pixel resolution, 72 Hz refresh rate) in a dark experimental room. Viewing was binocular and the viewing distance was 57 cm.
The detection of global motion embedded in noise was determined by presenting random dot patterns of varied motion coherence, i.e., the percentage of dots moving coherently on the same direction. The patient was instructed to maintain stable fixation and to report the direction of the coherent motion which could be either to the right or to the left (two-alternative forced choice). The sequence of trials, i.e., motion coherence of a given trial, was controlled by an adaptive staircase procedure (Taylor and Creelman, 1967). After training, the stimuli were presented, in separate blocks, in the right, in the left, and in the central visual field with short breaks between blocks. The perceptual threshold was defined as the percentage of motion coherence that yielded 75% correct responses (where 50% correct is the performance expected by chance). These thresholds were derived from Probit approximations (McKee et al., 1985).
During all tests, eye movements were monitored using an infrared iris reflection system (Amtech). Recordings were stored and analyzed online at a sampling rate of 200 Hz by a workstation, which also controlled the presentation of the stimuli. Deviations of eye position from the position of the given target exceeding 2° were fed back acoustically as errors and the corresponding trials were discarded. This training was also important to prepare the patient for the task during surgery. In particular, the patient learned not to move her eyes away from the fixation point when visual motion stimuli appeared in the visual periphery. Global motion detection was normal in the patient before surgery as compared with normal controls tested formerly in our laboratory (n = 29, mean age 47.5 years). Specifically, her mean thresholds obtained from different measurements were as follows.
Right visual field: 26.7% (control group, mean: 18.7%, std 10.1%; normal limit: 38.9%)
Left visual field: 30.15% (normal limit: 38.9%)
Central visual field: 19.3% (control group, mean: 13.1%, std 8.4%; normal limit: 29.8%)
Finally, also her pursuit eye movements recorded in a conventional step-ramp paradigm for two velocities (6°/s, 12°/s) also did not reveal any impairment or asymmetry.
Intraoperative protocol: visual stimulation, behavioral tests, and electrical stimulation
Visual stimuli were presented during intraoperative testing using an Eizo FlexScan L365 LCD monitor (1024 × 768 pixel resolution, 60 Hz refresh rate) horizontally aligned with the interocular axis at a distance of 50 cm and were generated using OpenGL rendering software operating on an Apple Macintosh G4 PowerBook Pro. The stimulus presentation was controlled over TCP/IP by an IBM PC-compatible Pentium class computer.
Because intraoperative stimulation would not allow for measurements of motion coherence thresholds requiring more than ∼20 presentations per condition (stimulation site), we decided to present visual motion stimuli at a clearly suprathreshold coherence level, i.e., 100% motion coherence. With normal preoperative perceptual thresholds the patient was expected to discriminate that level without any errors. Conversely, errors in discriminating global motion at 100% coherence would clearly indicate influences of electrical stimulation. In fact, during surgery the patient would be able to perform the task at ease as long as stimulation intensity was not larger than 4 mA. At higher intensities and at specific sites, however, consistent deficits occurred. To improve the detection of errors, the patient had to discriminate between four rather than two possible motion directions, i.e., the four cardinal directions, and was instructed to give a verbal response (“right,” “left,” “up,” “down”) without any time constraints. With four alternatives, the performance expected by chance was 25% correct responses. Although this paradigm thus accepted a bias for false-negative stimulations it seemed well suited to avoid false-positive results. Like in the preoperative psychophysical measurements, the visual motion stimuli were presented for 300 ms either in the right or the left visual hemifield, i.e., contralateral or ipsilateral to the surgical field. All other stimulus parameters were identical to the parameters of the fMRI experiment. The minimal duration between trials applying electrical stimulation was ∼10 s. Motion direction discrimination was tested in this four-alternative forced-choice paradigm in the absence and presence of intraoperative electrical stimulation.
Bupivacaine and xylocaine were infiltrated into the scalp before rigid fixation of the head with a three point Mayfield head-holder. A continuous infusion of propofol (5–7 mg/kg) was injected during skin incision and craniotomy. During cortical mapping no medication was administered.
Testing of language involvement was performed by presenting series of different objects from the Aachen Aphasia Test (subtest “naming”; Huber et al., 1983). Deficits in naming and/or speech arrest during stimulation identified language related areas.
Cortical stimulation parameters followed well established methods described previously (Ojemann, 1983; Gharabaghi et al., 2006). A bipolar electrode with 5 mm spaced tips delivered a biphasic current at a frequency of 60 Hz, with single pulse duration of 1 ms and pulse train duration of ∼2 s (Cortical Stimulator, Inomed). The stimulation electrode was placed directly onto the cortical surface.
Functional mapping started after the left temporo-occipital craniotomy and dural opening. In total, electrical stimulation was applied to 144 different cortical sites with amplitudes ranging from 2 to 6 mA. At 107 sites visual motion discrimination was tested and 37 locations were examined for aphasia testing. During the tests addressing visual motion perception, eye movements were monitored via direct visual inspection by an experienced rater. The patient was able to maintain stable fixation in all trials. In addition, we also performed EOG recordings which, however, could not be analyzed in a meaningful way due to severe baseline shifts that occurred during surgery.
Electrical stimulation: data analysis.
The analysis of the stimulation data required the determination of the electrode locations in a common intraoperative reference picture including its perspective correction. In a second step, we aimed at coregistration of radiological and intraoperative images to directly compare fMRI and electrical stimulation results.
The intraoperative reference picture was obtained from a digital image of the intraoperative video recording in which the complete surgical field was visible. To make sure that the projection origin of this image was located in the center of the surgical field and was orthogonal to the cortical surface, we calculated spherical projections from this selected reference picture and three additional intraoperative photographs with slight variations in projection origins using Autopano Pro version 1.4.2 (Kolor). Based on these data, the projection origin of the chosen reference image was adjusted accordingly and the linearity of the surface distances on the resulting intraoperative reference photograph was controlled via intraoperative markers.
The locations of the electrical stimulations were also determined from the intraoperative video recordings. To analyze the stimulation data, the determined electrode locations were represented on the intraoperative reference image. Because the normal gyral and sulcal pattern was not apparent in the patient due to space occupying effects of the tumor, the visible blood vessels were used as anatomic landmarks for the transfer of the intraoperative recording data to the reference photograph. In the common reference picture, a given electrical stimulation site was marked by a single solid circle, where the center between the tips of the bipolar electrode defined the center of the corresponding circle.
Coregistration of radiological and intraoperative images was achieved by identifying common points of the trepanation defect in both modalities (see Fig. 4). Although the trepanation defect was extracted from the intraoperative reference picture, it had to be determined by means of postoperative CT scans for the radiological data. Coregistration of postoperative CT and preoperative fMRI in turn allowed superimposition of BOLD responses and the trepanation defect and, in this manner, generation of a second reference image of the surgical field (see Fig. 4A). The projection origins of both, the radiological and the intraoperative, images used for the image registration process were selected in such a way that they were located in the center and orthogonal to the surface of the corresponding scene. To compare the two pictures, an image coregistration was performed using pairs of corresponding points (see Fig. 4). In total, 19 pairs of corresponding points were selected along the trepanation defect and identified in both images. Based on these image tie-points and using Matlab version 7.1 spatial transformation routines (MathWorks), the correspondence problem between the two images was solved by means of a piecewise linear mapping function (Goshtasby, 1986), finally resulting in one common frame of reference.
Results
Before surgery, fMRI was performed to separate area hMT from area hMST by resorting to techniques described recently (Dukelow et al., 2001; Huk et al., 2002; Becker et al., 2008; Helfrich et al., 2013). Based on the well known properties of MST neurons in the monkey brain, which have much larger receptive fields than MT neurons and which often extend into the ipsilateral hemifield (Desimone and Ungerleider, 1986; Komatsu and Wurtz, 1988; Duffy and Wurtz, 1991), area hMST was identified in both hemispheres by responses to visual motion presented in the ipsilateral visual field (Fig. 2). On the other hand, area hMT had to be extracted from the cluster of activation induced by contralateral motion stimuli, i.e., the hMT+ complex housing area hMT as well as hMST. This was achieved by excluding all voxels from the hMT+ complex that had earlier been classified as belonging to hMST. Corroborating earlier studies (Dukelow et al., 2001; Huk et al., 2002; Becker et al., 2008), the hMT+ complex preferring visual motion stimuli to static patterns was found at the temporo-occipital junction in both hemispheres of patient SH. Although the left hMT+ complex as a whole was slightly shifted in posterior direction due to tumor occupying effects, both areas hMT and hMST were spared from tumor infiltration.
Definitions and locations of the regions of interest (ROIs) constituting the hMT area and the hMST area of patient S.H. based on functional magnetic resonance imaging. BOLD activations were obtained by comparing responses to moving random dot kinematograms versus stationary dot patterns which were presented either in the left (A) or the right (B) visual hemifield. The resulting activations are superimposed on a structural axial slice in the plane of the ACPC, using a T-value threshold of 4.0 and a cluster threshold of k = 50 voxels (corresponding to p < 0.0001, uncorrected for multiple comparisons). A, Presentation of visual motion in the left visual hemifield induces a large patch of activation in contralateral (right) temporal cortex, corresponding to the hMT+, and a smaller patch in ipsilateral temporal cortex, defined as area hMST. B, Activations observed with the stimulus presented in the right visual hemifield. The pattern of activity is reversed. Note the mild posterior shift of area hMT+ on the left side due to the space-occupying tumor which is marked in green. C, Resultant ROIs attributed to area hMST (cyan) and area hMT (magenta), the first defined by responses to ipsilateral visual motion, the latter given by the voxels selectively responding to contralateral but not ipsilateral visual motion. D–I, ROIs of areas hMT and hMST superimposed on axial slices from top to bottom parallel to the ACPC plane (z = 8 to −12 mm). All axial images follow neurological convention, i.e., the left hemisphere is shown on the left. J, Three-dimensional surface reconstruction of the brain of patient S.H. and representation of the ROIs defining area hMT and area hMST. The ACPC plane (z = 0 mm) is indicated by blue lines.
Intraoperatively, the perception of visual motion was assessed during stationary fixation by random dot kinematograms (RDKs) presented for 300 ms in the left or the right visual hemifield. RDKs covered an area of 8.0° × 8.0° and were presented with their center placed 7° to the left or to the right of a central fixation point. All stimuli involved coherent visual motion at a velocity of 6 °/s, i.e., all dots were moving in the same direction. In each trial, motion was in one of four cardinal directions (up, down, right, or left) which the patient was instructed to report verbally after RDK offset. Direction discrimination in this four-alternative forced-choice paradigm was tested during electrical stimulation (Gharabaghi et al., 2006) delivered at amplitudes of 2–6 mA. The patient was trained and reinforced during the procedure to choose one of the four motion directions rather than describing her percept. In a postoperative interview, the patient reported that in some trials she had not experienced any specific motion direction. In all trials the dots had been visible for her. On no account had she experienced self motion, such as linear or circular vection.
Bipolar electrical stimulation was initially applied to 56 different locations at amplitudes between 2 and 4 mA while the patient observed visual motion in the right visual field, i.e., contralateral to the surgical field and to the hMT+ complex. For currents up to 4 mA, the patient performed only one single error. Thus, she was able to perform the task in the intraoperative environment with high accuracy (error probability per trial p = 1/56 = 0.0179, confidence interval = 0.0044, 0.0638) and stimulation amplitudes were obviously too low to evoke consistent deficits. Increasing the stimulation amplitude to 6 mA, in contrast, resulted in qualitative deficits, i.e., the inability to perceive the motion direction that we will also refer to as motion blindness or akinetopsia (Baker et al., 1991; Zeki, 1991) in the following. At 18 of 51 stimulation sites visited, motion blindness was induced. As depicted in Figure 3A, these deficits were elicited by electrical stimulation of a small, contiguous patch of cortex, covering ∼3 cm in the anterior–posterior direction (along the x-axis; Fig. 3) and 2 cm in the superior-inferior direction (along the y-axis). Motion blindness was induced not only in the right, i.e., the contralateral visual field (at 11 of 27 stimulation sites; Fig. 3A, magenta) but also in the left, i.e., the ipsilateral hemifield (at 7 of 24 sites; Fig. 3A, cyan). These deficits occurred during ongoing, stable fixation as visually monitored by an experienced rater. Although motion blindness in the contralateral visual field was observed after stimulation of the posterior part of the cluster of foci, ipsilateral akinetopsia occurred preferentially after stimulation of its more anterior parts with significant overlap in an intermediate zone. An anterior–posterior organization of the stimulation area was further supported by Mann–Whitney U tests comparing the coordinates of the stimulation sites inducing either contralateral or ipsilateral deficits which revealed significant differences along the x-axis (U = 41, p < 0.0408, Bonferroni-corrected) but not along the y-axis (U = 76, p > 0.8240). Importantly, the inability to report the motion direction seen was not due to a speech or language deficit resulting from stimulation of the dominant hemisphere. Speech arrest such as searched for using an object naming test (Gharabaghi et al., 2006) was induced at clearly distant stimulation sites, primarily affecting the angular gyrus (Fig. 3B). In fact, statistical analysis of the stimulation sites inducing visual motion as compared with the other foci eliciting speech deficits revealed highly significant differences for both cardinal axes (x-axis, U = 15, p < 0.0018; y-axis, U = 97, p < 0.013).
Results of electrical stimulation on motion perception (A) and object naming (B). The surgical field giving view on the tumor (green) and temporo-occipital cortex before tumor resection including the electrical stimulation sites and the stimulation effects (left, anterior; right, posterior). A, Electrical stimulation sites visited in the visual motion direction discrimination task are given in magenta or cyan depending on the visual hemifield being stimulated (sketch above). Foci which induced motion blindness in the contralateral, i.e., right, visual field are indicated by solid magenta circles with black borders. Sites inducing motion blindness in the ipsilateral, i.e., left, visual field are indicated by solid cyan circles with black borders. B, Electrical stimulation sites in the object naming task. Foci inducing speech arrest are indicated by solid blue circles. The mean coordinates of the effective electrical stimulation sites for the three tasks are marked by crosses in the corresponding color. Solid white circles denote the stimulation sites at which no behavioral consequences for the given task was observed (contralateral visual motion, magenta; ipsilateral visual motion, cyan; object naming, blue). Anterior is on the left and ventral is at the bottom of the surgical field. An orthogonal reference grid with a resolution of 5 × 5 mm is indicated by blue lines.
To relate the stimulation results to the functional organization of the hMT+ complex as delineated by fMRI, radiological and intraoperative stimulation data were coregistered. This was achieved by identifying image tie-points of the trepanation defect in both modalities (Fig. 4). Although the trepanation defect was easily extracted from intraoperative pictures, it had to be determined by means of postoperative computer tomography (CT) scans for the radiological data. Coregistration of postoperative CT and preoperative fMRI in turn allowed superimposition of BOLD responses and the trepanation defect and, in this manner, radiological reconstruction of the surgical field (Fig. 4) which was finally merged together with the reference picture obtained during surgery. As shown in Figure 5, the stimulation sites inducing motion blindness were in good agreement with the location and the anterior–posterior organization of the hMT+ complex as determined by fMRI. This correspondence lends strong support to the conclusion that visual cortex in and around the human inferior temporal sulcus indeed houses the homologs of macaque areas MT and MST and that ipsilateral motion blindness was induced by stimulation of area hMST.
Coregistration of radiological and intraoperative images was achieved by identifying common points of the trepanation defect in both modalities. A, Cropped and properly oriented three-dimensional surface reconstruction of the brain (dark gray) which was obtained from the preoperative MR images and which is overlaid here with the reconstructed skull (light gray) and its trepanation defect, the latter derived from postoperative CT images coregistered with the MR data. B, Picture of the surgical field giving view on the pial vessels and the cortical surface. In total 19 image tie-points were selected manually to delineate the trepanation defect (marked with cyan crosses in A and B). Anterior is on the left and ventral is at the bottom in both images.
Comparison of responses obtained from fMRI and electrical intraoperative stimulation. Coregistration of radiological and intraoperative images was achieved by identifying common points of the trepanation defect in both modalities. The reference picture of the surgical field including the stimulation sites (Fig. 3) was transformed to optimally fit into the trepanation defect such as reconstructed on the basis of postoperative computed tomography imaging. Coregistration of postoperative CT scans with preoperative fMR imaging in turn allowed superimposing of the ROIs of areas hMT (magenta) and hMST (cyan) onto the transformed surgical field, same color conventions as in Figures 2 and 3. A, Electrical stimulation sites at which contralateral (magenta) or ipsilateral (cyan) motion blindness was induced. B, Nonfoci. A shift of the electrical foci as compared with the BOLD responses is likely to reflect intraoperative displacement of the cortex in the course of the surgery (e.g., patient head positioning, skull trepanation, cerebrospinal fluid loss). The blue lines indicate the transformed reference grid of Figure 3.
Discussion
The areas MT and MST are considered core elements of the cortical visual motion system. In particular, the transition from retinal (MT) to more global coding of visual motion in nonretinal coordinates (MST) has established the view of areas MT and MST as a crucial sensorimotor interface and possible substrate of motion perception (for review, see Britten, 2008). Although the animal literature has differentiated between these two areas right from the beginning of their extensive study, it is more recently that the human homologs have also been separated based on functional imaging criteria (Dukelow et al., 2001; Huk et al., 2002; Smith et al., 2006; Becker et al., 2008). Specifically, activations resulting from ipsilateral visual motion stimulation in fMRI experiments have been attributed to hMST assuming that basic neuronal properties would have been preserved across species. Further imaging studies have revealed that hMST indeed resembles its monkey homolog in many important aspects, such as the preference for coherent motion as compared with motion noise (Becker et al., 2008; Fischer et al., 2012; Helfrich et al., 2013) or the selectivity for specific optic flow structures (Wall et al., 2008; Cardin et al., 2012).
Activation studies need to be supplemented by lesion mapping strategies to infer the functional significance of the activated brain areas with any certainty. Cortical lesions including the temporo-occipital junction in humans have been shown to entail deficits in visual motion perception (Zihl et al., 1983; Barton et al., 1995; Vaina et al., 2001). However, the effects of lesions to hMT and hMST have not been characterized separately, because of the immediate proximity of these two areas. This also holds true for the few studies available that have resorted to electrical stimulation techniques to address the hMT+ complex (Blanke et al., 2002; Rauschecker et al., 2011). The reason is that the spatial resolution of these studies was limited to the configuration and location of grids implanted in patients with intractable epilepsy before surgery. Combining fMRI with electrical stimulation that can be applied in a flexible manner during awake brain surgery, thus, offers a promising way to characterize the role of areas hMT and hMST in motion perception separately and in a most direct manner.
As a first main result we observed that stimulation of the hMT+ complex, as defined by fMRI, induces motion blindness in humans. Like previous stimulation studies (Blanke et al., 2002; Rauschecker et al., 2011), this result provides evidence for a causal link between activity in the hMT+ complex and the human perception of visual motion. Visual motion blindness was primarily observed in the contralateral visual field, but at specific sites it was also present in the ipsilateral hemifield. More specifically, ipsilateral akinetopsia was induced by stimulation of the more anterior parts of the hMT+ complex, i.e., the part which is believed to house the hMST homolog (Dukelow et al., 2001; Huk et al., 2002; Smith et al., 2006; Becker et al., 2008). The attribution of akinetopsia in the ipsilateral visual field to a transient lesion of area hMST seems straightforward at first glance, given the large receptive fields of MST neurons in the monkey. As far as we can tell, however, evidence for ipsilateral motion perception deficits resulting from area MST lesions is so far lacking. Studies in awake, behaving rhesus monkeys have established that focal lesions of area MT and/or MST induce profound visual motion perception deficits. These deficits, however, seem to be confined to the contralateral visual field (Newsome et al., 1985; Dürsteler and Wurtz, 1988; Newsome and Paré, 1988; Pasternak and Merigan, 1994; Rudolph and Pasternak, 1999). Likewise, patients with damage to extrastriate cortex including the posterior temporal lobe show impairments in the contralesional but not in the ipsilesional visual field (Barton et al., 1995; Vaina et al., 2001). Obviously the perception of visual motion does not necessarily depend on concerted activity of the hMT+ complexes of both hemispheres, but can rely on motion representations of the contralateral hMT+ complex alone.
If this is true, why then did intraoperative stimulation of area hMST result in ipsilateral akinetopsia in our patient, i.e., in a condition under which area hMT+ located contralaterally to the motion stimuli was not being directly affected? One possibility is that under the conditions of the experiment, compensatory mechanisms cannot be initiated quickly enough. It has been shown that unilateral inactivation of area MST in monkeys using muscimol produces almost negligible changes in optic flow perception while bilateral inactivation produces stronger effects (Gu et al., 2012). Potentially, the contralateral counterpart needs a critical period of time to compensate for unilateral inactivation. A second possibility is that the global deficit observed here reflects influences of stimulation on contralateral area hMST mediated via transcallosal projections. For this interpretation to be valid, two main conditions have to be met. First, area hMST must maintain direct connections to its contralateral counterpart. Second, electrical stimulation should be capable of inducing powerful remote effects to direct cortical projection sites. At least as far as monkey studies are concerned there is convincing evidence for both claims (Maunsell and van Essen, 1983; Ungerleider and Desimone, 1986; Boussaoud et al., 1990; Kaas and Morel, 1993; Tolias et al., 2005; Logothetis et al., 2010; Sultan et al., 2011). In particular, area MST maintains interhemispheric connections between representations of visual motion that go far beyond the vertical meridian, although they were found weaker in the visual periphery as compared with the more central visual field (Boussaoud et al., 1990). On the other hand, projections between hemispheres are sparse for area MT and seem to be confined to representations close to the vertical meridian (Maunsell and van Essen, 1983). Because remote effects of electrical stimulation as measured by fMRI are confined to cortical areas known to receive direct, i.e., monosynaptic connections from the stimulation site (Sultan et al., 2011), our stimulation effects are thus unlikely to reflect direct influences of stimulation on contralateral hMT.
Although we cannot exclude remote effects on areas other than hMST also housing bilateral representations of visual motion and although our line of arguments is derived primarily from studies of the monkey brain, we suggest from our data that activity in area hMT (and also early visual cortex, such as V1) alone is insufficient for motion perception. This may seem counterintuitive at first glance, given the severe impact of MT lesions on motion perception, however, there is indeed further evidence in favor of this interpretation. In a recent study, Hedges et al. (2011) demonstrated that the perception of global visual motion can grossly deviate from what MT neurons encode during the same task. Likewise, the perception of second order motion is not reflected by activity changes in MT neurons (Ilg and Churan, 2004). Another example for dissociation between motion perception and neuronal representations in area MT is offered by the case of self-induced retinal image motion such as resulting from smooth pursuit eye movements. Although human observers as well as rhesus monkeys perceive an almost perfectly stable world despite the retinal image shifts resulting from pursuit eye movements (Haarmeier et al., 1997; Dash et al., 2009), area MT signals image motion on the retina rather than perceptual stability experienced by the observer (Erickson and Thier, 1991; Tikhonov et al., 2004). Our finding of ipsilateral akinetopsia after unilateral stimulation of the hMT+ complex, thus, lends support to the hypothesis that we are not aware of neural activity in early visual cortex (Crick and Koch, 1995), even up to the level of area hMT. Rather, concerted activity of interconnected areas including area MST is necessary for visual motion perception.
Footnotes
This work was supported by the German Research Council (Deutsche Forschungsgemeinschaft) Grants DFG SFB 550/TP A2, DFG GH 94/2-1, and DFG EXC 307, the European Research Foundation Grant ERC 227632, the German Ministry of Education and Research Grants BMBF Bernstein 01GQ0761, and BMBF 16SV3783, BMBF 03160064B, BMBF V4UKF014, and the Hertie Foundation. We thank Mathias Röger for his help in acquiring the magnetic resonance images and Marina Liebsch for her help in acquiring the intraoperative electrical stimulation data. We also thank Rüdiger Berndt, Monika Fruhmann Berger, Rupert Kolb, Michael Erb, and Friedemann Bunjes for technical assistance, Peter Thier and Fahad Sultan for helpful discussions, and patient S.H. for participation in the study.
- Correspondence should be addressed to Dr Thomas Haarmeier, Department of Neurology, University of Aachen, RWTH, Pauwelsstrasse 30, D-52074 Aachen, Germany. thaarmeier{at}ukaachen.de
