Abstract
Visual control of locomotion is essential for most mammals and requires coordination between perceptual processes and action systems. Previous research on the neural systems engaged by self-motion has focused on heading perception, which is only one perceptual subcomponent. For effective steering, it is necessary to perceive an appropriate future path and then bring about the required change to heading. Using function magnetic resonance imaging in humans, we reveal a role for the parietal eye fields (PEFs) in directing spatially selective processes relating to future path information. A parietal area close to PEFs appears to be specialized for processing the future path information itself. Furthermore, a separate parietal area responds to visual position error signals, which occur when steering adjustments are imprecise. A network of three areas, the cerebellum, the supplementary eye fields, and dorsal premotor cortex, was found to be involved in generating appropriate motor responses for steering adjustments. This may reflect the demands of integrating visual inputs with the output response for the control device.
Introduction
Sensitivity to the direction of self-motion is fundamental for mobile animals. One of the main visual cues to direction of self-motion arises from optic flow, arising from the ground plane over which the animal travels (Gibson, 1958; Warren and Hannon, 1988). Research into the neural basis of sensitivity to optic flow and direction of self-motion has focused on the properties of two brain areas, dorsal medial superior temporal cortex (MSTd) and ventral intraparietal area (VIP). Neurons in macaque MSTd respond selectively to specific optic flow components (Tanaka et al., 1989; Duffy and Wurtz, 1991), and a human area with similar properties has been identified (Morrone et al., 2000; Smith et al., 2006). The sensitivity of macaque VIP to heading has been established by Bremmer et al. (2002). A functional magnetic resonance imaging (fMRI) study of human heading judgments, however, has highlighted additional areas (Peuskens et al., 2001).
Traveling across a homogenous ground plane that only provides optic flow, however, is not typical of real-world locomotion. In many environments, we also encounter paths, trails, or obstacles that set constraints on the future trajectory. These features prime the changes to heading that will be required in the next few seconds, and we refer to this as the perception of future path. Future path information is not exclusive to the man-made environment, and other animals routinely follow trails or negotiate obstacles.
The perception of future path can be dissociated from adjustments to the current path. A passenger in a car can perceive both current heading and future path but controls neither. The driver of a car is confronted with the same visual information as the passenger but uses it to perform an appropriate visuomotor transformation to achieve steering. Neurophysiological studies in monkeys and humans have so far focused on the recovery of heading from optic flow but not the use of this information for coordinated action. The purpose of this study was, first, to extend the work on the neural processing of heading to include the perception of future path, and, second, to explore the neural systems involved in transforming this information for visually controlled steering.
First, we report an exploratory study with three main aims. These were to identify human brain regions involved in the perception of future path, those regions that process error signals during simulated self-motion, and those specifically involved in controlling a steering device using visual information. We identified a superior parietal region associated with the perception of future path, and the anterior part of this region proved to be critical for processing error signals during self-motion. Comparing the active control of steering with the passive perception of heading revealed additional specific activations in the cerebellum, supplementary eye fields (SEFs), and dorsal premotor cortex. In a second study, using a subset of the participants from the first study, we added control conditions designed to partition the parietal responses attributable to eye movements and shifts of spatial attention (provoked by the stimulus) from those directly related to future path information.
Materials and Methods
Experiment 1: free gaze conditions
Experimental tasks.
In experiment 1, we placed no constraints on eye movements because we wanted to replicate natural viewing conditions for self-motion. Our eventual aim was to examine the complex task of visually controlled steering, but we began by isolating subcomponents of the steering task using a series of simulated self-motion stimuli of increasing complexity. In the simplest case, we presented simulated self-motion across a textured ground plane (Fig. 1, flow). The simulated self-motion followed a winding trajectory, and instantaneous heading information was available from optic flow. Participants pressed buttons to indicate whether they perceived themselves to be heading to the left or right, or refrained from pressing if they thought they were traveling straight ahead. All of our visual stimuli contained this information, so we used the heading judgment task in all visual conditions to ensure equivalent motor responses and ensure attention was directed toward the display. Participants were familiarized with the heading task before scanning. Feedback was never provided, but performance was accurate. Rest blocks were interleaved with the experimental blocks.
Still frames from animated visual stimuli used to convey the impression of self-motion to participants. The panels show four different stimulus configurations in which participants made a continuous heading judgment, holding down left/right buttons to indicate the direction of their curved trajectory. In each condition, the visual stimulus specified travel down a path generated by a sum of sinusoids (see Materials and Methods). In the flow condition, ground plane optic flow specified instantaneous curved heading, but future path information was not available. In the road condition, the same information was available, plus information about upcoming changes to heading from the road edges (future path). In the horizontal road condition, the visual information from the ground plane was the same as in the flow condition, and the undulating visual motion of the “horizontal” road was not related to future path but matched the road condition for low-level features. In the error condition, the sinusoidal path traveled was out of phase with the road, producing a time-varying error signal between current position and the road center. This was designed to match visually the imperfect performance in our final condition, which was active steering. In the active steering condition, participants viewed the same initial stimulus as in the road condition, but, instead of making heading judgments, they controlled their own course. The response of the control device was similar to a motorbike, in which the duration of the pressure applied to the left or right buttons (or handlebar) determined the rate of rotation. The active steering condition resulted in a visual stimulus intermediate between the road and error conditions.
In our second condition (Fig. 1, road), we added road edges to the textured ground plane, allowing participants to anticipate their future course. No instructions were given as to whether explicit attention should be directed to the road. The observers were required to indicate their instantaneous heading as in condition 1. Both flow and road gave them relevant information about the current trajectory and upcoming changes (respectively) so we anticipated that they may attend to both components.
As well as conveying information about future path, the road feature added a number of low-level properties to the scene, such as the presence of additional moving edges, defined by luminance and color contrast with the ground plane. In addition to their sensory properties, the road edges were likely to act as an exogenous factor directing spatial attention and eye movements. Our aim was not to reveal brain areas associated with these low-level properties but to isolate regions activated by the specific configuration of these properties in a way that was informative about future path. Therefore, our third condition (Fig. 1, horizontal road) included all of the low-level features of the road but was uninformative about future path. This was achieved by drawing the roadway horizontally at a fixed simulated distance 5 m front of the viewer. Although the distance of the central position and the width of this roadway did not change, we varied its curvature to mimic changes in position when moving along a real roadway, so the horizontal road repeatedly straightened and then curved throughout the duration of the trial.
In our fourth condition (Fig. 1, error), the road was drawn in the same way as for the road condition, but the path traveled was not an exact match to the road. This produced a visual input equivalent to that experienced by a passenger in a poorly driven car introducing a visual position error signal into the stimuli. For both the horizontal road and error stimuli, the task was still to indicate instantaneous heading, and there were no explicit instructions as to whether participants should attend to the moving road. In this respect, the response task was equivalent across all four conditions.
Our final condition was “active steering”. Here, the initial visual display contained the same future path information as in road, but, instead of making heading judgments while passively traveling down the road, participants had to control their own direction of motion using button presses to try to follow the roadway. This manual task produced a similar number of button presses to the heading task. Although car drivers are familiar with rotating a wheel for steering, many other locomotor devices require lateral pressure to be applied for a period to effect steering (e.g., motorbike, bicycle, skis, and skateboards), and the button-press task implemented a similar principle. Participants received practice on the steering task before scanning. Performance was not perfect and resulted in occasional visual error signals similar to those generated experimentally in the error condition (Fig. 2). Participants were informed whether an upcoming trial required manual inputs for heading judgments or steering adjustments through a brief change of the background screen color before the trial was initiated. The color of the road edges during the trial followed the same color scheme, so participants were constantly aware of which condition they were in. Sample movies of the different trial types are available on-line, including examples of participant-controlled steering trials (available at www.jneurosci.org as supplemental material).
Performance on heading and steering tasks. The top depicts how button pressing reflected actual curved heading. The vertical lines indicate the points in time when an ideal observer would switch from a “left” press to a “right” press. The solid sinusoidal line shows the normalized rotation within the world to provide an impression of the magnitude of the rotation components potentially available to the participant at each point in time. The three broken lines show averaged participant performance for the left/right judgments in three conditions. The dashed line shows performance for flow, dotted for road, and dot-dash for error. The normalization means that a value of 1 indicates that all participants were holding down the left button at that time point, whereas −1 indicates that all were holding down the right button. For a detailed description of how the button-press responses were normalized, see Materials and Methods. Responses were typically lagged by ∼200 ms compared with the stimulus, and there were only minimal differences between the conditions. The bottom shows the steering performance of a single participant. Mean steering errors from the center of the road varied within 0.5 to 1 m between participants. The participant shown here fell in the middle of this range with mean errors of ∼0.8 m. The bold lines indicate the road position over time. Each thin gray line shows the course taken during a single trial (half of the trials used a reflected roadway, and these have been reflected back to allow overlap). The dashed line shows the course introduced during the error condition, which shows a greater average deviation than the participant.
Rest blocks were interleaved with the experimental conditions. During a rest block, the ground plane was replaced with a black rectangle, and the blue sky was the same as in the other conditions. Participants were free to move their eyes during rest blocks, and the only requirement was to wait for the beginning of the next experimental block.
Details of visual stimuli and tasks.
Stimuli were backprojected onto a screen at the top of the bore of the magnet using a Sanyo (Toyko, Japan) PLC-XP40L projector from outside the scanner room using a waveguide. The refresh rate was 50 Hz, and the resolution was 1024 × 768 pixels. Participants viewed the projection screen via a mirror attached to the head coil. The distance from the projection screen to the mirror was 716 mm, and the distance from the center of the mirror to the participants' eyes was 150 mm. The display subtended ∼35° horizontally and 30° vertically. DirectX 8.1 graphics libraries were used to render perspective correct two-dimensional projections of the three-dimensional scenes.
Each trial presented the viewer with a visual environment that contained a ground plane textured with gravel over which they traveled at a constant speed of 8 ms−1 (29 kph). Most conditions also contained a superimposed curving roadway. The edges of the roadway were 2 m apart and were generated using the sum-of-sines formula to generate the forward (z) and lateral (x) coordinates in meters: x = Dr sin(z/15) + sin(z/10). Roadway direction (Dr = 1 or −1, to generate a roadway to the right or left of the current position, respectively) was varied between scanning sessions. In the error condition, steering errors were generated experimentally by creating a locomotor path that was similar to that of the road but with rates of steering (angular motion) increased by one-fifth. This caused the path to follow bends of equivalent amplitude but out of phase with the roadway.
In the active steering task, participants controlled their direction of motion using a button box resting on their lap and were asked to steer to ensure they stayed within the roadway, ideally at the center of the road (1 m from each edge). Button presses caused a standard 0.5°/s change in rotational velocity (left button causing a negative change and right button a positive one), but holding the button down for longer than 0.4 s caused this rotational rate to increase at 6.5°/s/s. After releasing the button, the rotation rate remained the same but could be modified by additional button pressing. Steering rates across whole trials resulted in average rotational speeds of ∼5°/s and accelerations of ∼3°/s/s.
Steering trials, heading trials, and the rest condition each had a duration of 16 s. Each of the heading trials was presented eight times, whereas steering was presented 16 times and rest was presented 36 times. The extra steering blocks were included as a precaution against large variation in steering performance, so we would be able to model the hemodynamic response during trials with unrepresentative levels of performance separately from “standard” trials. The total set of trials was divided in four and scanned in four separate sessions to allow participants a chance to rest. Each session also contained other experimental conditions that are not reported here.
Behavioral data analysis.
In most conditions, participants used response buttons to indicate whether they were taking a path that was moving to the left or the right. Not only did this match some of the basic motor response properties of the steering task, but it also allowed us to see whether there were differences in sensitivity when different visual information was made available. Perfect performance would have been a square wave, as shown in the top of Figure 2. Actual performance was limited by the sensitivity of detecting rotation components from the visual display, and this was influenced by the curvature of the path being followed. Figure 2 superimposes the actual rate of rotation, normalized to the largest angular change to give some impression of the amount of rotation. To characterize human performance, we took the button response (a value of −1, 0, or +1 depending on whether the left button, no button, or the right button was held down) for each individual time point and summed this with other button responses at the exact same time point across trials of the same condition. From this, we created a new time course that represented how often each participant indicated motion to the left, right, or straight ahead for each time point of a complete trial. These time courses were averaged across participants and then normalized (to a −1 to +1 scale) by dividing the time course values by the maximum possible number of button presses (Fig. 2, top).
Subjects.
Twelve (three female) right-handed volunteers with normal vision participated in the initial experiment. Ages varied between 25 and 35 years. Data from two of these were not used because of poor performance in the steering task. Five of the original participants (one female) took part in the follow-up study. Before being scanned, each participant was trained in the steering and heading tasks and learned the color-code system that indicated when to perform each task. Time spent practicing steering varied between 10 and 30 min. The methods and procedure for the heading and steering tasks were reviewed and approved by an ethics committee at the University of Reading. The protocols for human fMRI were also reviewed and approved by the University of Reading, and all participants gave informed consent before participation in each session.
fMRI scanning.
Functional MR images were acquired on a Siemens (Munich, Germany) Trio 3T whole-body scanner, using an eight-channel head array coil. Echo planar imaging parameters were as follows: echo time (TE), 30 ms; repetition time (TR), 2500 ms; flip angle, 90°; slice thickness, 3 mm; interslice gap, 0; image matrix, 64 × 64; field of view (FOV), 192 × 192 mm; and functional voxel size, 3 × 3 × 3 mm. Forty-six axial slices were acquired in an interleaved sequence covering the whole brain. Prospective motion correction [PACE (for prospective acquisition correction)] was used to track any small head movements made by participants, allowing slices to be automatically repositioned between each TR (Lee et al., 1996). Before the first functional MR session, high-resolution anatomical images were acquired in 4 min 32 s (TE, 30 ms; TR, 1960 ms; flip angle, 11°; slice thickness, 3 mm; interslice gap, 0; image matrix, 256 × 256; FOV, 256 × 256 mm; voxel size, 1 × 1 × 1 mm; and 176 sagittal slices). For the first experiment, a functional scanning session lasted for 9 min 52 s (237 TR), and consisted of an interleaved sequence made up of 16 s blocks of the different experimental tasks. There were four sessions, using four different task orders.
fMRI data analysis.
The data from each participant were processed with Statistical Parametric Mapping software (SPM2) (Wellcome Department of Cognitive Neurology, University College London, London, UK). The SPM2 motion correction algorithm was applied to identify any scans in which interscan motion had not been negated by PACE prospective motion correction. No such scans were identified. Volumes from each scanning session were all coregistered to the first volume of the first scan session to remove any differences in head position or orientation between sessions. Normalization to the Montreal Neurological Institute (MNI) template was performed, followed by spatial smoothing (6 mm). The time series was high-pass filtered using a 128 s cutoff. The activation for each participant was modeled using a linear combination of functions obtained by convolving the known temporal profile of the experimental conditions with the standard hemodynamic function of SPM, plus its time derivative. For some participants, two more functions were included in the model corresponding to steering trials in which error was unusually high. The four testing sessions were separated in the model. Because the TR was relatively short, temporal autocorrelation between volumes was a potential problem, and so the SPM2 correction for serial correlations was applied. Each participant's data were analyzed using t contrasts between each of the conditions and the resting baseline. Contrast maps between the conditions were also created to carry forward to group analysis.
Group analyses were performed with random-effects analyses. The second level model was a one-way ANOVA design, with nonsphericity correction and adjustment for correlated repeated measures. Results were inspected using t contrasts, with a height threshold corresponding to p < 0.0001, and an extent threshold of 10 voxels. For contrasts between experimental conditions, an inclusive mask was used, such that voxels had to be significantly associated with task–baseline at p < 0.001 before being considered in the contrast. This was to avoid reporting a positive difference between the experimental and control conditions, in which, in fact, the absolute level of activation was not different from resting baseline. Such differences can arise as a result of areas of deactivation relative to baseline in the control condition. Results are reported in the MNI space.
Experiment 2: eye movement controls
Experiment 2 had two aims. The first of these was to replicate the road and flow conditions of experiment 1 under conditions of fixation, to rule out eye movements as an explanation of the differences in brain activation between these conditions. The second aim was to localize the various cortical eye fields in our participants independently of our main stimulus set. We anticipated that just preventing actual eye movements during the viewing of the road condition would not prevent activation in the cortical eye fields. This is because, under conditions of fixation, road was likely to produce an increase in planned but unexecuted eye movements relative to flow. This is especially likely in the case of the parietal eye fields (PEFs), which are also involved in shifting spatial attention independently of actual eye movements (Gottlieb and Goldberg, 1999; Bisley and Goldberg, 2003). Therefore, we asked whether any of the activation produced by the road fell outside the area of parietal cortex that is activated by shifting attention and making eye movements. We addressed this question using individual subject analysis rather than group analysis because we expected the regions of interest to lie in adjacent regions of cortex, and group averaging might have obscured these differences. As a final control, we also measured the eye movements that take place when free viewing our flow and road stimuli and the incidence of unintentional gaze shifts that occur in the fixation tasks.
Experimental tasks.
We repeated flow and road with fixation crosses positioned 2° above the horizon in the scene. We also used a serial saccade task to activate cortical eye fields. The serial saccade task consisted of 22 s blocks, which all began with a central fixation target. The fixation target was repositioned in the horizontal plane every 500 ms. The maximum eccentricity of the target from the display center was 12.5°. Each new target location was determined by randomly generating a direction (left or right) relative to the current position and randomly generating a value between 4° and 12° for the distance from the current location. If these parameters placed the new target outside the allowed area, then the generation procedure was repeated until an acceptable location was produced. We used a simple fixation condition for subtraction from the saccade task. Although some cortical areas may be engaged by both fixation and saccades, the rapid and demanding nature of the saccade task was expected to produce much higher levels of activation in these areas than fixation. The fact that saccades were made in response to a moving target meant that this task was likely to produce shifts of spatial attention preceding the saccades. This suited our purpose because it served as a control for saccades and attention shifts that occurred under free eye movement in the road condition, as well as the premotor attention shifts that may occur under fixation conditions in which planned saccades are suppressed.
Eye-movement recording.
To determine whether there are fundamental differences in the pattern of eye movements elicited when viewing our different conditions, we repeated the conditions from experiment 1 and 2 outside the scanner, with three of the participants who undertook both experiments 1 and 2, and reproduced the visual angles of the display used in the bore. We then measured eye movements using an SR Instruments Eyelink II (Osgoode, Ontario, Canada) to monitor the position of the dominant eye at 250 Hz, with a reported spatial resolution of 0.01°. This allowed us to ask whether fixation was equally efficient in the presence of the different visual stimuli, as well as to assess whether eye movement patterns differed between conditions when fixation was not enforced. Furthermore, the optic flow from the ground plane in many of the stimuli may have induced optokinetic nystagmus (OKN) (Niemann et al., 1999; Wilkie and Wann, 2006). Although control of OKN is normally attributed to subcortical areas, we wanted to establish whether OKN was present and ensure that there were no differences between experimental conditions. To assist in identifying OKN, we included extra stimuli that were not used in the scanner: a road without optic flow from the ground plane, and a fixation-only baseline. The power spectrum for the eye-movement data were analyzed using Matlab (MathWorks, Natick, MA) to identify OKN, and Matlab routines were written to identify saccades that met the criteria of being spatial shifts >0.5° with a velocity >30°/s.
fMRI scanning and analyses.
A session lasted 4 min 8 s (100 TR) and consisted of interleaved blocks of saccades (22 s), fixation (22 s), rest (16 s), flow plus fixation (16 s), and road plus fixation (16 s). There were also two other exploratory conditions included that are not reported here. There were four sessions using four different task orders and a total of eight blocks of each task across the four sessions. The data from experiment 2 were combined with the data from experiment 1 by repeating the preprocessing stream and including the sessions from both experiments in the same coregistration, realignment, and normalization steps. Comparing normalized functional images from the two experiments when the preprocessing steams were performed separately resulted in much worse agreement than when the preprocessing streams were combined.
The most efficient way to compare the conditions from experiment 1 and experiment 2 was conjunction analysis between pairs of conditions drawn from each of the two experiments. We therefore reanalyzed the data from experiment 1 for the five individuals who participated in experiment 2 and included conditions from experiment 2 as additional sessions in the design matrix. In conjunction analysis, voxels of interest are identified on the basis that they are significant in two or more separate contrasts, e.g., A > B and C > D. One way we applied this technique was to identify voxels that responded to the presence of the road whether or not participants were required to fixate. Conjunction analyses were performed at the group level by carrying forward individual contrast images to the second-level design matrix that resulted from the individual contrast components that were to make up the conjunction. For example, one contrast image might be carried forward from a given participant for each of A > B and C > D to perform one conjunction analysis at the group level.
Results
Experiment 1: free gaze
Behavioral results
The time course of the heading responses under the different visual conditions is compared in Figure 2 (top). There were only minimal differences between the heading responses to different visual stimuli.
From the nature of the sinusoidal path with five changes of direction, we might expect that the button would be held down five times per trial. On average, there were 5.6 ± 1.10 button presses (mean ± SD) during the nonsteering button-pressing conditions. The tendency toward an extra press was presumably caused by the occasional mistake such as releasing the button too early and then holding it down again, or judging the wrong direction and then changing to press the other button. There were no differences in number of presses between the four kinds of heading judgment trials (F(3,27) = 3.6; NS). The fMRI data for active steering were contrasted with the fMRI data from the road condition. Therefore, we compared the number of button presses in these two conditions. Active steering trials produced slightly more button presses (mean ± SD, 7.0 ± 1.0 vs 5.8 ± 1.0 for road), a small but significant increase (F(1,9) = 58.2; p < 0.01). This small difference in button pressing, however, suggests a reasonable match between active steering conditions and heading judgment conditions, at least at a simple motor input/output level, and, as such, the heading responses in road provide a useful control for the motor responses made in the active generation of your own course. We would not expect to see differences in brain activation related to the small difference in the number of button presses because fMRI block designs lack the sensitivity to detect differences in brain activity related to small behavioral differences such as 5.8 presses versus 7.0 presses without a great deal of signal averaging across a large number of scans. Although we wanted to approximately equate the number of presses between steering and road, there are qualitative differences of the kind fMRI is sensitive to in terms of the processes leading to those presses.
Performance in the steering task varied between the participants. Two participants, whose average steering error was >2 m relative to the center of the road were excluded from the study. For the remaining participants, if a steering trial produced unusually poor performance that was not representative of the general performance of a participant, then the blood oxygenation level-dependent (BOLD) response to that trial was modeled separately. This was the case for eight trials (5%) across the 10 participants. The average steering error across the remaining trials was 0.98 m (between participants SD 0.35 m, most accurate participant 0.48 m, and least accurate participant 1.75 m). The position over time relative to the road center is shown in the bottom of Figure 2 for the modal participant, along with the steering error simulated in the error condition for comparison.
Each condition was repeated eight times (except steering conditions, which had 16 trials), but trials were interleaved so that stimulus conditions were never identical on consecutive trials. Participants were also trained in both active tasks before taking part in the experiment to ensure that they were at a relatively stable level of performance at the beginning of the experiment. To examine whether learning took place, we compared behavioral performance in the first block with the last block. The magnitude of steering errors for the first four steering trials versus the last four steering trials did not differ significantly (t(9) = 0.23; NS), nor did the average number of button presses for the first two heading trials versus the last two heading trials (t(9) = 0.6; NS).
Imaging results
When compared with resting baseline, all of our conditions containing optic flow generated predictable activations in visual cortex extending to cover the expected locations of the middle temporal area (MT) and MST. However, our aim was to establish how this information is then used at a higher level for locomotion control, so we used selective contrasts that stripped out the common component related to optic flow.
Future path
To locate voxels specifically associated with the presence of future path information, we performed a whole-brain, second-level, random-effects analysis of the contrast road > flow as described in Materials and Methods. The main activation was a bilateral one in the superior parietal lobule (SPL), extending medially to the precuneus in the right hemisphere. The left peak occurred at (coordinates in x, y, z) (−16, −66, 58) and the right peak at (14, −62, 60). These coordinates place the activation very close to the PEFs, the potential homolog of monkey lateral intraparietal area LIP, as assessed using monkey and human fMRI (Koyama et al., 2004). A second bilateral focus of activation occurred in the vicinity of the MT+ motion complex (containing both areas MT and MST), with coordinates slightly inferior to those typically reported in studies that seek to localize MT+ (left peak, −42, −78, −6; right peak, 46, −80, −10). Both of these activations are shown in Figure 3. Table 1 gives the details of these activations as well as those reported below.
Bilateral response in SPL to future path information. Contrasting road with flow isolated activation attributable to the addition of the road boundaries, which provided future path information. The main activation, shown here in red on the mean anatomical image of the 10 participants, was a strong bilateral site in the SPL and superior part of the intra parietal sulcus. The coordinates for this activation place it very close to the parietal eye fields [putative human LIP (Koyama et al., 2004)]. The second bilateral focus of activation produced by this contrast can be seen easily at slice y = −77 and corresponds to the coordinates of MT+. The salient visual motion produced by the road edges against the textured background may account for this. This interpretation is supported by the activation for the visual control condition, horizontal road > flow, which was confined to the blue area in the region of left MT+. The correspondence between voxel t values and the plotted color is shown in the inset.
The horizontal road condition shared low-level visual features with road condition. To isolate voxels responsive to these low-level features, we used the contrast horizontal road > flow. This contrast produced no activation in the SPL region identified using road > flow. However, it did produce left hemisphere activation in the vicinity of MT+ (compare the locations of blue/green and red/yellow activations in Fig. 3), and the location of this was similar to that for road > flow (peak, −42, −66, 0). This suggests that the SPL activation for road > flow was not attributable to low-level visual properties, whereas the MT+ activation it produced was. The same bilateral SPL region was also activated during the error condition and active steering condition in which the road was present.
Steering errors
The experiment included two conditions in which the locomotor path did not perfectly match the road (error and active steering). These introduced an additional source of visual information in the form of a position error, over and above instantaneous heading and future path that were present in the road condition. In the error condition, the only difference from the road stimuli was the presence of the visual error signal, whereas with active steering, there were a number of task differences. We isolated voxels responding specifically to the visual error signal using the conjunction of error > road and active steering > road. Figure 4 shows these voxels in green/blue. The main activation associated with the position error signal was in the left SPL lying just anterior to the area selective for future path (peak, −26, −54, 56). The future path activation (Fig. 3) and the error activation (Fig. 4) in the left hemisphere showed a small overlap, but the main part of the error activation was adjacent to the future path activation. This can be seen by comparing the slices between y = −59 and y = −47 in Figures 3 and 4. The future path activation was bilateral, whereas there was no activation in the corresponding location in the right hemisphere associated with the error signal. As well as lying slightly anterior to the future path selective area, the error signal area was more extended in the medial direction, and so part of it was located in the left precuneus. A much smaller activation associated with the conjunction of error and active steering occurred in the right occipital gyrus (shown in the top left slice of Fig. 4) (peak, 22, −86, 30).
Brain areas involved in registering steering errors and making steering adjustments. In the active steering condition, imperfect performance resulted in a position error signal relative to the road center. We also included a position error signal during the heading task in the error condition. We isolated voxels responding specifically to the visual error signal using the conjunction of error > road and active steering > road. These voxels are shown in blue. The main error-related area was in the left hemisphere, immediately anterior to the future path area shown in Figure 3. There were also a number of areas, shown in red, involved in active steering > road that were not active in error > road. These areas may be involved in generating steering adjustments from visual information. These activations were bilateral cerebellum, left dominant dorsal premotor cortex, and activation in dorsal medial frontal cortex, which we confirmed as the supplementary eye fields in experiment 2. Coronal slices in the figure are spaced 3 mm apart, but, at some points, we skip slices to save space; there was no activation in the skipped slices. The correspondence between voxel t values and the plotted color is shown in the inset.
Steering adjustments
We also identified voxels for active steering > road that were not active in error > road by using the latter contrast as an exclusive mask to the former, and we assume that these were involved in aspects of the steering task other than processing visual position error signals. These areas are shown in red/yellow in Figure 4. The first of these was on the dorsal surface of medial frontal cortex. We suspected this might be the SEFs and investigated this possibility further in experiment 2. Second, there was a right dominant bilateral activation of the anterior cerebellum. Finally, there was a left dominant bilateral activation in dorsal premotor cortex. Detailed coordinates for these activations can be found in Table 1.
Experiment 2: eye movement controls
Nature of activation produced by the road
All five participants showed bilateral SPL activation for road > flow when fixating in both conditions, just as they had for free eye movements. When eye movements were unconstrained, in experiment 1, activation in SPL may have occurred as a result of an increase in eye movements produced by the road. This activation was very close to the PEFs. The PEF contains a topographic map of visual saliency and is involved in selecting saccade targets as well as making shifts of spatial attention (Gottlieb, 2007). With fixation, this activation may have occurred as a result of shifts of spatial attention produced by the road or the deliberate suppression of eye movements. As a first step toward ruling out these factors as a complete explanation of the experiment 1 findings, we assessed the conjunction between the two contrasts, with and without fixation. This ruled out voxels in which the response to the road differed depending on whether eye movements were made or not. All five participants showed bilateral SPL activation in this analysis. The possibility remained that the activated areas were ones with a general involvement in saccade planning, spatial attention, and the suppression of eye movements. To remove this possibility, we repeated the contrast excluding all voxels that were more active at the liberal threshold of p < 0.01 in the saccade task than during fixation. After this step, extensive bilateral SPL activation remained in three of the five participants (Fig. 5, red/yellow areas). This activation was evident to a lesser extent in the fourth participant (KW) and was very limited in the fifth participant (RW) (for a magnified view of the middle three columns of Fig. 5, see supplemental Fig. 1, available at www.jneurosci.org as supplemental material). This refines our early observations, excluding the role of saccades or shifts of attention, and we propose that these areas are involved in the perceptual processing of future path information. We also identified voxels whose activation was attributable to eye movements and spatial attention shifts made in response to the road stimulus using the three-way conjunction of saccades > fixation and the two road > flow contrasts (with and without fixation). Figure 5 shows these areas in blue/green. Given that these areas were activated by the saccade task, they most probably correspond to the PEF.
SPL activation attributable to future path information partitioned into components related and unrelated to eye movements and spatial attention. Each row shows coronal slices from an individual participant. Red areas were active in the contrast road > flow, both with and without fixation, but were not activated by the saccade localizer task. Therefore, their activation is attributable to the future path information content of the road and is not attributable to eye movements or shifts of spatial attention that it caused. Blue areas were activated by the saccade task as well as road > flow, both with and without fixation. This indicates that the road did produce shifts of attention and eye movements, and the blue areas therefore indicate the location of the PEFs. The correspondence between voxel t values and the plotted color is shown in the inset.
Medial frontal cortex activation for steering is the supplementary eye field
Active steering requires coordinated eye and limb movements (Wilkie and Wann, 2003), and the task we used here could have activated both the SEF and the adjacent hand representation in the supplementary motor area (SMA), because of the need for accurately timed finger movements coordinated with eye movements. To assess this, we used the saccade task > fixation contrast to identify dorsal medial frontal cortical activation that corresponded to the SEFs. If activation for active steering > road occupied the same location as that identified using the saccade task, then this would suggest that it corresponded to the SEFs. If it lay adjacent to the saccade task activation, this would suggest that it corresponded to the adjacent SMA. To search for steering-related activation in the SEF, for each of the five participants, we ran the conjunction of active steering > road and saccade > fixation. In three of the five participants, this produced dorsal medial frontal activation. To search for steering-related activation in the SMA, we ran the contrast active steering > road excluding any voxels that reached a threshold of p < 0.01 in saccade > fixation. There was no activation for this contrast in the medial frontal cortex of any of the five participants. Together, these analyses suggest that the group activation shown in Figure 4 corresponds to the SEFs.
Cerebellar activation for steering cannot be explained by saccades
The bilateral, right dominant activation for active steering > road (Fig. 4, slices y = −44 and −41) may have reflected additional eye movements made in the steering task relative to the road condition. To rule out the making of saccades as an explanation of this activation, we repeated the contrast for the five participants, this time excluding voxels that reached a threshold of p < 0.01 in saccade > fixation. Bilateral cerebellar activation remained in all five participants, ruling out a simple explanation in terms of the number of saccades made.
Recorded eye movements
It could be anticipated that the optic flow stimuli might give rise to some degree of OKN, which for our simulated locomotor velocity should have been in the range of 1.5–2 Hz (cf. Niemann et al., 1999). We used Fourier analysis to estimate the spectral power attributable to OKN, and this accounted for only 2.9% of the power spectrum from 0–10 Hz. This percentage did not vary across the free gaze conditions containing flow only (3.0 ± 0.6%, mean ± SD), road only (2.9 ± 0.3%), both flow and road (2.9 ± 0.5%), or the active steering condition with flow and road (2.9 ± 0.6%). There were no differences in the frequency range expected for OKN found across the equivalent fixation conditions (3.6 ± 0.6, 3.2 ± 0.5, and 3.4 ± 0.4%) or with just fixation (3.1 ± 0.4%), and overall eye movements at the 1.5–2 Hz range accounted for 3.3% of the power spectrum. Because the power in the expected OKN range when optic flow was present was similar to that when it was not present (road only and simple fixation), this suggests that, with these particular stimuli, optic flow did not induce significant OKN. This is consistent with the findings of Wilkie and Wann (2006) regarding OKN with very similar displays. The difference compared with the findings of Niemann et al. (1999) might be attributed to the fact they used high-contrast black on white dots for their flow, whereas the flow here and in the study of Wilkie and Wann (2006) were homogenous textures lacking clear feature boundaries.
In terms of adherence to the fixation requirement, the SD of the gaze position was 0.17° with a maximum excursion of 0.54°, and this showed little variation across the flow (SD gaze position, 0.16; maximum excursion, 0.50), road without flow (SD gaze position, 0.18; maximum excursion, 0.61), or road with flow (SD gaze position, 0.15; maximum excursion, 0.49) experimental conditions. The numbers of saccades >0.5° was very few. During plain fixation, there were on average one saccade per 16 s of fixation. With flow, only there was a saccade once in 28 s, with road only once in 19 s, and when road and flow were present there was one in every 56 s of fixation. Eye movement traces for the fixation condition are shown in supplemental Figure 2 (available at www.jneurosci.org as supplemental material).
In the free gaze conditions, as expected, shifts of gaze (>0.5°) were much more common. For the experimental conditions with flow only, road only, both flow and road, or active steering, there were 2.13 ± 0.37, 1.56 ± 0.40, 1.82 ± 0.31, and 1.86 ± 0.55 (mean ± SD) saccades per second, respectively.
Given these observations, there are no reasons to suppose that the differences in the patterns of eye movements elicited by the stimuli underlie the differences we observed through selective contrasts of the BOLD signal.
Discussion
Visual control of the direction of locomotion is only possible because of the presence of coordinated neural systems. These involve a series of stages beginning with identifying the current direction of travel and the desired future path and then making the necessary visuomotor transformations for the current mode of travel. The most basic process is to pick up the current direction of travel (heading). In monkeys, both MSTd and the VIP have been implicated in processing the current direction of travel from ground plane flow, and human homologs are suggested by the studies of Morrone et al. (2000) and Smith et al. (2006). Area VIP is also sensitive to multisensory inputs, and this property was used to identify a human homolog of VIP (Bremmer et al., 2001). Our experimental contrasts did not reveal MSTd or VIP because the optic flow and heading information that these areas are sensitive to was present in the control conditions as well as the experimental conditions, but we identified brain areas involved in subsequent stages of the visual control of locomotion: future path processing, steering error detection, and making steering adjustments.
When we added information about future path to a stimulus that already specified instantaneous heading, it resulted in a strong bilateral activation of the SPL, as well as some activation in MT+. A control condition that shared low-level visual features, but had low behavioral salience, produced similar MT+ activation but did not activate the SPL. The presence of future path information resulted in strong activations, although the behavioral heading task did not require it to be processed. This suggests that participants were able to simultaneously process future path while attention was primarily directed toward judging heading.
Parietal cortex contains a topographic map of salience, the PEFs, and it is consistent with the proposed functions of this area that behaviorally relevant spatially distributed stimuli activate it, whereas a matched stimulus that lacks relevance does not (Gottlieb, 2007). Moving beyond the pervasive explanation that this is “just a spatial attention effect,” we would suggest that one of the reasons that functional regions such as PEFs have evolved is because of the fundamental survival behaviors of locomotion and foraging. Therefore, we do not view its activation here as artifactual. We wanted to refine our observations, however, and with experiment 2, we established that a part of the SPL activation attributable to future path information was not related to eye movements and shifts of attention provoked by the stimulus and therefore extends outside of the functions defined for the PEF. This suggests a new functional area, associated with path prediction. We propose that the area is specialized for visual information used to set the parameters for upcoming visuomotor transformations, which is a function that goes beyond the spatial allocation of attention in PEFs. For the stimuli used here, path prediction might be accomplished in the area we identified by processing the motion and optical curvature of road edges, but, in other situations, the specific features processed to achieve path prediction may be different. The presence of the road, however, did produce activation in PEFs, indicating that a complimentary function of future path information is to guide the allocation of attention and eye movement to parts of the visual scene that are critical for the visual control of locomotion.
Using the error and active steering conditions, we were able to identify an additional functional area in parietal cortex posterior to somatosensory cortex, associated with processing a visual feedback error signal. This activation was strongly lateralized to the left hemisphere. One possibility is that this lateralization reflects the use of the right hand in the steering task. Because the lateralized response to the presence of a visual error signal also occurred when the motor responses were matched across conditions and the task did not involve the actual correction of the error (error > road), it is also possible that this is a genuinely lateralized function.
The visual control of steering also requires the appropriate transformation of visual information to make a coordinated response. If steering were based purely on the moment-by-moment correction of heading error, this would result in an unstable system prone to oscillation. To enable prospective control, future path information must be combined with a model of the response properties of the steering device, thereby minimizing the iterative error correction component of the task. The bilateral activation of the anterior cerebellum we observed in the active steering condition is interesting in this regard. The cerebellum has been implicated in internal forward models (Ramnani, 2006). Internal forward models require updating and refinement, and this process may be achieved in the cerebellum by monitoring the discrepancy between the forward model prediction and sensory feedback of the actual consequences of action. In a positron emission tomography study, Blakemore et al. (2001) implicated the cerebellum in signaling this sensory discrepancy. The location of the activation they report is very close to that reported here for the steering task, although activation was unilateral in that study. The cerebellar activity could also reflect a number of other basic functions that have been attributed to the cerebellum. One possibility is that the bilateral activation reflected the need to coordinate the motor responses of two effectors, in this case eyes and fingers (Miall et al., 2000, 2001; Miall and Reckess, 2002; Miall and Jenkinson, 2005). Behavioral studies have highlighted the tight coupling between eye movements and steering adjustments (Land and Tatler, 2001; Wilkie and Wann, 2003). Our data enabled us to rule out a difference in the quantity of eye movements between steering and the control conditions as an explanation of the cerebellar activation, but we cannot rule out the requirements of temporal coordination at this stage. Another possibility is that the cerebellar activity reflected the precise timing requirements of the button presses in the steering task (Ivry and Spencer, 2004; Xu et al., 2006). Our control device mapped duration of press onto angle of rotation in the simulated visual scene. It was necessary to plan an appropriate duration of press and then execute the press for that duration. The button presses in the heading task, which was subtracted from active steering, did not require this kind of planning and execution. Relying only on visual feedback of when the correct locomotor path was achieved was an ineffective strategy because of the time lags in the system. This reflected real-world steering control, such as that involved in riding a motorbike, in which the duration for which pressure is applied to one side of the bike determines path curvature. We are conducting additional studies to establish the exact function of the cerebellum in the steering task.
We observed two more activations specific to active steering in both motor areas that are reciprocally connected with the cerebellum (Ramnani, 2006). Activation in the FEFs was similar for steering and the control task, but activation in the SEFs was notably higher for steering. Both tasks evoked eye movements, but, in steering, eye movements may be critical to identifying key features of the road and the control of steering (Land and Lee, 1994; Wilkie and Wann, 2003, 2006). The SEF activation may have reflected the greater planning element involved in the eye movements for steering. A final observation was of specific activation in dorsal premotor cortex caused by steering. This may be related to controlling finger movements contingent on the stringent timing requirements of our steering task. We reported previously activation in the same location specific to a time-to-contact task (Field and Wann, 2005), consistent with the idea that this area is involved in time critical motor responses.
In conclusion, we moved beyond previous observations on the detection of heading and identified neural activity that supports the visual control of locomotion. On the visual side of the task, we highlighted brain areas involved in detecting future path information and processing visual feedback error signals (e.g., current lane position). On the motor control side, we identified a network of three areas that may together form a control loop for generating steering corrections.
Footnotes
-
This work was supported by Engineering and Physical Sciences Research Council Grant EP/D055342/1. We thank Alice Cruikshank for assistance with eye movement recording.
- Correspondence should be addressed to David Field, Department of Psychology, University of Reading, Reading RG6 6AL, UK.d.t.field{at}reading.ac.uk
