Functional Magnetic Resonance Imaging Examination of Two Modular Architectures for Switching Multiple Internal Models

Introduction

Many studies have investigated the neural correlates of task switching using a variety of paradigms, such as switching a stimulus–response correspondence (Dove et al., 2000) or changing a game rule “cognitive set” (Konishi et al., 1998; Nakahara et al., 2002). These studies have revealed the involvement of the prefrontal (Konishi et al., 1998; Nakahara et al., 2002) and the parietal (Dove et al., 2000) cortices and the cerebellum (Bischoff-Grethe et al., 2002). However, little is known about the global mechanisms underlying the switching.

Two representative computational theories have most thoroughly been investigated for task switching: a mixture-of-experts model for centralized switching (Jacobs and Jordan, 1991; Gomi and Kawato, 1993; Graybiel et al., 1994; Jordan and Jacobs, 1994) and the MOSAIC (MOdular Selection And Identification for Control) model for parallel-distributed switching (Kawato and Wolpert, 1998; Wolpert and Kawato, 1998; Wolpert et al., 1998; Wolpert and Ghahramani, 2000; Haruno et al., 2001; Doya et al., 2002; Wolpert et al., 2003). In the mixture-of-experts model (Fig. 1A), outputs from function approximators known as expert modules are linearly combined by a classifier known as a gating module. They are trained so as to split the input data into subparts in which particular experts are specialized. The gating module weights the contribution of the output of each expert module to the final output according to its estimation of the probability that each expert is appropriate for the current context. Thus, the selection is centralized in the gating module and segregated from the expert modules. A mixture-of-experts architecture predicts that switching activity is segregated from an activity related to expert modules.

In contrast, in the MOSAIC (Fig. 1B), internal models that are experts at mimicking the input–output properties of controlled objects (Kawato et al., 1987; Miall et al., 1993; Shadmehr and Mussa-Ivaldi, 1994; Wolpert et al., 1995; Flanagan and Wing, 1997; Kawato, 1999) crucially contribute to selection. Multiple pairs of forward internal models (predictors) and inverse internal models (controllers) are tightly coupled as functional units. For example, when we use a new pair of scissors, forward models (Blakemore et al., 1998; Iacoboni, 2001; Miall et al., 2001) of various types of scissors simultaneously predict sensory feedback from an efference copy of motor commands. The prediction of each forward model is then compared with actual feedback. The smaller the error, the more likely that the forward model must have been an appropriate predictor in the current context. The inverse model paired with the appropriate predictor is considered as an appropriate controller. Thus, the selection mechanism crucially depends on the internal models (i.e., forward models must be active when switching internal models). Consequently, the MOSAIC predicts that the switch activity spatially and temporally overlaps the internal model activity. Our objective is to examine the predictions of the two architectures investigating brain activity related to task switching for novel tools. Our previous study (Imamizu et al., 2003) found modular organization of the internal models in the cerebellum, but the cerebro-cerebellar neural mechanism for the switching of the internal models is unknown.

Materials and Methods

Experimental design. Our previous studies demonstrated that activity reflecting an internal model of a novel tool (a rotated mouse, the cursor of which appears in a rotated position) increases in the cerebellum after learning use of the tool (Imamizu et al., 2000) and that the internal models send output to the premotor (PM) regions (Tamada et al., 1999). We also investigated cerebellar activity after learning use of two novel tools (the rotated mouse and a velocity mouse, the cursor velocity of which is proportional to the mouse position). Activities of the two different tools were segregated spatially with a very small overlap, suggesting that the multiple internal models are acquired in a modular manner (Imamizu et al., 2003). Based on these experimental paradigms, we investigated brain activity related to switching internal models for novel tools and examined the predictions of the two architectures.

Subjects manipulated the rotated, the velocity, or the normal mouse to track a randomly moving target on a screen with a cursor (a tracking task). The x and y components of the target path were each sums of sinusoids, the amplitude and frequency of which were determined randomly. After intensive training in using the novel mice, brain activity was scanned when the subjects conducted the tracking task using one of the three mice. The subjects manipulated the same type of mouse within a time block, the duration of which was determined randomly in a range of 26.5–42.4 sec. Four types of discontinuous transitions randomly split these blocks: (1) the mouse type changed, and cognitive cues were presented; (2) the mouse type did not change, and the cues were presented; (3) the mouse type changed, and the cues were not presented; and (4) the mouse type did not change, and the cues were not presented. The cognitive cues were cursor colors corresponding to the mouse type (red, violet, or green) and text presented on the screen (“rotate,” “velocity,” or “normal”). At every transition between blocks, the cursor was reset to the center of the screen while the target continued to move on a smooth path.

In our previous study (Imamizu et al., 2003), our purpose was to investigate regional difference in the cerebellar activity related to the internal models. Thus, the experimental design was simple (i.e., the type of mouse changed at a constant interval), and there was only one type of transition between the blocks (the mouse type changed, and cognitive cues were presented). However, in the current study, the experimental design was modified to investigate the switching mechanism of the internal models. The block was determined randomly so that the subjects could not anticipate when the mouse type would change. There were four types of transitions between blocks to dissociate various factors related to the switching (e.g., the mouse-type change, the cognitive cue, and the reset of the cursor). We scanned the whole brain to investigate the global switching mechanism.

Subjects. Ten neurologically normal subjects (21–39 years of age; two females and eight males) participated in the experiments. All participants were right-handed (Oldfield, 1971). Informed written consent was obtained from each subject. The protocol was approved by the ethics committee of Advanced Telecommunications Research Institute.

Tasks. A projector displayed the target and the cursor. A small white square target was presented on a dark background. The target moved within a square area, subtending horizontal and vertical visual angles of 23.36° and 19.71°, respectively. The subjects moved a small cross-hair cursor on the screen with the mouse.

The relationship between the cursor position and the mouse position for the rotated mouse was: (1) where (x_c, y_c) denotes the screen coordinates of the cursor (visual angle, °), and (x_m, y_m) denotes the mouse/hand coordinates (centimeters). In contrast, the velocity of the cursor (x_c, y_c) (degrees per second) was determined by the mouse position for the velocity mouse: (2) During the training sessions over the 5 d, brain activity was not scanned, but the subjects performed the task lying on a bed as they would in a magnetic resonance imaging (MRI) scanner. Two subjects did not participate in the training sessions because they had already received intensive training for using the novel mice in our other experiment (Imamizu et al., 2003). Each session lasted 5.63 min (days 1 and 2) or 14.22 min (days 3–5), and the mouse type did not change during the session. The subjects used the normal mouse in the first session of each morning and then were trained for only one of the two novel mice in the subsequent six sessions (days 1 and 2) or three sessions (days 3–5). Breaks between sessions lasted 5–10 min. After a 1–2 hr lunch break, the subjects used the normal mouse in the first session of each afternoon and then were trained for the other novel mouse with the same number of sessions as in the morning.

All subjects underwent six functional MRI (fMRI) scanning sessions after the training sessions, with each session lasting 14.22 min. In each session, the rotated mouse, the velocity mouse, and the normal mouse changed in a pseudo-random order. A cognitive cue was presented in every other session. The length of the time block in which the subjects continuously used the same type of mouse was pseudo-randomly determined in a range of 26.5–42.4 sec (five to eight fMRI scanning intervals).

Analysis of behavioral data. The cursor (x_c, y_c), the target (x_t, y_t), and the mouse (x_m, y_m) positions were sampled at 1 kHz. The distance between the cursor and the target at each sampling point: (3) was accumulated over 5.3 sec (position tracking error). A velocity tracking error: (4) was also accumulated over 5.3 sec. Because results obtained from the velocity error were similar to those from the position error, we only present the position tracking error in this article.

MRI acquisition. A 1.5 T MRI scanner (Shimadzu-Marconi) was used to obtain blood oxygen level-dependent (BOLD) contrast functional images. Images weighted with the apparent transverse relaxation time were obtained with an echoplanar imaging sequence (repetition time, 5.3 sec; echo time, 65 msec; flip angle, 90°). One hundred sixty-three sequential whole-brain volumes (64 × 64 × 44 voxels at 3.5 mm isotropic resolution) were acquired in each session. High-resolution anatomical images of all subjects were also acquired with a T1-weighted sequence.

MRI analysis. We used SPM99 software (http://www.fil.ion.ucl.ac.uk/spm/) for image processing and analysis. The first two volumes of images were discarded to allow for T1 equilibration, whereas the remaining 161 image volumes were realigned to the first volume and normalized to the Montreal Neurological Institute (MNI; Montreal, Canada) reference brain. The data were smoothed spatially with a Gaussian kernel with a 7 mm full-width at half maximum (FWHM). The voxel time series was smoothed temporally with a Gaussian filter (FWHM of 4 sec).

A model for investigating activity related to switching of mouse type. We conducted a multiple regression analysis to find regions related to the switching of the mouse type. Three factors were included as main explanatory variables: (5) Here, S^k_i denotes the fMRI signal at the i-th voxel in the k-th scan, and x, y, and z are explanatory variables representing the switching of the mouse type, the presentation of the cognitive cues for switching, and the cursor reset, respectively. Each variable was a pulse function assigned 1 in the scan immediately after each event occurred and 0 otherwise and convolved with a canonical hemodynamic function as implemented in SPM99. The four transition types were represented by combinations of the explanatory variables (x, y, z): (1) the mouse type changed, and cognitive cues were presented: (1, 1, 1); (2) the mouse type did not change, and the cues were presented: (0, 1, 1); (3) the mouse type changed, and the cues were not presented: (1, 0, 1); and (4) the mouse type did not change, and the cues were not presented: (0, 0, 1). This analysis allows us to distinguish activity related to the switching of the mouse type from activities related to the cognitive cues and behavioral changes that were evoked by the cursor reset (e.g., transient increase of the tracking error, attention, effort levels, and eye/hand movements).

We performed a random effect analysis as follows. Images of parameter estimates for the contrast of interest (α, β, and γ) were created for each subject (first-level analysis) and were then entered into a second-level analysis to test whether each parameter was significantly larger than zero, using a one-sample t test across subjects (t₍₉₎ > 4.3; p < 0.001 uncorrected; cluster size, >50 voxels).

In an event-related analysis of activity related to the switching of the mouse type (as shown in Figs. 5 and 6), we investigated time courses of the relative BOLD signal that is expressed as a percentage of signal increase at each voxel from the averaged signal across the whole brain and a session.

A model for investigating spatial overlap between activity related to behavioral switch and activity related to internal model. We conducted the second regression analysis to find regions in which behavioral switch activity spatially overlaps the internal model activity. In this regression analysis, the linear model was: (6) where s is an explanatory variable representing the behavioral switch. It was assigned 1 in the scans immediately after the switch of the mouse type, the cognitive cue presentation, or the cursor reset or 0 in the other scans. In the previous analysis, we investigated activity evoked by mouse-type switching, whereas, in the current analysis, we investigated activity related to the behavioral switch that was evoked by the discontinuous transitions between blocks (the switch of the mouse type, the cognitive cue presentation, or the cursor reset). We also included three explanatory variables (r, v, and n) representing the mouse type to investigate internal model activity. They correspond to the rotated mouse, the velocity mouse, and the normal mouse and were assigned 1 if the scan corresponded to its mouse type and 0 otherwise (assigned 0 in the first scan after the mouse type changed). Contrasts of interest in a first-level analysis were δ, (ϵ – η) and (ζ – η) for each subject. In a second-level analysis, we tested whether an estimated parameter for the behavioral switch was significantly larger than zero (δ > 0) and whether a parameter for the rotated mouse or the velocity mouse was larger than a parameter for the normal mouse (ϵ – η > 0or ζ – η > 0), using a one-sample t test across subjects (t₍₉₎ > 4.3; p < 0.001 uncorrected; cluster size, >50 voxels). Thus, regions related to the rotated mouse (or the velocity mouse) are those significantly more activated during use of the rotated mouse (or the velocity mouse) than the normal mouse.

We searched for regions in which the behavioral switch activity overlaps with activity related to the rotated mouse or the velocity mouse and defined the anatomical volume(s) of interest (VOI), including the overlapping regions. Regarding the cerebral cortex, the VOI was defined according to Tzourio-Mazoyer et al. (2002). The cerebellum was divided into six VOI: medial volume, including the vermis and the intermediate parts (–20 < x < 20 in MNI coordinates), and the remaining two lateral volumes were each divided into the superior and the inferior volumes by a horizontal plane (z = 40) that roughly corresponds to the horizontal fissure. We calculated the percentage of overlapping volume to the behavioral switch activity within each VOI.

Results

Discussion

Activity associated with the mouse-type change was found in area 46, the insula, the AP regions, and the anterior part of the lateral cerebellum. According to the event-related analysis, the ratio of the sustained internal model activity to the transient switching-related activity was large in the cerebellum and the AP region, whereas it was small in area 46 and the insula. In accordance with this temporal overlap analysis, spatial overlap between the behavioral switch activity and the internal model activity was not observed in area 46 or the insula but observed in the PM, the AP regions and the cerebellum. Furthermore, the largest overlapping volume was found in the cerebellum. Based on the predictions of the mixture-of-expert and MOSAIC architectures, the current results can be interpreted within the two different schemata as follows.

One of schema assumes, as shown in Figure 8A, that the internal models are located in the cerebellum, whereas the central executives of the MOSAIC are located among the cerebral regions. The responsibility estimator (RE) that decides the responsibility of each module for the current context may consist of several components. The components closely related to internal models (the blue and red part of the RE) are in the premotor (PM) and the AP regions, where the sustained activity was observed, whereas the components conducting the final computation of the responsibility (“genuine” switch functions; the red part) are in the insula and area 46, where the transient activity was dominant. This model provides one concrete computational mechanism for prefronto–parieto–cerebellar connectivity revealed by electrophysiological studies (Sasaki et al., 1977) and transneuronal–tracer studies (Middleton and Strick, 1994; Clower et al., 2001; Middleton and Strick, 2001; Dum and Strick, 2003; Kelly and Strick, 2003). In particular, a recent study (Kelly and Strick, 2003) identified putative, minute reciprocal one-to-one connections between the prefrontal (area 46) and the cerebellum. Internal models in the cerebellum compete to estimate the current context, and their estimation errors are sent to the prefrontal RE. Based on the error information, the prefrontal estimator decides the responsibility of each internal model and gives the cerebellum feedback about the degree of the contribution of the internal model to the total output. Therefore, reciprocal one-to-one connections between each cerebellar internal model and the prefrontal counterpart are essential for this neural implementation of the MOSAIC model.

Another component of the MOSAIC architecture is a responsibility predictor (RP) that predicts the responsibility of the module from sensory and cognitive information. For example, before lifting an object, we can roughly estimate its weight from visual information (e.g., object size and surface texture). The responsibility predictor can foresee the responsibility before movement onset. The activity in the occipital, temporal, and superior parietal regions increased immediately after the presentation of the cognitive cues for switching. Visual information processed in the occipital and temporal regions converges at the superior parietal regions. The responsibility predictor may also consist of several components distributed between the cerebrum (red parts) and the cerebellum (blue parts), and components closely related to sensory processing may exist in the superior parietal regions. This is also consistent with anatomical connectivity between the posterior parietal cortex and the cerebellum (Clower et al., 2001).

The other schema, illustrated in Figure 8B, assumes that the mixture-of-experts and MOSAIC architectures exist in distinct brain regions. Area 46 and the insula, where the transient switch activity was dominant and no spatial overlap with internal model activity was detected, probably assume the role of the gating module. The PM, the AP, and the cerebellar regions, where sustained activity specific to each novel mouse was found (Fig. 7), probably correspond to expert modules. Why are the two types of architecture necessary in the brain? Sensory feedback as a consequence of our own behavior is essential for the MOSAIC because the responsibility of the modules is determined according to the error between the prediction of the forward model and the actual sensory feedback. However, direct and continuous sensory feedback is sometimes unavailable for switching in several artificial circumstances with which modern people need to deal. In such cases, the mixture-of-experts or other mechanisms in the cerebral cortex are expected to play an important role. Many imaging studies have used tasks in which direct sensory feedback was unavailable. For example, Dove et al. (2000) used a task in which a simple relationship between a stimulus and a response changed repeatedly (switching of S-R mappings). They found that the supplementary motor area, the lateral prefrontal cortex, the parietal regions, the insula, and the cuneous/precuneous were related to the switching but the cerebellar activity was not reported. Bischoff-Grethe et al. (2002) suggested that the cerebellum also contributes to reassignment of the S-R mappings. The cerebellar activity (in about y = –66) was located posterior to that in our experiment (y = –48). This is consistent with the assumption that the more abstract cognitive functions such as arbitrary S-R mapping are located in posterior regions in comparison with more basic sensory–motor integration (for review, see Desmond and Fiez, 1998).

In the current study, transient activity related to the switching of the mouse type was observed in the dorsolateral prefrontal cortex (area 46). According to a previous study (D'Esposito et al., 1995), this region includes a central executive system that selects information stored in short-term memory buffers and plays an important role in dual-task performance (e.g., concurrent accomplishment of semantic–judgment tasks and spatial–judgment tasks). Other studies used a spatio-temporal working memory task and reported that area 46 was active when the subjects selected items from working memory rather than when they maintained the items in memory (Rowe and Passingham, 2001; Sakai et al., 2002). These studies suggest that area 46 is involved in the control of information flow according to contexts (i.e., what the gating module is supposed to do in the mixture-of-experts architecture). The Wisconsin Card Sorting Test (WCST) has been used to probe behavioral flexibility [i.e., the ability to shift from one response tendency (cognitive set) to another that is suitable for the current context]. Previous studies (Konishi et al., 1998; Nakahara et al., 2002) indicated that the posterior part of the inferior frontal sulcus (area 44/45) plays an essential role in the WCST. This region is located next to area 46 and probably has similar functions.

We examined the possibilities that the spatial and temporal overlaps found in the cerebellum were consequences of method limitations or artifacts of data processing. If the gating module and the expert module are located so closely that we cannot distinguish between them within the limit of fMRI spatial resolution, the transient activity related to the switching and the sustained activity specific to the novel mice may overlap each other even if they are anatomically distinct. However, this is improbable, first because we successfully separated the transient activity from the sustained activity in the cerebral cortex for the same subjects while using the same methods as in the cerebellum. Second, we conducted individual–subject analysis on the data, to which a spatial filter was not applied to preserve the resolution of the raw images, and obtained consistent results with Figure 7 (see the last paragraph in Results). Furthermore, we observed a marked temporal overlap of the transient and sustained activities for a single voxel of a single subject in the event-related analysis for several brain regions, including the cerebellum.

Our previous studies indicated that the internal models for the rotated mouse and the velocity mouse are acquired in the neighboring regions near the posterior superior fissure (Imamizu et al., 1998, 2000, 2003; Tamada et al., 1999). We have already confirmed that these sustained activations were not caused by behavioral factors such as tracking error, attention, effort, or eye/hand movements (Imamizu et al., 1998, 2000, 2003). The first regression analysis of the current study additionally suggests that the transient activity related to the mouse-type change was also free from the above confounding behavioral factors. This is because the mouse-type change activated only a few brain regions (Table 1), whereas the cursor reset that increased the above behavioral factors activated different various sensory–motor regions, as listed in Table 2.

In conclusion, the switch activity was segregated from the internal model activity in the prefrontal regions (area 46 and the insula), whereas it largely overlapped the internal model activity in the cerebellum and the parietal cortex. Therefore, switching mechanisms modeled by the gating module and the RE can be achieved in the prefrontal regions and that internal models of the MOSAIC can be achieved in the cerebellum and the parietal cortex. Our results suggest globally distributed switching mechanisms over the cerebro-cerebellar communication loop and distinct computational roles assigned to different parts of this loop in selecting appropriate internal models for given contexts.