To explore the extent to which functional systems within the human posterior parietal cortex and the superior temporal sulcus are involved in the perception of action, we measured cerebral metabolic activity in human subjects by positron emission tomography during the perception of simulations of biological motion with point-light displays. The experimental design involved comparisons of activity during the perception of goal-directed hand action, whole body motion, object motion, and random motion. The results demonstrated that the perception of scripts of goal-directed hand action implicates the cortex in the intraparietal sulcus and the caudal part of the superior temporal sulcus, both in the left hemisphere. By contrast, the rostrocaudal part of the right superior temporal sulcus and adjacent temporal cortex, and limbic structures such as the amygdala, are involved in the perception of signs conveyed by expressive body movements.
- parietal cortex
- superior temporal sulcus
- biological motion
- motion perception
- positron emission tomography
Lesions of the posterior part of the human cerebral cortex, involving the posterior parietal and adjacent temporal and occipital cortex, give rise to severe impairments in the production of action, as well as in the perception of meaningful action (Kimura and Archibald, 1974; De Renzi and Lucchelli, 1988; Goodale et al., 1991). These impairments, which are most frequently associated with damage to the left hemisphere, have been traditionally described under the term “apraxia” (Liepmann, 1908; Geschwind, 1967). Single-cell recording studies in monkeys have demonstrated that certain areas in the parietal cortex are involved in the reaching toward and manipulation of objects in space (Mountcastle et al., 1975; Taira et al., 1990; Sakata et al., 1992, 1995), as well as in the coding of the position of objects in relation to the body (Mountcastle et al., 1975; for review, see Andersen, 1987). These parietal areas receive input from the cortex of the caudal superior temporal sulcus (Seltzer and Pandya, 1978, 1984, 1991; Maunsell and Van Essen, 1983; Andersen et al., 1990; Boussaoud et al., 1990), where several areas involved in the perception of motion have been identified (Zeki, 1974; Van Essen et al., 1981; Tanaka et al., 1986). There is evidence that the “motion” areas in the caudal superior temporal sulcus are interacting functionally with the parietal cortex in the analysis of complex motion patterns (Tanaka and Saito, 1989; Graziano et al., 1994;Sakata et al., 1994). The above findings raise the question whether, in the human brain, the perception of action implicates specific systems within the superior temporal sulcus and the posterior parietal cortex. In addition, given the neuropsychological evidence that left posterior lesions give rise to “apraxia,” it would be of interest to find out whether this activity would be more prominent in the left hemisphere.
The experimental paradigm used to explore the above questions involved simulations of biological motion in the three-dimensional space using point-light displays. This technique, pioneered by Johansson (1973), permits the depiction of human movement by means of a few isolated points of light attached to major joints of the body. Naive observers readily interpret the moving cluster of light points as representing a human figure, despite the complete absence of form cues. In the present positron emission tomography (PET) study, the use of moving point-light displays, instead of ordinary featural displays, permits appropriate experimental control because the visual stimuli in all conditions are the same and, therefore, any difference between these conditions is the result of the perceptual processing by the nervous system. We measured cerebral blood flow with PET under conditions in which human subjects perceive goal-directed hand action, whole body motion, and object and random motion. Comparisons of activity between these conditions allowed the identification of neural activity specifically related to the interpretation of body movement.
A preliminary report of some of these data has been presented (Bonda et al., 1995a).
MATERIALS AND METHODS
Eleven volunteer subjects (19–27 years old; mean age 22.2), 10 female and 1 male, participated in the experiment. All subjects had normal uncorrected vision and were right-handed. Informed consent was obtained from the subjects, and the study was approved by the Ethics Committee of the Montreal Neurological Hospital.
Apparatus and stimuli
The stimuli consisted of sequences of point-light displays that were recorded in three spatial dimensions using the Optotrak, an optoelectronic imaging system. Sequences of dance-like whole-body movements, sequences of hand and arm movements reaching to pick up a cup and bring it to the mouth, and movements of light points attached to a mounting board were recorded. In each case, 13 light points were used. In all conditions, the shifting pattern of points was sampled at 100 Hz for 10 sec to create the point-light animation sequences.
The animation sequences were played back to the subjects on a computer monitor, and all trial-related events (display randomization, timing, etc.) were controlled by the computer. Each sequence consisted of the movements of the 13 light points (the recorded positions) seen against a black background. The 13 light points were visible at all times.
Experimental procedure and design
In the PET session, the subjects were tested under seven 60 sec scanning conditions, five of which constitute the present experiment. The order of these five conditions was balanced across subjects. Four conditions (body movement, hand action, object motion, and random motion) involved sequences of light points moving on the screen, and one condition (random static) involved static displays of such points. The subjects were asked to look at the point-light sequences very carefully because at the end of the scanning period, they would be given a memory test, during which they would have to recognize the sequences they saw. In each condition, four different sequences of point-lights were presented during scanning in blocks of four. Within each block, the four sequences were randomly presented. Thus, the subjects had equal exposure to all four sequences during the scan. The subjects were told to pay attention to the overall pattern of movement rather than to individual light points (in the four motion conditions), and to pay attention to the overall static display in the static condition. At the end of scanning in each condition, a recognition memory test was administered. During this test, eight sequences were presented; four of these were the ones presented during the scan and the other four were novel. The subject had to identify verbally the four familiar and the four new sequences.
All subjects were naive with regard to the stimulus material, and no practice session was administered before scanning. Each point-light sequence lasted 5 sec. The stimuli subtended 16.5° of visual angle. All light points disappeared during the inter-sequence interval, which was 0.5 sec for all conditions. The presentation of the sequences started ∼10 sec before the onset of PET data collection.
Body movement condition. In this condition, the point-light sequences depicted a frontal plane view of a human figure that moved backward and forward and from left to right, yielding three-dimensional movements of the whole body to the viewer. These moving sequences left a powerful and immediate impression on the viewer of a person dancing. To produce the body motion sequences, infrared light-emitting diode markers were placed bilaterally on the shoulders, elbows, hands, hips, knees, and ankles. The hip and shoulder markers defined the trunk. An additional marker was used to record the position of the head. The constellation of 13 points remained visible throughout the duration of each sequence. During scanning, four sequences of different body motion patterns were presented in four blocks. In the post-scanning memory test, in which the subjects had to discriminate these four body motion sequences from four new ones, the mean accuracy of performance was 81.8 (range, 62.5–100%). All subjects reported a compelling impression, obtained from the entire display, of a moving human body.
Hand action condition. In each sequence used in this condition, the 13 light points were placed on different parts of the hand: shoulder, elbow, wrist, and the two articulations defining the proximal phalanx of each finger. These sequences depicted a human left hand that reached out in space and simulated grasping a glass and bringing it to the mouth. All subjects reported a strong impression of perceiving this action. Four different sequences of the same hand action were presented in four blocks. In the post-scanning memory test, in which discrimination of these four hand action sequences from four new ones was required, the mean accuracy of performance was 81.8 (range, 50–100%).
Object motion condition. The point-light sequences used here depicted an abstract geometrical stimulus configuration. These sequences were constructed by placing the 13 light points on a square board that was moved in all three spatial dimensions by the experimenter during the recording session. This resulted in a display in which the motions of the 13 light points were coherent (as opposed to random), and of comparable spectral distribution to the body movement displays. Each sequence was constructed by recording motion involving translations of the object in the lateral, vertical, and forward directions and rotations of the object around these spatial axes to produce movements of the object involving pitch, roll, and yaw. Four different object motion sequences were presented in four blocks during scanning. The mean accuracy of performance in the discrimination of these four object motion sequences from four new ones was 79.5% (range, 75–100%).
Random motion condition. In this condition, the sequences presented were random motion patterns. These sequences had been produced by systematically altering the three-dimensional positions of the 13 light points used in the body movement sequences. Each of the three spatial coordinates of each light point was shifted by different constant amounts (up to 1.2 m in body space). The shifted points moved in different directions and with different speeds. The resulting displays contained the same spectral information as the body movement condition but were “meaningless” to the subject. Four different random motion sequences were presented during scanning. Mean accuracy in the post-scanning memory test, in which the subjects had to discriminate these four random motion sequences from four new random motion sequences, was 71.6% (range, 50–100%).
Random static condition. The aim of this condition was to provide a control for the random motion condition so that comparison of activation between them would identify, within the context of our study, the human homolog of motion area MT/V5 described previously (Watson et al., 1993; Tootel et al., 1995). To generate the stimuli used in this condition, static frames were taken from the random motion condition sequences. The subjects were told that they would see a pattern of stationary light points and that they would have to try to attend to the overall pattern of points. The four stationary patterns of lights used in this condition were presented for the same duration as the sequences in the other conditions. Performance in discriminating between the four familiar and the four new static displays in the post-scanning test was 53.4% (range, 12.5–62.5%).
PET and MRI data acquisition and analysis
PET scans were obtained using the Scanditronix PC-2048B tomograph, which produces 15 image slices with a transverse image resolution of 4.6–6.4 mm and an axial resolution of 5.4–7.1 mm. The regional distribution of cerebral blood flow (CBF) in experimental and control scanning conditions was measured by means of the water bolus H2 15O methodology (Raichle et al., 1983) without arterial blood sampling. The PET scans were of 60 sec duration. Each subject also underwent a high-resolution magnetic resonance imaging (MRI) scan (64 slices, 1 mm thick) obtained with a Philips ACS system (1.5 T). The MRI scans were resliced to be in register with the PET data (Evans et al., 1991). An orthogonal coordinate frame was then established on the basis of the anterior–posterior commissure line as identified in the MRI volume (Evans et al., 1992). These anatomical frame coordinates were used to apply a linear resampling of each matched pair of MRI and PET data sets into a standardized stereotaxic coordinate system (Talairach and Tournoux, 1988). The PET images were reconstructed with a 20 mm Hanning filter to overcome residual anatomical variability persisting after the stereotaxic standardization. These PET images were then normalized for global CBF value, and the mean intersubject state-dependent change in CBF was derived by subtracting the values in one state from those in another. The mean state-dependent change in CBF was converted to at statistic volume by dividing each voxel by the mean SD in normalized CBF for all intracerebral voxels (Worsley et al., 1992). Individual MRI images were subjected to the same averaging procedure such that composite stereotaxic image volumes were obtained for botht statistic and MRI volumes. The transformed volumes were 128 × 128 × 80 voxels in extent and sampled at 1.34 × 1.72 × 1.50 mm in the x, y, and z dimensions, respectively. Anatomical and functional images were merged into a single data set (Evans et al., 1992), a procedure that allows the following: (1) direct observation of t statistic peaks (identified by an automatic peak-detection algorithm) on the MRI images; and (2) the anatomical correlation of extended zones of CBF change patterns that cannot be expressed in terms of isolated peaks.
The statistical significance of focal changes in CBF was determined by a method based on three-dimensional Gaussian random field theory (Worsley et al., 1992). For all peaks within the gray matter volume of 600 cm3 or 200 resels (each of which has dimensions of 20 × 20 × 7.6 mm), the threshold for reporting a peak as significant was set at t = 3.5, corresponding to an uncorrected probability of p < 0.0004 (two-tailed). At value of 3.5 yields a false positive rate of 0.58 in the 200 resels explored. For predicted blood flow changes in particular brain areas, the threshold for significance was set at t = 3.00, corresponding to an uncorrected probability of p < 0.0013.
Subtraction of CBF between conditions was used to reveal activity changes in one condition with reference to the other. In comparison with the random static condition, the random motion condition revealed bilateral CBF changes within the occipitotemporal junction region, at the confluence of the superior temporal sulcus, the anterior occipital sulcus, and the middle occipital sulcus (Table 1). This region is coextensive with the presumed MT/V5 area, which was activated in previous PET (Watson et al., 1993) and functional MRI (Tootell et al., 1995) studies when activity during the perception of stationary stimuli was compared with activity during the perception of moving stimuli. In the present study, we use this focus as a basis of comparison to evaluate differences in the localization of significant CBF changes, within the superior temporal sulcal cortex, in other conditions.
CBF in the body movement and hand action conditions was compared with CBF in the random motion and object motion conditions. The significant changes in CBF in the hand action and body movement conditions in comparison with the random motion control condition are presented in Tables 2 and 3, respectively. The main pattern of results when the object motion condition was subtracted from the hand action and body movement conditions was essentially the same, and therefore only the differences are presented below.
Table 2 presents the results of the hand action condition compared with the random motion condition. Significant activity was observed within the superior temporal sulcal cortex, in its caudal extent. This activity was confined within the upper bank of the caudal section of the superior temporal sulcus (see Fig. 1) in the left hemisphere. This region may be involved in later stages in the processing of motion-related information within the human dorsal occipital pathway.
Table 3 presents the significant CBF changes in the body movement condition compared with the random motion condition. In contrast to the hand action condition, in the body movement minus random motion subtraction, the significant changes were confined to the rostrocaudal section of the superior temporal sulcus in the right hemisphere (Table 3, Fig. 2). This focus was within the ventral bank of the superior temporal sulcus and the adjacent lateral convexity of the middle temporal gyrus.
An extremely interesting result is the significant activity observed in the amygdala, bilaterally, in the body movement minus random motion subtraction (Table 3, Fig. 3). This activity extended into the adjacent entorhinal and perirhinal cortex. Significant changes were also observed in other structures that are architectonically related to the limbic system: the caudal orbitofrontal cortex, the fusiform gyrus, the subcallosal gyrus, and the septum. In addition, activity was observed in striatal areas, such as the nucleus accumbens and the body of the caudate nucleus.
Compared with the object motion condition, the hand action condition showed significant activity in the left caudal-most part of the superior temporal sulcus (−49, −68, 15; t = 3.94), whereas the body movement condition showed significant activity in the amygdala bilaterally (Right: 32, −1, −35; t = 3.26; Left: −25, −6, −21; t = 4.64), confirming the results of the previous subtractions. In addition, there was bilateral activity in the cortex of the superior parietal lobule (Right: 11, −63, 65; t = 4. 17; Left: −23, −64, 66; t = 3.00) in the hand action condition.
Table 4 presents the results of the subtraction of activity between the hand action and body movement conditions. This is a stringent test aimed at revealing CBF changes that are specific to one experimental task compared with the other. Note that the activity in the foci of the left and right superior temporal sulcus was now subtracted out, demonstrating that these regions were involved in both conditions. Therefore, the differential involvement of the superior temporal sulcus in the hand action and body movement conditions, revealed in the comparison of each one with the random motion condition, is relative. The results of the hand action and body movement comparison confirmed the involvement of the caudal superior parietal cortex (Fig. 4) and the adjacent medial part of the intraparietal sulcus in the hand action condition. Significant bilateral activity was also observed in the inferior parietal lobule, extending into the middle and anterior parts of the intraparietal sulcus in the left hemisphere (Fig.5).
For the body movement task in relation to the hand action task (see Table 4), significant changes were observed within the amygdala bilaterally, confirming once again the preeminent involvement of this structure in the body movement condition.
The present experiment explored the neural systems involved in the perception of meaningful body movement. Two types of motion were investigated: one involved goal-directed action of the hand (i.e., imitating the act of reaching toward a glass, picking it up, and bringing it to the mouth), and the other involved an expressive dancing-like movement of the whole body. Perception of a script of goal-directed action of the hand resulted in significant activity within the intraparietal sulcus, as well as in the caudal superior temporal sulcus, both in the left hemisphere (Tables 2, 4, Figs. 1, 5). Furthermore, a direct comparison between these two types of biological movements (see Table 4) showed that the perception of the action of the hand yielded the strongest activity within the parietal cortex (Table4, Fig. 5). This finding is of considerable interest in the context of an extensive literature going back to the pioneering work of Liepmann (1908), which has consistently shown that left posterior cortical lesions give rise to severe disturbances in the production and comprehension of goal-directed reaching (Fisk and Goodale, 1988;Winstein and Pohl, 1995). It has been suggested that the involvement of the left hemisphere in the control of action may be related to the temporal segmentation of the discrete components of the movement (Kimura, 1982).
Single-cell recording studies in behaving monkeys have shown that activity within the intraparietal sulcus and adjacent inferior parietal cortex is critical for the conversion of the retinal coordinates of visual stimuli into a body-centered frame of reference and the integration of visual with somatosensory and vestibular signals necessary for motor planning (Mountcastle et al., 1975; for review, seeAndersen, 1987). Furthermore, it has been shown that there is a class of neurons in the intraparietal sulcus and adjacent inferior parietal cortex that have response properties relating the pattern of hand movement to the visuospatial characteristics of the object to be manipulated (Taira et al., 1990; Sakata et al., 1992). More recently, Sakata and colleagues (1995) demonstrated that the majority of these highly selective “hand manipulation” neurons were located in the rostral part of the ventral bank of the intraparietal sulcus. Injection of muscimol into this region resulted in marked disorders in the preshaping of the hand during the manipulation of an object (Gallese et al., 1994). It is of considerable interest that our activation foci within the intraparietal sulcus in the hand action condition (Table 4, Fig. 5) extended from its midsection all the way to the anterior tip. There was no significant activity in the caudal part of the intraparietal sulcus.
The cortex of the intraparietal sulcus and adjacent inferior parietal lobule receives input from several areas within the caudal superior temporal sulcus that is involved in visual motion perception (Maunsell and Van Essen, 1983; Andersen et al., 1990; Boussaoud et al., 1990). Recent functional neuroimaging studies have shown that the presumed homolog of the motion area MT/V5 in the human brain lies at the tip of the ascending limb of the inferior temporal sulcus, namely at the dorsal border between the temporal and occipital lobes (Watson et al., 1993; Tootel et al., 1995). This area was activated when random motion was compared with static displays. In the present study, activity was also observed in this region in the random motion minus random static condition (Table 1). Thus, the hand action minus random motion condition reveals activation in which the effect of baseline random motion has been removed. This subtraction demonstrated a significant activity focus within the left caudal superior temporal sulcus (Table2, Fig. 1) that was located dorsal to the presumed MT/V5 homolog and probably represents activity in higher-order areas in the dorsally directed “motion” system leading to the parietal cortex. In this context, it is significant that posterior lesions of the superior temporal sulcus, as with those of the inferior parietal lobule, impair the monkey’s ability to move an object with its hand within a particular spatial trajectory (Petrides and Iversen, 1979).
The present study has revealed the important role of temporoparietal interactions involving the caudal superior temporal sulcus and the intraparietal sulcus, both in the left hemisphere, and the superior parietal cortex for the interpretation of scripts of goal-directed hand action. In an earlier study, we had shown that the left intraparietal sulcus and the superior parietal cortex were engaged when the subject was mentally transforming the position of his own hand (Bonda et al., 1995b). Thus, the perception of another person’s action involves parietal systems underlying mental transformations of the body.
A striking contrast between the activation patterns observed in relation to the two instances of biological motion investigated here lies in the involvement of the limbic structures. The body movement condition, but not the hand action condition, resulted in an increase in CBF within the amygdaloid region (Table 3, Fig. 3) and other limbic structures that are closely interconnected with the amygdala, such as the subcallosal gyrus, the septal area, and the caudal orbitofrontal cortex (Table 3). The increased activity in the amygdala in the body movement condition was a consistent finding and was confirmed in three separate subtractions: body movement minus random motion, body movement minus hand action, and body movement minus object motion.
The amygdaloid system has been traditionally discussed in terms of its involvement in stimulus-reward associations (Jones and Mishkin, 1972) and in the experience of various emotional states, such as fear (Gloor et al., 1982). Electrical stimulation of the amygdala in experimental animals (Ursin and Kaada, 1960; Kaada, 1972) and humans (Mark et al., 1972) evokes stereotyped motor patterns of defense, flight, orienting movements, etc. Lesions of the amygdala in the monkey severely impair behaviors necessary for the establishment and maintenance of social bonds (Rosvold et al., 1954; Steklis and Kling, 1985). For instance, such monkeys exhibit virtual absence of conspecific communication gestures (Dicks et al., 1969) and respond inappropriately to solicitation signals by normal monkeys (Kling and Cornell, 1971). Patients with amygdala lesions perform inadequately in perceptual tasks requiring recognition of facial expressions (Jacobson, 1986; Adolphs et al., 1994; Young et al., 1995). In the context of these data, we propose that the activity in the amygdala may reflect the emotional dis-position experienced by the perceiver in response to the behavior perceived. The present PET findings provide a demonstration of functional interactions between the temporal neocortex, endpoint of the visual perceptual system (Mishkin, 1982), and the amygdala. Note that the activity in the amygdala was restricted predominantly to the right hemisphere, the function of which has been related to emotional processing (for review, see Heilman et al., 1993).
The body movement condition also yielded significant activity in the lower bank of the rostrocaudal portion of the superior temporal sulcus, as well as in the adjacent middle temporal cortex (Table 3, Fig. 2). In the monkey, this part of the superior temporal sulcus and adjacent temporal cortex, which comprises primarily unimodal visual cortex (Seltzer and Pandya, 1978), is extensively connected with the amygdala (Aggleton, 1980; Amaral and Price, 1984; Yukie et al., 1990). Consistent with the PET findings, a patient with lesion involving the dorsal occipitoparietal cortex, but sparing the temporal lobe, showed specific deficits in many early aspects of motion analysis but normal performance in the recognition of patterns of light dots representing whole body movements (Vaina et al., 1990). Interestingly, some neurons in the superior temporal sulcus of the monkey, recorded in its anterior upper bank, were first reported by Bruce et al. (1981) to respond selectively to body movements, an observation subsequently confirmed by others (Perrett et al., 1985; Oram and Perrett, 1994).
The present study has demonstrated that the interpretation of different types of biological motion engages brain systems in a differential manner. Consistent with this is psychophysical evidence that different mechanisms operate for the perceptual analysis of various categories of biological motion (Dittrich, 1993). The perception of scripts of goal-directed hand action selectively activated areas in the posterior part of the left hemisphere within the intraparietal sulcus and the caudal superior temporal sulcus. By contrast, the perception of signs embedded in expressive body movements involves interaction between temporal neocortex and limbic areas critical for emotional effector patterns of behavior.
This work was supported by the McDonnell-Pew Program in Cognitive Neuroscience; the Medical Research Council and Natural Sciences and Engineering Research Council (Canada); and a predoctoral fellowship to E.B. from the Ministère de la Recherche et de la Technologie (France). We thank Jorge Moreno, Stephen Frey, and Sylvain Milot for technical assistance.
Correspondence should be addressed to Eva Bonda, Cognitive Neuroscience Unit, Montréal Neurological Institute, McGill University, 3801 University Street, Montréal, Québec, Canada H3A 2B4.