The sensitivity of primate STS neurons to walking sequences and to the degree of articulation in static images

https://doi.org/10.1016/S0079-6123(06)54007-5Get rights and content

Abstract

We readily use the form of human figures to determine if they are moving. Human figures that have arms and legs outstretched (articulated) appear to be moving more than figures where the arms and legs are near the body (standing). We tested whether neurons in the macaque monkey superior temporal sulcus (STS), a region known to be involved in processing social stimuli, were sensitive to the degree of articulation of a static human figure. Additionally, we tested sensitivity to the same stimuli within forward and backward walking sequences. We found that 57% of cells that responded to the static image of a human figure was also sensitive to the degree of articulation of the figure. Some cells displayed selective responses for articulated postures, while others (in equal numbers) displayed selective responses for standing postures. Cells selective for static images of articulated figures were more likely to respond to movies of walking forwards than walking backwards. Cells selective for static images of standing figures were more likely to respond to movies of walking backwards than forwards. An association between form sensitivity and walking sensitivity could be consistent with an interpretation that cell responses to articulated figures act as an implied motion signal.

Introduction

Artists use many tricks to convey information about movement. One method commonly used is to illustrate a person with legs and arms outstretched or articulated as if the artist had captured a snapshot of the person mid-stride during walking or running. When we see such static images we commonly interpret the human as moving, walking or running forwards through the scene. Although no real movement occurs, the articulated human figure ‘implies’ movement forward by its configuration or form. There is considerable evolutionary advantage in this ability to infer information about movement from the posture; we can interpret movement direction and speed from a momentary glimpse of a figure.

Traditionally, form and motion information have been thought to be processed along anatomically separate pathways; relatively little effort has been spent investigating how the pathways interact and how motion and form are integrated. Recently, however, three fMRI studies have shown that the brain structure that processes motion, hMT+/V5 (Zeki et al., 1991; Watson et al., 1993; Tootell et al., 1995), is more active to images implying motion when compared to similar images where motion in not implied (Kourtzi and Kanwisher, 2000; Senior et al., 2000; Krekelberg et al., 2005). In each study very different images were used to imply motion; Kourtzi and Kanwisher used images of athletes and animals in action, Senior et al. used images of moving objects and Krekelberg et al. used ‘glass patterns’, i.e., arrangements of dots suggesting a path of motion. These papers all argue that information regarding the form of static images is made available to hMT+/V5 for coding motion.

Neurons in the monkey homologue of human hMT+/V5, the medial temporal (MT) and medial superior temporal (MST) areas, also respond to glass patterns, where motion is implied (Krekelberg et al., 2003). Areas MT and MST contain neurons that respond to motion (Dubner and Zeki, 1971; Desimone and Ungerleider, 1986) and respond in correlation with the monkey's perception of motion (Newsome et al., 1986; Newsome and Pare, 1988). Neurons in MT/MST area respond maximally to movement in one direction; Krekelberg et al. (2003) showed that they respond preferentially to both real dot motion and implied motion in the preferred direction. Presentation of contradictory implied motion and real motion results in a compromised MT/MST neural response and compromises the monkey's perception of coherent movement.

The blood-oxygen level-dependent (BOLD) activity seen in human hMT+/V5 to complex images implying motion (Kourtzi and Kanwisher, 2000; Senior et al., 2000) could be explained by input from other regions of the cortex. Measurement of event-related potentials (ERP) responses from a dipole pair in the occipital lobe, consistent with localization to hMT+/V5, showed that the responses to the real motion of a random-dot field were 100 ms earlier than responses to static images containing human figures implying motion (Lorteije et al., 2006). The delay in the implied motion response indicates that this information arrives via a different and longer pathway. Kourtzi and Kanwisher (2000) concluded that since inferring information about still images depends upon categorization and knowledge, this must be analysed elsewhere. The activation of hMT+/V5 by implied motion of body images could be due to top-down influences. Senior et al. (2000) suggested that the activation they saw in hMT+/V5 is more likely due to processing of the form of the image in temporal cortex without the need for engagement of conceptual knowledge. At present, there is no evidence that cells in monkey MT are sensitive to articulated human figures implying motion despite active search (Jeanette Lorteije, personal communication).

Information about body posture and articulation in a human figure is likely to come from regions of the cortex that contain neurons sensitive to body form. The superior temporal sulcus (STS) in monkeys and the superior temporal gyrus (STG) and nearby cortex in humans is widely believed to be responsible for processing socially important information. Monkey STS contains neurons that respond to movement of human bodies (Bruce et al., 1981; Perrett et al., 1985), the form (view) of human bodies (Wachsmuth et al., 1994) and many appear to integrate motion and form to code walking direction (Oram and Perrett, 1996; Jellema et al., 2004). It is not known, however, if cells exist that are sensitive to the pattern of articulation that may differentiate postures associated with motion from those associated with standing still.

Giese and Poggio (2003) extended models of object recognition (Riesenhuber and Poggio, 1999, Riesenhuber and Poggio, 2002) to generate a plausible feed-forward model of biological motion recognition. A critical postulate of Giese and Poggio's model is the existence of ‘snapshot’ neurons, neurons tuned to differing degrees of articulation of bodies. Giese and Poggio suggest that these neurons should be found in inferotemporal (IT) or STS cortex, and would feed-forward to neurons coding specific motion patterns, e.g., walking (Oram and Perrett, 1996; Jellema et al., 2004).

In this study we set out to investigate if neurons in temporal cortex can code the degree of articulation of a human figure. Video taping a person walking or running produces a series of stills capturing discrete moments in time. Some of these stills show the person in an articulated pose, others in less-articulated poses akin to standing still. We made use of such video footage in order to compare the responses of STS neurons to a human figure articulated and standing. Neurons in STS sensitive to non-walking articulated postures are also sensitive to actions leading to such postures (Jellema and Perrett, 2003). It is possible, however, to arrive at a posture from two different directions, by walking forwards, or by walking backwards, both movement directions are consistent with the same static form. We therefore used the video footage played forwards and backwards to investigate how form sensitivity was related to walking.

Following Giese and Poggio (2003) we hypothesized that STS neurons would discriminate articulated postures from standing postures. We also hypothesized that the ability to differentiate posture in static images would relate to sensitivity to motion type for the same neurons. To this end we explore the cells’ sensitivity to images of static figures taken from video and movies containing the same images, played forward and in reverse. We also investigate the sensitivity to body view since cells sensitive to static and moving bodies exhibit viewpoint sensitivity (Perrett et al., 1991; Oram and Perrett, 1996).

Section snippets

Physiological subjects, recording and reconstruction techniques

One rhesus macaque, aged 9 years, was trained to sit in a primate chair with head restraint. Using standard techniques (Perrett et al., 1985), recording chambers were implanted over both hemispheres to enable electrode penetrations to reach the STS. Cells were recorded using tungsten microelectrodes inserted through the dura mater. The subject's eye position (±1°) was monitored (IView, SMI, Germany). A Pentium IV PC with a Cambridge electronics CED 1401 interface running Spike 2 recorded eye

Results

We tested 55 cells that responded significantly (see methods) to either static images of a human figure or movies of a human walking.

Discussion

The results of this study show two main findings: (1) Fifty-seven per cent of STS neurons that respond to static images of a human figure are sensitive to the degree of articulation of the figure itself. (2) There is an association between STS neuronal response sensitivity to the degree of articulation of a human figure and sensitivity to the compatibility between the direction of locomotion and view of the body of a human walking. For the cells that were sensitive to both the degree of

Acknowledgments

This work was funded by grants from the EU and the Wellcome Trust.

References (40)

  • C.J. Bruce et al.

    Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque

    J. Neurophysiol.

    (1981)
  • C.J. Bruce et al.

    Both striate cortex and superior colliculus contribute to visual properties of neurons in superior temporal polysensory area of macaque monkey

    J. Neurophysiol.

    (1986)
  • R. Desimone et al.

    Multiple visual areas in the caudal superior temporal sulcus of the macaque

    J. Comp. Neurol.

    (1986)
  • C. Distler et al.

    Cortical connections of inferior temporal area TEO in macaque monkeys

    J. Comp. Neurol.

    (1993)
  • R. Edwards et al.

    Color sensitivity of cells responsive to complex stimuli in the temporal cortex

    J. Neurophysiol.

    (2003)
  • P. Foldiak et al.

    Rapid serial visual presentation for the determination of neural selectivity in area STSa

    Prog. Brain Res.

    (2003)
  • M.A. Giese et al.

    Neural mechanisms for the recognition of biological movements

    Nat. Rev. Neurosci.

    (2003)
  • K. Hikosaka et al.

    Polysensory properties of neurons in the anterior bank of the caudal superior temporal sulcus of the macaque monkey

    J. Neurophysiol.

    (1988)
  • T. Jellema et al.

    Single cell integration of animate form, motion and location in the superior temporal cortex of the macaque monkey

    Cereb. Cortex

    (2004)
  • G. Johansson

    Visual perception of biological motion and a model for its analysis

    Percept. Psychophys.

    (1973)
  • Cited by (0)

    View full text