Perceiving the unusual: Temporal properties of hierarchical motor representations for action perception

Abstract

Recent computational approaches to action imitation have advocated the use of hierarchical representations in the perception and imitation of demonstrated actions. Hierarchical representations present several advantages, with the main one being their ability to process information at multiple levels of detail. However, the nature of the hierarchies in these approaches has remained relatively unsophisticated, and their relation with biological evidence has not been investigated in detail, in particular with respect to the timing of movements. Following recent neuroscience work on the modulation of the premotor mirror neuron activity during the observation of unpredictable grasping movements, we present here an implementation of our HAMMER architecture using the minimum variance model for implementing reaching and grasping movements that have biologically plausible trajectories. Subsequently, we evaluate the performance of our model in matching the temporal dynamics of the modulation of cortical excitability during the passive observation of normal and unpredictable movements of human demonstrators.

Introduction

An increased interest in computational mechanisms that will allow robots to observe, imitate and learn from human actions has resulted in a number of computational architectures that allow the matching of demonstrated actions to the observer robot's equivalent motor representations (Alissandrakis et al., 2002, Billard, 2000, Demiris and Hayes, 2002, Schaal et al., 2003). These architectures, whilst sharing common computational components such as modules for processing and classifying visual information and retrieving motor representations, differ in the way that the perceptual information is coded and classified, the organisation of the motor system, and the stage at which the motor representations are used. The final aspect, at what stage the motor representations are used, differentiate architectures that follow the general ‘observe, classify, imitate’ decomposition (Kuniyoshi, Inaba, & Inoue, 1994), from those that advocate a stronger involvement of the motor systems in the perception process, through a ‘rehearse, predict, observe, reinforce’ decomposition (Demiris and Hayes, 2002, Demiris & Johnson, 2003, Schaal et al., 2003). In the latter, the observer robot invokes its motor systems to rehearse potential actions, predicting and confirming incoming observed states during the demonstration. This approach has gained biological credibility with the discovery of the mirror system in monkeys and humans (Grezes et al., 2003, Rizzolatti et al., 1996). Not all theoretical models advocate the actual rehearsal of candidate actions as our previous work has done (Demiris & Hayes, 2002), opting instead for a weaker version of this motor theory of perception, usually termed ‘motor resonance’, in which the motor representations are retrieved through a resonance mechanism rather than a generative mechanism.

For imitation approaches that advocate the use of motor systems during the perception stage it becomes crucial to have a clear and flexible motor system organisation. Hierarchical representations, involving primitive motor structures at the lowest level, while increasing their complexity in higher levels, have been proposed (Demiris & Johnson, 2003, Wolpert et al., 2003), and tested in robotic systems (Demiris & Johnson, 2003), which successfully learned and used sequences of actions by observation. However, little has been done with respect to the temporal dimension of these representations, including how they can be coordinated, as well as their relation to biological data.

In this paper, we will examine in detail the issue of hierarchical representations, and in particular examine how higher level models can be composed from (and coordinate) lower levels primitives. Our approach will use representations based on the biologically plausible minimum variance model of movement control (Harris and Wolpert, 1998, Simmons and Demiris, 2005), which leads to principled and biologically plausible coordination of the underlying components. We subsequently compare a particular instantiation of our hierarchical attentive multiple models for execution and recognition (HAMMER) architecture (Demiris & Khadhouri, in press) for reaching and grasping actions, with transcranial magnetic stimulation (TMS) data from humans during the passive observation of grasping movements by a demonstrator (Gangitano et al., 2004).