Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Meaningful interactions can enhance visual discrimination of human agents

Nature Neuroscience volume 9, pages 1186–1192 (2006)Cite this article

Abstract

The ability to interpret and predict other people's actions is highly evolved in humans and is believed to play a central role in their cognitive behavior. However, there is no direct evidence that this ability confers a tangible benefit to sensory processing. Our quantitative behavioral experiments show that visual discrimination of a human agent is influenced by the presence of a second agent. This effect depended on whether the two agents interacted (by fighting or dancing) in a meaningful synchronized fashion that allowed the actions of one agent to serve as predictors for the expected actions of the other agent, even though synchronization was irrelevant to the visual discrimination task. Our results demonstrate that action understanding has a pervasive impact on the human ability to extract visual information from the actions of other humans, providing quantitative evidence of its significance for sensory performance.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Motion capture and stimulus design.
Figure 2: Sensitivity is greater for Sync as opposed to deSync stimuli.
Figure 3: The Sync versus deSync effect correlates with the detectability of synchronization.

Similar content being viewed by others

References

  1. Puce, A. & Perrett, D. Electrophysiology and brain imaging of biological motion. Phil. Trans. R. Soc. Lond. B 358, 435–445 (2003).

    Article  Google Scholar 

  2. Nelissen, K., Vanduffel, W. & Orban, G.A. Charting the lower superior temporal region, a new motion-sensitive region in monkey superior temporal sulcus. J. Neurosci. 26, 5929–5947 (2006).

    Article  CAS  Google Scholar 

  3. Rizzolatti, G. & Craighero, L. The mirror-neuron system. Annu. Rev. Neurosci. 27, 169–192 (2004).

    CAS  PubMed  Google Scholar 

  4. Gallese, V., Fadiga, L., Fogassi, L. & Rizzolatti, G. Action representation and the inferior parietal lobule. in Common Mechanisms in Perception and Action: Attention and Performance XIX (eds. Prinz, W. & Hommel, B.) 247–266 (Oxford Univ. Press, Oxford, 2002).

    Google Scholar 

  5. Giese, M.A. & Poggio, T. Neural mechanisms for the recognition of biological motion. Nat. Rev. Neurosci. 4, 179–192 (2003).

    Article  CAS  Google Scholar 

  6. Bülthoff, I., Bülthoff, H. & Sinha, P. Top-down influences on stereoscopic depth-perception. Nat. Neurosci. 1, 254–257 (1998).

    Article  Google Scholar 

  7. Neri, P., Morrone, M.C. & Burr, D.C. Seeing biological motion. Nature 395, 894–896 (1998).

    Article  CAS  Google Scholar 

  8. Blake, R., Turner, L.M., Smoski, M.J., Pozdol, S.L. & Stone, W.L. Visual recognition of biological motion is impaired in children with autism. Psychol. Sci. 14, 151–157 (2003).

    Article  Google Scholar 

  9. Barlow, H.B. Cerebral cortex as model builder. in Models of the Visual Cortex (eds. Rose, D. & Dobson, V.G.) 37–46 (John Wiley & Sons, New York, 1985).

    Google Scholar 

  10. Lange, J. & Lappe, M. A model of biological motion perception from configural form cues. J. Neurosci. 26, 2894–2906 (2006).

    Article  CAS  Google Scholar 

  11. Morrone, M.C., Burr, D.C. & Vaina, L.M. Two stages of visual processing for radial and circular motion. Nature 376, 507–509 (1995).

    Article  CAS  Google Scholar 

  12. Sumi, S. Upside down presentation of the Johannson moving light spot pattern. Perception 13, 283–286 (1984).

    Article  CAS  Google Scholar 

  13. Pavlova, M. & Sokolov, A. Orientation specificity in biological motion perception. Percept. Psychophys. 62, 889–899 (2000).

    Article  CAS  Google Scholar 

  14. Brown, W.M. et al. Dance reveals symmetry especially in young men. Nature 438, 1148–1150 (2005).

    Article  CAS  Google Scholar 

  15. Oram, M. & Perrett, D.I. Responses of anterior superior temporal polysensory (STPa) neurons to 'biological motion' stimuli. J. Cogn. Neurosci. 6, 99–116 (1994).

    Article  CAS  Google Scholar 

  16. Bonda, E., Petrides, M., Ostry, D. & Evans, A. Specific involvement of human parietal systems and the amygdala in the perception of biological motion. J. Neurosci. 16, 3737–3744 (1996).

    Article  CAS  Google Scholar 

  17. Grossman, E.D. & Blake, R. Brain areas active during visual perception of biological motion. Neuron 35, 1167–1175 (2002).

    Article  CAS  Google Scholar 

  18. Castelli, F., Happé, F., Frith, U. & Frith, C. Movement and mind: a functional study of perception and interpretation of complex intentional movement patterns. Neuroimage 12, 314–325 (2000).

    Article  CAS  Google Scholar 

  19. Martin, A. & Weisberg, J. Neural foundations for understanding social and mechanical concepts. Cogn. Neuropsychol. 20, 575–587 (2003).

    Article  Google Scholar 

  20. Schultz, J., Friston, K.J., O'Doherty, J., Wolpert, D.M. & Frith, C.D. Activation in posterior superior temporal sulcus parallels parameter inducing the percept of animacy. Neuron 45, 625–635 (2005).

    Article  CAS  Google Scholar 

  21. Gallese, V. & Goldman, A. Mirror neurons and the simulation theory of mind-reading. Trends Cogn. Sci. 2, 493–501 (1998).

    Article  CAS  Google Scholar 

  22. Kohler, E. et al. Hearing sounds, understanding actions: action representation in mirror neurons. Science 297, 846–848 (2002).

    Article  CAS  Google Scholar 

  23. Umiltà, M.A. et al. I know what you are doing: a neurophysiological study. Neuron 31, 155–165 (2001).

    Google Scholar 

  24. Nelissen, K., Luppino, G., Vanduffel, W., Rizzolatti, G. & Orban, G.A. Observing others: multiple action representation in the frontal lobe. Science 310, 332–336 (2005).

    Article  CAS  Google Scholar 

  25. Rizzolatti, G., Fogassi, L. & Gallese, V. Neurophysiological mechanisms underlying the understanding and imitation of action. Nat. Rev. Neurosci. 2, 661–670 (2001).

    Article  CAS  Google Scholar 

  26. Fadiga, L., Fogassi, L., Pavesi, G. & Rizzolatti, G. Motor facilitation during action observation: a magnetic stimulation study. J. Neurophysiol. 73, 2608–2611 (1995).

    Article  CAS  Google Scholar 

  27. Iacoboni, M. et al. Cortical mechanisms of human imitation. Science 286, 2526–2528 (1999).

    Article  CAS  Google Scholar 

  28. Peelen, M.V., Wiggett, A.J. & Downing, P.E. Patterns of fMRI activity dissociate overlapping functional brain areas that respond to biological motion. Neuron 49, 815–822 (2006).

    Article  CAS  Google Scholar 

  29. Nishitani, N. & Hari, R. Temporal dynamics of cortical representation for action. Proc. Natl. Acad. Sci. USA 97, 913–918 (2000).

    Article  CAS  Google Scholar 

  30. Iacoboni, M. et al. Grasping the intentions of others with one's own mirror neuron system. PLoS Biol. 3, E79 (2005).

    Article  Google Scholar 

  31. Calvo-Merino, B., Glaser, D.E., Grzes, J., Passingham, R.E. & Haggard, P. Action observation and acquired motor skills: an fMRI study with expert dancers. Cereb. Cortex 15, 1243–1249 (2005).

    Article  CAS  Google Scholar 

  32. Frith, C.D. & Frith, U. The neural basis of mentalizing. Neuron 50, 531–534 (2006).

    Article  CAS  Google Scholar 

  33. Adolphs, R. Cognitive neuroscience of human social behaviour. Nat. Rev. Neurosci. 4, 165–178 (2003).

    Article  CAS  Google Scholar 

  34. Gallese, V., Keysers, C. & Rizzolatti, G. A unifying view of the basis of social cognition. Trends Cogn. Sci. 8, 396–403 (2004).

    Article  Google Scholar 

  35. Williams, J.H., Whiten, A., Suddendorf, T. & Perrett, D.I. Imitation, mirror neurons and autism. Neurosci. Biobehav. Rev. 25, 287–295 (2001).

    Article  CAS  Google Scholar 

  36. Oberman, L.M. et al. EEG evidence for mirror neuron dysfunction in autism spectrum disorders. Brain Res. Cogn. Brain. Res. 24, 190–198 (2005).

    Article  Google Scholar 

  37. Dapretto, M. et al. Understanding emotions in others: mirror neuron dysfunction in children with autism spectrum disorders. Nat. Neurosci. 9, 28–30 (2006).

    Article  CAS  Google Scholar 

  38. Dittrich, W.H., Troscianko, T., Lea, S. & Morgan, D. Perception of emotion from dynamic point-light displays represented in dance. Perception 25, 727–738 (1996).

    Article  CAS  Google Scholar 

  39. Bassili, J.N. Facial motion in the perception of faces and of emotional expression. J. Exp. Psychol. Hum. Percept. Perform. 4, 373–379 (1978).

    Article  CAS  Google Scholar 

  40. Barraclough, N.E., Xiao, D., Baker, C.I., Oram, M.W. & Perrett, D.I. Integration of visual and auditory information by superior temporal sulcus neurons responsive to the sight of actions. J. Cogn. Neurosci. 17, 377–391 (2005).

    Article  Google Scholar 

  41. Keysers, C. & Perrett, D.I. Demystifying social cognition: a Hebbian perspective. Trends Cogn. Sci. 8, 501–507 (2004).

    Article  Google Scholar 

  42. Iacoboni, M. et al. Reafferent copies of imitated actions in the right superior temporal cortex. Proc. Natl. Acad. Sci. USA 98, 13995–13999 (2001).

    Article  CAS  Google Scholar 

  43. Kersten, D., Mamassian, P. & Yuille, A. Object perception as Bayesian inference. Annu. Rev. Psychol. 55, 271–304 (2004).

    Article  Google Scholar 

  44. Burr, D. Motion smear. Nature 284, 164–165 (1980).

    Article  CAS  Google Scholar 

Download references

Acknowledgements

Supported by US National Institutes of Health grant RO1EY01728.

Author information

Authors and Affiliations

  1. School of Optometry, University of California at Berkeley, Berkeley, 94720-2020, California, USA

    Peter Neri, Jennifer Y Luu & Dennis M Levi

Authors
  1. Peter Neri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jennifer Y Luu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dennis M Levi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peter Neri.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary Video 1

Motion-captured fighting and dancing sequences. Top panels show original movies of actors wearing light markers while performing a fighting (left) and a dancing (right) sequence. Bottom panels show corresponding motion-captured sequences after tracking of the joints. (MPG 2976 kb)

Supplementary Video 2

Comparison between Sync and de-Sync stimuli. The four panels containing point-light agents show an animated version of Fig. 2d–g. Black arrows schematically depict the operation of computing inter-dot distance for all possible dot pairs on each frame. Black traces in the upper-right graph plot the average magnitude of this distance for the Sync sequences, grey for the de-Sync sequences. Yellow arrows show the average motion vector for all dots (computed between position at frame t and frame t+1). Dark yellow traces in the upper-right graph plot the magnitude of this vector for Sync sequences, lighter yellow for de-Sync. Green double-arrows point from the average dot position of one agent to the average dot position for the other agent. Dark green traces in upper-right graph plot the magnitude of this inter-agent distance for de-Sync sequences, lighter green for Sync. The graph at the bottom-right plots distributions for the quantities shown in the upper graph, updated as the sequences progress over time. These distributions are indistinguishable at P > 0.05 for first and second-order statistics. We also ran programs that attempted to distinguish between Sync and de-Sync using extracts from these sequences that were equal in length to the ones we used in our experiments, but failed to differentiate between the two. We also measured these statistics for the inverted control sequences and found them to be virtually identical to those in the original non-inverted sequence. All sequences shown here are for fighters, but we obtained similar results with dancers. (MPG 2498 kb)

Supplementary Video 3

Examples of limited-lifetime sampling, noise and agent inversion. This demo consists of 6 iterations, 3 for the sequences used in the main experiment and 3 for the sequences containing one inverted agent that were used in the control experiment. All sequences shown here are for fighters. Left panels show two examples of 'target' and 'non-target' intervals for Sync trials, right panels for de-Sync. Three modes of presentation are demonstrated (labelled in red at bottom): unlimited dot lifetime, limited dot lifetime, and limited dot lifetime with added noise dots. Each iteration lasts more than 3 seconds for ease of inspection, but we used a presentation time of 1.5 seconds for fighters in the actual experiments. To ease comparison across the different panels, similar extracts have been used in all of them. However, the trajectories extracted from the original movie for display in the psychophysical experiments were randomly selected independently for every interval on every trial. Based on the characteristics of the sequences we used and on the presentation modes that are demonstrated in this movie, we can list at least five reasons why we think inter-agent occlusion could not be the source of the differential Sync/de-Sync effect we report in Fig. 2. First, because dots have a limited lifetime in our stimuli (see also Fig. 1i–j) they effectively disappear and reappear all the time, thus disappearance by occlusion is not special (see second iteration in this movie). Second, in the presence of noise dots, occlusion rules are constantly violated by the noise dots themselves, which can appear anywhere within and outside the agents (see third iteration in this movie). Third, the inverted control experiment preserves occlusion almost entirely, because both agents are still present and occlude each other in almost identical ways, despite the fact that one of them is upside-down (see second set of iterations in this movie). Fourth, the percentage of occlusion induced by one agent on the other agent (inter-agent occlusion) is only a fraction of the occlusion induced by one agent on himself/herself (self-occlusion): 46% for the fighting sample, 23% for the dancing sample. Self-occlusion is unrelated to synchronization between agents. If occlusion plays any role, it would be dominated by self-occlusion, leaving little scope for inter-agent occlusion to generate the differential effect reported in Fig. 2. Finally, the psychophysical effect was stronger for dancing than for fighting (compare blue and red symbols in Fig. 2a). If it were due to inter-agent occlusion, we would expect that such occlusion would be more pronounced in the dancing sequence than in the fighting sequence. To the contrary, inter-agent occlusion is more pronounced in the fighting sequence (8% of total sequence as opposed to 6% for dancers). The unlimited dot lifetime experiment (open black symbols in Fig. 2b and first iteration in this Video) serves as a further control for a potential role of inter-agent occlusion, because such occlusion is enhanced in fully sampled sequences. If inter-agent occlusion were the source of the effect, we would expect it to be even stronger in the fully sampled condition (the opposite of what we observe). Finally, this experiment adds evidence to the notion that the reduced performance on de-Sync trials cannot be attributed to observers being 'put off' by desynchronization and refusing to perform the task altogether (criterion change). Not only were they not aware of desynchronization during testing, but they could perform extremely well on de-Sync trials when the joint trajectories are fully sampled, a manipulation that enhances the perceptual impact of synchronization (mean percent correct for detecting synchronization was 90% in the fully sampled condition, as opposed to 77% in the undersampled version; paired t-test across observers for former greater than latter returns P < 0.05). (MPG 1964 kb)

Supplementary Video 4

Procedure used to measure noise and scrambling thresholds. For noise thresholds (top), as the number of noise dots progressively increases, performance drops from 100% correct (1) to chance (0.5) in the graph at top right. Yellow dots show real data for one subject (from the inset in Fig. 2a), dot size scales with number of trials. Blue line shows weighted (by square root of number of trials) Weibull fit. The number of noise dots is gradually increased in this demonstration, but in the actual experiments it was only varied from one trial to the next and never within a presentation. For scrambling thresholds (bottom), this demonstration shows three iterations of a motion sequence where one of the two agents in the non-target stimulus is progressively less scrambled. As scrambling decreases, the ability to distinguish target from non-target interval drops from 100% correct to chance. Yellow dots show real data from one of the insets in Fig. 2b. (MPG 2009 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Neri, P., Luu, J. & Levi, D. Meaningful interactions can enhance visual discrimination of human agents. Nat Neurosci 9, 1186–1192 (2006). https://doi.org/10.1038/nn1759

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn1759

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing