In Sync: Metaphor, Mechanism or Marker of Mutual Understanding?

The capacity for mutual understanding, often metaphorically expressed as “being in sync,” is one of the great scientific enigmas (Levinson, 2006). How can we understand what another is thinking or feeling just by observing their actions? For example, how does my friend know I am suggesting we enter a pub when I point toward a nearby bicycle that both of us know belongs to his girlfriend?

In a recent study in The Journal of Neuroscience, Nummenmaa and colleagues (2014) studied the brain mechanism underlying understanding another's actions and feelings using functional magnetic resonance imaging (fMRI). They were particularly interested in the role of the “action-observation network” in the process of action understanding. This parietofrontal circuit exhibits spatially similar activation patterns during perception and execution of motor actions. Therefore, this network has been suggested to support action understanding through an automatic sensorimotor mechanism (“mirroring”) that links the mental representation of a perceived action to the representation of an executed action, and the latter to its outcome (Rizzolatti and Craighero, 2004). Building on this notion, Nummenmaa and colleagues (2014) reasoned that similar states of the action-observation network might reflect similar mind states, possibly providing a platform for individuals to view and understand the external world in a similar fashion. Accordingly, the authors hypothesized that when two or more people share a third person's feelings, this should evoke neural synchrony across the action-observation networks of those individuals, possibly revealing the basic neural mechanism that promotes mutual understanding.

Nummenmaa and colleagues (2014) had healthy participants watch ∼10 s video clips of professional boxing matches, but with different levels of engagement. In the WATCH condition, indicated by tags placed over both boxers (Nummenmaa et al., 2014, their Fig. 1), participants were instructed to watch the video cuts as if they were watching TV. In the WIN and LOSE conditions, indicated by a single tag placed over one boxer only, the participants' task was to mentally simulate, as accurately as possible, the actions, thoughts, and feelings of the tagged individual, without actually performing those actions. The WIN and LOSE labels refer to whether the tagged boxer won or lost that particular match, respectively. Instantaneous intersubject phase synchronization provided a scan-by-scan measure of voxelwise synchrony between BOLD signals evoked across participants during the presentation of the boxers' videos. Similarly to moment-to-moment intersubject correlation computed with a sliding temporal window, this measure captures the degree of synchronization of hemodynamic brain activity across observers, with a temporal resolution as high as the sampling rate (1 TR of fMRI acquisition).

The main finding of the study was that mental simulation of a boxer's actions and feelings yielded stronger BOLD synchrony across participants' parietofrontal circuits than passively viewing the same video material (Nummenmaa et al., 2014, their Fig. 5). Both the WIN and LOSE conditions showed enhanced synchrony, with no differential effects between those mental simulation conditions.

The study by Nummenmaa and colleagues (2014) opens the way for studying whether neural synchrony across individuals represents a basic mechanism supporting action understanding. Neural synchrony might increase not only across observers, as in the action observation setting of Nummenmaa and colleagues (2014), but perhaps even during mutual understanding, when interactions between agents and observers are required. However, even when considering effects shared only across observers, the findings of Nummenmaa and colleagues (2014) raise a number of questions. For instance, if neural synchrony across action-observation networks of different observers represents the basic mechanism underlying the sharing of feelings, it remains unclear whether this mechanism pertains to sharing any emotional experience, or whether it is sensitive to the valence of the shared experience. The WIN and LOSE conditions evoked different feelings about the observed person and different activity in brain regions associated with affective and sensory processing (e.g., anterior insula and sensory cortices). Yet, there were no differences in neural synchronization patterns with regard to the feelings that were shared. This observation suggests that the strict time-locking of BOLD activity across individuals could have been driven by components other than those involved when mentally simulating the valence of a third person's feelings. Together with the fact that mental simulation yielded stronger synchrony in eye movements across participants than passive viewing, it seems plausible that the enhanced synchrony of participants' parietofrontal circuits is a consequence of increased levels of visuospatial attention to a boxer's actions. Increased visuospatial attention might be a prerequisite for mutual understanding, and increased attention might lead to increased synchrony across observers of the same sensory material, but this does not imply that neural synchrony is the basic mechanism that allows us to interact and create mutual understanding. Future work involving manipulations of mutual understanding between agents and observers, measuring whether feelings and thoughts are shared rather than levels of visuospatial attention, should provide more insight in the mechanisms supporting mutual understanding.

Mechanistically, it remains unclear how tight synchronization of action-observation networks across individuals could facilitate action understanding. The authors appeal to “resonance between the observers.” Resonance between two oscillating system components can increase the mechanical gain of that system, but this requires those components to influence each other in a predictable way and to show periodicity in the first place. It remains unclear whether these conditions can be applied to the domain of mutual understanding. Even if activity in those parietofrontal circuits was periodic, it remains a mystery how automatic sensorimotor resonances could deal with the complexity inherent to action understanding and thus by-pass many-to-many mappings between actions and their meanings. Arguably, understanding another's actions requires computations that go beyond the situational sensory evidence, as when distinguishing between a reflexive and an embarrassing cough, or between a compliment and a criticism (Basnáková et al., 2013). Mental action simulation could play an important role in action coordination with others as a convenient source of information about the mechanics of an action (Csibra, 2008). However, it seems unlikely that neural synchrony per se automatically results in grasping the potential content of that action. In sum, pending future work showing the social relevance of tight neural synchronization between two or more brains, currently there is no strong theoretical argument or empirical evidence that neural synchrony between sensorimotor systems is a mechanism for mutual understanding.

Does this mean that being in sync is just a metaphor for mutual understanding? Hypothetically, neural activity that is temporally synchronized across two or more individuals could be a marker of mutual understanding, even if it is not a mechanism. However, testing this hypothesis would require simultaneously studying action production and action understanding in two or more people engaged in truly interactive and dynamic social encounters, rather than studying synchronous processes in people processing the same sensory input. This approach is necessary for understanding how inferred knowledge and beliefs of our interlocutor(s) is adjusted in our minds as we interact (Brennan and Clark, 1996). In fact, coordination of this knowledge is a sine qua non for allowing mutual understanding that goes beyond mapping meaning to a string of actions or sounds, a cognitive feat achieved every day in our social interactions (Clark, 1996). For instance, when my friend tells me about his recent split from his girlfriend, the mutual adjustment of inferred knowledge shared between us could drive simultaneous changes in our neural activity and my pointing toward the bike to acquire a completely different meaning: indicating, perhaps, that we should not enter the pub.