Elsevier

NeuroImage

Volume 38, Issue 2, 1 November 2007, Pages 331-345
NeuroImage

Double dissociation between rules and memory in music: An event-related potential study

https://doi.org/10.1016/j.neuroimage.2007.07.034Get rights and content

Abstract

Language and music share a number of characteristics. Crucially, both domains depend on both rules and memorized representations. Double dissociations between the neurocognition of rule-governed and memory-based knowledge have been found in language but not music. Here, the neural bases of both of these aspects of music were examined with an event-related potential (ERP) study of note violations in melodies. Rule-only violations consisted of out-of-key deviant notes that violated tonal harmony rules in novel (unfamiliar) melodies. Memory-only violations consisted of in-key deviant notes in familiar well-known melodies; these notes followed musical rules but deviated from the actual melodies. Finally, out-of-key notes in familiar well-known melodies constituted violations of both rules and memory. All three conditions were presented, within-subjects, to healthy young adults, half musicians and half non-musicians. The results revealed a double dissociation, independent of musical training, between rules and memory: both rule violation conditions, but not the memory-only violations, elicited an early, somewhat right-lateralized anterior-central negativity (ERAN), consistent with previous studies of rule violations in music, and analogous to the early left-lateralized anterior negativities elicited by rule violations in language. In contrast, both memory violation conditions, but not the rule-only violation, elicited a posterior negativity that might be characterized as an N400, an ERP component that depends, at least in part, on the processing of representations stored in long-term memory, both in language and in other domains. The results suggest that the neurocognitive rule/memory dissociation extends from language to music, further strengthening the similarities between the two domains.

Introduction

Language and music share a range of characteristics (Besson and Schön, 2001, Maess et al., 2001, Patel, 2003, Koelsch, 2005). Of particular interest here, both domains crucially rely on both rules and memorized representations. In both language and music, higher-order structures (e.g., sentences and melodies) are made up of basic units (e.g., words and notes) that are arranged in a rule-governed hierarchical configuration. Language and music also both depend on long-term memorized representations that are at least partly idiosyncratic, that is, not entirely derivable from rules. For example, in language one must memorize various types of lexical knowledge, such as the sequence of phonemes /c/, /a/, and /t/ and that this refers to a furry, four-legged animal, as well as the fact that the combination of words “let the cat out of the bag” means “disclose a secret.” Likewise in music, a familiar tune must be memorized as a particular sequence of notes and rhythms.

The neurocognition of this rule/memory distinction has been well-studied in language but not in music. In language, evidence suggests that the two capacities rely on at least partially distinct neural bases. Double dissociations between rules and memory have been found using a variety of empirical approaches, including lesion studies of patients (Damasio, 1992, Goodglass, 1993, Ullman et al., 1997), neuroimaging (Newman et al., 2001, Friederici et al., 2003b), and event-related potentials (ERPs) (De Vincenzi et al., 2003, Sanders and Neville, 2003, Friederici et al., 2004).

ERPs have been particularly revealing. Violations of grammatical rules, e.g., in syntactic phrase structure or morpho-syntax, often elicit a “biphasic” pattern of ERPs. Initial negativities, generally found around 150–200 or 300–500 ms, and commonly referred to as (early) left anterior negativities – (E)LANs – due to their typical scalp distribution, are followed by later positivities (500–1000 ms), often called P600s due to their positivity and typical onset latency (Neville et al., 1991, Friederici et al., 1996, Coulson et al., 1998). LANs seem to be automatic, have been linked to aspects of grammatical structure building, and appear to depend at least in part on left frontal lobe structures (Hahne and Friederici, 1997, Friederici et al., 1998, Friederici et al., 1999b, Hagoort and Brown, 2000, Newman et al., 2007). The late positivities, by contrast, have a central-posterior distribution, seem to be controlled rather than automatic, have been implicated (at least in part) in aspects of syntactic integration, and appear to involve left temporal/temporo-parietal structures and the basal ganglia (Osterhout and Holcomb, 1992, Hahne and Friederici, 1997, Friederici et al., 1998, Friederici et al., 2003a, Kaan et al., 2000, Friederici, 2002).

Lexical-conceptual processing generally elicits bilateral central-to-posterior negativities known as N400s, which typically occur from about 250 to 500 ms post-stimulus (Kutas and Hillyard, 1980, Friederici et al., 1999a, Kutas and Federmeier, 2000, Hinojosa et al., 2001, Laurent et al., 2006). N400s seem to reflect one or more possibly related processes, including access to representations in long-term memory (thus the component is modulated by the frequency of the stimulus) and contextual integration, that is, the integration of the given stimulus with the current context held in working memory and with established long-term knowledge (Kutas and Federmeier, 2000). For example, reading or hearing a word that does not match the preceding context and long-term knowledge (e.g., “He shaved off his moustache and city.”) elicits an N400, as would seeing a picture of a city or hearing a sound of a car horn honking following the same sentence fragment (Van Petten and Rheinfelder, 1995, McPherson and Holcomb, 1999, Kutas and Federmeier, 2000, Federmeier and Kutas, 2001). Based on findings from patient, neuroimaging, and intracranial recording studies, N400s seem to reflect activity in multiple brain areas, including modality-specific regions of the temporal neocortex, ventrolateral prefrontal areas involved in processing information across modalities, and medial temporal lobe structures, including the hippocampus (Kutas and Federmeier, 2000).

ERP studies of rule violations in music have yielded results intriguingly similar to those found in language, suggesting at least some shared neurocognitive correlates across the domains. In a series of musical chords, harmonically inappropriate or dissonant chords generally elicit anterior negativities in a similar time window (about 150 to 280 ms) as some of the earlier LANs (i.e., ELANs) found in language studies (Verleger, 1990, Paller et al., 1992, Patel et al., 1998, Koelsch et al., 2000, Koelsch et al., 2002b, Koelsch et al., 2002c, Loui et al., 2005, Leino et al., 2007). However, whereas the negativities elicited by rule violations in language are generally left lateralized, those elicited by rule violations in music are usually right lateralized, or sometimes bilateral – hence the term early (right) anterior negativity, or E(R)AN – and have been linked to frontal lobe structures in both hemispheres (Maess et al., 2001, Koelsch et al., 2002a).

Later bilateral or right-lateralized anterior negativities (N5s/Late Negativities—450–1500 ms) have also been elicited by the same types of rule violations in music (Koelsch et al., 2000, Koelsch et al., 2002b, Koelsch et al., 2002c, Loui et al., 2005). Although it has been suggested that these negativities may be similar to the N400s found in language studies, reflecting musical “meaning” conveyed by the violations (Koelsch, 2005), the fact that they have an anterior scalp distribution and are elicited by rule-based violations suggests that they might also or instead be related to rule processing. Finally, musical rule violations also typically elicit late central to posterior positivities, in both melodies (Verleger, 1990, Paller et al., 1992, Besson and Faïta, 1995) and chord sequences (Patel et al., 1998, Koelsch et al., 2000, Koelsch et al., 2002c, Regnault et al., 2001). Interestingly, the later anterior negativities seem to be observed only (though not always; Steinbeis et al., 2006) when subjects do not attend to the violations (Koelsch et al., 2000, Koelsch et al., 2002b, Koelsch et al., 2002c, Loui et al., 2005), resulting in the reduction of the temporally overlapping late positivities that may mask the negativities due to additivity effects (Paller et al., 1992, Besson and Faïta, 1995, Koelsch et al., 2002c).

In contrast, the processing of idiosyncratic, non-rule-based, aspects of long-term memorized representations has not been well studied in music. In particular, we are aware of no ERP studies examining this issue. Moreover, to our knowledge, N400s have not been directly elicited by music in any studies. Note that although one study investigated in-key note (“diatonic”) violations in melodies, these notes actually appeared to violate musical rules, for example, non-tonic notes following leading notes (Besson and Faïta, 1995, Besson and Schön, 2001). Indeed, in a separate group of subjects, these violations in unfamiliar melodies were detected 46% of the time by musicians and 31% of the time by non-musicians, suggesting that at least some of these notes violated non-memory-based aspects of the melodies. Interestingly, the “diatonic” violations, as well as out-of-key note (“non-diatonic”) violations, appeared to elicit negativities between 200 and 400 ms in familiar melodies but not in unfamiliar melodies. However, these negativities were only shown in the figures and not reported in the text, and were immediately followed by large positivities that may have masked the negativities in later time windows due to additivity effects. It is thus unclear whether or not these negativities were related to N400s.

Despite the fact that N400s have not clearly been elicited by note violations in familiar melodies, neuroimaging studies have revealed that a similar network of brain areas is involved in processing memorized aspects of both music and language. For instance, semantic judgments of single words (Friederici et al., 2000, Booth et al., 2002, Pilgrim et al., 2002, Chou et al., 2006) and of idioms or metaphors (Rapp et al., 2004, Eviatar and Just, 2006, Zempleni et al., 2007) have been linked to activation of left or bilateral temporal/temporo-parietal regions, including Brodmann's areas (BA) 20 and 21, as well as left or bilateral anterior ventro-lateral prefrontal cortex (BA 45/47). The processing of semantic anomalies has additionally been linked to activation of medial temporal lobe structures, namely the hippocampus and parahippocampal gyrus (Newman et al., 2001). Each of these brain regions has also been activated during familiarity judgments of music, or while processing familiar as compared to unfamiliar melodies (Platel et al., 1997, Platel et al., 2003, Satoh et al., 2006, Plailly et al., 2007), suggesting at least partly overlapping neural substrates involved in processing memorized aspects of both music and language.

In the present ERP study, we examined the rule/memory contrast in music within subjects. Two types of note violations were examined in the same set of well-known melodies, as well as in matched novel melodies. “In-key” deviant notes in well-known melodies were appropriate with respect to musical rules but differed from the actual melody, thus serving as memory-based violations in well-known but not novel melodies. “Out-of-key” deviant notes violated rules of musical tonality. In novel melodies, out-of-key notes served solely as rule violations, whereas in well-known melodies, the notes violated both rules and memory. To ensure that each subject was actually familiar with each well-known melody, only those well-known melodies that that particular subject rated as familiar and novel melodies rated as unfamiliar in a separate test session were included for analysis. To reduce the attention-related late positivity, subjects were not informed of the in-key and out-of-key violations, but instead made responses about the timbre of the last note in each melody. We predicted a double dissociation between rule and memory violations: ERANs for rule violations but not for purely memory violations, and N400s for memory violations but not for purely rule violations. Finally, to probe whether these effects vary according to musical training, we included musical training as a factor in the experimental design: half the subjects were musicians and half were non-musicians.

Section snippets

Participants

We tested 64 adult (32 male, 32 female) monolingual native speakers of American English. Because knowledge of the well-known melodies in this study is largely culturally dependent, none of the subjects had lived outside the United States for more than six months before the age of 18. None of the subjects had any known developmental, neurological, or psychiatric disorders, and all had normal or corrected hearing and vision. Half the subjects (16 male, 16 female) were musicians, having at least

Behavioral results

In the ERP experiment, subjects accurately judged the presence (mean = 97.4%, SE = 1.2%) and absence (mean = 98.1%, SE = 0.6%) of timbre violations, with no significant differences between musicians and non-musicians (ps > 0.09). In the follow-up behavioral session, the familiarity ratings of the well-known melodies (mean = 4.23, SE = 0.06) differed between musicians (mean = 4.44, SE = 0.06) and non-musicians (mean = 4.02, SE = 0.09; t(52) = 3.76, p < 0.001), whereas the ratings of novel melodies (mean = 1.49, SE = 0.04)

Summary and discussion

In this study, a double dissociation was observed between rule and memory violations in music. Early anterior-central negativities were elicited by out-of-key violations in both unfamiliar and familiar melodies, whereas posterior negativities were elicited by both in-key and out-of-key violations in familiar melodies. The amplitude of the early anterior-central negativity did not differ between the two out-of-key (rule) violation conditions, while the amplitude of the posterior negativity did

Acknowledgments

We thank Aaron Newman, Petr Janata, John Drury, Matthew Walenski, Matthew Moffa, Jocelyn Curchack, Marco Piñeyro, Heidi Getz, and Brittany Sonnichsen for help on various aspects of this study. Support was provided to RAM from an NSF Graduate Research Fellowship and support from Georgetown University, and to MTU from NIH R01 MH58189 and R01 HD049347.

References (72)

  • S. Koelsch et al.

    Bach speaks: a cortical “language-network” serves the processing of music

    NeuroImage

    (2002)
  • M. Kutas et al.

    Electrophysiology reveals semantic memory use in language comprehension

    Trends Cogn. Sci.

    (2000)
  • J.-P. Laurent et al.

    On understanding idiomatic language: the salience hypothesis assessed by ERPs

    Brain Res.

    (2006)
  • S. Leino et al.

    Representation of harmony rules in the human brain: further evidence from event-related potentials

    Brain Res.

    (2007)
  • P. Loui et al.

    Effects of attention on the neural processing of harmonic syntax in Western music

    Cogn. Brain Res.

    (2005)
  • P. Messerli et al.

    Hemispheric dominance for melody recognition in musicians and non-musicians

    Neuropsychologia

    (1995)
  • T.F. Münte et al.

    Brain potentials in patients with music perception deficits: evidence for an early locus

    Neurosci. Lett.

    (1998)
  • A.J. Newman et al.

    An ERP study of regular and irregular English past tense inflection

    NeuroImage

    (2007)
  • R.C. Oldfield

    The assessment and analysis of handedness: the Edinburgh inventory

    Neuropsychologia

    (1971)
  • L. Osterhout et al.

    Event-related brain potentials elicited by syntactic anomaly

    J. Mem. Lang.

    (1992)
  • L. Pilgrim et al.

    Overcoming confounds of stimulus blocking: an event-related fMRI design of semantic processing

    NeuroImage

    (2002)
  • H. Platel et al.

    Semantic and episodic memory of music are subserved by distinct neural networks

    NeuroImage

    (2003)
  • A.M. Rapp et al.

    Neural correlates of metaphor processing

    Brain Res. Cogn. Brain Res.

    (2004)
  • L.D. Sanders et al.

    An ERP study of continuous speech processing: I. Segmentation, semantics, and syntax in native speakers

    Cogn. Brain Res.

    (2003)
  • M.T. Ullman

    Contributions of memory circuits to language: the declarative/procedural model

    Cognition

    (2004)
  • C. Van Petten et al.

    Conceptual relationships between spoken words and environmental sounds: event-related brain potential measures

    Neuropsychologia

    (1995)
  • M.-Z. Zempleni et al.

    Evidence for bilateral involvement in idiom comprehension: an fMRI study

    NeuroImage

    (2007)
  • M. Besson et al.

    An event-related potential (ERP) study of musical expectancy: comparison of musicians with nonmusicians

    J. Exp. Psychol. Hum. Percept. Perform.

    (1995)
  • M. Besson et al.

    Comparison between language and music

    Ann. N. Y. Acad. Sci.

    (2001)
  • T.G. Bever et al.

    Cerebral dominance in musicians and nonmusicians

    Science

    (1974)
  • J.R. Booth et al.

    Modality independence of word comprehension

    Hum. Brain Mapp.

    (2002)
  • V. Bostanov et al.

    Recognition of affective prosody: continuous wavelet measures of event-related brain potentials to emotional exclamations

    Psychophysiology

    (2004)
  • S. Coulson et al.

    Expect the unexpected: event-related brain response to morphosyntactic violations

    Lang. Cogn. Processes

    (1998)
  • A.R. Damasio

    Aphasia

    N. Engl. J. Med.

    (1992)
  • M. De Vincenzi et al.

    Differences in the perception and time course of syntactic and semantic violations

    Brain Lang.

    (2003)
  • S. Evers et al.

    The cerebral haemodynamics of music perception. A transcranial Doppler sonography study

    Brain

    (1999)
  • Cited by (100)

    • Musical and linguistic syntactic processing in agrammatic aphasia: An ERP study

      2022, Journal of Neurolinguistics
      Citation Excerpt :

      There are mixed findings regarding how this response differs between musicians and non-musicians. In response to out-of-key violations in unfamiliar melodies, some studies found P600 responses with larger amplitudes and earlier latencies in musicians (Besson et al., 1994), while others found no differences in P600 responses between musicians and non-musicians (Miranda & Ullman, 2007). However, for familiar melodies, Miranda and Ullman (2007) found that musicians had a larger amplitude P600 response to out-of-key violations.

    • Behavioral evidence of a harmonic surprise effect on preference in popular music

      2022, Current Research in Behavioral Sciences
      Citation Excerpt :

      The resulting variability in listeners’ familiarity with different songs could therefore be a confounding factor mediating any results reported to be associated with harmonic surprise. Two distinct neural mechanisms are implicated in the learning of harmonic musical information (Miranda and Ullman, 2007). One involves the processing of statistically learned information about musical harmony.

    View all citing articles on Scopus

    Portions of this article were included as part of a doctoral dissertation by R.A. Miranda.

    View full text