Research ReportRepresentation of harmony rules in the human brain: Further evidence from event-related potentials
Introduction
When listening to music, a sequence of chords generates expectancies for notes or chords to follow. Such expectancy is vital to musical experience (e.g., Meyer, 1956). In particular, the harmonic context of a chord sequence primes the processing of chords related to the context and induces expectations by activating tonal representations existing in the mind of the listener (Bharucha and Stoeckig, 1987). Expectancies generated by a harmonic context reflect the innate or learned mental representation of tonal relationships, or tonality. Tonality (also termed musical key) refers to the organization of pitches in such a way that one central pitch dominates and attracts the others and gives name to the key (see e.g., Bharucha, 1984, Bharucha and Krumhansl, 1983, Krumhansl, 2000). The core of a functional harmonic context is often composed of tonic (T), dominant (D), and subdominant (S) chords. Compared to other chord functions, these three chord functions are perceived as more closely related to each other (Bharucha and Krumhansl, 1983). The dominant is perceived as tension-creating, demanding resolution to the stable position of the tonic, whereas the subdominant reflects an intermediate position between these two oppositions. Chords incorporating notes outside the prevailing harmonic context usually demand resolution to more stable harmonies of the system (Bharucha and Krumhansl, 1983). The mental representation of tonality may be established by very few notes or chords (Berent and Perfetti, 1993, Krumhansl and Kessler, 1982). Chords breaking these harmonic expectations may be perceived as erroneous, and a reinterpretation of the current tonality may occur (Berent and Perfetti, 1993).
Recently, several event-related potential (ERP) studies investigating the violation of harmonic expectations based on the rules of Western tonal music have been conducted (e.g., Koelsch et al., 2000, Koelsch et al., 2005, Loui et al., 2005, Maess et al., 2001). In these studies, inappropriate chords within or at the end of harmonic cadences (chord sequences) elicited an event-related potential (ERP) component called the early right anterior negativity (ERAN). The ERAN occurs at an early latency (150–250 ms after stimulus onset) and is maximal over anterior regions of the scalp with a tendency to be lateralized to the right. The ERAN has been most commonly recorded in a semi-attended paradigm, where the subjects’ attention is directed towards the music by asking them to respond to infrequent chords played on a deviant instrument. It can also be elicited preattentively (Koelsch et al., 2002b), however its amplitude is modulated by the attentional load (Loui et al., 2005). The ERAN has been shown to be larger in musical experts than in novices (Koelsch et al., 2002c). Utilizing magnetoencephalography (MEG), Maess et al. (2001) localized the sources of the magnetic ERAN response in the Broca’s area (BA44) and its right-hemispheric homologue with a non-significant tendency towards right-hemispheric superiority. The left-hemispheric Broca’s area is involved in the processing of linguistic syntax (see e.g., Friederici et al., 2000). This result has been interpreted to demonstrate an analogy of the rules of harmony in music with the rules of linguistic syntax since both comprise of ordered succession of meaningful sound events in time (e.g., Koelsch and Siebel, 2005, Patel, 2003).
In previous studies, the violation of the rules of harmony was realized by placing an inappropriate chord, the Neapolitan subdominant (Sn), into the authentic cadence (see Table 1). The Neapolitan subdominant is the first inversion of the major triad built on the flattened second degree of the scale (in C major or minor: F-Ab-Db). The Neapolitan chord includes notes outside of the prevailing tonality (or key) and is used in Western classical music as a variation of the subdominant, most often preceding a V-I cadence. In the previous experiments, the Neapolitan chord was presented either at position 3 or at the ending position 5 of the cadence. It is noteworthy that each trial in the previous ERAN studies was performed in a different key. Therefore the previous findings, showing a prominent ERAN mainly in position 5 (T) compared to position 3 (S), might be a consequence of a less well established key in position 3 rather than a violation of expectancies based on harmony rules of chord succession. Also note that at position 5 the Neapolitan chord always followed a dominant chord, which created a strong expectation for a tonic chord. In contrast, at position 3 the Neapolitan chord always followed a chord that created an expectation for a subdominant chord. According to Western musical theory, when used to replace a subdominant chord, the Neapolitan generates a chord progression more accepted in the theory of functional harmony than when placed after a dominant chord (Drabkin, 2002). The most appropriate resolution of the dominant is a tonic chord. When placed after a dominant chord, i.e. replacing the strongly expected tonic chord, the Neapolitan therefore creates an inappropriate chord succession. This is especially true when the chord that is to be replaced is the final chord of the cadence. The comparison used in the previous studies between Neapolitan chords at position 3 and at position 5 of a five-chord cadence is therefore unbalanced as it contrasts expectations which are formed by either the building of tonality or by the harmony rules of ordering of musical events. Consequently, also the elicitation of a larger ERAN by a Neapolitan in position 5 than in position 3 of the cadence (e.g., Koelsch et al., 2000) may be the result of a confound between the effects of the building of a tonal context and the processing of the harmony violation.
In our study we wished to separate the effects of tonality establishment within a musical context and the effects of a violation of harmony rules, which determine the succession and music-structural importance of chords within a cadence. To accomplish this, the present study was based on a chord sequence containing seven chords instead of five chords as in previous ERAN studies (see Table 2 and Fig. 1). Our standard chord sequence formed an authentic cadence (T-D-T-T3-S-D-T) following the rules of Western functional harmony. Three types of chord sequences containing Neapolitan chords addressed the abovementioned problems by reversing the order of expectation. At positions 3 and 7, the Neapolitan chord always followed a dominant chord, hence replacing a tonic chord. In this way, according to harmony rules, a very inappropriate succession was created. In one case, at position 3, the tonality was not yet established and in the other case, at position 7, the tonality was well established and a strong expectation for the final tonic chord was created (for the prominent role of the last chord of a cadence for evoking a feeling of closure, see Boltz, 1989, Palmer and Krumhansl, 1987). The effects of a harmony violation can therefore be assessed for both final and non-final cadence positions.
Moreover, the Neapolitan chord at position 5 always followed a tonic chord in the first inversion, hence replacing a subdominant chord and producing a more appropriate harmonic succession than at positions 3 and 7. On the other hand, the Neapolitan chord replacing a subdominant chord at position 5 contrasted in turn with the better established tonality expectations for the chord at position 5 than for the chord at the earlier position 3 (the chord at position 3 contrasting more with the expectations based on the rules of functional harmony). Thus, assuming that the ERAN response mainly reflects the processing of violations of harmony rules, as suggested previously, and not the establishment of tonality, the ERAN elicited by a Neapolitan chord in position 3 should be larger in amplitude than the ERAN elicited by a Neapolitan chord in position 5.
There was an additional manipulation of the experimental paradigm introduced into our standard sequence mistuned chords, where the fifth of the chord was raised in pitch from its standard position in the musical scale. Similarly as the Neapolitan chords, the mistuned chords were included at positions 3, 5, and 7 of the chord cadence. This manipulation aimed at further studying the cognitive nature of electrical brain responses elicited by violations of chord progression. We hypothesized that the violation of the tuning of a chord would represent a musical-scale violation. This kind of a violation would not be regulated by the rules of chord progressions or by their music-structural importance within the cadence, but would be based on the comparison with the scale steps used in the previous chords of the cadence. In other words, this manipulation violated the tuning properties of all the sounds upon which harmony is based.
Violations of simple rules governing the properties of auditory information are known to elicit an ERP component called the mismatch negativity (MMN: see e.g., Näätänen and Winkler, 1999, Picton et al., 2000). The MMN has been shown to reflect the automatic formation of a short-term neural model of the physical or abstract regularities in the auditory environment (Winkler et al., 1996). The MMN is a fronto-central negative potential with sources in the primary and non-primary auditory cortex and a latency of 150–250 ms. In our paradigm, if the MMN reflected a tuning rule violation, it should be similar in amplitude and latency for all the three different cadence positions. In contrast, the ERAN, if truly an index of harmony violations, should be modulated by the development of a cognitive representation of tonality in the course of cadence presentation. The ERAN is also expected to have a different latency, amplitude, and scalp distribution than the MMN.
Section snippets
Results
When presented within chord cadences following the rules of Western functional harmony, statistically significant ERP responses were elicited by both Neapolitan chords (p < 0.05) and mistuned chords (p < 0.01). Neapolitan chords, violating the rules of harmony, elicited a negative ERP response peaking on average 236 ms post-stimulus (see Fig. 2). Mistuned chords, violating the rules of musical-scale tuning, elicited a negative ERP response peaking on average 270 ms post-stimulus (see Fig. 3).
The
Discussion
The present study was conducted to elucidate the relationship between the neural processing of harmony rules, tuning rules, and the establishment of tonality. Harmonically inappropriate chords (Neapolitan subdominants) were found to elicit a brain response which was modulated by the degree of the violation of harmony rules. The results allow us to discern the effects of tonality establishment within a chord cadence from those of the violation of the rules of Western functional harmony on a
Participants and stimuli
Ten right-handed subjects (5 male, 5 female; age range 22–30 years, mean age 26 years) participated in the experiment. All participants had normal hearing and no musical expertise or explicit knowledge of music theory. Written informed consent was received from all participants. Participants received monetary compensation for taking part in the experiment.
The stimuli used in the experiment were digitally generated piano and organ chords organized into cadences. The stimulus chords were prepared
Acknowledgments
The authors would like to thank Dr. Pauli Brattico and the anonymous reviewer for helpful comments on previous versions of the manuscript, Dr. Sandra Khromatidi for help in data analysis, and Pia Dam Petersen for help in preparing the stimuli. The project was supported by the Academy of Finland (project 200522).
References (40)
- et al.
An on-line method in studying music parsing
Cognition
(1993) Anchoring effects in music: the resolution of dissonance
Cogn. Psychol.
(1984)- et al.
The representation of harmonic structure in music: hierarchies of stability as a function of context
Cognition
(1983) - et al.
A module for syntactic processing in music?
Trends Cogn. Sci.
(2006) - et al.
Musical scale properties are automatically processed in the human auditory cortex
Brain Res.
(2006) - et al.
Differential task effects on semantic and syntactic processes as revealed by ERPs
Cogn. Brain Res.
(2002) - et al.
The capacity for music: what is it, and what’s special about it?
Cognition
(2006) Neural substrates of processing syntax and semantics in music
Curr. Opin. Neurobiol.
(2005)- et al.
Towards a neural basis of music perception
Trends Cogn. Sci.
(2005) - et al.
Bach speaks: a cortical “language-network” serves the processing of music
NeuroImage
(2002)
Effects of attention on the neural processing of harmonic syntax in Western music
Cogn. Brain Res.
Separate cortical networks involved in music perception: preliminary functional MRI evidence for modularity of music processing
NeuroImage
To musicians, the message is in the meter pre-attentive neuronal responses to incongruent rhythm are left-lateralized in musicians
NeuroImage
It don’t mean a thing… Keeping the rhythm during polyrhythmic tension, activates language areas (BA47)
NeuroImage
Adaptive modeling of the unattended acoustic environment reflected in the mismatch negativity event-related potential
Brain Res.
Perceptual musical analysis: segmentation and perception of tension
Music. Sci.
Priming of chords: spreading activation or overlapping frequency spectra?
Percept. Psychophys.
Rhythm and “good endings”: effects of temporal structure on tonality judgments
Percept. Psychophys.
Context effects on pitch perception in musicians and nonmusicians: evidence from event-related potential recordings
Mus. Percept.
The musical work as discourse and text
Music. Sci.
Cited by (84)
Contextual prediction modulates musical tension: Evidence from behavioral and neural responses
2021, Brain and CognitionCitation Excerpt :In line with these theories, researchers have conducted a series of empirical studies on music tension by replacing a harmonically expected chord with a harmonically unexpected chord. In one case, unexpected chord appears at the end of the chord sequence (Koelsch et al., 2010a; 2010b; 2008a; 2008b; 2002; 2000; Leino, Brattico, Tervaniemi, & Vuust, 2007; Steinbeis et al., 2008; 2006; Villarreal, Brattico, Leino, Østergaard, & Vuust, 2011). Under this case, it is found that tension increased while the harmonic chords were unexpected (Steinbeis, Koelsch, & Sloboda, 2006).
Mapping Tonal Hierarchy in the Brain
2021, NeuroscienceCitation Excerpt :The ERAN is an increase in negativity that occurs around 150–250 ms after the violating stimulus, and is best visualized using a difference wave that is calculated by subtracting the EEG response of a stimulus that violates musical syntax from one that does not. It typically peaks around 200 ms after the syntax-violating stimulus, tends to have a right, fronto-anterior scalp distribution, and can be evoked even when the listener is not attending to the musical stimuli (Koelsch, 2009b; Koelsch et al., 2001; Koelsch and Mulder, 2002; Leino et al., 2007). The ERAN is therefore influenced by the obligatory components of the auditory evoked response that occur during a similar epoch, in this case the N1 and P2 components at that peak at approximately 100 ms and 200 ms post-stimulus respectively (Crowley and Colrain, 2004; Näätänen and Picton, 1987).
Event related potentials at initial exposure in third language acquisition: Implications from an artificial mini-grammar study
2020, Journal of NeurolinguisticsEffects of global and local contexts on chord processing: An ERP study
2018, NeuropsychologiaMusical Expectations Enhance Auditory Cortical Processing in Musicians: A Magnetoencephalography Study
2018, NeuroscienceCitation Excerpt :For example, GBD–CEG can be regarded as V (dominant) – I (tonic chord) in a key of C major and I–IV (subdominant chord) in a key of G major (Regnault et al., 2001; Poulin-Charronnat et al., 2006). Such regularities establish musical syntax, which has been reported to be processed in right-lateralized structures in the frontal cortex (Maess et al., 2001; Leino et al., 2007; Kim et al., 2011; Koelsch et al., 2013), while the effects of the spectral properties of sound, training or experience, regardless of musical context, have been reported mainly in subcortical regions or auditory cortices (Marmel et al., 2011a; Fritz et al., 2013; Bidelman et al., 2014). The present question is whether harmonic expectancies generated in the frontal cortex influence auditory cortical processing in a top-down manner.