Elsevier

NeuroImage

Volume 53, Issue 2, 1 November 2010, Pages 638-646
NeuroImage

Neural correlates of implicit and explicit combinatorial semantic processing

https://doi.org/10.1016/j.neuroimage.2010.06.055Get rights and content

Abstract

Language consists of sequences of words, but comprehending phrases involves more than concatenating meanings: A boat house is a shelter for boats, whereas a summer house is a house used during summer, and a ghost house is typically uninhabited. Little is known about the brain bases of combinatorial semantic processes. We performed two fMRI experiments using familiar, highly meaningful phrases (lake house) and unfamiliar phrases with minimal meaning created by reversing the word order of the familiar items (house lake). The first experiment used a 1-back matching task to assess implicit semantic processing, and the second used a classification task to engage explicit semantic processing. These conditions required processing of the same words, but with more effective combinatorial processing in the meaningful condition. The contrast of meaningful versus reversed phrases revealed activation primarily during the classification task, to a greater extent in the right hemisphere, including right angular gyrus, dorsomedial prefrontal cortex, and bilateral posterior cingulate/precuneus, areas previously implicated in semantic processing. Positive correlations of fMRI signal with lexical (word-level) frequency occurred exclusively with the 1-back task and to a greater spatial extent on the left, including left posterior middle temporal gyrus and bilateral parahippocampus. These results reveal strong effects of task demands on engagement of lexical versus combinatorial processing and suggest a hemispheric dissociation between these levels of semantic representation.

Introduction

Comprehending language involves more than just understanding individual words; the meanings of individual words are fluently combined to produce larger structures expressing relations between the constituent words. Although the neural structures that support the comprehension of isolated words have been studied extensively (Binder & Price, 2001, Pulvermüller, 1999), less is known about the brain bases of combinatorial semantic processes. We investigated these processes using simple noun–noun phrases such as lake house and house lake. The meaning of lake house depends on the meanings of the two words but expresses a further relation between them: a lake house is a house located on or near a lake. House lake, however, does not express an easily interpretable relation between the same words (Gagné and Shoben, 1997). This difference arises from the underlying semantic structure of the constituent nouns, which determines how naturally or automatically their meanings are combined. For example, large, stationary objects like houses have a fixed location and thus can be felicitously combined as head nouns with a modifying noun describing a larger object on which the head noun is located (e.g., country house, city house, beach house, mountain house, prairie house, etc.). House lake violates this semantic constraint because lakes are larger than houses. This is not to say that house lake cannot be interpreted with some additional effort (a lake on which there are numerous houses?), but we assume that in such cases the combination is constructed less successfully, resulting in little meaning or one marked by considerable residual ambiguity. In such cases, the typicality of the relationship between words influences ease of comprehension. The modifier noun mountain, for example, more often indicates a location relationship with the head noun (mountain stream) than an “about” relationship (mountain magazine), and the more typical relations are associated with faster sensibility judgments (Gagné, 2001, Gagné & Shoben, 1997, Gagné & Spalding, 2004, Gagné & Spalding, 2009).

We examined these combinatorial semantic processes by contrasting highly meaningful noun–noun phrases with their reversed, minimally meaningful forms. The conditions differ in meaningfulness but are matched with respect to word-specific properties. The three aims of this study were (1) to identify the neural systems that support successful combinatorial processing, (2) to identify the neural correlates of lexical (word-level) processing as distinct from combinatorial processing, and (3) to compare activation in these neural systems for tasks that engage explicit compared to implicit semantic processing. In contrasting meaningful phrases with their reversed versions, we expected that the reversed phrases would elicit greater effort, attention, and working memory in searching for a viable interpretation, particularly during an explicit semantic judgment task. Our main focus, however, is on the neural signature of successful conceptual combination, as identified by higher levels of activation for items that participants judged to be meaningful than for items they judged to be meaningless. Our interest in this aspect of fluent semantic processing arises from its ubiquity and central importance in everyday language use.

In addition to the relation-based account described above, several other mechanisms have been proposed to underlie combinatorial semantic processing. These mechanisms are not mutually exclusive, and our study was not intended to distinguish among them. The inventory of relation-based interpretations derives from world knowledge (e.g., concerning properties and functions of objects, the contexts in which they are used, and so on), encoded by knowledge structures such as schemas (Costello & Keane, 2000, Smith et al., 1988). Murphy, 1988, Murphy, 2002 focuses on cases in which phrases must be interpreted with respect to relatively specific world knowledge. To take an example from the current study, the meaning of flower girl does not derive in any obvious way from a relation between the head noun and modifier, nor do the properties of girl and flower appear to align in any useful way. While flower girl refers to a girl who carries or scatters flowers, knowing that this is done at a wedding by a girl who is too young to be a bridesmaid is critical to understanding the phrase. A somewhat different proposal holds that noun–noun combinations are interpreted in terms of their shared properties (Wisniewski and Love, 1998). For example, in the relation approach, robin hawk could be a hawk that preys on robins. According to Wisniewski and Love (1998) this phrase could be interpreted in terms of the properties of the nouns, where a robin hawk could be a hawk with a red breast. Thus, interpretations in which “one or more properties of the modifier concept apply in some way to the head concept,” also play a role in conceptual combination, as demonstrated for about 30% of their noun–noun phrases. The current study focused on the processing of phrases for which a meaningful interpretation can be readily derived (e.g., flower girl), by comparison to phrases such as girl flower that lack conventional meanings and can only be interpreted with effort and in varying ways. We assume that activations for meaningful compared to reversed phrases reveal the neural systems used to derive phrase-level meaning through lexical semantic combination under typical conditions involving compatible semantic constraints provided by the head and modifier nouns.

The above theories share the assumption that determining the meanings of noun–noun phrases involves combinatorial processes. It is also possible, however, that many such combinations are stored as lexical entries. A phrase like lake house, for example, having been encountered in the past, may be stored as a lexical item, much like cottage or farmhouse. Numerous studies have examined how phrase-level frequency affects the comprehension of noun–noun constructions. At one extreme, very frequent combinations are often labeled compound words or collocations, although the boundary between “noun–noun phrase” and “compound word” is not well defined or reliably indicated by typography (e.g., inclusion of a hyphen or space between the words; Marchand, 1969). For example, front door is almost always written with a space, whereas back door is nearly equally written with and without the space. Familiar phrases and compounds might be stored as lexical entries, obviating the need for combinatorial processing. This hypothesis has been offered as an alternative interpretation of previous studies of conceptual combination (Gagné & Spalding, 2006, Murphy & Wisniewski, 2006, Wisniewski & Murphy, 2005). Recent evidence from lexical decision during online sentence comprehension suggests that the lexical constituents of familiar phrases are processed both individually and combinatorially (Swinney et al., 2007), but the issue is not settled (Murphy and Wisniewski, 2006). In the present study, we addressed this issue empirically by including in the fMRI analyses a continuous regressor for whole-phrase frequency (see Methods for details). This enabled the effects of combinatorial processing to be examined separately from any effects of phrase usage frequency.

Several areas of prior imaging research are relevant to the current study. Combinatorial semantic processing is presumably related to the process of semantic integration in sentence comprehension. Electrophysiological investigations of semantic integration often involve sentence stimuli in which the beginning of the sentence sets up a semantic context (e.g., I like my coffee with cream and) that is violated by the final word (e.g., socks). This manipulation typically results in a negative-going current peaking around 400 ms after the stimulus of interest (socks) (Kutas and Federmeier, 2000). Although this result, referred to as the N400, is often taken to reflect attempts to semantically integrate the target word with its preceding context (Hagoort, 2008), an alternate interpretation is that the incongruent context leads to increased difficulty of lexical access for the target word (Lau et al., 2008). Relevant to the current fMRI study, recent reviews have tentatively localized N400 effects to primarily left-hemisphere (LH) regions within the temporoparietal and inferior frontal lobes (Lau et al., 2008, Van Petten & Luka, 2006). Regarding conceptual combination, Koester et al. (2009) presented German compounds for semantic judgment and found an increased N400 for less plausible head constituents. Similarly, El Yagoubi et al. (2008) presented Italian compounds (e.g., capobanda, band leader) for lexical decision, using nonword trials constructed by reversing the order of the constituents in the compound words (e.g., bandacapo). A significantly larger N400 was found for nonwords compared to compound words. Although relevant in the sense that they deal with compounds, these results stand in contrast to the goal of the current study, which is to reveal the neural correlates of successful combinatorial semantic processing, the conditions for which are maximized by presenting highly meaningful phrases and minimized by presenting phrases for which the reversed form has minimal meaning. Thus if N400 effects increase with difficulty of lexical processing, and occur primarily in left temporoparietal and inferior frontal areas, then we might expect to see activation related to lexical processing in the LH that is distinct from areas related to combinatorial semantic processing.

To distinguish “the amount of effort needed to perform semantic integration” from the “degree to which the target word is pre-activated by context,” Pylkkänen and McElree (2007) used magnetoencephalography (MEG) to compare activation for sentences such as the author began the book, in which a meaning (in this case, writing) is implied but not stated, with both control (the author wrote the book) and anomalous sentences (the author disgusted the book). They found increased signal amplitude for implied-meaning phrases compared to anomalous and control phrases that peaked around 400 ms after presentation of the critical word (in this case, book) and localized to the anterior midline region. This study and a follow-up that found similar results using a different task (Pylkkänen et al., 2009) are instructive in that they reveal a neural correlate of N400-like effects that are distinct from those induced by semantic anomaly, and point to a possible candidate area for the kind of semantic integration that may also take place in conceptual combination.

Regarding lexical-level processing, several recent studies have demonstrated LH activation associated with increased levels of lexical-semantic information, as indexed by high word frequency and imageability (Bedny & Thompson-Schill, 2006, Binder et al., 2005a, Binder et al., 2005b, Carreiras et al., 2009, Graves et al., in press, Jessen et al., 2000, Prabhakaran et al., 2006, Sabsevitz et al., 2005). Thus, if lexical processing occurs in parallel with or just prior to combinatorial processing, LH systems that support lexical-semantic processing should be activated to the extent that a phrase contains familiar lexical units. The lexical constituents of these combinations are presumably processed prior to computing the phrase-level concept, as suggested, for example, by the results of the Swinney et al. (2007) study discussed above. In the present study we investigated the neural correlates of lexical processing by performing an fMRI analysis using the sum of the frequencies of the lexical items in each phrase. This analysis, performed for both of the experiments reported here, also included terms for meaningful and reversed phrases, thereby potentially revealing separate neural correlates for lexical compared to phrase-level semantic processing for the same stimuli.

The first experiment used a 1-back task that required monitoring for repetition of single words across phrases. Although this task does not require conceptual combination, it had the advantage of placing similar performance demands on meaningful and reversed phrases, thereby allowing any activation differences between meaningful and reversed phrases to be attributable to implicit (i.e., obligatory) combinatorial semantic processing. A small preliminary behavioral study (N = 10) was performed outside the scanner to confirm that these conditions were equated in terms of difficulty as measured by reaction time and error rate. Because of the possibility that conceptual combination might not occur during the 1-back task, a second fMRI experiment was performed using the same stimuli but with a classification task that required phrases to be judged for meaningfulness, a more explicit task. If similar processes are engaged for implicit and explicit semantic processing, similar activation patterns for the conditions of interest should obtain across the two experiments, but to a somewhat greater extent for the second experiment due to the more extensive processing needed to perform the semantic judgment task. Alternatively, if the 1-back task primarily engages lexical processing and the semantic judgment task primarily engages combinatorial processing, then there may be little or no overlap across the two tasks.

Section snippets

Stimulus selection and norming

The same 400 stimuli of interest were used in both experiments, and a complete list is provided in the supplemental material (Table S2). Stimulus selection began by compiling a list of all English words in the CELEX database (Baayen et al., 1995) that have a higher noun than verb or adjective frequency. The 500 most highly imageable words in this list were selected using a database of imageability ratings compiled from six sources (Bird et al., 2001, Clark & Paivio, 2004, Cortese & Fugett, 2004

Behavioral results

Response times (RTs) for correct 1-back responses following meaningful and reversed phrases were compared, as were error rates across subjects. No reliable performance differences (either RT or error) were observed for 1-back responses to meaningful (mean RT: 915 ms, SD: 271, mean subject-wise percent error rate: 8.5, SD: 8.2) compared to reversed (RT: 929 ms, SD: 270, error rate: 9.1, SD: 7.5) phrases.

Phrase-level imaging results

For Experiment 1 using the 1-back task, the contrast of meaningful (forward) compared to

Discussion

This study examined the neural correlates of combinatorial semantic processing, as distinct from lexical-level processing, during processing of noun–noun combinations in which words were presented in either meaningful or reversed order. We assumed that the process of successfully combining two concepts to form a third concept produces a neural signature detectable by fMRI. In contrast, when two concepts do not combine in a clearly meaningful way, this neural signature representing the

Acknowledgments

We thank Jon Willits for help obtaining phrase frequencies, David A. Medler, Ph.D., for providing the composite imageability database, and Edward Possing for help with data collection. This work was supported by NIH grants from the NINDS to author JRB (R01 NS033576) and the NICHD to author WWG (F32 HD056767).

References (57)

  • R.C. Oldfield

    The assessment and analysis of handedness: the Edinburgh inventory

    Neuropsychologia

    (1971)
  • R. Prabhakaran et al.

    An event-related fMRI investigation of phonological–lexical competition

    Neuropsychologia

    (2006)
  • L. Pylkkänen et al.

    The anterior midline field: coercion or decision making?

    Brain Lang.

    (2009)
  • J. Rodd et al.

    Modelling the effects of semantic ambiguity in word recognition

    Cogn. Sci.

    (2004)
  • D.S. Sabsevitz et al.

    Modulation of the semantic system by word imageability

    Neuroimage

    (2005)
  • E.E. Smith et al.

    Combining prototypes: a selective modification model

    Cogn. Sci.

    (1988)
  • C. Van Petten et al.

    Neural localization of semantic context effects in electromagnetic and hemodynamic studies

    Brain Lang.

    (2006)
  • E.J. Wisniewski et al.

    Relations versus properties in conceptual combination

    J. Mem. Lang.

    (1998)
  • R.H. Baayen et al.

    The CELEX lexical database

  • M. Beeman et al.

    Summation priming and coarse semantic coding in the right hemisphere

    J. Cogn. Neurosci.

    (1994)
  • J. Binder et al.

    Functional neuroimaging of language

  • J.R. Binder et al.

    Distinct brain systems for processing concrete and abstract concepts

    J. Cogn. Neurosci.

    (2005)
  • J.R. Binder et al.

    Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies

    Cereb. Cortex

    (2009)
  • H. Bird et al.

    Age of acquisition and imageability ratings for a large set of words, including verbs and function words

    Behav. Res. Meth. Instrum. Comput.

    (2001)
  • M. Carreiras et al.

    Syllable congruency and word frequency effects on brain activation

    Hum. Brain Mapp.

    (2009)
  • J.M. Clark et al.

    Extensions of the Paivio, Yuille, and Madigan (1968) norms

    Behav. Res. Meth. Instrum. Comput.

    (2004)
  • M.J. Cortese et al.

    Imageability ratings for 3, 000 monosyllabic words

    Behav. Res. Meth. Instrum. Comput.

    (2004)
  • R.W. Cox et al.

    Real-time 3D image registration of functional MRI

    Magn. Reson. Med.

    (1999)
  • Cited by (103)

    • Age-related differences in the neural network interactions underlying the predictability gain

      2022, Cortex
      Citation Excerpt :

      These findings point towards a potential demand-related role of right angular gyrus in semantic processing. There is mounting evidence that both left and right angular gyri are involved in (combinatorial) semantic processes (Binder et al., 2009; Golestani et al., 2013; Graessner, Zaccarella, & Hartwigsen, 2021; Graves, Binder, Desai, Conant, & Seidenberg, 2010; Price, Bonner, Peelle, & Grossman, 2015). During speech comprehension under challenging listening conditions, right angular gyrus showed increased activity for related words at higher intelligibility in young participants, but at a lower threshold than left angular gyrus (Golestani et al., 2013).

    View all citing articles on Scopus
    View full text