Dissociable neural correlates for familiarity and recollection during the encoding and retrieval of pictures

Abstract

Results from behavioral studies have supported the idea that recognition memory can be supported by at least two different processes, recollection and familiarity. However, it remains unclear whether these two forms of memory reflect neurally distinct processes. Furthermore, it is unclear whether recollection and familiarity can be best conceived as differing primarily in terms of retrieval processing, or whether they additionally differ at encoding. To address these issues, we used event-related brain potentials (ERPs) to monitor neural correlates of familiarity and recollection at both encoding and retrieval. Participants studied pictures of objects in two types of study blocks and subsequently made remember–know and source memory judgments during retrieval. Results showed that, during encoding, neural correlates of subsequent familiarity and recollection onsetted in parallel, but exhibited differences in scalp topography and time course. Subsequent familiarity-based recognition was associated with a left-lateralized enhanced positivity and observed at anterior scalp sites from 300 to 450 ms, whereas subsequent recollection was associated with a topographically distinct right-lateralized positivity at anterior scalp sites from 300 to 450 ms and bilateral activity from 450 to 600 ms. During retrieval, neural correlates of familiarity emerged earlier than correlates of recollection. Familiarity was associated with an enhanced positivity at frontopolar scalp sites from 150 to 450 ms, whereas recollection was associated with positive ERP modulations over bilateral frontal (300–600 ms) and parietal (450–800 ms) sites. These results demonstrate that familiarity and recollection reflect the outcome of neurally distinct memory processes at both encoding and retrieval.

Introduction

Findings from several studies have supported the idea that recognition memory may be supported by at least two processes: the assessment of an item's familiarity and the recollection of specific aspects of the episode during which an item was encountered [1], [2], [36], [73]. Several behavioral methods have been used to successfully dissociate recollection and familiarity [21], [22], [23], [28], but it is unclear whether these forms of memory reflect qualitatively and neurally distinct processes [34], [59], [74], [75].

One hypothesis regarding familiarity and recollection is that they are largely overlapping, relying on the same neural system [34], [39] but that recollection necessitates additional strategic processing at retrieval [31]. In support of this view, a recent meta-analysis of multiple empirical studies suggests that while various experimental manipulations can produce double dissociations between recollection and familiarity during retrieval, similar manipulations produce only single dissociations during encoding (see [73] for review). Collectively, these data would seem to suggest that recollection and familiarity rely on a similar neural system during encoding and may only be independent and dissociable at the time of retrieval, when controlled processing is essential for full recognition.

An alternative hypothesis regarding familiarity and recollection is that they rely on functionally distinct neural systems (see [28], [72] for reviews). Support for this view comes from studies of patients with medial temporal lobe lesions [2], [75], suggesting that the hippocampus may be specifically critical for recollection, whereas surrounding regions in the rhinal cortex may be sufficient to support familiarity. Furthermore, some recent functional neuroimaging studies have shown that encoding related activity in the hippocampus and posterior parahippocampal cortex may support subsequent recollection whereas perirhinal activity may support subsequent familiarity [12], [50]. If distinct neural systems support familiarity and recollection, as these studies suggest, then these processes should be dissociable both at the time of encoding and retrieval.

Further insight into the neural substrates of familiarity and recollection might be gained through the use of event-related brain potentials (ERPs) to monitor neural activity associated with these forms of memory. Most ERP studies of episodic memory that have attempted to dissociate neural correlates of familiarity and recollection have focused on measures of brain activity during episodic retrieval. Results from these studies suggest that different ERP “old–new” effects (i.e. ERP differences between studied and unstudied items) appear to differentiate between familiarity-based recognition and recollection, suggesting that the two forms of memory may rely on different neural processes at retrieval. For example, one old–new effect that has been termed the “FN400” appears as an enhanced negativity for new items compared to correctly recognized old items over frontal locations between 300 and 500 ms (see [19] for review). Some researchers have suggested that the FN400 old–new effect may be a neural correlate of familiarity-based recognition, because it dissociates recognized from correctly rejected (CR) unstudied items, but is insensitive to recollection [10], [11]. Additionally, one recent study identified an early onsetting (100 ms) old–new effect, observed over frontopolar locations, that may also reflect familiarity [65]. However, to our knowledge, this latter effect has yet to be replicated or associated with familiarity-based recognition.

In contrast to early onsetting old–new effects, several late onsetting old–new effects have been proposed to be correlates of successful recollection (e.g. [19], [48], [49] for review). For example, many studies have identified a parietal maximal old–new effect occurring between 400 and 800 ms that has often been associated with recollection (see [30], [52] for reviews). The “parietal old–new” effect is sensitive to factors believed to influence recollection, such as depth of processing [53], [57], and is largest for items that elicited correct source [49], [63], [70], [71] or “remember” judgments [14], [60], [64]. However, it remains unclear whether the parietal old–new effect is purely reflective of recollection or a unitary retrieval process that varies in a graded fashion [14], [60], [63], [64], [70], [71].

Several ERP studies have shown that, in addition to retrieval, patterns of brain activity during encoding can differentiate items that will be subsequently correctly recognized from items that will subsequently be forgotten (e.g. [58] see also [30], [52] for reviews). This activity typically takes the form of an enhanced positivity for subsequently recognized compared to subsequently forgotten items. These ERP effects have been termed “differential neural activity due to memory” or “Dm” effects [45] and have been posited to reflect neural correlates of successful memory formation [16], [45].

If recollection and familiarity are supported by different types of representations, one would expect these types of memory to be associated with different patterns of Dm effects. Unfortunately, few studies have attempted to dissociate Dm effects at the time of encoding [20], [37], [60]. Although each of these studies [20], [37], [60] used the “remember–know” procedure [66] to differentiate items that were subsequently recollected from items that were subsequently recognized on the basis of familiarity, results from these studies were inconsistent. One study reported topographically widespread Dm effects for all subsequently recognized items between 200 and 900 ms but no differences were observed as a function of recollection or familiarity [60]. Another study found Dm effects for recollected (i.e. items that subsequently elicited a “remember” response), but not for familiar items (i.e. items that subsequently elicited a “know” response) between 400 and 1100 ms, maximal over left frontal sites [20]. A third study found what appeared to be distinct neural correlates of both recollection and familiarity Dm effects [37]. They showed that while a left temporal maximal negativity at 400 ms was correlated with subsequent familiarity, sustained bilateral frontal positivity between 1000 and 2000 ms predicted recollection. However, they did not report topographical analyses to indicate whether these effects were qualitatively distinct [40], [51]. Thus, it remains unclear whether distinct neural processes may support recollection and familiarity at the time of encoding.

In addition to determining whether recollection and familiarity would be associated with distinct patterns of activity, another objective of the present study was to characterize the nature of visual memory representation. Several behavioral studies of recognition memory have demonstrated that visual stimuli are better remembered if presented to the same rather than opposite visual hemifield at study and test (e.g. [3], [24]). In addition, one ERP study found that when abstract visual line patterns were presented laterally at encoding and centrally at retrieval, a greater physiological response, as measured by ERPs, was elicited over the hemisphere contralateral to initial stimulus presentation during encoding [24]. Together, these data suggest that visual memories may be organized in a contralateral fashion. If so, then it follows that recollection and familiarity effects for laterally presented stimuli might also be associated with contralaterally enhanced memory traces.

The present study, schematically depicted in Fig. 1, was designed to address the aforementioned issues. ERPs were recorded while participants studied, and subsequently retrieved from memory, photographs of concrete objects. In alternating blocks of study trials, subjects either performed animacy (“Is this living or nonliving?”) or manipulability (“Is this a manipulable object?”) judgments on laterally presented objects. During test blocks, a series of studied and unstudied foil objects were centrally presented and subjects made “remember–know–new” judgments on these objects. For items eliciting remember or know responses, subjects additionally made source decisions about which study block (animacy or manipulability) the object was encountered in. This allowed us to verify that remember and know responses were associated with recollection (contextual) and familiarity (a contextual) processes, respectively (see [72] for review). ERPs were then sorted by test responses for both encoding and retrieval phases.

We hypothesized that if recollection and familiarity-based recognition are supported by different types of neural representations [1], [2], [15], we would expect that they would be associated with qualitatively distinct patterns of neural activity at encoding and retrieval. In addition, we hypothesized that if visual memories are contralaterally organized for laterally presented objects, then contralateral enhancements of memory effects should be seen at encoding and retrieval.