Abstract
Several recent models of medial temporal lobe (MTL) function have proposed that the parahippocampal cortex processes context information, the perirhinal cortex processes item information, and the hippocampus binds together items and contexts. While evidence for a clear functional distinction between the perirhinal cortex and other regions within the MTL has been well supported, it has been less clear whether such a dissociation exists between the hippocampus and parahippocampal cortex. In the current study, we use a novel approach applying a functional magnetic resonance imaging adaptation paradigm to address these issues. During scanning, human subjects performed an incidental target detection task while viewing trial-unique sequentially presented pairs of natural scenes, each containing a single prominent object. We observed a striking double dissociation between the hippocampus and parahippocampal cortex, with the former showing a selective sensitivity to changes in the spatial relationship between objects and their background context and the latter engaged only by scene novelty. Our findings provide compelling support for the hypothesis that rapid item–context binding is a function of the hippocampus, rather than the parahippocampal cortex, with the former acting to detect relational novelty of this nature through its function as a match–mismatch detector.
Introduction
There is general agreement that the medial temporal lobe (MTL) is responsible for the rapid one-shot encoding of information about objects and the background context in which they occur, in the service of episodic memory (Scoville and Milner, 1957; O'Keefe and Nadel, 1978; Cohen and Eichenbaum, 1993; Aggleton and Brown, 1999; Nadel and Moscovitch, 2001; Squire et al., 2004; Morris, 2006). An important goal of current research, however, is defining the nature of information processing performed by regions within the MTL and establishing whether clear distinctions exist between the function of individual brain structures within this region (Lee et al., 2005; Aggleton and Brown, 2006; Davachi, 2006; Bussey and Saksida, 2007; Diana et al., 2007; Eichenbaum et al., 2007; Mayes et al., 2007).
One hypothesis that has risen in prominence in recent years, termed the binding of item and context model, argues that the perirhinal cortex processes item information, the parahippocampal cortex (PHc) supports contextual representations, and the hippocampus binds together items and contexts (Davachi, 2006; Diana et al., 2007; Eichenbaum et al., 2007). Recent evidence has provided considerable support for certain aspects of this theory, in particular demonstrating that the function of the perirhinal cortex and PHc/hippocampus may be most appropriately characterized in terms of a dissociation between the type of information represented (objects vs contexts), rather than purely in terms of processes supported (i.e., familiarity vs recollection). Less clear, however, is whether similar distinctions can be drawn between the PHc and hippocampus. Typically, both regions have been associated with successful source recognition judgments, such as remembering the color of the background in which an object is present (Diana et al., 2010), leaving open the question of whether these structures participate in the representation of item–context bindings, or the context itself. Based on current evidence, therefore, it remains uncertain how exactly the hippocampus and PHc contribute to representing the context in which events occur (Davachi, 2006; Diana et al., 2007; Eichenbaum et al., 2007).
Here we use a different approach to investigate these issues, drawing on recent ideas that the characteristics of novelty responses generated by individual regions within the MTL may provide a window into the nature of the representations and computations they support (Kumaran and Maguire, 2009), under incidental processing conditions mirroring the automatic creation of episodic memories in the real world. Our study therefore addressed three main questions: (1) Does the hippocampus support neural representations that mediate the binding of objects within the background context in which they are experienced? (2) Can this putative function of the hippocampus be dissociated from that of the PHc, with the latter playing a greater role in supporting representations of the overall scene? (3) Are hippocampal novelty responses generated in response to changes in object–contextual bindings consistent with the operation of an associative match–mismatch detector, or instead a familiarity-based mechanism?
Materials and Methods
Experiment 1: functional magnetic resonance imaging
Participants.
Twenty-two right-handed, healthy volunteers [11 males; age, 19–34 years; mean age, 25.3 years (SD, 3.9 years)] with normal or corrected-to-normal vision participated in this experiment. All subjects were free from neurological and psychiatric disease and gave informed written consent in accordance with the local research ethics committee. Two participants (one male, one female) were excluded from the study because of excessive head movement during scanning.
Stimuli.
A total of 289 color-picture stimuli, comprising unique object and background combinations, were used in this study. Photographs of objects were either obtained from the Hemera Photo-Objects image collection or from a variety of other sources on the Internet. These included birds, mammals, household objects, cars and other vehicles, but no faces or human bodies. The sizes of the objects ranged from 61 to 274 pixels in width and from 52 to 327 pixels in height. One hundred and thirty-nine objects were animate, and 150 were inanimate. Animacy was counterbalanced across our experimental conditions. Backgrounds were either photographed by the experimenter or obtained from various sources on the Internet. A wide variety of environments were used, such as offices, deserts, mountains, corridors, and warehouses. The number of indoor (n = 139) and outdoor (n = 150) backgrounds was counterbalanced across our experimental conditions. Selected backgrounds were 700 × 525 pixels in size, contained no salient objects, had an easily distinguishable depth dimension, and were unfamiliar (i.e., no famous landscapes were chosen). CorelDraw Graphics Suite X4 was used to combine objects with backgrounds by overlaying objects on backgrounds to form the stimuli for the experiment. Objects were paired with backgrounds so that they were contextually congruent (i.e., boats were positioned on water, and planes were in the sky), yet salient within. The horizontal position of each object in its background was constrained to one of three positions (central, left, or right), but the vertical position of the object varied. The combined objects and backgrounds formed our “scenes,” which were then inserted into frames that were 1024 × 768 pixels to create the “pictures” that comprised our stimulus set (Fig. 1A).
Stimulus design and presentation sequence. A, Each colored-picture stimulus comprised the conjunction of an object and background to form a scene, which was subsequently embedded within a grayscale frame. B, During scanning, picture stimuli were presented to subjects in pairs. Each picture within the pair was presented for 1500 ms, separated by a 250 ms interstimulus interval. An intertrial interval of 2000 ms separated each pair. During all intervals, a black screen, with a centrally presented white fixation cross, was shown. Subjects performed an incidental target (butterfly) detection task throughout the experiment, an example of which is shown in pair 2.
Experimental design and procedures.
Trials comprised pairs of sequentially presented pictures, each presented for 1500 ms, with an interstimulus interval of 250 ms (Fig. 1B). Each pair was separated by an intertrial interval of 2000 ms. During all intervals, a black screen, with a centrally positioned white fixation cross, was presented. To explore different types of novelty, we created a number of experimental conditions by manipulating the second picture presented.
For associative novelty, we manipulated the position of the object and the background independently to create four conditions (each 40 trials): (1) the object changed to a new position on the projection screen, moving horizontally left or right of where it was previously located (Object_move); (2) the background changed to a new position horizontally left or right of where it was previously located on the projection screen (Background_move); (3) the background and the object both changed to a new position, horizontally left or right of where they were previously located, with the each relocating in the opposite direction (Object&background_move); and (4) no movement of any element of the picture (No_move) (Fig. 2). In addition to our main experimental conditions, we also included 40 trials of a condition in which the object and background both moved in the same direction horizontally left or right of where they were previously located, such that the scene moved as a whole (Scene_move) (Fig. 2). For all conditions, an equal number of left and right object positions and background positions were used in each condition; therefore, the changes in the positions of both the objects and the backgrounds occurred equally toward the left and the right. The change in location of the object or the background was always by 210 pixels (30% of the background width, 4.3 visual degrees). None of the pictures presented as the first picture had an object located on the far left or far right of the projection screen. This was because it would have enabled subjects to predict that the next picture could not be an Object&background_move trial, given that it would have been impossible to move the object and background in opposite directions while keeping them both on the projection screen.
Experimental conditions. These were created by manipulating the second picture presented. One example picture (red buoy on sea shore) is shown to illustrate the experimental conditions. The position of the object (highlighted by the red triangle with the letter O below) and the background image (highlighted by the black triangle with the letter B below) were manipulated independently to create five conditions: (1) there was no movement of any element of the picture (No_move); (2) the object changed to a new position on the projection screen, moving horizontally left or right of where it was previously located (Object_move); (3) the background changed to a new position horizontally left or right of where it was previously located on the projection screen (Background_move); (4) the background and the object both changed to a new position, horizontally left or right of where they were previously located, with the each relocating in the opposite direction (Object&background_move); or (5) the whole scene (the object and background together) moves horizontally left or right of where it was previously located (Scene_move). In the final condition illustrated here, a completely new object and background was presented as the second picture (Novel_scene). The Repeat_scene condition (in which a previously seen scene was presented multiple times) is not shown. For each of the conditions, the table displayed indicates whether the object, the background, both, or neither moved in picture 2 relative to the picture 1. Arrows illustrate the direction of movement in the examples shown.
To explore scene novelty, we created a condition in which a completely new object and background was presented as the second picture (Novel_scene) (Fig. 2). We also created a condition in which novelty was diminished by presenting subjects with a familiar scene (Repeat_scene). These trials were identical to the No_move condition except that the pictures used had been presented in previous trials and initially once outside the scanner during practice. Two pictures were used for each subject, and the object was always presented in the middle of each background. There were 20 trials of the Novel_scene and Repeat_scene conditions. Also, for each of these conditions, the background was presented on the left of the screen in half of the trials, and for the other half it was presented on the right.
Finally, we also included target pictures (24 trials) as part of our incidental task (see below). These pictures contained a butterfly, with each picture containing a different butterfly on a unique background. Half of the target pictures were presented as the first picture in a trial, and the other half was presented as the second picture. If presented first, target pictures were followed by the fixation screen. If presented second, target pictures were preceded by a pretarget picture (Fig. 1B). Target and pretarget pictures were equally likely to appear on the left or the right of the screen.
Task during scanning.
Participants were informed that they were to complete a target detection task, for which they would be rewarded by extra financial compensation according to their accuracy. They were not informed about the nature of changes in any of the other experimental trial types. When participants encountered a target picture containing a butterfly, they were required to press a button with their right index finger, which, during the scanning phase, was recorded for analysis. Before scanning, a training session was conducted to ensure that participants understood the task requirements and that they were preexposed to all possible trial types. A total of 19 trials were viewed, and these were identical for all participants.
Scanning was conducted in two sessions. Stimulus presentation commenced after 6 functional “dummy” scans had been taken, and stimulus presentation was resynchronized with the scanner periodically throughout each session. Trials were presented in a subject-specific, pseudorandom order with the constraint that no more than two trials of the same type were viewed consecutively and target trials had to occur at least once in every 30 trials. The study was a within-subjects design. Across both sessions, participants viewed a total of 264 stimulus trials (132 per session). For each trial type, the total number of viewings was divided equally between the two sessions. Sixty null trials (30 per session) were also included to improve the experimental design efficiency. In these null trials, the participant viewed the fixation screen for the duration of the entire trial (3250 ms). MATLAB 7.5 and the Cogent 2000 version 1.28 Toolbox (http://www.vislab.ucl.ac.uk/cogent_2000.php) were used to control stimulus presentation, interact with the scanner, and record response data.
Postscan memory task.
Immediately after scanning, participants completed a surprise task testing background image familiarity and cued object recall. In this task, all the backgrounds viewed during scanning were re-presented via a laptop (screen size, 12 inch). The backgrounds (700 × 525 pixels in size) were presented centrally on a white background screen, without the object and their outer frame present. The two backgrounds used in the Repeat_scene condition were always presented first, but the remaining backgrounds were presented in a fully randomized order. Participants were first asked whether they thought the background image was familiar. They were then asked to recall the object that had been previously presented with the background. The correct object was then shown to the participant on a separate laptop computer (screen size also 12 inches), and they were required to indicate, using a mouse click, where they thought the object had been located on the background. MATLAB version 7.5 and the Cogent2000 version 1.28 Toolbox were used to control stimulus presentation. The position of the two laptop computers used in this part of the study was counterbalanced such that for half the participants, the backgrounds were presented on the left laptop and the objects on the right, and vice versa for the other half of subjects. Statistics on behavioral data was conducted with SPSS.
Functional magnetic resonance imaging parameters and acquisition.
Participants were scanned at the Birkbeck-UCL Neuroimaging Centre using a 1.5 T Siemens (Siemens Medical Systems) Avanto magnetic resonance imaging (MRI) scanner, with a 32-channel head coil. The experimental task was performed over two sessions, each lasting 16 min, 14 s. A total of 644 functional scans were acquired in each session using a gradient-echo echoplanar imaging (EPI) sequence (repetition time, 3000 ms; echo time, 48 ms; 205 × 205 field of view; 64 × 64 matrix). In each volume, 36 oblique axial slices, approximately parallel to the hippocampus and 3.2 mm thick, were acquired. After this, a high-resolution T1 structural scan was acquired (magnetization prepared rapid gradient echo; 176 slices; 1 × 1 × 1 mm resolution). Foam padding was used to minimize head motions, and ear plugs were used to dampen the noise of the scanner. Stimuli were projected centrally onto a screen at the front of the magnet, which participants viewed using a mirror mounted on the head coil (21 × 13° of visual angle of the whole screen).
Functional MRI preprocessing and statistical analysis.
The first six functional volumes of each session (dummy scans) were discarded to permit T1 equilibrium. Initial inspection of the structural images revealed that two participants moved excessively during their scans, and these were excluded before preprocessing. Statistical parametric mapping (SPM8; http://www.fil.ion.ucl.ac.uk/spm/software/spm8/) was used for spatial preprocessing and subsequent analyses. Images were spatially realigned to the first volume of the first session to correct for motion artifacts, normalized to a standard EPI template in Montreal Neurological Institute (MNI) space with a resampled voxel size of 3 × 3 × 3 mm, and smoothed with an 8 mm full-width at half-maximum Gaussian kernel filter.
After preprocessing, the smoothed, normalized functional imaging data were entered into a voxel-wise subject-specific general linear model (i.e., the first-level design matrix) with seven regressors of interest: No_move, Object_move, Background_move, Object&background_move, Scene_move, Novel_scene, Repeat_scene. One regressor of no interest, coding target trials, was included. Each of the regressors was then convolved with the canonical hemodynamic response function. Furthermore, six subject-specific movement parameters (derived from the realignment phase of preprocessing) were also included as regressors of no interest in each model. A high-pass filter with a cutoff of 128 s was used to remove low-frequency drifts. Temporal autocorrelation was modeled using an AR(1) process. At the first level, linear weighted contrasts were used to identify effects of interest, providing contrast images for group effects analyzed at the second (random-effects) level. The MarsBar SPM toolbox (http://marsbar.sourceforge.net/) was used to extract mean responses from single subjects using defined regions of interest (10 mm spheres) in the left hippocampus (Kumaran and Maguire, 2006) and bilateral parahippocampus (Goh et al., 2004).
Given our a priori anatomical hypotheses, we report activations in the hippocampus and PHc at a threshold of p < 0.005 (uncorrected for multiple comparisons) and a minimum of 10 contiguous voxels. For these regions, we also used a small volume correction (SVC) (10 mm sphere) located at specific MNI coordinates on the basis of previous studies. These were for the left hippocampus [−27, −18, −18 (Kumaran and Maguire, 2006)] and parahippocampus [29, −35, −7 (Goh et al., 2004)]. We report activations outside these regions at a threshold of p < 0.05 corrected for whole-brain volume.
Experiment 2: eye tracking
Because our functional MRI (fMRI) experiment involved manipulating object positions and presenting novel spatial scenes, we reasoned that our experimental manipulations might give rise to systematic differences in eye movements between our conditions. Such differences would mean that differences in the fMRI signal between such conditions would have to be interpreted in relation to both stimuli effects and eye-movement patterns. To investigate this possibility, eye-tracking data were collected during the presentation of the same experimental stimuli and task with a separate, naive participant group.
Participants.
A separate group of 16 naive, right-handed, healthy volunteers [nine males; age, 19–35 years; mean age, 25.0 years (SD, 4.6 years)] with normal or corrected-to-normal vision participated in this experiment. All subjects were free from neurological and psychiatric disease and gave informed written consent in accordance with the local research ethics committee.
Stimuli and experimental conditions.
The same stimulus set and experimental conditions used in experiment 1 were used.
Experimental task, procedures, and design.
The same task used in experiment 1 was used. All aspects of the timing of stimuli presentation were identical to that in experiment 1, except that in the middle of each 2000 ms intertrial interval there was a drift correction, where participants were required to focus their gaze on a white circle located in the center of a black screen.
Before the collection of eye-tracking data, the same training session used in experiment 1 was given. To mimic the scanning protocol as closely as possible the eye-tracking experiment was conducted in two sessions, each lasting ∼20 min and separated by a 5 min rest period. The initiation of each session was contingent on the successful completion of a nine-point calibration procedure. Furthermore, the eye tracker was recalibrated every 54th trial (twice during each session) to maintain the accuracy of the eye tracker. Trials were presented in a subject-specific, pseudorandom order, with the constraint that no more than two trials of the same type were viewed consecutively and the different types of trial were distributed evenly across the two sessions. The same total number of trials and of each type of trial used in experiment 1 was used in this study.
Eye-tracking data collection and analysis.
For both the main experimental session and the practice session, stimuli were presented on a 20 inch computer monitor with a resolution of 1920 × 1200 pixels and a refresh rate of 60 Hz. Participants were seated in a dimly lit room at a viewing distance of ∼113 cm from the monitor, with a viewing angle of 26°. Eye movements were monitored at the UCL Multimodal Lab using a head-mounted EyeLink (EyeLink II; SR Research) eye tracker, tracking the right pupil only, with a temporal resolution of 500 Hz and a spatial resolution of 0.5°. Experiment Builder (SR Research) was used to present stimuli, synchronize with the eye tracker, and record data, including the recording of eye position throughout each experimental trial. Only data collected during presentation of the second picture was analyzed. Because the interpicture interval was 250 ms, it did not provide a sufficient duration for a new saccade to be initiated. Therefore, to remove noise generated by lingering fixations from the first picture, the first 350 ms of the presentation of the second picture was removed from the analysis. A total of 350 ms was chosen because it corresponds to the average time needed to execute a saccade (Castelhano and Henderson, 2008). Three analyses were applied. The first analysis examined two measures across all conditions: (1) the mean total number of fixations and (2) mean saccade amplitude (in degrees of visual angle). The second analysis examined whether fixations were located within the region of (1) the object (a rectangular box 120% the size of the object) or (2) the background (the remainder of the scene after accounting for the region of the prestimulus fixation cross) (Fig. 3). The third analysis was restricted to those conditions in which the object was moved relative to the projection screen (Object_move, Object&background_move, and Scene_move) and examined the number of fixations on (1) the current object region, (2) the region previously occupied by the object, and (3) the remaining background (Fig. 3).
Eye-tracking analysis. The regions of interest (rectangular boxes) used for data analysis were as follows: (1) the current object position; (2) the position of prestimulus fixation cross; (3) the position previously occupied by the object for conditions in which it had moved relative to the projection screen; and (4) the remainder of the scene. In this example, the elephant was previously positioned on the right side of the scene on the first presentation of this picture. Fixations (overlaid circles) are shown, the size of which corresponds to their duration (i.e., larger circles indicate longer durations). For this particular trial, three fixations were recorded on the current object region and accounted for 54.4% of the total viewing duration, and the remaining two fixations (45.6% of duration) occurred on the background region.
Results
Experiment 1: fMRI
Behavioral results
Participants performed the incidental target detection task during scanning with 96.0% accuracy (SD, 9.2%) and a mean reaction time of 819 ms (SD, 118.4 ms). Target trials after pretarget scenes could be classified in different conditions depending on whether or not the butterfly and its background were presented in the same positions or moved to new positions on the screen. A one-way ANOVA revealed no significant differences in the number of errors (F(3,57) = 0.24; p > 0.05) or reaction time (F(3,57) = 0.99; p > 0.05) across these categories.
Neuroimaging data
We examined the response of the hippocampus and PHc to changes in the spatial relationship between objects and backgrounds and to scene novelty. Our results revealed a double dissociation between brain regions, with the left hippocampus selectively responsive to changes in the spatial relationship between the object and the background context, and the PHc selectively responsive to scene novelty. These regions survived a threshold of p < 0.005 uncorrected and a small volume correction at a threshold of p < 0.05 corrected (see Materials and Methods for details). Notably, the perirhinal cortex showed no response in any of our contrasts. For completeness, all MTL regions active in our contrasts at a threshold of p < 0.005 uncorrected and all other regions, outside of the MTL, active in our contrasts at a threshold of p < 0.001 uncorrected are available in Table 1.
MNI coordinates for peak voxels that showed a significant response in contrasts of interest
Left hippocampal novelty responses to changes in the spatial relationship between object and background context
To examine the effect of changes in the spatial relationship between objects and backgrounds, we first contrasted activity in the Object_move condition with the No_move condition (i.e., Object_move > No_move). This revealed significant activity in the left hippocampus (Fig. 4A). Importantly, this finding cannot reflect a response to stimulus novelty, given that the objects and backgrounds were equally familiar (i.e., had been experienced once before during the first presentation) in both Object_move and No_move conditions. In addition, no significant difference in hippocampal activity was observed in a setting when the entire scene itself moved on the projection screen, an additional (Scene_move) condition included outside the factorial design proper, indicating that changes in object–background context relationships drive hippocampal novelty signals in our paradigm.
Hippocampal novelty responses to changes in the spatial relationship between object and background context are maximal under conditions of associative match–mismatch. A, Increased activity was observed in the left anterior hippocampus when the Object_move and No_move conditions were contrasted (Object_move > No_move). Coronal and axial sections at the peak level for this contrast are displayed (x, y, z = −24, −13, −20; t = 3.60). Threshold is set at p < 0.005 (uncorrected for multiple comparisons). R, Right side of brain. Activation in the hippocampus is significant at p < 0.005 (uncorrected for multiple comparisons); the cluster size is >10 contiguous voxels and also survives SVC at a threshold p < 0.05 (corrected) (see Materials and Methods). B, Increased activity was also observed in the left anterior hippocampus when the Object_move and Object&background_move conditions were contrasted (Object_move > Object&background_move). Coronal and axial sections at the peak level for this contrast are displayed (x, y, z = −21, −16, −20; t = 4.49). Threshold is set at p < 0.005 (uncorrected for multiple comparisons). Activation in the hippocampus is significant at p < 0.005 (uncorrected for multiple comparisons); the cluster size is >10 contiguous voxels and also survives SVC at a threshold p < 0.05 (corrected). C, Increased activity observed in the left anterior hippocampus for the interaction contrast [(Object_move + Background_move) > (Object&background_move + No_move)] provides further evidence that this region operates as an associative match–mismatch detector. Coronal and axial sections at the peak level for this contrast are displayed (x, y, z = −15, −22, −23; t = 4.44). Threshold is set at p < 0.005 (uncorrected for multiple comparisons). Activation in the hippocampus is significant at p < 0.005 (uncorrected for multiple comparisons); the cluster size is >10 contiguous voxels and also survives SVC at a threshold p < 0.05 (corrected). All peak coordinates are reported in MNI space. D, Condition-specific parameter estimates (β) in arbitrary units at the peak voxel in left anterior hippocampus shown in C. Gray bars are the conditions used in the interaction contrast [(Object_move + Background_move) > (Object&background_move + No_move)].
While the presence of hippocampal novelty responses to changes in the relationship between items and their background contexts (i.e., Object_move > No_move) provides evidence that this region represents this kind of information, the pattern of responses observed offers clues to the underlying computations (Kumaran and Maguire, 2009). In this respect, it is illuminating that significant activation in the left hippocampus was also observed when the Object_move condition was contrasted with the Object&background_move condition (i.e., Object_move > Object&background_move) (Fig. 4B), indicating that the response of this region was modulated depending on whether the background had either moved or remained static. As such, hippocampal novelty responses were found to be maximal when the spatial relationship between objects and their background was altered to a moderate amount (i.e., Object_move), rather than a larger degree (i.e., Object&background_move) favoring the operation of an associative match–mismatch, rather than a familiarity-based, mechanism (Sokolov, 1963; Gray, 1982; Levy, 1989; Hasselmo and Schnell, 1994; Knight, 1996; Lisman and Otmakhova, 2001; Moser and Paulsen, 2001; Vinogradova, 2001; Fyhn et al., 2002; Kumaran and Maguire, 2007a,b, 2009). Critically, associative match–mismatch models predict that hippocampal mismatch signals are generated whenever novel input patterns contain a novel (i.e., mismatch) and familiar (i.e., match) associative component (i.e., as in the Object_move condition) (Kumaran and Maguire, 2007b, 2009). Additional evidence consistent with the operation of such a match–mismatch mechanism comes from examining the interaction contrast [i.e., (Object_move + Background_move) > (Object&background_move + No_move)] that contrasts conditions containing match–mismatch with conditions containing only pure match or pure mismatch. This replicates the interaction contrast used by Kumaran and Maguire (2007b) to examine match–mismatch in their previous study. This contrast revealed significant effects in the left hippocampus (Fig. 4C). Together, therefore, these results suggest that the hippocampus supports representations that bind together the items and the contexts within which they occur, acting to detect changes in such relationships through its function as an associative match–mismatch detector. Importantly, no significant effects were observed in the PHc in any of the aforementioned contrasts, suggesting that a putative item–contextual binding function is specific to the hippocampus itself.
The PHc responds to scene novelty
Given that no significant effects were observed in the PHc in relation to changes in object–background context relationships, we next examined potential effects of scene novelty (i.e., Novel_scene > Repeat_scene). Significant activations were found only in the bilateral PHc (Fig. 5A) but not in the hippocampus. We also found the same PHc region was significantly more active when the Novel_scene condition was contrasted with the No_move condition (i.e., Novel_scene > No_move) (L PHc: x = −33, y = −31, z = −23, t = 5.59; R PHc: x = 36, y = −34, z = −20, t = 5.96).
Parahippocampal cortex responds to scene novelty. A, Increased activity was observed bilaterally in the PHc when the Novel_scene and Repeat_scene conditions were contrasted (Novel_scene > Repeat_scene). Coronal and axial sections at the peak levels for this contrast are displayed (right: x, y, z = 33, −37, −17; t = 5.22; left: x, y, z = −33, −37, −23; t = 3.96). Threshold is set at p < 0.005 (uncorrected for multiple comparisons). Activation in the PHc is significant at p < 0.005 (uncorrected for multiple comparisons); the cluster size is >10 contiguous voxels and also survives SVC at a threshold p < 0.05 (corrected). Peak coordinates are reported in MNI space. B, Condition-specific parameter estimates (β) in arbitrary units at the peak voxel in the right PHc. Gray bars are the conditions used in the scene novelty contrast (Novel_scene > Repeat_scene).
Dissociation between the hippocampus and PHc
Given the differential pattern of responses found in the hippocampus and PHc to object–background context and scene novelty, respectively, we next examined the evidence for a more formal dissociation between their response patterns. To consider this issue, we asked whether there was a statistically reliable brain region × contrast (object–background novelty and scene–novelty) interaction. This was done by extracting each subject's mean response from the predefined anatomical loci, in relation to the two relevant contrasts: object–background novelty [i.e., (Object_move + Background_move) > (Object&background_move + No_move)] and scene–novelty (Novel_scene > Repeat_scene) (see Materials and Methods). A 2 × 2 repeated-measures ANOVA yielded a significant brain region × contrast interaction (F(1,19) = 4.66; p < 0.05) and no significant main effects for brain region or contrast.
Responses are independent of subsequent familiarity or recall
Activity in the MTL has been associated with subsequent memory effects (Brewer et al., 1998). We therefore tested whether our novelty responses were related to, or independent of, postscan familiarity for the scenes or recall of the objects. After scan, participants classified an average of 34.7% (SD, 17.7%) of backgrounds as familiar and were able to correctly recall an average of 5.9% (SD, 3.54%) of objects (Table 2). These familiarity and recall scores were significantly positively correlated (r = 0.66; p < 0.005). We found that the number of backgrounds judged familiar by subjects did not differ statistically across our conditions (F(5,90) = 2.28; p > 0.5), but the number of objects recalled did (F(5,95) = 3.68; p = 0.04). Post hoc tests revealed that this difference was driven by significantly more objects recalled from the Object_move condition being remembered than the Novel_scene condition (p < 0.05). Additional statistical comparisons following the logic of the scene–novelty and match–mismatch contrasts found no differences between conditions [Novel_scene vs No_move: t(19) = 1.72, p = 0.10; (Object_move + Background_move) vs (Object&background_move + No_move): t(19)= −0.403, p = 0.69]. Thus, we find no evidence that our MTL responses were related to subsequent familiarity or the cued recall of objects.
Measures of subsequent postscan background image familiarity and cued object recall
Experiment 2: eye tracking
Behavioral results
Subjects performed at 94.8% (SD, 5.7%) accuracy on the target detection task. Their accuracy did not differ significantly from the fMRI participant group in experiment 1 (t(34) = 0.53; p = 0.60).
Eye-tracking results
Eye-tracking data were analyzed using t tests, which replicated the same statistical comparisons used in our fMRI design. We found no significant differences in the mean total number of fixations or mean saccade amplitude for any comparison (p > 0.05). This was the case for our analysis of both the whole scene and the region-of-interest analysis (see Materials and Methods). Thus, it is unlikely our fMRI results reported above were driven by consistent differences in the number of fixations or saccade amplitudes across subjects. For further scrutiny of the data, one-way repeated-measures ANOVAs and post hoc corrected tests were also conducted on the data (see Table 3 for details).
Measures of eye movements
Discussion
We used an fMRI repetition paradigm to characterize the contributions of different regions within the MTL to representing scenes and the object–background relationships within them. Our results demonstrate a double dissociation between the hippocampus and PHc, with the former engaged specifically by changes in the spatial relationship between objects and their background context and the latter selectively by scene novelty. Together, our findings provide compelling support for recent proposals (Davachi, 2006; Diana et al., 2007; Eichenbaum et al., 2007) in which the PHc is viewed to support neural representations of the global context within which events occur, with the hippocampus playing a specific role in the rapid creation of item–context bindings.
A key finding in this study was that hippocampal novelty signals were observed selectively in relation to changes in the spatial relationship between objects and their background. In contrast, the design of a previous adaptation study did not permit definite conclusions to be drawn about whether the activation of hippocampus, and PHc, was driven by relational novelty of this nature, or instead by stimulus (i.e., object or scene) novelty (Goh et al., 2004). More specifically, in our study hippocampal activity was not observed in relation to scene novelty, nor could it be accounted for solely by changes in the egocentric position of objects on the screen, given that no significant effects were observed in a condition in which both object and background moved (i.e., Object&background_move). Furthermore, a second behavioral eye-tracking experiment confirmed that the relevant experimental conditions did not differ significantly in terms of the average number of fixations or saccade amplitudes, rendering it unlikely that eye movement differences account for the observed pattern of novelty responses. Our data therefore provide evidence for the hypothesis that this region plays a specific role in sustaining neural representations that bind together objects and the context in which they occur (Davachi, 2006; Diana et al., 2007; Eichenbaum et al., 2007), a function that sits comfortably with ideas concerning its role in relational memory (Cohen and Eichenbaum, 1993; Eichenbaum, 2004; Hannula et al., 2006) and spatial mapping (O'Keefe and Nadel, 1978).
Our experiment, in addition to providing constraints concerning the nature of information represented within MTL regions, also affords insights into the type of computations performed by the hippocampus. As has been argued previously (Kumaran and Maguire, 2009), the presence of hippocampal novelty responses to a given change in sensory input (i.e., object–scene relationships) provides evidence concerning the type of information represented, whereas the pattern of novelty responses observed can be used to make inferences about underlying computations. Specifically, in our study, hippocampal novelty responses were observed in response to moderate changes (i.e., Object_move condition), but not large (i.e., Object&background_move condition) changes, in the spatial relationship between object and background. This pattern of responses and previous work (Kumaran and Maguire, 2006; 2007b) argues strongly against the operation of a familiarity mechanism within the hippocampus (Gillund and Shiffrin, 1984; Hintzman, 2001), which predicts that the condition with the greatest change in object–background relationship (i.e., Object&background_move) would be associated with maximal hippocampal activation. Instead, our data support theoretical predictions that the hippocampus operates as an associative match–mismatch detector (Sokolov, 1963; Gray, 1982; Levy, 1989; Hasselmo and Schnell, 1994; Knight, 1996; Lisman and Otmakhova, 2001; Moser and Paulsen, 2001; Vinogradova, 2001; Fyhn et al., 2002; Kumaran and Maguire, 2009), a function perhaps subserved by the CA1 subregion based on receipt of dual inputs concerning both the external sensory environment (arriving from the entorhinal cortex) and past experience (from the CA3 subregion). Critically, according to such models, novelty signals are viewed to be contingent on the initial recall of stored information perhaps through an autoassociative mechanism (e.g., pattern completion), a process that only occurs under conditions (i.e., Object_move) when novel sensory inputs overlap sufficiently with existing representations (i.e., relating to the first presentation) (Kumaran and Maguire, 2009; Norman and O'Reilly, 2003).
Our finding that left anterior hippocampal activity was greater during match–mismatch conditions than either match or mismatch conditions is in close agreement with previous fMRI studies examining match–mismatch using the repetition of quartets of sequentially presented objects (Kumaran and Maguire, 2006, 2007b). In each case, activity in a left hippocampal region situated in close anatomical proximity to the area identified in our study was observed to be maximal under conditions of associative match–mismatch. Based on previous data, one might conclude that hippocampal mismatch signals are generated primarily in relation to the sequential presentation of multiple items, a line of reasoning that accords with work implicating a left lateralized role in the temporal processing of episodes (Maguire, 2001; Spiers et al., 2001a,b) (but see Lehn et al., 2009). Our findings, however, provide evidence that the left hippocampus plays a more general role in match–mismatch processing, both in the temporal and spatial dimensions, in this case relating to changes between objects and their backgrounds.
Our results demonstrate that the PHc exhibited scene novelty responses and was not sensitive to changes in the spatial relationships between objects and the background in which they are presented, an opposite pattern to that observed in the hippocampus. Our data are consistent with the notion that the PHc supports neural representation of the global context of episodes (Bar and Aminoff, 2003; Davachi, 2006; Diana et al., 2007; Eichenbaum et al., 2007) and also with models that argue the PHc represents the local layout of an environment (Epstein and Kanwisher, 1998; Epstein et al., 2005; Byrne et al., 2007; Epstein, 2008). However, it is also important to bear in mind that the present findings are consistent with PHc responses being driven by stimulus novelty, given that both objects and background context were new in the Novel_scene condition, although, notably, the PHc has been found to be less responsive to changes in object identify than scene identity or scene viewpoint (Epstein et al., 2003). It is also interesting to note that previous studies have observed that the PHc is engaged by object–location novelty, albeit typically in the setting of tasks involving explicit recognition memory judgements (Düzel et al., 2003; Köhler et al., 2005) or prefamiliarized object arrays (Pihlajamäki et al., 2004). Furthermore, the PHc has been implicated in associating objects with navigational relevance (Janzen and van Turennout, 2004; Janzen and Weststeijin, 2007; Janzen et al., 2008). Together, therefore, the current study provides evidence that the rapid, automatic creation of neural representations that bind objects to their backgrounds is a specific function of the hippocampus, rather than the PHc. It will be important, however, for future studies to clarify the exact nature of representations within the PHc, an issue that we believe fMRI adaptation techniques are ideally suited to address.
The neural mechanisms supporting the one-shot creation of episodic memories, and the contribution of individual regions within the MTL, remains a fundamental issue in memory research. Previous evidence, garnered mostly from explicit recognition memory paradigms (Diana et al., 2007, Eichenbaum et al., 2007), has provided support for the role of the hippocampus and PHc in binding events within their context but found it difficult to isolate the individual contributions of these regions. The current study used a novel approach to address this question, leveraging novelty responses to provide insights into the nature of information processing performed by different MTL regions. In doing so, we provide evidence that item–contextual binding is a specific function of the hippocampus, with this region acting to detect changes in such relational encodings through its function as an associative match–mismatch detector.
Footnotes
-
This work was supported by a Biotechnology and Biological Sciences Research Council studentship to L.R.H., a Wellcome Trust Advanced Training Fellowship to H.J.S., and a Wellcome Trust Intermediate Clinical Fellowship to D.K. We thank Marty Sereno for help optimizing the scanning sequences, Robin Thompson for advice on eye-tracking analysis, Emily Collins for help with programming and eye-tracking data collection, Robin Hayman for help with programming, and Qing Ruan for help with behavioral data collection.
- Correspondence should be addressed to Dr. Hugo J. Spiers, Institute of Behavioural Neuroscience, Research Department of Cognitive, Perceptual, and Brain Sciences, Division of Psychology and Language Sciences, University College London, London WC1H 0AP, UK. h.spiers{at}ucl.ac.uk
