Abstract
Learning sequences of events (e.g., a-b-c) is conceptually a simple problem that can be solved using asymmetrically linked cell assemblies [e.g., “phase sequences” (Hebb, 1949)], provided that the elements of the sequence are unique. When elements repeat within the sequence, however (e.g., a-b-c-d-b-e), the same element belongs to two separate “contexts,” and a more complex sequence encoding mechanism is required to differentiate between the two contexts. Some neural structure must form sequential-context-dependent, or “differential,” representations of the two contexts (i.e., b as an element of “a-b-c” as opposed to “d-b-e”) to allow the correct choice to be made after the repeated element. To investigate the possible role of hippocampus in complex sequence encoding, rats were trained to remember repeated-location sequences under three conditions: (1) reward was given at each location; (2) during training, moveable barriers were placed at the entry and exit of the repeated segment to direct the rat and were removed once the sequence was learned; and (3) reward was withheld at the entry and exit of the repeated segment. In the first condition, hippocampal ensemble activity did not differentiate the sequential context of the repeated segment, indicating that complex sequences with repeated segments can be learned without differential encoding within the hippocampus. Differential hippocampal encoding was observed, however, under the latter two conditions, suggesting that long-term memory for discriminative cues present only during training, working memory of the most recently visited reinforcement sites, or anticipation of the subsequent reinforcement site can separate hippocampal activity patterns at the same location.
Introduction
Sequential-context-dependent representations that could aid in learning complex sequences might be formed in the hippocampus because of its propensity, under some conditions, to form distinct patterns for similar inputs. Such “pattern separation” (Marr, 1969), or “orthogonalization,” enables discrimination between inputs that differ only slightly, for example, when current sensory inputs are constant but internal, contextual variables (e.g., hunger, task demands, contents of working, or long-term memory, etc.) differ. Numerous empirical studies have shown that pattern separation is a basic feature of hippocampal dynamics (Quirk et al., 1990; Bostock et al., 1991; Knierim et al., 1995; Markus et al., 1995; Skaggs and McNaughton, 1998; Tanila, 1999; Lever et al., 2002; Wirth et al., 2003).
A direct role for the hippocampus in encoding sequences of places or events has been proposed on theoretical grounds (McNaughton and Morris, 1987; Treves and Rolls, 1991; Abbott and Blum, 1996; Levy, 1996; Touretzky and Redish, 1996; Wallenstein et al., 1998; Lisman, 1999; Sato and Yamaguchi, 2003). Empirical support for such a role includes the observations that hippocampal lesions impair the ability of rats to recognize different sequences of odors (Dusek and Eichenbaum, 1997; Agster et al., 2002; Fortin et al., 2002; Kesner et al., 2002) (but see Van Elzakker et al., 2003) and that hippocampal place fields expand asymmetrically in an experience- and NMDA receptor-dependent manner during repeated route following (Mehta et al., 1997; Ekstrom et al., 2001), as predicted by models of the formation of sequential associations.
Recently, several empirical studies have addressed the issue of whether the hippocampus creates differential representations when external cues are held constant but internal cues vary. For sequences containing repeated spatial segments, Wood et al. (2000) found differential activity in the hippocampus on the common segment of a T-maze (modified to allow return to the base of the T), although all external variables were held constant once the task had been learned. A similar result was obtained by Frank et al. (2000) and Ferbinteanu and Shapiro (2003). In contrast, however, Lenck-Santini et al. (2001) failed to observe differential hippocampal activity during learned spatial alternation on a Y-maze.
The two principal questions addressed here are as follows: (1) is the differentiation of sequential context within the hippocampus necessary for the solution of repeated-element sequence learning problems, and (2) what are the factors that underlie the discrepancy in the literature in which hippocampal differentiation of sequential-context has been observed in some cases but not others? The first experiment was similar in design to previous repeated-element sequence studies but used a more complex sequence. In this case, there was no evidence of hippocampal differentiation along the repeated element. The next two experiments addressed procedural variables (i.e., reward locations and training protocols) that may have contributed to the observation of differential activity in the hippocampus in the case of Wood et al. (2000) but failed to produce differential activity in other cases (Lenck-Santini et al., 2001).
Parts of this work have been published previously in abstract form (Bower et al., 2001).
Materials and Methods
Subjects and pretraining
Three male Brown Norway/Fisher 344 hybrid rats (6-9 months old, 250-350 g) were housed in Plexiglas home cages, maintained on a reversed 24 h light/dark cycle, and food deprived to 80% of their ad libitum weight. Training and recording sessions occurred during the dark portion of this cycle. The rats were trained to find food pellets at one of eight equally spaced reward zones on the edge of a 1.3-m-diameter circular arena. Reward zones were marked by wooden clothespins attached to the edge of the arena, each supporting a light emitting diode (LED). Each LED was located ∼5 cm above the table surface. A training session lasted ∼30 min and comprised a number of separate trials. Each trial began with two, simultaneously presented cues: a nondirectional 4 kHz tone that signaled the availability of reward somewhere in the arena, and the illumination of one LED that marked the correct reward zone. To enhance the salience of the visual cue, the LED marking the correct reward zone flashed at 1 Hz. Rats were trained to run for food reward to the vicinity of the correct reward zone (within 10 cm), the food being placed on the maze only after the rat arrived in the correct zone. This training continued until each rat made direct trajectories to reward locations and >50 rewards were received in a training session. During this phase, the reward sites were presented pseudorandomly.
Surgery
National Institutes of Health guidelines and approved Institutional Animal Care and Use Committee protocols were followed for all surgical procedures. Each rat was anesthetized with Nembutal (sodium pentobarbital, 40 mg/kg; Abbott Laboratories, Abbott Park, IL), placed in a stereotaxic holder, and injected with Bicillin (6000 IU/hindleg, i.m.; Wyeth Laboratories, Madison, NJ). The skull was cleared of skin and fascia, and craniotomies were opened for two stimulating electrodes targeting the medial forebrain bundle (MFB) (+0.25 mm anteroposterior, 1.9 mm mediolateral bilaterally, 8.5 mm ventral, projecting 19.5° caudal) and a hyperdrive containing 12 tetrodes (±2.0 mm mediolateral, -4.0 mm anteroposterior) (Gothard et al., 1996). Each stimulating electrode consisted of two, Teflon-coated, stainless-steel wires (coated diameter, 0.0045 inches; part number 316SS3T; Medwire, Mt. Vernon, NY) twisted together with ∼1 mm of insulation removed from one tip. Each tetrode consisted of four, polyimide-coated, nichrome wires (diameter, 14 μm; Kanthal Palm Coast, Palm Coast, FL) twisted together. The hyperdrive targeted the right hippocampus in rats 2 and 3 and the left hippocampus in rat 1. No apparent hemispheric differences were noted, and so all analyses were pooled. Rats were allowed to recover for 1 week after surgery. After surgery, rats were returned to ad libitum feeding for the remainder of the experiment.
MFB stimulation
After surgery, rats were trained on the same task involving randomly presented reward zones, except that food reward was replaced by electrical stimulation of the MFB as the reinforcing reward (Olds and Milner, 1954) (for review and training techniques, see Liebman and Cooper, 1989; Kobayashi et al., 1997). Optimal stimulation parameters were obtained using an operant conditioning chamber equipped to deliver MFB reward stimulation when the rat performed an operant response (nose poke or lever press). Optimal MFB stimulation consisted of a train of 100 μs width, 70-110 μA, diphasic current pulses, delivered at 150 Hz for 300 ms. Rats were trained to run to goals presented in random order for electrical stimulation reward until their performance approximately matched that observed for food reward before surgery.
Recording apparatus and protocol
During recording sessions, the hyperdrive was connected to a recording head stage (Neuralynx, Tucson, AZ) that enabled low-noise transmission of neural data to the recording system and connection from a stimulation isolation unit (SIU) (A365D-A; World Precision Instruments, Sarasota, FL) to the MFB stimulating electrodes. The head stage also contained an array of LEDs that could be detected by an overhead CCD camera, enabling the tracking of the position of the rat on the maze at 60 frames per second. All data were recorded using a Cheetah recording system (Neuralynx) running in combination with a Pentium-based personal computer. Single-unit data from each tetrode was amplified, filtered between 0.6 and 6 kHz (Assembly Hunter amplifiers; Neuralynx), and digitized at 32 kHz. Single units were recorded with respect to a reference electrode placed in or near the corpus callosum above the hippocampus. Video spatial resolution was approximately three pixels per centimeter. Electroencephalogram (EEG) data were collected from each tetrode, along with data from an electrode located near the hippocampal fissure (∼320 μm below the CA1 cell body layer, stratum pyramidale). EEG data were recorded with respect to the callosal reference, filtered between 1 and 300 Hz, and sampled at 1 kHz.
Each tetrode was advanced through parietal cortex toward the hippocampus from 0 to 160 μm/d while monitoring the unit activity with an audio amplifier (Grass Instruments, West Warwick, RI). As the cell body layer of CA1 was approached, 200-300 Hz oscillations (“ripples”) (O'Keefe and Nadel, 1978; Buzsaki, 1983) could be heard over the audio monitor and observed on the computer monitor. Tetrodes were then advanced gradually into the cell body layer of CA1 until multiple single units were observed.
At the start of the experiment, the rat was brought into the recording room and placed in a towel-lined bowl in the center of the recording arena. The head stage was attached to the hyperdrive, and electrical power was supplied to the low-noise amplifiers and LEDs on the head stage. The signal gain and threshold of each tetrode was determined, data recording was initiated, and the rat was allowed to rest for 20-30 min in the towel-lined bowl. After this rest period, the rat was moved from the bowl to the arena. The behavioral phase began when the first cue (consisting of a light and tone) was given and lasted from 30 to 60 min. During an experiment, a computerized control program monitored the spatial location of the rat and delivered rewarding MFB stimulation when the rat moved within a 10 cm reward zone by sending transistor-transistor logic pulses to the SIU. The rat was then returned to the towel-lined bowl and allowed to rest for another 20-30 min. Data acquired during the two rest epochs was combined with data from the behavioral epoch to enable better identification of single cells during cluster cutting (described below) and to verify stability.
Order of sequence training
All sequence training in all rats occurred after surgery. Sequence tasks were presented to each rat by cueing reward zones in cyclic order. After a rat completed a sequence three times with guidance from LED cues (a “cued” block of sequences), a 5 s delay was inserted between the nonspatial, audio cue and the illumination of cue lights, providing time for the rat to recall the next reward location and move to it without the aid of the spatial, visual cue. After the rat completed the sequence three times with a delayed visual cue (a “delayed-cue” block of sequences), audio and visual cues were presented simultaneously, again starting another block of cued sequences. Blocks of three complete sequences alternated between cued and delayed-cue throughout the duration of the recording session.
Three different sequences were used, each containing at least one repeated segment (see Fig. 1). For each sequence, rats were trained over several days until they reached asymptotic performance, normally when they met the zone selection criteria (described below) on 70% of delayed-cue trials after the choice point. This final training session on a given sequence was called the “criterion day” for that sequence. The “complex sequence” consisted of eight segments and contained two contiguous, repeated segments that followed the same physical path. MFB stimulation was given at each reward location. The use of two contiguous repeated segments caused the physical trajectory of the rat to be highly stereotypic on the second of the segments, regardless of the sequential context. The remaining two sequences were intended to approximate different aspects of the task used by Wood et al. (2000). For the “barrier-trained” sequence, rats learned to traverse a sequence of six segments containing one repeated segment and were guided, during training, by two moveable, wooden barriers, as in the initial training protocol used by Wood et al. (2000). The barriers (wood blocks, 10 × 15 × 3 cm) were placed on the maze during the first five to seven training sessions and moved manually so as to force the rat to follow alternate paths on each pass through the repeated segment. These barriers were removed completely by the final, or criterion, day on the task. A pair of guide rails (2 cm high, 0.7 m long, 10 cm separation) was used during this and the next task to limit the lateral movement of the rat along the repeated segment, again to approximate aspects of the task used by Wood et al. (2000). The “skipped-reward” sequence consisted of eight segments with one repeated segment. As in the complex sequence, no barriers were used during training for this task. During training, reward was given at both the beginning and end of the repeated segment on a probabilistic basis. The probability of reward was reduced across five to seven sessions until no reward was given at either end of the repeated segment.
The arena and schematics of routes for each sequence task. A photograph of the arena with clothespin-mounted light-emitting diodes (which served as cues) mounted around the perimeter is shown to the left. Neural data were recorded from the rat through a lightweight tether. The three figures on the right show top-down, schematic views of the routes for each sequence. The legs of each sequence are shown as lines ending in arrowheads, in which arrowheads note the location where reward was given. Dotted lines represent the first segment of a context. In CCW contexts (blue), a CCW turn at the choice point is the most efficient for reaching the next goal, whereas in CW contexts (red), the most efficient turn direction at the choice point is a CW turn. Parallel red and blue lines denote the repeated segment(s). All other segments are shown in black (e.g., in the skipped-reward task). In the barrier-trained task, the black rectangles represent wooden blocks that were used during training to guide the rat through the sequence. Once the rat had learned the sequence, these blocks were removed from the arena. In the skipped-reward task, the probability of reward at the ends of the repeated segment (denoted by an X) was reduced across sessions until no reward was given, allowing rats to make continuous trajectories into, through, and out of the repeated segment. To make the two tasks and corresponding spatial locations less similar to the rat, the direction of the repeated segment was rotated by 90° between the barrier-trained and skipped-reward tasks (as shown in the schematic). To enable easier comparisons of unit responses, the orientation of sequences has been rotated in subsequent figures such that the repeated segments are vertical.
Data analysis
Preprocessing. The duration of stimulation was 300 ms. Artifact arising from MFB stimulation was removed by deleting all spikes recorded within 1 s of each stimulation onset, which provided sufficient time for the fading of any transients in the amplifiers. Putative single neurons were isolated using an interactive procedure based on the relative action potential amplitudes on the different tetrode channels and other wave-form parameters (McNaughton et al., 1983b; Wilson and McNaughton, 1994), producing a collection of timestamps associated with each action potential, or spike, from a given unit. Pyramidal cells were identified on the basis of wave shape, a bimodal interspike interval distribution reflecting complex bursting, and a mean firing rate below 5 Hz across the recording session (Muller et al., 1987). Position during each video frame was extracted by finding the center of mass of the image of the head stage LEDs.
Performance criteria. Each sequence contained a series of “segments,” beginning with the onset of the tone and ending with entry into the reward zone and delivery of MFB stimulation. Typically, rats actively sought goals and ran quickly to them, but this activity was interrupted at times by pauses or grooming bouts lasting from several seconds to >1 min. Unlike alternation tasks run on elevated tracks (Frank et al., 2000; Wood et al., 2000; Lenck-Santini et al., 2001; Ferbinteanu and Shapiro, 2003), these tasks were run on an open, flat surface, which allowed rats to move anywhere (even over the guide rails) at any time, allowing rats to make more and different types of errors than are possible when movement is constrained by an elevated track. To ensure that data used for analyses came only from segments during which the rat was highly motivated to find the goal, a set of criteria was established to remove trials containing other behaviors. Acceptable segment traversals satisfied two criteria: accuracy and continuous movement. A segment passed the accuracy criterion if the rat entered the correct zone before cue onset (i.e., within 5 s) after having passed through no more than one other zone. This allowance for an extra zone was required for those cases when rats made the correct turn after the repeated segment but then passed through a zone en route to the correct target, not those cases in which rats simply turned in the wrong direction and entered the wrong reward zone first. The continuous movement criterion was used to disqualify those trials in which the rat's trajectory was interrupted by a drop in velocity below 0.2 m/s (max velocity during runs was ∼0.6 m/s) or if the rat failed to begin the segment promptly after receiving the preceding reward. Most errors involved either the rat pausing along the repeated segment or straying too far off the regular path taken on valid trials and hence not reaching the goal zone within the time limit. The percentage of all trials on which the rat chose the correct goal within the allotted time was used as the measure of “behavioral proficiency.”
Computation of binned mean firing rates
Binning method. Only data from delayed-cue trials were used. The segments leading into, through, and out of the repeated segment were grouped together into contexts. Contexts were defined as clockwise (CW) or counterclockwise (CCW) according to the most efficient turn required to reach the next goal after the choice point. The average path taken by the rat through the repeated segment was divided into five spatial bins of equal length, as in the procedure used by Wood et al. (2000). For each segment, the binned firing rate of each unit was computed by dividing the total number of spikes in the bin by the total bin occupancy time. Because the body orientation of the rat often varied with turn direction at the beginning and end of the repeated segment, data in the first and last bins were dropped from additional analysis. In addition, two other factors that are known to influence place cell firing were considered: velocity and path deviation. The average velocity in a bin was computed by dividing the number of pixels traversed between the bin edges (i.e., arc length along the path) by the time spent in that bin. To compute the path deviation along the repeated segment, a straight line between start and goal was computed, and the deviation from the actual path to that line was found at nine equally spaced points. More specifically, the path deviation for each traversal was found by taking the intersection of each true path with a series of perpendicular lines extending from each of the nine sampling points along the straight-line path, yielding nine measurements for path deviation for each traversal. The median path was found by taking the median of these path deviations. All deviation values used in subsequent analyses indicate the distance of given traversal from the median path. To compensate for the effects of differences in path deviation or velocity on the firing rate of a given cell, a regression analysis was run on the data within each bin, using spike rate as the dependent variable and deviation and velocity as independent variables. The adjusted rates for each traversal were found by adding the residual of the regression to the mean firing rate for that bin: Yadjijk = Yijk - Ypredijk + Ymeanj, where Yijk is the raw firing rate for the ith context in the jth bin during the kth traversal, Ypredijk is the firing rate predicted by the regression equation, Ymeanj is the mean firing rate across context and traversal, and Yadjijk is the adjusted firing rate. For comparison purposes, the ANOVA was computed for both raw and adjusted firing rates. Cells whose maximum firing rate across all bins failed to exceed 1 Hz in any bin were excluded from analysis. For the remaining pyramidal cells, place-related activity and the effects of context on the repeated segment were assessed using a two-factor ANOVA (bin by context) according to the procedure of Wood et al. (2000). Place-related activity was defined by a significant bin effect (p < 0.05). Context-related effects were defined as either a main effect of context or a bin-by-context interaction (p < 0.05). Conducting multiple tests using a fixed criterion level will, of course, lead to some false-positive results. Because context-related activity was dependent on two independent tests (a main effect of context or a bin-by-context interaction), each with a significance criterion of p < 0.05, the number of cells with place-specific responses that were expected to show context-related effects by chance was 9.75%. It should also be noted that the use of a regression analysis to account for the effects of path deviation and velocity produces a conservative estimate of context-dependent differences in firing rate, because any variability attributable to either of these causes is removed before comparison of firing rate differences attributable to context.
Fisher linear discriminant. The similarity of the ensemble of place-related activity for the two contexts was determined by the construction of a Fisher linear discriminant (Duda et al., 2001) using a leave-one-out method. For each trial, the firing rate population vectors for each of the three middle bins were computed for all n recorded units. These vectors were joined together to form a composite vector for each segment; i.e., the composite vector for each segment had a length k, which was equal to n units multiplied by three bins. Composite vectors across trials were grouped according to CW and CCW contexts. A classifier (a k - 1 hyperplane through k-dimensional space) was constructed using all of the composite vectors from the two contexts, with the exception of one vector from one context, which became the test vector for that particular classifier. A classifier, or discriminant, can be thought of as a plane that places the largest possible number of example vectors from the two contexts on its opposite sides. The class of a test vector is determined by finding on which side of the discriminant plane it lies. In this manner, the classifier was used to classify the left-out, test vector.
This process was repeated for each vector in both contexts, in which only the test vector was “left out” during the construction of the classifier. If the class for the test vector was the context to which it actually belonged, this was considered a correct classification. The percentage of correct classifications across both groups, referred to as a separability score, served to quantify the degree of separation between the two groups of population vectors. Such a leave-one-out method produced an estimate of the best possible separation that could be obtained using the observed data, and so the separability score can be thought of as describing a lack of similarity in the data for the two contexts. An estimate of the variance of separability scores was obtained using a “bootstrap” procedure, in which context memberships for each composite vector were randomized, and the separability score was computed. This process was repeated 100 times. To determine which tasks showed separability above chance, the mean separability score was compared with the chance distribution obtained by the randomized bootstrap procedure, using a t test (p < 0.05). This procedure was computed for both raw and adjusted firing rate data. Given the conservative nature of the regression analysis, these two results should bracket the true separability achievable for a given data set.
Results
Behavior
Three rats were trained on the complex-sequence task, and two of these rats (rats 2 and 3) were trained on both the barrier-trained and skipped-reward tasks (Fig. 1). Training alternated between blocks of three cued sequences, during which the cue light marking the next correct reward zone was illuminated at the start of segment traversal and three delayed-cue sequences, during which the illumination of the cue light was delayed by 5 s, giving the rat the opportunity to recall the next correct reward location from memory. Behavioral proficiency was assessed during delayed-cue blocks, according to the performance criteria described in Materials and Methods (specifically, the accuracy of behavioral choice and the continuity of movement) (Fig. 2). It should be noted that, if only the two possibly correct zones were considered, then the chance level of behavioral performance would be 50%. Considering all possible reward zones and the possibility that the rat would not reach any reward zone before cue illumination, chance performance would be less than one in seven. As shown in Figures 2 and 3, rats were able to learn sequence tasks rapidly, and the same rat could learn many different sequences within a few weeks time (Table 1).
Behavioral analysis for the three sequence tasks. For each sequence, the schematic of the sequence (see Fig. 1) is shown at the top left, the behavioral accuracy scores (i.e., the percentage of traversals that met the behavioral criterion) for each rat are shown in the bar graphs at the bottom left, and the actual paths followed by each rat are shown in the right portion. The paths labeled “Selected,” a subset of “All,” are those that met the performance criteria described in Materials and Methods and were used for subsequent analysis. Behavioral performance is shown for the complex-sequence task (A), barrier-trained task (B), and skipped-reward task (C). Only traversals that occurred during delayed-cue blocks are considered (see Materials and Methods).
Rats solved the sequence tasks using a spatial rather than a motor strategy. In both graphs, the horizontal axis shows the number of delayed-cue blocks (sets of 3 complete sequences, during which the illumination of the cue light was delayed by 5 s) after exposure to a novel or previously learned but rotated sequence. The change or rotation occurred while the rat was performing the previously learned sequence and was initiated at the start of a cued block of trials. The vertical axis shows the percentage of zones reached before cue onset. The “Novel” sequence shows the disruption in performance during delayed-cue segment traversals when rats were exposed to a sequence with which they had no previous experience (for details, see Materials and Methods). The “Rotated” sequence shows the disruption in performance during delayed-cue segment traversals when rats were exposed to a rotated version of a previously learned sequence. As denoted by an asterisk, performance during the first block after the change in both conditions is significantly worse than the average performance on the preceding 10 blocks (t test; p < 0.05). In addition, the relearning curves are similar.
The number of sessions to reach behavioral criterion for all rats and all tasks
These sequences could potentially be learned in two different ways: by using a motor strategy consisting of a series of specific turns or by using a spatial strategy relying on the arrangement of proximal and distal cues. It was important to determine which strategy was being used because spatial learning requires an intact hippocampus, whereas motor strategies rely on other structures (for review, see Redish, 1999). If rats used a motor strategy to learn a sequence, then a rotation of that sequence should have little effect on performance of the task. If, however, rats used a spatial strategy to learn a sequence, then the rotation of a familiar sequence with respect to proximal and distal cues should disrupt performance in the same way as the presentation of a novel sequence. To test these possibilities, a previously learned (i.e., “familiar”) sequence was rotated by 180°; nothing in the environment was rotated or changed, only the sequence of reward locations (a manipulation made possible by use of video-tracker-based control of the experiment). Performance was evaluated by noting the time required to reach each zone in the sequence. The relearning curves for these familiar but rotated sequences were compared with those observed when rats 2 and 3 initially learned the complex-sequence and barrier-trained sequences. As shown in Figure 3, performance was disrupted equivalently after the introduction of a novel sequence and the rotation of a previously learned sequence. In addition, the learning curves for the novel and rotated sequences were comparable. This indicates that rats did not use a motor strategy to learn these sequences but relied on proximal and/or distal spatial cues.
The complex-sequence task
For rats 1, 2, and 3, a total of 99 units were recorded on the criterion day (57 from rat 1, 29 from rat 2, and 13 from rat 3). Of these, 88 were identified as pyramidal cells (49 from rat 1, 29 from rat 2, and 10 from rat 3). Only data from the segment leading to the choice point (the second of the two contiguous repeated segments shown in Fig. 1) were analyzed because of the variability in the rat's heading when entering the first segment. Unit activity along the second repeated segment was binned, firing rates were corrected for path deviation and running speed using regression, and a two-way ANOVA (with factors of bin and context) was computed, as described in Materials and Methods. Within the second of the repeated segments, 21 of the 88 pyramidal cells showed a significant effect of bin on firing rate (ANOVA; p < 0.05; 10 of 49 from rat 1, 8 of 29 from rat 2, and 3 of 10 from rat 3), i.e., showed significant place-related activity in the second repeated segment. Of pyramidal cells with significant activity in the second repeated segment, 0 of 21 showed a significant main effect of context, or a bin-by-context interaction (ANOVA; p < 0.05) (Fig. 4).
The complex-sequence task was solved without differential encoding in the hippocampus. Each row depicts data from the same rat during a single session; each subpanel depicts a different cell and the corresponding cell identification number. The actual paths taken by the rat during clockwise (light gray paths; blue dots and bars) and counterclockwise (dark gray paths; red dots and bars) contexts are overlaid, showing the similarity between the paths that were taken. The location of the rat when each spike occurred is overlaid on the paths. The mean binned firing rates and SEs are shown by bar graphs adjacent to the paths, with calibration bars shown in the bottom right of each subpanel. The cells displayed were chosen because they showed the greatest difference between the two sequential contexts. None of the cells analyzed exhibited statistically significant differences in firing rate attributable to sequential context in any bin (ANOVA; p < 0.05).
Barrier-trained task
A possible explanation for the failure to observe differential responses similar to those observed by Wood et al. (2000), Frank et al. (2000), and Ferbinteanu and Shapiro (2003) involves differences in training protocols. For example, Wood et al. (2000) used barriers to force rats to follow the correct path during training. Thus, sequential-context-dependent sensory cues were available to their animals to aid in solving the task during initial training. It is possible that the presence of these cues per se may have led to the context-dependent differentiation of hippocampal activity in the repeated segment. To test this, a barrier-trained task was used in which barriers were placed at the entry and exit of the repeated segment during training, forcing the rat into the correct next segment, as well as providing sensory cues that changed reliably with the current sequential context. Because rats tended to run around these barriers on an open maze, guide rails (see Materials and Methods) were added to restrict entry into the repeated segment; these rails remained in fixed positions throughout all training and recording sessions.
On the first day of training (with barriers present), a total of 40 units were recorded (22 from rat 2 and 18 from rat 3). Of these, 31 units were identified as pyramidal cells (19 from rat 2 and 12 from rat 3). Of pyramidal cells, 19 of 31 showed a significant effect of bin on firing rate along the repeated segment (ANOVA; p < 0.05; 14 of 19 from rat 2 and 5 of 12 from rat 3). Of these, 13 of 19 units showed a significant main effect of context or bin-by-context interaction, i.e., showed a significant difference in firing rate between the sequential contexts (ANOVA; p < 0.05; 10 of 14 from rat 2 and 3 of 5 from rat 3). The number of pyramidal cells that showed differential place-related activity is more than would be expected by chance. Thus, the physical presence of sequential-context-dependent sensory cues was sufficient to differentiate the hippocampal activity.
Once the sequence had been learned to criterion and the barriers removed, a total of 51 cells were recorded on the criterion day (22 from rat 2 and 29 from rat 3). Unit activity along the repeated segment was binned, firing rates were corrected for path deviation and running speed using regression, and a two-way ANOVA (with factors of bin and context) was computed, as described in Materials and Methods. For all recorded cells, 40 of 51 were identified as pyramidal cells (16 of 22 from rat 2 and 24 of 29 from rat 3). Of these pyramidal cells, 19 of 40 showed a significant effect of bin on firing rate along the repeated segment (ANOVA; p < 0.05; 11 of 16 from rat 2 and 8 of 24 from rat 3). Of these, 7 of 19 units showed a significant main effect of context or a bin-by-context interaction (ANOVA, p < 0.05; 4 of 11 from rat 2 and 3 of 8 from rat 3) (Fig. 5). The number of pyramidal cells that showed differential place-related activity is more than would be expected by chance. Consistent with the results described by Wood et al. (2000), differential activity was observed in the form of both qualitative and quantitative differences in place fields. Some fields were present in only one context, some were shifted with context, and some exhibited context-dependent peak firing rates. Thus, the presence of sequential-context-dependent sensory cues during acquisition was sufficient to produce differential hippocampal activity during subsequent performance in the complete absence of the cues. This indicates that information stored in long-term memory was responsible for the differentiation of the hippocampal activity.
Training with barriers (the barrier-trained task) produced persistent differential activity. Each row depicts data from three different cells during the same recording session from the same rat, performing the task without the aid of barriers. The asterisk denotes significant sequential-context-specific activity as determined by either a significant main effect of context or bin-by-context interaction (ANOVA; p < 0.05). All other details are the same as those given in Figure 4.
Skipped-reward task
A second possible explanation for the lack of differential responses in the complex-sequence task involves the presence or absence of reward within the repeated segment itself. In the studies by Wood et al. (2000), Frank et al. (2000), and Ferbinteanu and Shapiro (2003), no reward was given at the choice point at the end of the repeated segment. Unlike the complex-sequence task just described, the choice point in the aforementioned studies was not itself a “goal” in the sense of being a source of reward. Assuming that a future reward site constitutes a goal and that a previous reward site may be represented in working memory, then these sources of context-dependent input to the hippocampus may have contributed to the differentiation of hippocampal activity in the two sequential contexts in the previous studies. If so, then the inclusion of rewards at the beginning and end of the repeated segment may have blocked these differentiating inputs. To test this hypothesis, a skipped-reward task was introduced, in which reinforcement was withheld at the ends of the repeated segment. This enabled the rat to make smooth, continuous trajectories into, through, and from the repeated segment, possibly creating context-dependent input to the hippocampus attributable to different goals or sequential-context-dependent working memory of the most recent reward site while the rat was in the repeated segment. The barriers used in the previous experiment were not used at any time in this procedure.
On the first day of training (when reward was given with a probability of 50% at the entry and exit of the repeated segment), a total of 47 units were recorded (25 from rat 2 and 22 from rat 3). The performance of rat 3 on the first day of this task was very poor, and so these data were dropped from additional analysis. Of the 25 units from rat 2, 19 were identified as pyramidal cells. Of these, 8 of 19 showed a significant effect of bin on firing rate along the repeated segment (ANOVA; p < 0.05). Of these, three of eight showed a weak, but significant main effect of context or a bin-by-context interaction (ANOVA; p < 0.05). These differences, although slightly more than expected by chance, were mainly quantitative (see below).
The reward probabilities at the entry and exit of the repeated segment were gradually reduced to zero across several training sessions, until rats could perform the sequence without receiving reward at the entry or exit of the repeated segment. On the criterion day, a total of 65 units were recorded (25 from rat 2 and 40 from rat 3). Unit activity along the repeated segment was binned, firing rates were corrected for path deviation and running speed using regression, and a two-way ANOVA (with factors of bin and context) was computed, as described in Materials and Methods. For all recorded cells, 50 of 65 were identified as pyramidal cells (17 from rat 2 and 33 from rat 3). Along the repeated segment, 20 of 50 pyramidal cells showed a significant effect of bin on firing rate (ANOVA; p < 0.05; 9 of 17 from rat 2 and 11 of 33 from rat 3). Of these, 11 of 20 cells showed a significant main effect of context or bin-by-context interaction (ANOVA; p < 0.05; 7 of 9 from rat 2 and 4 of 11 from rat 3) (Fig. 6). The number of pyramidal cells that exhibited significant differential activity was more than would be expected by chance. Again, consistent with Wood et al. (2000), differential activity was observed in the form of place fields that were present in only one context, were shifted with respect to context, or exhibited changes in context-dependent peak firing rates within the same place field. Compared with the first training session with 50% reward probability, the magnitude of the sequential-context-dependent differences increased substantially once the rats learned to run smoothly into, through, and out of the repeated segment in the complete absence of reward. Thus, the presence of differential input to the hippocampus, attributable to either different goals or different working memory of the most recent reward site, was sufficient to induce differential activity in the hippocampus. Cell counts for each condition are summarized in Table 2.
Training without rewards at the entry or exit of the repeated segment (the skipped-reward task) induced differential activity. In the schematic at the top, X symbols denote zones in which reward was withheld. Each row depicts data from three different cells during the same recording session from the same rat. All other details are the same as those given in Figure 4.
The numbers of units by task, rat, and type
Discriminant analysis
The proportion of cells exhibiting significant differences in firing rate in the two sequential contexts, as presented above, does not take into account the magnitude of the differences in the overall ensemble activity. To obtain a more quantitative estimate, the Fisher linear discriminant, which finds the plane, or discriminant, that best separates two samples, was applied to the data of the present study. Whereas a correlation determines the degree of similarity between two samples, a discriminant determines the degree of separability between them (Duda et al., 2001). When applied to the composite population vectors (see Materials and Methods) obtained after the sequences had been learned in the complex, barrier-trained, and skipped-reward tasks, the latter two showed separability scores that were substantially and significantly larger than the chance value of 50% (Fig. 7). In addition to considering the separability after several days of training (i.e., the criterion day), the separability observed on the first training day (i.e., the “initial” day) was computed for the barrier-trained and skipped-reward task (Fig. 7, BTi, SRi). Separability on the initial day of the barrier-trained task was significantly greater than chance, whereas the separability observed for the initial day of training on the skipped-reward task was not (consistent with the weakness of the effects observed in the three cells on that day that exhibited significant differences).
Ensemble neural activity discriminated between the two contexts for barrier-trained and skipped-reward tasks but not for the complex-sequence task. Black bars show the behavioral proficiency, i.e., the percentage of segment traversals in which the rat met behavioral criterion, as described in Materials and Methods. Light gray bars show the mean percentage of correct classifications of context based on raw firing rate data using a Fisher linear discriminant analysis of the neural firing patterns, i.e., how different the ensemble firing patterns on the repeated segment were between the two contexts. Dark gray bars show the mean percentage of correct classifications of context based on firing rate data adjusted for path deviations and velocity. CS, The complex-sequence task; BTi, the initial day of the barrier-trained task; BTc, the criterion day of the barrier-trained task; SRi, the initial day of the skipped-reward task; SRc, the criterion day of the skipped-reward task. The asterisk denotes prediction that was significantly greater than chance, as defined in Materials and Methods. The lack of an error bar for “SRi” was attributable to having only one rat for this condition.
One possibility was that sequential-context-dependent hippocampal responses were present only when rats were highly trained on a sequence task. A regression analysis showed, however, no significant correlation between the behavioral proficiency and the separability score (r2 = 0.310; p > 0.05). In addition, a post hoc analysis was conducted in which the five tasks were grouped into those with separability scores significantly greater than chance (i.e., barrier-trained initial, barrier-trained criterion, and skipped-reward criterion, which were referred to collectively as the “separable” tasks) and those with separability scores no different than chance (i.e., complex sequence and skipped-reward initial, which were referred to collectively as the “nonseparable” tasks). No significant difference was found between the behavioral proficiency for the two groups (ANOVA; F(1,9) = 2.01; p > 0.05). Thus, separability scores above chance (i.e., orthogonalization) did not result from rats being better trained on some sequences than others.
Acquisition of differential representations
The change in separability scores on the skipped-reward task from the first to the criterion days was examined across intervening days (Fig. 8). Persistent changes in place specificity of CA1 place cells can arise both within a single recording session (Wilson and McNaughton, 1993) and across multiple recording sessions (Lever et al., 2002). One possibility was that the two sequential contexts became more separable as proficiency on the task improved, but this was not the case (r2 = 0.028; p > 0.05). There was, however, a significant correlation between separability and the number of sessions run on the Skipped-reward task (r2 = 0.7118; p < 0.05). The improvement in separability scores across days suggests that the differential encodings for the same physical location of the repeated element became progressively orthogonal across days, i.e., the ensemble activity became more specific to each sequential context with repeated exposure to the task.
Separability of neural activity increases with repeated exposures. The horizontal axis shows the number of the training day on the skipped-reward task; “C” represents the criterion day for each rat (day 6 for rat 2 and day 10 for rat 3). The vertical axis shows the separability score as determined by discriminant analysis. Data are shown for two rats (squares for rat 2; dots for rat 3). Data for day 1 for rat 3 were not used because of poor proficiency on the task; data for days 4 and 5 for rat 2 were not used because too few units were recorded on those days to allow analysis. Guide rails, which limited lateral position variability on the repeated segment, were present on each day.
Proportion of cells with place-specific activity on the repeated segment
When comparing the percentage of cells that became active within the repeated segment with those cases in which differential hippocampal activity was observed (as in the study by Wood et al., 2000 or in the barrier-trained and skipped-reward tasks in these studies) and those when it was not (as in the study by Lenck-Santini et al., 2001 or in the complex sequence in the present studies), two scenarios are possible. In one scenario, the same overall percentage of cells could become active within the repeated segment as for any other segment, with half responding in each context. This would suggest that the fixed population of cells that would normally have been allocated to the repeated segment was divided into two groups, one for each context. Alternatively, the same overall percentage of cells could become active in each sequential context as would have been allocated for any other segment. This would result in a larger overall percentage of pyramidal cells showing place-related activity along the repeated segment during the entire task. The latter case would be expected on the basis of numerous other studies, in which different, but overlapping populations of cells with place fields were observed in two different environments (Skaggs and McNaughton, 1998) or within the same environment (Markus et al., 1995) in different contexts, thus increasing the overall percentage of observed place cells when both contexts or environments were considered jointly.
Using the post hoc grouping described in the discriminant analysis section above (nonseparable and separable), the percentage of pyramidal cells showing place-specific activity along the repeated segment (regardless of context) was significantly greater for the separable group (54.1 ± 8.9%) than for the nonseparable group (29.2 ± 5.2%) (ANOVA; F(1,9) = 5.35; p < 0.05). The percentages for each rat and for each session are shown in Figure 9. Thus, the differentiation of the two contexts by the hippocampus is consistent with the allocation of different, but overlapping, populations of the same numbers of neurons to each context, which is consistent with other studies.
Tasks in which differential hippocampal responses occurred activated a larger percentage of pyramidal cells. The vertical axis shows the percentage of pyramidal cells in one recording task that showed a significant main effect of bin along the repeated segment (i.e., showed a place-selective response regardless of context). Sequence tasks were grouped according to whether the discriminant analysis (as summarized in Fig. 7) could predict the context at better than chance levels. “Non-separable” denotes tasks in which prediction was no greater than chance; “Separable” denotes tasks in which prediction was greater than chance. The separable tasks showed a significantly larger percentage of units with place-specific responses on the repeated segment (ANOVA; F(1,9) = 5.3491; p < 0.05). CS, The complex-sequence task; BTi, the initial day of the barrier-trained task; BTc, the criterion day of the barrier-trained task; SRi, the initial day of the skipped-reward task; SRc, the criterion day of the skipped-reward task.
Discussion
Rats learned complex spatial sequences that included repeated segments. Performance was disrupted by rotation of the sequences with respect to distal and proximal cues. Thus, the encoding involved spatial information and not simply motor sequences. An involvement of the hippocampus in task performance is therefore strongly implicated (O'Keefe and Conway, 1978; O'Keefe and Nadel, 1978; Muller et al., 1987). Hippocampal neural activity exhibited the expected spatial selectivity across all sequence tasks. Nevertheless, differential activity specific to the sequential contexts of the repeated segments was not observed in the complex-sequence task, despite the fact that the task was arguably more difficult than others in which differential activity was observed (Frank et al., 2000; Wood et al., 2000; Ferbinteanu and Shapiro, 2003). Thus, differential hippocampal activity is not required to separate spatial-sequence-dependent contexts behaviorally.
Differential activity was observed during the barrier-trained and skipped-reward tasks, in agreement with Wood et al. (2000), Frank et al. (2000), and Ferbinteanu and Shapiro (2003). This result was observed even given the highly conservative method used to exclude variability attributable to path differences and velocity (see Materials and Methods). A distinction should be made, however, between two possible explanations for this effect. The first explanation involves the chaining together of place cells with overlapping place fields. Several theories suggest that temporally asymmetric synaptic plasticity could link neurons into chains capable of encoding sequences of events or locations (Hebb, 1949; McNaughton and Morris, 1987; Abbott and Blum, 1996; Levy, 1996). This hypothesis is supported by observations that the temporal order in which place cells are activated during maze running is preserved in subsequent sleep (Skaggs and McNaughton, 1996) and by experience- and NMDA-receptor-dependent place field expansion during repeated route following (Mehta et al., 1997; Ekstrom et al., 2001). In the study by Frank et al. (2000), differential CA1 activity was observed mostly near the choice point, which could potentially be explained by asymmetric, synaptic plasticity leading to the backwards expansion of place fields (in the manner reported by Mehta et al., 1997) into the repeated segment from segments after the choice point. Asymmetric associations would be necessary for sequence learning with or without a repeated element, but this mechanism cannot account for differential hippocampal activity located far from the choice point, as observed in the present experiments and by others.
The second explanation involves independent encodings of the repeated segment for each context, as if the contexts were in completely different physical locations. In the study by Wood et al. (2000) and in the present study, differential activity, when it occurred, was observed along the entire repeated segment. This suggests a context-dependent encoding of the repeated segment, which may reflect the ability of the hippocampus to amplify differences in the neural input received from other brain structures. The present results are consistent with many other studies showing that the hippocampus can exhibit nonlinear changes in its encoding when afferent information changes. This property, often referred to as orthogonalization or remapping, may play an important role in maximizing associative memory capacity (Marr, 1969; McNaughton et al., 1989; Treves and Rolls, 1991; O'Reilly and McClelland, 1994). Hippocampal activity at a given location can be strongly influenced by internal state variables not associated with any current external cue. For example, orthogonalization in the hippocampus has been shown to be affected by long-term memory (Quirk et al., 1990; Bostock et al., 1991), working memory and/or path integrator coordinates (O'Keefe and Conway, 1978; Skaggs and McNaughton, 1998), behavioral set (Foster et al., 1989; Markus et al., 1995), or current orientation of the head-direction system (Knierim et al., 1995). In the present study, a higher percentage of place-specific units was recorded in the repeated segment during tasks in which the hippocampus exhibited sequential-context disambiguation, confirming the direct prediction of the orthogonalization hypothesis that different, but overlapping, subsets of the same relative proportion of neurons are allocated to each context (Guzowski et al., 1999).
Such recoding might assist complex sequence learning by reducing the demand for working memory. On longer behavioral timescales, it might also contribute to the encoding of episodic memories that involve the same items in different contexts (Wallenstein et al., 1998; Agster et al., 2002). In contrast to the conclusions of Agster et al. (2002) and Fortin et al. (2002), however, unique sequential-context-dependent activity in the hippocampus is not necessary to perform spatial sequence tasks involving repeated elements.
Given that hippocampal disambiguation of the repeated elements of a spatial sequence is not necessary for repeated-sequence learning, the question remains as to why such disambiguation occurs in some cases. As one would expect, physical differences along the repeated segment, such as the blocks used during early phases of the barrier-trained task, can cause differential hippocampal activity; however, on the final day of testing with barrier-trained and skipped-reward tasks, all external information was identical, yet differential activity was still observed. Working memory for recently visited locations, or (equivalently) an internal representation of future goals, however, was probably available. Working memory of recently visited locations is capable of orthogonalizing hippocampal codes for similar environments (Skaggs and McNaughton, 1998) and could have had a similar effect in the present case. Although the differential activity induced by the blocks in the barrier-trained task cannot account for the activity on the criterion day, it does demonstrate that physical cues during training can establish differential representations of the environment that persist even in their absence. It is possible that, once differential activity patterns become established on the basis of differential physical cues, they can be chained together associatively, allowing the differential representation to persist in the absence of the cues. Indeed, this would constitute a form of working memory that might well assist the animal in making the correct choice. Alternatively, the persistent differential activity might reflect pattern completion resulting from pairing information in a more traditional item-based working memory with the differentiating cues.
The lack of hippocampal orthogonalization on the first day of the skipped-reward task (and in the complex-sequence task) may reflect masking of the differentiating effect of working memory information by interference from additional, nondifferential task variables; reinforcement at the entry to the repeated segment, and possibly expectation of reinforcement at the exit, provided a common component to the hippocampal input in the two sequential contexts. This common information might have overridden any sequential-context-specific activity (e.g., working memory) transmitted to the hippocampus from other structures. Once the reinforcement was removed, the latter activity was apparently sufficient to induce orthogonalization of the hippocampal output. A similar explanation can be suggested for why, during random foraging (i.e., no particular consistent goal at any given location), spatial selectivity can be completely independent of direction of travel through the place field (Muller et al., 1987), whereas, during shuttling on linear tracks in which the goals are consistently different at a given location, spatial selectivity is almost completely directionally dependent (McNaughton et al., 1983a). An exception, however, to the directional dependence of place fields on linear tracks was recently noted (Battaglia et al., 2004). Multiple discrete, local cues on a linear track induce bidirectionality in many hippocampal cells, presumably arising from direction-invariant object representations in neocortical afferents. The presence of consistent starting locations and goals could explain why Lenck-Santini et al. (2001) failed to observe differential activity in the Y-maze alternation task. In this case, trajectories in the two contexts had either a common starting location or a common goal, which could have prevented the orthogonalization of the respective context encodings. The present results suggest that differential hippocampal encodings of the same sequence location emerged over time only in the presence of reliable internal and/or external context-specific cues.
There are thus two principal conclusions of this study. The first is that hippocampal differentiation of the sequential context of a repeated segment during a spatial sequence task is unnecessary for the animal to learn the sequence successfully (including the repeated component). Unless the differential codes are established elsewhere in the brain, performance of such tasks without the aid of differential hippocampal activity would rely on working memory or a trace-strength comparison and hence might be more subject to interference, distractions, or delays than would be the case if differential encodings were produced. Thus, hippocampal-lesioned animals might be more impaired at learning sequence tasks involving delays (Ainge and Wood, 2003). The second conclusion is that differential hippocampal activity within the repeated segment can be induced by context-dependent external or internal cues after repeated exposures to a given sequence, provided that the net input to the hippocampus is sufficiently distinct. These conclusions lead to two predictions. When compared with rats trained such that differential hippocampal codes are not produced (as in the complex-sequence task), rats with differential hippocampal codes should be less distracted by delays on the repeated segment, because differential hippocampal activity could help sustain working memory. Conversely, making the common segment more distinct (e.g., by adding local cues, as in the study by Battaglia et al., 2004) and thereby biasing the hippocampus to form identical codes for the two sequential contexts might, paradoxically, make performance of the repeated element sequence task less robust.
Footnotes
This work was supported by National Institutes of Health Grants NS20331 and MH01565 and the Japan Science and Technology Corporation-Core Research for Evolutional Science and Technology. We acknowledge Erik Ells-worth, Nathalie Gebara, Kim Bohne, Samantha de Dios-Goodwin, Jennifer Dees, and Kate Poneta for help with data collection and analysis and Steve Cowen, Hiroaki Wagatsuma, and Drs. Ephron Rosenzweig, Francesco Battaglia, and Edvard Moser for technical assistance and helpful comments.
Correspondence should be addressed to Dr. Bruce L. McNaughton, Arizona Research Laboratories, Division of Neural Systems, Memory, and Aging, Life Sciences North Building, Room 384, University of Arizona, Tucson, AZ 85724-5115. E-mail: bruce{at}nsma.arizona.edu.
Copyright © 2005 Society for Neuroscience 0270-6474/05/251313-11$15.00/0
