QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

HOME  |   SEARCH  |   ARCHIVE  |   SUBSCRIBE  |   CONTACT  |   HELP

This Article Abstract Full Text (PDF) Submit an eLetter Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in Web of Science Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Web of Science (33) Citing Articles via Google Scholar Google Scholar Articles by Corbit, L. H. Articles by Balleine, B. W. Search for Related Content PubMed PubMed Citation Articles by Corbit, L. H. Articles by Balleine, B. W.

 Previous Article  |  Next Article 

The Journal of Neuroscience, June 1, 2000, 20(11):4233-4239

The Role of the Hippocampus in Instrumental Conditioning

Laura H. Corbit and Bernard W. Balleine

Department of Psychology, University of California, Los Angeles, Los Angeles, California 90095

	ABSTRACT

TOP ABSTRACT INTRODUCTION EXPERIMENT 1A EXPERIMENT 1B EXPERIMENT 2 GENERAL DISCUSSION REFERENCES

Considerable evidence suggests that, in instrumental conditioning, rats can encode both the specific action-outcome associations to which they are exposed and the degree to which an action is causal in producing its associated outcome. Three experiments assessed the involvement of the hippocampus in encoding these aspects of instrumental learning. In each study, rats with electrolytic lesions of the dorsal hippocampus and sham-lesioned controls were trained while hungry to press two levers, each of which delivered a unique food outcome. Experiments 1A and 1B used an outcome devaluation procedure to assess the effects of the lesion on encoding the action-outcome relationship. After training, one of the two outcomes was devalued using a specific satiety procedure, after which performance on the two levers was assessed in a choice extinction test. The lesion had no detectable effect on either the acquisition of instrumental performance or on the rats' sensitivity to outcome devaluation; lesion and sham groups both reduced responding on the lever associated with the devalued outcome compared with the other lever. In experiment 2, the sensitivity of hippocampal rats to the causal efficacy of their actions was assessed by selectively degrading the contingency between one of the actions and its associated outcome. Whereas sham rats selectively reduced performance on the lever for which the action-outcome contingency had been degraded, hippocampal rats did not. These results suggest that, in instrumental conditioning, lesions of the dorsal hippocampus selectively impair the ability of rats to represent the causal relationship between an action and its consequences.

Key words: hippocampus; instrumental conditioning; outcome devaluation; reward; contingency; rat

	INTRODUCTION

TOP ABSTRACT INTRODUCTION EXPERIMENT 1A EXPERIMENT 1B EXPERIMENT 2 GENERAL DISCUSSION REFERENCES

The ability of a hungry rat to acquire seemingly arbitrary actions, such as lever pressing, to gain access to food is one of the most robust forms of learning one can observe in the animal laboratory. Nevertheless, there have been few recent attempts to systematically assess the neural basis of this form of learning. Although considerable research has investigated the brain processes that mediate learning generally, the vast majority of this work has focussed on the learning of predictive relationships between events primarily using the Pavlovian conditioning paradigm. As a consequence, the neural structures controlling instrumental conditioning remain poorly understood.

Significant advances have, however, been made in our understanding of the psychological determinants of instrumental action in the rat. Although for many years instrumental learning was characterized solely in stimulus-response (S-R) terms, recent evidence from outcome devaluation studies has made it clear that, in instrumental conditioning, rats are able to encode the specific consequences of their actions and that the encoded action-outcome (A-O) relationship plays a critical role in the initial acquisition and performance of an instrumental action (for review, see Colwill and Rescorla, 1986; Dickinson and Balleine, 1994). Nevertheless, evidence that A-O associations play a role in instrumental conditioning does not rule out the involvement of an S-R process. In fact there is considerable evidence suggesting that, when overtrained, instrumental performance can become stimulus-bound, independent of the current value of the outcome and so impervious to outcome devaluation (Adams, 1982; Dickinson and Balleine, 1995; Balleine and Dickinson, 1998a). It is evident, therefore, that both A-O and S-R processes can contribute to instrumental performance, although they serve quite distinct functions. The A-O process controls initial acquisition and performance of goal-directed actions, whereas the S-R process exerts more control over performance as an action becomes more habitual (cf. Dickinson and Balleine, 1994, 1995).

This functional distinction between the A-O and S-R learning processes has often been interpreted as implying that different memory systems contribute to instrumental conditioning. For example, a number of authors have suggested that A-O associations are encoded in declarative memory, whereas S-R associations are encoded in procedural memory (Winograd, 1975; Dickinson, 1980; Dickinson and Balleine, 1993; Squire and Zola-Morgan, 1996). With respect to the neural bases of instrumental conditioning, considerable recent evidence suggests that declarative memory is dependent on the integrity of the hippocampal formation (Squire, 1992; Eichenbaum et al., 1996; Squire and Zola-Morgan, 1996). This suggests that, in instrumental conditioning, A-O learning is instantiated in declarative memory and therefore, that this form of learning may be hippocampally dependent. This hypothesis predicts that damage to the hippocampus should render animals unable to encode the relationship between actions and their outcomes with the effect that their instrumental performance should be controlled predominantly by the S-R process. As a consequence, the instrumental performance of hippocampal rats should be relatively insensitive to outcome devaluation. This prediction was assessed in experiment 1.

	EXPERIMENT 1A

TOP ABSTRACT INTRODUCTION EXPERIMENT 1A EXPERIMENT 1B EXPERIMENT 2 GENERAL DISCUSSION REFERENCES

In experiment 1A, rats with electrolytic lesions of the dorsal hippocampus and sham-lesioned controls were trained to press two levers, with one lever delivering food pellets and the other delivering a polycose solution. After training, one of these two outcomes was devalued using a specific satiety treatment (Balleine and Dickinson, 1998a) (for review, see Dickinson and Balleine, 1994, 1995). In this procedure, the animals are allowed access to one outcome ad libitum for 1 hr immediately before a choice extinction test conducted on the two levers. In line with previous findings, we anticipated that, on test, sham rats would show an outcome devaluation effect, i.e., they would perform fewer responses on the lever that, in training, delivered the now devalued outcome. In contrast, if, as predicted above, the hippocampus is essential to the formation of A-O associations during instrumental training, then any outcome devaluation effect established in controls should be severely attenuated in the lesioned group.

Method

Subjects and apparatus

The subjects were 17 experimentally naive adult female Long-Evans rats. Subjects were housed singly and were handled daily for 1 week before surgery. Training and testing took place in twelve Med Associates (East Fairfield, VT) operant chambers housed within sound- and light-resistant shells. Each chamber was equipped with a pump fitted with a syringe that delivered 0.1 ml of a 20% polycose solution into a recessed magazine in the chamber. Each chamber was also equipped with a pellet dispenser that delivered one 45 mg Noyes pellet (formula A/I) when activated. The chambers contained two retractable levers that could be inserted to the left and right of the magazine and were illuminated by a 3 W, 24 V house light mounted on the top center of the wall opposite the magazine. Microcomputers equipped with the MED-PC program (Med Associates) controlled the equipment, delivered reinforcers, and recorded the lever presses. For the outcome specific satiety prefeeding, animals received either 50 ml of the polycose solution in a calibrated glass drinking tube affixed to the front of the animal's home cage or a bowl containing 50 gm of Noyes pellets placed inside the home cage.

Surgery

At the time of surgery, animals weighed between 270 and 325 gm. Animals were anesthetized using sodium pentobarbital (Nembutal, 50 mg/kg), treated with atropine (0.1 mg), and then placed in a stereotaxic frame with the incisor bar adjusted so that lambda and bregma were level. Half the subjects received electrolytic lesions of the dorsal hippocampus, and the other half received sham surgery that consisted of lowering the electrode into the hippocampus without running any current. Electrodes, made from insect pins (size 00) covered in Epoxylite except for 1 mm at the tip, were lowered into four sites in the dorsal hippocampus (all coordinates relative to bregma; anteroposterior, 2.8 and 4.2; mediolateral, ±2.0 and ±3.0; dorsoventral, 4.0). When the electrodes were in place, a 1 mA, 20 sec current was passed through the electrode. After surgery, the animals were given 1 week to recover, during which they were handled daily.

Histology 

At the end of the experiment, the animals were scarificed using a lethal barbiturate overdose and perfused transcardially with 0.9% saline, followed by 10% formalin solution. The brains were stored in 10% formalin solution for 48 hr and then transferred to a 25% sucrose-formalin solution before 40 µm coronal sections were cut throughout the region of the hippocampus. The slices were stained using thionin. Slides were examined for placement and extent of the lesion, with the latter assessed by microscopically examining sections for areas of marked cell loss and gliosis.

Procedure

After recovery from surgery, subjects were placed on a food deprivation schedule such that they received 15 gm of their maintenance diet daily to maintain them at ~85% of their free-feeding weight. During training, the animals were fed after each training session. Animals had access to tap water ad libitum while in the home cage. Each session started with the illumination of the house light and insertion of the levers when appropriate and ended with the retraction of the levers and turning off of the house light. All sessions were 30 min in duration.

Magazine training. Initially, all subjects received two 30 min sessions of magazine training in which 15 presentations of each of the two reinforcers were given on a random time 60 sec schedule with the levers withdrawn.

Lever acquisition. In the next session, the animals were trained to press one of the two levers until 100 reinforcers had been earned. The animals were then trained on the other lever until 100 of the alternate reinforcer were earned. Half of the animals were trained with the left lever earning pellets and the right lever earning polycose, and the remaining animals received the opposite action-outcome assignments. In this initial phase of lever training, outcomes were delivered on a fixed interval schedule (FI-20). Once all animals had earned 100 of each reinforcer type, they were shifted to a random ratio (RR) schedule of reinforcement in which the appropriate outcome was delivered at a fixed probability after each action.

Lever training. The animals were first trained on an RR-5 schedule (i.e., each action delivered an outcome with a probability of 0.2). After 3 d of training, this was changed to an RR-10 (or a probability of 0.1) schedule for 3 d and then to an RR-20 schedule (or a probability of 0.05) for an additional 3 d of training. The animals received two training sessions each day, one with each action-outcome pair. The animals had a break of at least 30 min between sessions.

Devaluation test. After the final day of RR-20 training, all of the rats were given access to one of the two outcomes ad libitum for 1 hr in the home cage. Half of the animals in each action-outcome pair assignment received pellets, and the remaining animals received polycose. Immediately after the prefeeding, the animals were placed in the operant chambers. A 20 min choice extinction test was then conducted in which both levers were extended and the number of presses was counted on each lever. No outcomes were delivered during the test.

Results and Discussion

Histology

No recovery problem or weight loss was observed after surgery. In all lesioned animals (n = 8), damage to the dorsal hippocampus was bilateral and complete except at the rostral and caudal extremes. Figure 1 summarizes the maximum and minimum damage resulting from the lesions for the animals included in the behavioral analysis. There was no systematic damage to the overlying cortex observed in either group.

View larger version (40K): [in this window] [in a new window]   Figure 1.   Line drawings of coronal sections from the brains of subjects with the maximum and minimum damage resulting from lesions in the hippocampal group. Starting from the top, sections are taken from the following points in the anteroposterior plane (in millimeters relative to bregma): 2.56, 3.14, 3.60, 4.16, and 4.52. Drawings are from Paxinos and Watson (1998).

Acquisition

No effect of the lesion on the acquisition of lever pressing was observed. The two groups acquired the lever press response in the same number of days. Furthermore, no difference in the level of lever pressing between the lesion [mean (M), 350; SE, 111.8] and sham (M, 408; SE, 100.6) groups was evident at the end of initial training (F < 1).

Devaluation test

Two hippocampal subjects failed to consume the free outcome in the home cage and were excluded from the analysis. The results of the extinction test are illustrated in Figure 2. It is clear from this figure that both the sham and the lesioned groups performed fewer responses on the lever that, in training, earned the subsequently devalued outcome. As such, this figure suggests that a comparable outcome devaluation effect was observed in both groups. This description was confirmed by the statistical analysis. A 2 × 2 mixed ANOVA was conducted with a between-subjects factor of group and a within-subjects factor of devaluation, the later separating performance on the devalued and nondevalued lever. This analysis revealed a significant effect of devaluation (F_(1,13) = 16.562; p < 0.01), but not an effect of group (F < 1) or a group × devaluation interaction (F_(1,13) = 1.406; p > 0.05). Inspection of Figure 2 may suggest that the devaluation effect was smaller in the hippocampal animals, but the hippocampal animals showed numerically smaller (although not statistically different) response levels in training, which may account for the lower response levels in the devaluation test.

View larger version (27K): [in this window] [in a new window]   Figure 2.   Mean lever press responses for the devalued and nondevalued outcomes in a two-lever choice extinction test. Error bars represent the SED for the within-subjects comparison for each group.

The results of experiment 1A failed to confirm the prediction that, in instrumental conditioning, A-O learning depends on the integrity of the hippocampal formation. Despite considerable destruction to the dorsal hippocampus, lesioned rats showed a clear outcome devaluation effect in the choice extinction test.

	EXPERIMENT 1B

TOP ABSTRACT INTRODUCTION EXPERIMENT 1A EXPERIMENT 1B EXPERIMENT 2 GENERAL DISCUSSION REFERENCES

The results of experiment 1A suggest that animals with hippocampal lesions are sensitive to changes in the value of an outcome and can integrate this change in value with an association between a specific action and the paired outcome encoded during training. It has been shown that, in normal animals, the incentive value of an outcome can be mediated by a motivationally arbitrary stimulus, such as taste (Balleine and Dickinson, 1998a; Rescorla, 1990). This is a much more subtle and presumably more difficult discrimination to make. In experiment 1A, the outcomes differed not only in taste but also in texture and nutritional value. The purpose of experiment 1B was to examine whether animals with hippocampal lesions are capable of the more difficult discrimination of outcomes that differ only in their taste.

Method

Subjects and apparatus

Seventeen experimentally naive female Long-Evans rats were housed and maintained under the same conditions and trained in the apparatus as described for experiment 1A, except that a second pump and syringe assembly was added to deliver a second solution. Both pumps delivered a 20% sucrose solution, but for one pump, the solution was flavored with orange Kool-Aid, whereas for the other pump, the solution was flavored with grape Kool-Aid.

Surgery

At the time of surgery, animals weighed between 290 and 380 gm. The surgical and histological procedures were identical to those described for experiment 1A.

Procedure

The procedure was similar to that of experiment 1A with the following exceptions. The left lever earned grape-flavored sucrose and right lever earned orange-flavored sucrose for half the animals, whereas for the remainder, these action-outcome relationships were reversed. The animals were given 2 d of magazine training and then trained on the FI-20 schedule until they earned 100 orange outcomes and then 100 grape outcomes as described for experiment 1A. After acquisition, training on the random ratio schedules was conducted exactly as in experiment 1A, except that the rats received 5 d of RR-5 and 5 d of RR-10 training on each action before testing.

Devaluation test. All animals underwent the outcome specific satiety devaluation treatment described above; each animal received 1 hr access to one of the two outcomes in the home cage ad libitum. Immediately after this exposure period, the animals were placed in the operant chambers and given a single two-lever choice extinction test as described for experiment 1A.

Results and Discussion

Histology

Representative lesions for animals included in the lesion group are displayed in Figure 1.

Training

Over the course of training, both groups acquired the lever press response in the same number of days and, at the end of training, there was no difference in response rate between the lesion (M, 294.5; SE, 47.7) and sham (M, 247.6; SE, 55.6) groups (F < 1).

Devaluation test

Figure 3 illustrates the data from the extinction test. As was observed in experiment 1A, a comparable outcome devaluation effect was observed in both the lesion and sham rats, with both groups animals performing fewer presses on the lever that, in training, had delivered the subsequently devalued outcome. Analysis of the extinction test data reveals a significant effect of devaluation (F_(1,15) = 7.325; p < 0.01), but not an effect of group or a group × lesion interaction (F < 1 for both).

View larger version (32K): [in this window] [in a new window]   Figure 3.   Lever press responses for devalued and nondevalued outcomes in a two-lever choice extinction test. Error bars represent the SED for each group.

This finding replicates that of experiment 1A and indicates that animals with lesions of the dorsal hippocampus are able to discriminate and encode the value of outcomes based on a single, motivationally irrelevant flavor feature. Additionally, it should be noted that there is no suggestion that the devaluation effect may be smaller in hippocampal animals as may have been suspected in experiment 1A. Together, experiments 1A and 1B provide consistent evidence against the suggestion that the hippocampus mediates the encoding of the action-outcome association in memory. Indeed, these experiments provide no evidence whatever to suggest that the hippocampus plays a role in instrumental conditioning. It has been suggested by White and colleagues (McDonald and White, 1993) that there are multiple memory systems that may have different roles in specific tasks, and so, it remains possible that sensitivity to outcome devaluation may be dependent on another neural structure, such as the amygdala.

	EXPERIMENT 2

TOP ABSTRACT INTRODUCTION EXPERIMENT 1A EXPERIMENT 1B EXPERIMENT 2 GENERAL DISCUSSION REFERENCES

There are a number of features of instrumental conditioning that stand discordant with a simple S-R approach and that support the view that animals can encode the relationship between an action and its consequences. The first of these has been described above, namely demonstrations that instrumental conditioning is sensitive to devaluation of the instrumental outcome. A second demonstration of this same principle comes from experiments indicating that the instrumental performance of animals is sensitive to changes in the contingency between a particular action and its consequences (Hammond, 1980; Dickinson and Mulatero, 1989; Balleine and Dickinson, 1998b).

In one study, for example, Balleine and Dickinson (1998b) found that, when the response-outcome contingency was degraded for one of two action-outcome pairs, animals selectively decreased performance of that response. With respect to the current analysis, this demonstration is important because it suggests that, although hippocampal animals may be able to encode adventitious action-outcome relationships that they are exposed to in training, it is still possible that they remain insensitive to the causal consequences of their actions. There is, in fact, some evidence that accords with this suggestion. Devenport (1979, 1980) and Devenport and Holloway (1980) argued that the ability of rats to detect causal, as opposed to adventitious, relationships in their environment depends on the hippocampus; that, without an intact hippocampus, instrumental performance is controlled solely by response-reward contiguity as envisaged within S-R reinforcement theory. In support of this claim, Devenport and Holloway (1980) found that rats with hippocampal lesions and trained to lever press for food continued to press at high rates, even when the causal relationship between action and outcome was removed and reward delivery was shifted from a random-interval to a random-time schedule.

Although suggestive, there are several features of these experiments that make this conclusion premature. The results of Devenport and Holloway's (1980) study are particularly open to question because, in their study, the hippocampal lesioned group was pressing at much higher rates than intact controls before the shift to the random time schedule. Response-reward contiguity is positively related to baseline response rate on random-time schedules, and so there are good grounds for suggesting that the effectiveness of the shift in contingency differed between groups. If this is the case, the claim that the instrumental performance of hippocampal animals suffered any general loss of control by the action-outcome contingency must be reexamined.

In experiment 2, therefore, we assessed whether the hippocampal rats in our studies were sensitive to the causal consequences of their actions using a procedure pioneered by Hammond (1980) but as modified in the experiment of Balleine and Dickinson (1998b). To achieve this, the rats used in experiment 1 were given several sessions of retraining after the outcome devaluation test, after which they were given several training sessions in which one of the two instrumental action-outcome contingencies was degraded. Finally, the rats were tested in extinction on the levers to assess the impact of the shift in contingency. If, as suggested by Devenport and Holloway (1980), the hippocampus is critical for encoding the causal relationship between an action and outcome, hippocampal lesions should significantly impair the sensitivity of rats to this shift in the instrumental contingency. As such, although it is predicted that sham rats should reduce performance of the response that is no longer causal with respect to its outcome relative to the other response, hippocampal rats should not show this effect and perform both responses at comparable rates in the extinction test.

Method

Subjects and apparatus

The subjects were those described for experiment 1A above. The equipment and general apparatus were also the same as that described above.

Surgery

Surgical procedures are described in experiment 1.

Procedure

Contingency degradation training. After the training and testing described for experiment 1A, the animals were retrained on RR-20 schedules for 2 d, after which one of the two instrumental contingencies was degraded. At the end of training, each lever earned a unique outcome (pellets or a 20% polycose solution) with a fixed probability, p(O/A) = 0.05. In subsequent sessions, in addition to being earned by one of the actions, one of the outcomes was now also delivered noncontingently with the same probability [p(O/A) = 0.05] in each second without a response. For one lever, the free reinforcer was the same as that which was earned by a response on that lever. Thus, the experienced probability of the delivery of that particular outcome was the same whether or not the animals performed that action, a procedure that should have acted to degrade that action-outcome contingency. For the other lever, the free reinforcer was different from the earned reinforcer, and so this contingency was nondegraded. For half of the animals, the degraded contingency involved pellet delivery, whereas for the remainder, it involved the delivery of the polycose solution. The animals had two 30 min training sessions per day, one on each lever and, hence, on each contingency. The animals had a break of ~1 hr between sessions, and the order of the sessions was alternated each day. This training continued for 3 d.

Contingency test. On the day after the final day of contingency training, rats in both groups received a choice extinction test. The test began with the insertion of the levers and the onset of the house light and ended 20 min later with the retraction of the levers and the offset of the house light. No outcomes were presented during this session.

Results and Discussion

Histology

Representative lesions displaying the maximum and minimum damage resulting from the lesions are represented in Figure 1.

Contingency degradation training

Figure 4 shows the effects of contingency degradation across training days on the lever press response. The animals were sensitive to the degradation of the instrumental contingency, performing more responses on the lever for which the contingency had not been degraded. The statistical analysis of the contingency training data revealed a main effect of contingency (F_(1,15) = 10.540; p < 0.01), a main effect of training day (F_(2,30) = 12.241; p < 0.01), but no main effect of group (F < 1). Examination of Figure 4 suggests that the hippocampal animals were not sensitive to the degradation of the instrumental contingency; however, the group × contingency interaction failed to reach significance (F_(1,15) = 1.660; p > 0.05). None of the other interactions were significant.

View larger version (20K): [in this window] [in a new window]   Figure 4.   Lever responses for the degraded and nondegraded outcomes across days of contingency degradation training in hippocampal (left) and sham-lesioned (right) animals.

Extinction test

The critical data from the experiment, those arising from the choice extinction test, are presented in Figure 5. It is clear that the two groups performed very differently on this test. Whereas the sham group showed a clear difference in performance of the action whose contingency was degraded relative to the other action, this was not true of the hippocampal rats, which appeared to perform both actions at a comparable rate. The analysis of the extinction test data confirmed this description. This analysis revealed no main effect of contingency (F_(1,15) = 2.204; p > 0.05) and a marginal main effect of group (F_(1,15) = 3.948; p = 0.06), but, most importantly, there was a significant group × contingency interaction (F_(1,15) = 5.862; p = 0.02). Simple main effects analysis revealed that, for the sham group, there is a significant difference induced by the degraded and nondegraded contingency treatments (F₍₈₎ = 7.88; p < 0.01). As shown in Figure 5, these animals performed fewer presses on lever for which the contingency had been degraded. However, for the lesion group, there was no difference in response rate on the two levers after the contingency treatment (F < 1).

View larger version (24K): [in this window] [in a new window]   Figure 5.   Lever responses for hippocampal and sham-lesioned animals in a two-lever choice extinction test after the selective degradation of one of the instrumental contingencies. Error bars represent the SED for each group.

These results are consistent with the suggestion that the hippocampal lesion rendered rats insensitive to the causal consequences of their instrumental actions. Thus, whereas the sham rats showed considerable sensitivity to the selective degradation of a specific instrumental contingency, the hippocampal animals did not appear sensitive to this manipulation and continued to perform both actions at a comparable and somewhat depressed rate.

	GENERAL DISCUSSION

TOP ABSTRACT INTRODUCTION EXPERIMENT 1A EXPERIMENT 1B EXPERIMENT 2 GENERAL DISCUSSION REFERENCES

The general aim of this study was to assess the effects of dorsal hippocampal lesions on instrumental conditioning. Specifically, we investigated the effects of these lesions on the acquisition and maintenance of instrumental performance, sensitivity to changes in the value of the instrumental outcome, and sensitivity to degradation of the instrumental contingency.

The results indicate that these hippocampal lesions had no effect on the acquisition of the instrumental action. The lesioned animals did not differ from shams in either the rate of acquisition of the lever press response or the level of responding at the end of training. Furthermore, the hippocampal lesioned rats were just as sensitive as sham controls to devaluation of the instrumental outcome. Indeed, in experiment 1B, it was found that lesioned animals still showed this selective devaluation effect even when the sole factor discriminating between the two outcomes was a motivationally irrelevant factor: the flavor. These results provide evidence that animals with lesions of the hippocampus both encode the value of an outcome and are able to establish an A-O association that controls their selective responding on test.

Of greater interest, however, is the finding of experiment 2 that hippocampal rats were relatively insensitive to treatments that ensured the degradation of the action-outcome contingency. When evaluated in a two-lever choice extinction test, intact animals decreased responding on the lever previously associated with the noncontingent reinforcer relative to the other lever, whereas rats with hippocampal lesions continued to respond at equal rates on both levers. These results tend to confirm, therefore, the suggestion of Devenport and colleagues (Devenport, 1979; Devenport and Holloway, 1980) that the hippocampus is critical for the detection of the causal relationship between an action and outcome. As suggested by experiment 2, without an intact hippocampus, animals continue to respond as though reward is dependent on their actions, even when one action is no longer causal with respect to the delivery of its specific outcome. Thus, it appears that the hippocampus plays a relatively specific role in instrumental conditioning; without an intact hippocampus, animals seem unable to differentiate between actions that are causal with respect to their associated outcomes and those that are merely adventitiously related. Because it is the learning of this relationship that critically distinguishes instrumental learning from other forms of relational learning, this interpretation may be thought to accord with the suggested role of the hippocampus in declarative memory (Squire and Zola-Morgan, 1996).

However, when considered together with the data from the devaluation experiments above, the interpretation becomes less clear. In experiments 1A and 1B, it was demonstrated that animals with hippocampal lesions selectively decrease their responding after the devaluation of one instrumental outcome. This shows that they must still depend on an intact A-O association. The deficit in hippocampal animals is not their ability to associate their actions with subsequent outcomes but their sensitivity to the probability of outcome delivery in the absence of an action. An alternative explanation is therefore that animals with hippocampal lesions are unable to calculate background rates of reinforcement. Whereas contingent reinforcers are best predicted by an A-O association, noncontingent reinforcers are better predicted by the training context, and thus the observed deficit in hippocampal animals may be attributable to a failure to form a context-outcome association. Rather than mediating the encoding of causal as opposed to chance relationships between events, the role of the hippocampus is often thought to be to encode the context within which two events are related (Jarrard, 1995; Holland and Bouton, 1999). It is important, therefore, to consider the degree to which the current data accord with this kind of position.

Although often obscured within cognitive analyses of relational learning, considerable evidence suggests that the ability to encode situational cues (i.e., contexts) is essential if animals are to discern causal from merely contiguous relationships between events. This point was recognized quite early on in the associative analysis of Pavlovian conditioning and subsequently encapsulated within the concept of predictive validity (Dweck and Wagner, 1970; Rescorla and Wagner, 1972). According to this position, the learning of predictive relationships between events is subject to a competitive process; encoding one event as predictive of another is subject to competition from other events and so is determined by how good a predictor it is relative to those other events. If, for example, a hungry rat is strongly motivated to predict the delivery of food, then learning that food follows the presentation of, say, a tone would accord with that need, and an association between the tone and delivery of food is likely to be formed. However, if the food is also delivered at other times in the absence of the tone, this association is expected to be substantially weaker. It was argued, and subsequently supported by experimental evidence, that degrading the predictive status of the tone was based on the animal learning to encode the relationship between the food and the background or situational cues in which the noncontiguous food presentations occurred (Dweck and Wagner, 1970; Baker, 1990).

In the current situation, a comparable analysis is available. Thus, for example, when the delivery of a specific food outcome follows the performance of an instrumental action and at no other time, the rat appears to learn that the performance of the action is causally related to the delivery of that outcome. However, when the outcome is also delivered unpaired with performance of the action, any association between the context and the food will naturally compete with the action for association with food delivery and so degrade its causal relationship with the outcome. Indeed, when an outcome is equally probable in the presence of an action (A) and in the presence of a context cue (C) alone, it is clear, given that the action is also performed in C (i.e., AC), that C is a better predictor of the food than is A.

This argument suggests that sensitivity to noncontingent outcome delivery should strongly depend on the ability of rats to form context-outcome associations. If hippocampal animals fail to form an association between the context and the noncontingent outcome, they may associate the delivery of both outcomes only with their actions. This account would explain why the lesioned animals in the current study maintain equal response rates for the contingent and noncontingent outcomes and is consistent with the suggestion that the hippocampus is important for contextual retrieval (Hirsh, 1974).

There are, in fact, numerous experiments that suggest that animals with lesions of the hippocampus show deficits in contextual conditioning (Honey and Good, 1993; Maren and Fanselow, 1997) (but see Holland, 1997). For example, Honey and Good (1993) found that hippocampal lesions did not impair the initial acquisition of conditioned responding but did disrupt the contextual specificity of conditioning in a latent inhibition paradigm. Additionally, there is evidence that, despite their impairment in contextual conditioning, hippocampal animals remain able to form associations with discrete cues (Jarrard, 1993). Related findings have been reported by Winocur and Olds (1978) who found that a reversal learning deficit in hippocampal animals could be reduced if external cues were added, which served to enhance the discriminability of successive tasks and minimized the need for context retrieval. This argument predicts, therefore, that if a discrete cue were used to signal delivery of both the outcomes that are paired with performance of the instrumental action and those that are unpaired, this cue would be a better predictor of that outcome than either the action or the context. Given that hippocampal animals are able to form associations with such cues, this should alleviate their deficit, and these lesioned animals should then to be as sensitive to a change in the instrumental contingency as intact controls.

In summary, lesions of the dorsal hippocampus produce an interesting set of effects on instrumental performance. Although the initial acquisition of the instrumental response appears normal, animals with hippocampal lesions do not modify their behavior when the A-O contingency is selectively degraded. This is not because of a simple failure to encode the difference between two actions, two outcomes, or even the A-O association because these animals were shown to be able to selectively control their responding when one of the two outcomes was devalued. What appears to be impaired, therefore, is the ability of the hippocampal animals to distinguish between contingent and free reinforcers. Although this may suggest that hippocampal animals are unable to encode causal relationships between their actions and their consequences, it remains a possibility that this effect reflects a failure of hippocampal animals to calculate background rates of reinforcement based on the formation of context-outcome associations. It is an important aim for future experimentation to differentiate between these two accounts of the role of the hippocampus in instrumental conditioning.

	FOOTNOTES

Received Feb. 9, 2000; revised March 13, 2000; accepted March 13, 2000.

This work was supported by National Institute of Mental Health Grant MH 56446. We thank Sandra Cetl and Chris Park who assisted with data collection.

Correspondence should be addressed to Laura Corbit, Department of Psychology, University of California, Los Angeles, Box 951563, Los Angeles, CA 90095. E-mail: corbit{at}ucla.edu.

	REFERENCES

TOP ABSTRACT INTRODUCTION EXPERIMENT 1A EXPERIMENT 1B EXPERIMENT 2 GENERAL DISCUSSION REFERENCES

• Adams CD (1982) Variations in the sensitivity of instrumental responding to reinforcer devaluation. Q J Exp Psychol 34B:77-98.
• Baker AG (1990) Contextual conditioning during free-operant extinction: unsignaled, signaled, and backward-signaled noncontingent food. Anim Learn Behav 18:59-70.
• Balleine BW, Dickinson A (1998a) The role of incentive learning in instrumental outcome revaluation by sensory-specific satiety. Anim Learn Behav 26:46-59.
• Balleine BW, Dickinson A (1998b) Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology 37:407-419[Web of Science][Medline].
• Colwill RC, Rescorla RA (1986) Associative structures in instrumental learning. In: The psychology of learning and motivation, Vol 20 (Bower GH, ed), pp 55-104. New York: Academic.
• Devenport LD (1979) Superstitious bar pressing in hippocampal and septal rats. Science 205:721-723[Abstract/Free Full Text].
• Devenport LD (1980) Response-reinforcer relations and the hippocampus. Behav Neural Biol 29:105-110[Web of Science][Medline].
• Devenport LD, Holloway FA (1980) The rat's resistance to superstition: role of the hippocampus. J Comp Physiol Psychol 4:691-705.
• Dickinson A (1980) In: Contemporary animal learning theory. Cambridge: Cambridge UP.
• Dickinson A, Balleine BW (1993) Actions and responses: the dual psychology of behaviour. In: Spatial representation (Eilan N, McCarthy R, Brewer MW, eds), pp 277-293. Oxford: Basil Blackwell.
• Dickinson A, Balleine BW (1994) Motivational control of goal-directed action. Anim Learn Behav 22:1-18.
• Dickinson A, Balleine BW (1995) Motivational control of instrumental action. Curr Direct Psychol Sci 4:162-167.
• Dickinson A, Mulatero CW (1989) Reinforcer specificity of the suppression of instrumental performance on a non-contingent schedule. Behav Processes 19:167-180.
• Dweck CS, Wagner AR (1970) Situational cues and correlation between CS and US as determinants of the conditioned emotional response. Psychon Sci 18:145-147.
• Eichenbaum H, Schoenbaum G, Young B, Bunsey M (1996) Functional organization of the hippocampal memory system. Proc Natl Acad Sci USA 93:13500-13507[Abstract/Free Full Text].
• Hammond LJ (1980) The effect of contingency upon the appetitive conditioning of free-operant behavior. J Exp Anal Behav 34:297-304[Medline].
• Hirsh R (1974) The hippocampus and contextual retrieval from memory: a theory. Behav Biol 12:421-444[Web of Science][Medline].
• Holland PC (1997) Brain mechanisms for changes in processing of conditioned stimuli in Pavlovian conditioning: implications for behavior theory. Anim Learn Behav 25:373-399.
• Holland PC, Bouton ME (1999) Hippocampus and context in classical conditioning. Curr Opin Neurobiol 9:195-202[Web of Science][Medline].
• Honey RC, Good M (1993) Selective hippocampal lesions abolish the contextual specificity of latent inhibition and conditioning. Behav Neurosci 107:23-33[Web of Science][Medline].
• Jarrard LE (1993) On the role of the hippocampus in learning and memory in the rat. Behav Neural Biol 60:9-26[Web of Science][Medline].
• Jarrard LE (1995) What does the hippocampus really do? Behav Brain Res 71:1-10[Web of Science][Medline].
• Maren S, Fanselow MS (1997) Electrolytic lesions of the fimbria/fornix, dorsal hippocampus, or entorhinal cortex produce anterograde deficits in contextual fear conditioning in rats. Neurobiol Learn Mem 67:142-149[Web of Science][Medline].
• McDonald RJ, White NM (1993) A triple dissociation of memory systems: Hippocampus, amygdala, and dorsal striatum. Behav Neurosci 107:3-22[Web of Science][Medline].
• Paxinos G, Watson C (1998) In: The rat brain in stereotaxic coordinates. San Diego: Academic.
• Rescorla RA (1990) Instrumental responses become associated with reinforcers that differ in one feature. Anim Learn Behav 18:206-211.
• Rescorla RA, Wagner AR (1972) A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and non-reinforcement. In: Classical conditioning. II. Current research and Theory (Black AH, Prokasy WF, eds), pp 64-99. New York: Appelton-Century-Crofts.
• Squire LR (1992) Memory and the hippocampus: a synthesis from findings with rats, monkeys, and humans. Psychol Rev 99:195-231[Web of Science][Medline].
• Squire LR, Zola-Morgan S (1996) Structure and function of declarative and nondeclarative memory systems. Proc Natl Acad Sci 93:13515-13522[Abstract/Free Full Text].
• Winograd T (1975) Frames representations and the declarative-procedural controversy. In: Representation and understanding (Bobrow DG, Collins A, eds), pp 185-210. New York: Academic.
• Winocur G, Olds J (1978) Effects of context manipulation on memory and reversal learning in rats with hippocampal lesions. J Comp Physiol Psychol 92:312-321[Web of Science][Medline].

---

Copyright © 2000 Society for Neuroscience  0270-6474/00/20114233-07$05.00/0

This article has been cited by other articles:

				B. Lex and W. Hauber The Role of Dopamine in the Prelimbic Cortex and the Dorsomedial Striatum in Instrumental Conditioning Cereb Cortex, August 12, 2009; (2009) bhp151v2. [Abstract] [Full Text] [PDF]

				J. Lisman and A.D. Redish Prediction, sequences and the hippocampus Phil Trans R Soc B, May 12, 2009; 364(1521): 1193 - 1201. [Abstract] [Full Text] [PDF]

				B. W. Balleine, A. S. Killcross, and A. Dickinson The Effect of Lesions of the Basolateral Amygdala on Instrumental Conditioning J. Neurosci., January 15, 2003; 23(2): 666 - 675. [Abstract] [Full Text] [PDF]

				L. H. Corbit, S. B. Ostlund, and B. W. Balleine Sensitivity to Instrumental Contingency Degradation Is Mediated by the Entorhinal Cortex and Its Efferents via the Dorsal Hippocampus J. Neurosci., December 15, 2002; 22(24): 10976 - 10984. [Abstract] [Full Text] [PDF]

				L. H. Corbit, J. L. Muir, and B. W. Balleine The Role of the Nucleus Accumbens in Instrumental Conditioning: Evidence of a Functional Dissociation between Accumbens Core and Shell J. Neurosci., May 1, 2001; 21(9): 3251 - 3260. [Abstract] [Full Text] [PDF]

				B. W. Balleine and A. Dickinson The Effect of Lesions of the Insular Cortex on Instrumental Conditioning: Evidence for a Role in Incentive Memory J. Neurosci., December 1, 2000; 20(23): 8954 - 8964. [Abstract] [Full Text] [PDF]

This Article Abstract Full Text (PDF) Submit an eLetter Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in Web of Science Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Web of Science (33) Citing Articles via Google Scholar Google Scholar Articles by Corbit, L. H. Articles by Balleine, B. W. Search for Related Content PubMed PubMed Citation Articles by Corbit, L. H. Articles by Balleine, B. W.

Home  |   Search  |   Archive  |   Subscribe  |   Contact  |   Help