Invited reviewTransition from ‘model-based’ to ‘model-free’ behavioral control in addiction: Involvement of the orbitofrontal cortex and dorsolateral striatum
Section snippets
Differential roles of the dorsolateral striatum and orbitofrontal cortex in model-free versus model-based behavior
The model-free system endows cues with pre-computed value based on their history of reinforcement. As a result, while model-free learning is fast and efficient on one hand, it is also limited in its flexibility (Niv et al., 2006). This limitation is easily captured by experiments in which decisions or learning are especially dependent on new or specific information regarding expected outcome rather than that pre-computed value. Typically such experiments involve reinforcer devaluation (Holland
Altered model-free and model-based representations by cocaine exposure
We have seen that overtraining makes performance insensitive to changes in reward value in reinforcer devaluation tasks (Holman, 1975, Colwill and Rescorla, 1988, Balleine and Dickinson, 1998, Killcross and Coutureau, 2003, Tricomi et al., 2009) and that the DLS is involved in this deficit (Yin et al., 2004). Nelson and Killcross (2006) have demonstrated that animals sensitized with amphetamine show no devaluation effect, even after limited training (Fig. 1). Animals were sensitized with
Conclusions
Cocaine addiction is characterized by poor judgment and maladaptive decision-making. Here we have reviewed evidence to support the proposal that such maladaptive behaviors associated in addicted subjects may be due, at least in part, to a drug-induced change in the balance between the model-based and model-free systems (Fig. 6). The transition between the two systems is consistent with impairments in the ability to generate and retrieve specific information about expected outcomes and an
Acknowledgments
The work of writing it was supported by funding from the intramural programs of NIDA. Correspondence should be addressed to G.S. ([email protected]).
References (94)
- et al.
Goal-directed instrumental action: contingency and incentive learning and their cortical substrates
Neuropharmacology
(1998) - et al.
Dissecting components of reward: ‘liking’, ‘wanting’, and learning
Curr. Opin. Pharmacol.
(2009) - et al.
Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex
Neurosci. Biobehav. Rev.
(2002) - et al.
Differential effects of inactivation of the orbitofrontal cortex on strategy set-shifting and reversal learning
Neurobiol. Learn. Mem.
(2008) - et al.
Delay discounting in currently using and currently abstinent cocaine-dependent outpatients and non-drug-using matched controls
Addict. Behav.
(2006) Some conditions for the dissociation of consummatory and instrumental behavior in rats
Learn. Motiv.
(1975)- et al.
Impairments of reversal learning and response perseveration after repeated, intermittent cocaine administrations to monkeys
Neuropsychopharmacology
(2002) - et al.
Temporal difference models and reward-related learning in the human brain
Neuron
(2003) Cognitive planning in humans: neuropsychological, neuroanatomical and neuropharmacological perspectives
Prog. Neurobiol.
(1997)- et al.
Frontal cortex and reward-guided learning and decision-making
Neuron
(2011)