Skip to main content

Main menu

  • HOME
  • CONTENT
    • Early Release
    • Featured
    • Current Issue
    • Issue Archive
    • Collections
    • Podcast
  • ALERTS
  • FOR AUTHORS
    • Information for Authors
    • Fees
    • Journal Clubs
    • eLetters
    • Submit
  • EDITORIAL BOARD
  • ABOUT
    • Overview
    • Advertise
    • For the Media
    • Rights and Permissions
    • Privacy Policy
    • Feedback
  • SUBSCRIBE

User menu

  • Log in
  • My Cart

Search

  • Advanced search
Journal of Neuroscience
  • Log in
  • My Cart
Journal of Neuroscience

Advanced Search

Submit a Manuscript
  • HOME
  • CONTENT
    • Early Release
    • Featured
    • Current Issue
    • Issue Archive
    • Collections
    • Podcast
  • ALERTS
  • FOR AUTHORS
    • Information for Authors
    • Fees
    • Journal Clubs
    • eLetters
    • Submit
  • EDITORIAL BOARD
  • ABOUT
    • Overview
    • Advertise
    • For the Media
    • Rights and Permissions
    • Privacy Policy
    • Feedback
  • SUBSCRIBE
PreviousNext
Featured ArticleResearch Articles, Behavioral/Cognitive

Reward-Mediated, Model-Free Reinforcement-Learning Mechanisms in Pavlovian and Instrumental Tasks Are Related

Neema Moin Afshar, François Cinotti, David Martin, Mehdi Khamassi, Donna J. Calu, Jane R. Taylor and Stephanie M. Groman
Journal of Neuroscience 18 January 2023, 43 (3) 458-471; DOI: https://doi.org/10.1523/JNEUROSCI.1113-22.2022
Neema Moin Afshar
1Department of Psychiatry, Yale School of Medicine, New Haven, Connecticut 06511
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
François Cinotti
2Department of Experimental Psychology, University of Oxford, Oxford OX2 6GG, United Kingdom
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for François Cinotti
David Martin
3Department of Anatomy and Neurobiology, University of Maryland School of Medicine, Baltimore, Maryland 21201
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Mehdi Khamassi
4Institute of Intelligent Systems and Robotics, Centre National de la Recherche Scientifique, Sorbonne University, 75005 Paris, France
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Mehdi Khamassi
Donna J. Calu
3Department of Anatomy and Neurobiology, University of Maryland School of Medicine, Baltimore, Maryland 21201
5Program in Neuroscience, University of Maryland School of Medicine, Baltimore, Maryland 21201
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jane R. Taylor
1Department of Psychiatry, Yale School of Medicine, New Haven, Connecticut 06511
6Department of Psychology, Yale University, New Haven, Connecticut 06520
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Stephanie M. Groman
1Department of Psychiatry, Yale School of Medicine, New Haven, Connecticut 06511
7Department of Neuroscience, University of Minnesota Medical School, Minneapolis, Minnesota 55455
8Department of Psychology, University of Minnesota, Minneapolis, Minnesota 55455
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Stephanie M. Groman
  • Article
  • Figures & Data
  • Info & Metrics
  • eLetters
  • PDF
Loading

Abstract

Model-free and model-based computations are argued to distinctly update action values that guide decision-making processes. It is not known, however, if these model-free and model-based reinforcement learning mechanisms recruited in operationally based instrumental tasks parallel those engaged by pavlovian-based behavioral procedures. Recently, computational work has suggested that individual differences in the attribution of incentive salience to reward predictive cues, that is, sign- and goal-tracking behaviors, are also governed by variations in model-free and model-based value representations that guide behavior. Moreover, it is not appreciated if these systems that are characterized computationally using model-free and model-based algorithms are conserved across tasks for individual animals. In the current study, we used a within-subject design to assess sign-tracking and goal-tracking behaviors using a pavlovian conditioned approach task and then characterized behavior using an instrumental multistage decision-making (MSDM) task in male rats. We hypothesized that both pavlovian and instrumental learning processes may be driven by common reinforcement-learning mechanisms. Our data confirm that sign-tracking behavior was associated with greater reward-mediated, model-free reinforcement learning and that it was also linked to model-free reinforcement learning in the MSDM task. Computational analyses revealed that pavlovian model-free updating was correlated with model-free reinforcement learning in the MSDM task. These data provide key insights into the computational mechanisms mediating associative learning that could have important implications for normal and abnormal states.

SIGNIFICANCE STATEMENT Model-free and model-based computations that guide instrumental decision-making processes may also be recruited in pavlovian-based behavioral procedures. Here, we used a within-subject design to test the hypothesis that both pavlovian and instrumental learning processes were driven by common reinforcement-learning mechanisms. Sign-tracking and goal-tracking behaviors were assessed in rats using a pavlovian conditioned approach task, and then instrumental behavior was characterized using an MSDM task. We report that sign-tracking behavior was associated with greater model-free, but not model-based, learning in the MSDM task. These data suggest that pavlovian and instrumental behaviors may be driven by conserved reinforcement-learning mechanisms.

  • computational psychiatry
  • decision-making
  • incentive salience
  • model-based learning
  • model-free learning

SfN exclusive license.

View Full Text

Member Log In

Log in using your username and password

Enter your Journal of Neuroscience username.
Enter the password that accompanies your username.
Forgot your user name or password?

Purchase access

You may purchase access to this article. This will require you to create an account if you don't already have one.
Back to top

In this issue

The Journal of Neuroscience: 43 (3)
Journal of Neuroscience
Vol. 43, Issue 3
18 Jan 2023
  • Table of Contents
  • Table of Contents (PDF)
  • About the Cover
  • Index by author
  • Masthead (PDF)
Email

Thank you for sharing this Journal of Neuroscience article.

NOTE: We request your email address only to inform the recipient that it was you who recommended this article, and that it is not junk mail. We do not retain these email addresses.

Enter multiple addresses on separate lines or separate them with commas.
Reward-Mediated, Model-Free Reinforcement-Learning Mechanisms in Pavlovian and Instrumental Tasks Are Related
(Your Name) has forwarded a page to you from Journal of Neuroscience
(Your Name) thought you would be interested in this article in Journal of Neuroscience.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Print
View Full Page PDF
Citation Tools
Reward-Mediated, Model-Free Reinforcement-Learning Mechanisms in Pavlovian and Instrumental Tasks Are Related
Neema Moin Afshar, François Cinotti, David Martin, Mehdi Khamassi, Donna J. Calu, Jane R. Taylor, Stephanie M. Groman
Journal of Neuroscience 18 January 2023, 43 (3) 458-471; DOI: 10.1523/JNEUROSCI.1113-22.2022

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Respond to this article
Request Permissions
Share
Reward-Mediated, Model-Free Reinforcement-Learning Mechanisms in Pavlovian and Instrumental Tasks Are Related
Neema Moin Afshar, François Cinotti, David Martin, Mehdi Khamassi, Donna J. Calu, Jane R. Taylor, Stephanie M. Groman
Journal of Neuroscience 18 January 2023, 43 (3) 458-471; DOI: 10.1523/JNEUROSCI.1113-22.2022
Reddit logo Twitter logo Facebook logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • Abstract
    • Introduction
    • Materials and Methods
    • Results
    • Discussion
    • Footnotes
    • References
  • Figures & Data
  • Info & Metrics
  • eLetters
  • PDF

Keywords

  • computational psychiatry
  • decision-making
  • incentive salience
  • model-based learning
  • model-free learning

Responses to this article

Respond to this article

Jump to comment:

No eLetters have been published for this article.

Related Articles

Cited By...

More in this TOC Section

Research Articles

  • Face-selective patches in marmosets are involved in dynamic and static facial expression processing
  • Sex differences in the impact of electronic nicotine vapor on corticotropin-releasing factor receptor 1 neurons in the mouse ventral tegmental area
  • CIB2 and CIB3 regulate stereocilia maintenance and mechanoelectrical transduction in mouse vestibular hair cells
Show more Research Articles

Behavioral/Cognitive

  • A learned map for places and concepts in the human MTL
  • Genetic Disruption of System xc-Mediated Glutamate Release from Astrocytes Increases Negative-Outcome Behaviors While Preserving Basic Brain Function in Rat
  • Neural Substrates of Body Ownership and Agency during Voluntary Movement
Show more Behavioral/Cognitive
  • Home
  • Alerts
  • Visit Society for Neuroscience on Facebook
  • Follow Society for Neuroscience on Twitter
  • Follow Society for Neuroscience on LinkedIn
  • Visit Society for Neuroscience on Youtube
  • Follow our RSS feeds

Content

  • Early Release
  • Current Issue
  • Issue Archive
  • Collections

Information

  • For Authors
  • For Advertisers
  • For the Media
  • For Subscribers

About

  • About the Journal
  • Editorial Board
  • Privacy Policy
  • Contact
(JNeurosci logo)
(SfN logo)

Copyright © 2023 by the Society for Neuroscience.
JNeurosci Online ISSN: 1529-2401

The ideas and opinions expressed in JNeurosci do not necessarily reflect those of SfN or the JNeurosci Editorial Board. Publication of an advertisement or other product mention in JNeurosci should not be construed as an endorsement of the manufacturer’s claims. SfN does not assume any responsibility for any injury and/or damage to persons or property arising from or related to any use of any material contained in JNeurosci.