Skip to main content

Main menu

  • HOME
  • CONTENT
    • Early Release
    • Featured
    • Current Issue
    • Issue Archive
    • Collections
    • Podcast
  • ALERTS
  • FOR AUTHORS
    • Information for Authors
    • Fees
    • Journal Clubs
    • eLetters
    • Submit
    • Special Collections
  • EDITORIAL BOARD
    • Editorial Board
    • ECR Advisory Board
    • Journal Staff
  • ABOUT
    • Overview
    • Advertise
    • For the Media
    • Rights and Permissions
    • Privacy Policy
    • Feedback
    • Accessibility
  • SUBSCRIBE

User menu

  • Log out
  • Log in
  • My Cart

Search

  • Advanced search
Journal of Neuroscience
  • Log out
  • Log in
  • My Cart
Journal of Neuroscience

Advanced Search

Submit a Manuscript
  • HOME
  • CONTENT
    • Early Release
    • Featured
    • Current Issue
    • Issue Archive
    • Collections
    • Podcast
  • ALERTS
  • FOR AUTHORS
    • Information for Authors
    • Fees
    • Journal Clubs
    • eLetters
    • Submit
    • Special Collections
  • EDITORIAL BOARD
    • Editorial Board
    • ECR Advisory Board
    • Journal Staff
  • ABOUT
    • Overview
    • Advertise
    • For the Media
    • Rights and Permissions
    • Privacy Policy
    • Feedback
    • Accessibility
  • SUBSCRIBE
PreviousNext
Articles, Behavioral/Cognitive

Hierarchical Learning Induces Two Simultaneous, But Separable, Prediction Errors in Human Basal Ganglia

Carlos Diuk, Karin Tsai, Jonathan Wallis, Matthew Botvinick and Yael Niv
Journal of Neuroscience 27 March 2013, 33 (13) 5797-5805; https://doi.org/10.1523/JNEUROSCI.5445-12.2013
Carlos Diuk
1Psychology Department and Princeton Neuroscience Institute, Princeton University, Princeton, New Jersey 08544,
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Karin Tsai
2Computer Science Department, Princeton University, Princeton, New Jersey 08540, and
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jonathan Wallis
3Department of Psychology and Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, California 94720
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Matthew Botvinick
1Psychology Department and Princeton Neuroscience Institute, Princeton University, Princeton, New Jersey 08544,
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Yael Niv
1Psychology Department and Princeton Neuroscience Institute, Princeton University, Princeton, New Jersey 08544,
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Figures & Data
  • Info & Metrics
  • eLetters
  • PDF
Loading

Article Figures & Data

Figures

  • Tables
  • Figure 1.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 1.

    Sample trial: the participant chooses to play in the left casino, the door opens and displays a target number of points (indicated by red bar). After 2.5–3.5 s, the four slot machines appear. The participant plays upper-left slot and, after another 2.5–3.5 s, the points obtained in that machine are shown inside the machine (as a green bar plus a roman numeral). The corresponding part of the target points turns yellow, indicating the points accumulated with the first slot machine play. The rest is still red, indicating the points still necessary to win the casino. The participant plays the bottom-right slot machine and obtains sufficient points to win the casino. The target bar turns green and a message appears indicating the casino win (10¢).

  • Figure 2.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 2.

    HRL representation of the casino task. The top level shows the task of playing a casino, and the bottom level decomposes this task into the subtasks of playing slot machines. Prediction errors under the Outcome Model and Slot-Points Model are shown (in this example, “slot-3” indicates the name of the slot machine just played). Note that the prediction error for playing the left casino and the second slot machine occur simultaneously.

  • Figure 3.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 3.

    Logistic regression on casino (left) and slot machine (right) choices. We estimated the relationship between casino choices and the outcome of the casino on the last four times it was chosen, as well as the total slot machine points obtained in the corresponding trials. We similarly estimated the relationship between choices of each slot machine and the outcomes of the last four plays of this slot machine, as well as the casino outcomes during those same trials. Plotted are the regression weights for the last four outcomes of each type. Stars indicate significance at a between-participants Bonferroni-corrected level (p < 0.0063).

  • Figure 4.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 4.

    Average posterior probability per choice trial for the Outcome Model and the Target Model per participant. The Outcome Model assigns a higher average probability per trial to the choices of 22 of 28 participants (points lying above the solid equal-likelihood line). The average probability of a choice trial was calculated as the likelihood of the whole sequence of choice data divided by the number of choice trials. Dashed lines indicate chance.

  • Figure 5.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 5.

    Average posterior probability per choice trial for the Slot-Points Model and Six-Armed-Bandit Model per participant. The Slot-Points Model assigns a higher average probability per trial to the choices of 25 of 28 participants (points lying above the solid equal-likelihood line). Chance = 0.16, indicated by dashed lines.

  • Figure 6.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 6.

    A, Activations that survived a whole-brain FWE-corrected threshold of p < 0.05, cluster size >5, in the random effects contrast for the Casino regressor. Images are centered at voxel (18,14,−8) to better depict the extent of the activation. B, Activations that survived an uncorrected threshold of p < 0.001, cluster size >5, in the random effects contrast for the combined Slot regressor. Images are centered at voxel (−9, 11, −5).

Tables

  • Figures
    • View popup
    Table 1.

    All activations that survived a whole-brain FWE-corrected threshold of p < 0.05 in the random-effects contrast for a casino prediction error signal

    Anatomical locationPeak x,y,z (mm)Cluster sizePeak intensity (T)
    Right ventral putamen15, 11, −8297.15
    Left ventral putamen−18, 8, −11177.05
    Left lingual gyrus−3, −64, 131−6.91
    Right lingual gyrus9, −73, −57−6.66
    Occipital lobe−12, −88, 25485−12.68
    • Anatomical locations were determined through inspection with respect to the average anatomical image of all 28 participants.

Back to top

In this issue

The Journal of Neuroscience: 33 (13)
Journal of Neuroscience
Vol. 33, Issue 13
27 Mar 2013
  • Table of Contents
  • Table of Contents (PDF)
  • About the Cover
  • Index by author
  • Advertising (PDF)
  • Ed Board (PDF)
Email

Thank you for sharing this Journal of Neuroscience article.

NOTE: We request your email address only to inform the recipient that it was you who recommended this article, and that it is not junk mail. We do not retain these email addresses.

Enter multiple addresses on separate lines or separate them with commas.
Hierarchical Learning Induces Two Simultaneous, But Separable, Prediction Errors in Human Basal Ganglia
(Your Name) has forwarded a page to you from Journal of Neuroscience
(Your Name) thought you would be interested in this article in Journal of Neuroscience.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Print
View Full Page PDF
Citation Tools
Hierarchical Learning Induces Two Simultaneous, But Separable, Prediction Errors in Human Basal Ganglia
Carlos Diuk, Karin Tsai, Jonathan Wallis, Matthew Botvinick, Yael Niv
Journal of Neuroscience 27 March 2013, 33 (13) 5797-5805; DOI: 10.1523/JNEUROSCI.5445-12.2013

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Respond to this article
Request Permissions
Share
Hierarchical Learning Induces Two Simultaneous, But Separable, Prediction Errors in Human Basal Ganglia
Carlos Diuk, Karin Tsai, Jonathan Wallis, Matthew Botvinick, Yael Niv
Journal of Neuroscience 27 March 2013, 33 (13) 5797-5805; DOI: 10.1523/JNEUROSCI.5445-12.2013
Twitter logo Facebook logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • Abstract
    • Introduction
    • Materials and Methods
    • Results
    • Discussion
    • Footnotes
    • References
  • Figures & Data
  • Info & Metrics
  • eLetters
  • PDF

Responses to this article

Respond to this article

Jump to comment:

No eLetters have been published for this article.

Related Articles

Cited By...

More in this TOC Section

Articles

  • Memory Retrieval Has a Dynamic Influence on the Maintenance Mechanisms That Are Sensitive to ζ-Inhibitory Peptide (ZIP)
  • Neurophysiological Evidence for a Cortical Contribution to the Wakefulness-Related Drive to Breathe Explaining Hypocapnia-Resistant Ventilation in Humans
  • Monomeric Alpha-Synuclein Exerts a Physiological Role on Brain ATP Synthase
Show more Articles

Behavioral/Cognitive

  • Attention Alters Population Spatial Frequency Tuning
  • Complex Impact of Stimulus Envelope on Motor Synchronization to Sound
  • The Molecular Substrates of Second-Order Conditioned Fear in the Basolateral Amygdala Complex
Show more Behavioral/Cognitive
  • Home
  • Alerts
  • Follow SFN on BlueSky
  • Visit Society for Neuroscience on Facebook
  • Follow Society for Neuroscience on Twitter
  • Follow Society for Neuroscience on LinkedIn
  • Visit Society for Neuroscience on Youtube
  • Follow our RSS feeds

Content

  • Early Release
  • Current Issue
  • Issue Archive
  • Collections

Information

  • For Authors
  • For Advertisers
  • For the Media
  • For Subscribers

About

  • About the Journal
  • Editorial Board
  • Privacy Notice
  • Contact
  • Accessibility
(JNeurosci logo)
(SfN logo)

Copyright © 2025 by the Society for Neuroscience.
JNeurosci Online ISSN: 1529-2401

The ideas and opinions expressed in JNeurosci do not necessarily reflect those of SfN or the JNeurosci Editorial Board. Publication of an advertisement or other product mention in JNeurosci should not be construed as an endorsement of the manufacturer’s claims. SfN does not assume any responsibility for any injury and/or damage to persons or property arising from or related to any use of any material contained in JNeurosci.