Reward expectation modulates feedback-related negativity and EEG spectra

Abstract

The ability to evaluate outcomes of previous decisions is critical to adaptive decision-making. The feedback-related negativity (FRN) is an event-related potential (ERP) modulation that distinguishes losses from wins, but little is known about the effects of outcome probability on these ERP responses. Further, little is known about the frequency characteristics of feedback processing, for example, event-related oscillations and phase synchronizations. Here, we report an EEG experiment designed to address these issues. Subjects engaged in a probabilistic reinforcement learning task in which we manipulated, across blocks, the probability of winning and losing to each of two possible decision options. Behaviorally, all subjects quickly adapted their decision-making to maximize rewards. ERP analyses revealed that the probability of reward modulated neural responses to wins, but not to losses. This was seen both across blocks as well as within blocks, as learning progressed. Frequency decomposition via complex wavelets revealed that EEG responses to losses, compared to wins, were associated with enhanced power and phase coherence in the theta frequency band. As in the ERP analyses, power and phase coherence values following wins but not losses were modulated by reward probability. Some findings between ERP and frequency analyses diverged, suggesting that these analytic approaches provide complementary insights into neural processing. These findings suggest that the neural mechanisms of feedback processing may differ between wins and losses.

Introduction

To optimize behavior, organisms must evaluate outcomes of their actions, and use these evaluations to guide decision-making. The neural mechanisms of feedback evaluation are receiving increasing attention in cognitive neuroscience. In particular, researchers using event-related potentials (ERPs) have identified a component of the feedback-locked ERP that is sensitive to the valence of the feedback. This feedback-related negativity (FRN) is a relatively negative deflection in the ERP following losses or error feedback compared to wins or positive feedback. The FRN peaks at around 300 ms and is maximal at fronto-central scalp electrode sites (Hajcak et al., 2005, Holroyd et al., 2003, Yasuda et al., 2004). Convergent findings from source modeling, fMRI, and single-unit recording studies suggest that the FRN is generated in the medial frontal cortex, and probably in the anterior cingulate cortex (Amiez et al., 2005, Brown and Braver, 2005, Mars et al., 2005, Miltner et al., 2003, Niki and Watanabe, 1979, Paulus et al., 2004, Ridderinkhof et al., 2004, Shidara and Richmond, 2002, Tsujimoto et al., 2006, van Schie et al., 2004, Williams et al., 2004). Topographically and functionally similar feedback-locked ERP modulations have been called the medial frontal negativity and feedback error-related negativity (Gehring and Willoughby, 2002, Holroyd et al., 2003). These effects also share many similarities with the error-related negativity (ERN), a negative-going mid-frontally distributed potential elicited by erroneous responses on speeded response tasks. These potentials are thought to reflect activation of a reinforcement learning system that rapidly evaluates outcomes of decisions to guide reward-seeking behavior (Holroyd and Coles, 2002, Nieuwenhuis et al., 2004). This system is capable of rapidly determining whether feedback is better or worse than expected, and encodes this difference between expectations and actual outcomes as a reward prediction error. The anterior cingulate cortex might use these prediction errors to improve performance due to its role in cognitive control and action monitoring (Barber and Carter, 2005, Bokura et al., 2001, Botvinick et al., 2004, Kerns et al., 2004).

Given that a reward prediction error is the difference between an expected and received reward, differences in expectations of rewards should modulate the size of prediction error signals. Single-unit recording studies in nonhuman primates suggest that this is indeed the case, with more unexpected outcomes yielding larger neural responses in midbrain dopamine neurons (Fiorillo et al., 2003). It is unclear whether the magnitude of the FRN is also modulated by reward expectation, because previous studies have yielded inconsistent findings. In two studies (Holroyd et al., 2003, Yasuda et al., 2004), the magnitude of the FRN was larger when outcomes were unexpected. In another study, no statistically significant modulation was observed (Hajcak et al., 2005), although from visual inspection, it appears that the FRN was larger for unexpected than expected outcomes. Of the two studies that found a significant modulation, Yasuda and colleagues (2004) found that ERPs following both losses and wins were enhanced. In the Holroyd et al. (2003) study, however, it appears from visual inspection that only the win-related ERPs were modulated, although a statistical test of this asymmetry was not reported. We designed an experiment to investigate this issue further by examining not only how reward probability might modulate outcome-locked ERPs, but also how changes in reward expectation that occur during learning might further modulate ERPs.

Because the FRN (and ERPs in general) is measured by averaging single-trial EEG traces, this potential will not reflect oscillatory activity that varies in phase from trial-to-trial (particularly in high frequencies, such as gamma). Such event-related oscillations can be assessed using time–frequency decomposition analyses such as complex wavelet convolutions, from which one can obtain estimates of instantaneous power (i.e., energy at different frequencies) and inter-trial phase coherence (i.e., consistency of oscillation onset across trials). Recent findings using this approach have revealed novel insights into task-related cognitive processes beyond what is evident in averaged ERPs (Fell et al., 2004, Makeig et al., 2002, Salinas and Sejnowski, 2001). Although the frequency characteristics of feedback processing are largely unknown, research into the frequency characteristics of the response-related ERN (Bernat et al., 2005, Luu and Tucker, 2001, Luu et al., 2004, Trujillo and Allen, submitted for publication) suggests it reflects enhanced theta (i.e., 4–8 Hz) activity following incorrect compared to correct responses. Based on the idea that the ERN and FRN reflect similar mechanisms of monitoring and controlling behavior (Holroyd and Coles, 2002), we hypothesized that feedback processing would therefore induce increased EEG theta activity for losses compared to wins.

In the present study, we sought to investigate the effects of reward probability on ERP and oscillatory correlates of neural feedback processing. Subjects chose one of two targets on each trial, and received positive or negative feedback (± 10 cents) following each choice. In blocks of 80–150 trials, we manipulated the probability of winning and losing such that subjects had to learn which of the two targets rewarded more often in order to maximize their winnings. This design allowed us to examine neural responses to winning and losing as a function of the probability of wins and losses, using both conventional ERP and time–frequency analyses.