Article Figures & Data
Figures
- Figure 1.
Roulette gambling task. Participants were given 1000 ms to select a colored option. After 1000 ms, if an option was selected (here, as denoted by the lightened color, green was selected), a spinner appeared and spun for 3000 ms, and after stopping, remained on-screen for a further 500 ms. The reward was displayed for 1000 ms. If the spinner stopped on the selected color, as was the case here, the participant received €2; otherwise, he or she won nothing and the same coin superimposed with a red X was displayed. ITI was jittered, with a mean = 8000 ms.
- Figure 2.
Behavioral results and modeling. The choice data for each subject were separately fit to the RL model described in the text. Subjects were classified according to the weighted RL metric into three patterns of behavior: environmental adherence, RL-consistent, and GF-consistent. The panels show the mean choice behavior (data points) and model predictions (lines), separated by pattern (environmental adherence in the top, RL-consistent in the middle, GF-consistent in the bottom), and plotted according to the number of consecutive identical outcomes (horizontal axis) and the probability of choosing the outcome that is “streaking” (vertical axis). The different possible streaks are Hi Win (red), Lo Win (green), Hi Lose (blue), and Lo Lose (black). Note that the RL model predictions follow the qualitative trend of behavior for the environmental adherer and RL-consistent patterns but not the GF-consistent pattern.
- Figure 3.
Contrast of RL- with GF-consistent choices. Coronal (left) and sagittal (right) views of the tmap of activation in the RL–GF contrast, thresholded at pUnc < 0.001, k > 100, and overlaid onto the mean of the participants' structural images. Using a small volume correction, the left caudate activity was significant at pFWE < 0.05.
- Figure 4.
Striatal activity at reward and choice. A, Coronal (left) and sagittal (right) views of the tmap of activation from the correlation of BOLD activation with reward PE from an RL model, thresholded at pUnc < 0.005, k > 60, and overlaid onto the mean of the participants' structural images. Using a small volume correction, the left caudate activity was significant at pFWE < 0.05. B, Coronal view of the tmap of activation in the ventral striatum from the same contrast and threshold as shown in A. The left ventral striatal activity was significant at pFWE < 0.05, corrected for the whole brain. C, Coronal (left) and sagittal (right) views of the significant dorsal striatal clusters that show increased activity during RL- compared with GF-consistent choice at the time of stimulus onset (red), a positive correlation with PE at the time of reward (blue), and where the two clusters overlap (magenta).
- Figure 5.
Correlation of BOLD activity with RL metric. A, Coronal (left) and sagittal (right) views of the tmap of activation in the ventromedial prefrontal cortex from the second-level correlation of BOLD activity with the RL metric in the gambling–control contrast, thresholded at pUnc < 0.001, k > 60, and overlaid onto the mean of the participants' structural images. Using a small volume correction, the cluster was significant at pFWE < 0.05. B, Coronal (left) and sagittal (right) views of the tmap of activation in bilateral amygdala from the same contrast and threshold as shown in A. C–E, Scatterplot showing individual data points for the RL metric plotted against the BOLD activity in the ventromedial prefrontal cortex (C), left amygdala (D), and right amygdala (E) clusters, together with regression lines of best fit through the data. As indicated by the weighted RL metric, circles are colored to denote group membership: GF-consistent (blue), RL-consistent (red). The BOLD data in each plot were independently obtained by taking an 8 mm sphere centered on the same small volume coordinates for each region that were used to test for significant contrasts.
Tables
1 2 3 4 5 6 7 Hi Win 43.65 16.42 6.06 2.45 1.16 0.52 0.10 Hi Lose 39.10 22.97 14.10 8.42 4.77 2.74 1.39 Lo Win 75.81 20.55 6.23 1.52 0.42 0.16 0.03 Lo Lose 80.35 54.06 36.16 23.19 15.39 10.39 6.52 -
Data represent the mean frequencies across subjects for consecutive identical outcomes; e.g., column 2 of Hi Win shows that the mean number per subject of consecutive wins for the Hi option having a streak of 2 or greater is 16.42.
-
Category Previous choice Previous outcome Current behavior Current choice RL Hi Not Hi Switch Not Hi RL Not Hi Hi Switch Hi RL Hi Hi Stay Hi RL Not Hi Not Hi Stay Not Hi GF Not Hi Not Hi Switch Hi GF Hi Hi Switch Not Hi GF Not Hi Hi Stay Not Hi GF Hi Not Hi Stay Hi -
Not Hi refers to either of the two Lo probability options. Switching between Lo options was relatively rare; thus, while separate regressors were created for these events, they were not included in the contrast. Each of the above regressors contained streaks of two or more identical consecutive outcomes.
-
Model Group BIC χ λ 1/θ RL Environmental adherers −0.43 N/A 0.00 0.02 RL RL-consistent 9.10 N/A 0.03 0.21 RL GF-consistent −0.89 N/A 0.00 0.10 GF Environmental adherers −0.43 1.82 0.00 0.02 GF RL-consistent 2.87 2.27 −0.04 0.17 GF GF-consistent 2.71 5.50 −0.01 0.10 -
All data were fit at the individual level. BIC is positive where the experimental model outperformed the baseline model. Parameters: χ is the crossover tolerance parameter and is used by the GF model to switch between RL- and GF-consistent behavior; λ is the updating or learning rate parameter; θ is the temperature parameter and controls the extent to which a participant behaves deterministically.
-
- Table 4.
Descriptive statistics for choosing the Hi option, conditioned on previous outcomes
N Choose Hi | Hi Won Choose Hi | Hi Lost Environmental adherers 8 0.92 (0.12) 0.94 (0.11) RL-consistent 6 0.77 (0.16) 0.47 (0.17) GF-consistent 17 0.33 (0.16) 0.65 (0.13) Total 31 0.57 (0.31) 0.69 (0.21) -
The mean (standard deviation) conditional probabilities for choosing the Hi option conditioned on it winning or losing consecutively 2 or more times are given, separated by group. High variance within subject choice is characterized by choice probabilities within a cell that fail to load on either 0 or 1.
-
