Abstract
Perception depends on the interplay of ongoing spontaneous activity and stimulus-evoked activity in sensory cortices. This raises the possibility that training ongoing spontaneous activity alone might be sufficient for enhancing perceptual sensitivity. To test this, we trained human participants to control ongoing spontaneous activity in circumscribed regions of retinotopic visual cortex using real-time functional MRI-based neurofeedback. After training, we tested participants using a new and previously untrained visual detection task that was presented at the visual field location corresponding to the trained region of visual cortex. Perceptual sensitivity was significantly enhanced only when participants who had previously learned control over ongoing activity were now exercising control and only for that region of visual cortex. Our new approach allows us to non-invasively and non-pharmacologically manipulate regionally specific brain activity and thus provide “brain training” to deliver particular perceptual enhancements.
Introduction
Even in the absence of external stimulation, the brain undergoes spontaneous activity fluctuations that have an impact on perception (Arieli et al., 1996; Boly et al., 2007; Fox and Raichle, 2007). For example, successful visual perception depends on the interplay between ongoing spontaneous activity in visual cortex and that evoked by a stimulus (Hesselmann et al., 2008a, 2008b). Although the precise nature of that interaction remains under debate, the level of ongoing activity in visual cortex at the time a stimulus is presented determines whether or not it will subsequently be perceived (Ress et al., 2000).
Such observations raise the possibility that improvements in perceptual sensitivity might be achieved through learned control of the level of ongoing spontaneous activity before stimulus presentation. Here, we tested this hypothesis by examining whether systematically training participants to control the level of ongoing activity in visual cortex led to changes in their sensitivity for detecting visual stimuli when that control was exercised.
Typically, the level of ongoing activity in visual cortex is outside voluntary control because we do not have conscious access to it. We therefore used a new approach, using real-time functional MRI (fMRI) neurofeedback to provide direct feedback to participants about the ongoing activity level in their own retinotopic visual cortex (Weiskopf et al., 2004a; deCharms, 2008). Rather than studying differences in activity levels associated with ongoing spontaneous fluctuations, this new approach allowed us to selectively train participants to voluntarily “clamp” visual cortex activity at high or low levels. The visual cortex is a particularly well-suited target for neurofeedback because its activity can be modulated by top-down control mechanisms, such as attention and imagery (Kastner et al., 1998; Brefczynski and DeYoe, 1999; Kosslyn et al., 2001; Slotnick et al., 2005; Li et al., 2008; Stokes et al., 2009). Furthermore, in the visual domain, potential behavioral consequences of neurofeedback training can be precisely assessed using standard behavioral paradigms, such as visual threshold measurements. We hypothesized that increasing the mean level of ongoing activity in visual cortex would lead to enhanced visual sensitivity (compared with not exerting control) when previously unseen weak visual stimuli were presented for detection.
If voluntarily increasing ongoing visual cortex activity caused improved perception for stimuli that were entirely unrelated to the neurofeedback training, then such an improvement should generalize to any visual stimulus. Hence, this new approach could potentially allow us to voluntarily and temporarily increase perceptual sensitivity without relying on repeated exposure to (or training with) a specific visual stimulus. Other means of improving visual sensitivity, such as perceptual learning, require repeated exposure to a specific stimulus, are not under voluntary control, and are permanent.
Our new approach also goes beyond conventional imaging studies that can only correlate changes in ongoing activity with changes in visual perception. Neurofeedback allows manipulation of ongoing brain activity and thereby to establish a causal link between such ongoing activity and visual perception.
Materials and Methods
Experimental design overview.
To test our hypothesis, we recruited 16 healthy volunteers and sought to train them to control the level of ongoing activity in a circumscribed target region of interest (ROI) in early retinotopic visual cortex, which was delineated in separate functional localizer scans (Fig. 1). This ROI represented a specific location in the visual field. Training participants to control activity in this ROI was undertaken in several separate scanning sessions spread over the course of several days. For neurofeedback training, the fMRI blood oxygenation level-dependent (BOLD) signals from the ROI were processed in near real time, and the level of ongoing activity in the ROI was fed back to the participant in the form of a visual display projected in front of the participant during scanning (Fig. 2). A target level of activity was delineated on the display, and participants attempted to learn, by trial and error and using a freely chosen strategy, to upregulate the level of activity measured in the target ROI to the target level. No other visual stimuli were presented. After the neurofeedback training, participants were required to detect the presence of weak visual stimuli presented at different positions in the visual field while “clamping” activity in the visual target ROI at either high or low levels using the voluntary control they had attempted previously to learn (behavioral test runs).
Experimental design overview. In the first scanning session, a high-resolution structural scan was acquired, and the visual target ROI was defined with a functional localizer. After receiving written instructions, participants underwent several sessions of neurofeedback training (on separate days). Per session, participants did an average of approximately two feedback runs of 8.3 min each. A feedback run was composed of 38 s baseline blocks (gray) interleaved with 38 s upregulation blocks (red). After the training, participants tried self-regulation in the absence of feedback (transfer run). The behavioral testing was performed in several separate scanning sessions spread over the course of several days. To test the effect of self-regulation on behavior, participants were required to detect the presence of near-threshold stimuli at different positions in the visual field while exerting voluntary control over activity in the target ROI. We hypothesized that participants who learned to increase activity in their left/right visual cortex would improve in visual sensitivity in the contralateral but not in the ipsilateral visual field (visual stimuli are processed in the contralateral visual cortex).
Flow of data in the neurofeedback experiment. Sixteen healthy volunteers were trained to learn to control local visual cortex activation with the help of neurofeedback. Brain activation was measured with a 3 T Siemens Allegra MR scanner using the BOLD effect, which detects the neurovascular response to brain activation. Data preprocessing (e.g., 3D head motion correction) and analysis (e.g., percentage of signal change calculation) were performed online with Turbo-BrainVoyager (Brain Innovation) and custom-made software. Custom-made software was used to continuously provide visual feedback of local brain activation to the participant in the scanner. The neurofeedback display consisted of a thermometer, and the temperature reading indicated the current level of activity in the visual cortex ROI. A dashed line indicated the target activation level, which could either be high (upregulation condition) or low (baseline condition).
fMRI data acquisition.
All experiments were performed on a 3 T Magnetom Allegra head-only scanner, using a standard transmit–receive head coil (Siemens Healthcare). Functional data were acquired with a single-shot gradient echo planar imaging sequence (matrix size, 64 × 64; field of view, 192 × 192 mm; isotropic resolution, 3 × 3 × 3 mm; 32 slices with ascending acquisition; slice thickness, 2 mm; slice gap, 1 mm; echo time (TE), 30 ms; repetition time (TR), 1920 ms; flip angle, 90°; receiver bandwidth, 3551 Hz/pixel). In the middle of each scanning session, double-echo fast, low-angle shot sequence (FLASH) field maps (TE1, 10 ms; TE2, 12.46 ms; resolution, 3 × 3 × 2 mm; slice gap, 1 mm) were acquired and used to correct geometric distortions in the images attributable to field inhomogeneities.
The neurofeedback setup used Turbo-BrainVoyager (Brain Innovation), custom real-time image export tools programmed in ICE VA25 (Siemens Healthcare) (Weiskopf et al., 2004b), and custom scripts running on MATLAB (MathWorks). This allowed participants to be shown visual representations of BOLD signal changes in specific brain regions (in the form of a thermometer display projected into the scanner) with a delay of <2 s from the acquisition of the image. Head motion was corrected in real time using Turbo-BrainVoyager. Heart rate and respiration were continuously monitored throughout the experiment (setup similar to Hutton et al., 2011).
Participants.
Sixteen naive human volunteers (six male, between 18 and 37 years old, all right handed) with normal or corrected-to normal vision gave written informed consent to participate in the experiment, which was approved by the local ethics committee. Before the experiment, they completed the Vividness of Visual Imagery Questionnaire (VVIQ) (Marks, 1973, 1995) and received written instructions describing that they will learn to regulate their visual cortex activity with the help of neurofeedback. The instructions included an explanation of the neurofeedback thermometer display (Fig. 2) and recommended as potential regulation strategies the use of visual imagery with high-resolution details as well as changing stimulus quality (color, shape) and intensity (brightness) spatially overlapping with the target ROI. We also suggested that participants prepare a few imagined patterns in advance and to try them repeatedly. It was emphasized that participants should find an individual strategy that worked best for them. Furthermore, they were instructed to fixate on the central fixation point throughout the experiment, to breathe steadily, and to remain as still as possible.
We also explained to the participants that the feedback was delayed by ∼8 s (the hemodynamic delay plus the real-time analysis processing time). In a separate scanning run, this delay was illustrated to the participants by asking them to move their fingers while providing feedback from the motor cortex. By observing the delay of the neurofeedback signal change with respect to the onset and offset of finger movements, participants gained an intuitive understanding of the delay. All participants reported that they could easily account for the delay during the actual neurofeedback training.
After each scanning session, participants were asked to fill in a written questionnaire and, among other questions, describe how they tried to manipulate the feedback signal (including drawing any visual imagery), how effective their strategy was, and how they rated the attentional demands.
The control group was matched to the experimental group with respect to age, gender, and education (Table 1).
Details about the experimental groups, the visual target ROIs, the visual sensitivity measurements, and the CNR of the feedback display
Retinotopic mapping and localizer.
Stimulus display (a large circular projection screen at the rear of the scanner bore with a mirror positioned within the head coil) and response collection were controlled by MATLAB (MathWorks) using the COGENT toolbox (developed by the Cogent 2000 team at the Wellcome Trust Centre for Neuroimaging and the UCL Institute of Cognitive Neuroscience, and Cogent Graphics developed by John Romaya at the Wellcome Department of Imaging Neuroscience, London, UK).
In a separate scanning session before the neurofeedback training, we collected from each participant a high-resolution T1-weighted structural scan of the whole brain [3D modified driven equilibrium Fourier transform (MDEFT); 1 mm isotropic resolution; matrix size, 256 × 240 mm; field of view, 256 × 240 mm; 176 sagittal partitions; TE, 2.4 ms; TR, 7.92 ms; inversion time, 910 ms; flip angle, 15°; readout bandwidth, 195 Hz/pixel; spin tagging in the neck with flip angle 160° to avoid flow artifacts] for superposition of functional maps (Deichmann et al., 2004). In the same scanning session, we acquired for each participant two retinotopic mapping runs of 200 volumes each, during which participants viewed standard stimuli that mapped the horizontal and vertical meridians (Sereno et al., 1995; Wandell, 1999; Wandell et al., 2000).
We also determined the visual target ROI from which participants received neurofeedback by acquiring two functional localizer runs of 150 volumes each. In each localizer run, participants were fixating on a central fixation point while a flickering circular checkerboard (100% contrast, 10 Hz contrast reversal) with a diameter of 2° visual angle was presented on a gray background. The checkerboard disc was presented for ∼13 s in each of the four quadrants of the visual field (eccentricity, 3° visual angle), with a baseline condition of the same duration once after the stimulus had been presented in each quadrant. To ensure fixation, participants had to count and report color changes of the fixation dot. The visual target ROI for neurofeedback was restricted to those voxels in occipital cortex that exhibited a positive BOLD response to the checkerboard stimulus (p < 0.0001, Bonferroni's corrected for multiple comparisons) in either the lower left or the lower right visual quadrant (for details about the visual target ROIs, see Table 1). Participants received feedback from the visual target ROI corresponding to the lower left or the lower right visual field (randomly assigned). The target ROI for the control group, i.e., the ventral striatum, was anatomically defined using Brain Voyager QX (Brain Innovation).
Neurofeedback training.
Participants took part in at least three neurofeedback training sessions spread over the course of several days. Each training session started with a 4.5 min T1-weighted structural scan of the whole brain (3D FLASH; 1 mm isotropic resolution; matrix size, 256 × 224; field of view, 256 × 224 mm; 176 sagittal partitions; TE, 3.5 ms; TR, 9.5 ms; flip angle, 18°; readout bandwidth, 199 Hz/pixel; anatomical image data exported in near real time similar to Weiskopf et al., 2004b). This anatomical image was used for coregistration of the current head position with the high-resolution T1-weighted structural MDEFT scan using Turbo-BrainVoyager. From the resulting coregistration matrix, the position of the neurofeedback target ROI in the current head position of the current run was determined. This ensured that the same ROI was targeted in all training sessions, which took place on different days.
For each training session, participants performed on average two training runs of 8.3 min each. The training runs were composed of seven 38-s baseline blocks interleaved with upregulation blocks of the same duration. During the baseline blocks, the target-level indicator of the thermometer display was low (i.e., three pixels above the fixation point), which indicated to the participants that they should mentally count backward from 99 in steps of −7 to maintain a stable baseline activity (Fig. 2). During the upregulation blocks, the target-level indicator moved up (i.e., 30 pixels above the fixation point), which indicated to the participants that they should increase activity in their visual target ROI. Participants were presented feedback about their success via the thermometer reading, which indicated the percentage of signal change compared with the previous baseline block (sliding window baseline). To avoid strong fluctuations of the thermometer display and baseline drifts, we applied temporal filtering and detrending. Temporal filtering was accomplished by averaging over the previous three time points. Detrending was accomplished by applying an exponential moving average algorithm to the ROI time course (see Eq. 1):
where m is the current time point, α is the smoothing parameter that is set to 0.97, y is the time point value, x is the filter value, and x̂ is the filter output.
To normalize the percentage signal change values to the thermometer scale (which ranged from 70 pixels below the fixation point to 70 pixels above the fixation point), the signal change values were scaled according to Equation 2:
where m is the current time point, t is the temperature reading of the thermometer, psc is the percentage of signal change, limitlow/limitup are the mean of the five lowest/highest signal change values that have been acquired so far, pixelmax is 70, and pixelmin is −70.
After the neurofeedback training, participants performed self-regulation but now in the absence of feedback (transfer run), i.e., only the target level indicator was visible but not the thermometer reading.
To be considered as learners, participants needed to fulfill two criteria. (1) They had to show a signal increase in the upregulation blocks compared with the baseline blocks in the online analysis of the last two training runs. (2) They had to show an overall signal increase during upregulation, i.e., on average, the signal difference between upregulation blocks and baseline blocks had to be positive.
To validate the feedback signal after the experiment, we calculated the contrast-to-noise ratio (CNR) of the feedback display according to Equation 3:
where displayTCup/base is the time course of the feedback display during upregulation and baseline conditions, respectively. Because self-regulation is not constant and we were interested in the periods of successful self-regulation, we calculated the CNR for the top 20% of each time course.
Behavioral testing without self-regulation before and after the neurofeedback training.
To test whether neurofeedback training led to general perceptual improvements that were independent from actively upregulating visual cortex activity, we measured each participant's perceptual threshold before and after the neurofeedback training, i.e., before the first training scan and after the last behavioral test scan. For this, participants had to detect a Gabor grating (same size and location as the visual target ROI localizer; presentation duration, 500 ms; 1.5 cycles/°; orientation, 0°; phase, 0.25°; σ of its Gaussian, 0.65°; trimmed off edges) whose contrast was adjusted according to the adaptive procedure QUEST (Watson and Pelli, 1983). Immediately after the Gabor presentation, the fixation point changed color for 500 ms, which indicated to the participants that they had to respond using a keypad if they saw a visual stimulus. The individual detection thresholds were determined by averaging over three QUEST runs of 60 trials each. The pretraining and posttraining threshold measurements were performed in the MR scanner. Please note that, for technical reasons, posttraining threshold measurements from one learner and from one control participant are missing.
Behavioral testing during self-regulation.
Behavioral testing during self-regulation was performed in several separate scanning sessions spread over the course of several days. The behavioral test was performed during transfer runs, i.e., the participants did not receive neurofeedback information. While participants clamped activity in the visual target ROI at high or low levels, a Gabor grating was presented at randomly jittered times in the visual field location corresponding to the visual target ROI and/or the corresponding location in the contralateral lower visual field. The Gabor grating was the same as the one used for determining the visual threshold without self-regulation. The contrast of the Gabor grating corresponded to individual detection thresholds of the participants that were determined in the MR scanner before the first training session, i.e., it was adjusted for inter-individual differences in perceptual threshold and presented at this threshold during all of his/her behavioral test runs. Immediately after the Gabor presentation (or no presentation in the Gabor absent trials; 50% presentation probability independently for the lower left and the lower right visual field), the fixation point changed color for 500 ms, which indicated to the participants that they had to respond using a keypad if they saw a visual stimulus in the lower left and/or the lower right visual field. To ensure that participants could build up proper self-regulation after responding to the Gabor presentation, the Gabor presentation frequency was sparse (approximately every 10 s). This explains why a large number of behavioral test sessions were needed for a reliable estimate of visual sensitivity during upregulation and baseline blocks. The hits, misses, false alarms, and correct rejections were recorded and analyzed to yield the visual sensitivity measure d−, which is defined as z(hit rate) − z(false alarm rate), with z defined as the inverse of the cumulative density function of the normal distribution. Visual sensitivity (d′) was calculated for upregulation and baseline blocks, independently for the retinotopic location overlapping with the visual target ROI and for the corresponding contralateral visual field position. To assess changes in visual sensitivity, paired t tests compared upregulation and baseline blocks for each experimental group (two-tailed; statistical significance threshold of p < 0.05).
Initial offline data preprocessing.
Offline data analysis used SPM8 (Wellcome Trust Centre for Neuroimaging, Queen Square, London, UK; http://www.fil.ion.ucl.ac.uk/) and BrainVoyager QX (Brain Innovation). The first three volumes of each run were excluded from statistical analysis because it takes a few volumes for T1-related equilibration to occur at the start of each fMRI run. The remaining images were corrected for slice time acquisition differences, realigned to the first scan of each run, corrected for static magnetic field (B0) inhomogeneities (Hutton et al., 2002), coregistered to the structural scan and smoothed with an isotropic Gaussian kernel with 4 mm full-width at half-maximum.
Offline ROI analysis.
The visual target ROI time courses were extracted and detrended with linear and quadratic terms, and the mean percentage of signal change in the upregulation blocks compared with the baseline blocks was calculated for each run. Because the number of completed training runs varied slightly across participants, the mean percentage of signal change values for each subject were grouped and averaged into six bins (=minimum number of training runs that all participants completed) in a nearest-neighbor manner. A linear regression of the mean percentage of signal change of each experimental group over training runs was calculated to assess neurofeedback training success. In addition, paired t tests were calculated for each experimental group to examine learning success in the last training, the transfer, and the behavioral test runs (two-tailed; statistical significance threshold of p < 0.05).
To analyze the composition of the visual target ROIs, we functionally identified each participant's V1, V2, and V3 and quantified the proportion of ROI voxels that fell within V1, V2, or V3. We also investigated whether the proportion of the visual target ROI (in terms of V1, V2, and V3 voxels) predicted training success. For this, we calculated the Pearson's correlation between the number of V1, V2, or V3 voxels for each participant and his/her achieved signal change in the last training run. Furthermore, we extracted for each participant the signal change across training runs for those V1, V2, and V3 voxels that overlap with the participant's visual target ROI. To show that learning was specific to the visual target ROI, we compared this with the signal change across training runs for all voxels in V1, V2, or V3. Please note that, for technical reasons, one learner could not be included in this analysis.
Exploration of connectivity changes using psychophysical interaction analysis.
To explore functional coupling between different brain areas and activity in the visual target ROI, we used a psychophysiological interaction (PPI) analysis (Friston et al., 1997). Functional images were normalized to the MNI standard template using DARTEL (Ashburner, 2007). Images of those participants whose visual target ROI was located in the left hemisphere were flipped so that all visual target ROIs were displayed on the right side.
We then specified general linear models with regressors for the respective visual target ROI time course, for the experimental conditions (i.e., a boxcar function representing upregulation and baseline blocks convolved with the canonical hemodynamic response function in SPM8) and for the interaction between the two. For each run of each participant, a positive contrast was applied to parameters estimated for the last regressor (i.e., the interaction term).
To reveal areas of the brain whose connectivity to the visual target ROI changed depending on whether or not the participant was upregulating, we calculated a voxelwise one-sample t test of the interaction term contrast images of each learner's last training runs. At a lower statistical threshold (i.e., p = 0.001 uncorrected for multiple comparisons), this t test revealed stronger correlations between the visual target ROI and the contralateral superior parietal lobe (SPL), and weaker correlations between the visual target ROI and the ipsilateral middle frontal gyrus as well as the contralateral superior temporal gyrus. Brain ROIs were labeled using the SPM anatomy toolbox (Eickhoff et al., 2005, 2006, 2007).
Because these three areas might mediate learning self-regulation, we investigated how the PPI between the visual target ROI and these three areas changed across training within the three different groups (i.e., learners, non-learners, and controls). For this, we used the ROIs from the previous step (i.e., SPL, middle frontal gyrus, superior temporal gyrus), to extract the mean PPI parameter estimates from these ROIs for each training run. Next, we averaged the mean PPI parameter estimates over each of the three experimental groups (i.e., the learners, the non-learners, and the control groups). This was done separately for each ROI and for each of the six training runs. The mean values were transformed into percentage change in PPI from the first training run.
Results
Learning voluntary control of visual cortex activity
Every participant completed at least six neurofeedback training runs spread over several days. Over the course of this training, seven participants successfully learned to control BOLD signals in the visual target ROI. Specifically, these individuals showed a significant increase in BOLD signals in the ROI associated with training (linear regression: r2 = 0.85, F(1,5) = 23.40, p < 0.01) and a significant difference in signals comparing blocks in which they were asked to increase the level of ongoing activity in the visual target ROI with baseline blocks in which active control was not exerted (Fig. 3A, learners in green; paired t test to compare upregulation and baseline blocks in the last training run: t = 5.07, df = 6, p < 0.01). Increased visual cortex activity was locally specific and confined to the visual target ROI (Fig. 4A,B). Furthermore, the voluntary signal changes in the visual cortex were not related to cardiorespiratory artifacts, i.e., heart rate and respiration showed no difference between baseline and upregulation blocks (Fig. 5; heart rate paired t test: t = 0.14, df = 6, p = 0.89; respiration paired t test: t = 0.70, df = 6, p = 0.51).
Neurofeedback training and behavioral effects of self-regulation. A, Visual ROI regulation success was measured as the mean percentage signal change in the upregulation blocks compared with the baseline blocks. The seven learners showed an increase in visual cortex control with training that was maintained during transfer and behavioral test runs. The non-learners and the controls did not learn to control their visual cortex activity. B, Visual sensitivity (d′) of the learners improved significantly in the upregulation compared with the baseline blocks. The non-learners showed a nonsignificant decrease in visual sensitivity, and the controls showed no difference in visual sensitivity. Error bars represent 1 SEM.
Composition of visual target ROI. A, Learning is found in all subregions of the visual target ROI, i.e., there is no significant difference between V1, V2, and V3 voxel components of the visual target ROI. B, Self-regulation of visual cortex activity is confined to the parts of V1, V2, and V3 that overlap with the visual target ROI and is not found in all V1, V2, and V3 voxels of the corresponding quadrant (depending on the location of the visual target ROI activity in either the left or right dorsal portions of the visual regions are shown). For comparison, the learning curve for the visual target ROI is plotted in gray. Error bars represent 1 SEM. C, The proportion of V1, V2, and V3 voxels that constitute the visual target ROI is not significantly different between the experimental groups. Also, the proportion of V1, V2, and V3 voxels does not correlate with learning success in terms of achieved signal change in the last training run (V1: Pearson's correlation, r(14) = 0.04, p = 0.89, n = 16; V2: Pearson's correlation, r(14) = −0.16, p = 0.57, n = 16; V3: Pearson's correlation, r(14) = 0.02, p = 0.94, n = 16).
Physiological measures show no difference between upregulation and baseline blocks. A, Mean heart beat is not different between conditions (upregulation/baseline) and experimental groups (learners/non-learners; because of technical problems, there is insufficient data for the controls). B, Mean respiration, plotted as the SD over 3 TRs (Birn et al., 2008; Chang et al., 2009), is not different between conditions (upregulation/baseline) and experimental groups (learners/non-learners/controls). Error bars represent 1 SEM.
This learned ability to control spontaneous visual cortex activity was subsequently maintained in the absence of neurofeedback. This was shown in “transfer” runs, in which we tested the ability of trained participants to upregulate BOLD signals in the visual target ROI to the previously trained target level but now in the absence of neurofeedback (or indeed any other visual stimulation). In this new situation, we found that participants could exert voluntary control of activity in the target ROI in the absence of feedback, and this was specific to those individuals who had demonstrated learning previously (Fig. 3A, transfer run column; paired t test: t = 8.43, df = 6, p < 0.01). This shows that, once participants had learned to self-regulate in the visual target ROI, this learned skill could be used even in the absence of neurofeedback and that changes in brain activity during learning did not reflect changing visual input from the neurofeedback display.
Four participants did not learn to increase visual cortex activity (Fig. 3A, non-learners in red; linear regression: r2 = 0.06, F(1,2) = 0.27, p = 0.63), although they did not differ from the learners with respect to the composition of the visual target ROI (Fig. 4C), the size of the ROI (Table 1; one-way ANOVA: F(2,13) = 0.06, df = 15, p = 0.95), the amount of training, the mental strategies used (Fig. 6), their attentional efforts (Fig. 7A), or their vividness of visual imagery (Fig. 7B,C).
Examples of visual imagery. As part of the debriefing after the neurofeedback training sessions, participants depicted the contents and locations of their imagery. Verbally, participants described their imagery as follows (from top left to bottom right): learners, “I imagined pictures of someone I like.”/“I saw pictures in the fuzz.”/“I imagined various spinning wheels and moving spirals.”/“I was observing a model with a fancy wedding dress walking down the stage.”/“I imagined details of my cat moving.”; non-learners, “I imagined people doing various things.”/“I imagined trees and a road leading to an entrance of a building.”/“I imagined Tinkerbell movies.”/“I imagined writing my name.”; controls, “I imagined viewing a dental cast in different angles.”/“I imagined a beach at the ocean.”/“I imagined me running in a park.”/“I imagined a fish.” For illustration purposes, an orange circle indicates the location corresponding to the visual target ROI; this circle was not presented to the participants. Please note that two participants from the learner group and one from the control group frequently changed strategies and were therefore not able to provide a representative drawing/description of their imagery.
Attentional effort and vividness of visual imagery. A, After the training sessions, participants rated on a scale from 1 to 5 if they were focused or absent-minded during the training runs. There was no significant difference with respect to their attentional involvement between the experimental groups (one-way ANOVA: F(2,13) = 0.23, df = 15, p = 0.80). B, After the training sessions, participants rated their visual vividness during upregulation and baseline blocks on a scale from 1 to 8. Vividness was not different between the experimental groups, and it was much higher during the upregulation blocks compared with the baseline blocks (some participants reported some weak visual imagery of numbers while counting backward during the baseline blocks). C, Before the neurofeedback training, participants performed the VVIQ, which is a standard test to measure individual differences in vividness of visual imagery (Marks, 1973, 1995). Please note that higher VVIQ scores indicate lower vividness of the visual imagery. There was no significant difference of visual image vividness between the experimental groups (one-way ANOVA: F(2,13) = 0.79, df = 15, p = 0.47). Error bars represent 1 SEM.
To determine whether contingent feedback from the visual target ROI was necessary to learn to control its activity, we scanned a control group of five participants. Participants in this control group were provided with the same instructions and underwent the identical training procedure but received feedback from an area not involved in visual processing, i.e., the ventral striatum. Participants in this group did not learn to control visual cortex activity (Fig. 3A, controls in blue; linear regression: r2 = 0.50, F(1,3) = 3.99, p = 0.12).
These training results were also reflected in the CNR values of the feedback display (for details, see Table 1). The CNR values were significantly higher for the learners compared with the non-learners and the controls (one-way ANOVA: F(2,13) = 11.02, df = 15, p < 0.01). Only the learners showed a significant CNR increase associated with training (learners: r2 = 0.70, F(1,5) = 9.19, p = 0.03; non-learners: r2 = 0.24, F(1,5) = 1.28, p = 0.32; controls: r2 = 0.06, F(1,5) = 0.26, p = 0.64).
Behavioral effects of self-regulation
Having trained participants to voluntarily control ongoing activity in visual cortex, we could now proceed to test our hypothesis that voluntary enhancement of this activity would affect visual sensitivity for detecting targets presented in the visual field location corresponding to the controlled target ROI. Behavioral testing was performed in several separate scanning sessions spread over the course of several days (and was therefore completely independent from the classification into learners and non-learners, which was based on the training performance alone). Participants were now required to detect the presence of near-threshold visual stimuli presented at different positions in the visual field while clamping activity in the visual target ROI at either high or low levels using the voluntary control they had previously attempted to learn but now without feedback. These weak visual stimuli and detection task had never been presented to the participants previously during training. The visual field positions where the visual stimuli were presented corresponded to either the retinotopic location of the trained target ROI or the untrained region in the opposite visual field. During this experiment, those participants who had previously learned voluntary control showed significant increases in activity in the visual target ROI during “upregulation” blocks compared with “baseline” blocks (Fig. 3A, test run column; paired t test: t = 5.99, df = 6, p < 0.01). As anticipated, because of the simultaneous behavioral experiment, this self-regulation of the visual target ROI was slightly less successful than during the training and transfer runs (but still demonstrated statistically significant differences from baseline). Nevertheless, the increase in visual cortex activity controlled by the learners was also associated with a significant improvement in visual sensitivity (d′) for detecting the weak visual stimuli in the upregulation compared with the baseline blocks (Fig. 3B; paired t test: t = 3.36, df = 6, p = 0.02; for details, see Table 1). This improvement was attributable to a significantly increased hit rate (i.e., the percentage of stimuli that were correctly detected; paired t test: t = 4.05, df = 6, p < 0.01). Importantly, increased visual sensitivity was only found in learners for stimuli that were presented at locations in the visual field overlapping with the visual target ROI and were not found for stimuli presented in other visual field locations.
In contrast, during behavioral testing, non-learners showed (as expected) no significant differences in activity in the visual target ROI during upregulation compared with baseline blocks (Fig. 3A, test run column; paired t test: t = −0.74, df = 3, p = 0.51), and they also showed no significant differences in visual sensitivity (d′) for detecting the weak visual stimuli (Fig. 3B; paired t test: t = −1.85, df = 3, p = 0.16; for details, see Table 1). Similarly, the control participants also showed no significant differences in activity in the visual target ROI during upregulation (Fig. 3A, test run column; paired t test: t = −0.18, df = 4, p = 0.87) and no significant changes in d′ for visual detection (Fig. 3B; paired t test: t = 0.86, df = 4, p = 0.44; for details, see Table 1). Collapsing across all three groups (learners, non-learners, and control groups), there was a significant positive correlation between the degree to which an individual could increase activity in the visual target ROI during upregulation blocks and the improvement in visual sensitivity on the behavioral task that they exhibited (Fig. 8; Pearson's correlation: r(14) = 0.52, p = 0.04, n = 16).
Changes in visual sensitivity correlate with changes in visual cortex activity. There is a significant positive correlation between self-regulated activity in the visual target ROI (i.e., the signal change differences between upregulation and baseline blocks in the last training run) and visual sensitivity (i.e., the d′ differences between upregulation and baseline blocks; Pearson's correlation, r(14) = 0.51, p = 0.04, n = 16).
Over the course of neurofeedback training, there was no BOLD signal increase in the upregulation blocks compared with the baseline blocks in the homolog of the visual target ROI in the corresponding location of the contralateral lower visual field (Fig. 9A, linear regression training run column; learners: r2 = 0.12, F(1,5) = 0.55, p = 0.50; non-learners: r2 < 0.01, F(1,3) < 0.01, p = 0.99; controls: r2 = 0.03, F(1,2) = 0.14, p = 0.73). Also during the transfer runs and during the behavioral test runs, there were no differences between upregulation blocks and baseline blocks (Fig. 9A, paired t test transfer run column; learners: t = 1.17, df = 6, p = 0.29; non-learners: t = −0.26, df = 3, p = 0.81; controls: t = 046, df = 4, p = 0.67; paired t test, test run column; learners: t = −0.19, df = 6, p = 0.86; non-learners: t = −0.60, df = 3, p = 0.59; controls: t = 0.78, df = 4, p = 0.48). Moreover, no significant changes in d′ for visual detection were found for the contralateral hemisphere (Fig. 9B; paired t test learners: t = −0.36, df = 6, p = 0.74; non-learners: t = 2.43, df = 3, p = 0.09; controls: t = 0.73, df = 4, p = 0.50).
No learning and behavioral effects in the contralateral homolog of the ROI. A, Regulation success in the homolog of the visual target ROI in the corresponding location of the contralateral lower visual field was measured as the mean percentage signal change in the upregulation blocks compared with the baseline blocks. None of the participants showed an increase in the contralateral visual cortex with training, during transfer, nor during behavioral test runs. B, Visual sensitivity (d′) of the learners, the non-learners, and the controls was not significantly different in the upregulation compared with the baseline blocks. Error bars represent 1 SEM.
Behavioral effects without self-regulation
We tested whether neurofeedback training led to general perceptual improvements that are independent of actively upregulating visual cortex activity, by measuring each participant's perceptual threshold before (Fig. 10A) and after (Fig. 10B) the neurofeedback training. We found that perceptual thresholds without self-regulation were not significantly different between the experimental groups (two-way ANOVA; main effect of group: F(2,11) = 0.02, df = 11, p = 0.98; for details, see Table 1) or over time, i.e., between pretraining and posttraining measurements (main effect of time: F(1,11) = 0.04, df = 11, p = 0.85). The time × experimental group interaction was also nonsignificant (F(2,11) = 0.84, df = 11, p = 0.46).
Behavioral effects without self-regulation. To test whether neurofeedback training led to general perceptual improvements that are independent of actively upregulating visual cortex activity, we measured each participant's perceptual threshold before (A) and after (B) the neurofeedback training in the MR scanner. Thresholds were not different between the experimental groups or over time.
Processes underlying self-regulation
To provide additional insight into the neural substrates of the learning effect that we observed, we used PPIs with activity in the visual target ROI (Friston et al., 1997). Such an analysis reveals brain areas whose effective connectivity to the visual target ROI changed depending on whether participants were upregulating or not. We found that, during upregulation, activity in the visual target ROI was more correlated with activity in the contralateral SPL but less correlated with activity in the ipsilateral middle frontal gyrus and the contralateral superior temporal gyrus (Fig. 11A). To further explore the modulatory influences that these three regions might thus have on the visual target ROI, we next characterized how the PPI between these areas and the visual target ROI changed across training (Fig. 11B). Although there were no significant training-associated changes in PPI between the visual target ROI and the middle frontal and the superior temporal gyrus, we found that the PPI with the contralateral SPL increased significantly over the course of training (linear regression: r2 = 0.65, F(1,5) = 7.28, p = 0.03, one-tailed). No such changes in connectivity were found in the non-learners and the controls. This pattern of connectivity between the visual target ROI and the contralateral SPL might explain how learning visual cortex control is mediated and why some participants learned voluntary control whereas others failed to do so. Future work could test these possibilities explicitly.
Regions with differential connectivity to the visual target ROI attributable to self-regulation. A, A PPI analysis of the learner's last training runs revealed stronger correlations between the visual target ROI and the contralateral superior parietal cortex [MNI coordinates, (22, −58, 63)] and weaker correlations with the ipsilateral middle frontal gyrus [not shown for clarity; MNI coordinates, (38, 48, 0)] as well as with the contralateral superior temporal gyrus [not shown for clarity; MNI coordinates, (56, −29, 17)]. Please note that, for this exploratory analysis, the statistical parametric maps were thresholded at p = 0.001 (uncorrected). Please also note that, to allow for group comparisons, the images of the participant whose target ROI was in the left visual cortex had been flipped, and therefore the right hemisphere in the illustration refers to the hemisphere ipsilateral to the visual target ROI. B, In the learners, the PPI between the contralateral SPL and the visual target ROI increased significantly with training. Such training-related changes in connectivity were not found in the non-learners and in the controls. No significant changes with training were found for the middle frontal gyrus and the superior temporal gyrus. Error bars represent 1 SEM.
Discussion
We showed that participants could use neurofeedback to learn to voluntarily control the ongoing activity in a circumscribed retinotopic region of their early visual cortex in a self-organized, endogenous manner. When participants voluntarily regulated activity in that region after training, significant improvements in visual sensitivity for a previously untrained detection task were seen. These improvements were specific to the participants who were able to successfully learn, to the time when participants were upregulating visual cortex activity, and to the region (and thus visual field location) trained. Hence, learning to control ongoing activity in retinotopic early visual cortex is possible and has profound perceptual consequences even for an unrelated and previously untrained visual detection task.
Generalizability of the perceptual effect
Very recently, it has been shown that neurofeedback training of visual cortex activation patterns that correspond to a specific visual stimulus can induce enhanced perceptual sensitivity specific to that stimulus (Shibata et al., 2011). Here, we complement this finding by showing that neurofeedback training can improve perceptual sensitivity for visual stimuli and task that were previously untrained and were not related to the neurofeedback training. Rather than training a specific pattern of visual cortex activity that corresponded to a specific visual stimulus (Shibata et al., 2011), we trained self-regulation of mean visual cortex activity. We then showed the perceptual consequences of learned self-regulation using previously unseen near-threshold visual stimuli and a previously untrained simple detection task. They were chosen to maximize effect size (weak visual stimulus) and to minimize dual task interference with self-regulation (simple detection task). However, other studies have shown that spontaneous fluctuations of ongoing neural activity have an impact on perception using a variety of tasks and stimuli (Arieli et al., 1996; Ress et al., 2000; Boly et al., 2007; Fox and Raichle, 2007; Hesselmann et al., 2008a,b). We therefore expect that self-regulation of early visual cortex activity will have a similar impact on other stimuli/tasks that are processed in the visual target ROI (although they might be much harder to detect).
Shibata et al. (2011) found enhanced perceptual sensitivity after neurofeedback training even when participants did not actively self-regulate their visual cortex activity. They interpreted this as evidence that neurofeedback training induces lasting plastic changes in early visual areas. We found that the perceptual effects in our participants were specific to the time when they were upregulating (Fig. 3). We did not find general perceptual improvements that were independent of actively self-regulating visual cortex activity (Fig. 10). Hence, the enhanced visual sensitivity that we observed is a result of temporally increased ongoing activity in visual cortex attributable to participant's cognitive efforts rather than lasting plastic changes.
Excluding effects not related to neurofeedback training
How can we be sure that the enhanced visual sensitivity was a consequence of the brain changes attributable to neurofeedback training? Could such sensitivity differences have already existed before neurofeedback training? Because we did not measure perceptual threshold differences between upregulation and baseline blocks before the neurofeedback training, we cannot completely exclude preexisting differences in visual sensitivity between these two conditions. However, if we assume that the visual sensitivity difference between baseline and upregulation blocks that we found in the learners existed already before training, then we would have also observed this perceptual difference in the non-learners and the controls. It is rather unlikely that we by chance assigned participants to the experimental groups in a way that only the learners show a significant positive difference in visual sensitivity. Participants were randomly assigned to the experimental groups, and the groups were well matched with respect to age, gender, and education. We nevertheless estimated the likelihood for such a scenario by running Monte Carlo simulations. For this, we assumed that the visual sensitivity differences between upregulation and baseline were preexisting for each participant. We then drew randomly with replacement 10,000 possible assignments of the participants to the different experimental groups. The simulations show that, when assuming preexisting visual sensitivity differences, the likelihood that we by chance obtain a significant positive effect for the learners is 0.042. The likelihood that we by chance obtained a significant positive effect only for the learners but not for the control subjects and not for the non-learners was 0.041. Hence, even with the limited number of participants in our study, it is extremely unlikely that our results are attributable to preexisting differences.
Likewise, if we assume that the visual sensitivity difference between baseline and upregulation blocks that we found in the learners is attributable to other factors that are unrelated to neurofeedback learning (such as, for example, artifacts of the experimental design, dual task interference between self-regulation and visual detection, shifts of attention that are independent of the neurofeedback training, etc.), then such factors would also be evident in the non-learners and the control group. However, only the learners showed such a perceptual difference, so we conclude that it must be attributable to the neurofeedback training.
The nature of neurofeedback learning
What could account for the ability of some of our participants to learn voluntary control? Self-regulation of visual cortex activity required contingent neurofeedback (participants in the control group did not learn self-regulation; Fig. 3), improved with training (self-regulation is absent at the beginning of the training; Fig. 3), and was spatially specific, being confined to those parts of early visual cortex that overlap with the visual target ROI (Figs. 4, 9). To elucidate the underpinnings of successful neurofeedback learning, we (1) debriefed the participants after each training session, (2) ran standardized psychological questionnaires, (3) included a control group with feedback from a non-visual region, (4) investigated training-related changes in brain activity, and (5) looked into changes in brain connectivity as a function of neurofeedback training.
Based on introspective measures, we found no difference between those participants who learned to control their visual cortex activity and those who did not. Non-learners and controls used similar visual imagery strategies as the learners (Fig. 6), and they showed the same attentional effort and vividness of visual imagery as the learners (Fig. 7). However, this does not exclude the possibility that the neurofeedback was used to acquire an optimal cognitive control strategy for those participants who successfully learned controlling their visual cortex activity. Our introspective measures might have not been sensitive enough to detect subtle differences between the experimental groups.
Although we found no obvious indication that participants who successfully learned self-regulation differed in their mental strategy (Fig. 6) and psychological (Fig. 7) as well as peripheral–physiological (Fig. 5) measures, we nevertheless found a neural signature in terms of connectivity changes between the trained visual ROI and the SPL (Fig. 11). The SPL is involved in directing covert visuospatial attention (Hopfinger et al., 2000; Yantis et al., 2002; Kelley et al., 2008; Greenberg et al., 2010) and cognitive control (Chiu and Yantis, 2009; Koenigs et al., 2009; Greenberg et al., 2010). Furthermore, covert shifts of attention activate corresponding visual areas (Kastner et al., 1998; Brefczynski and DeYoe, 1999; Li et al., 2008) and improve perception at these locations (Carrasco and McElree, 2001; Carrasco et al., 2004; Liu et al., 2005). Hence, for the learners, the observed increase in connectivity between the visual target ROI and the SPL with training might be a correlate of increasing attentional and cognitive control to learn self-regulation and mediate the improvement in visual sensitivity. The absence of such connectivity in the non-learners and the controls might explain their failure.
However, it is not known whether the ability to control the feedback signal is attributable to a feedback-guided search for an explicit cognitive control strategy or attributable to operant conditioning based on reinforcement by the feedback (Thorndike, 1898; Skinner, 1953). Even in animals, whose cognitive abilities are probably not as evolved as in humans, volitional control of the activity of even single neurons can be achieved through neurofeedback (Olds, 1965; Fetz, 1969, 2007). Feedback studies using electroencephalography (EEG) or physiological signals, such as heart rate, found evidence for both mechanisms (Lacroix and Roberts, 1978; Roberts et al., 1984, 1989; Dunn et al., 1986; Schober and Lacroix, 1986; Utz, 1987, 1994; Siniatchkin et al., 2000; Kotchoubey et al., 2002; Neumann et al., 2003). The learning process seems to also depend on the period of time over which is learned (it might change over time from the search for a control strategy to more automatic reinforcement learning) and on the population that is tested. Children, for example, are more sensitive to reward and reinforcement than adults who have more advanced cognitive abilities (Gelman, 1978; Dragoi and Staddon, 1999; Siniatchkin et al., 2000). However, because of the differences between EEG- and real-time fMRI-based neurofeedback, these findings might not be transferable. In the field of real-time fMRI-based neurofeedback, only Bray et al. (2007) explicitly used operant conditioning of neural activity by presenting monetary reward feedback rather than feedback of brain activity. It is therefore possible that the learning processes by which the participants learn control over the feedback signal are rather implicit and do not necessarily require explicit cognitive processing, which is accessible via introspection. Future studies could test this hypothesis directly.
Implications of brain training with neurofeedback
Until now, neurofeedback was mainly used to train self-regulation of autonomic functions or of specific EEG components. In healthy volunteers, EEG feedback was successfully used to manipulate brain activity in a way that resulted in, for example, decreased reaction times in a motor task (Rockstroh et al., 1990), faster lexical decisions (Pulvermuller et al., 2000), altered emotional responses (Allen et al., 2001), increased attention (Egner and Gruzelier, 2001; Vernon et al., 2003), and enhanced musical performance (Egner and Gruzelier, 2003). EEG feedback was also successfully used in patients. For example, it was applied to communicate with severely paralyzed patients (Birbaumer et al., 1999; Kübler et al., 2001), to suppress epileptic activity (Kotchoubey et al., 2001; Sterman and Egner, 2006), and to treat symptoms of attention deficit hyperactivity disorder (Thompson and Thompson, 1998; Monastra et al., 2002; Fuchs et al., 2003; Heinrich et al., 2004). Although EEG-based feedback benefits from a very high temporal resolution, it is limited with respect to spatial specificity and thus the choice of brain regions that can be targeted.
Neurofeedback based on real-time fMRI offers the advantage of learning to control spatially localized brain activity in the range of millimeters across the entire brain. So far, few studies have used this technically challenging method. They have demonstrated the feasibility of self-regulating activation in specific brain areas (Yoo and Jolesz, 2002; Posse et al., 2003; Weiskopf et al., 2004b; deCharms et al., 2004; Yoo et al., 2006; Johnston et al., 2010; Mathiak et al., 2010). Some studies have even shown that self-regulation leads to behavioral effects that are specific to the functional role of the targeted cortical area (Weiskopf et al., 2003, 2004a; deCharms et al., 2005; Bray et al., 2007; Caria et al., 2007; Rota et al., 2009; Haller et al., 2010; Shibata et al., 2011; Subramanian et al., 2011).
The neurofeedback approach goes beyond conventional fMRI studies that are correlational. It allowed us to induce changes in brain activity locally and without drugs, in keeping with recent invasive neurofeedback studies in nonhuman primates (Schafer and Moore, 2011). Similar to approaches that disrupt brain activity either temporarily (e.g., transcranial magnetic stimulation) or permanently (e.g., brain lesions), the neurofeedback approach now permits us to regard perception or behavior as the variable dependent on manipulation of brain activity, thus enabling us to establish a causal link between brain activity and perception.
Footnotes
This work was supported by the Swiss National Science Foundation, the European Union, and the Wellcome Trust. We thank Christian Kaul, Lauri Jalkanen, and Eric Featherstone for help with data collection. We thank Hester Breman, Nico Bunzeck, Rainer Goebel, Narly Golestani, Michael Herzog, Frank Jaekel, Amanda Kaas, Yury Koush, Toni Saarela, Sam Schwarzkopf, Ranganatha Sitaram, Bettina Sorger, and Mikhail Zvyagintsev for insightful discussions.
- Correspondence should be addressed to Frank Scharnowski; Wellcome Trust Centre for Neuroimaging, UCL Institute of Neurology, University College London, 12 Queen Square, London WC1N 3BG, UK. frank.scharnowski{at}fil.ion.ucl.ac.uk