Abstract
The neural correlates for retention of visual information in visual short-term memory are considered separate from those of sensory encoding. However, recent findings suggest that sensory areas may play a role also in short-term memory. We investigated the functional relevance, spatial specificity, and temporal characteristics of human early visual cortex in the consolidation of capacity-limited topographic visual memory using transcranial magnetic stimulation (TMS). Topographically specific TMS pulses were delivered over lateralized occipital cortex at 100, 200, or 400 ms into the retention phase of a modified change detection task with low or high memory loads. For the high but not the low memory load, we found decreased memory performance for memory trials in the visual field contralateral, but not ipsilateral to the side of TMS, when pulses were delivered at 200 ms into the retention interval. A behavioral version of the TMS experiment, in which a distractor stimulus (memory mask) replaced the TMS pulses, further corroborated these findings. Our findings suggest that retinotopic visual cortex contributes to the short-term consolidation of topographic visual memory during early stages of the retention of visual information. Further, TMS-induced interference decreased the strength (amplitude) of the memory representation, which most strongly affected the high memory load trials.
Introduction
The visual short-term memory (VSTM) system effectively retains visual sensory information well after the sensory stimulation has ended. Short-term retention may include the consolidation of visual information into the neural system, in which the information can be manipulated and used to guide our future behavior (Jonides et al., 2008). Neurophysiological and brain imaging studies suggest that the neural correlates of VSTM are separate from those that encode the sensory information (Goldman-Rakic, 1995; Miller et al., 1996; Courtney et al., 1997; D'Esposito et al., 2000; Munk et al., 2002; Linden et al., 2003). However, there is increasing evidence that sensory cortex may play an important role in STM (Fuster, 1995; Pasternak and Greenlee, 2005). Several animal neurophysiology studies reported increased activity in higher-level (Fuster and Jervey, 1982; Miyashita and Chang, 1988) as well as low-level (Supèr et al., 2001) visual areas during visual short-term retention. Further, while most functional magnetic resonance imaging (fMRI) studies of VSTM did not reveal increased activity in sensory brain areas during retention (Courtney et al., 1997; Munk et al., 2002; Linden et al., 2003), recent fMRI studies showed that visual cortex may contribute to retention of visual information in other ways than increased activity (Harrison and Tong, 2009; Tambini et al., 2010).
The temporal window for visual short-term consolidation may last ∼500 ms into the retention interval (Shapiro et al., 1994; Chun and Potter, 1995; Giesbrecht and Di Lollo, 1998; Jolicoeur and Dell'Acqua, 1998; Lalonde and Chaudhuri, 2002; Vogel et al., 2006). Earlier studies using the attentional blink paradigm showed that identification of a target stimulus is impaired if it is presented 200–500 ms after an initial stimulus (Shapiro et al., 1994; Chun and Potter, 1995). More recent behavioral experiments showed impaired memory performance if a distractor, or memory mask, is presented up to 500 ms after presentation of the memory item (Magnussen et al., 1996; Jolicoeur and Dell'Acqua, 1998; Vogel et al., 2006). The distractor may interfere with the memory representation of the target stimulus, thereby affecting the window of consolidation more directly (Magnussen and Greenlee, 1999; Enns and Di Lollo, 2000).
It remains unclear how sensory cortex contributes to this consolidation window. Here, we investigated the functional role of human early visual areas in VSTM consolidation using transcranial magnetic stimulation (TMS). TMS can be used to manipulate activity in retinotopic visual cortex (Amassian et al., 1989; Kastner et al., 1998; Kammer et al., 2005a,b; Sack et al., 2009a; Thielscher et al., 2010) and chart the time course of behavioral relevance of the stimulated area (Pascual-Leone et al., 2000). In our study, participants saw a visual stimulus array in either their left or right visual field and had to report whether a subsequent test array differed from the previously presented memory array. We administered TMS to one visual field (spatial specificity) at different time points (temporal specificity) during the consolidation window, which allowed us to directly test the functional relevance, topography, and temporal characteristics of early visual cortex contribution in VSTM.
Materials and Methods
Participants.
We ran a behavioral pilot experiment, for which we recruited 11 participants. Nine of the 11 participants reached the performance threshold during training [five females; one left-handed; mean (SD) age of 24.8 (2.2) years].
For the TMS experiment, we recruited 19 participants. On 14 participants, we positioned the TMS coil over the scalp using phosphene localization; six of these participants were later excluded (two because of failing to reach the performance threshold during training and four because phosphenes could not be elicited reliably). In the other five participants, we used fMRI-based neuronavigation to position the coil; the data of one participant were excluded because of seeing phosphenes during the main part of the experiment. The remaining 12 participants (six females; one left-handed) had a mean (SD) age of 25.3 (3.8) years. The TMS protocol followed published safety guidelines (Wassermann, 1998; Rossi et al., 2009) and was approved by the Medical Ethical Committee of the Academic Hospital of Maastricht.
For the memory-masking experiment, we recruited 12 participants. All participants reached the required performance threshold during training [seven females; four left-handed; mean (SD) age of 20.8 (1.3) years].
Participants were recruited from the academic environment of Maastricht University, The Netherlands. All participants provided written informed consent before participation, and TMS participants also filled out and signed health screening forms that were approved by a medical supervisor and an independent physician. All participants received financial compensation or course credits for their participation.
Stimulus material and presentation.
Pictures were gray surfaces (luminance, 68.7 cd/m2) of nonnatural shapes (BORTS: blurred outlines of random Tetris shapes; courtesy of N. Kriegeskorte, MRC Cognition and Brain Sciences Unit, Cambridge, UK), which could not be easily verbalized or conceptualized (Linden et al., 2003; Peters et al., 2009). Pictures spanned 1.5° in width and height and were drawn on a white background (227 cd/m2) that spanned the entire computer screen (34° × 27° visual angle). For the pilot and memory-masking experiments (see below), we created a distractor stimulus (memory mask) from the average of all the abstract shapes used in the experiments. Participants were seated in front of a PC-controlled computer screen (refresh rate, 60 Hz) at an eye–screen distance of ∼57 cm. Stimulus presentation and timing, and logging of individual responses were controlled using the Presentation software (Neurobehavioral Systems).
The experimental task was a variant of the change detection task. A memory trial (Fig. 1) started with a brief presentation (500 ms) of an arrow (< or > symbol) at the fixation location to indicate in which visual field the trial would be presented. The arrow was briefly replaced by a fixation symbol (+, duration 500 ms), after which the remainder of the trial started, including presentation of the sample stimulus array, retention of the sample in working memory, presentation of the test stimulus array, and the participant's response.
The sample array contained one or three gray-scale abstract pictures, which were presented simultaneously. Pictures were centered at an eccentricity of 6° visual angle in the lower visual fields on either the left or right side of the fixation symbol. The respective polar angle of the pictures were 25°, 45°, and 65° from the horizontal meridian in the left and right lower quadrants (for stimulus presentation for the lower left visual field, see Fig. 1, top left). The sample array was presented for 150 ms and was followed by a blank screen and superimposed fixation symbol for 1500 ms, during which the sample array had to be retained in memory. A test array followed the retention period at the same location as the sample array. The test array contained either the same pictures as the sample array or had one different picture. For the three-picture stimulus arrays, the altered picture had an equal probability to be presented at any of the three positions. Participants were required to judge whether the test array was the same as the sample array (i.e., SAME response, pressing the Z key with the left hand) or not (DIFFERENT response, pressing the/key with the right hand). Participants were instructed that accuracy was more important than response speed, but that there was a time limit of 5 s.
General procedure.
Participants first completed a number of practice sessions, in which 24 change detection trials were presented (four blocks of six trials, two blocks in each visual field). Participants were required to attain a minimum accuracy of 20/24 trials, which was commonly attained within two to five sessions. For each practice trial, participants received feedback about their performance (green or red fixation cross indicated correct or false judgment, respectively). Afterward, participants completed a baseline session (60 trials across two hemifields and two memory load conditions; six blocks of 10 trials each) during which no TMS pulses or masking stimuli were delivered. Finally, participants completed the main memory experiment, in which a TMS pulse or memory mask was presented at one of three time points during the retention interval [i.e., after offset of the sample array; interstimulus intervals (ISI) of 100, 200, and 400 ms; 60 trials per time point; 18 blocks of 10 trials each]. Consecutive trials in each block were separated by 3000 ms during which participants only saw the fixation cross at the center of the screen. Visual field position of the memory trials varied in a block-by-block fashion (i.e., left-right-left). During the training and baseline experiments, memory load was randomized within and across blocks. During the TMS and memory-masking experiments, memory load and time point of interference (TMS pulse or memory mask) were randomized within and across blocks, with equal probability for each Load × Time point combination. In the pilot experiment, the location of the memory mask was always spatially congruent with the memory trial. In the TMS and the memory-masking experiment, the pulse or memory mask was presented at a fixed visual field location so that it was spatially congruent to 50% of the memory trials. Specifically, in the TMS experiment, pulses were delivered to the left or right visual field in 11 or one participants, respectively. In the memory masking experiment, distractor stimuli were presented in the left or right visual field in eight or four participants, respectively.
Transcranial magnetic stimulation.
Biphasic TMS pulses were delivered over lateral visual cortex using a figure-of-eight coil (MC-B70, the inner and outer radii of the two coil loops were 1.2 and 5.4 cm, respectively Fig. 1, bottom left) and a MagPro R 30 stimulator (maximum stimulator output, 1.9 T; Medtronic Functional Diagnostics).
We used phosphene localization to determine the coil position over the scalp in eight participants (Pascual-Leone and Walsh, 2001; Campana et al., 2002; Silvanto et al., 2005; Romei et al., 2008; Cattaneo et al., 2009; Sack et al., 2009a). Here, the location and intensity of pulse delivery was determined by localizing TMS-induced phosphenes as close as possible to the visual field location of the center stimulus of the sample array. TMS-induced phosphenes have been associated with local transient changes in activity of early visual areas (Amassian et al., 1989; Meyer et al., 1991; Kammer et al., 2005a).
Phosphene localization was performed with a black computer screen, which facilitated phosphene detection and localization. Phosphene localization started with 60% of maximum stimulator output intensity and the center of the coil was positioned ∼2 cm above the inion. The coil was then moved laterally and single TMS pulses were applied until a clear phosphene was perceived in the contralateral visual field. Phosphene perception had to depend on stimulated hemisphere and retinotopic organization to be considered a valid TMS-induced percept (Meyer et al., 1991; Sack et al., 2009a). This procedure aimed at positioning the TMS coil in such a way that the induced phosphenes overlapped with or were in close proximity to the spatial location of the sample and test arrays. We used custom-written software to allow participants to monitor and register phosphene locations in the visual field. After localization, the TMS coil was fixed in a mechanical arm coil holder and placed tangentially on the skull. Three phosphenes were then elicited to ensure that phosphene location was not altered. In seven of the eight participants, we administered TMS pulses to the right hemisphere (left visual field), thereby using the left hemisphere as control. In one participant, reliable phosphenes could only be elicited in the left hemisphere (right visual field), with the right hemisphere used as control.
Phosphene localization is inherently based on subjective reports of phosphene perceptions, in which phosphenes may not be reliably produced in some participants. Therefore, we used fMRI-based neuronavigation (Sack et al., 2009b) to position the coil over the V1/V2 complex of the upper convex of the calcarine fissure (lower visual field) of the right hemisphere in four participants. Neuronavigation was based on the results of an fMRI localizer of the stimulus position in the lower left visual field (6° eccentricity), which was acquired in a separate, unrelated study (De Weerd et al., in press). Note that for these participants, the coil position was independent of whether or how reliably phosphenes could be induced. TMS neuronavigation followed procedures described previously (Sack et al., 2009b). After coil positioning, we attempted to induce phosphenes. Two participants saw phosphenes in the same part of the visual field as the participants in whom coil position was based on phosphene localization did. We could not elicit phosphenes in the other two participants.
During the main experiment, single-pulse TMS was applied at stimulator output intensity equal to 110% of phosphene threshold to correct for the white background of the screen during presentation of the memory experiment. Visual exposure to dimmed or dark luminance markedly decreases phosphene thresholds (Boroojerdi et al., 2000). Stimulus presentation and pulse triggering were controlled using Presentation (Neurobehavioral Systems). For the two participants who did not see phosphenes, stimulation intensity during the remainder of the experiment was set at 45% machine output.
Analysis.
The dependent variable in all experiments was accuracy, which was estimated according to signal detection theory rationale that a decision criterion to correctly judge the presence of a signal results from the deviation between signal and noise distributions (MacMillan and Creelman, 2005). Hit rate (H) was calculated as the number of hits (correctly judging that sample and probe were the same) divided by the sum of number of hits and misses (i.e., H = hits/(hits + misses)). False alarm rate (FA) was calculated as the number of false alarms (erroneously judging that sample and probe were the same) divided by the sum of the number of false alarms and correct rejections (i.e., FA = false alarms/(false alarms + correct rejections)). We used the nonparametric A′ (A-prime) index (Snodgrass and Corwin, 1988; Stanislaw and Todorov, 1999; MacMillan and Creelman, 2005) as a measure of accuracy independent of response bias. Originally, A′ is estimated using two formulas: when H > FA, and when FA > H.
These equations can be rewritten into a single formula, as follows (Stanislaw and Todorov, 1999): A′ presents a nonparametric alternative to the more often used d′ (d-prime) when assumptions of normality that are required for d′ are suspect to violation (Stanislaw and Todorov, 1999; MacMillan and Creelman, 2005). In our case, we used A′ rather than d′ because we obtained a relatively small sample of responses per cell, and because some participants showed perfect performance (i.e., H = 1) in some conditions. We also ran the main analyses using d′, and results were similar.
Accuracy was analyzed using repeated-measures ANOVA (RMANOVA) models. For the pilot experiment, we first verified that responses in the two visual fields were not significantly different, using a Field (left, right) × Load (1, 3) × ISI (100, 200, 400) RMANOVA model with first- and second-order interaction terms. We then proceeded with a Load (1, 3) × Mask (100, 200, 400) RMANOVA model that also included the interaction term. For the TMS experiment, we used a Field (TMS, Control) × Load (1, 3) × TMS (100, 200, 400) RMANOVA model with first- and second-order interaction terms. We used a similar model for analysis of the memory-masking experiment [i.e., Field (Mask, Control) × Load (1, 3) × ISI (100, 200, 400), with first- and second-order interaction terms]. Effects with p values at or below the alpha level of 0.05 were considered significant. Post hoc comparisons of mean differences were performed using two-tailed one-sample t tests (Holm-Bonferroni corrected for multiple comparisons). Effect sizes for RMANOVA were further estimated for significant effects [partial η squared (η2) for ANOVA effects and Cohen's d for t test effects].
Results
Pilot experiment of short-term memory and visual masking
In this experiment, we presented lateralized short-term memory trials and memory masks that were always spatially congruent with the memory trials to investigate the effect of memory masking independent of visual field. In an initial analysis, we verified that memory masks presented in each visual field provided similar response patterns. We analyzed percentage accuracy of memory performance using a 2 × 2 × 3 RMANOVA with factors Visual Field (left or right), Load (1 or 3), and ISI (100, 200, or 400). We found a significant main effect of Load (F(1,8) = 238.2, p < 0.001), but no significant effects related to Field (F(1,8) = 1.02, p = 0.34), which confirmed that memory masking did not differentially affect performance of the two visual fields. Further, we found trend effects for ISI (F(1,8) = 3.0, p = 0.079) and for the Load × ISI interaction (F(2,16) = 3.1, p = 0.073).
To obtain more power, we pooled the results across the two visual fields and recalculated a 2 × 3 RMANOVA (Load × ISI). We found a significant main effect of Load (F(1,8) = 241.0, p < 0.001, η2 = 0.97) and a significant Load × ISI interaction effect (F(2,16) = 3.6, p = 0.052, η2 = 0.31). Figure 2 shows the mean accuracies across all conditions (data pooled across Field; Table 1). The interaction effect was based on a decrease of accuracy when the mask was presented 200 ms into the retention interval (mean A′ = 0.71), compared with the other two time points (mean A′ ISI 100 = 0.78; ISI 400 = 0.78), only during high memory load trials (Table 1). These results indicated that presentation of a memory mask ∼200 ms into the retention interval interfered with subsequent recognition in high memory load trials.
Short-term memory and TMS
In this experiment, we presented lateralized short-term memory trials while TMS pulses were always delivered over retinotopic occipital cortex (affecting one visual field location). Thus, TMS pulses were spatially congruent with 50% of the memory trials.
Accuracy was analyzed using a 2 × 2 × 3 RMANOVA model (Field × Load × TMS). To verify that the results were not biased by the two different methods of coil positioning, we added Positioning Method (phosphenes, neuronavigation) as between-subject factor.
With respect to positioning method, we found a significant Load × Method interaction effect (F(1,10) = 5.2, p = 0.046, η2 = 0.34). Participants of the neuronavigation method showed slightly better performance for low memory load trials [mean (SD) A′ = 0.94 (0.04)] compared with those of with phosphene localization [mean (SD) A′ = 0.89 (0.04)], but slightly worse performance for high memory load trials [mean (SD) A′ = 0.64 (0.11)] than those with the phosphene localization [mean (SD) A′ = 0.71 (0.12)]. All other interactions with Method were not significant (ps > 0.22). Thus, localization method did not affect the pattern of results. We pooled the data of participants from both positioning methods in the following analyses.
For the within-subject factors, we found a significant main effect of Load (F(1,11) = 57.8, p < 0.001, η2 = 0.84), a significant Field × TMS interaction effect (F(2,22) = 3.9, p = 0.037, η2 = 0.26), and a significant Field × Load × TMS interaction effect (F(2,22) = 4.3, p = 0.026, η2 = 0.28). To parcel out these effects, we calculated 2 × 3 RMANOVAs for each of the two memory loads. For Load 1, we found no significant main or interaction effects (ps > 0.59). For Load 3, we found a significant Field × TMS interaction effect (F(2,22) = 5.0, p = 0.016, η2 = 0.31). Figure 3 shows the mean accuracies across the conditions (Table 1).
These results corroborated and extended the pilot memory-masking results. The interactions were based on a decrease of accuracy when the spatially congruent TMS pulse was presented 200 ms into the retention interval, compared with the spatially incongruent pulse, only for the high memory load (Table 1). Post hoc contrasts between TMS and control visual fields for the time points during high memory load trials showed a significant TMS-induced impairment in performance at the 200 ms time point (mean decrease A′ = −0.21; t(11) = −2.9, p = 0.013, Cohen's d = 0.85), but not at the other two time points. Further post hoc one-tailed comparisons in the TMS condition of the decrease in accuracy at 200 ms compared with the other two time points were significant ([A′100 > A′200]: t(11) = −1.9, p = 0.044; [A′400 > A′200]: t(11) = −2.3, p = 0.021). We found no significant differences within the control condition.
Short-term memory and visual masking
In this experiment, we presented lateralized short-term memory trials while memory masks were always presented in one visual field location. Thus, memory masks were spatially congruent with 50% of the trials. Accuracy was analyzed using a 2 × 2 × 3 RMANOVA model (Field × Load × ISI). We verified that the choice of control visual field (right in eight participants, left in four) did not affect the results by adding Mask Location (left, right) as between-subject factor. Mask Location did not show interaction effects with any of the within-subject factors (ps > 0.32), which further corroborated the finding from the pilot study that memory performance did not differ between visual fields.
For the within-subject factors, we found a significant main effect of Load (F(1,11) = 239.1, p < 0.001, η2 = 0.96) and a significant Field × ISI interaction effect (F(2,22) = 5.5, p = 0.012, η2 = 33). To parcel out these effects, we recalculated 2 × 3 RMANOVAs for each of the two memory loads (i.e., Field × ISI). For Load 1, we found no significant main or interaction effects (ps > 0.31). For Load 3, we found a significant Field × TMS interaction effect (F(2,22) = 4.1, p = 0.031, η2 = 0.27). Figure 3 shows the mean accuracies across the conditions (see also Table 1).
These results corroborated and extended the pilot and TMS results. The interactions were based on a decrease of accuracy when the spatially congruent mask was presented 200 ms into the retention interval, compared with the spatially incongruent mask (Table 1). This effect was present only for the high memory load. Post hoc contrasts between mask and control visual fields for the 200 ms time point showed that the mask-induced performance impairment was close to significant at the uncorrected p value (mean decrease A′ = −0.12; t(11) = −2.1, p = 0.056, Cohen's d = 0.62), but not at the corrected p value. The other two masking time points showed no significant differences (ps > 0.7). Further post hoc comparisons in the TMS condition of the decrease in accuracy at 200 ms compared with the other two time points were significant ([A′100 > A′200]: t(11) = −2.1, p = 0.028; [A′400 > A′200]: t(11) = −2.4, p = 0.016). We found no significant differences within the control condition. These results suggest that participants showed a similar pattern of interference at 200 ms compared with the TMS experiment.
Discussion
To investigate the functional relevance, topography, and temporal characteristics of early visual cortex contribution to VSTM consolidation, we presented memory masks or administered topographically specific TMS pulses affecting the visual field contralateral to the side of TMS at different time points within 100–400 ms into the retention interval of a change detection task with low and high memory loads, and used the other visual field as within-subject control. We found a very strong correspondence in memory interference between the TMS and masking experiments. Specifically, memory masks and TMS pulses decreased accuracy only in the targeted visual field, revealing the topographic specificity of our TMS-induced memory consolidation interference in early visual cortex. Furthermore, accuracy in the targeted visual field decreased only when the memory mask or TMS pulse was administered at 200 ms into the retention window (temporal specificity), and most strongly for the high memory load (capacity limit). These findings not only provide direct evidence for the functional necessity of intact neural processing in early visual cortex at 200 ms into the retention phase, but also suggest a load dependence of this functional relevance in memory consolidation. The similar findings of both experiments suggest that interference of activity in visual cortex reliably impaired memory consolidation, thereby affecting memory performance. Our findings are in line with previous interference studies that showed a critical window for VSTM consolidation up to 500 ms after stimulus offset (Chun and Potter, 1995; Magnussen et al., 1996; Giesbrecht and Di Lollo, 1998; Jolicoeur and Dell'Acqua, 1998; Lalonde and Chaudhuri, 2002; Vogel et al., 2006), and reveal a neurophysiological correlate of this consolidation window that includes activation states of topographic early visual cortex, which suggests that VSTM representations may be formed or maintained in sensory brain areas that encoded the stimuli. To our knowledge, we are the first to show that topographic early visual cortex is functionally relevant for short-term consolidation of sensory visual information.
To date, very few studies have investigated the role of occipital cortex in VSTM. In a previous study (Beckers and Hömberg, 1991), participants completed a delayed match-to-sample (DMS) task of face stimuli, during which TMS was applied over the occipital pole before (i.e., during the retention phase) or after (i.e., during memory scanning) presentation of the memory probe. The authors found no significant change in reaction times after pulses were delivered during the memory retention phase. However, pulses delivered during the memory scanning phase significantly decreased the scanning rate. A recent TMS study found that TMS administered at the onset of the DMS retention phase increased reaction times, whereas TMS at the end of the retention phase decreased reaction times (Cattaneo et al., 2009). Crucially, in these studies, effects of TMS on memory performance were found in altered reaction times, but not in accuracy, which suggests that their effects may be mediated by neural mechanisms other than those studied here.
TMS may interfere with signal processing by decreasing signal strength, rather than by adding random noise (Harris et al., 2008). Therefore, TMS may decrease the strength of or overwrite the neural representation of the memory trace in visual cortex. This suggestion is in line with propositions of object substitution in visual memory after presentation of a competing stimulus (Giesbrecht and Di Lollo, 1998; Enns and Di Lollo, 2000). Furthermore, our finding of a lateralized TMS effect on memory performance suggested that the neural memory representation was retinotopically organized, with limited transfer of memorized information to other parts of the visual field (Karni and Sagi, 1993; Schoups et al., 1995; Dill and Fahle, 1997). This spatial specificity of our findings further supports the notion of a local neural representation of memory in visual cortex. Importantly, the interference of TMS on memory performance was not due to impaired visual awareness of the to-be-memorized items. Visual awareness is commonly impaired if a visual mask is presented 60–120 ms after a very brief presentation of the target stimulus (Breitmeyer and Ogmen, 2000). A number of TMS studies that presented pulses in a similar temporal window reported similar impairments of awareness (Amassian et al., 1989; Beckers and Hömberg, 1991; Kammer, 2007; Sack et al., 2009a). In our study, participants were not impaired in their memory performance if TMS or the mask was presented at 100 ms into the retention interval. Instead, TMS impaired memory performance at a later time window, and only for the high memory load, which suggests that TMS pulses interfered with poststimulus processes beyond those of initial encoding.
Also, it is unlikely that TMS decreased accuracy through phosphenes acting as visual masks. Most participants reported seeing no or few phosphenes during the memory experiment, similarly to previous studies (Silvanto et al., 2005; Cattaneo et al., 2009). Furthermore, even if participants saw phosphenes but failed to report them, phosphene masking cannot explain the larger TMS-induced impairment compared with the smaller impairment in the memory-masking experiment. Finally, TMS-induced phosphenes could even increase memory performance. Silvanto and colleagues showed that TMS-induced phosphenes may contain previously learnt or memorized information (Silvanto et al., 2007; Silvanto and Cattaneo, 2010). Participants could possibly use such phosphenes to rehearse the information during retention and increase recognition accuracy.
Alternatively, TMS may have interfered with ongoing communication between visual cortex and higher-level areas that may occur during periods of consolidation, as is shown in recent fMRI studies (Lewis et al., 2009; Tambini et al., 2010). Furthermore, the time point of impairment may be associated with dynamically altering states of brain activity in sensory and higher-level areas. For example, Lamme and Roelfsema (2000) proposed that visual perception is the result of sweeps of activity running up and down the visual hierarchy. Incoming visual sensory information is initially processed in a feedforward sweep up the hierarchy that occurs within 40 and 70 ms after stimulus onset. Higher-level brain areas than feed information back to early visual areas within 80 and 120 ms after stimulus onset. The feedforward sweep may be associated with initial stimulus encoding, while the feedback sweep may be associated with visual awareness or attention. In our study, the time point of interference may indicate that alteration of activity in early visual cortex may have affected a later feedback sweep to early visual cortex during short-term memory consolidation, suggesting that memory consolidation requires additional forward and backward sweeps beyond those needed for visual awareness.
Finally, we found that the functional relevance of early visual cortex to memory consolidation depended on the capacity constraints of VSTM (Luck and Vogel, 1997; Cowan, 2000). Retaining memory loads that approach the capacity limit is more easily or more profoundly impaired by the processing of distractors (Cowan, 2000; Vogel et al., 2006). Previous fMRI studies showed a neural correlate for the capacity limitation of VSTM in frontal and parietal cortex (Linden et al., 2003; Todd and Marois, 2004), in which brain activity increased monotonically with higher memory loads until the capacity limit was reached. Furthermore, retention of higher memory loads may be associated with increased functional coupling between these brain areas (Honey et al., 2002; Woodward et al., 2006). Thus, higher memory loads may require more neural resources for memory retention, which may leave limited resources available to protect against interfering signals. This suggestion could further be relevant when considering that we used complex memory items. Previous studies have shown that increased object complexity may constitute a higher information load, which thereby limits the object capacity of short-term memory (Alvarez and Cavanagh, 2004; Eng et al., 2005; Luria et al., 2010). Therefore, complexity-imposed shrinkage in VSTM capacity could be particularly costly when the number of memory items approaches the capacity limit. In this sense, trials of high memory and information load should be more prone to interfering signals, which is exactly what we found. However, the nature of the TMS-induced interference on the memory representation in early visual cortex may be different for simple stimuli, such as color or gratings.
The larger TMS-induced impairment compared with that after memory masking may indicate that the nature of the TMS interference is different from that of memory masking. TMS pulses may have directly affected relevant neural populations, interfering with cortical activity as the memory was processed. In contrast, the memory mask was presented to the retina, and traversed the same neural pathways as did the to-be-remembered stimuli. Here, interference effects may be much more subtle. Alternatively, the memory mask may not have provided enough overlap with relevant stimulus dimensions of the abstract shapes, thereby limiting its interference on memory-related processing of the target stimuli (Magnussen and Greenlee, 1999; Lalonde and Chaudhuri, 2002).
In conclusion, we showed that early visual cortex topographically contributes to consolidation of visual information in VSTM early within the retention window. Furthermore, TMS likely decreased the signal strength of the memory representation, which affects performance most strongly when the memory load approaches the capacity limit of VSTM. Finally, the specific time point of interference appears in line with a sequential pattern of discrete temporal windows during which visual cortex contributes to visual perception.
Footnotes
-
This work was supported by the Netherlands Organization for Scientific Research (Grant 451-07-014 to V.V.) and by the European Research Council under the European Union's Seventh Framework Programme (FP7/2007-2013)/European Research Council Grant Agreement Number 263472 (to A.T.S.). We kindly thank Drs. David E. J. Linden and Christoph Bledowski for very helpful comments on a previous version of this manuscript, and Peter Schubert and Liselot Kerpershoek for assistance in data acquisition. We thank our medical supervisor Cees van Leeuwen and independent physician Martin van Boxtel.
- Correspondence should be addressed to Dr. Vincent van de Ven, Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, PO Box 616, 6200 MD, Maastricht, The Netherlands. v.vandeven{at}maastrichtuniversity.nl