Abstract
Search for target objects in visual scenes is guided by mental representations of target features (attentional templates). However, it is unknown when such templates are activated during each search episode and whether this can be controlled by temporal expectations. We used electrophysiological measures to track search template activation processes in real time. In three experiments, female and male humans searched for a color-defined target object in search displays where targets were accompanied by distractors in different nontarget colors. Brief task-irrelevant color singleton probes that matched the target template were flashed rapidly (every 200 ms) throughout each block. Probes presented at times when the target template is active should capture attention, whereas probes presented at other times should not. To assess this, N2pc components were measured as markers of attentional capture, separately for probes at each successive temporal position between two search displays. Results demonstrated that search templates were active from ∼1000 ms before the arrival of the next search display, and were deactivated after each search episode, even when the preceding search display did not contain a target object. Templates were activated later when the predictable interval between search displays was increased. Results demonstrate that search templates are not continuously active but are transiently activated during the preparation for each new search episode. These activation states are regulated in a top-down fashion by temporal expectations about when an attentional template will become task-relevant.
SIGNIFICANCE STATEMENT It is often assumed that observers prepare for a visual search task by activating mental representations of search target objects (attentional templates). However, the time course of such template activation processes is completely unknown. By using a new sequential probe presentation technique and electrophysiological measures of attentional processing, we demonstrate that target templates are rapidly activated and deactivated before and after each successive search display, and that these template activation states are tuned to observers' temporal expectations. These results provide novel insights into the temporal dynamics of cognitive control processes in visual attention. They show that attentional templates for visual search are preparatory states that are activated in a transient fashion before each new search episode.
- attention
- cognitive control
- event-related brain potentials
- visual search
- working memory
Introduction
In visual search tasks, observers have to find task-relevant target objects and ignore task-irrelevant distractors. When attributes of search targets are known in advance, the search process can be controlled by attentional templates, working memory representations of the features that distinguish targets from nontarget objects (e.g., Duncan and Humphreys, 1989; Wolfe and Horowitz, 2004). Search templates guide selective attention toward objects with template-matching features. Once selected, these objects can then be identified as targets or distractors (e.g., Cunningham and Wolfe, 2014). To facilitate search, target templates must be available at the right moment in time. For this reason, it is often assumed that attentional templates are activated in a preparatory fashion before a particular search episode (e.g., Desimone and Duncan, 1995), but little is known about the time course of such template activation processes. Search templates may become activated as soon as the identity of a target is known, and remain active until they are no longer task-relevant. Such sustained activation processes have been found in recent fMRI studies that used decoding methods to detect memorized target templates during the preparation for a memory test (e.g., Lewis-Peacock et al., 2015). Another possibility is that template activation is triggered in a reactive fashion by the onset of a search display. In this case, templates may be represented in working memory during the preparation for search but remain dormant until they are activated by the arrival of the search display (for the distinction between active target templates and accessory working memory representations, see Olivers et al., 2011). A third possibility is that target templates are activated during the preparation for search, before the arrival of a search display, but that this process is controlled flexibly by temporal expectations. In this case, search templates should be activated transiently only during the period when a search display is likely to appear.
To dissociate these possibilities, methods are required that can track the activation states of search templates with high temporal precision. In this study, we used a new rapid serial probe presentation procedure and used online electrophysiological markers of template activation during the preparation for search. Participants' task was to find and discriminate target objects defined by a particular color in multicolor displays, a task that requires a color-specific search template. Task-irrelevant color singleton probes were presented at a fast rate (every 200 ms) before, during, and after each search display, at different locations closer to fixation (Fig. 1). Critically, half (Experiment 1) or all (Experiments 2 to 4) probes appeared in the target color, and thus matched the search template. These target-color probes should therefore attract attention when the search template is activated (task-set contingent attentional capture) (e.g., Folk et al., 1992), but not when they appear at a time when the template is inactive. To measure probe-induced attentional capture, we recorded EEG during task performance and computed N2pc components for probes at each temporal position between two successive search displays. The N2pc is an enhanced negativity at posterior scalp electrodes contralateral to candidate target objects that typically emerges 180–200 ms after stimulus onset, is generated in ventral extrastriate visual areas (Hopf et al., 2000), and reflects the rapid allocation of attention to candidate target objects in multistimulus displays (e.g., Luck and Hillyard, 1994; Eimer, 1996; Woodman and Luck, 1999; for review, see Eimer, 2014). Previous work has demonstrated close links between the N2pc and attentional capture, with visual stimuli triggering N2pc components if, and only if, they match current target features and also trigger behavioral capture effects (Eimer and Kiss, 2008; Lien et al., 2008; Eimer et al., 2009). By measuring N2pc components to each successive target-color probe, we assessed which of these probes captured attention, to track the temporal profile of template activation processes and to determine whether search templates are activated in a sustained, reactive, or transient fashion.
Materials and Methods
We conducted four experiments to track the temporal profile of template activation processes. In Experiment 1, participants searched for a color-defined bar in circular search displays. This target bar appeared among five nontarget bars in five different colors to ensure that a feature-specific attentional template (e.g., “red”) was required to find the target. For each participant, the target color remained constant for the entire experiment. The task was to report the orientation of the target bar (horizontal or vertical). Throughout each experimental block, circular displays that contained a color singleton probe were flashed every 200 ms, and participants were told to ignore them. These probe displays appeared at a different eccentricity (closer to fixation) than the target displays because pilot testing had shown that presenting multiple rapidly flashing color singleton probes at the same eccentricity as the target displays was extremely distracting and made it very difficult for observers to find and discriminate target items in the search displays. Singleton probes in the target color and in a different distractor color were equally likely, and randomly intermixed. The interval between two successive search displays was constant (1600 ms) and thus fully predictable. One probe display appeared simultaneously with each search display (Probe S), and seven others were presented successively at 200 ms intervals between two search displays (Probes 1–7). N2pc components were measured independently for each of these probes, separately for target-color and distractor-color probes. N2pcs were also computed for target objects in the search displays. Target-color probes that appear at a time when the search template for the target color is active should be able to capture attention and thus trigger N2pc components. The critical question was when during the preparation for the next search episode these Probe N2pcs would emerge. If color templates are activated in a sustained fashion until they are no longer relevant, N2pcs should be triggered by all target-color probes, regardless of their temporal position. If these templates are only triggered reactively, probes that appear in the interval between two search displays should not elicit any N2pc components. Finally, if search templates are activated transiently, and in line with temporal expectations, probes that appear in close temporal proximity to the predicted onset of the next search display (e.g., Probe 7) should elicit N2pc components, whereas no N2pc should be found for earlier probes (e.g., probes presented shortly after the previous search display). Distractor-color singleton probes were included in Experiment 1 to confirm that any capture of attention by target-color probes was mediated exclusively by active color templates, and not by the bottom-up salience of these probes. If this was correct, distractor-color probes should never trigger any N2pc components, regardless of their temporal position.
In Experiment 2, we investigated whether and how the selection and processing of the target object in a search display would affect the subsequent activation of an attentional template during preparation for the next search episode. Here, only half of all search displays contained a target, whereas the other half included six nontarget-color objects. All probe displays now contained a target-color probe, and N2pc components were computed separately for probes that followed target-present and target-absent search displays. If the processing of a search target on the previous trial influences template activation processes on the next trial, the time course of N2pc components should differ between these two types of probes.
Experiment 3 investigated whether and how the time course of template activation processes can be modulated by temporal expectations. In the first two experiments, the interval between two successive search displays was always constant (1600 ms). In different blocks of Experiment 3, this interval was short (1000 ms, including four probes), intermediate (1800 ms, eight probes), or long (2600 ms, 12 probes). If the moment when a search template becomes active can be regulated in line with the expected time of arrival of the next search display, template activation processes should start later when intervals between search displays are longer, and this should be reflected by corresponding temporal differences in the emergence of Probe N2pc components. Another condition of Experiment 3 investigated the time course of search template activation under conditions where the intervals between successive search displays were no longer fixed and predictable but varied randomly across trials (1000, 1400, or 1800 ms).
Because singleton probes and targets always appeared at different locations (i.e., different eccentricities) in Experiments 1–3, no behavioral spatial cueing effects (faster responses for targets at locations previously occupied by a color-matching probe) (Folk et al., 1992) could be measured as additional markers of attentional capture by target-color probes. In Experiment 4, probe and search display items were presented at the same positions, so that targets could appear at the same location that was previously occupied by a target-color probe on some trials. Seven probe displays were presented successively between two search displays. To avoid the processing of search displays to be compromised by the preceding rapid sequence of multiple target-color singleton probes at possible search target locations, only one of the probe displays (Probe 2, 5, or 7) included a target-color singleton on each trial. Behavioral spatial cueing effects and N2pc components were measured separately for these three probe positions. Target-color probes presented in close temporal proximity to the next search display (i.e., Probe 7) were expected to elicit clear N2pc components and also trigger reliable spatial cueing effects for target reaction times (RTs). Such a link between behavioral and electrophysiological markers of attentional capture would confirm that N2pcs to probe stimuli were associated with the allocation of attention to these probes. In contrast, no behavioral spatial cueing effects and no reliable N2pc components should be triggered by probes presented early after the previous search display (i.e., Probe 2).
Participants.
Sixteen paid participants were tested in Experiment 1. One was excluded due to exceedingly high error rates (>40%). Of the remaining 15 participants, 25–42 years of age (mean age 31.4 years), 8 were female and 3 were left-handed. The same 15 participants were paid to take part in Experiment 2, which was conducted in a different session on a different day. Eighteen different paid participants took part in Experiment 3. Three were excluded due to excessive eye movement activity, resulting in a loss of >50% of all data. The remaining 15 participants were 23–43 years of age (mean age 31.8 years). Seven were female and 3 were left-handed. In Experiment 4, 9 different paid participants were tested. They were 23–44 years of age (mean age 33.4 years). Seven were female and 1 was left-handed. All participants tested had normal or corrected-to-normal vision and normal color vision (tested with the Ishihara color vision test) (Ishihara, 1972).
Stimuli and procedures.
Participants were seated in an electrically shielded and dimly illuminated testing booth. Visual stimuli were presented on a computer monitor at a viewing distance of 100 cm. In Experiments 1 and 2, a 17-inch Samsung wide Syncmaster 753S monitor (1280 × 1024 pixels resolution, 100 Hz refresh rate) was used. Stimulus presentation, timing, and response collection were controlled by a LG Pentium PC running under Windows XP, using the Cogent 2000 toolbox (RRID:SCR_015672; www.vislab.ucl.ac.uk/cogent_2000.php) for MATLAB (RRID:SCR_001622; www.mathworks.com/products/matlab/). Experiments 3 and 4 were run in a different laboratory than the first two experiments, using a 24-inch BenQ GL2450HE HD monitor (1280 × 1024 pixels resolution, 100 Hz refresh rate) for stimulus presentation and a Corsair desktop computer with 3.3 GHz quad-core processor (running on Windows 7) for stimulus presentation, timing, and response collection, using MATLAB and the Cogent 2000 toolbox.
Figure 1 shows the time course of stimulation in Experiment 1. Stimuli were presented against a black background. A gray fixation point (CIE x, y color coordinates: 0.287/0.312; 0.2° × 0.2° of visual angle) was continuously presented throughout each experimental block. Each trial contained eight consecutive displays, which were presented for 50 ms and separated by a stimulus onset asynchrony (SOA) of 200 ms. The interval between the offset of the eighth display in a trial and the onset of the first display in the next trial was 150 ms, so that all displays were shown in a continuous serial presentation stream within each block. Seven successive displays contained probe displays (Probes 1–7), and the eighth displays contained both a probe display and a task-relevant search display at different eccentricities (simultaneous target and probe: Probe S). Because search displays were presented in every eighth display, the SOA between two successive search displays was 1600 ms. Search displays contained six differently colored vertical or horizontal bars (each covering 0.2° × 0.6°), which were presented at the 1, 3, 5, 7, 9, and 11 o'clock positions of an imaginary clock face at an eccentricity of 1.4° from central fixation. The six bar orientations were independently and randomly selected in each trial. Possible bar colors were red (0.609/0.327), green (0.296/0.581), blue (0.174/0.149), yellow (0.389/0.512), cyan (0.227/0.376), magenta (0.216/0.110), and gray (0.287/0.312). All colors were equiluminant (∼10.9 cd/m2). Participants' task was to report the orientation of the target-color bar (vertical, horizontal) by pressing the respective of two purpose-built vertically aligned response keys. The response-to-key mapping and the hand-to-key mapping was counterbalanced across participants but was kept constant for each participant for the duration of the whole experiment. For each participant, one of three colors (red, blue, or green) served as target color. The distractor color that would appear in half of all probe displays was chosen from the same set of three colors. Each of these three colors served as target or distractor color for five participants. Search displays always contained one target-color bar, one distractor-color bar, and four bars in yellow, cyan, magenta, and gray.
All probe displays contained six items (0.25° × 0.25° per item) presented at the 1, 3, 5, 7, 9, and 11 o'clock positions of an imaginary clock face at an eccentricity of 0.5°. Each item in these probe displays consisted of two closely aligned vertical and horizontal pairs of dots (0.1° × 0.1° per dot). In each probe display, one of the items was a color singleton and the other five items were uniformly gray (0.287/0.312). The colored probe item either matched the target or the distractor color for each individual participant. The color and location of a singleton probe were selected independently for each probe display, with the exception that immediate repetitions of color probes at the same location were not allowed, and that successive probes would equally likely appear on the same or opposite display sides across displays (to ensure that each color probe was preceded equally often by a target- or a distractor-color probe on the same or opposite display side). Participants were informed that probe displays were task-irrelevant. Because target-color singleton probes appeared in 50% of all probe displays, these probes were predictive of the color of the target (as in most contingent attentional capture experiments) (e.g., Folk et al., 1992). However, previous studies have shown that task-irrelevant color-matching items retain their ability to attract attention, even when they appear among other items in multiple different colors and are thus no longer uniquely predictive of the current target color (Lamy et al., 2004; Eimer et al., 2009).
Experiment 1 contained 40 blocks, with 12 trials per block. Blocks were short to minimize the presence of eye blinks within each block. The 12th search display was followed by seven additional probe displays to keep response conditions identical across all trials in a block. Consequently, each block contained 12 search displays and 103 probe displays (13 for Probes 1–7, and 12 for Probe S, which was presented together with the search display). Before the first experimental block, participants received two practice blocks.
In Experiment 2, the same stimuli and procedures were used, except for the fact that there were no distractor-color probes, as all probe displays contained a target-color singleton. The critical difference to Experiment 1 was that on half of all trials, a target-absent search display was shown that contained six differently colored nontarget bars but no target-color bar. On the other 50% of trials, a target-present search display appeared that was identical to Experiment 1. Target-present and target-absent search displays were randomly intermixed within each block. Participants' task was to indicate the orientation of the target-color bar (vertical, horizontal) and to refrain from responding to target-absent displays. Red, green, or blue served as target color for five participants each. Experiment 2 contained 40 blocks, with 12 trials per block (six trials with target-absent and six with target-present search displays).
In Experiment 3, the critical new manipulation concerned the interval between two successive search displays. In different blocks, there were three constant (predictable) SOAs between successive search displays. These displays were separated by SOAs of 1000 ms (with four intermediate probe displays), 1800 ms (with eight intermediate probe displays), or 2600 ms (with 12 intermediate probe displays) in short, medium, and long SOA blocks, respectively. In addition, there were also variable SOA blocks, with three possible SOAs between consecutive search displays (1000, 1400, and 1800 ms, with 4, 6, and 8 intermediate probe displays, respectively) that varied randomly and unpredictably between trials. Each of the three constant SOA conditions (short, medium, long) was tested in 20 successive blocks. The variable SOA condition was tested in 60 successive blocks. All blocks included 12 trials. Eight participants were presented with the three constant SOA conditions first (with order of SOA condition randomized between participants); the other 7 started with the variable SOA blocks. All probe displays in Experiment 3 contained a target-color singleton.
Experiment 4 contained 100 blocks, with 12 trials per block. Procedures were similar to Experiment 1, with the following exceptions. All probe and search display items were now presented at the same locations, at an eccentricity of 1.4° from central fixation. Seven probe displays were presented in the interval between two search displays, and only one of them (Probe 2, 5, or 7) contained a target-color singleton on each trial. The other six probe displays contained six gray items. Trials with a target-color singleton in Probe 2, 5, or 7 were randomly intermixed in each block. The location of these singleton probes was independent and thus not predictive of the location of the target in the next search display. Target-color probes and target objects could only appear at the 1, 5, 7, and 11 o'clock positions and never at the 3 or 9 o'clock positions, so that the likelihood of a target being presented at the same location as a target-color probe was 25%.
The stimulus colors used in Experiments 3 and 4 were the same as in Experiments 1 and 2, but due to the use of a different monitor, color parameters were slightly different (red: 0.635/0.329; green: 0.291/0.599; blue: 0.155/0.086; yellow: 0.471/0.458; cyan: 0.214/0.308; magenta: 0.265/0.142; and gray: 0.299/0.314; all equiluminant: ∼9.3 cd/m2).
EEG recording.
In all four experiments, EEG was DC-recorded from 27 scalp sites at standard positions of the extended 10/20 system with a sampling rate of 500 Hz. It was digitally low-pass filtered at 40 Hz with no other filters applied after data acquisition. Impedances were kept <5 kΩ. All channels were referenced to the left earlobe during recording and were rereferenced offline to the average of both earlobes. Data from the first seven and the last seven probe displays in each block, as well as trials with incorrect, anticipatory (faster than 200 ms), very slow (slower than 1500 ms), or missing responses did not enter analysis. Furthermore, data contaminated with artifacts (eye movements exceeding ±30 μV in the bipolar horizontal EOG [HEOG] channel; blinks exceeding ±60 μV at Fpz; muscular movements exceeding ±80 μV in all other channels) were excluded from EEG analyses. This led to the removal of 6.6% of all data (ranging between 0.3% and 20.5% across participants) in Experiment 1, of 5.7% (ranging between 0.8% and 13.7%) in Experiment 2, of 9.6% (ranging between 1.3% and 24.5%) in Experiment 3, and of 8.9% (ranging between 3.3% and 23.4%) in Experiment 4. The remaining EEG was segmented into 500 ms time windows ranging from 100 ms before to 400 ms after the onset of a particular display. For color probes, averages were computed for each of the probe displays presented during the interval between two successive search displays (Probes 1 to S, where Probe 1 is the probe that immediately followed the preceding search display, and Probe S is the probe that was presented simultaneously with the next search display). All EEG data processing was performed with Brain Vision Analyzer Software (RRID:SCR_002356; www.brainproducts.com).
Experimental design and statistical analyses.
The experiments were designed to track the activation of target templates during the preparation for an upcoming search display by measuring N2pc components to each of the color singleton probes that were presented at different times during the preparation interval. N2pc components to probes were quantified on the basis of event-related potential (ERP) mean amplitudes measured at lateral posterior electrodes PO7 and PO8 contralateral and ipsilateral to the side of a probe within an 80 ms time window starting at 190 ms after probe display onset. The start of this time window was determined in Experiment 1 by measuring the point in time when the ascending flank of the averaged Probe N2pc (pooled across all eight target-color probes in a trial) reached 50% of the peak amplitude (at −0.26 μV). In Experiments 1 and 2, N2pcs were computed for each of the eight successive probes on each trial (Probes 1–7 and Probe S), separately for target-color and distractor-color probes (Experiment 1), or for probes that appeared concurrently with or followed a target-present or a target-absent search display (Experiment 2). In Experiment 3, Probe N2pc components were computed separately for each of the three constant (short, medium, long) and for variable SOA blocks. Because the different SOAs were presented in an intermixed fashion in variable SOA blocks, and participants could not know whether the next search display would appear after a short (four intermediate probes), medium (six intermediate probes), or long interval (eight intermediate probes), ERPs triggered in response to Probes 1–4 were pooled across all three SOAs, and ERPs in response to Probes 5 and 6 were pooled across trials with medium and long SOAs. In Experiment 4, Probe N2pc components were computed separately for Probes 2, 5, and 7. In addition, RTs to targets at the same location that was previously occupied by a target-color singleton in these probe displays were compared with RTs on trials where targets appeared at a different location than the preceding target-color singleton probe. In all four experiments, N2pc components were also computed for target objects in the search displays, within the same 190–270 ms poststimulus time window used for the Probe N2pc analyses.
The presence of reliable N2pc components in response to singleton probe items at different temporal positions was assessed with repeated-measures ANOVAs of ERP mean amplitudes with the factors laterality (electrode contralateral or ipsilateral to the side of the probe) and probe number (temporal position of the probe relative to the preceding search display). In Experiment 1, the additional factor probe color (target-color or distractor-color probe) was included. Experiment 2 included the additional factor probe context (probes following target-present or target-absent search displays). In Experiment 3, analyses included the additional factor SOA condition (for details, see Results of Experiment 3). Whenever these analyses revealed a significant interaction between laterality and probe number, planned follow-up comparisons (paired t tests) were conducted on contralateral and ipsilateral ERPs for individual probes. Additional Bonferroni-corrected t tests were conducted to compare N2pc components for probes at different temporal positions. Effect sizes are reported in terms of Cohen's d (Cohen, 1988), with a CI of 95%, for t tests, and partial η squared (ηp2) for F tests. All statistical analyses were performed with SPSS software (version 22, IBM; RRID:SCR_002865; www-01.ibm.com/software/uk/analytics/spss).
Results
Experiment 1
Behavioral results
Mean correct RTs and error rates in response to search targets in displays where they were accompanied by a target-color or a distractor-color probe (523 vs 522 ms), and error rates for these two types of displays (3.5 vs 3.4%) did not differ (both t(14) ≤ 0.7, p ≥ 0.482, d ≤ 0.05).
N2pc components
To illustrate the temporal pattern of ERPs to target and probe displays in this RSVP task, Figure 2 shows grand-averaged ERPs across all trials for two successive search displays and the seven probe displays that appeared in the interval between them. These waveforms show ERPs at electrodes PO7/8 contralateral and ipsilateral to the target-color bar in the second search display. Search displays triggered large visual P1 and N1 components. Much smaller P1/N1 components are also visible for each of the probe displays that were presented during the interval between successive search displays. Clear N2pc components were triggered contralateral to target objects in the search displays (comparison of contralateral vs ipsilateral mean ERP amplitudes in the 190–270 ms interval after search display onset; t(14) = 10.6, p < 0.001, d = 0.49).
Because the location of targets in a particular search display was independent of the locations of the preceding color probes or target location in the previous search displays, no N2pc components to these items are visible in Figure 2. N2pcs to target-color and distractor-color probes were extracted by computing ERPs at posterior sites PO7/8 contralateral and ipsilateral to the side of a probe, separately for each of the eight successive temporal probe positions. The resulting ERP waveforms for target-color probes are shown in Figure 3A for the 350 ms interval after the onset of a particular probe display, relative to a 100 ms prestimulus baseline. To visualize the time course of Probe N2pc components across the seven successive probes between two search displays directly, Figure 3B shows continuous N2pc difference waveforms (obtained by subtracting ipsilateral from contralateral ERPs), separately for target-color and distractor-color probes. To present these difference waves in a temporally continuous fashion, Figure 3B shows 200 ms poststimulus time segments for each probe, which correspond to the 200 ms SOA between successive probes. For Probe 1, difference waves are shown from 100 ms before to 350 ms after probe onset. For all other probes (Probe 2 to Probe S), Figure 3B shows the time interval from 150 to 350 ms after probe onset, interpolated between adjacent data points from intervals obtained for two successive probes (resulting in some positive-going offsets that can be seen in Fig. 3B between adjacent probe intervals at a latency of 150 ms relative to probe onset). The onset of each probe is marked with vertical lines, and the N2pc time windows for each probe (190–270 ms after stimulus) are indicated with gray bars. Because probes were presented every 200 ms, the onset of each individual probe n in Figure 3B occurs within the N2pc time interval for the preceding probe n − 1.
Figure 3 shows that target-color probes that appeared simultaneously with or immediately after a search display (Probes S, 1, and 2) did not elicit any N2pc components. N2pcs emerged from Probe 3 onwards and were largest in size for Probe 7, immediately before the onset of the next search display. In contrast, no N2pcs appear to be present for any of the distractor-color probes. A similar temporal pattern of Probe N2pc components was also found at other lateral posterior electrode pairs (e.g., PO9/10). An omnibus repeated-measures ANOVA on N2pc mean amplitudes to color probes with the factors probe color (target-color, distractor-color), probe number (Probes 1–7, Probe S), and laterality (electrode ipsilateral, contralateral to the side of the probe) obtained a main effect of laterality (F(1,14) = 9.0, p = 0.009, ηp2 = 0.39). Importantly, laterality interacted with probe color (F(1,14) = 18.1, p = 0.001, ηp2 = 0.56) as well as with probe number (F(7,98) = 7.9, p < 0.001, ηp2 = 0.36), indicating that N2pc components were larger for target-color than for distractor-color probes, and larger for later compared with earlier probes. There was also a three-way interaction (F(7,98) = 2.4, p = 0.029, ηp2 = 0.14), reflecting differences in the time course of N2pc components between target-color and distractor-color probes. To assess this, ANOVAs were conducted separately for both types of probes. For distractor-color probes, there was no effect of laterality (F(1,14) = 0.04, p = 0.854, ηp2 < 0.01) and no interaction between laterality and probe number (F(7,98) = 1.6, p = 0.131, ηp2 = 0.11). This confirms that probes that did not match the target color template did not trigger N2pc components and thus did not capture attention at any point in time.
For target-color probes, a main effect of laterality (F(1,14) = 16.6, p = 0.001, ηp2 = 0.54) was accompanied by an interaction between laterality and probe number (F(7,98) = 8.2, p < 0.001, ηp2 = 0.37). Follow-up t tests showed that the first two probes (Probes 1 and 2), as well as Probe S, which was presented together with the search display, did not trigger reliable N2pcs (all t(14) ≤ 1.5, p ≥ 0.154, d ≤ 0.05). All other probes (Probes 3–7) elicited significant N2pc components (all t(14) ≥ 2.4, p ≤ 0.030, d ≥ 0.14), suggesting that the target color template was active from ∼1000 ms before the start of the next search episode. As shown in Figure 3B, N2pc components to target-color probes were largest for Probe 7, just before the fully predictable onset of a search display. Probe 7 elicited a significantly larger N2pc than Probe 6 (t(14) = 3.8, p = 0.004, d = 0.92), whereas there were no reliable differences between N2pc amplitudes between Probes 3 and 4, 4 and 5, and 5 and 6 (all t(14) ≤ 1.2, p ≥ 0.472, d ≤ 0.35). This suggests that template activation states were not constant but changed in line with temporal expectations. To rule out the possibility that the N2pcs elicited to Probes 3–7 were contaminated by eye movements toward these probes, HEOG waveforms obtained in the 350 ms interval after the onset of a particular target-color probe display (averaged across Probes 3–7) were computed for EOG electrodes contralateral and ipsilateral to the side of a target-color singleton probe. These are shown in Figure 3C, together with the corresponding contralateral/ipsilateral difference waveforms. Any tendency to move the eyes toward the side of a target-color probe would be reflected by an enhanced contralateral negativity in these HEOG waveforms, but no such effect was found.
The absence of an N2pc for Probe S is presumably due to competitive interactions with the search target object, which itself triggered substantial N2pcs (Fig. 2). The absence of N2pcs to Probes 1 and 2 that immediately followed the preceding search displays suggests that template activation was not sustained, and that templates were briefly deactivated after each search episode. However, it could also reflect a phenomenon similar to the attentional blink (e.g., Raymond et al., 1992), that is, an impaired sensitivity of selective attention to task-relevant information after a target has been encountered. As can be seen in Figure 2, Probes 1 and 2 coincided with the target P3 component, which is linked to target-related identification and response selection processes (e.g., Polich, 2007). The inability of these probes to capture attention may thus not be due to a transient template deactivation, but to a sustained competition with the preceding search target. If this were the case, reliable N2pc components should be triggered by all probe stimuli that follow search displays that do not contain a target. This was tested in Experiment 2, where target-absent search displays containing six nontarget-color objects were presented on half of all trials, and Probe N2pcs were computed separately for probes that followed target-present and target-absent displays.
Experiment 2
Behavioral results
Mean correct RT on target-present trials was 561 ms, and error rate on these trials was 4.6%. False alarms were present on 3.7% of all target-absent trials. The inclusion of target-absent trials in Experiment 2 resulted in slower RTs relative to Experiment 1 (t(14) = 4.7, p < 0.001, d = 0.76), but error rates did not differ between these two experiments (t(14) = 1.5, p = 0.148, d = 0.46).
N2pc components
Figure 4A shows ERPs at posterior sites PO7/8 contralateral and ipsilateral to the side of the target in target-present search displays. As expected, a clear N2pc was triggered by targets, and a comparison of contralateral versus ipsilateral mean amplitudes obtained in the 190–270 ms time window confirmed that this target N2pc was reliably present (t(14) = 8.2, p < 0.001, d = 0.54). Figure 4B shows N2pc difference waveforms illustrating the time course of Probe N2pc components when these probes appeared concurrently (Probe S) or after (Probes 1–7) a target-present search display (top row) or a target-absent search display (bottom row). These difference waves were generated and plotted in the same way as the N2pc difference waves in Figure 3B for Experiment 1. For probes following target-present search displays, the temporal pattern of N2pc components confirmed the results of Experiment 1 (except that reliable Probe N2pcs only emerged from Probe 4 onwards). A different pattern was found for probes that followed target-absent search displays. N2pc components were triggered by probes that appeared simultaneously with or immediately after a target-absent display (Probes S and 1), then disappeared for intermediate probes (Probes 2–4), and reemerged for probes that preceded the subsequent search display (Probes 5–7).
An ANOVA of mean N2pc amplitudes measured in the 190–270 ms postprobe time window with the factors probe context (concurrent with/following a target-present vs target-absent search display), probe number (Probe S, Probes 1–7), and laterality obtained a main effect of laterality (F(1,14) = 13.9, p = 0.002, ηp2 = 0.50), and an interaction between laterality and probe number (F(7,98) = 10.8, p < 0.001, ηp2 = 0.44), showing that Probe N2pcs differed across time. There was no interaction between laterality and probe context (F(1,14) = 2.8, p = 0.117, ηp2 = 0.17), but a three-way interaction between laterality, probe number, and probe context (F(7,98) = 3.1, p = 0.005, ηp2 = 0.18), indicating that the time course of Probe N2pcs was different after target-present and target-absent search displays. To assess this directly, we ran separate ANOVAs with the factors probe context and laterality for Probe N2pcs at each of the eight temporal positions. Interactions between laterality and probe context were present for Probe S (F(1,14) = 8.6, p = 0.011, ηp2 = 0.38) and Probe 1 (F(1,14) = 6.5, p = 0.024, ηp2 = 0.32). As in Experiment 1, these two probes failed to elicit an N2pc with and after target-present search (both t(14) ≤ 0.7, p ≥ 0.509, d ≤ 0.03) but triggered reliable N2pcs in the context of target-absent displays (Probe S: t(14) = 3.1, p = 0.008, d = 0.09; Probe 1: t(14) = 2.5, p = 0.024, d = 0.08). In other words, probes captured attention when they appeared simultaneously with or immediately after a search display that included only distractors, but not when this display contained a competing task-relevant target object. Critically, the N2pc results for Probes 2–4 confirmed that target templates are temporarily deactivated between two successive search episodes, even when the previous search display does not contain a target. Probes 2 and 3 did not trigger N2pcs in either context. There were no effects of laterality (both F(1,14) ≤ 2.5, p ≥ 0.137, ηp2 ≤ 0.15) or interactions with probe context (both F(1,14) ≤ 0.4, p ≥ 0.551, ηp2 ≤ 0.03). Probe 4 elicited reliable N2pcs only after target-present (t(14) = 3.5, p = 0.003, d = 0.14), but not after target-absent search displays (t(14) = 1.6, p = 0.130, d = 0.13), as confirmed by a significant laterality × probe context interaction (F(1,14) = 5.7, p = 0.031, ηp2 = 0.29). In the final 600 ms before the onset of the next search display, evidence for reactivated search templates was found following both target-absent and target-present displays. For Probes 5 and 6, there were significant main effects of laterality (both F(1,14) ≥ 17.5, p ≤ 0.001, ηp2 ≥ 0.56) but no interactions with probe context (both F(1,14) ≤ 0.2, p ≥ 0.660, ηp2 < 0.01), demonstrating that these probes elicited reliable and equivalent N2pcs in both contexts. Probe 7 also elicited reliable N2pcs in both contexts (both t(14) ≥ 3.7, p ≤ 0.002, d ≥ 0.41), which were larger following target-absent displays (interaction between laterality and probe context) (F(1,14) = 10.7, p = 0.006, ηp2 = 0.43).
Experiment 3
Behavioral results
RTs were faster and errors more frequent for shorter versus longer constant SOAs (short: 533 ms, 5.5%; medium: 574 ms, 3.5%; long: 596 ms, 3.2%), with intermediate RTs and error rates in the variable SOA condition (563 ms, 4.6%). These differences were reflected by main effects of SOA condition for both RTs and error rates (both F(3,42) ≥ 6.5, p ≤ 0.001, ηp2 ≥ 0.32).
N2pc components
Figure 5A shows target N2pc components (averaged across all four SOA conditions). Reliable N2pcs were triggered by target objects in search displays in all SOA conditions. A repeated-measures ANOVA on N2pc mean amplitudes revealed a main effect of laterality (F(1,14) = 87.5, p < 0.001, ηp2 = 0.86), but no interaction between laterality and SOA condition (F(3,42) = 2.2, p = 0.101, ηp2 = 0.14), indicating that target N2pc amplitudes did not differ between blocks with different SOAs. Figure 5B shows contralateral-ipsilateral N2pc difference waveforms illustrating the time course of Probe N2pc components for the variable SOA condition. Figure 6 shows the same Probe N2pc difference waves for the three constant SOA conditions. These continuous difference waves illustrate the time course of Probe N2pcs during the interval between two search displays in the same way as Figures 3B and 4B for Experiments 1 and 2, respectively.
The Probe N2pcs found when two search displays were separated by a constant SOA of 1800 ms (medium SOA blocks) mirrored the results of Experiments 1 and 2, with Probe N2pc components emerging from Probe 3 onwards, and increasing in amplitude toward the expected arrival of the next search display. Reducing the SOA between two successive search displays to 1000 ms did not result in an earlier emergence of Probe N2pcs relative to medium SOA blocks. The first two probes that immediately followed the preceding search display still failed to trigger N2pc components. To confirm this statistically, an ANOVA compared N2pcs to Probes 1–4 in the short and medium constant SOA conditions. There was no main effect of laterality (F(1,14) = 3.9, p = 0.068, ηp2 = 0.22), but an interaction between probe number and laterality (F(3,42) = 7.7, p < 0.001, ηp2 = 0.36), reflecting the absence of N2pcs for the earliest probes and their presence for subsequent probes. Critically, there was no significant interaction between SOA condition and laterality (F(1,14) = 2.7, p = 0.123, ηp2 = 0.16), and no three-way interaction between SOA, probe number, and laterality (F(3,42) = 2.0, p = 0.133, ηp2 = 0.12), suggesting that this temporal pattern of Probe N2pc components did not differ between blocks with short and medium constant SOAs. Follow-up t tests (pooled across these two SOA conditions) confirmed that reliable N2pcs were triggered by Probes 3 and 4 (both t(14) ≥ 2.9, p ≤ 0.011, d ≥ 0.07) but not by Probes 1 and 2 (both t(14) ≤ 1.3, p ≥ 0.218, d ≤ 0.03).
In contrast, increasing the constant SOA to 2600 ms had a strong impact on the time course of search template activation processes, reflected by a marked delay of Probe N2pcs, which only emerged from Probe 9 onwards in long SOA blocks. This delay was statistically assessed in an ANOVA that compared N2pcs with Probes 5–8 in medium and long SOA blocks. There was a main effect of laterality (F(1,14) = 16.6, p = 0.001, ηp2 = 0.54) and, critically, an interaction between SOA condition and laterality (F(1,14) = 18.8, p = 0.001, ηp2 = 0.57). In medium SOA blocks, Probes 5–8 all triggered reliable N2pc components (all t(14) ≥ 2.2, p ≤ 0.045, d ≥ 0.14). In long SOA blocks, N2pcs to these probes were entirely absent (all t(14) ≤ 1.5, p ≥ 0.155, d ≤ 0.19). Here, only the four probes that preceded the next search display (Probes 9–12) triggered significant N2pcs (all t(14) ≥ 4.6, p < 0.001, d ≥ 0.35). This shows that search templates were activated much later in long compared with medium SOA blocks, in line with the objective increase of the predictable interval between two search displays.
The temporal pattern of Probe N2pcs in the variable SOA blocks was equivalent to the pattern observed with a constant medium SOA (1800 ms). This was confirmed in an ANOVA that compared N2pcs with Probes 1–8 between these two types of blocks. A main effect of laterality (F(1,14) = 17.4, p = 0.001, ηp2 = 0.55) was accompanied by an interaction between probe number and laterality (F(7,98) = 13.6, p < 0.001, ηp2 = 0.49), reflecting larger N2pcs for later probes and the absence of N2pcs for early probes. Critically, there was no interaction between SOA condition, probe number, and laterality (F(7,98) = 1.1, p = 0.400, ηp2 = 0.07), indicating that the time course of template activation did not differ between blocks with constant and variable SOAs. The same result was found in a second ANOVA that compared N2pcs with Probes 1–4 between blocks with short constant SOAs (where the next search display always appeared after Probe 4) and blocks with variable SOAs (where search displays followed Probe 4 only on one-third of all trials). A main effect of laterality (F(1,14) = 6.8, p = 0.021, ηp2 = 0.33) was accompanied by a probe number × laterality interaction (F(3,42) = 15.5, p < 0.001, ηp2 = 0.53), reflecting the fact that N2pcs were triggered by Probes 3 and 4, but not by Probes 1 and 2. Again, there was no three-way interaction between SOA condition, probe number, and laterality (F(3,42) = 1.4, p = 0.257, ηp2 = 0.09), indicating that template activated processes were unaffected by whether or not the onset of the next search display was fully predictable. Probes that appeared together with search displays in variable SOA blocks elicited a very small N2pc (mean amplitude of Probe S: −0.16 μV; Fig. 5B) that was however significant (t(14) = 2.8, p = 0.013, d = 0.06). The absence of a reliable N2pc to Probe S in all other task conditions of the present study and the presence of such an N2pc in variable SOA blocks may reflect an enhanced ability of these probes to attract attention in blocks where they coincide with a search display which could not be precisely anticipated.
Experiment 4
Behavioral results
For RTs, the presence of spatial cueing effects indicative of attentional capture was assessed in an ANOVA with the factors target location (same vs different as the location previously occupied by a target-color probe) and probe number (Probe 2, Probe 5, Probe 7). There was a main effect of target location (F(1,8) = 22.7, p = 0.001, ηp2 = 0.74), reflecting faster RTs for targets at the same location as the preceding color singleton probe. Importantly, this effect interacted with probe number (F(2,16) = 15.6, p < 0.001, ηp2 = 0.66). A reliable spatial cueing effect was present on trials where the color probe display appeared immediately before the search display (Probe 7), with faster RTs to targets at the same location as the color probe (558 ms) relative to targets at a different location (577 ms) (t(8) = 8.1, p < 0.001, d = 0.30). No such spatial cueing effects were found for trials where the color singleton appeared in probe display 2 (567 ms vs 571 ms) (t(8) = 1.4, p = 0.187, d = 0.06) or in probe display 5 (572 ms vs 570 ms) (t(8) = 1.0, p = 0.368, d = 0.03). There was no main effect of probe number on RTs (F(2,16) = 0.6, p = 0.561, ηp2 = 0.07). Error rate was 3.3% in Experiment 4. There was no effect of target location (F(1,8) = 3.7, p = 0.090, ηp2 = 0.32) and no interaction between target location and probe number, or main effect of probe number for error rates (both F(2,16) ≤ 0.1, p ≥ 0.942, ηp2 ≤ 0.01).
N2pc components
Figure 7 shows ERPs at posterior sites PO7/8 contralateral and ipsilateral to the side of a target-color singleton in probe displays 2, 5, and 7, together with ERPs triggered contralateral and ipsilateral to the target in the search displays (Fig. 7, right). The Probe N2pc results were very similar to those observed in the previous experiments, with a clear N2pc for Probe 7, a smaller N2pc for Probe 5, and little lateralized activity in the N2pc time range for Probe 2. This was confirmed in an ANOVA of mean N2pc amplitudes measured in the 190–270 ms postprobe time window with the factors probe number (Probe 2, Probe 5, Probe 7) and laterality. There was a main effect of laterality (F(1,8) = 15.7, p = 0.004, ηp2 = 0.66) and, critically, an interaction between laterality and probe number (F(2,16) = 14.9, p < 0.001, ηp2 = 0.65). Follow-up t tests revealed a significant N2pc for probes presented immediately before a search display (Probe 7; t(8) = 4.5, p = 0.002, d = 0.58), which also triggered behavioral spatial cueing effects indicative of attentional capture. No significant N2pc was present for Probe 2 (t(8) = 1.6, p = 0.154, d = 0.04). For Probe 5, a small but reliable N2pc was observed (t(8) = 4.0, p = 0.004, d = 0.39), confirming similar observations from Experiments 1–3 and suggesting that these probes were able to capture attention. The absence of spatial cueing effects for these probes suggests that this attentional capture was relatively short-lived, and thus no longer affected RTs when search display targets appeared 600 ms later, and after the presentation of two additional uniformly gray probe displays at positions 6 and 7 (for evidence that the recovery time from contingent attentional capture is typically 150–300 ms, see also Fukuda and Vogel, 2011). As expected, target objects in search displays elicited reliable N2pc components (t(8) = 7.1, p < 0.001, d = 0.55).
Discussion
We used a new rapid serial probe presentation procedure and recorded electrophysiological markers of attentional capture to investigate the time course of target template activation processes in visual search. Participants searched for color-defined target objects and ignored color singleton probe displays that were flashed every 200 ms. To determine when a color-specific target template was active, N2pc components to template-matching probes were measured at successive time points between two search displays. Experiment 1 showed that probes matching the current target color template triggered N2pcs, whereas nonmatching probes did not, demonstrating that the ability of these probes to capture attention was mediated by the current target template (see also Folk et al., 1992; Eimer and Kiss, 2008). Target-color probes presented well ahead of the next search episode elicited N2pc components, demonstrating that attentional templates were activated in a preparatory fashion, and were not triggered purely reactively by the arrival of a search display. The observation that N2pcs were typically triggered from Probe 3 onwards shows that search templates are activated as early as 1000 ms prior before the next search display is expected to appear.
Results also demonstrate that template activation states are not maintained in a sustained fashion between and across successive search episodes. In all experiments, N2pcs were absent for the earliest probes, increased in amplitude toward the end of the interval between two search displays, and were maximal for the probe that appeared immediately before the next search display. Experiment 4 demonstrated that the presence of large N2pcs for late probes and the absence of N2pcs for early probes were associated with the presence versus absence of behavioral spatial cueing effects indicative of attentional capture. These observations suggest that the target template activation states vary across time, with higher activation levels when the next search episode is imminent. Alternatively, the increase of Probe N2pc amplitudes at the end of the preparation period could reflect temporal variability in the start of template activation processes between trials. In this case, the target template would have been active on only a subset of all trials during the period when Probes 3–6 were presented, but on virtually all trials when the probe that preceded a search display appeared. Because Probe N2pcs were computed by averaging across many individual trials, this would result in larger N2pcs for later probes. Both interpretations imply that the activation of search templates is not constant but changes rapidly when observers prepare for the next search episode. The absence of N2pcs to probes that appeared simultaneously with or shortly after a search display is most likely due to competition with concurrent target objects preventing these probes from attracting attention. This was shown in Experiment 2, where reliable N2pcs were triggered by probes that were presented together with or immediately after a target-absent search display, suggesting that search templates remain active for a brief period after a search display has been encountered. However, the absence of N2pcs for probes that followed target-absent displays at intermediate latencies in this experiment shows that search templates were still temporarily deactivated, even when the preceding search episode did not include a target object.
The clearest evidence for rapid changes in the activation states of search templates, and for the ability of observers to control these changes strategically in line with temporal expectations, was provided in Experiment 3. Here, Probe N2pcs emerged much later during the interval between two search displays when this interval was known to be long (2600 ms) than when it was shorter (1800 ms). With long intervals, probes that appeared during the initial 1600 ms interval after the previous search display did not elicit any N2pcs, demonstrating that target templates were inactive in this period. The absence of an active search template during this period cannot be due to ongoing target processing from the previous search episode, and thus is likely to reflect a strategic and presumably voluntary postponement of preparatory template activation processes during time periods where this template is not needed. The absence of N2pcs to some probes in Experiments 1 and 2 could in principle have been due to the fact that probes were presented at task-irrelevant locations (i.e., outside the focus of spatial attention), resulting in a general attenuation of N2pc amplitudes, making it more difficult to detect the effects of sustained template activation for each probed time point. However, this explanation is very unlikely to account for the observation that Probe N2pcs were consistently absent for the first eight successive probes in the long SOA condition of Experiment 3. In the variable SOA blocks of Experiment 3, the temporal pattern of Probe N2pcs did not differ from a condition where the next search display was known to appear after a short interval. This suggests that, when intervals between two search episodes were not predictable, participants always prepared for the earliest possible arrival time of the next target.
The transient nature of template activation states revealed by the current results is not entirely consistent with fMRI-based decoding studies (e.g., Lewis-Peacock et al., 2015), which suggest that working memory representations in matching-to-sample tasks are typically sustained for extended periods. However, the representations activated in such tasks may differ from attentional templates that control target selection in visual search tasks. Also, and in contrast to EEG-based measures, the poor temporal resolution of fMRI signals limits their ability to track template activation processes in a temporally precise fashion. Evidence for transient template activation processes was provided in a recent eye tracking study by Olmos-Solis et al. (2017). These authors found that microsaccades elicited during the preparation for a search task with color-defined targets were biased toward task-irrelevant distractor objects that matched the target color, and that these biases were sensitive to observers' expectations about when the next search display would appear.
It has previously been suggested that in tasks where observers search for the same target object for extended periods, target templates are only held in working memory for a brief time, and are then transferred to a different longer-term memory store where they are retained in a stable fashion without the involvement of active attentional maintenance processes (Woodman et al., 2007; Carlisle et al., 2011). Such a transfer of search templates to long-term memory could also have occurred in the present experiments, where participants always searched for the same color-defined target object throughout the entire experimental session. However, the time course of template activation observed here suggests otherwise. The transient pattern of template activation and deactivation processes before and after individual search episodes, and the sensitivity of these processes to temporal expectations indicate that target templates were controlled by top-down attentional mechanisms that regulated the activation states of these templates on a short-term basis, and in line with current task requirements. Because such characteristics are generally associated with maintaining and processing information in working memory, it appears implausible to assume that target templates were held in a different more permanent long-term memory store.
This raises the general question why observers would choose to switch search templates on and off across successive search episodes in a task where the same color remained task-relevant throughout. Under these circumstances, automatizing color-based target selection by maintaining a stable color template in long-term memory would presumably be less demanding than exerting continuous top-down control over template activation processes in working memory. To resolve this puzzle, it may be important to take account of the fact that target templates play two roles in the control of visual search. During an early stage of visual processing, these templates guide attention toward objects with template-matching features, thus increasing the probability that these objects will be selected for further analysis. During a later object identification stage, selected objects are compared with the currently active target template, to determine their target or nontarget status (Cunningham and Wolfe, 2014). It is possible that the termination of each memory matching process results in the deactivation of the target template involved in this process, which would render it unavailable for subsequent attentional guidance processes (for additional electrophysiological evidence for rapid template deactivation, see Grubert et al., 2017). In this case, this template would have to be reactivated by top-down control mechanisms during the preparation for the next search episode. The fact that memory matching processes are typically terminated later when no target is present (e.g., Chun and Wolfe, 1996) could explain the observation of Experiment 2 that template deactivation and reactivation processes were both delayed following target-absent compared with target-present search displays.
In conclusion, the current study has provided new insights into the time course of template activation processes in visual search. Search templates are not triggered reactively by the arrival of a search display, but are activated proactively, and can already be online from ∼1000 ms before the next search episode. Template activation states are not maintained in a sustained fashion, but change rapidly across time, and these changes are regulated by temporal expectations about when a template will become relevant.
Footnotes
- Received February 13, 2018.
- Revision received July 31, 2018.
- Accepted August 28, 2018.
This work was supported by Economic and Social Research Council (United Kingdom) Grant ES/L016400/1.
The authors declare no competing financial interests.
- Correspondence should be addressed to Dr. Martin Eimer, Department of Psychological Sciences, Birkbeck, University of London, Malet Street, London WC1E 7HX, United Kingdom. m.eimer{at}bbk.ac.uk
- Copyright © 2018 the authors 0270-6474/18/389527-12$15.00/0