Abstract
Determining the direction of visual motion poses a serious problem for any visual system, given the inherent ambiguities. Geisler (1999)has suggested that motion streaks left in the wake of a moving target provide a rich source of potential information that could aid in resolving direction ambiguities. Here we provide strong experimental evidence that the human visual system does in fact exploit motion streaks in direction discrimination. Masks comprising oriented random noise impeded direction discrimination of moving dots when the masks were oriented parallel to the direction of motion but had very little effect when oriented orthogonal to the direction of motion. The masking effect decreased systematically with increasing bandwidth for the parallel masks and increased with bandwidth for the orthogonal masks. Importantly, these masks had little effect on neither contrast sensitivity for detecting the moving stimuli nor for speed discrimination. Experiments with “Glass patterns” (moirépatterns constructed from random dot pairs) confirmed that misleading pattern information can impede motion detection. The results show that the oriented streaks left by moving stimuli provide fundamental information about the direction of visual motion; removing these streaks or augmenting them with erroneous streaks severely confounds motion direction discrimination.
Cartoonists have long used “speedlines,” very successfully, to indicate motion in a stationary scene. There is no mistaking the path of the brick on Figure1, nor that it moved at high speed. Here we ask whether it is possible that traces left by moving objects not only represent motion in some symbolic sense but also actually determine the axis of visual motion under normal physiological conditions.
Determining the direction of motion can be one of the more difficult tasks for any visual system, given the inherent ambiguities created by motion sensing mechanisms of necessarily limited spatial extent (commonly termed the “aperture problem”). A clear illustration of the problem is the well known “barber-pole illusion” (Wallach, 1935; English translation by Wuerger et al., 1996), in which a striped pole rotating sideways seems to move upward along the direction of the pole (that in effect forms its aperture). Several explanations have been proposed to account for how the visual system attempts (and sometimes fails) to deal with the ambiguities of local motion signals, the commonest being the “intersection of constraints” model (Horn and Schunck, 1981; Adelson and Movshon, 1982) and the “vector-sum” model (Wilson et al., 1992).
Recently, Geisler (1999) has proposed that “motion streaks” (effectively speedlines) aid in determining direction of motion. As the visual system integrates information for ∼100 msec, moving objects should leave behind detectable “streaks,” betraying direction of motion. The streaks could easily be detected by orientationally selective mechanisms and be combined with information from a standard motion unit to compute the velocity vector (Geisler, 1999, his Fig. 1). Geisler has provided a detailed model of how this may occur, with mathematical simulations and psychophysical and electrophysiological support (Geisler, 1999; Geisler et al., 2001), primarily based on the predicted behavior at high speed.
Additional evidence comes from recent demonstrations from Ross et al. (2000), who showed that dynamic sequences of Glass patterns create a powerful impression of motion, ambiguous in direction but oriented in accordance with the structure of the Glass pattern. Glass patterns (Glass, 1969) are created from randomly placed pairs of dots, with pairs oriented coherently to produce a global impression of streakiness: the streaks can either lie parallel or be organized more complexly, such as in radial or circular patterns. Successive presentations of Glass patterns with different random pairs produces motion energy in all directions but no physically coherent motion; however, coherent motion is seen in the direction of the patterns created by the dot pairs. One possibility is that the dot pairs (each falling on the extreme of a short line) simulate motion streaks that would have been left by coherent motion (Burr, 2000) and that these pseudostreaks serve to structure the incoherent motion of the random presentations.
The aim of the present study was to provide the first direct evidence that speedlines can alter the perceived direction of motion. With techniques involving bandpass-limited noise masking and Glass patterns, we show that speedlines selectively influence direction but do not affect other aspects of motion perception, such as contrast sensitivity and speed discrimination.
MATERIALS AND METHODS
All stimuli were generated by Visual Stimulus Generator framestore (Cambridge Research Systems, Rochester, UK) and displayed on the face of a Hitachi (Tokyo, Japan) monitor at 512 × 512 pixels resolution and 140 Hz. The visible screen was a circle 28 cm in diameter, subtending 20° from the viewing distance of 80 cm, with mean luminance of 20 cd/m2. For the first experiments (see Figs. 2, 3), the stimulus comprised 200 randomly positioned dots moving rightward at 33°/sec. Dot diameter was 11.7 arc minutes (′) (five pixels). One hundred were white and 100 were black, with Michelson contrast of 50% (except when contrast thresholds were measured). Dot moved with a “limited lifetime” paradigm, in that they were displaced for one frame to be reborn in a random position on the next frame. The motion sequence of each trial lasted 10 pairs of frames, 143 msec. The monitor was equipped with P22 phosphor, which decayed to <10% (below detection level; see Fig. 3) in <1 msec.
The moving dots were superimposed on noise patterns, displayed on alternate video frames to allow separate control of test and mask contrast and to ensure linear summation. The noise was bandpass filtered in both spatial frequency (Gaussian filter on log frequency, centered at 3 cycles/degree with three octave full-bandwidth) and orientation (Gaussian filter centered at 0 or 90°, with variable bandwidth, indicated in the abscissa of Fig. 2 or the captions of Fig. 3). Twenty fresh noise patterns were constructed before each experimental session. For the static presentation, a new noise pattern was chosen at random each trial; for the dynamic presentation, a new pattern was chosen on each (interspersed) frame, giving it a refresh rate of 70 Hz.
Glass patterns comprised 200 pairs of dots, 100 black pairs and 100 white pairs, all at 95% Michelson contrast. They were 11.7′ in diameter as before and separated by 14′. A certain proportion of the dots were oriented in a coherent way, and the others were oriented at random. For the translational patterns, all coherent dots had the same orientation, chosen at random between −20 and +20° (with 0° referring to horizontal). For the spiral patterns, the coherent pairs were all orientated at a particular angle relative to the radii, so 0° described a radial pattern, 90° described a circular pattern, and 45° described a spiral pattern (with equal radial and circular components). Again, the angle of coherence varied randomly from trial to trial between −20 and +20°. With these patterns, the direction of motion also followed a spiral trajectory defined by the angle relative to the radii (with local speed constant of 33°/sec). In both cases, the motion was a variant of the limited-lifetime technique, in which each pair had a certain probability of being moved coherently (either translation or spiral motion) or dying to be reborn at random. For observers D.B. and A.M.W., the probability of a pair moving was 0.5 (on average, equal to a limited lifetime of 2), whereas for J.R., it was 0.25.
The observers' task was usually to identify the direction of motion of the dots, upward or downward from horizontal for the translating patterns, and whether there was more expansion or rotation for the spiral patterns. The magnitude of the deviation from horizontal or spiral depended on subject responses, following the adaptive QUEST (Watson and Pelli, 1983) algorithm. Thresholds were determined by fitting a cumulative Gaussian to the resultant probability of seeing curves (on logarithm abscissa). For the contrast and speed thresholds, there were two intervals separated by 200 msec and marked by an auditory tone. Subjects had to identify the interval containing the moving dots (compared with a blank interval) for the contrast thresholds or the interval containing the faster motion (compared with the standard of 33°/sec) for the speed discrimination. Again, contrast or speed difference was varied by QUEST, and thresholds were determined by Gaussian fit. In all cases, erroneous responses were indicated by auditory feedback.
RESULTS
We first measured thresholds for discriminating the direction of motion of a field of random dots (upward or downward), with and without superimposed visual noise. The noise was filtered in the orientation domain, either parallel to the average direction of motion (horizontal) or orthogonal to it (vertical). The results are plotted in Figure2 as a function of the orientation bandwidth of the noise. Noise filtered parallel to the direction of motion was more detrimental to direction thresholds than orthogonal noise. The effect was strongest for narrow bandpass noise, in which parallel noise increased direction thresholds by a factor of ∼8. For orthogonal filtered noise, the narrowest bandwidths had little effect on motion direction discrimination (baseline indicated by thedashed line). As the bandwidth of noise increased, the effect of the parallel noise decreased and the effect of orthogonal noise increased. Most of the data (indicated by circles) were collected with static noise. The squares indicate thresholds for dynamic noise (updated at 70 Hz) of one orientation bandwidth (11°). The dynamic noise was slightly more detrimental than static noise, but the difference between parallel and orthogonal remained.
To test whether the effect of the oriented noise was specific to direction discrimination, we measured contrast sensitivity and speed discrimination thresholds under similar conditions. The left panels of Figure 3 show contrast thresholds for detecting the moving dots in the presence of static or dynamic parallel or orthogonal noise. Neither parallel nor orthogonal noise had much effect on thresholds under any conditions. All effects were less than a factor of 1.5, in the order of the variability of threshold measurements. Geisler (1999) found that thresholds for a single moving dot were raised more by parallel than by orthogonal noise, but his motion was more nearly continuous than ours, presumably creating greater responses in V1 neurons (“streak detectors”) tuned in the direction of motion (Geisler et al., 2001). The right panels of Figure 3 show results for speed discrimination thresholds. Again, the effects were small, but orthogonal noise had more detrimental effects on speed discrimination than parallel noise (that produced no measurable differences at all).
The study of Ross et al. (2000) suggested that Glass patterns might mimic motion streaks and impose a coherent pattern on random motion. If this were the case, then Glass patterning in coherently moving dots should act as the source of an extraneous direction signal. We created random dot patterns similar to those used in the previous study, again caused to translate rightward. In this experiment, however, all dots were arranged in pairs: a proportion of the pairs, referred to as “coherent,” were all oriented in the same direction to create a Glass pattern; the remainder were oriented at random. The orientation of the coherent pairs varied randomly on each trial between −20 and +20° (with 0° referring to horizontal); if they gave signals about axis of motion, these signals would be randomly misleading. As before, observers were required to identify whether the motion were upward or downward from true horizontal. The left panel of Figure4 shows direction discrimination thresholds as a function of percentage of coherence of the dipoles. For all three observers, the presence of the randomly oriented coherent dipoles was detrimental to discrimination performance, and the effect depended systematically on coherence, reaching a factor of 10 for total coherence for one observer (J.R.).
The experiment was repeated for spiral motion combined with spiral Glass patterning. The procedure was identical to that described previously, except that dots moved in a clockwise-expanding spiral motion, and observers were required to identify whether the motion seemed more to expand or to rotate. As before, all dots were organized in pairs, with a proportion of them oriented coherently in a spiral direction and the others at random. The orientation of the coherent dots (compared with the radii) was chosen at random from 25 to 65° in the radial-circular space (see Materials and Methods). The results (Fig. 4, right panel) are similar to those for translational motion, except that the effects are slightly stronger. The presence of the coherent dipoles impeded direction discrimination in the spiral motion space, with the effect depending monotonically on the percentage of coherent motion.
DISCUSSION
In these studies, we used discontinuous motion in conjunction with oriented noise or Glass patterns to expose the visual mechanisms that normally compute the direction of continuous motion in the natural world. The results strongly support the suggestion that the traces left by a moving target (motion streaks, or speedlines) may be used by the visual system to help determine the direction of motion. Narrow band noise oriented in the direction of the motion increased discrimination thresholds by a factor of 6–8, whereas similar noise oriented in the orthogonal direction had very little effect on direction thresholds. The parallel noise may have acted to mask or in someway distort the motion streaks created by the moving dots. There are several ways this could occur: the mask may desensitize orientation-selective mechanisms that would normally have responded to the streaks or may act as pseudostreaks, providing false (but un-ignorable) information. Either way, only noise oriented at or near the direction of motion raises direction discrimination thresholds. Furthermore, parallel noise impedes only direction discrimination, having little or no effect on either contrast sensitivity or speed discrimination. This further supports a model whereby speed and direction are encoded at least in part by independent mechanisms.
The results with Glass patterns reinforced the masking results. The presence of coherent dot pairs not only affects the apparent direction of motion, as Ross et al. (2000) have shown, but also impairs discrimination performance considerably, presumably by supplying false motion signals in randomly chosen directions (between −20 and +20° from horizontal). This is consistent with recent observations that cells in the middle temporal area (MT) and the medial superior temporal area respond to these dynamic glass patterns as they do to real motion, with the same directional selectivity (Krekelberg et al., 2001). Observers attempted to ignore these signals, often responding against the apparent direction of motion, trying to double guess the task but with little success (despite auditory error feedback on each trial). This shows that the sensation of motion created by Glass patterns is “obligatory,” activating basic perceptual mechanisms, and not merely creating a “higher-order” cognitive interpretation that could be ignored at will. Because the Glass patterns form the endpoints of line segments, the most parsimonious explanation for their influence on motion directions is that they masquerade as motion streaks.
The individual pairing of Glass patterns is not obvious except on close inspection. Motion streaks are normally not obvious either (Burr, 1980), provided the motion duration is sufficient. It would seem that motion streaks are used somehow in aiding the analysis of motion direction, without appearing to perception as streaks per se. This is supported by demonstrations that, although stimuli moving over long tracks may not appear blurred, the motion blur nevertheless affects performance in blur discrimination (Burr and Morgan, 1997).
The neural mechanisms for the action of motion streaks are far from clear at this stage. Geisler et al. (2001) have shown that many neurons in V1 of cat and monkey show direction selectivity to both motion perpendicular to their preferred orientation (as is typically reported) and parallel to the preferred orientation. Both directions show a similar temporal frequency preference, but the parallel direction is spatially low-pass instead of bandpass. As they argue convincingly, the response to parallel motion could well correspond to motion streaks. Furthermore, Pack and Born (2001) have shown that neurons in MT change their directional selectivity over time. When stimulated with diagonally moving small bars, they initially respond to the motion direction orthogonal to the bar direction, like V1 cells (Movshon et al., 1985). Over a period of ∼120 msec, the preferred direction changes to that of the global motion of the lines. It is possible that the later but more veridical tuning of the cells arises from information from neural units tuned to the stationary streaks left by the line terminations (in the veridical direction of motion). Because these units are tuned to stationary rather than moving stimuli, it seems reasonable that they have a slower temporal response and hence arrive in MT after the basic motion information. Interestingly, human observers also misjudge motion direction of similar stimuli when presented briefly but tend to veridicality for longer durations (Lorenceau et al., 1993). It would be interesting to study the dynamics of the masking effects of simulated streaks to see whether this is also delayed with respect to speed information.
Finally, it is worth remarking that artists have often anticipated the discoveries of visual science in their attempts to represent form, depth, and color in their painting (Werner and Ratliff, 1999) and, more recently, of the effect of size on shape (Pelli, 1999). In using speedlines to indicate motion, cartoonists may also have anticipated scientific discovery.
Footnotes
Correspondence should be addressed to David Burr, Istituto di Neurofisiologia del Consiglio Nazionale delle Ricerche, via Moruzzi 1, Pisa 56100, Italy. E-mail: dave{at}in.pi.cnr.it.