Visual object recognition is considered to be largely translation invariant. An earlier study (Foster & Kahn, 1985), however, has indicated that recognition of complex novel stimuli is partially specific to location in the visual field: It is significantly easier to determine the identity of two briefly displayed random patterns if both stimuli are presented at the same, rather than at different, locations. In a series of same/different discrimination tasks, we characterize the processes underlying this "displacement effect": Horizontal and vertical translations are equally effective in reducing performance. Making the task more difficult by increasing pattern similarity leads to even higher positional specificity. The displacement effect disappears after rotation or contrast reversal of the patterns, indicating that positional specificity depends on relatively low levels of processing. Control experiments rule out explanations that are independent of visual pattern memory, such as spatial attention, eye movements, or retinal afterimages. Positional specificity of recognition is found only for same trials. Our results demonstrate that position invariance, a widely acknowledged property of the human visual system, is limited to specific experimental conditions. Normalization models involving mental shifts of an early visual representation or of a window of attention cannot easily account for these findings.