The visual system is thought to process luminance (first-order) and contrast (second-order) information by dedicated cortical streams. To explore the spatial characteristics of the second-order pathway, we examined the effect of adaptation on spatial localization in human subjects. We show that, unlike first-order adaptation, second-order positional adaptation via cortical mechanisms transfers across orientations but not across spatial frequencies. These results support physiological evidence that these two processing streams are distinct and suggest that the cortical mechanism mediating second-order positional adaptation maintains spatial frequency information but sums signals across orientations.