Spatial frequency-based information plays an important role in visual perception. By combining behavioral and electroencephalogram (EEG) measurements, we investigated the mechanisms of the interaction and information integration between different spatial frequency bands. The observers performed a scene categorization task on hybrid images that were generated by combining the low spatial frequency (LSF) component of one image with the high spatial frequency (HSF) component of another image. The results showed that the recognition of the HSF component was interfered by the non-attended LSF component at semantic level. The strength of the semantic interference was modulated by the physical similarity between the LSF and HSF components. Analyses of the EEG data revealed an early anterior N1 component (122 ms from stimulus onset) that was related to the observed interaction of the semantic and physical information between the LSF and HSF components. These findings demonstrate that the semantic information from different spatial frequency bands can be integrated at early stage of the perceptual processing. This early integration is likely to occur at frontal areas in order to initiate top-down facilitation.