Processing of shape information in human peripheral visual fields is impeded beyond what can be expected by poor spatial resolution. Visual crowding, the inability to identify objects in clutter, has been shown to be the primary factor limiting shape perception in peripheral vision. Despite the well-documented effects of crowding, its underlying causes remain poorly understood. Given that spatial attention both facilitates learning of image statistics and directs saccadic eye movements, we propose that the acquisition of image statistics in peripheral visual fields is confounded by eye-movement artifacts. Specifically, the image statistics acquired under a peripherally deployed spotlight of attention are systematically biased by saccade-induced image displacements. These erroneously represented image statistics lead to inappropriate contextual interactions in the periphery and cause crowding.