The developing brain responds to the environment by using statistical correlations in input to guide functional and structural changes—that is, the brain displays neuroplasticity. Experience shapes brain development throughout life, but neuroplasticity is variable from one brain system to another. How does the early loss of a sensory modality affect this complex process? We examined cross-modal neuroplasticity in anatomically defined subregions of Heschl's gyrus, the site of human primary auditory cortex, in congenitally deaf humans by measuring the fMRI signal change in response to spatially coregistered visual, somatosensory, and bimodal stimuli. In the deaf Heschl's gyrus, signal change was greater for somatosensory and bimodal stimuli than that of hearing participants. Visual responses in Heschl's gyrus, larger in deaf than hearing, were smaller than those elicited by somatosensory stimulation. In contrast to Heschl's gyrus, in the superior-temporal cortex visual signal was comparable to somatosensory signal. In addition, deaf adults perceived bimodal stimuli differently; in contrast to hearing adults, they were susceptible to a double-flash visual illusion induced by two touches to the face. Somatosensory and bimodal signal change in rostrolateral Heschl's gyrus predicted the strength of the visual illusion in the deaf adults in line with the interpretation that the illusion is a functional consequence of the altered cross-modal organization observed in deaf auditory cortex. Our results demonstrate that congenital and profound deafness alters how vision and somatosensation are processed in primary auditory cortex.