Whether visual imagery and visual short-term memory (STM) share the same neural resources, and the extent to which the early visual cortex (V1/V2) is involved in these processes, has been the subject of much debate. Here, we used transcranial magnetic stimulation (TMS) in two separate experiments to contrast the neural states associated with visual imagery and visual STM in the early visual cortex. In Experiment 1, we investigated V1/V2 activation states at the end of the retention phase in a visual imagery and a visual STM task. V1/V2 TMS facilitated performance in both tasks; the finding that imagery and STM interacted with TMS in the same way suggests that the two processes have similar effects on early visual cortical excitability. In Experiment 2, we investigated V1/V2 activation states at the beginning of the retention phase. V1/V2 TMS impaired performance in the visual STM task, whereas it had no effect on the imagery task. Taken together, our findings show that the late phases of the early visual cortical activation state associated with visual imagery and visual STM are similar; differences between the two processes are apparent in the early phases of the tasks. Our results also suggest that the causal role of the early visual cortex in visual STM includes both the initial translation of the visual input into working memory and the subsequent maintenance of the mental representation. Finally, our findings indicate that visual STM sensory recruitment in working memory might act via excitability modulation of V1/V2 neurons.