Inhibitory neurons regulate the adaptation of neural circuits to sensory experience, but the molecular mechanisms by which experience controls the connectivity between different types of inhibitory neuron to regulate cortical plasticity are largely unknown. Here we show that exposure of dark-housed mice to light induces a gene program in cortical vasoactive intestinal peptide (VIP)-expressing neurons that is markedly distinct from that induced in excitatory neurons and other subtypes of inhibitory neuron. We identify Igf1 as one of several activity-regulated genes that are specific to VIP neurons, and demonstrate that IGF1 functions cell-autonomously in VIP neurons to increase inhibitory synaptic input onto these neurons. Our findings further suggest that in cortical VIP neurons, experience-dependent gene transcription regulates visual acuity by activating the expression of IGF1, thus promoting the inhibition of disinhibitory neurons and affecting inhibition onto cortical pyramidal neurons.