Contextual modulation of neuronal responses by surrounding environments is a fundamental attribute of sensory processing. In the mammalian retina, responses of On-Off direction selective ganglion cells (DSGCs) are modulated by motion contexts. However, the underlying mechanisms are unknown. Here, we show that posterior-preferring DSGCs (pDSGCs) are sensitive to discontinuities of moving contours owing to contextually modulated cholinergic excitation from starburst amacrine cells (SACs). Using a combination of synapse-specific genetic manipulations, patch clamp electrophysiology and connectomic analysis, we identified distinct circuit motifs upstream of On and Off SACs that are required for the contextual modulation of pDSGC activity for bright and dark contrasts. Furthermore, our results reveal a class of wide-field amacrine cells (WACs) with straight, unbranching dendrites that function as "continuity detectors" of moving contours. Therefore, divergent circuit motifs in the On and Off pathways extend the information encoding of On-Off DSGCs beyond their direction selectivity during complex stimuli.