The primary source of top-down attentional control in object perception is the prefrontal cortex. This region is involved in the maintenance of goal-related information as well as in attentional selection and set shifting. Recent approaches have emphasized the role of top-down processes during elementary visual processes as exemplified in bistable vision where perception oscillates automatically between two mutually exclusive states. The prefrontal cortex might influence this process either by maintaining the dominant pattern while protecting it against the competing representation, or by facilitating perceptual switches between the two competing representations. To address this issue, we investigated reported perceptual reversals in patients with circumscribed lesions of the prefrontal cortex and healthy control participants in three experimental conditions: hold (maintaining the dominant view), speed (inducing as many perceptual switches as possible), and neutral (no intervention). Results indicated that although the patients showed normal switching rates in the neutral condition and were able to control perceptual switches in the hold condition as much as control subjects were, they were less able to facilitate reversals specifically in the speed condition. These results suggest that the prefrontal cortex is necessary to bias the selection of visual representations in accord with current goals, but is less essential for maintaining selected information active that is continuously available in the environment. As for attentional selection, the present results suggest that the prefrontal cortex initiates perceptual reversals by withdrawing top-down support from the dominant representation without (or prior to) boosting the suppressed view.