Knowledge or experience is voluntarily recalled from memory by reactivation of the neural representations in the cerebral association cortex. In inferior temporal cortex, which serves as the storehouse of visual long-term memory, activation of mnemonic engrams through electric stimulation results in imagery recall in humans, and neurons can be dynamically activated by the necessity for memory recall in monkeys. Neuropsychological studies and previous split-brain experiments predicted that prefrontal cortex exerts executive control upon inferior temporal cortex in memory retrieval; however, no neuronal correlate of this process has ever been detected. Here we show evidence of the top-down signal from prefrontal cortex. In the absence of bottom-up visual inputs, single inferior temporal neurons were activated by the top-down signal, which conveyed information on semantic categorization imposed by visual stimulus-stimulus association. Behavioural performance was severely impaired with loss of the top-down signal. Control experiments confirmed that the signal was transmitted not through a subcortical but through a fronto-temporal cortical pathway. Thus, feedback projections from prefrontal cortex to the posterior association cortex appear to serve the executive control of voluntary recall.