Learning is more than memory. It is not simply the building of a look-up table of labelled images, or a phone-directory-like list of motor acts and the corresponding sequences of muscle activation. Central to learning and intelligence is the ability to predict, that is, to generalize to new situations, beyond the memory of specific examples. The key to generalization, in turn, is the architecture of the system, more than the rules of synaptic plasticity. We propose a specific architecture for generalization for both the motor and the visual systems, and argue for a canonical microcircuit underlying visual and motor learning.