In arbitrary visuomotor mapping, an object instructs a particular action or target of action, but does so in a particular way. In other forms of visuomotor control, the object is either the target of action (termed standard mapping) or its location provides the information needed for targeting (termed transformational mapping). By contrast, in arbitrary mapping, the object's location bears no systematic spatial relationship with the action. Neuropsychological and neurophysiological investigation has, in large part, identified the neural network that underlies the rapid acquisition and performance of arbitrary visuomotor mappings. This network consists of parts of the premotor (PM) and prefrontal (PF) cortex, the hippocampal system (HS), and the basal ganglia (BG). Here, we propose specialized contributions of the network's different components to its overall function. To do so, we invoke the concept of distributed information-processing architectures, or modules, which may involve a variety of neural structures. According to this view, recurrent neural networks involving cortex, basal ganglia, and thalamus operate largely in parallel. Each of these interacting networks can be termed a cortical-BG module. A large number of these modules include PM neurons, and they can be termed PM cortical-BG modules. A comparable number include PF neurons, termed PF cortical-BG modules. We propose that PM and PF cortical-BG modules compute specific object-to-action mappings, in which the network learns the action associated with a given input. These mappings serve as specific solutions to arbitrary visuomotor mapping problems. However, they are also exemplars of more abstract rules, such as the knowledge that nonspatial visual information (e.g., color) can guide the choice of action. We propose that PF cortical-BG modules subserve abstract rules of this kind, along with other problem-solving strategies. This view should not be taken to imply that the PF network lacks the capacity to compute specific mappings, but rather that it has higher-order mapping functions in addition to its lower-order ones. Furthermore, it seems likely that PF provides PM with pertinent sensory information. The hippocampal system appears to play a role parallel to that of both neocortical-BG networks discussed here. However, in accord with several models, it operates mainly in the intermediate term, pending the consolidation of the relevant information in those neocortical-BG networks.