This article addresses the representation of numerical information conveyed by nonsymbolic and symbolic stimuli. In a first simulation study, we show how number-selective neurons develop when an initially uncommitted neural network is given nonsymbolic stimuli as input (e.g., collections of dots) under unsupervised learning. The resultant network is able to account for the distance and size effects, two ubiquitous effects in numerical cognition. Furthermore, the properties of the network units conform in detail to the characteristics of recently discovered number-selective neurons. In a second study, we simulate symbol learning by presenting symbolic and nonsymbolic input simultaneously. The same number-selective neurons learn to represent the numerical meaning of symbols. In doing so, they show properties reminiscent of the originally available number-selective neurons, but at the same time, the representational efficiency of the neurons is increased when presented with symbolic input. This finding presents a concrete proposal on the linkage between higher order numerical cognition and more primitive numerical abilities and generates specific predictions on the neural substrate of number processing.