Children take years to learn symbolic arithmetic. Nevertheless, non-human animals, human adults with no formal education, and human infants represent approximate number in arrays of objects and sequences of events, and they use these capacities to perform approximate addition and subtraction. Do children harness these abilities when they begin to learn school mathematics? In two experiments in different schools, kindergarten children from diverse backgrounds were tested on their non-symbolic arithmetic abilities during the school year, as well as on their mastery of number words and symbols. Performance of non-symbolic arithmetic predicted children's mathematics achievement at the end of the school year, independent of achievement in reading or general intelligence. Non-symbolic arithmetic performance was also related to children's mastery of number words and symbols, which figured prominently in the assessments of mathematics achievement in both schools. Thus, non-symbolic and symbolic numerical abilities are specifically related, in children of diverse socio-economic backgrounds, near the start of mathematics instruction.