We used an equivalent noise (EN) paradigm to examine how the human visual system pools local estimates of direction across space in order to encode global direction. Observers estimated the mean direction (clockwise or counter-clockwise of vertical) of a field of moving band-pass elements whose directions were drawn from a wrapped normal distribution. By measuring discrimination thresholds for mean direction as a function of directional variance, we were able to infer both the precision of observers' representation of each element's direction (i.e., local noise) as well as how many of these estimates they were averaging (i.e., global pooling). We estimated EN for various numbers of moving elements occupying regions of various sizes. We report that both local and global limits on direction integration are determined by the number of elements present in the display (irrespective of their density or the size of region they occupy), and we go on to show how this dependence can be understood in terms of neural noise. Specifically, we use Monte Carlo simulations to show that a maximum-likelihood operator, operating on pooled directional signals from visual cortex corrupted by Poisson noise, accounts for psychophysical data across all conditions tested, as well as motion coherence thresholds (collected under similar experimental conditions). A population vector-averaging scheme (essentially a special case of ML estimation) produces similar predictions but out-performs subjects at high levels of directional variability and fails to predict motion coherence thresholds.