During functional magnetic resonance imaging (fMRI), our participants selectively attended to tone streams at the left or right, and occasionally shifted their attention from one stream to another as guided by a centrally presented visual cue. Duration changes in the to-be-attended stream served as targets. Loudness deviating tones (LDTs) occurred infrequently in both streams to catch attention in a bottom-up manner, as indicated by their effects on reaction times to targets. LDTs activated the right temporo-parietal junction (TPJ), posterior parts of the left inferior/middle frontal gyrus (IFG/MFG), ventromedial parts of the superior parietal lobule (SPL), and left frontal eye field/premotor cortex (FEF/PMC). In addition, LDTs in the to-be-ignored sound stream were associated with enhanced activity in the ventromedial prefrontal cortex (VMPFC) possibly related to evaluation of the distracting event. Top-down controlled cue-guided attention shifts (CASs) activated bilateral areas in the SPL, intraparietal sulcus (IPS), FEF/PMC, TPJ, IFG/MFG, and cingulate/medial frontal gyrus, and crus I/II of the cerebellum. Thus, our results suggest that in audition top-down controlled and bottom-up triggered shifting of attention activate largely overlapping temporo-parietal, superior parietal and frontal areas. As the IPS, superior parts of the SPL, and crus I/II were activated specifically by top-down controlled attention shifts, and the VMPFC was specifically activated by bottom-up triggered attention shifts, our results also suggest some differences between auditory top-down controlled and bottom-up triggered shifting of attention.