The salience map is a crucial concept for many theories of visual attention. On this map, each object in the scene competes for selection - the more conspicuous the object, the greater its representation, and the more likely it will be chosen. In recent years, the firing patterns of single neurons have been interpreted using this framework. Here, we review evidence showing that the expression of salience is remarkably similar across structures, remarkably different across tasks, and modified in important ways when the salient object is consistent with the goals of the participant. These observations have important ramifications for theories of attention. We conclude that priority--the combined representation of salience and relevance--best describes the firing properties of neurons.