Laboratory studies on a range of animals have identified a bias that seems to violate basic principles of rational behavior: a preference is shown for feeding options that previously provided food when reserves were low, even though another option had been found to give the same reward with less delay. The bias presents a challenge to normative models of decision making (which only take account of expected rewards and the state of the animal at the decision time). To understand the behavior, we take a broad ecological perspective and consider how valuation mechanisms evolve when the best action depends upon the environment being faced. We show that in a changing and uncertain environment, state-dependent valuation can be favored by natural selection: Individuals should allow their hunger to affect learning for future decisions. The valuation mechanism that typically evolves produces the kind of behavior seen in standard laboratory tests. By providing an insight into why learning should be affected by the state of an individual, we provide a basis for understanding psychological principles in terms of an animal's ecology.