Phasic dopamine signals: from subjective reward value to formal economic utility
Introduction
The function of reward is derived from the biological needs for nutritional and other substances and reproduction. Thus, rewards have specific value for individual survival and gene propagation. Although rewards have physical aspects that are detected by sensory receptors, there are no specific receptors for the typically polysensory rewards, and their value needs to be inferred from eliciting preferences in behavioral choices. Furthermore, reward value depends on the organism's momentary requirements. Satiation induced by a meal reduces the value of foods but may render liquids such as digestive drinks more attractive. Thus, value is subjective and constructed by the brain; it cannot be estimated entirely from the physical parameters and sensory properties of the rewards. The usual way to estimate subjective value in animals involves behavioral measures, including break points in fixed ratio schedules, preferences in binary choices and psychophysical indifference points against a common reference reward (subjective equivalents). Subjective value estimated in these ways is expressed in physical measures of break points, choice frequency or reference reward amount (e.g. ml of juice or numbers of pellets). By contrast, a more general, and theoretically well defined, measure for subjective value is formal economic utility, which constitutes a mathematical characterization of reward preferences and provides an internal metric of subjective value (sometimes called util) [1]. Individuals have the best chance to survive by preferring rewards with the highest subjective value. Economic theory formalizes this idea with axioms defining the conditions for utility maximization [2].
Maximization of subjective value and utility requires decision mechanisms in which inputs from neuronal value signals compete with each other, and only the option with the highest value gets selected. Neuronal reward signals that serve as appropriate inputs to competitive decision mechanisms should process subjective value or, in their best defined form, economic utility, in a monotonic but usually nonlinear relationship to objective value.
This review describes the neuronal coding of subjective value and formal economic utility in one of the brain's prominent reward systems, the dopamine neurons. We review both the electrophysiological responses of midbrain dopamine neurons and the voltametrically assessed dopamine concentration changes in axonal terminal areas in nucleus accumbens. We also address recent issues concerning voltammetric changes reflecting subjective value in rats.
Section snippets
Concepts and behaviour
How can we maximize subjective value when choosing between apples and oranges? These objects contain important substances for bodily functions, like glucose and water, but their precise contents are difficult to quantify. As different rewards often have no common physical unit, one can assign a ‘common currency’ value to one particular object, called ‘numeraire’ in economic theory. Behavioral preferences serve to estimate the subjective values of all other objects relative to this common
Concepts
Rewards are inherently risky. The terms of risk avoidance and risk seeking refer to individual psychological tendencies of hating or loving risk and characterize the influence of risk on subjective reward value. Risk avoiders value risky outcomes lower than safe outcomes with same objective value, whereas risk seekers do the opposite. Risk is distinct from probability, as it increases up to probabilities of p = 0.5 and then declines again. The most simple and best controlled behavioral risk test
Concepts and behaviour
The subjective value of rewards decays with the delay between a stimulus or action and the reward delivery. This temporal discounting may have its biological origin in the physical decay of many nutrient rewards. Temporal discounting applies to rewards in general, even when they remain physically unchanged and do not decay. Intertemporal choices between a variable early and a set late reward serve to psychophysically assess temporal discounting. The subjective value of the late reward is
Concepts and behaviour
Caloric rewards provide energy for body functions. However, reward acquisition often involves effort, which amounts to energy expenditure. The gain from reward is reduced by the loss. This notion can be extended to all rewards; effort is considered an economic cost that should be subtracted from income value (however, subtracting cost from income does not define formal economic utility as viewed by economists [2, 33, 34]). Monkeys show longer reaction times, more task errors and lower task
Concepts
The quantification of subjective reward value relative to a numeraire employs an objective scale. The £20,000 price of car is an objective money amount, even though my preference for that car over another car reveals my subjective value. By contrast, formal economic utility advances by one crucial step in providing a mathematical function of objective value u (x). Knowing such a function allows to determine the subjective value for goods solely based on their objective amounts, without every
Conflict of interest statement
Nothing declared.
References and recommended reading
Papers of particular interest, published within the period of review, have been highlighted as:
• of special interest
•• of outstanding interest
Acknowledgements
Our work has been supported by Wellcome Trust (WS: 058365, 093270, 095495), European Research Council (WS: ERC, 293549), NIH Conte Center at Caltech (WS: P50MH094258) and NIH (RMC: DA034021; RMW: DA010900).
References (56)
- et al.
Adaptive coding of reward value by dopamine neurons
Science
(2005) - et al.
Two types of dopamine neuron distinctively convey positive and negative motivational signals
Nature
(2009) - et al.
Playing for peanuts: why is risk seeking more common for low-stakes gambles?
Organ Behav Hum Decis Process
(2005) - et al.
Microeconomic Theory
(1995) Impulse control in pigeons
J Exp Anal Behav
(1974)- et al.
Determination of discount functions in rats with an adjusting-amount procedure
J Exp Anal Behav
(1997) - et al.
Influence of reward delays on responses of dopamine neurons
J Neurosci
(2008) - et al.
Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning
Science
(2009) - et al.
Noradrenaline and dopamine neurons in the reward/effort trade-off: a direct electrophysiological comparison in behaving monkeys
J Neurosci
(2015) - et al.
Dynamic gain control of dopamine delivery in freely moving animals
J Neurosci
(2004)