Neural basis of utility estimation

doi:10.1016/S0959-4388(97)80008-6

Current Opinion in Neurobiology

Volume 7, Issue 2, April 1997, Pages 198-208

https://doi.org/10.1016/S0959-4388(97)80008-6 Get rights and content

Abstract

The allocation of behavior among competing activities and goal objects depends on the payoffs they provide. Payoff is evaluated among multiple dimensions, including intensity, rate, delay, and kind. Recent findings suggest that by triggering a stream of action potentials in myelinated, medial forebrain bundle axons, rewarding electrical brain stimulation delivers a meaningful intensity signal to the process that computes payoff.

References (76)

M Cabanac
Pleasure: the common currency
J Theor Biol
(1992)
G Fouriezos et al.
Task difficulty increases thresholds of rewarding brain stimulation
Behav Brain Rev
(1990)
J Rossi et al.
Analysis of the relationships between self-stimulation sniffing and brain-stimulation sniffing
Physiol Behav
(1992)
KD Carr et al.
Chronic food restriction and weight loss produce opioid facilitation of perifornical hypothalamic self-stimulation
Brin Res
(1993)
P Shizgal et al.
On the neural computation of utility
Curr Dir Psychol Sci
(1996)
CR Gallistel et al.
A portrait of the substrate for self-stimulation
Psychol Rev
(1981)
WM Baum et al.
Choice as time allocation
J Exp Anal Behav
(1969)
P Shizgal et al.
Electrical stimulation of the rat diencephalon: differential effects of interrupted stimulation on on- and off-responding
Brain Res
(1977)
JE Mazur et al.
Self-control choice with electrical stimulation of the brain
Behav Processes
(1987)
L Green et al.
Economic substitutability of electrical brain stimulation, food, and water
J Exp Anal Behav
(1991)

H Rachlin et al.

Substitutability in time allocation

Psychol Rev

(1980)

RA Wise

Addictive drugs and brain stimulation reward

Annu Rev Neurosci

(1996)

C Bielajew et al.

Behaviorally derived measures of conduction velocity in the substrate for rewarding medial forebrain bundle stimulation

Brain Res

(1982)

P Shizgal et al.

Behavioral methods for inferring anatomical linkage between rewarding brain stimulation sites

J Comp Physiol Psychol

(1980)

A Arvanitogiannis et al.

Effects of excitotoxic lesions of the basal forebrain on MFB self-stimulation

Physiol Behav

(1996)

A Arvanitogiannis et al.

Increased ipsilateral expression of Fos following lateral hypothalamic self-stimulation

Brain Res

(1996)

C Flores et al.

Fos-like immunoreactivity in forebrain regions following self-stimulation of the lateral hypothalamus and the ventral tegmental area

Behav Brain Res

(1997)

B Murray et al.

Physiological measures of conduction velocity and refractory period for putative reward-relevant MFB axons arising in the rostral MFB

Physiol Behav

(1996)

J Malette et al.

Interhemispheric links in brain stimulation reward

Behav Brain Res

(1995)

JS Yeomans

Electrically evoked behaviors: axons and synapses mapped with collision tests

Behav Brain Res

(1995)

JS Yeomans et al.

Excitability properties of medial forebrain bundle axons of A9 and A10 dopamine cells

Brain Res

(1988)

RA Wise

Action of drugs of abuse on brain reward systems

Pharmacol Biochem Behav

(1980)

CR Gallistel

The role of the dopaminergic projections in MFB self-stimulation

Behav Brain Res

(1986)

O Kofman et al.

Differential effects of atropine, procaine and dopamine in the rat ventral tegmentum on lateral hypothalamic rewarding brain stimulation

Behav Brain Res

(1990)

J Barth

J Bentham

KL Conover et al.

Competition and summation between rewarding effect of sucrose and lateral hypothalamic stimulation in the rat

Behav Neurosci

(1994)

KL Conover et al.

Effects of sodium depletion on competition and summation between rewarding effects of salt and lateral hypothalamic stimulation in the rat

Behav Neurosci

(1994)

KL Conover et al.

Differential effects of postingestive feedback on the reward value of sucrose and lateral hypothalamic stimulation in the rat

Behav Nuerosci

(1994)

B Woodside et al.

Administration of ovarian steroid hormones does not change the reward effectiveness of lateral hypothalamic stimulation in ovariectomized rats

Psychobiology

(1996)

EP Bless et al.

The effects of gonadal steroids on brain stimulation reward in female rats

Behav Brain Res

(1997)

GC Abrahamsen et al.

Curve-shift analysis of self-stimulation in food-restricted rats: relationships between daily meal, plasma corticosterone and reward sensitization

Brain Res

(1995)

JS Yeomans

(1990)

JM Simmons et al.

Saturation of subjective reward magnitude as a function of current and pulse frequency

Behav Neurosci

(1994)

GM Heyman et al.

Reinforcer magnitude (sucrose concentration) and the matching law theory of response strength

J Exp Anal Behav

(1994)

AL Hamilton et al.

Reward, performance, and the response strength method in self-stimulating rats: validation and neuroleptics

Physiol Behav

(1985)

RJ Herrnstein

Relative and absolute strength of response as a function of frequency of reinforcement

J Exp Anal Behav

(1961)

RJ Herrnstein

On the law of effect

J Exp Anal Behav

(1970)

Cited by (144)

The role of norepinephrine in the pathophysiology of schizophrenia
2020, Neuroscience and Biobehavioral Reviews
Several lines of evidence have suggested for decades a role for norepinephrine (NE) in the pathophysiology and treatment of schizophrenia. Recent experimental findings reveal anatomical and physiological properties of the locus coeruleus-norepinephrine (LC-NE) system and its involvement in brain function and cognition. Here, we integrate these two lines of evidence. First, we review the functional and structural properties of the LC-NE system and its impact on functional brain networks, cognition, and stress, with special emphasis on recent experimental and theoretical advances. Subsequently, we present an update about the role of LC-associated functions for the pathophysiology of schizophrenia, focusing on the cognitive and motivational deficits. We propose that schizophrenia phenomenology, in particular cognitive symptoms, may be explained by an abnormal interaction between genetic susceptibility and stress-initiated LC-NE dysfunction. This in turn, leads to imbalance between LC activity modes, dysfunctional regulation of brain network integration and neural gain, and deficits in cognitive functions. Finally, we suggest how recent development of experimental approaches can be used to characterize LC function in schizophrenia.
In search of a definition of reinforcer value: Some successes and failures of the multiplicative hyperbolic model
2019, Behavioural Processes
The concept of ‘value’ has enjoyed a central position in many theoretical accounts of choice behaviour. Several definitions of ‘value’ are contrasted in this paper, and one particular approach is defended, whereby value is defined as a dimensionless intervening variable. This definition is a cornerstone of the multiplicative hyperbolic model of choice (MHM), which was proposed twenty years ago as a modification of Mazur’s (1987) hyperbolic model of delay discounting. This paper reviews some of the merits and shortcomings of MHM, and suggests some ways in which MHM might be extended and improved. A formal link between ‘value’ and the related concept of ‘response strength’ is suggested, and revisions of the model are proposed which may enable it to accommodate several behavioural phenomena not considered in the original formulation. Broadening the scope of MHM comes at the cost of adding to its burden of free parameters, and it is emphasised that addition of any new parameters needs empirical justification. The status of value as a dimensionless intervening variable is upheld; however it is noted that a growing body of empirical evidence for links between neurobiological phenomena and value suggests that interpretation of value as a hypothetical construct may be warranted.
Valence processing differs across stimulus modalities
2018, NeuroImage
Citation Excerpt :
It has been suggested that the medial orbitofrontal cortex (OFC) – the most ventral part of the mPFC – integrates multimodal sensory information and is the most likely area in the brain to mediate valence processing for stimuli originating from different sensory modalities (Kringelbach, 2005). This idea of a common neuronal brain region or network for neuronal processing of valence across stimulus modalities is shared by Montague & Berns' predictor-valuation model (Montague and Berns, 2002) and Shizgal’ utility estimation model (Shizgal, 1997). These models argue that the value of multimodal stimuli is converted into a common scale (or currency) allowing the organism to compare their value or ‘utility’ to guide future behaviors (Grabenhorst et al., 2010; Montague and Berns, 2002).
Although it is often assumed that valence processing in the prefrontal cortex (PFC) is similar for stimuli originating from different sensory modalities, evidence supporting this view is lacking. To address this, we recruited 20 male participants and used a delayed-response fMRI design to test whether perceived pleasantness of flavors and images is similarly processed in the PFC. As predicted, significant correlations were observed between image and flavor pleasantness ratings, and PFC response to these stimuli; however, these responses were spatially different, with flavor pleasantness reflected in more ventrally located PFC regions than image pleasantness. These results indicate that, contrary to the general assumption of a singular circuit representing pleasantness, distinct PFC circuits are recruited depending upon stimulus modality. We argue that the ventral-dorsal distinction may be attributed to a difference in proximal versus distal stimulus representations.
It's all in the type of the task: Dopamine modulates kinematic patterns during competitive vs. cooperative interaction in Parkinson's disease
2016, Neuropsychologia
Citation Excerpt :
As postulated by some, engagement in social interaction and rewards processing require basal ganglia (BG) involvement (Izuma et al., 2008; Spreckelmeyer et al., 2009; Lebreton et al., 2009; Pfeiffer et al., 2014), and dopamine neurons play a central role in the reward circuit (Schultz, 2002; Wise, 2002). Behavioral and pharmacological studies on dopamine pathways have described associations between the mesolimbic and nigrostriatal pathways and reward and motor activity (Koob and Le Moal, 1997; Panksepp, 1998; Phillips et al., 1992; Shizgal, 1997; Wise, 2004). Much of the causal evidence supporting the hypothesis that the dopamine mediates the brain reward system is based on studies examining the pharmacological blockade of dopamine receptors in animals.
Increasing evidence suggests that a dysfunctional dopaminergic system affects the ability to socially interact. Since Parkinson's disease (PD) provides a model for assessing dopaminergic dysfunctions in humans, our study was designed to investigate social interactions in PD patients receiving dopamine replacement therapy (Levodopa=l-Dopa) and in neurologically healthy controls. We focused on the kinematics of one action, reaching to grasp a wooden block, which was performed within the context of two basic modes of social cognition, namely cooperation and competition. During the cooperative tasks, two participants were instructed to reach and grasp their respective objects and to cooperate in forming a specific configuration on the working table. During the competitive tasks, two participants were instructed to compete to place their own object at the bottom of a tower to be built on the working table. PD patients’ ability to modulate motor patterning depending on the intention motivating the action they were about to perform was evaluated in both “on” (with l-Dopa) and “off” (without l-Dopa) states. Study results revealed that both the healthy controls and the ‘on’ PD patients had distinct kinematic patterns for cooperative and competitive actions and that these differed from patterns mirroring similar actions performed by those same participants in non social conditions. The kinematic patterns of the healthy controls and the ‘on’ patients were highly correlated during the cooperative tasks. The ‘off’ PD patients were, instead, unable to differentiate between isolated and social conditions. These results support the hypothesis that dopaminergic neurotransmission is involved in shaping the mechanisms underlying social interactions.
Toward a systems-oriented approach to the role of the extended amygdala in adaptive responding
2016, Neuroscience and Biobehavioral Reviews
Research into the structure and function of the basal forebrain macrostructure called the extended amygdala (EA) has recently seen considerable growth. This paper reviews that work, with the objectives of identifying underlying themes and developing a common goal towards which investigators of EA function might work. The paper begins with a brief review of the structure and the ontological and phylogenetic origins of the EA. It continues with a review of research into the role of the EA in both aversive and appetitive states, noting that these two seemingly disparate avenues of research converge on the concept of reinforcement – either negative or positive – of adaptive responding. These reviews lead to a proposal as to where the EA may fit in the organization of the basal forebrain, and an invitation to investigators to place their findings in a unifying conceptual framework of the EA as a collection of neural ensembles that mediate adaptive responding.
The neural substrates for the rewarding and dopamine-releasing effects of medial forebrain bundle stimulation have partially discrepant frequency responses
2016, Behavioural Brain Research
Citation Excerpt :
For example, dopamine receptor antagonists block or reduce ICSS whereas indirect dopamine agonists, such as amphetamine and cocaine, potentiate ICSS [14,15]. However, behavioral studies also implicate neurons with highly excitable, myelinated axons in the rewarding effect of MFB stimulation [16–20,6], and electrophysiological studies have identified non-dopaminergic neurons, directly driven by MFB stimulation, that have characteristics consistent with those inferred from the behavioral studies [21–23]. Fast-scan cyclic voltammetry (FSCV) confirms that midbrain dopamine neurons are activated trans-synaptically by rewarding MFB stimulation.
Midbrain dopamine neurons have long been implicated in the rewarding effect produced by electrical brain stimulation of the medial forebrain bundle (MFB). These neurons are excited trans-synaptically, but their precise role in intracranial self-stimulation (ICSS) has yet to be determined. This study assessed the hypothesis that midbrain dopamine neurons are in series with the directly stimulated substrate for self-stimulation of the MFB and either perform spatio-temporal integration of synaptic input from directly activated MFB fibers or relay the results of such integration to efferent stages of the reward circuitry. Psychometric current-frequency trade-off functions were derived from ICSS performance, and chemometric trade-off functions were derived from stimulation-induced dopamine transients in the nucleus accumbens (NAc) shell, measured by means of fast-scan cyclic voltammetry. Whereas the psychometric functions decline monotonically over a broad range of pulse frequencies and level off only at high frequencies, the chemometric functions obtained with the same rats and electrodes are either U-shaped or level off at lower pulse frequencies. This discrepancy was observed when the dopamine transients were recorded in either anesthetized or awake subjects. The lack of correspondence between the psychometric and chemometric functions is inconsistent with the hypothesis that dopamine neurons projecting to the NAc shell constitute an entire series stage of the neural circuit subserving self-stimulation of the MFB.

View all citing articles on Scopus

View full text

Neural basis of utility estimation

Abstract

J Theor Biol

Behav Brain Rev

Physiol Behav

Brin Res

Curr Dir Psychol Sci

Psychol Rev

J Exp Anal Behav

Brain Res

Behav Processes

J Exp Anal Behav

Psychol Rev

Annu Rev Neurosci

Brain Res

J Comp Physiol Psychol

Physiol Behav

Brain Res

Behav Brain Res

Physiol Behav

Behav Brain Res

Behav Brain Res

Brain Res

Pharmacol Biochem Behav

Behav Brain Res

Behav Brain Res

Competition and summation between rewarding effect of sucrose and lateral hypothalamic stimulation in the rat

Behav Neurosci

Effects of sodium depletion on competition and summation between rewarding effects of salt and lateral hypothalamic stimulation in the rat

Behav Neurosci

Differential effects of postingestive feedback on the reward value of sucrose and lateral hypothalamic stimulation in the rat

Behav Nuerosci

Administration of ovarian steroid hormones does not change the reward effectiveness of lateral hypothalamic stimulation in ovariectomized rats

Psychobiology

The effects of gonadal steroids on brain stimulation reward in female rats

Behav Brain Res

Curve-shift analysis of self-stimulation in food-restricted rats: relationships between daily meal, plasma corticosterone and reward sensitization

Brain Res

Saturation of subjective reward magnitude as a function of current and pulse frequency

Behav Neurosci

Reinforcer magnitude (sucrose concentration) and the matching law theory of response strength

J Exp Anal Behav

Reward, performance, and the response strength method in self-stimulating rats: validation and neuroleptics

Physiol Behav

Relative and absolute strength of response as a function of frequency of reinforcement

J Exp Anal Behav

On the law of effect

J Exp Anal Behav