Choice behavior in a discrete-trial concurrent VI-VR: A test of maximizing theories of matching

doi:10.1016/0023-9690(85)90025-6

Learning and Motivation

Volume 16, Issue 4, November 1985, Pages 423-443

https://doi.org/10.1016/0023-9690(85)90025-6 Get rights and content

Abstract

Rats were trained on a discrete-trial procedure in which one alternative (VR) was correlated with a constant probability of reinforcement while the other was correlated with a VI schedule which ran during the intertrial intervals and held the scheduled reinforcer until they were obtained by the next VI response. Relative reinforcement rate was varied in series of conditions in which the VR schedule was varied and in series in which the VI was varied. Choice behavior was described well by the generalized matching law, although moderate undermatching occurred for all subjects. Contrary to the predictions of molar maximizing (optimality) theories, there was no consistent bias in favor of the ratio alternative, and the sensitivity to reinforcement allocation was not systematically affected by whether the ratio or interval schedule was varied. The results were also contrary to momentary maximizing accounts, as there was no correspondence between the probability of a changeover to the VI behavior and the time since the last response to the VI alternative. Neither variety of maximizing theory appears to provide a general explanation of matching in concurrent schedules.

References (26)

R.J. Herrnstein et al.
Melioration and behavioral allocation
W.H. Baum
On two types of deviation from the matching law: Bias and undermatching
Journal of the Experimental Analysis of Behavior
(1974)
W.H. Baum
Time-based and count-based measurement of preference
Journal of the Experimental Analysis of Behavior
(1976)
W.H. Baum
Matching, undermatching, and overmatching in studies of choice
Journal of the Experimental Analysis of Behavior
(1979)
W.H. Baum
Optimization and the matching law as accounts of instrumental behavior
Journal of the Experimental Analysis of Behavior
(1981)
M. Davison
Preference in concurrent variable-interval fixed-ratio schedules
Journal of the Experimental Analysis of Behavior
(1982)
M. Fleshler et al.
A progression for generating variable-interval schedules
Journal of the Experimental Analysis of Behavior
(1962)
L. Green et al.
Matching and maximizing with concurrent ratio-interval schedules
Journal of the Experimental Analysis of Behavior
(1983)
R.J. Herrnstein
Relative and absolute strength of response as a function of frequency of reinforcement
Journal of the Experimental Analysis of Behavior
(1961)
R.J. Herrnstein
On the law of effect
Journal of the Experimental Analysis of Behavior
(1970)

R.J. Herrnstein et al.

Is matching compatible with reinforcement maximization on concurrent variable interval, variable ratio?

Journal of the Experimental Analysis of Behavior

(1979)

J.M. Hinson et al.

Hill-climbing by pigeons

Journal of the Experimental Analysis of Behavior

(1983)

J.M. Hinson et al.

Matching, maximizing, and hillclimbing

Journal of the Experimental Analysis of Behavior

(1983)

Cited by (18)

Matching Behaviours and Rewards
2021, Trends in Cognitive Sciences
Citation Excerpt :
In the concurrent VI VI schedule paradigm, it is worth switching to a schedule because there is now likely to be a reward. The extent to which choices can be predicted by maximising the probability of reward (momentary maximising [33]) has received considerable discussion [34–36]. Some analyses of concurrent VI VI schedules assume that the gain from a schedule is just a function of total time allocated to it [11,37].
Matching describes how behaviour is related to rewards. The matching law holds when the ratio of an individual’s behaviours equals the ratio of the rewards obtained. From its origins in the study of pigeons working for food in the laboratory, the law has been applied to a range of species, both in the laboratory and outside it (e.g., human sporting decisions). Probability matching occurs when the probability of a behaviour equals the probability of being rewarded. Input matching predicts the distribution of individuals across habitats. We evaluate the rationality of the matching law and probability matching, expose the logic of matching in real-world cases, review how recent neuroscience findings relate to matching, and suggest future research directions.
Dynamic decision making and value computations in medial frontal cortex
2021, International Review of Neurobiology
Citation Excerpt :
In this task, matching is not the optimal solution. When these types of tasks have been tested, matching behavior has been observed (Herrnstein & Heyman, 1979; Vyse & Belke, 1992; Williams, 1985), including in humans (Savastano & Fantino, 1994). Under the theory proposed by Sakai and Fukai, these findings indicate that animals behave as if they are not aware of environmental statistics.
Dynamic decision making requires an intact medial frontal cortex. Recent work has combined theory and single-neuron measurements in frontal cortex to advance models of decision making. We review behavioral tasks that have been used to study dynamic decision making and algorithmic models of these tasks using reinforcement learning theory. We discuss studies linking neurophysiology and quantitative decision variables. We conclude with hypotheses about the role of other cortical and subcortical structures in dynamic decision making, including ascending neuromodulatory systems.
The implications of imperfect measurement for free-choice carry-over effects: Reply to M. Keith Chen's (2008) "Rationalization and cognitive dissonance: Do choices affect or reflect preferences?"
2009, Journal of Experimental Social Psychology
Examinations of post-choice decision-making behavior often involve two successive choices. At time 1, participants choose between two equally attractive items. At time 2, participants choose between the unchosen item from time 1 and a new item that is roughly equal in attractiveness to the other two. The option rejected at time 1 will tend to again be rejected at time 2, a tendency often attributed to a psychological carry-over effect. Chen (2008) (Available from: http://cowles.econ.yale.edu/P/cd/d16b/d1669.pdf) discounts these psychological explanations. He argues that, given the time 1 choice, there is, in fact, a baserate probability of 66.7% that the initially unchosen item will be rejected again at time 2. However, Chen’s argument rests on the unwarranted assumption that the time 1 choice provides a perfectly reliable measure of subjects’ preference for the chosen item over the unchosen item. With more realistic estimates of the association between preference and choice, Chen’s statistical explanation cannot fully account for the carry-over effect. Alternative experimental methodologies that eliminate Chen’s statistical explanation are discussed.
Is choice a reliable predictor of choice? A comment on Sagarin and Skowronski
2009, Journal of Experimental Social Psychology
In a recent working paper, Chen (2008) argues that a methodology central to the cognitive dissonance literature (the free-choice paradigm) has suffered from an inability to separately measure how much choices affect people’s preferences, and how much they simply reflect those preferences, by failing to fully control for the fact that subjects tend to choose goods they prefer. Although Sagarin and Skowronski concede this, they discount Chen’s argument, claiming that for revealed preferences to completely account for observed choice-effects the relationship between choice and preference would have to be unrealistically high. In this comment, we argue that their critique both misses the crux of Chen’s analysis, and is incorrect. Specifically, to properly test whether choices affect preferences, it is essential that researchers experimentally control for revealed preferences rather than speculate how much of a role they may play. Moreover, Sagarin and Skowronski’s critique rests on two fundamental errors—a misunderstanding of the function of the null-hypothesis and a misunderstanding of preference-measurement psychometrics. These errors leads Sagarin and Skowronski to suggest alternative experimental designs which while a good first step, do not address the problems identified in Chen (2008).
Reinforcement and Choice
1994, Animal Learning and Cognition
The effects of response contingency and reinforcement identity on response suppression by alternative reinforcement
1989, Learning and Motivation
Rats were trained on a multiple schedule in which lever pressing was reinforced in one component and nose poking was reinforced in the other. The reinforcers contingent on the two responses were always different, either corn oil or Noyes pellets. A schedule of alternative reinforcement was superimposed in each component, either presenting the same reinforcer as that contingent on the lever/key, or the alternative reinforcer. When the alternative reinforcers were freely presented on a variable time (VT) schedule, their suppressive effects were substantially larger when the free reinforcer was the same as the reinforcer contingent on the lever/key than when it was different. But when the alternative reinforcers were presented according to a variable-interval schedule for a third response (rod pressing), the differences caused by the nature of reinforcement were greatly reduced and not statistically significant. Similar effects occurred when the alternative reinforcement was presented on a VT schedule but preceded by a signal. The results suggest that the effects of alternative reinforcement cannot be explained solely in terms of relative reinforcement rate.

View all citing articles on Scopus

^☆: This research was supported by NIMH Research Grant 1 RO1 MH 35572-02 and NSF Research Grant BNS84-08878 to the University of California, San Diego.

View full text

Choice behavior in a discrete-trial concurrent VI-VR: A test of maximizing theories of matching☆

Abstract

On two types of deviation from the matching law: Bias and undermatching

Journal of the Experimental Analysis of Behavior

Time-based and count-based measurement of preference

Journal of the Experimental Analysis of Behavior

Matching, undermatching, and overmatching in studies of choice

Journal of the Experimental Analysis of Behavior

Optimization and the matching law as accounts of instrumental behavior

Journal of the Experimental Analysis of Behavior

Preference in concurrent variable-interval fixed-ratio schedules

Journal of the Experimental Analysis of Behavior

A progression for generating variable-interval schedules

Journal of the Experimental Analysis of Behavior

Matching and maximizing with concurrent ratio-interval schedules

Journal of the Experimental Analysis of Behavior

Relative and absolute strength of response as a function of frequency of reinforcement

Journal of the Experimental Analysis of Behavior

On the law of effect

Journal of the Experimental Analysis of Behavior

Is matching compatible with reinforcement maximization on concurrent variable interval, variable ratio?

Journal of the Experimental Analysis of Behavior

Hill-climbing by pigeons

Journal of the Experimental Analysis of Behavior

Matching, maximizing, and hillclimbing

Journal of the Experimental Analysis of Behavior