Representation of Action-Specific Reward Values in the Striatum
SciencePublished 25 November 2005
Kazuyuki Samejima, Yasumasa Ueda, Kenji Doya, Minoru Kimura
Citations975
SJR quartileQ1
SJR score10.42
SNIP6.62
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
Representation of action values in the striatum is suggested, which can guide action selection in the basal ganglia circuit in monkeys who chose between left and right handle turns.
Abstract
The estimation of the reward an action will yield is critical in decision-making. To elucidate the role of the basal ganglia in this process, we recorded striatal neurons of monkeys who chose between left and right handle turns, based on the estimated reward probabilities of the actions. During a delay period before the choices, the activity of more than one-third of striatal projection neurons was selective to the values of one of the two actions. Fewer neurons were tuned to relative values or action choice. These results suggest representation of action values in the striatum, which can guide action selection in the basal ganglia circuit.
Keywords
Neuroscience
ScienceA Neural Substrate of Prediction and Reward
9,610 Citations1997Wolfram Schultz, Peter Dayan +1 more
Findings in this work indicate that dopaminergic neurons in the primate whose fluctuating output apparently signals changes or errors in the predictions of future salient and rewarding events can be understood through quantitative theories of adaptive optimizing control.
ScienceDissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
2,151 Citations2004John P. O’Doherty, Peter Dayan +4 more
This work scanned human participants with functional magnetic resonance imaging while they engaged in instrumental conditioning to suggest partly dissociable contributions of the ventral and dorsal striatum to the critic and the actor.
The MIT Press eBooksModels of Information Processing in the Basal Ganglia
1,187 Citations1994
Part 1 Fundamentals: information processing in modular circuits linking basal ganglia and cerebral cortex, James C. Houk context-dependent activity in primate striatum reflecting past and future behavioural events and a model of how the basal ganglia generate and use neural signals that predict reinforcement are presented.
Current Opinion in NeurobiologyComplementary roles of basal ganglia and cerebellum in learning and motor control
963 Citations2000Kenji Doya
This work has suggested that the cerebellum, the basal ganglia, and the cerebral cortex are specialized for different types of learning: namely, supervised learning, reinforcement learning and unsupervised learning, respectively.
Behavioral BiologyThe hippocampus and contextual retrieval of information from memory: A theory
962 Citations1974Richard F. Hirsh
A theory which holds that information is normally stored within a specialized memory rather than the system immediately responsible for the performance of behavior is advanced and the effects of hippocampal ablation are explained in terms of the elimination of such transfer.
NatureA cellular mechanism of reward-related learning
899 Citations2001John N. J. Reynolds, Brian I. Hyland +1 more
It is proposed that stimulation of the substantia nigra when the lever is pressed induces a similar potentiation of cortical inputs to the striatum, positively reinforcing the learning of the behaviour by the rats.
ScienceBuilding Neural Representations of Habits
847 Citations1999Mandar Jog, Yasuo Kubota +3 more
Large and widely distributed changes in the neuronal activity patterns occurred in the sensorimotor striatum during behavioral acquisition, culminating in task-related activity emphasizing the beginning and end of the automatized procedure.
NatureDifferent time courses of learning-related activity in the prefrontal cortex and striatum
728 Citations2005Anitha Pasupathy, Earl K. Miller
During associative learning in monkeys, neural activity in these areas changes at different rates: the striatum showed rapid, almost bistable, changes compared with a slower trend in the prefrontal cortex that was more in accordance with slow improvements in behavioural performance.
Nature NeurosciencePrefrontal cortex and decision making in a mixed-strategy game
708 Citations2004Dominic J. Barraclough, Michelle Conroy +1 more
The role of prefrontal cortex (PFC) in dynamic decision making in monkeys was investigated and it was found that neurons in the dorsolateral prefrontal cortex encoded the animal's past decisions and payoffs, as well as the conjunction between the two, providing signals necessary to update the estimates of expected reward.
NatureReward expectancy in primate prefrental neurons
685 Citations1996Masataka Watanabe
It is reported that the delay activity in prefrontal neurons is dependent also on the particular reward received for the behavioural response, and to the way the reward is given.
ScienceRole for Cingulate Motor Area Cells in Voluntary Movement Selection Based on Reward
658 Citations1998Keisetsu Shima, Jun Tanji
It is suggested that cells in the rostral cingulate motor area, one of the higher order motor areas in the cortex, play a part in processing the reward information for motor selection.
Nature NeuroscienceExpectation of reward modulates cognitive signals in the basal ganglia
642 Citations1998Reiko Kawagoe, Yoriko Takikawa +1 more
The results indicate that the caudate contributes to the determination of oculomotor outputs by connecting motivational values (for example, expectation of reward) to visual information.
Cerebral CortexFeature Article: Distributed Modular Architectures Linking Basal Ganglia, Cerebellum, and Cerebral Cortex: Their Role in Planning and Controlling Action
593 Citations1995James C. Houk, Steven P. Wise
The motor system includes structures distributed widely through the CNS, and a scheme for how they might cooperate in the control of action is presented, including interconnections among the basal ganglia, cerebellum, and cerebral cortex.
NeuronCoincident but Distinct Messages of Midbrain Dopamine and Striatal Tonically Active Neurons
579 Citations2004Genela Morris, David Arkadir +3 more
It is concluded that the striatal cholinergic and dopaminergic systems carry distinct messages by different means, which can be integrated differently to shape the basal ganglia responses to reward-related events.
Trends in NeurosciencesThe corticostriatal projection: from synaptic plasticity to dysfunctions of the basal ganglia
461 Citations1996Paolo Calabresi, Antonio Pisani +2 more
Alterations in striatal synaptic plasticity might be implicated in Parkinson's disease and Huntington's disease, and these forms of synaptic Plasticity could be involved in motor learning.
Journal of NeurophysiologyEffects of Expectations for Different Reward Magnitudes on Neuronal Activity in Primate Striatum
363 Citations2003Howard C. Cromwell, Wolfram Schultz
Reliable preferences in reward choice trials and differences in anticipatory licks, performance errors, and reaction times indicated that animals differentially expected the various reward amounts predicted by the instruction cues.
Journal of NeuroscienceCorrelated Coding of Motivation and Outcome of Decision by Dopamine Neurons
305 Citations2003Takemasa Satoh, Sadamu Nakai +2 more
The dual correlated coding of motivation and REEs suggested the involvement of the dopamine system, both in reinforcement in more elaborate ways than currently proposed and in motivational function in reward-based decision-making and learning.
Current Opinion in NeurobiologyNeural mechanisms of reward-related motor learning
289 Citations2003Jeffery R. Wickens, John N. J. Reynolds +1 more
Dopamine-dependent synaptic plasticity may provide a link between the reward-related firing of dopamine cells and the acquisition of changes in striatal cell activity during learning, and play a special role in the translation of reward signals into context-dependent response probability or directional bias in movement responses.
Journal of NeuroscienceVisual and Anticipatory Bias in Three Cortical Eye Fields of the Monkey during an Adaptive Decision-Making Task
215 Citations2002Brian C. Coe, Kazuya Tomihara +2 more
The results suggest that although all three cortical eye fields reflect attentional and intentional aspects of sensorimotor processing, SEF plays an earlier and perhaps more cognitive role in internally guided decision-making processes for saccades.
Journal of NeuroscienceNeuronal Signals in the Monkey Ventral Striatum Related to Progress through a Predictable Series of Trials
208 Citations1998Munetaka Shidara, Thomas Aigner +1 more
The ventral striatum might be part of a circuit that supports keeping track of progress through learned behavioral sequences that, when successfully completed, lead to reward.
Journal of NeuroscienceIndependent Coding of Movement Direction and Reward Prediction by Single Pallidal Neurons
114 Citations2004David Arkadir, Genela Morris +2 more
The effects of movement-direction and reward probability on neural activity are linearly summed and thus reflect two independent modulations of pallidal activity, and it is proposed that GPe neurons are uniquely suited for independent processing of a multitude of parameters.