States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning.

No abstract is available for this paper.

• Publication year

  2010

• Venue

  Neuron

• Publication date

  2010-05-27

• Fields of study

  Medicine, Computer Science, Psychology

• Identifiers
  DOI 10.1016/j.neuron.2010.04.016PMID 20510862PMCID PMC2895323
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• Predictive Reward Signal of Dopamine Neurons

  2011influential reference
• THE VISUAL EVOKED POTENTIAL IN MAN

  2010cited by this paper
• Determining a role for ventromedial prefrontal cortex in encoding action-based value signals during reward-related decision making.

  2009cited by this paper
• Human Reinforcement Learning Subdivides Structured Action Spaces by Learning Effector-Specific Values

  2009cited by this paper
• Visualization of Group Inference Data in Functional Neuroimaging

  2009cited by this paper
• Novelty and target processing during an auditory novelty oddball: A simultaneous event-related potential and functional magnetic resonance imaging study

  2008cited by this paper
• Neural correlates of perceptual learning in a sensory-motor, but not a sensory, cortical area

  2008influential reference
• BOLD Responses Reflecting Dopaminergic Signals in the Human Ventral Tegmental Area

  2008cited by this paper
• Regulating the expectation of reward via cognitive strategies

  2008cited by this paper
• Posterior parietal cortex encodes autonomously selected motor plans.

  2007cited by this paper
• A common framework for perceptual learning.

  2007cited by this paper
• The Role of the Dorsal Striatum in Reward and Decision-Making

  2007cited by this paper
• Learning the value of information in an uncertain world

  2007cited by this paper
• Multiple Representations of Belief States and Action Values in Corticobasal Ganglia Loops

  2007cited by this paper
• Different neural correlates of reward expectation and reward expectation error in the putamen and caudate nucleus during stimulus-action-reward association learning.

  2006influential reference
• Midbrain dopamine neurons encode decisions for future action

  2006cited by this paper
• Cortical substrates for exploratory decisions in humans

  2006cited by this paper
• Activity in the lateral prefrontal cortex reflects multiple steps of future events in action plans.

  2006cited by this paper
• Distributed Neural Representation of Expected Value

  2005cited by this paper
• Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control

  2005influential reference
• Valid conjunction inference with the minimum statistic

  2005influential reference
• Midbrain dopamine neurons encode a quantitative reward prediction error signal.

  2005cited by this paper
• Reinforcement learning and decision making in monkeys during a competitive game.

  2004cited by this paper
• Temporal difference models describe higher-order learning in humans

  2004cited by this paper
• Matching Behavior and the Representation of Value in the Parietal Cortex

  2004cited by this paper
• Prefrontal cortex and decision making in a mixed-strategy game

  2004cited by this paper
• Temporal prediction errors in a passive learning task activate human striatum.

  2003influential reference
• The primate basal ganglia: parallel and integrative networks.

  2003cited by this paper
• Temporal difference models and reward-related learning in the human brain.

  2003cited by this paper
• Optimized EPI for fMRI studies of the orbitofrontal cortex

  2003cited by this paper
• Multiple Model-Based Reinforcement Learning

  2002cited by this paper
• Transient neural activity in human parietal cortex during spatial attention shifts

  2002cited by this paper
• The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity.

  2002cited by this paper
• ReviewBanburismus and the Brain : Decoding the Relationship between Sensory Stimuli , Decisions , and Reward

  2002cited by this paper
• Banburismus and the brain: decoding the relationship between sensory stimuli, decisions, and reward.

  2002cited by this paper
• STEVENS' HANDBOOK OF EXPERIMENTAL PSYCHOLOGY

  2002cited by this paper
• The 28th Bartlett Memorial Lecture Causal Learning: An Associative Analysis

  2001cited by this paper
• Anticipation of Increasing Monetary Reward Selectively Recruits Nucleus Accumbens

  2001cited by this paper
• The 28th Bartlett Memorial Lecture. Causal learning: an associative analysis.

  2001influential reference
• Neurophysiological investigation of the basis of the fMRI signal

  2001cited by this paper
• Responses of human frontal cortex to surprising events are predicted by formal associative learning theory

  2001influential reference
• Voluntary orienting is dissociated from target detection in human posterior parietal cortex

  2000cited by this paper
• Dissociating the role of the dorsolateral prefrontal and anterior cingulate cortex in cognitive control.

  2000cited by this paper
• Tracking the hemodynamic responses to reward and punishment in the striatum.

  2000cited by this paper
• Neural correlates of decision variables in parietal cortex

  1999cited by this paper
• What are the computations of the cerebellum, the basal ganglia and the cerebral cortex?

  1999cited by this paper
• The functional neuroanatomy of novelty processing: integrating ERP and fMRI results.

  1999cited by this paper
• Experience-Weighted Attraction Learning in Coordination Games: Probability Rules, Heterogeneity, and Time-Variation.

  1998influential reference
• Predictive reward signal of dopamine neurons.

  1998cited by this paper
• Reinforcement Learning: An Introduction

  1998cited by this paper
• Removal of Cholinergic Input to Rat Posterior Parietal Cortex Disrupts Incremental Processing of Conditioned Stimuli

  1998influential reference
• A Neural Substrate of Prediction and Reward

  1997influential reference
• Changes in brain activity patterns in aging: the novelty oddball.

  1995cited by this paper
• Spatial registration and normalization of images

  1995cited by this paper
• A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli.

  1980cited by this paper
• Cognitive maps in rats and men.

  1948cited by this paper
• A PROOF OF THE LAW OF EFFECT.

  1933cited by this paper
• Degrees of hunger, reward and non-reward, and maze learning in rats, and Introduction and removal of reward, and maze performance in rats

  year unknowncited by this paper

• Day-to-day fluctuations in cognitive precision predict the domain-general intention-behavior gap

  2026cites this paper
• Modeling metacognition and executive functions in the metacognitive wisconsin card sorting test using the neuropsychological digital-twin method

  2026cites this paper
• Efficient Learning of Predictive Maps for Flexible Planning

  2026cites this paper
• Deciding for others diminishes model-based decision-making but depends on individual prosociality

  2026cites this paper
• Building Goal-Directed Cognitive Graphs

  2026cites this paper
• A Cognitive Framework for Autonomous Agents: Toward Human-Inspired Design

  2026cites this paper
• Cathodal tDCS over right orbitofrontal cortex enhances the ability in subclinical obsessive-compulsive disorder to shift to model-based control.

  2026cites this paper
• Adaptive integration of model-based and model-free strategies in human reinforcement learning of reachable space

  2026cites this paper
• The prefrontal–striatal signatures of reduced model-based learning in depressed patients

  2026cites this paper
• Implicit extrapolation and the beliefs channel of investment demand

  2026cites this paper
• Information uncertainty influences learning strategy from sequentially delayed rewards

  2026cites this paper
• Realist representational explanations of agency should require some unity of purpose

  2026cites this paper
• Locus coeruleus norepinephrine neurons facilitate orbitofrontal cortex remapping and behavioral flexibility.

  2025cites this paper
• Fragmentation and multithreading of experience in the default-mode network

  2025cites this paper
• The brain-AI convergence: Predictive and generative world models for general-purpose computation

  2025cites this paper
• Reinforcement learning at the interface of artificial intelligence and cognitive science.

  2025cites this paper
• Deep Reinforcement Learning Based Energy Management Strategy for Vertical Take-Off and Landing Aircraft with Turbo-Electric Hybrid Propulsion System

  2025cites this paper
• Impaired arbitration between reward-related decision-making strategies in Alcohol Users compared to Alcohol Non-Users: a computational modeling study

  2025cites this paper
• Environmental dynamics shape human learning: change points versus random walks

  2025cites this paper
• Differences in resting state functional connectivity in striatal and frontal networks associated with task-behaviour elicited by gamified two-stage decision task.

  2025cites this paper
• Orbitofrontal-Hippocampal State Coding Dynamics during Reversal Learning

  2025cites this paper
• Annual Research Review: Neural mechanisms of eating disorders in youth - from current theory and findings to future directions.

  2025cites this paper
• Reduced Model-Based Control in Gambling Disorder Despite Intact Neural Value and Task Structure Representations

  2025influential citation
• Transition ability to safe states reduces fear responses to height

  2025cites this paper
• The neural correlates of novelty and variability in human decision-making under an active inference framework

  2025cites this paper
• An experimental and computational investigation of executive functions and inner speech in schizophrenia spectrum disorders

  2025cites this paper
• Neural correlates of trial outcome monitoring during long-term learning in primate posterior parietal cortex

  2025cites this paper
• Brain-like neural dynamics for behavioral control develop through reinforcement learning

  2025cites this paper
• Deformed Probability Estimation in Goal-Directed reinforcement learning model explains anxious-depression dimensions of psychiatric disorders

  2025cites this paper
• Tachykinin 1-expressing neurons in the lateral habenula signal negative reward prediction error

  2025cites this paper
• The representation and valuation of subgoals in the human brain during model-based hierarchical behavior

  2025influential citation
• Looking deeper into the algorithms underlying human planning.

  2025cites this paper
• Altered model-based prediction error signaling in the lateral orbitofrontal cortex in patients with obsessive-compulsive disorder: An fMRI study

  2025cites this paper
• Neural correlates of reduced sensitivity to information about uncertainty during valuation in older adults: An fNIRS study

  2025cites this paper
• The distinct functions of working memory and intelligence in model-based and model-free reinforcement learning

  2025cites this paper
• Modeling natural neural networks of decision making with artificial neural networks.

  2025cites this paper
• Novelty as a drive of human exploration in complex stochastic environments

  2025cites this paper
• Shaping Memory from the Start: Initial Prediction Errors during First Encoding

  2025cites this paper
• Finding the sweet spot of memory modification: An fMRI study on episodic prediction error strength and type

  2025cites this paper
• When models matter: Environmental demand guides the arbitration between model-based and model-free control

  2025cites this paper
• Striatal arbitration between choice strategies guides few-shot adaptation

  2025cites this paper
• A corticostriatal learning mechanism linking excess striatal dopamine and auditory hallucinations

  2025cites this paper
• Interaction between Model-based and Model-free Mechanisms in Motor Learning

  2025cites this paper
• Higher motivation and pleasure scores predict more reliance on model-free decision making

  2025cites this paper
• Cognitive computational model reveals repetition bias in a sequential decision-making task

  2025cites this paper
• Exploring the Specificity of Linguistic Rule Learning Through Reinforcement Learning: Semantic and Syntactic Perspectives.

  2025cites this paper
• Causal necessity of human hippocampus for structure-based inference in learning

  2025cites this paper
• How prediction error drives memory updating: role of locus coeruleus-hippocampal interactions.

  2025cites this paper
• Computational modelling and neural correlates of reinforcement learning following three-week escitalopram: a double-blind, placebo-controlled semi-randomised study

  2025cites this paper
• Thalamic regulation of reinforcement learning strategies across prefrontal-striatal networks

  2025influential citation
• Goal-directed behavior and hippocampal activity predict real-life impact of drinking intentions in alcohol use disorder

  2025cites this paper
• Distributional dual-process model predicts strategic shifts in decision-making under uncertainty

  2025cites this paper
• What Makes a Good Diffusion Planner for Decision Making?

  2025cites this paper
• A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Models

  2025cites this paper
• Distributed activity in the human posterior putamen distinguishes goal-directed from habitual control

  2025cites this paper
• Working Memory Deficits in School-Age Children With Cochlear Implants Are Primarily Explained by Deficits in the Processing of Auditory and Lexical Information.

  2025cites this paper
• Cross-frequency routing in a hippocampo–cortical circuit during probabilistic reversal learning

  2025cites this paper
• Higher-order and distributed synergistic functional interactions encode information gain in goal-directed learning

  2025cites this paper
• Neurocomputational modeling of rule abstraction and memorization during probabilistic stimulus-reward learning

  2025cites this paper
• Model-based and model-free valuation signals in the human brain vary markedly in their relationship to individual differences in behavioral control

  2025influential citation
• Integration of contextual reward motivation with category representations

  2025cites this paper
• Factorized embedding of goal and uncertainty in the lateral prefrontal cortex guides stably flexible learning

  2025cites this paper
• The neural basis for uncertainty processing in hierarchical decision making

  2025cites this paper
• The Cerebellum and Striatum in Reward Processing: Caring About Being Right vs. Caring About Reward

  2025cites this paper
• A model-centered introduction to curiosity

  2025cites this paper
• Non-invasive ultrasonic neuromodulation of the human nucleus accumbens impacts reward sensitivity

  2025cites this paper
• The neuropsychological correlates of urban navigation: A meta-analysis of 27 years of functional neuroimaging studies

  2025cites this paper
• Internally Formed Preferences for Options only Influence Initial Decisions in Gambling Tasks, while the Gambling Outcomes do not Alter these Preferences

  2024cites this paper
• Self-Adaptive Electronics using Artificial Neural Networks

  2024cites this paper
• Cognitive Computational Model Reveals Repetition Bias in a Sequential Decision-Making Task

  2024cites this paper
• Online Learning Based Control for Mobile Vehicle to Avoid Static Obstacles via Kernel Based Safe Dynamic Model

  2024cites this paper
• How do anxiety and stress affect soccer referees? An ERPs study

  2024cites this paper
• Reconciling flexibility and efficiency: medial entorhinal cortex represents a compositional cognitive map

  2024cites this paper
• Leveraging individual differences in cue-reward learning to investigate the psychological and neural basis of shared psychiatric symptomatology: The sign-tracker/goal-tracker model.

  2024cites this paper
• Heterogeneity in strategy use during arbitration between experiential and observational learning

  2024cites this paper
• Neurocomputational model of compulsivity: deviating from an uncertain goal-directed system

  2024cites this paper
• Computational Mechanisms Underlying Motivation to Earn Symbolic Reinforcers

  2024cites this paper
• Surprise-minimization as a solution to the structural credit assignment problem

  2024cites this paper
• Hippocampus and striatum show distinct contributions to longitudinal changes in value-based learning in middle childhood

  2024cites this paper
• Modelling cognitive flexibility with deep neural networks

  2024cites this paper
• Observational reinforcement learning in children and young adults

  2024cites this paper
• From brain to body: exploring the connection between altered reward processing and physical fitness in schizophrenia.

  2024cites this paper
• A Sequential Sampling Approach to the Integration of Habits and Goals

  2024cites this paper
• Characterizing Human Habits in the Lab

  2024cites this paper
• Active reinforcement learning versus action bias and hysteresis: control with a mixture of experts and nonexperts

  2024cites this paper
• Midbrain signaling of identity prediction errors depends on orbitofrontal cortex networks

  2024cites this paper
• Encoding of Predictive Associations in Human Prefrontal and Medial Temporal Neurons During Pavlovian Appetitive Conditioning

  2024cites this paper
• Decoding Human Prediction Errors for Human-Robot Value Alignment

  2024cites this paper
• Monkey Prefrontal Cortex Learns to Minimize Sequence Prediction Error

  2024cites this paper
• Whoa, No-Go: Evidence consistent with model-based strategy use in horses during an inhibitory task

  2024cites this paper
• Over the Quantum Rainbow: Explaining Hybrid Quantum Reinforcement Learning

  2024cites this paper
• Reinforcement Learning-based Autonomous Driving Path Planning for Smart Connected Vehicles

  2024cites this paper
• Construction and use of mental models: Organizing principles for the science of brain and mind.

  2024cites this paper
• The effect of transcranial Direct Current Stimulation on the Iowa Gambling Task: a scoping review

  2024cites this paper
• The role of training variability for model-based and model-free learning of an arbitrary visuomotor mapping

  2024cites this paper
• Policy consequences of the new neuroeconomic framework

  2024cites this paper
• Doubling down on dual systems: A cerebellum-amygdala route towards action- and outcome-based social and affective behavior.

  2024cites this paper
• Making sense of feelings

  2024cites this paper
• Global neural encoding of behavioral strategies in mice during perceptual decision-making task with two different sensory patterns

  2024cites this paper
• Evaluating the Influence of Musical and Monetary Rewards on Decision Making through Computational Modelling

  2024cites this paper