A gradual temporal shift of dopamine responses mirrors the progression of temporal difference error in machine learning

No abstract is available for this paper.

• Publication year

  2022

• Venue

  Nature Neuroscience

• Publication date

  2022-07-07

• Fields of study

  Medicine, Computer Science

• Identifiers
  DOI 10.1038/s41593-022-01109-2PMID 35798979PMCID PMC9624460
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• Reporting for specific materials, systems and methods

  2021cited by this paper
• Next-generation GRAB sensors for monitoring dopaminergic activity in vivo

  2020influential reference
• Distinct temporal difference error signals in dopamine axons in three regions of the striatum in a decision-making task

  2020cited by this paper
• Deep Reinforcement Learning and its Neuroscientific Implications

  2020cited by this paper
• A systems-neuroscience model of phasic dopamine.

  2020cited by this paper
• Cue-Evoked Dopamine Promotes Conditioned Responding during Learning.

  2020cited by this paper
• A distributional code for value in dopamine-based reinforcement learning

  2020cited by this paper
• Specialized coding of sensory, motor, and cognitive variables in VTA dopamine neurons

  2019cited by this paper
• Dissociable dopamine dynamics for learning and motivation.

  2019cited by this paper
• Causal evidence supporting the proposal that dopamine transients function as temporal difference prediction errors

  2019cited by this paper
• Causal evidence supporting the proposal that dopamine transients function as temporal difference prediction errors

  2019cited by this paper
• High-performance calcium sensors for imaging activity in neuronal populations and microcompartments

  2019cited by this paper
• A unified framework for dopamine signals across timescales

  2019influential reference
• FISSA: A neuropil decontamination toolbox for calcium imaging signals

  2018cited by this paper
• Reward prediction error does not explain movement selectivity in DMS-projecting dopamine neurons

  2018cited by this paper
• The timing of action determines reward prediction signals in identified midbrain dopamine neurons

  2018cited by this paper
• A Suite of Transgenic Driver and Reporter Mouse Lines with Enhanced Brain-Cell-Type Targeting and Functionality.

  2018cited by this paper
• Belief state representation in the dopamine system

  2018cited by this paper
• Dysfunction of ventrolateral striatal dopamine receptor type 2-expressing medium spiny neurons impairs instrumental motivation

  2017cited by this paper
• Neural Circuitry of Reward Prediction Error.

  2017cited by this paper
• Learning and Stress Shape the Reward Response Patterns of Serotonin Neurons

  2017cited by this paper
• Suite2p: beyond 10,000 neurons with standard two-photon microscopy

  2016cited by this paper
• Midbrain dopamine neurons signal aversion in a reward-context-dependent manner

  2016cited by this paper
• Distributed and Mixed Information in Monosynaptic Inputs to Dopamine Neurons.

  2016influential reference
• The integrative role of orexin/hypocretin neurons in nociceptive perception and analgesic regulation

  2016cited by this paper
• Opposite initialization to novel cues in dopamine signaling in ventral and posterior striatum in mice

  2016influential reference
• Habenula Lesions Reveal that Multiple Mechanisms Underlie Dopamine Prediction Errors.

  2015cited by this paper
• Human-level control through deep reinforcement learning

  2015cited by this paper
• Dopamine neurons projecting to the posterior striatum form an anatomically distinct subclass

  2015cited by this paper
• Reward Contexts Extend Dopamine Signals to Unrewarded Stimuli

  2014cited by this paper
• Dopamine Encoding of Pavlovian Incentive Stimuli Diminishes with Extended Training

  2013cited by this paper
• Ultra-sensitive fluorescent proteins for imaging neuronal activity

  2013cited by this paper
• Paxinos and Franklin's the Mouse Brain in Stereotaxic Coordinates

  2012cited by this paper
• Neuron-type specific signals for reward and punishment in the ventral tegmental area

  2011cited by this paper
• Optogenetic interrogation of neural circuits: technology for probing mammalian brain structures

  2010cited by this paper
• A selective role for dopamine in reward learning

  2010cited by this paper
• A pallidus-habenula-dopamine pathway signals inferred stimulus values.

  2010cited by this paper
• Hierarchically organized behavior and its neural foundations: a reinforcement learning perspective.

  2009cited by this paper
• A robust and high-throughput Cre reporting and characterization system for the whole mouse brain

  2009cited by this paper
• A Local Circuit Model of Learned Striatal and Dopamine Cell Responses under Probabilistic Schedules of Reward

  2008cited by this paper
• Influence of Reward Delays on Responses of Dopamine Neurons

  2008cited by this paper
• Synaptic glutamate release by ventromedial hypothalamic neurons is part of the neurocircuitry that prevents hypoglycemia.

  2007cited by this paper
• Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards

  2007cited by this paper
• Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens

  2007cited by this paper
• PVLV: the primary value and learned value Pavlovian learning algorithm.

  2007cited by this paper
• Efficient reinforcement learning: computational theories, neuroscience and robotics.

  2007cited by this paper
• Towards a Unified Theory of State Abstraction for MDPs

  2006cited by this paper
• Characterization of a mouse strain expressing Cre recombinase from the 3′ untranslated region of the dopamine transporter locus

  2006cited by this paper
• Dopamine Cells Respond to Predicted Events during Classical Conditioning: Evidence for Eligibility Traces in the Reward-Learning Network

  2005influential reference
• Behavioral and Brain Functions

  2005cited by this paper
• Speed and accuracy of olfactory discrimination in the rat

  2003cited by this paper
• Dopamine: generalization and bonuses

  2002cited by this paper
• How the Basal Ganglia Use Parallel Excitatory and Inhibitory Learning Pathways to Selectively Respond to Unexpected Rewarding Cues

  1999cited by this paper
• Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning

  1999cited by this paper
• Reinforcement Learning: An Introduction

  1998cited by this paper
• Dopamine neurons report an error in the temporal prediction of reward during learning

  1998influential reference
• A Neural Substrate of Prediction and Reward

  1997influential reference
• A framework for mesencephalic dopamine systems based on predictive Hebbian learning

  1996influential reference
• Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task

  1993cited by this paper
• A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement

  1972cited by this paper

• Dopamine signatures of excessive and compulsive cocaine and fentanyl use

  2026cites this paper
• Duration between rewards controls the rate of behavioral and dopaminergic learning.

  2026cites this paper
• Consensus Paper: Models of Cerebellar Functions

  2026cites this paper
• Actor–critic networks with analogue memristors mimicking reward-based learning

  2025cites this paper
• Trial-by-trial learning of successor representations in human behavior

  2025cites this paper
• Dopamine release effects on striatal blood oxygenation and whole brain plasticity underlying associative learning

  2025cites this paper
• Local Reinforcement Learning with Action-Conditioned Root Mean Squared Q-Functions

  2025cites this paper
• Individual differences in decision-making shape how mesolimbic dopamine regulates choice confidence and change-of-mind

  2025cites this paper
• Flexible value coding in the mesolimbic dopamine system depending on internal water and sodium balance

  2025cites this paper
• Learning in Temporally Complex Problems: The Role of External Knowledge

  2025cites this paper
• Prospective contingency explains behavior and dopamine signals during associative learning

  2025influential citation
• Ventral Tegmental Area Dopamine Neural Activity Switches Simultaneously with Rule Representations in the Medial Prefrontal Cortex and Hippocampus

  2025cites this paper
• Neurocomputational underpinnings of suboptimal beliefs in recurrent neural network-based agents

  2025cites this paper
• A statistical framework for analysis of trial-level temporal dynamics in fiber photometry experiments

  2025cites this paper
• Interpretable deep learning for deconvolutional analysis of neural signals.

  2025cites this paper
• Contextual cues facilitate dynamic value encoding in the mesolimbic dopamine system.

  2025cites this paper
• Reconciling time and prediction error theories of associative learning

  2025cites this paper
• Distributed representations of temporally accumulated reward prediction errors in the mouse cortex

  2025cites this paper
• Trial-by-trial learning of successor representations in human behavior

  2025cites this paper
• Individual differences in decision-making shape how mesolimbic dopamine regulates choice confidence and change-of-mind

  2024cites this paper
• Predictive reward-prediction errors of climbing fiber inputs integrate modular reinforcement learning with supervised learning

  2024cites this paper
• A generalizable computational mechanism underlying the interaction between momentary craving and decision-making

  2024cites this paper
• Interpretable deep learning for deconvolutional analysis of neural signals

  2024cites this paper
• CoBjeason: Reasoning Covered Object in Image by Multi-Agent Collaboration Based on Informed Knowledge Graph

  2024cites this paper
• Striatal dopamine signals reflect perceived cue–action–outcome associations in mice

  2024cites this paper
• Frontal norepinephrine represents a threat prediction error under uncertainty.

  2024cites this paper
• The role of prospective contingency in the control of behavior and dopamine signals during associative learning

  2024cites this paper
• Glutamate inputs send prediction error of reward, but not negative value of aversive stimuli, to dopamine neurons.

  2024cites this paper
• Dimensionality reduction beyond neural subspaces with slice tensor component analysis

  2024cites this paper
• A dopamine mechanism for reward maximization

  2024cites this paper
• Functional diversity of dopamine axons in prefrontal cortex during classical conditioning

  2024cites this paper
• Rethinking dopamine‐guided action sequence learning

  2024influential citation
• Learning to express reward prediction error-like dopaminergic activity requires plastic representations of time

  2024influential citation
• TDSTDP

  2024cites this paper
• Explaining dopamine through prediction errors and beyond

  2024cites this paper
• Distinct cholinergic circuits underlie discrete effects of reward on attention

  2024cites this paper
• Predictive Coding of Reward in the Hippocampus

  2024cites this paper
• Ventral tegmental area dopamine neural activity switches simultaneously with rule representations in the prefrontal cortex and hippocampus

  2024cites this paper
• Dopamine release plateau and outcome signals in dorsal striatum contrast with classic reinforcement learning formulations

  2024cites this paper
• Dopamine dynamics are dispensable for movement but promote reward responses

  2024influential citation
• A light at the end of the axon: genetically encoded fluorescent indicators shine light on the dopamine system

  2024cites this paper
• Opponent control of reinforcement by striatal dopamine and serotonin

  2024cites this paper
• “PyTDL”: A versatile temporal difference learning algorithm to simulate behavior process of decision making and cognitive learning

  2024cites this paper
• A Statistical Framework for Analysis of Trial-Level Temporal Dynamics in Fiber Photometry Experiments

  2023cites this paper
• Glutamate inputs send prediction error of reward but not negative value of aversive stimuli to dopamine neurons

  2023cites this paper
• Tonic dopamine and biases in value learning linked through a biologically inspired reinforcement learning model

  2023cites this paper
• Prediction error in dopamine neurons during associative learning.

  2023influential citation
• Striatal integration of inverse dopamine and serotonin signals gates learning

  2023cites this paper
• Mesostriatal dopamine is sensitive to specific cue-reward contingencies

  2023cites this paper
• Learning to Express Reward Prediction Error-like Dopaminergic Activity Requires Plastic Representations of Time

  2023influential citation
• Subsecond fluctuations in extracellular dopamine encode reward and punishment prediction errors in humans

  2023cites this paper
• Dual credit assignment processes underlie dopamine signals in a complex spatial environment

  2023cites this paper
• Contexts facilitate dynamic value encoding in the mesolimbic dopamine system

  2023cites this paper
• PLCγ1 in dopamine neurons critically regulates striatal dopamine release via VMAT2 and synapsin III

  2023cites this paper
• Striatal Dopamine Signals and Reward Learning

  2023cites this paper
• Biologically plausible local synaptic learning rules robustly implement deep supervised learning

  2023cites this paper
• Deep-brain optical recording of neural dynamics during behavior.

  2023cites this paper
• Slow‐rising and fast‐falling dopaminergic dynamics jointly adjust negative prediction error in the ventral striatum

  2023influential citation
• Dopamine Release Plateau and Outcome Signals in Dorsal Striatum Contrast with Classic Reinforcement Learning Formulations

  2023cites this paper
• Nucleus accumbens dopamine release reflects the selective nature of pair bonds

  2022cites this paper
• Prefrontal norepinephrine represents a threat prediction error under uncertainty

  2022cites this paper
• Pupillary dynamics of mice performing a Pavlovian delay conditioning task reflect reward-predictive signals

  2022cites this paper
• Temporal scaling and computing time in neural circuits: Should we stop watching the clock and look for its gears?

  2022cites this paper
• Mesolimbic dopamine release conveys causal associations

  2022cites this paper
• Mesolimbic dopamine adapts the rate of learning from action

  2021cites this paper