Mechanisms of hierarchical reinforcement learning in corticostriatal circuits 1: computational analysis.

No abstract is available for this paper.

• Publication year

  2012

• Venue

  Cerebral Cortex

• Publication date

  2012-03-01

• Fields of study

  Medicine, Computer Science, Psychology

• Identifiers
  DOI 10.1093/cercor/bhr114PMID 21693490PMCID PMC3278315
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• Parallel Organization

  2014cited by this paper
• A Framework for content-adaptive photo manipulation macros: Application to face, landscape, and global manipulations

  2011cited by this paper
• Neurogenetics and Pharmacology of Learning, Motivation, and Cognition

  2011cited by this paper
• Frontal cortex and the discovery of abstract action rules.

  2010influential reference
• Mechanisms underlying dopamine-mediated reward bias in compulsive behaviors.

  2010cited by this paper
• Human and Rodent Homologies in Action Control: Corticostriatal Determinants of Goal-Directed and Habitual Action

  2010cited by this paper
• Frontal theta links prediction errors to behavioral adaptation in reinforcement learning

  2010cited by this paper
• Selective impairment of prediction error signaling in human dorsolateral but not ventral striatum in Parkinson's disease patients: evidence from a model-based fMRI study

  2010cited by this paper
• Neural mechanisms of acquired phasic dopamine responses in learning.

  2010cited by this paper
• The role of dopamine in modulating the structure and function of striatal circuits.

  2010cited by this paper
• Prefrontal cortex and basal ganglia contributions to visual working memory

  2010cited by this paper
• Neural response to working memory load varies by dopamine transporter genotype in children

  2010cited by this paper
• Keeping Memory Clear and Stable—The Contribution of Human Basal Ganglia and Prefrontal Cortex to Working Memory

  2010cited by this paper
• Bayesian model selection for group studies

  2009cited by this paper
• Prefrontal organization of cognitive control according to levels of abstraction.

  2009cited by this paper
• Faculty Opinions recommendation of Hierarchical cognitive control deficits following damage to the human frontal lobe.

  2009cited by this paper
• Pharmacological modulation of subliminal learning in Parkinson's and Tourette's syndromes

  2009cited by this paper
• Hierarchical cognitive control deficits following damage to the human frontal lobe

  2009cited by this paper
• Instructional control of reinforcement learning: a behavioral and neurocomputational investigation.

  2009cited by this paper
• Developing PFC representations using reinforcement learning.

  2009cited by this paper
• Hierarchically organized behavior and its neural foundations: a reinforcement learning perspective.

  2009cited by this paper
• Striatal Dopamine Predicts Outcome-Specific Reversal Learning and Its Sensitivity to Dopaminergic Drug Administration

  2009cited by this paper
• Personality, addiction, dopamine: insights from Parkinson's disease.

  2009cited by this paper
• Motivation and cognitive control in the human prefrontal cortex

  2009cited by this paper
• Human Reinforcement Learning Subdivides Structured Action Spaces by Learning Effector-Specific Values

  2009cited by this paper
• Hierarchical models of behavior and prefrontal function.

  2008cited by this paper
• Dichotomous Dopaminergic Control of Striatal Synaptic Plasticity

  2008influential reference
• Prefrontal cortex and basal ganglia control access to working memory

  2008cited by this paper
• Working Memory Capacity Predicts Dopamine Synthesis Capacity in the Human Striatum

  2008cited by this paper
• A dopaminergic basis for working memory, learning and attentional shifting in Parkinsonism.

  2008cited by this paper
• Cognitive control, hierarchy, and the rostro-caudal organization of the frontal lobes.

  2008cited by this paper
• Evidence for Segregated and Integrative Connectivity Patterns in the Human Basal Ganglia

  2008cited by this paper
• Anterior Prefrontal Function and the Limits of Human Decision-Making

  2007cited by this paper
• Towards an executive without a homunculus: computational models of the prefrontal cortex/basal ganglia system

  2007cited by this paper
• Relationship between the corticostriatal terminals from areas 9 and 46, and those from area 8A, dorsal and rostral premotor cortex and area 24c: an anatomical substrate for cognition to action

  2007cited by this paper
• Experienced-Weighted Attraction Learning in Normal Form Games

  2007influential reference
• Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning

  2007influential reference
• An information theoretical approach to prefrontal executive function.

  2007cited by this paper
• Functional Magnetic Resonance Imaging Evidence for a Hierarchical Organization of the Prefrontal Cortex

  2007cited by this paper
• Impulsive Personality Predicts Dopamine-Dependent Changes in Frontostriatal Activity during Component Processes of Working Memory

  2007cited by this paper
• L-DOPA Disrupts Activity in the Nucleus Accumbens during Reversal Learning in Parkinson's Disease

  2007cited by this paper
• Multiple Representations of Belief States and Action Values in Corticobasal Ganglia Loops

  2007cited by this paper
• Multilevel structure in behaviour and in the brain: a model of Fuster's hierarchy

  2007cited by this paper
• Dopamine modulation in the basal ganglia locks the gate to working memory

  2006influential reference
• Neural evidence for dissociable components of task-switching.

  2006cited by this paper
• Hold your horses: A dynamic computational role for the subthalamic nucleus in decision making

  2006cited by this paper
• Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans

  2006cited by this paper
• Basal ganglia functional connectivity based on a meta-analysis of 126 positron emission tomography and functional magnetic resonance imaging publications.

  2006cited by this paper
• Cortical substrates for exploratory decisions in humans

  2006influential reference
• Reversal learning in Parkinson's disease depends on medication status and outcome valence.

  2006cited by this paper
• A mechanistic account of striatal dopamine function in human cognition: psychopharmacological studies with cabergoline and haloperidol.

  2006influential reference
• Anatomy of a decision: striato-orbitofrontal interactions in reinforcement learning, decision making, and reversal.

  2006cited by this paper
• Making Working Memory Work: A Computational Model of Learning in the Prefrontal Cortex and Basal Ganglia

  2006cited by this paper
• A Physiologically Plausible Model of Action Selection and Oscillatory Activity in the Basal Ganglia

  2006cited by this paper
• Net influx of plasma 6‐[18F]fluoro‐l‐DOPA (FDOPA) to the ventral striatum correlates with prefrontal processing of affective stimuli

  2006cited by this paper
• The Role of the Ventromedial Prefrontal Cortex in Abstract State-Based Inference during Decision Making in Humans

  2006cited by this paper
• Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control

  2005cited by this paper
• Representation of Action-Specific Reward Values in the Striatum

  2005cited by this paper
• Prefrontal cortex and flexible cognitive control: rules without symbols.

  2005cited by this paper
• Agents of the mind

  2005cited by this paper
• Dynamic Dopamine Modulation in the Basal Ganglia: A Neurocomputational Account of Cognitive Deficits in Medicated and Nonmedicated Parkinsonism

  2005cited by this paper
• How laminar frontal cortex and basal ganglia circuits interact to control planned and reactive saccades

  2004cited by this paper
• Diffusion tensor fiber tracking shows distinct corticostriatal circuits in humans

  2004cited by this paper
• 3-D diffusion tensor axonal tracking shows distinct SMA and pre-SMA projections to the human striatum.

  2004cited by this paper
• Dynamic Analysis of Learning in Behavioral Experiments

  2004cited by this paper
• By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism

  2004cited by this paper
• Evaluating self-generated information: anterior prefrontal contributions to human cognition.

  2003cited by this paper
• The Effect of Cingulate Cortex Lesions on Task Switching and Working Memory

  2003cited by this paper
• The Architecture of Cognitive Control in the Human Prefrontal Cortex

  2003cited by this paper
• The primate basal ganglia: parallel and integrative networks.

  2003cited by this paper
• : Evidence from

  2002cited by this paper
• Multiple Model-Based Reinforcement Learning

  2002influential reference
• Dopamine-dependent plasticity of corticostriatal synapses

  2002cited by this paper
• Limitations of object-based feature encoding in visual short-term memory.

  2002cited by this paper
• Role of the human medial frontal cortex in task switching: a combined fMRI and TMS study.

  2002cited by this paper
• The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity.

  2002cited by this paper
• A computational model of action selection in the basal ganglia. II. Analysis and simulation of behaviour

  2001cited by this paper
• An integrative theory of prefrontal cortex function.

  2001cited by this paper
• A computational model of action selection in the basal ganglia. I. A new functional anatomy

  2001cited by this paper
• Toward a unified model of attention in associative learning

  2001cited by this paper
• Mechanisms of cognitive set flexibility in Parkinson's disease.

  2001cited by this paper
• Interactions between frontal cortex and basal ganglia in working memory: A computational model

  2001influential reference
• The connections of the dopaminergic system with the striatum in rats and primates: an analysis with respect to the functional and compartmental organization of the striatum.

  2000cited by this paper
• The Prefrontal Cortex: Anatomy, Physiology, and Neuropsychology of the Frontal Lobe, 3rd ed.

  2000cited by this paper
• Dissociating the role of the medial and lateral anterior prefrontal cortex in human planning.

  2000cited by this paper
• Corticostriatal and corticosubthalamic input zones from the presupplementary motor area in the macaque monkey: comparison with the input zones from the supplementary motor area.

  1999cited by this paper
• A Feature-Segmentation Model of Short-Term Visual Memory

  1999cited by this paper
• The Prefrontal Cortex: Anatomy, Physiology and Neuropsychology of the Frontal Lobe, 3rd Edition.

  1999cited by this paper
• Experience‐weighted Attraction Learning in Normal Form Games

  1999influential reference
• The Prefrontal Cortex

  1997cited by this paper
• Long Short-Term Memory

  1997influential reference
• A Neural Substrate of Prediction and Reward

  1997cited by this paper
• A framework for mesencephalic dopamine systems based on predictive Hebbian learning

  1996cited by this paper
• Patterns of intrinsic and associational circuitry in monkey prefrontal cortex

  1996cited by this paper
• The basal ganglia: focused selection and inhibition of competing motor programs.

  1996cited by this paper
• Distributed modular architectures linking basal ganglia, cerebellum, and cerebral cortex: their role in planning and controlling action.

  1995cited by this paper
• Building action repertoires: memory and learning functions of the basal ganglia.

  1995cited by this paper
• Adaptive Mixtures of Local Experts

  1991influential reference
• Parallel organization of functionally segregated circuits linking basal ganglia and cortex.

  1986cited by this paper
• The prefrontal cortex: Anatomy, physiology and neuropsychology of the frontal lobe

  1981cited by this paper

• Hybrid neural-cognitive models reveal how memory shapes human reward learning.

  2026cites this paper
• Causal evidence for a shared mechanism linking language and tool use via the putamen

  2026cites this paper
• Dopamine modulates striatal activity when controlling the output of working memory

  2026cites this paper
• Adaptive learning via BG-thalamo-cortical circuitry

  2025cites this paper
• Striatal dopamine can enhance both fast working memory, and slow reinforcement learning, while reducing implicit effort cost sensitivity

  2025influential citation
• Mechanisms of Output Gating for visual working memory

  2025cites this paper
• The neural basis of cognitive processing: A review and a speculative architecture.

  2025cites this paper
• Adaptive chunking improves effective working memory capacity in a prefrontal cortex and basal ganglia circuit

  2025cites this paper
• Chartist: Task-driven Eye Movement Control for Chart Reading

  2025cites this paper
• Simulating Errors in Touchscreen Typing

  2025cites this paper
• Adaptive Cost-Benefit Control Fueled by Striatal Dopamine.

  2025cites this paper
• Constraints on multi-item working memory access: performance costs and retrieval dynamics

  2025cites this paper
• Reinforcement learning increasingly relates to memory specificity from childhood to adulthood

  2025cites this paper
• Abstract rule learning promotes cognitive flexibility in complex environments across species

  2025influential citation
• Striatal Gradient in Value-Decay Explains Regional Differences in Dopamine Patterns and Reinforcement Learning Computations

  2025cites this paper
• Developing algorithmic psychiatry via multi-level spanning computational models

  2025cites this paper
• Transdiagnostic prevention in youth mental health, Part I: rationale, shared risk factors

  2025influential citation
• Dual process impairments in reinforcement learning and working memory systems underlie learning deficits in physiological anxiety

  2025cites this paper
• What Is Cognitive Control?

  2025cites this paper
• Timescales of dopamine release in the striatum as a window into hierarchical control

  2025cites this paper
• Striatal Dopamine Represents Valence on Dynamic Regional Scales

  2025cites this paper
• Limits on selecting multiple items from working memory: the role of context and item competition

  2025cites this paper
• Schemas, reinforcement learning and the medial prefrontal cortex

  2025cites this paper
• From Cognitive Function to Treatment Efficacy in Obsessive–Compulsive Disorder: Insights from a Multidimensional Meta-Analytic Approach

  2024cites this paper
• MARLUI: Multi-Agent Reinforcement Learning for Adaptive Point-and-Click UIs

  2024cites this paper
• The dynamic interplay between in-context and in-weight learning in humans and neural networks

  2024cites this paper
• Neural basis of cognitive control signals in anterior cingulate cortex during delay discounting

  2024cites this paper
• A feature-specific prediction error model explains dopaminergic heterogeneity

  2024cites this paper
• CRTypist: Simulating Touchscreen Typing Behavior via Computational Rationality

  2024cites this paper
• Multiplexed Levels of Cognitive Control through Delta and Theta Neural Oscillations

  2024cites this paper
• Transformer Mechanisms Mimic Frontostriatal Gating Operations When Trained on Human Working Memory Tasks

  2024influential citation
• Task structure tailors the geometry of neural representations in human lateral prefrontal cortex

  2024cites this paper
• Adaptive chunking improves effective working memory capacity in a prefrontal cortex and basal ganglia circuit

  2024influential citation
• Cognitive Control

  2024cites this paper
• Striatal Dopamine Can Enhance Learning, Both Fast and Slow, and Also Make it Cheaper

  2024cites this paper
• Multimodal neuroimaging of hierarchical cognitive control.

  2024cites this paper
• Hypothetical Minds: Scaffolding Theory of Mind for Multi-Agent Tasks with Large Language Models

  2024cites this paper
• Contributions of Attention to Learning in Multidimensional Reward Environments

  2024cites this paper
• Artificial neural networks for model identification and parameter estimation in computational cognitive models

  2024cites this paper
• Reward Reinforcement Creates Enduring Facilitation of Goal-directed Behavior

  2024cites this paper
• Human Curriculum Effects Emerge with In-Context Learning in Neural Networks

  2024cites this paper
• Dopamine transients follow a striatal gradient of reward time horizons

  2024cites this paper
• From Tripping and Falling to Ruminating and Worrying: A Meta-Control Account of Repetitive Negative Thinking.

  2024cites this paper
• Surprise-minimization as a solution to the structural credit assignment problem

  2024cites this paper
• Adaptive circuits for action and value information in rodent operant learning.

  2024cites this paper
• The challenge of learning adaptive mental behavior.

  2024cites this paper
• From Frege to chatGPT: Compositionality in language, cognition, and deep neural networks

  2024cites this paper
• Hierarchical categorization learning is associated with representational changes in the dorsal striatum and posterior frontal and parietal cortex

  2023cites this paper
• Pharmacological modulation of dopamine receptors reveals distinct brain-wide networks associated with learning and motivation in non-human primates

  2023influential citation
• Policy abstraction as a predictor of cognitive effort avoidance.

  2023cites this paper
• Working Memory Gating in Obesity: Insights from a Case-Control fMRI Study

  2023cites this paper
• State-specific alterations in the neural computations underlying inhibitory control in women remitted from bulimia nervosa

  2023cites this paper
• Artificial neural networks for model identification and parameter estimation in computational cognitive models

  2023cites this paper
• Humans adaptively select different computational strategies in different learning environments

  2023cites this paper
• MARLUI: Multi-Agent Reinforcement Learning for Goal-Agnostic Adaptive UIs

  2022cites this paper
• Dynamics of Hierarchical Task Representations

  2022cites this paper
• From Concrete to Abstract Rules: A Computational Sketch

  2022cites this paper
• Adaptive control of synaptic plasticity integrates micro- and macroscopic network function

  2022cites this paper
• Fast rule switching and slow rule updating in a perceptual categorization task

  2022cites this paper
• Topographic organization of the human caudate functional connectivity and age-related changes with resting-state fMRI

  2022cites this paper
• Altered Functional Connectivity of Basal Ganglia in Mild Cognitive Impairment and Alzheimer’s Disease

  2022influential citation
• Sensitive caregiving and reward responsivity: A novel mechanism linking parenting and executive functions development in early childhood.

  2022cites this paper
• Abstract task representations for inference and control.

  2022cites this paper
• Distributed Neural Systems Support Flexible Attention Updating during Category Learning

  2022cites this paper
• The role of conjunctive representations in prioritizing and selecting planned actions

  2022cites this paper
• The Higher, More Complicated: The Neural Mechanism of Hierarchical Task Switching on Prefrontal Cortex

  2022cites this paper
• Opponent control of behavior by dorsomedial striatal pathways depends on task demands and internal state

  2022cites this paper
• Interplay between rule learning and rule switching in a perceptual categorization task

  2022cites this paper
• Survey of Computational Modeling of the Functional Parts of the Brain

  2022cites this paper
• MARLUI: Multi-Agent Reinforcement Learning for Adaptive UIs

  2022cites this paper
• A dual-task approach to inform the taxonomy of inhibition-related processes.

  2022cites this paper
• Cognitive Control as a Multivariate Optimization Problem

  2021cites this paper
• Generalized attention-weighted reinforcement learning

  2021cites this paper
• Modelling Drivers’ Adaptation to Assistance Systems

  2021cites this paper
• Dissociating the Neural Correlates of Planning and Executing Tasks with Nested Task Sets

  2021cites this paper
• Dopaminergic specializations for flexible behavioral control: linking levels of analysis and functional architectures

  2021cites this paper
• Reinforcement-learning in fronto-striatal circuits

  2021cites this paper
• Executive Functions in Aging: An Experimental and Computational Study of the Wisconsin Card Sorting and Brixton Spatial Anticipation Tests

  2021influential citation
• Metacognitive resources for adaptive learning.

  2021cites this paper
• Advances in modeling learning and decision-making in neuroscience

  2021cites this paper
• Corticostriatal Regulation of Language Functions

  2021cites this paper
• Unlocking the reinforcement-learning circuits of the orbitofrontal cortex.

  2021cites this paper
• Touchscreen Typing As Optimal Supervisory Control

  2021cites this paper
• Moving beyond domain-specific versus domain-general options in cognitive neuroscience.

  2021cites this paper
• Wave-like dopamine dynamics as a mechanism for spatiotemporal credit assignment.

  2021cites this paper
• Value signals guide abstraction during learning

  2021cites this paper
• The evolution of hierarchical structure building capacity for language and music: a bottom-up perspective

  2021cites this paper
• Hierarchical clustering optimizes the tradeoff between compositionality and expressivity of task structures for flexible reinforcement learning

  2021cites this paper
• Examining the Trainability and Transferability of Working-Memory Gating Policies

  2021cites this paper
• Human Fronto-Striatal Connectivity is Organized into Discrete Functional Subnetworks

  2021cites this paper
• Computational Neuroscientific Models of Working Memory

  2021cites this paper
• How the Mind Creates Structure: Hierarchical Learning of Action Sequences

  2021cites this paper
• Beyond Gradients: Noise Correlations Control Hebbian Plasticity to Shape Credit Assignment

  2021cites this paper
• Evidence for a gradient within the medial temporal lobes for flexible retrieval under hierarchical task rules

  2021cites this paper
• Hierarchical control as a shared neurocognitive mechanism for language and music.

  2021cites this paper
• Examining the Trainability and Transferability of Working-Memory Gating Policies

  2021cites this paper
• A Spectrum of Time Horizons for Dopamine Signals

  2021cites this paper
• Predicted Utility Modulates Working Memory Fidelity in the Brain

  2021cites this paper
• Neural Circuits and Symbolic Processing.

  2021cites this paper
• PLANNING AND EXECUTING HIERARCHICAL TASK SETS 1 Dissociating the Neural Correlates of Planning and Executing Hierarchical Task Sets

  2021cites this paper