Hierarchically organized behavior and its neural foundations: a reinforcement learning perspective.

No abstract is available for this paper.

• Publication year

  2009

• Venue

  Cognition

• Publication date

  2009-12-01

• Fields of study

  Medicine, Computer Science, Psychology

• Identifiers
  DOI 10.1016/j.cognition.2008.08.011PMID 18926527PMCID PMC2783353
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• New Directions in Attribution Research

  2018cited by this paper
• Cognitive control, hierarchy, and the rostro-caudal organization of the frontal lobes.

  2008cited by this paper
• Action outcomes are represented in human inferior frontoparietal cortex.

  2008cited by this paper
• Hierarchical models of behavior and prefrontal function.

  2008influential reference
• Automatic discovery and transfer of MAXQ hierarchies

  2008cited by this paper
• Proc. Computational and Systems Neuroscience COSYNE '08, Salt Lake City, Utah (2008)

  2008cited by this paper
• A hierarchical biased-competition model of domain-dependent working memory mainatenance and executive control

  2007cited by this paper
• Categorization of behavioural sequences in the prefrontal cortex

  2007cited by this paper
• Prefrontal Neural Correlates of Memory for Sequences

  2007cited by this paper
• Event perception: a mind-brain perspective.

  2007cited by this paper
• Angelic Semantics for High-Level Actions

  2007cited by this paper
• Multilevel structure in behaviour and in the brain: a model of Fuster's hierarchy

  2007cited by this paper
• Evolutionary Development of Hierarchical Learning Structures

  2007cited by this paper
• From cognitive to neural models of working memory

  2007cited by this paper
• Heterarchical reinforcement-learning model for integration of multiple cortico-striatal loops: fMRI examination in stimulus-action-reward association learning

  2006cited by this paper
• Such stuff as habits are made on: A reply to Cooper and Shallice (2006).

  2006influential reference
• Biologically Inspired Approaches to Advanced Information Technology, Second International Workshop, BioADIT 2006, Osaka, Japan, January 26-27, 2006, Proceedings

  2006cited by this paper
• Towards a Unified Theory of State Abstraction for MDPs

  2006cited by this paper
• Activity of tonically active neurons in the monkey putamen during initiation and withholding of movement.

  2006cited by this paper
• Working memory as an emergent property of the mind and brain.

  2006cited by this paper
• Recent breakthroughs in basal ganglia research

  2006cited by this paper
• Effects of end-goal on hand shaping.

  2006cited by this paper
• Making Working Memory Work: A Computational Model of Learning in the Prefrontal Cortex and Basal Ganglia

  2006cited by this paper
• Anatomy of a decision: striato-orbitofrontal interactions in reinforcement learning, decision making, and reversal.

  2006cited by this paper
• Hierarchical control of cognitive processes: switching tasks in sequences.

  2006cited by this paper
• Advances in Neural Information Processing Systems 17: Proceedings of the 2004 Conference

  2006cited by this paper
• Neural Activity in Monkey Prefrontal Cortex Is Modulated by Task Context and Behavioral Instruction during Delayed-match-to-sample and Conditional ProsaccadeAntisaccade Tasks

  2006cited by this paper
• Absolute coding of stimulus novelty in the human substantia nigra/VTA.

  2006cited by this paper
• Encoding of time-discounted rewards in orbitofrontal cortex is independent of value representation.

  2006cited by this paper
• A Comparison of Abstract Rules in the Prefrontal Cortex, Premotor Cortex, Inferior Temporal Cortex, and Striatum

  2006cited by this paper
• The short-latency dopamine signal: a role in discovering novel actions?

  2006cited by this paper
• Actor-critic algorithms for hierarchical Markov decision processes

  2006cited by this paper
• Actions , Policies , Values , and the Basal Ganglia

  2005cited by this paper
• Prefrontal cortex and flexible cognitive control: rules without symbols.

  2005cited by this paper
• Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control

  2005cited by this paper
• Production of Negative Transfer in a Problem-Solving Task

  2005cited by this paper
• Uncertainty-based competition between prefrontal and striatal systems for behavioural control

  2005cited by this paper
• Brain-based mechanisms underlying complex causal thinking.

  2005cited by this paper
• Pulling out the intentional structure of action: the relation between action processing and action production in infancy.

  2005cited by this paper
• Identifying useful subgoals in reinforcement learning by local graph partitioning

  2005cited by this paper
• A causal approach to hierarchical decomposition of factored MDPs

  2005cited by this paper
• Integrating Reinforcement Learning with Soar

  2005cited by this paper
• Infants' Sensitivity to the Causal Features of Means‐End Support Sequences in Action and Perception

  2005cited by this paper
• Reinforcement learning with via-point representation

  2004cited by this paper
• Upper processing stages of the perception-action cycle.

  2004influential reference
• Coincident but distinct messages of midbrain dopamine and striatal tonically active neurons.

  2004cited by this paper
• A Agent Environment StatesActions Rewards Critic B Agent Internal Environment Rewards Critic External Environment Sensations StatesDecisions Actions " Organism " Figure 1

  2004cited by this paper
• An integrated theory of the mind.

  2004influential reference
• Mechanisms of theory formation in young children.

  2004cited by this paper
• Intrinsically Motivated Reinforcement Learning

  2004cited by this paper
• A theory of causal learning in children: causal maps and Bayes nets.

  2004cited by this paper
• Biologically Inspired Reinforcement Learning: Reward-Based Decomposition for Multi-goal Environments

  2004cited by this paper
• Dynamic abstraction in reinforcement learning via clustering

  2004cited by this paper
• Basal ganglia neural mechanisms of natural movement sequences.

  2004cited by this paper
• How we use rules to select actions: A review of evidence from cognitive neuroscience

  2004cited by this paper
• Doing without schema hierarchies: a recurrent connectionist approach to normal and impaired routine sequential action.

  2004influential reference
• Fast Remapping of Sensory Stimuli onto Motor Actions on the Basis of Contextual Modulation

  2004cited by this paper
• Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning

  2004cited by this paper
• Action sets and decisions in the medial frontal cortex.

  2004cited by this paper
• Production Compilation: A Simple Mechanism to Model Complex Skill Acquisition

  2003cited by this paper
• Dissociating Valence of Outcome from Behavioral Control in Human Orbital and Ventral Prefrontal Cortices

  2003cited by this paper
• Human prefrontal cortex: processing and representational perspectives

  2003cited by this paper
• The Architecture of Cognitive Control in the Human Prefrontal Cortex

  2003cited by this paper
• Recent Advances in Hierarchical Reinforcement Learning

  2003influential reference
• Embodied artificial intelligence

  2003cited by this paper
• Teleological reasoning in infancy: the nai;ve theory of rational action.

  2003cited by this paper
• Intrinsic and Extrinsic Motivation

  2003cited by this paper
• SHOP2: An HTN Planning System

  2003cited by this paper
• From Syllables to Syntax: Multilevel Statistical Learning by 12-Month-Old Infants

  2003influential reference
• Maximizing Learning Progress: An Internal Reward System for Development

  2003cited by this paper
• Perception of action boundaries in patients with frontal lobe damage.

  2003cited by this paper
• A neural model of frontostriatal interactions for behavioural planning and action chunking

  2003cited by this paper
• Recent Advances in Hierarchical Reinforcement Learning

  2003influential reference
• Task switching.

  2003cited by this paper
• Prediction precedes control in motor learning.

  2003cited by this paper
• Encoding strategies dissociate prefrontal activity from working memory demand.

  2003cited by this paper
• Representation of Action Sequence Boundaries by Macaque Prefrontal Cortical Neurons

  2003cited by this paper
• From rule to response: neuronal processes in the premotor and prefrontal cortex.

  2003cited by this paper
• Computational Models of the Amygdala and the Orbitofrontal Cortex: A Hierarchical Reinforcement Learning System for Robotic Control

  2002cited by this paper
• State abstraction for programmable reinforcement learning agents

  2002cited by this paper
• Always Under Construction

  2002cited by this paper
• Actor-critic models of the basal ganglia: new anatomical and computational perspectives

  2002cited by this paper
• Near-Optimal Reinforcement Learning in Polynomial Time

  2002cited by this paper
• The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity.

  2002cited by this paper
• Discovering Hierarchy in Reinforcement Learning with HEXQ

  2002cited by this paper
• Timing and Partial Observability in the Dopamine System

  2002cited by this paper
• Q-Cut - Dynamic Discovery of Sub-goals in Reinforcement Learning

  2002cited by this paper
• Autonomous discovery of temporal abstractions from interaction with an environment

  2002cited by this paper
• Dynamic Variations in Adult Cognitive Microdevelopment

  2002cited by this paper
• PolicyBlocks: An Algorithm for Creating Useful Macro-Actions in Reinforcement Learning

  2002cited by this paper
• Representing task context: proposals based on a connectionist model of action

  2002cited by this paper
• Functional anatomy of the basal ganglia

  2002cited by this paper
• Basal-ganglia 'projections' to the prefrontal cortex of the primate.

  2002cited by this paper
• Intentions and Intentionality: Foundations of Social Cognition

  2001cited by this paper
• Sequential learning in non-human primates.

  2001cited by this paper
• Human brain activity time-locked to perceptual event boundaries

  2001influential reference
• Motor prediction.

  2001cited by this paper
• How infants make sense of intentional action

  2001cited by this paper
• Modeling functions of striatal dopamine modulation in learning and planning.

  2001cited by this paper
• The prefrontal cortex--an update: time is of the essence.

  2001cited by this paper

• An Analysis of Multi-Task Architectures for the Hierarchic Multi-Label Problem of Vehicle Model and Make Classification

  2026cites this paper
• Forging EMPIRE: A data-driven agent-based model for scenario-based generalizability in spatio-temporal human behavior modeling

  2026cites this paper
• Hierarchical Brain-Inspired Deep Learning for Autonomous Decision-Making in Complex Dynamic Environments

  2026cites this paper
• When Should an AI Act? A Human-Centered Model of Scene, Context, and Behavior for Agentic AI Design

  2026cites this paper
• Efficient Learning of Predictive Maps for Flexible Planning

  2026cites this paper
• Hybrid neural-cognitive models reveal how memory shapes human reward learning.

  2026cites this paper
• Hierarchical Reward Design from Language: Enhancing Alignment of Agent Behavior with Human Specifications

  2026cites this paper
• Parsing Reward Processing Substages With Multimodal EEG–fMRI

  2026cites this paper
• Reinforcement Learning in Artificial Intelligence and Neurobiology

  2025cites this paper
• The adaptive value of stubborn goals.

  2025cites this paper
• Cognitive Foundations for Reasoning and Their Manifestation in LLMs

  2025cites this paper
• Hyper-GoalNet: Goal-Conditioned Manipulation Policy Learning with HyperNetworks

  2025cites this paper
• Schemas, reinforcement learning and the medial prefrontal cortex

  2025influential citation
• Spectral envelopes of rhythmic facial movements predict intention and motor cortical representations

  2025cites this paper
• On the origin of the reward function: Exploring the role of conditioned reinforcement and social learning

  2025cites this paper
• Naturally disengaging control to reveal habits

  2025cites this paper
• HiERO: understanding the hierarchy of human behavior enhances reasoning on egocentric videos

  2025cites this paper
• Cognitively-plausible reinforcement learning in epidemiological agent-based simulations

  2025cites this paper
• Natural-Born Intelligence as the Invocation of Emotion = Politics

  2025cites this paper
• Complementary roles for hippocampus, anterior cingulate cortex, and orbitofrontal cortex in composing continuous choice

  2025cites this paper
• It’s Hard to Prepare for Task Novelty: Cueing the Novelty of Upcoming Tasks Does Not Facilitate Task Performance

  2025cites this paper
• Structuring the environment nudges participants toward hierarchical over shortest path planning.

  2025cites this paper
• Hierarchical Visual Policy Learning for Long-Horizon Robot Manipulation in Densely Cluttered Scenes

  2025cites this paper
• Learning to Decompose: Human-Like Subgoal Preferences Emerge in Neural Networks Learning Graph Traversal

  2025cites this paper
• Emergence of Value and Action Codes in Bimodular Spiking Actor–Critic Networks

  2025cites this paper
• Eye and hand coarticulation during problem solving reveals hierarchically organized planning

  2025cites this paper
• DHP: Discrete Hierarchical Planning for Hierarchical Reinforcement Learning Agents

  2025cites this paper
• From Priors to Predictions: Explaining and Visualizing Human Reasoning in a Graph Neural Network Framework

  2025cites this paper
• Goals and the Structure of Experience

  2025cites this paper
• Expert navigators deploy rational complexity–based decision precaching for large-scale real-world planning

  2025cites this paper
• A representational framework for learning and encoding structurally enriched trajectories in complex agent environments

  2025cites this paper
• A taxonomy of spatial navigation in mammals: Insights from computational modeling.

  2025cites this paper
• The devilish details affecting TDRL models in dopamine research.

  2025cites this paper
• DHP: Discrete Hierarchical Planning for Hierarchical Reinforcement Learning Agents

  2025cites this paper
• ARC-NCA: Towards Developmental Solutions to the Abstraction and Reasoning Corpus

  2025cites this paper
• The valuationist model of human agent architecture

  2025cites this paper
• Action chunking as conditional policy compression.

  2025cites this paper
• Discounting and Drug Seeking in Biological Hierarchical Reinforcement Learning

  2025influential citation
• The role of affective states in computational psychiatry

  2025cites this paper
• LLM-Powered Deep Reinforcement Learning for Predictive Cyber Threat Mitigation

  2025cites this paper
• A unified neural representation model for spatial and conceptual computations

  2025cites this paper
• The representation and valuation of subgoals in the human brain during model-based hierarchical behavior

  2025cites this paper
• Instruction-induced modulation of the visual stream during gesture observation.

  2025cites this paper
• ACC representations of reward-driven motivation over hierarchically-organized behavior

  2025cites this paper
• MRSD: Multi-Resolution Skill Discovery for HRL Agents

  2025cites this paper
• Disentangling the Component Processes in Complex Planning Impairments Following Ventromedial Prefrontal Lesions

  2025cites this paper
• ICAL: Continual Learning of Multimodal Agents by Transforming Trajectories into Actionable Insights

  2024cites this paper
• Contextualizing predictive minds.

  2024cites this paper
• The temporal dynamics of metacognitive experiences track rational adaptations in task performance

  2024cites this paper
• Over the Quantum Rainbow: Explaining Hybrid Quantum Reinforcement Learning

  2024cites this paper
• VLM Agents Generate Their Own Memories: Distilling Experience into Embodied Programs of Thought

  2024cites this paper
• Value modulation of self-defeating impulsivity.

  2024cites this paper
• Hierarchical behavior control by a single class of interneurons

  2024cites this paper
• An algorithmic account for how humans efficiently learn, transfer, and compose hierarchically structured decision policies.

  2024influential citation
• The visual statistical learning overcomes scene dissimilarity through an independent clustering process

  2024cites this paper
• Multilevel Graph Reinforcement Learning for Consistent Cognitive Decision-making in Heterogeneous Mixed Autonomy

  2024cites this paper
• Computation noise promotes zero-shot adaptation to uncertainty during decision-making in artificial neural networks

  2024cites this paper
• Hierarchical control over foraging behavior by anterior cingulate cortex.

  2024cites this paper
• Problem-Solving as a Language: A Computational Lens into Human and Monkey Intelligence

  2024cites this paper
• Human foraging strategies flexibly adapt to resource distribution and time constraints

  2024cites this paper
• Identifying resource-rational heuristics for risky choice.

  2024cites this paper
• A language model of problem solving in humans and macaque monkeys.

  2024cites this paper
• Reinforcement learning and meta-decision-making

  2024cites this paper
• Prefrontal cortex drives the flexibility of whole-brain orchestration of cognition

  2024cites this paper
• Human learning of hierarchical graphs.

  2024cites this paper
• Neuroplasticity Meets Artificial Intelligence: A Hippocampus-Inspired Approach to the Stability–Plasticity Dilemma

  2024cites this paper
• Learning environment-specific learning rates

  2024cites this paper
• Bridging flexible goal-directed cognition and consciousness: The Goal-Aligning Representation Internal Manipulation theory

  2024cites this paper
• Environmental complexity modulates information processing and the balance between decision-making systems

  2024cites this paper
• Cognitive, Emotional, and Daily Functioning Domains Involved in Decision-Making among Patients with Mild Cognitive Impairment: A Systematic Review

  2024cites this paper
• A transdisciplinary view on curiosity beyond linguistic humans: animals, infants, and artificial intelligence

  2024cites this paper
• London taxi drivers exploit neighbourhood boundaries for hierarchical route planning

  2024cites this paper
• Exploring the steps of learning: computational modeling of initiatory-actions among individuals with attention-deficit/hyperactivity disorder

  2024cites this paper
• Compositional pretraining improves computational efficiency and matches animal behavior on complex tasks

  2024cites this paper
• Reinforcement Learning with Options and State Representation

  2024cites this paper
• Temporally extended successor feature neural episodic control

  2024cites this paper
• Humans rationally balance detailed and temporally abstract world models

  2024cites this paper
• Modernizing Smart Homes into Cognitive Residences

  2024cites this paper
• A sensory–motor theory of the neocortex

  2024cites this paper
• A feature-specific prediction error model explains dopaminergic heterogeneity

  2024cites this paper
• Human-like decision making for lane change based on the cognitive map and hierarchical reinforcement learning

  2023cites this paper
• Human Learning of Hierarchical Graphs

  2023cites this paper
• Integrating large language models and active inference to understand eye movements in reading and dyslexia

  2023cites this paper
• Coding Dynamics of the Striatal Networks During Learning

  2023cites this paper
• The Relationships Between Neural Activity and In-Game Hitting Performance in Baseball.

  2023cites this paper
• Strategy inference during learning via cognitive activity-based credit assignment models

  2023cites this paper
• The successor representation subserves hierarchical abstraction for goal-directed behavior

  2023cites this paper
• The Situated Assessment Method (SAM2): Establishing individual differences in habitual behavior

  2023cites this paper
• A Hierarchical Robot Learning Framework for Manipulator Reactive Motion Generation via Multi-Agent Reinforcement Learning and Riemannian Motion Policies

  2023cites this paper
• Prediction error in dopamine neurons during associative learning.

  2023cites this paper
• Navigation through the complex world -- the neurophysiology of decision-making processes

  2023cites this paper
• A shared novelty-seeking basis for creativity and curiosity

  2023cites this paper
• Humans adaptively select different computational strategies in different learning environments

  2023cites this paper
• Reinforcement Learning Inspired by Psychology and Neuroscience

  2023cites this paper
• Anxiety as a disorder of uncertainty: implications for understanding maladaptive anxiety, anxious avoidance, and exposure therapy

  2023cites this paper
• Hierarchical behavior control by a single 1 class of interneurons 2

  2023cites this paper
• Having multiple selves helps learning agents explore and adapt in complex changing worlds

  2023cites this paper
• On the convergence of the MLE as an estimator of the learning rate in the Exp3 algorithm

  2023cites this paper
• Adaptive Feature Fusion: Enhancing Generalization in Deep Learning Models

  2023cites this paper
• Exploring the hierarchical structure of human plans via program generation

  2023cites this paper