A Behavioural and Representational Evaluation of Goal-Directedness in Language Model Agents

Understanding an agent's goals helps explain and predict its behaviour, yet there is no established methodology for reliably attributing goals to agentic systems. We propose a framework for evaluating goal-directedness that integrates behavioural evaluation with interpretability-based analyses of models'internal representations. As a case study, we examine an LLM agent navigating a 2D grid world toward a goal state. Behaviourally, we evaluate the agent against an optimal policy across varying grid sizes, obstacle densities, and goal structures, finding that performance scales with task difficulty while remaining robust to difficulty-preserving transformations and complex goal structures. We then use probing methods to decode the agent's internal representations of the environment state and its multi-step action plans. We find that the LLM agent non-linearly encodes a coarse spatial map of the environment, preserving approximate task-relevant cues about its position and the goal location; that its actions are broadly consistent with these internal representations; and that reasoning reorganises them, shifting from broader environment structural cues toward information supporting immediate action selection. Our findings support the view that introspective examination is required beyond behavioural evaluations to characterise how agents represent and pursue their objectives.

• Publication year

  2026

• Venue

  Unknown venue

• Publication date

  2026-02-09

• Fields of study

  Computer Science

• Identifiers
  arXiv 2602.08964
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Auditing Games for Sandbagging

  2025cited by this paper
• Goal-Directedness is in the Eye of the Beholder

  2025cited by this paper
• Evaluating the Goal-Directedness of Large Language Models

  2025cited by this paper
• Interpreting Emergent Planning in Model-Free Reinforcement Learning

  2025cited by this paper
• Auditing language models for hidden objectives

  2025cited by this paper
• Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs

  2025cited by this paper
• Emergent Response Planning in LLMs

  2025cited by this paper
• Propositional Interpretability in Artificial Intelligence

  2025cited by this paper
• Spilling the Beans: Teaching LLMs to Self-Report Their Hidden Objectives

  2025cited by this paper
• Lessons from a Chimp: AI "Scheming" and the Quest for Ape Language

  2025cited by this paper
• AgentMisalignment: Measuring the Propensity for Misaligned Behaviour in LLM-Based Agents

  2025cited by this paper
• The Limits of Predicting Agents from Behaviour

  2025influential reference
• The Strawberry Problem: Emergence of Character-level Understanding in Tokenized Language Models

  2025cited by this paper
• AI Sandbagging: Language Models can Strategically Underperform on Evaluations

  2024cited by this paper
• Do LLMs Build World Representations? Probing Through the Lens of State Abstraction

  2024cited by this paper
• The Reasons that Agents Act: Intention and Instrumental Goals

  2024cited by this paper
• Towards Measuring Goal-Directedness in AI Systems

  2024cited by this paper
• CUTE: Measuring LLMs’ Understanding of Their Tokens

  2024cited by this paper
• Planning in a recurrent neural network that plays Sokoban

  2024cited by this paper
• Unlocking the Future: Exploring Look-Ahead Planning Mechanistic Interpretability in Large Language Models

  2024cited by this paper
• The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets

  2023cited by this paper
• Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Tasks

  2023cited by this paper
• Emergent Linear Representations in World Models of Self-Supervised Sequence Models

  2023cited by this paper
• Language Models Represent Space and Time

  2023cited by this paper
• Future Lens: Anticipating Subsequent Tokens from a Single Hidden State

  2023cited by this paper
• Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task

  2022cited by this paper
• The alignment problem from a deep learning perspective

  2022cited by this paper
• Implicit Representations of Meaning in Neural Language Models

  2021cited by this paper
• Risks from Learned Optimization in Advanced Machine Learning Systems

  2019cited by this paper
• A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress

  2018cited by this paper
• Inverse Reward Design

  2017cited by this paper
• Cooperative Inverse Reinforcement Learning

  2016cited by this paper
• Action understanding as inverse planning.

  2009cited by this paper
• Maximum Entropy Inverse Reinforcement Learning

  2008influential reference
• Theory of Games and Economic Behavior

  2006cited by this paper
• Apprenticeship learning via inverse reinforcement learning

  2004cited by this paper
• Radical interpretation

  2004cited by this paper
• Algorithms for Inverse Reinforcement Learning

  2000cited by this paper
• A Door Opening

  1997cited by this paper
• A Neural Substrate of Prediction and Reward

  1997cited by this paper
• Dynamic Programming

  1993cited by this paper
• The Interpretation of Texts, People and Other Artifacts

  1990cited by this paper
• IMPORTANT

  1950cited by this paper
• Samuelson's Foundations: Its Mathematics

  1948cited by this paper
• Cognitive maps in rats and men.

  1948cited by this paper

• No citing papers are available for this paper.