Higher-order representation in AI

Higher-order representations are those that are about other representations. Humans and some other animals form higher-order mental representations concerning representations in our own minds, through the operation of processes of metacognition and introspection. These have been linked with a wide range of mental capacities and attributes, including consciousness. Recent research on large language models (LLMs) has explored their knowledge of their own ‘minds’, sometimes suggesting that these models represent their own inner representational states in activations. This paper surveys this research, arguing that there is some evidence of higher-order representation in LLMs but that substantial empirical and philosophical questions remain unresolved.

• Publication year

  2026

• Venue

  Philosophy and the Mind Sciences

• Publication date

  2026-02-27

• Fields of study

  Not labeled

• Identifiers
  DOI 10.33735/phimisci.2026.12032
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Large Language Models lack essential metacognition for reliable medical reasoning

  2025cited by this paper
• Quantifying uncert-AI-nty: Testing the accuracy of LLMs’ confidence judgments

  2025cited by this paper
• Self-Interpretability: LLMs Can Describe Complex Internal Processes that Drive Their Decisions, and Improve with Training

  2025cited by this paper
• Metacognitive sensitivity: The key to calibrating trust and optimal decision making with AI

  2025cited by this paper
• Metacognition and Uncertainty Communication in Humans and Large Language Models

  2025cited by this paper
• Tell me about yourself: LLMs are aware of their learned behaviors

  2025cited by this paper
• A comprehensive assessment of current methods for measuring metacognition

  2025cited by this paper
• On the attribution of confidence to large language models

  2024cited by this paper
• Metacognition in nonhuman primates: a review of current knowledge

  2024cited by this paper
• Imagining and building wise machines: The centrality of AI metacognition

  2024cited by this paper
• Looking Inward: Language Models Can Learn About Themselves by Introspection

  2024influential reference
• Concepts at the Interface

  2024cited by this paper
• Me, Myself, and AI: The Situational Awareness Dataset (SAD) for LLMs

  2024cited by this paper
• Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback

  2023cited by this paper
• What Forms Could Introspective Systems Take? A Research Programme

  2023cited by this paper
• Metacognition and Confidence: A Review and Synthesis.

  2023cited by this paper
• Optimal metacognitive decision strategies in signal detection theory

  2023cited by this paper
• Consciousness in Artificial Intelligence: Insights from the Science of Consciousness

  2023cited by this paper
• Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs

  2023cited by this paper
• Attention Schema in Neural Agents

  2023cited by this paper
• Role play with large language models

  2023cited by this paper
• Evaluating large language models in theory of mind tasks

  2023cited by this paper
• Model-free metacognition.

  2022cited by this paper
• Chain of Thought Prompting Elicits Reasoning in Large Language Models

  2022cited by this paper
• Large Language Models are Zero-Shot Reasoners

  2022cited by this paper
• Teaching Models to Express Their Uncertainty in Words

  2022cited by this paper
• Language Models (Mostly) Know What They Know

  2022cited by this paper
• Unit Testing for Concepts in Neural Networks

  2022cited by this paper
• Language Models as Agent Models

  2022cited by this paper
• Time-sensitive prefrontal involvement in associating confidence with task performance illustrates metacognitive introspection in monkeys

  2021cited by this paper
• Confidence Tracks Consciousness

  2021cited by this paper
• Metacognition

  2021cited by this paper
• Truthful AI: Developing and governing AI that does not lie

  2021cited by this paper
• Perceptual confidence judgments reflect self-consistency

  2021cited by this paper
• How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering

  2020cited by this paper
• Knowing Ourselves Together: The Cultural Origins of Metacognition

  2020cited by this paper
• Self-Knowledge

  2020cited by this paper
• Deep learning: A philosophical introduction

  2019cited by this paper
• Understanding the Higher-Order Approach to Consciousness.

  2019cited by this paper
• Representation in Cognitive Science

  2018cited by this paper
• PERCEIVING THE WORLD OUTSIDE: HOW TO SOLVE THE DISTALITY PROBLEM FOR INFORMATIONAL TELEOSEMANTICS

  2018cited by this paper
• Transcranial magnetic stimulation to visual cortex induces suboptimal introspection.

  2017cited by this paper
• Superior colliculus neuronal ensemble activity signals optimal rather than subjective confidence

  2017cited by this paper
• HMeta-d: hierarchical Bayesian estimation of metacognitive efficiency from confidence ratings

  2016cited by this paper
• Action-Specific Disruption of Perceptual Confidence

  2015cited by this paper
• Reward Prediction Error Signals are Metarepresentational

  2014cited by this paper
• How to measure metacognition

  2014cited by this paper
• Orbitofrontal cortex is required for optimal waiting based on decision confidence.

  2014cited by this paper
• The Philosophy of Metacognition: Mental Agency and Self-Awareness

  2013cited by this paper
• Direct injection of noise to the visual cortex decreases accuracy but increases decision confidence.

  2012cited by this paper
• Knowing what I want

  2011cited by this paper
• Theta-burst transcranial magnetic stimulation to the prefrontal cortex impairs metacognitive visual awareness

  2010cited by this paper
• Know thyself: metacognitive networks and measures of consciousness.

  2010cited by this paper
• Multiple demonstrations of metacognition in nonhumans: Converging evidence or multiple mechanisms?

  2009cited by this paper
• Metacognition in animals: Trends and challenges.

  2009cited by this paper
• Transfer of Metacognitive Skills and Hint Seeking in Monkeys

  2007cited by this paper
• Higher‐Order Theories of Consciousness

  2007cited by this paper
• Misinformation

  2006cited by this paper
• Type 2 tasks in the theory of signal detectability: Discrimination between correct and incorrect decisions

  2003cited by this paper
• Misrepresenting & malfunctioning

  1995cited by this paper
• The Varieties of Reference.

  1984cited by this paper
• The perceptual reality monitoring theory

  year unknowncited by this paper

• No citing papers are available for this paper.