On the Relationship Between Active Inference and Control as Inference

Active Inference (AIF) is an emerging framework in the brain sciences which suggests that biological agents act to minimise a variational bound on model evidence. Control-as-Inference (CAI) is a framework within reinforcement learning which casts decision making as a variational inference problem. While these frameworks both consider action selection through the lens of variational inference, their relationship remains unclear. Here, we provide a formal comparison between them and demonstrate that the primary difference arises from how value is incorporated into their respective generative models. In the context of this comparison, we highlight several ways in which these frameworks can inform one another.

• Publication year

  2020

• Venue

  International Workshop on Affective Interactions

• Publication date

  2020-06-23

• Fields of study

  Mathematics, Philosophy, Computer Science

• Identifiers
  DOI 10.1007/978-3-030-64919-7_1arXiv 2006.12964
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Agent57: Outperforming the Atari Human Benchmark

  2020cited by this paper
• Evaluating the neurophysiological evidence for predictive processing as a model of perception

  2020cited by this paper
• Reinforcement Learning through Active Inference

  2020cited by this paper
• Deep active inference agents using Monte-Carlo methods

  2020cited by this paper
• Whence the Expected Free Energy?

  2020cited by this paper
• Deep Active Inference as Variational Policy Gradients

  2019cited by this paper
• End-to-End Pixel-Based Deep Active Inference for Body Perception and Action

  2019cited by this paper
• Learning action-oriented models through active inference

  2019cited by this paper
• Scaling Active Inference

  2019cited by this paper
• Neuronal message passing using Mean-field, Bethe, and Marginal approximations

  2019cited by this paper
• Implementing Predictive Processing and Active Inference: Preliminary Steps and Results

  2019cited by this paper
• Efficient Exploration via State Marginal Matching

  2019cited by this paper
• Combining Active Inference and Hierarchical Predictive Coding: A Tutorial Introduction and Case Study

  2019cited by this paper
• Soft Actor-Critic Algorithms and Applications

  2018cited by this paper
• Generalised free energy and active inference

  2018cited by this paper
• Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review

  2018influential reference
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor

  2018cited by this paper
• The modularity of action and perception revisited using control theory and active inference

  2018cited by this paper
• Maximum a Posteriori Policy Optimisation

  2018cited by this paper
• Model-Based Active Exploration

  2018cited by this paper
• Active Inference: A Process Theory

  2017cited by this paper
• Rainbow: Combining Improvements in Deep Reinforcement Learning

  2017cited by this paper
• Mastering the game of Go without human knowledge

  2017cited by this paper
• Deep active inference

  2017cited by this paper
• The graphical brain: Belief propagation and active inference

  2017cited by this paper
• Active inference and robot control: a case study

  2016cited by this paper
• VIME: Variational Information Maximizing Exploration

  2016cited by this paper
• Active inference and epistemic value

  2015cited by this paper
• Continuous control with deep reinforcement learning

  2015cited by this paper
• Radical predictive processing

  2015cited by this paper
• Human-level control through deep reinforcement learning

  2015cited by this paper
• On probabilistic inference approaches to stochastic optimal control

  2013cited by this paper
• Relative entropy and free energy dualities: Connections to Path Integral and KL control

  2012cited by this paper
• Planning as inference.

  2012cited by this paper
• On Stochastic Optimal Control and Reinforcement Learning by Approximate Inference (Extended Abstract)

  2012cited by this paper
• Risk Sensitive Path Integral Control

  2010cited by this paper
• Approximate Inference and Stochastic Optimal Control

  2010cited by this paper
• Modeling Purposeful Adaptive Behavior with the Principle of Maximum Causal Entropy

  2010cited by this paper
• The free-energy principle: a unified brain theory?

  2010cited by this paper
• Reinforcement Learning or Active Inference?

  2009cited by this paper
• Robot trajectory optimization using approximate inference

  2009cited by this paper
• Graphical Models, Exponential Families, and Variational Inference

  2008cited by this paper
• Hierarchical Models in the Brain

  2008cited by this paper
• General duality between optimal control and estimation

  2008cited by this paper
• A free energy principle for the brain.

  2006cited by this paper
• Variational algorithms for approximate Bayesian inference

  2003cited by this paper
• Planning by Probabilistic Inference

  2003cited by this paper
• MASTERING THE GAME

  1998cited by this paper
• Reinforcement Learning: An Introduction

  1998cited by this paper
• Maximum a Posteriori

  1997cited by this paper
• Duality between the discrete-time Kalman filter and LQ control law

  1995cited by this paper

• Pragmatic Curiosity: A Hybrid Learning-Optimization Paradigm via Active Inference

  2026cites this paper
• Maximum Likelihood Reinforcement Learning

  2026cites this paper
• Distributional Active Inference

  2026cites this paper
• EcoNet: Multiagent Planning and Control Of Household Energy Resources Using Active Inference

  2025cites this paper
• Graph Distance as Surprise: Free Energy Minimization in Knowledge Graph Reasoning

  2025cites this paper
• Why is cognitive effort experienced as costly?

  2025cites this paper
• Is the 'Agent' Paradigm a Limiting Framework for Next-Generation Intelligent Systems?

  2025cites this paper
• A coalgebraic perspective on predictive processing

  2025cites this paper
• Simulating Biological Intelligence: Active Inference with Experiment-Informed Generative Model

  2025cites this paper
• Emergence of Goal-Directed Behaviors via Active Inference with Self-Prior

  2025influential citation
• Counteracting uncertainty: exploring the impact of anxiety on updating predictions about environmental states

  2025influential citation
• Active Inference as a Model of Agency

  2024cites this paper
• Limitations of Agents Simulated by Predictive Models

  2024cites this paper
• Free-Gate: Planning, Control And Policy Composition via Free Energy Gating

  2024cites this paper
• Real-World Robot Control Based on Contrastive Deep Active Inference With Demonstrations

  2024cites this paper
• Learning in Hybrid Active Inference Models

  2024cites this paper
• Hybrid Recurrent Models Support Emergent Descriptions for Hierarchical Planning and Control

  2024cites this paper
• Value of Information and Reward Specification in Active Inference and POMDPs

  2024cites this paper
• A Probabilistic Treatment of (PO)MDPs with Multiplicative Reward Structure

  2024cites this paper
• A framework for the use of generative modelling in non-equilibrium statistical mechanics

  2024cites this paper
• A computationally informed distinction of interoception and exteroception.

  2024cites this paper
• When in Doubt, Think Slow: Iterative Reasoning with Latent Imagination

  2024cites this paper
• Deep Hybrid Models: Infer and Plan in a Dynamic World

  2024cites this paper
• A Survey on Cyber-Physical Security of Autonomous Vehicles Using a Context Awareness Method

  2023cites this paper
• The Human Affectome

  2023cites this paper
• World models and predictive coding for cognitive and developmental robotics: frontiers and challenges

  2023cites this paper
• Hierarchical predictive coding in distributed pain circuits

  2023cites this paper
• An analytical model of active inference in the Iterated Prisoner's Dilemma

  2023cites this paper
• On efficient computation in active inference

  2023influential citation
• Life-inspired Interoceptive Artificial Intelligence for Autonomous and Adaptive Agents

  2023cites this paper
• Reward Maximization Through Discrete Active Inference

  2023influential citation
• A Unified View on Solving Objective Mismatch in Model-Based Reinforcement Learning

  2023influential citation
• cpp-AIF: A multi-core C++ implementation of Active Inference for Partially Observable Markov Decision Processes

  2023cites this paper
• Synergizing habits and goals with variational Bayes

  2023cites this paper
• Stochastic Motion Planning as Gaussian Variational Inference: Theory and Algorithms

  2023cites this paper
• pymdp: A Python library for active inference in discrete state spaces

  2022cites this paper
• Towards a Geometry and Analysis for Bayesian Mechanics

  2022cites this paper
• Successor Representation Active Inference

  2022cites this paper
• Hybrid predictive coding: Inferring, fast and slow

  2022cites this paper
• A Worked Example of the Bayesian Mechanics of Classical Objects

  2022cites this paper
• Model Reduction Through Progressive Latent Space Pruning in Deep Active Inference

  2022cites this paper
• Flexible intentions: An Active Inference theory

  2022cites this paper
• On Bayesian mechanics: a physics of and by beliefs

  2022cites this paper
• Efficient search of active inference policy spaces using k-means

  2022cites this paper
• The Free Energy Principle for Perception and Action: A Deep Learning Perspective

  2022cites this paper
• Predictive Coding: Towards a Future of Deep Learning beyond Backpropagation?

  2022cites this paper
• How Active Inference Could Help Revolutionise Robotics

  2022cites this paper
• The free energy principle made simpler but not too simple

  2022cites this paper
• Online reinforcement learning with sparse rewards through an active inference capsule

  2021cites this paper
• Kalman filters as the steady-state solution of gradient descent on variational free energy

  2021cites this paper
• Understanding the origin of information-seeking exploration in probabilistic objectives for control

  2021cites this paper
• Multimodal VAE Active Inference Controller

  2021cites this paper
• Chance-Constrained Active Inference

  2021cites this paper
• Whole brain Probabilistic Generative Model toward Realizing Cognitive Architecture for Developmental Robots

  2021cites this paper
• Active Inference: Demystified and Compared

  2021cites this paper
• Adaptation Through Prediction: Multisensory Active Inference Torque Control

  2021cites this paper
• Active Inference in Robotics and Artificial Agents: Survey and Challenges

  2021cites this paper
• Contrastive Active Inference

  2021cites this paper
• On Solving a Stochastic Shortest-Path Markov Decision Process as Probabilistic Inference

  2021cites this paper
• Towards Stochastic Fault-tolerant Control using Precision Learning and Active Inference

  2021cites this paper
• Active Inference for Stochastic Control

  2021cites this paper
• A Unified View of Algorithms for Path Planning Using Probabilistic Inference on Factor Graphs

  2021cites this paper
• A whole brain probabilistic generative model: Toward realizing cognitive architectures for developmental robots

  2021cites this paper
• Bayesian mechanics of perceptual inference and motor control in the brain

  2020cites this paper
• The Emperor’s New Markov Blankets

  2020cites this paper
• The relationship between dynamic programming and active inference: the discrete, finite-horizon case

  2020cites this paper
• Whence the Expected Free Energy?

  2020cites this paper
• Deep Active Inference and Scene Construction

  2020cites this paper
• Reward Maximisation through Discrete Active Inference

  2020influential citation
• Application of the Free Energy Principle to Estimation and Control

  2019cites this paper
• Active inference: demystified and compared

  2019cites this paper
• 2021 Special Issue on AI and Brain Science: AI-powered Brain Science A whole brain probabilistic generative model: Toward realizing cognitive architectures for developmental robots

  year unknowncites this paper
• D EEP HYBRID MODELS : INFER AND PLAN IN THE REAL WORLD

  year unknowncites this paper