Complete Diagrammatic Axiomatisations of Relative Entropy

Relative entropy is a fundamental class of distances between probability distributions, with widespread applications in probability theory, statistics, and machine learning. In this work, we study relative entropy from a categorical perspective, viewing it as a quantitative enrichment of categories of stochastic matrices. We consider two natural monoidal structures on stochastic matrices, given by the Kronecker product and the direct sum. Our main results are complete axiomatisations of Kullback-Leibler divergence and, more generally, of R\'enyi divergences of arbitrary order, for each such structure. Our axiomatic theories are formulated within the framework of quantitative monoidal algebra, using a graphical language of string diagrams enriched with quantitative equations.

• Publication year

  2026

• Venue

  Unknown venue

• Publication date

  2026-03-04

• Fields of study

  Mathematics, Computer Science

• Identifiers
  arXiv 2603.04530
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Tapes as Stochastic Matrices of String Diagrams

  2026cited by this paper
• Order in Partial Markov Categories

  2025cited by this paper
• Parametric Iteration in Resource Theories

  2025influential reference
• An Algebraic Approach to Moralisation and Triangulation of Probabilistic Graphical Models

  2025cited by this paper
• A Complete Diagrammatic Calculus for Conditional Gaussian Mixtures

  2025cited by this paper
• Lifting Algebraic Reasoning to Generalized Metric Spaces. (Relèvement du raisonnement algébrique aux espaces métriques généralisés)

  2024cited by this paper
• Quantitative Monoidal Algebra: Axiomatising Distance with String Diagrams

  2024influential reference
• A Diagrammatic Algebra for Program Logics

  2024cited by this paper
• A Complete Axiomatisation of Equivalence for Discrete Probabilistic Programming

  2024influential reference
• Graphical Quadratic Algebra

  2024cited by this paper
• Polynomial Lawvere Logic

  2024cited by this paper
• On the Lattice of Program Metrics

  2023cited by this paper
• An Introduction to String Diagrams for Computer Scientists

  2023influential reference
• Pearl's and Jeffrey's Update as Modes of Learning in Probabilistic Programming

  2023cited by this paper
• Propositional Logics for the Lawvere Quantale

  2023cited by this paper
• Absolute continuity, supports and idempotent splitting in categorical probability

  2023cited by this paper
• Discrete equational theories

  2022cited by this paper
• Markov Categories and Entropy

  2022influential reference
• BASIC CONCEPTS OF ENRICHED CATEGORY THEORY

  2022cited by this paper
• On Quantitative Algebraic Higher-Order Theories

  2022cited by this paper
• The Information Loss of a Stochastic Map

  2021cited by this paper
• Learning from What's Right and Learning from What's Wrong

  2021cited by this paper
• Causal inference via string diagram surgery: A diagrammatic approach to interventions and counterfactuals

  2021cited by this paper
• Generative Adversarial Networks

  2021cited by this paper
• Combining Nondeterminism, Probability, and Termination: Equational and Metric Reasoning

  2020cited by this paper
• AUTO-ENCODING VARIATIONAL BAYES

  2020cited by this paper
• Monads and Quantitative Equational Theories for Nondeterminism and Probability

  2020cited by this paper
• A synthetic approach to Markov kernels, conditional independence, and theorems on sufficient statistics

  2019cited by this paper
• Formal verification of higher-order probabilistic programs: reasoning about approximation, convergence, Bayesian inference, and optimization

  2018cited by this paper
• An Algebraic Theory of Markov Processes

  2018cited by this paper
• An Elementary Introduction to Information Geometry

  2018cited by this paper
• Deconstructing Lawvere with distributive laws

  2018cited by this paper
• The Theory of Traces for Systems with Nondeterminism, Probability, and Termination

  2018cited by this paper
• ∞-Categories for the Working Mathematician

  2018cited by this paper
• Picturing Quantum Processes

  2017cited by this paper
• A short characterization of relative entropy

  2017cited by this paper
• A Categorical Characterization of Relative Entropy on Standard Borel Spaces

  2017cited by this paper
• A Complete Quantitative Deduction System for the Bisimilarity Distance on Markov Chains

  2017cited by this paper
• Quantitative Algebraic Reasoning

  2016influential reference
• On Measures of Entropy and Information

  2015cited by this paper
• Distributional Sentence Entailment Using Density Matrices

  2015cited by this paper
• Metric reasoning about λ-terms: The affine case

  2015cited by this paper
• A Bayesian Characterization of Relative Entropy

  2014cited by this paper
• The Algorithmic Foundations of Differential Privacy

  2014cited by this paper
• Beyond Differential Privacy: Composition Theorems and Relational Logic for f-divergences between Probabilistic Programs

  2013cited by this paper
• A tutorial on variational Bayesian inference

  2012cited by this paper
• A Survey of Graphical Languages for Monoidal Categories

  2009influential reference
• A presentation of the category of stochastic matrices

  2009influential reference
• A Mathematical Theory of Communication

  2006cited by this paper
• A behavioural pseudometric for probabilistic transition systems

  2005cited by this paper
• MODELS OF HORN THEORIES

  2004cited by this paper
• The Uniformity Principle on Traced Monoidal Categories

  2003cited by this paper
• The role of relative entropy in quantum information theory

  2001cited by this paper
• Metrics for Labeled Markov Systems

  1999cited by this paper
• Algebraic Theories of Quasivarieties

  1998cited by this paper
• On Information and Sufficiency

  1997cited by this paper
• Information Theory

  1987influential reference
• A functional equation and its application to information theory

  1974cited by this paper
• Measurable solutions of functional equations related to information theory

  1973cited by this paper
• Metric spaces, generalized logic, and closed categories

  1973cited by this paper
• A new theorem of information theory

  1969cited by this paper
• Postulates for the barycentric calculus

  1949cited by this paper

• No citing papers are available for this paper.