Glivenko-Cantelli for $f$-divergence

We extend the celebrated Glivenko-Cantelli theorem, sometimes called the fundamental theorem of statistics, from its standard setting of total variation distance to all $f$-divergences. A key obstacle in this endeavor is to define $f$-divergence on a subcollection of a $\sigma$-algebra that forms a $\pi$-system but not a $\sigma$-subalgebra. This is a side contribution of our work. We will show that this notion of $f$-divergence on the $\pi$-system of rays preserves nearly all known properties of standard $f$-divergence, yields a novel integral representation of the Kolmogorov-Smirnov distance, and has a Glivenko-Cantelli theorem. We will also discuss the prospects of a Vapnik-Chervonenkis theory for $f$-divergence.

• Publication year

  2025

• Venue

  Unknown venue

• Publication date

  2025-03-21

• Fields of study

  Mathematics, Computer Science

• Identifiers
  arXiv 2503.17355
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Glivenko-Cantelli for f-divergence

  2025cited by this paper
• Survey of Distances between the Most Popular Distributions

  2023cited by this paper
• Statistical Learning Theory

  2021cited by this paper
• Geometric Approximation Theory

  2021cited by this paper
• AUTO-ENCODING VARIATIONAL BAYES

  2020cited by this paper
• Measure theory

  2019cited by this paper
• On the rate of convergence of empirical measure in $\infty $-Wasserstein distance for unbounded density function

  2018cited by this paper
• GENERATIVE ADVERSARIAL NETS

  2018cited by this paper
• Non Additive Measure And Integral

  2016cited by this paper
• On the f-divergence for non-additive measures

  2016cited by this paper
• Rényi Divergence Variational Inference

  2016cited by this paper
• A Glivenko–Cantelli theorem for almost additive functions on lattices

  2016cited by this paper
• Introduction To Measure And Probability

  2016cited by this paper
• On Measures of Entropy and Information

  2015cited by this paper
• $f$ -Divergence Inequalities

  2015influential reference
• Geometric Modeling in Probability and Statistics

  2014cited by this paper
• Understanding Machine Learning - From Theory to Algorithms

  2014cited by this paper
• A note on derivatives of functions with respect to fuzzy measures

  2013cited by this paper
• Functional analysis for probability and stochastic processes

  2013cited by this paper
• Almost Sure Convergence of Random Variables

  2011cited by this paper
• A Lower Bound on Arbitrary $f$--Divergences in Terms of the Total Variation

  2009cited by this paper
• Empirical Processes with Applications to Statistics

  2009influential reference
• Introduction to Nonparametric Estimation

  2008cited by this paper
• Measures, Integrals and Martingales

  2007cited by this paper
• Distances defined by Choquet integral

  2007cited by this paper
• Pattern Recognition and Machine Learning

  2006cited by this paper
• On Pinsker's and Vajda's Type Inequalities for Csiszár's $f$ -Divergences

  2006cited by this paper
• Quantitative Concentration Inequalities for Empirical Measures on Non-compact Spaces

  2005cited by this paper
• Stability and Generalization

  2002cited by this paper
• Some inequalities for information divergence and related measures of discrimination

  2000influential reference
• Local Radon-Nikodym Derivatives of Set Functions

  1997cited by this paper
• Divergence measures based on the Shannon entropy

  1991cited by this paper
• Monographs on statistics and applied probability

  1990cited by this paper
• Applied Analysis

  1986influential reference
• Asymptotic Methods In Statistical Decision Theory

  1986influential reference
• A differential geometric approach to statistical inference on the basis of contrast functionals

  1985cited by this paper
• Real Analysis: Modern Techniques and Their Applications

  1984cited by this paper
• Chervonenkis: On the uniform convergence of relative frequencies of events to their probabilities

  1971influential reference
• Note on discrimination information and variation (Corresp.)

  1970influential reference
• A General Class of Coefficients of Divergence of One Distribution from Another

  1966cited by this paper
• Markov Processes and the H -Theorem

  1963cited by this paper
• Theory of capacities

  1954cited by this paper
• A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations

  1952cited by this paper
• On the Criterion that a Given System of Deviations from the Probable in the Case of a Correlated System of Variables is Such that it Can be Reasonably Supposed to have Arisen from Random Sampling

  1900cited by this paper
• Theory of Probability

  year unknowncited by this paper

• No citing papers are available for this paper.