Strong Asymptotic Assertions for Discrete MDL in Regression and Classification

We study the properties of the MDL (or maximum penalized complexity) estimator for Regression and Classification, where the underlying model class is countable. We show in particular a finite bound on the Hellinger losses under the only assumption that there is a “true” model contained in the class. This implies almost sure convergence of the predictive distribution to the true one at a fast rate. It corresponds to Solomono’s central theorem of universal induction, however with a bound that is exponentially larger.

• Publication year

  2005

• Venue

  arXiv.org

• Publication date

  2005-01-31

• Fields of study

  Mathematics, Computer Science

• Identifiers
  arXiv math/0502315
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Convergence of Discrete MDL for Sequential Prediction

  2004cited by this paper
• On the Convergence Speed of MDL Predictions for Bernoulli Sequences

  2004cited by this paper
• Convergence and Loss Bounds for Bayesian Sequence Prediction

  2003cited by this paper
• Optimality of universal Bayesian prediction for general loss and alphabet

  2003cited by this paper
• Convergence rates of posterior distributions

  2000cited by this paper
• Minimum description length induction, Bayesianism, and Kolmogorov complexity

  1999cited by this paper
• The Minimum Description Length Principle in Coding and Modeling

  1998cited by this paper
• Fisher information and stochastic complexity

  1996cited by this paper
• Minimum complexity density estimation

  1991cited by this paper
• Information-theoretic asymptotics of Bayes methods

  1990cited by this paper
• Asymptotics In Statistics

  1990cited by this paper
• Complexity-based induction systems: Comparisons and convergence theorems

  1978cited by this paper
• Mathematical Statistics

  1944cited by this paper
• and as an in

  year unknowncited by this paper

• Short Biography

  2016cites this paper
• Maximizing Fun by Creating Data with Easily Reducible Subjective Complexity

  2013cites this paper
• Intrinsically Motivated Learning in Natural and Artificial Systems

  2013cites this paper
• Formal Theory of Creativity, Fun, and Intrinsic Motivation (1990–2010)

  2010cites this paper
• Driven by Compression Progress: A Simple Principle Explains Essential Aspects of Subjective Beauty, Novelty, Surprise, Interestingness, Attention, Curiosity, Creativity, Art, Science, Music, Jokes

  2008cites this paper
• MDL convergence speed for Bernoulli sequences

  2006cites this paper
• Asymptotics of discrete MDL for online prediction

  2005cites this paper