An Empirical Study of MDL Model Selection with Infinite Parametric Complexity

Parametric complexity is a central concept in MDL model selection. In practice it often turns out to be infinite, even for quite simple models such as the Poisson and Geometric families. In such cases, MDL model selection as based on NML and Bayesian inference based on Jeffreys’ prior can not be used. Several ways to resolve this problem have been proposed. We conduct experiments to compare and evaluate their behaviour on small sample sizes. We find interestingly poor behaviour for the plug-in predictive code; a restricted NML model performs quite well but it is questionable if the results validate its theoretical motivation. The Bayesian model with the improper Jeffreys’ prior is the most dependable.

• Publication year

  2005

• Venue

  arXiv.org

• Publication date

  2005-01-14

• Fields of study

  Mathematics, Computer Science

• Identifiers
  arXiv cs/0501028
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• BAYESIAN THEORY

  2008cited by this paper
• Advances in Minimum Description Length: Theory and Applications

  2005cited by this paper
• Elements of Information Theory

  2005cited by this paper
• Asymptotic Log-Loss of Prequential Maximum Likelihood Codes

  2005cited by this paper
• MDL Denoising

  2000cited by this paper
• The Minimum Description Length Principle in Coding and Modeling

  1998influential reference
• IEEE Transactions on Information Theory

  1998cited by this paper
• Statistical Inference, Occam's Razor, and Statistical Mechanics on the Space of Probability Distributions

  1996cited by this paper
• Fisher information and stochastic complexity

  1996cited by this paper
• Jeffreys' prior is asymptotically least favorable under entropy risk

  1994cited by this paper
• On Predictive Least Squares Principles

  1992cited by this paper
• Information-theoretic asymptotics of Bayes methods

  1990cited by this paper
• Stochastic Complexity in Statistical Inquiry

  1989cited by this paper
• A Predictive Least-Squares Principle

  1986cited by this paper
• Universal coding, information, prediction, and estimation

  1984cited by this paper
• Estimating the Dimension of a Model

  1978cited by this paper
• Modeling By Shortest Data Description*

  1978cited by this paper
• Advances in Minimum Description

  year unknowncited by this paper

• Following the Flattened Leader

  2010influential citation
• Prequential plug-in codes that achieve optimal redundancy rates even if the model is wrong

  2010cites this paper
• Intelligent Algorithms in Ambient and Biomedical Computing

  2006cites this paper
• A First Look at the Minimum Description Length Principle

  2006cites this paper
• MDL Denoising Revisited

  2006cites this paper
• MDL model selection using the ML plug-in code

  2005cites this paper
• MDL model selection using the ML plug-in code

  2005influential citation
• Asymptotic Log-Loss of Prequential Maximum Likelihood Codes

  2005cites this paper