MDL Denoising Revisited

We refine and extend an earlier minimum description length (MDL) denoising criterion for wavelet-based denoising. We start by showing that the denoising problem can be reformulated as a clustering problem, where the goal is to obtain separate clusters for informative and noninformative wavelet coefficients, respectively. This suggests two refinements, adding a code-length for the model index, and extending the model in order to account for subband-dependent coefficient distributions. A third refinement is the derivation of soft thresholding inspired by predictive universal coding with weighted mixtures. We propose a practical method incorporating all three refinements, which is shown to achieve good performance and robustness in denoising both artificial and natural signals.

• Publication year

  2006

• Venue

  IEEE Transactions on Signal Processing

• Publication date

  2006-09-25

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1109/TSP.2009.2021633arXiv cs/0609138
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Minimum Description Length Principle.

  2010influential reference
• A Wavelet Tour of Signal Processing, Third Edition: The Sparse Way

  2008cited by this paper
• Tooth shape reconstruction from ct images using spline Curves

  2007cited by this paper
• The Minimum Description Length Principle (Adaptive Computation and Machine Learning)

  2007cited by this paper
• Model selection by Normalized Maximum Likelihood

  2006cited by this paper
• Minimum Description Length Denoising With Histogram Models

  2006cited by this paper
• Accumulative prediction error and the selection of time series models

  2006cited by this paper
• Information and Complexity in Statistical Modeling

  2006cited by this paper
• An Empirical Study of MDL Model Selection with Infinite Parametric Complexity

  2005cited by this paper
• On the Behavior of MDL Denoising

  2005cited by this paper
• An MDL Framework for Data Clustering

  2005cited by this paper
• A Note on the Applied Use of MDL Approximations

  2004cited by this paper
• A tutorial introduction to the minimum description length principle

  2004cited by this paper
• Robust denoising of electrophoresis and mass spectrometry signals with minimum description length principle

  2004cited by this paper
• Image denoising using scale mixtures of Gaussians in the wavelet domain

  2003cited by this paper
• Lectures on Statistical Modeling Theory

  2002cited by this paper
• Kolmogorov's structure functions and model selection

  2002cited by this paper
• Exact minimax strategies for predictive density estimation, data compression, and model selection

  2002cited by this paper
• Strong optimality of the normalized ML models as universal codes and information in data

  2001cited by this paper
• The consistency of the BIC Markov order estimator

  2000cited by this paper
• Wavelet thresholding via MDL for natural images

  2000cited by this paper
• MDL Denoising

  2000cited by this paper
• The Competitive Complexity Ratio

  2000cited by this paper
• Analysis of Multiresolution Image Denoising Schemes Using Generalized Gaussian and Complexity Priors

  1999cited by this paper
• Scale Mixtures of Gaussians and the Statistics of Natural Images

  1999cited by this paper
• A BAYESIAN DECISION THEORETIC APPROACH TO THE CHOICE OF THRESHOLDING PARAMETER

  1999cited by this paper
• Minimax Description Length for Signal Denoising and Optimized Representation

  1999cited by this paper
• Wavelet thresholding via a Bayesian approach

  1998cited by this paper
• Nonlinear wavelet shrinkage with Bayes rules and Bayes factors

  1998cited by this paper
• A wavelet tour of signal processing

  1998influential reference
• Asymptotic minimax regret for data compression, gambling, and prediction

  1997cited by this paper
• Complexity-regularized image denoising

  1997cited by this paper
• Model selection using wavelet decomposition and applications

  1997cited by this paper
• APPROACHES FOR BAYESIAN VARIABLE SELECTION

  1997cited by this paper
• Adaptive Bayesian Wavelet Shrinkage

  1997cited by this paper
• Fisher information and stochastic complexity

  1996cited by this paper
• The context-tree weighting method: basic properties

  1995cited by this paper
• Adapting to Unknown Smoothness via Wavelet Shrinkage

  1995influential reference
• Filtering random noise from deterministic signals via data compression

  1995cited by this paper
• Wavelet filter evaluation for image compression

  1995cited by this paper
• Ideal spacial adaptation via wavelet shrinkage

  1994influential reference
• Ideal spatial adaptation by wavelet shrinkage

  1994cited by this paper
• Simultaneous noise suppression and signal compression using a library of orthonormal bases and the minimum-description-length criterion

  1994cited by this paper
• Image compression through wavelet transform coding

  1992cited by this paper
• Ten Lectures on Wavelets

  1992cited by this paper
• A Theory for Multiresolution Signal Decomposition: The Wavelet Representation

  1989cited by this paper
• Bayesian Variable Selection in Linear Regression

  1988cited by this paper
• Modeling By Shortest Data Description*

  1978cited by this paper
• Estimating the Dimension of a Model

  1978cited by this paper
• The Contribution of Parameters to Stochastic Complexity

  year unknowncited by this paper
• Submitted to Ieee Transactions on Image Processing Adaptive Wavelet Thresholding for Image Denoising and Compression

  year unknowncited by this paper

• Partial reciprocity-based precoding matrix prediction in FDD massive MIMO with mobility

  2024cites this paper
• Denoising CSEM Data Using Least-Squares Method Based on Mixed Basis of Fourier Series and Legendre Polynomials

  2023cites this paper
• Low-complexity eigenvector prediction-based precoding matrix prediction in massive MIMO with mobility

  2023cites this paper
• Granger Causal Inference in Multivariate Hawkes Processes by Minimum Message Length

  2023cites this paper
• On the selection of predictors by using greedy algorithms and information theoretic criteria

  2023cites this paper
• Change Point Detection and Node Clustering for Time Series of Graphs

  2022cites this paper
• Extending the Use of MDL for High-Dimensional Problems: Variable Selection, Robust Fitting, and Additive Modeling

  2022cites this paper
• Estimation of Information in Parallel Gaussian Channels via Model Order Selection

  2020cites this paper
• Learning to compress and search visual data in large-scale systems

  2019cites this paper
• Adaptive-Size Dictionary Learning Using Information Theoretic Criteria

  2019cites this paper
• High-dimensional penalty selection via minimum description length principle

  2018cites this paper
• Maximum Entropy Expectation-Maximization Algorithm for Fitting Latent-Variable Graphical Models to Multivariate Time Series

  2018cites this paper
• Speech intelligibility and quality: A comparative study of speech enhancement algorithms

  2017cites this paper
• Effects of ageing and Alzheimer disease on haemodynamic response function: a challenge for event-related fMRI

  2017influential citation
• Sparse Graphical Modeling via Stochastic Complexity

  2017cites this paper
• On the number of iterations for the matching pursuit algorithm

  2017cites this paper
• Effects of ageing and Alzheimer disease on haemodynamic response function: a challenge for eventâ•’related fMRI

  2017influential citation
• Minimum Description Length Principle Based Atomic Norm for Synthetic Low-Rank Matrix Recovery

  2016cites this paper
• Research Advances on Dictionary Learning Models, Algorithms and Applications

  2015cites this paper
• Evaluation of Hemodynamic Response Function in Vision and Motor Brain Regions for the Young and Elderly Adults

  2015cites this paper
• Generalized Context Modeling With Multi-Directional Structuring and MDL-Based Model Selection for Heterogeneous Data Compression

  2015cites this paper
• A fixed-point image denoising algorithm with automatic window selection

  2014cites this paper
• Denoising and compression of intracortical signals with a modified MDL criterion

  2014influential citation
• An analysis of CS algorithms efficiency for sparse communication signals reconstruction

  2014cites this paper
• Optimization of MDL-based wavelet denoising for fMRI data analysis

  2014cites this paper
• Effects of aging on BOLD hemodynamic response: Healthy aging versus Alzheimer disease

  2014cites this paper
• A Tutorial on Model Selection

  2014cites this paper
• Vibration signals denoising using minimum description length principle for detecting impulsive signatures

  2014cites this paper
• Normalized maximum likelihood methods for clustering and density estimation

  2013cites this paper
• Probabilistic, Information-Theoretic Models for Etymological Alignment

  2013cites this paper
• A confidence-set approach to signal denoising

  2013cites this paper
• MDL-based joint denoising and compression of intracortical signals

  2012influential citation
• Some New Results on the Estimation of Sinusoids in Noise

  2012cites this paper
• Second generation sparse models

  2011cites this paper
• Variable selection in linear regression: Several approaches based on normalized maximum likelihood

  2011cites this paper
• Short Course : Introduction to Information-Theoretic Modeling

  2011cites this paper
• Denoising Speech Signals for Digital Hearing Aids: A Wavelet Based Approach

  2011cites this paper
• An MDL Framework for Sparse Coding and Dictionary Learning

  2011cites this paper
• Sparse coding and dictionary learning based on the MDL principle

  2010cites this paper
• Model selection by sequentially normalized least squares

  2010cites this paper
• Confidence sets in time-series filtering

  2010cites this paper
• On the use of Kolmogorov structure function for periodogram smoothing

  2010influential citation
• MML Invariant Linear Regression

  2009cites this paper
• A FRAMEWORK FOR MDL CLUSTERING

  2009cites this paper
• Extensions to MDL denoising

  2008cites this paper
• Recent Breakthroughs in Minimum Description Length Learning Workshop

  2008cites this paper
• Shrinkage and Denoising by Minimum Message Length

  2008influential citation
• ON SEQUENTIALLY NORMALIZED MAXIMUM LIKELIHOOD MODELS

  2008cites this paper
• A soft thresholding approach for MDL denoising

  2007cites this paper
• Statistical analysis and modeling for biomolecular structures

  2007cites this paper
• FMRI BASELINE DRIFT ESTIMATION METHOD BY MDL PRINCIPLE

  2007cites this paper