An analysis of entropy estimators for blind source separation

An extensive analysis of a non-parametric, information-theoretic method for instantaneous blind source separation (BSS) is presented. As a result a modified stochastic information gradient estimator is proposed to reduce the computational complexity and to allow the separation of sub-Gaussian sources. Interestingly, the modification enables the method to simultaneously exploit spatial and spectral diversity of the sources. Consequently, the new algorithm is able to separate i.i.d, sources, which requires higher-order spatial statistics, and it is also able to separate temporally correlated Gaussian sources, which requires temporal statistics. Three reasons are given why Renyi's entropy estimators for Information-Theoretic Learning (ITL), on which the proposed method is based, is to be preferred over Shannon's entropy estimators for ITL. Also contained herein is an extensive comparison of the proposed method with JADE, Infomax, Comon's MI, FastICA, and a non-parametric, information-theoretic method that is based on Shannon's entropy. Performance comparisons are shown as a function of the data length, source kurtosis, number of sources, and stationarity/correlation of the sources.

• Publication year

  2006

• Venue

  Signal Processing

• Publication date

  2006-01-01

• Fields of study

  Mathematics, Computer Science, Engineering

• Identifiers
  DOI 10.1016/j.sigpro.2005.04.015
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Probability Theory

  2019cited by this paper
• A Mathematical Theory of Communication

  2006cited by this paper
• Elements of Information Theory

  2005cited by this paper
• FAST ALGORITHM FOR ESTIMATING MUTUAL INFORMATION, ENTROPIES AND SCORE FUNCTIONS

  2003cited by this paper
• Online entropy manipulation: stochastic information gradient

  2003influential reference
• Blind source separation using Renyi's -marginal entropies

  2002influential reference
• Blind source separation of time-varying instantaneous mixtures using an on-line algorithm

  2002cited by this paper
• [Expression of fusion proteins in beta(2)GP I gene-transfected HEp-2 cells and its clinical application].

  2002cited by this paper
• Generalized information potential criterion for adaptive system training

  2002influential reference
• CONTRAST FUNCTIONS FOR BLIND SEPARATION AND DECONVOLUTION OF SOURCES

  2001cited by this paper
• Blind source separation using Renyi's mutual information

  2001cited by this paper
• Learning from Examples with Information Theoretic Criteria

  2000cited by this paper
• Fast and robust fixed-point algorithms for independent component analysis

  1999influential reference
• SURVEY OF INDEPENDENT COMPONENT ANALYSIS

  1999cited by this paper
• Efficiency of high-order moment estimates

  1998cited by this paper
• Blind signal separation: statistical principles

  1998cited by this paper
• Infomax and maximum likelihood for blind source separation

  1997cited by this paper
• Adaptive Online Learning Algorithms for Blind Separation: Maximum Entropy and Minimum Mutual Information

  1997cited by this paper
• Maximum Likelihood Blind Source Separation: A Context-Sensitive Generalization of ICA

  1996cited by this paper
• Neural Learning in Structured Parameter Spaces - Natural Riemannian Gradient

  1996cited by this paper
• Equivariant adaptive source separation

  1996cited by this paper
• An Information-Maximization Approach to Blind Separation and Blind Deconvolution

  1995influential reference
• Empirical Entropy Manipulation for Real-World Problems

  1995cited by this paper
• Independent component analysis, A new concept?

  1994cited by this paper
• Blind beamforming for non-gaussian signals

  1993influential reference
• Signal processing

  1980cited by this paper
• On Estimation of a Probability Density Function and Mode

  1962cited by this paper
• Article in Press

  year unknowncited by this paper

• A Novel Extension of Entropy Measures in Quantifying Financial Markets Uncertainty:

  2025cites this paper
• Local Shannon, Rényi, and Tsallis Entropy for Useful Content Extraction from Choi-Williams and Zhao-Atlas-Marks Time-Frequency Distributions

  2022cites this paper
• Basic tools for vibration analysis with applications to predictive maintenance of rotating machines: an overview

  2021cites this paper
• On Change Detection in the Complexity of the Time Series with Multiscale Renyi Entropy Processing

  2020cites this paper
• A Feature Selection and Extraction Method from Time-Frequency Images

  2020influential citation
• A novel approach to extracting useful information from noisy TFDs using 2D local entropy measures

  2020cites this paper
• VIBROCHANGE—a development system for condition monitoring based on advanced techniques of signal processing

  2019cites this paper
• Big Data Viewpoint On Channel Information Measures Based on ACE Algorithm

  2018cites this paper
• Modified GSO Based Kurtosis Maximization Criterion for BSS in Hindi Speech Processing System Help of LDA

  2018cites this paper
• SEPARATION OF HINDI SOURCE SIGNAL IN BSS USING GDA-AGSO APPROACH

  2018cites this paper
• Shannon Shakes Hands with Chernoff: Big Data Viewpoint On Channel Information Measures

  2017cites this paper
• A Comprehensive Learning-Based Model for Power Load Forecasting in Smart Grid

  2017cites this paper
• An Introduction to Information Theoretic Learning, Part II: Applications

  2016cites this paper
• On the change detection methods with sensitivity at variance of the processed signal

  2016cites this paper
• An EEMD-ICA Approach to Enhancing Artifact Rejection for Noisy Multivariate Neural Data

  2016cites this paper
• Artifact Removal in EEG Recordings

  2016cites this paper
• Bio-Inspired and Information-Theoretic Signal Processing

  2015cites this paper
• Discriminative Kernel Feature Extraction and Learning for Object Recognition and Detection

  2015cites this paper
• Feature Extraction and Learning Using Context Cue and Rényi Entropy Based Mutual Information

  2015cites this paper
• Spatial separation of sound sources

  2014cites this paper
• Blind separation of sound sources from the principle of least spatial entropy

  2014cites this paper
• Enhanced Automatic Wavelet Independent Component Analysis for Electroencephalographic Artifact Removal

  2014cites this paper
• On change detection for improved reliability

  2013cites this paper
• Automatic Artifact Rejection From Multichannel Scalp EEG by Wavelet ICA

  2012cites this paper
• Speaker independent robust phoneme recognition using Higher-Order statistics and entropic-based features in adverse environments

  2011cites this paper
• A linear discriminant analysis method based on mutual information maximization

  2011influential citation
• Independent component analysis for harmonic source estimation from piecewise constant parameter mixed measurements

  2011cites this paper
• A new algorithm of Infomax for small numbers of sound signal separation

  2010cites this paper
• A computational framework to support the automated analysis of routine electroencephalographic data

  2010influential citation
• The Equivalency of Minimum Error Entropy Criterion and Minimum Dispersion Criterion for Symmetric Stable Signal Processing

  2010cites this paper
• An Information Theoretic Linear Discriminant Analysis Method

  2010cites this paper
• Efficient Independent Component Analysis on a GPU

  2010cites this paper
• Self-Organizing ITL Principles for Unsupervised Learning

  2010cites this paper
• Correntropy for Random Processes: Properties and Applications in Signal Processing

  2010cites this paper
• Studies of information quantities and information geometry of higher order cumulant spaces

  2010cites this paper
• WAVELET ENTROPY AND POWER OF AE SIGNALS AS TOOLS TO EVALUATE DAMAGE IN COATINGS SUBMITTED TO SCRATCH TEST

  2009cites this paper
• Mutual information based data association

  2009cites this paper
• Mutual information strategies for sensor registration

  2009cites this paper
• The design and analysis of information processing systems ] From Linear Adaptive Filtering to Nonlinear Information Processing

  2009cites this paper
• The International Journal on Advances in Intelligent Systems is Published by IARIA

  2009cites this paper
• Mutual information based on Renyi's entropy feature selection

  2009cites this paper
• An Expectation–Maximization Method for Spatio–Temporal Blind Source Separation Using an AR-MOG Source Model

  2008influential citation
• On the Risk of Using RÉnyi's Entropy for Blind Source Separation

  2008cites this paper
• An overview of Renyi Entropy and some potential applications

  2008cites this paper
• Generalized Negentropy for Statistical Dependence

  2007cites this paper
• Is the General Form of Renyi's Entropy a Contrast for Source Separation?

  2007cites this paper
• Robust Independent Component Analysis Using Quadratic Negentropy

  2007cites this paper
• Fetal cardiac signal extraction from magnetocardiographic data using a probabilistic algorithm

  2007cites this paper
• Performance comparison of six independent components analysis algorithms for fetal signal extraction from real fMCG data

  2007influential citation
• From linear adaptive filtering to nonlinear information processing - The design and analysis of information processing systems

  2006cites this paper
• Feature extraction using information-theoretic learning

  2006cites this paper
• Performance comparison of independent component analysis algorithms for fetal cardiac signal reconstruction: a study on synthetic fMCG data

  2006cites this paper
• An EM Method for Spatio-temporal Blind Source Separation Using an AR-MOG Source Model

  2006cites this paper
• Information Theoretic Learning

  2005cites this paper
• > Replace This Line with Your Paper Identification Number (double-click Here to Edit) <

  year unknowncites this paper