Effects of Outliers on the Maximum Correntropy Estimation: A Robustness Analysis

Recently, maximum correntropy criterion (MCC) has been widely and successfully used in robust signal processing and machine learning, in which the correntropy is maximized instead of minimizing the popular mean square error (MSE) to improve the robustness with respect to outliers or impulsive noises. A lot of efforts have been devoted to derive different adaptive algorithms under MCC, but to date, little insight has been gained as to how the MCC solution will be influenced by outliers. In this paper, we investigate this problem and our focus is mainly on the parameter estimation of a simple linear errors-in-variables (EIVs) model with scalar variables. Under some conditions, we derive an upper bound on the absolute value of the estimation error and show that the MCC solution can get very close to the true value of the unknown parameter even with arbitrarily large outliers in both the input and output variables. Illustrative examples are provided to verify and clarify the theory.

• Publication year

  2021

• Venue

  IEEE Transactions on Systems, Man, and Cybernetics: Systems

• Publication date

  2021-06-01

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1109/TSMC.2019.2931403
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Maximum Correntropy Criterion-Based Sparse Subspace Learning for Unsupervised Feature Selection

  2019cited by this paper
• Robust High-Order Manifold Constrained Sparse Principal Component Analysis for Image Representation

  2019cited by this paper
• A New Look at Boundedness of Error Covariance of Kalman Filtering

  2018cited by this paper
• Errors-in-variables methods in system identification

  2018cited by this paper
• Short-Term Wind Speed Forecasting via Stacked Extreme Learning Machine With Generalized Correntropy

  2018cited by this paper
• Towards enhancing stacked extreme learning machine with sparse autoencoder by correntropy

  2017cited by this paper
• Fuzzy Transform Based Compression of Electric Signal Waveforms for Smart Grids

  2017cited by this paper
• Probabilistic Inference-Based Least Squares Support Vector Machine for Modeling Under Noisy Environment

  2016cited by this paper
• Correntropy Maximization via ADMM: Application to Robust Hyperspectral Unmixing

  2016cited by this paper
• Kernel recursive maximum correntropy

  2015cited by this paper
• A variable step-size adaptive algorithm under maximum correntropy criterion

  2015cited by this paper
• Robust Hammerstein Adaptive Filtering under Maximum Correntropy Criterion

  2015cited by this paper
• Generalized Correntropy for Robust Adaptive Filtering

  2015cited by this paper
• Half-Quadratic-Based Iterative Minimization for Robust Sparse Representation

  2014cited by this paper
• Robust Recovery of Corrupted Low-RankMatrix by Implicit Regularizers

  2014cited by this paper
• Steady-State Mean-Square Error Analysis for Adaptive Filtering under the Maximum Correntropy Criterion

  2014cited by this paper
• Convex Combination of Adaptive Filters under the Maximum Correntropy Criterion in Impulsive Interference

  2014cited by this paper
• Correntropy Induced L2 Graph for Robust Subspace Clustering

  2013cited by this paper
• System Parameter Identification: Information Criteria and Algorithms

  2013cited by this paper
• Training extreme learning machine via regularized correntropy criterion

  2013cited by this paper
• Maximum Correntropy Estimation Is a Smoothed MAP Estimation

  2012cited by this paper
• Robust Principal Component Analysis Based on Maximum Correntropy Criterion

  2011cited by this paper
• Maximum Correntropy Criterion for Robust Face Recognition

  2011cited by this paper
• Kernel adaptive filtering with maximum correntropy criterion

  2011cited by this paper
• A loss function for classification based on a robust similarity metric

  2010cited by this paper
• Information Theoretic Learning: Renyi's Entropy and Kernel Perspectives

  2010cited by this paper
• Using Correntropy as a cost function in linear adaptive filters

  2009cited by this paper
• Overview of total least-squares methods

  2007cited by this paper
• Correntropy: Properties and Applications in Non-Gaussian Signal Processing

  2007cited by this paper
• On the analysis of average time complexity of estimation of distribution algorithms

  2007cited by this paper
• A Study on the Global Convergence Time Complexity of Estimation of Distribution Algorithms

  2005cited by this paper
• Robust Regression and Outlier Detection

  2005cited by this paper
• A recursive least M-estimate algorithm for robust adaptive filtering in impulsive noise: fast algorithm and convergence performance analysis

  2004cited by this paper
• On the convergence of a class of estimation of distribution algorithms

  2004cited by this paper
• Canonical Correlation Analysis: An Overview with Application to Learning Methods

  2004cited by this paper
• Application of structured total least squares for system identification

  2004cited by this paper
• On the finite sample breakdown points of redescending M-estimates of location

  2004cited by this paper
• Principal Component Analysis

  2003cited by this paper
• Least absolute deviations estimation for ARCH and GARCH models

  2003cited by this paper
• Fundamentals of Adaptive Filtering

  2002cited by this paper
• Fast least mean M-estimate algorithms for robust adaptive filtering in impulse noise

  2000cited by this paper
• Breakdown points of t-type regression estimators

  2000cited by this paper
• Signal processing with alpha-stable distributions and applications

  1995cited by this paper
• Global total least squares modeling of multivariable time series

  1995cited by this paper
• Signal processing with fractional lower order moments: stable processes and their applications

  1993cited by this paper
• The Breakdown Point of Simultaneous General M Estimates of Regression and Scale

  1991cited by this paper
• Asymptotics for Least Absolute Deviation Regression Estimators

  1991cited by this paper
• Adaptive filter theory (2nd ed.)

  1991cited by this paper
• A unifying theorem for linear and total linear least squares

  1990cited by this paper
• HIGH BREAKDOWN-POINT AND HIGH EFFICIENCY ROBUST ESTIMATES FOR REGRESSION

  1987cited by this paper
• Least absolute deviations estimation for the censored regression model

  1984cited by this paper
• Finite Sample Breakdown of $M$- and $P$-Estimators

  1984cited by this paper
• Convergence of a fixed point algorithm

  1983cited by this paper
• An analysis of the total least squares problem

  1980cited by this paper
• The Kalman filter: A robust estimator for some classes of linear quadratic problems

  1976cited by this paper
• > Replace This Line with Your Paper Identification Number (double-click Here to Edit) <

  year unknowncited by this paper

• Complex-valued mixture generalized modified Blake–Zisserman algorithm for widely linear active noise control system

  2026cites this paper
• Adaptive Ensemble Control for Stochastic Systems With Mixed Asymmetric Laplace Noises

  2026cites this paper
• Adaptive Unscented Kalman Filter under Minimum Error Entropy with Fiducial Points for Non-Gaussian Systems

  2026cites this paper
• Spline Prioritization Optimization Lncosh Adaptive Filter for Heavy-Tailed Non-Gaussian Noise

  2025cites this paper
• Robust Decorrelated Stochastic Configuration Networks Ensemble via Weighted Negative Correlation Learning

  2025cites this paper
• Robust Volterra Filter for Nonlinear Censored Regression and Its Applications to Nonlinear Acoustic Echo Cancellation

  2025cites this paper
• Fixed-point fully adaptive interpolated Volterra filter under recursive maximum correntropy

  2025cites this paper
• One Class Restricted Kernel Machines

  2025cites this paper
• Privacy-Preserving Distributed Entropy Filtering for Microgrids With Innovation Decomposition

  2025cites this paper
• Robust Sparse Bayesian Learning Based on Minimum Error Entropy for Noisy High-Dimensional Brain Activity Decoding

  2025cites this paper
• Robust Fault-Aware Extreme Learning Machine Based on Maximum Correntropy

  2025cites this paper
• Incomplete Multi-View Clustering via Correntropy and Complement Consensus Learning

  2024cites this paper
• A Proportionate Maximum Total Complex Correntropy Algorithm for Sparse Systems

  2024cites this paper
• A Novel DOA Estimation Algorithm Based on Robust Mixed Fractional Lower-Order Correntropy in Impulsive Noise

  2024cites this paper
• Diffusion Constrained Adaptive Filtering Algorithm Based on Half-Quadratic Criterion for System Identification

  2024cites this paper
• Sparse Unmixing in the Presence of Mixed Noise Using ℓ0-Norm Constraint and Log-Cosh Loss

  2024cites this paper
• Maximum entropy intrinsic learning for spiking networks towards embodied neuromorphic vision

  2024cites this paper
• Adaptive Unscented Kalman Filter under Minimum Error Entropy with Fiducial Points for Non-Gaussian Systems

  2023cites this paper
• SNIB: Improving Spike-Based Machine Learning Using Nonlinear Information Bottleneck

  2023cites this paper
• Generalized Modified Blake–Zisserman Robust Sparse Adaptive Filters

  2023cites this paper
• Multikernel correntropy based robust least squares one-class support vector machine

  2023cites this paper
• Robust Generalized Maximum Blake–Zisserman Total Correntropy Adaptive Filter for Generalized Gaussian Noise and Noisy Input

  2023cites this paper
• Multi-Kernel Correntropy Regression: Robustness, Optimality, and Application on Magnetometer Calibration

  2023cites this paper
• Multikernel Correntropy-Based Orientation Estimation of IMUs: Gradient Descent Methods

  2023cites this paper
• A class of adaptive filtering algorithms based on improper complex correntropy

  2023cites this paper
• Heterogeneous Ensemble-Based Spike-Driven Few-Shot Online Learning

  2022cites this paper
• Robust Spike-Based Continual Meta-Learning Improved by Restricted Minimum Error Entropy Criterion

  2022cites this paper
• Mixture correntropy based robust multi-view K-means clustering

  2022cites this paper
• Adaptive weighted robust iterative closest point

  2022cites this paper
• Error Loss Networks

  2021cites this paper
• Complex Shannon Entropy Based Learning Algorithm and Its Applications

  2021cites this paper
• On the resilience of a class of Correntropy-based state estimators

  2020cites this paper
• Broad Learning System Based on Maximum Correntropy Criterion

  2019cites this paper