On the Estimation of Tsallis Entropy and a Novel Information Measure Based on Its Properties

This letter explores a plug-in estimator of second-order Tsallis entropy based on Kernel Density Estimation (KDE) and its implicit regularization process. First, it is shown that the expected value of the estimator corresponds to the entropy of an Additive White Gaussian Noise (AWGN) model. Then, we prove various relevant properties of the Tsallis entropy: It is monotonically non-decreasing under random variables addition, its derivative with respect to the Gaussian noise power is monotonically non-increasing, and it is concave in the additive noise power. From these, we derive an information metric that provides an alternative to the strategy of regularization.

• Publication year

  2023

• Venue

  IEEE Signal Processing Letters

• Publication date

  Unknown publication date

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1109/LSP.2023.3291308
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Performance analysis of entropy thresholding for successful image segmentation

  2022cited by this paper
• Entropy-Based Non-Data-Aided SNR Estimation

  2019cited by this paper
• A Proof of de Bruijn Identity based on Generalized Price’s Theorem

  2019cited by this paper
• Entropy-based covariance determinant estimation

  2017cited by this paper
• Table Of Integrals Series And Products

  2016cited by this paper
• Monotonicity of Entropy and Fisher Information: A Quick Proof via Maximal Correlation

  2016cited by this paper
• Efficient ISAR autofocus via minimization of Tsallis Entropy

  2016cited by this paper
• On Rényi Entropy Power Inequalities

  2016cited by this paper
• On Measures of Entropy and Information

  2015cited by this paper
• Matrix Analysis, 2nd Ed

  2012cited by this paper
• Rényi Divergence and Kullback-Leibler Divergence

  2012cited by this paper
• Information geometry and its applications

  2012cited by this paper
• Information Theoretic Learning: Renyi's Entropy and Kernel Perspectives

  2010cited by this paper
• Universal Estimation of Information Measures for Analog Sources

  2009cited by this paper
• Probability and Random Processes

  2007cited by this paper
• Elements of Information Theory

  2005cited by this paper
• Solution of Shannon's problem on the monotonicity of entropy

  2004cited by this paper
• Image thresholding using Tsallis entropy

  2004cited by this paper
• An error-entropy minimization algorithm for supervised training of nonlinear adaptive systems

  2002cited by this paper
• A comparison of SNR estimation techniques for the AWGN channel

  2000cited by this paper
• Estimation of entropy and other functionals of a multivariate density

  1989cited by this paper
• Possible generalization of Boltzmann-Gibbs statistics

  1988cited by this paper
• Density Estimation for Statistics and Data Analysis

  1987cited by this paper
• Density Estimation for Statistics and Data Analysis

  1987cited by this paper
• A note on the estimation of the integral of ⨍2(x)

  1984cited by this paper
• Detection, Estimation, and Modulation Theory, Part I

  1968cited by this paper
• On Estimation of a Probability Density Function and Mode

  1962cited by this paper
• Some Inequalities Satisfied by the Quantities of Information of Fisher and Shannon

  1959cited by this paper

• Scalable and reliable deep transfer learning for intelligent fault detection via multi-scale neural processes embedded with prior knowledge

  2025cites this paper
• Estimation of Tsallis entropy for exponentially distributed several populations

  2024cites this paper
• On the Quantum Analysis by Using Semantic Integration and Covert Communication for Next-Generation Networks

  2024cites this paper
• Enhancing Learning in Fine-Tuned Transfer Learning for Rotating Machinery via Negative Transfer Mitigation

  2024cites this paper