Comparing Measures of Sparsity

Sparsity of representations of signals has been shown to be a key concept of fundamental importance in fields such as blind source separation, compression, sampling and signal analysis. The aim of this paper is to compare several commonly-used sparsity measures based on intuitive attributes. Intuitively, a sparse representation is one in which a small number of coefficients contain a large proportion of the energy. In this paper, six properties are discussed: (Robin Hood, Scaling, Rising Tide, Cloning, Bill Gates, and Babies), each of which a sparsity measure should have. The main contributions of this paper are the proofs and the associated summary table which classify commonly-used sparsity measures based on whether or not they satisfy these six propositions. Only two of these measures satisfy all six: the pq-mean with p les 1, q > 1 and the Gini index.

• Publication year

  2008

• Venue

  IEEE Transactions on Information Theory

• Publication date

  2008-11-28

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1109/TIT.2009.2027527arXiv 0811.4706
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Robust Face Recognition via Sparse Representation

  2009cited by this paper
• The application of joint sparsity and total variation minimization algorithms to a real-life art restoration problem

  2009cited by this paper
• Reduce and Boost: Recovering Arbitrary Sets of Jointly Sparse Vectors

  2008cited by this paper
• Sparse approximations for joint source-channel coding

  2008cited by this paper
• Sparse Sampling of Signal Innovations

  2008cited by this paper
• Sparse Representation for Color Image Restoration

  2008cited by this paper
• A Frame Construction and a Universal Distortion Bound for Sparse Representations

  2008cited by this paper
• On the Uniqueness of Nonnegative Sparse Solutions to Underdetermined Systems of Equations

  2008cited by this paper
• Sparse and Redundant Modeling of Image Content Using an Image-Signature-Dictionary

  2008cited by this paper
• Demo: Robust face recognition via sparse representation

  2008cited by this paper
• Compressive Sampling and Lossy Compression

  2008cited by this paper
• Stable Recovery of Sparse Signals Via Regularized Minimization

  2008cited by this paper
• Sparse Registration for Three-Dimensional Stress Echocardiography

  2008cited by this paper
• Erratum to "Axiomatic characterization of the Gini coefficient and Lorenz curve orderings" [J. Econ. Theory 101 (2001) 115-132]

  2008cited by this paper
• Sparse Pseudo Inverse of the Discrete Plane Wave Transform

  2008cited by this paper
• Fast Solution of (cid:0) (cid:0) -Norm Minimization Problems When the Solution May Be Sparse

  2008cited by this paper
• Comparing measures of sparsity

  2008cited by this paper
• Enhancing Sparsity by Reweighted ℓ1 Minimization

  2007cited by this paper
• Fast Sparse Approximation for Least Squares Support Vector Machine

  2007cited by this paper
• Blind Speech Separation

  2007cited by this paper
• Estimation of Rapidly Time-Varying Sparse Channels

  2007cited by this paper
• Maximizing Sparsity of Wavelet Representations via Parameterized Lifting

  2007cited by this paper
• Convolutive Blind Source Separation in the Frequency Domain Based on Sparse Representation

  2007cited by this paper
• The DUET Blind Source Separation Algorithm

  2007cited by this paper
• Multi-frame representations in linear inverse problems with mixed multi-constraints

  2007cited by this paper
• Lp Norm Iterative Sparse Solution for EEG Source Localization

  2007cited by this paper
• Sparse sources are separated sources

  2006cited by this paper
• $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation

  2006cited by this paper
• Sparse Representations Are Most Likely to Be the Sparsest Possible

  2006cited by this paper
• Stable recovery of sparse overcomplete representations in the presence of noise

  2006cited by this paper
• Just relax: convex programming methods for identifying sparse signals in noise

  2006cited by this paper
• Equivalence principle for optimization of sparse versus low‐spread representations for signal estimation in noise

  2005cited by this paper
• Decoding by linear programming

  2005cited by this paper
• Recovery of exact sparse representations in the presence of bounded noise

  2005cited by this paper
• Parameterized Lifting for Sparse Signal Representations Using the Gini Index

  2005cited by this paper
• Survey of sparse and non‐sparse methods in source separation

  2005cited by this paper
• The Mean-Gini Efficient Portfolio Frontier

  2005cited by this paper
• K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation

  2005cited by this paper
• Sparse ICA for blind separation of transmitted and reflected images

  2005cited by this paper
• The Gini index of speech

  2004cited by this paper
• Blind separation of speech mixtures via time-frequency masking

  2004cited by this paper
• Non-negative Matrix Factorization with Sparseness Constraints

  2004cited by this paper
• Sparse coding of sensory inputs.

  2004cited by this paper
• Sparse representations in unions of bases

  2003cited by this paper
• MEASURING SPARSENESS OF NOISY SIGNALS

  2003cited by this paper
• Axiomatic Characterization of the Gini Coefficient and Lorenz Curve Orderings

  2000cited by this paper
• An affine scaling methodology for best basis selection

  1999cited by this paper
• Measures and algorithms for best basis selection

  1998cited by this paper
• Highly Scalable Parallel Algorithms for Sparse Matrix Factorization

  1997cited by this paper
• A simple way to calculate the gini coefficient, and some implications

  1997cited by this paper
• Majorization and the Lorenz Order: A Brief Introduction

  1987cited by this paper
• The Measurement of the Inequality of Incomes

  1920cited by this paper
• Methods of Measuring the Concentration of Wealth

  1905cited by this paper

• Exploring Sparsity and Smoothness of Arbitrary $\ell_p$ Norms in Adversarial Attacks

  2026cites this paper
• A novel multi-chirp rate demodulation-based time-frequency transform method for non-stationary underwater acoustic signals

  2026cites this paper
• Memory-guided mask reconstruction with central contrastive learning for robust multivariate time series anomaly detection.

  2026cites this paper
• Intrinsic Interpretability at Parity: Attention-Based RL–MIL for Student Outcome Prediction

  2026cites this paper
• The spatiotemporal band-gated modal decomposition method and its application in compound fault diagnosis of gearbox

  2026cites this paper
• A New Family of Poisson Non-negative Matrix Factorization Methods Using the Shifted Log Link

  2026cites this paper
• Non-convex Extreme Learning Machine with L1 over L2 Norm Regularization and Exponential Squared Loss for Sparse and Robust Regression

  2026cites this paper
• Interpreting Emergent Extreme Events in Multi-Agent Systems

  2026influential citation
• Wealth exchange under ceiling and flooring constraints: a modified Bennati-Dragulescu-Yakovenko model

  2026cites this paper
• Simplicial Embeddings Improve Sample Efficiency in Actor-Critic Agents

  2025influential citation
• MVCUN: A New CNN-Based Autoencoder With Minimum Volume Constraint for Hyperspectral Unmixing

  2025cites this paper
• Connecting Parameter Magnitudes and Hessian Eigenspaces at Scale using Sketched Methods

  2025cites this paper
• Boosting the visual interpretability of CLIP via adversarial fine-tuning

  2025cites this paper
• Across-speaker articulatory reconstruction from sensorimotor cortex for generalizable brain-computer interfaces

  2025cites this paper
• Designing lensless imaging systems to maximize information capture

  2025cites this paper
• A Global Dataset-Specific Any-Order Minimal Expectation Baseline for Saliency Scores

  2025cites this paper
• A permutation-equivariant deep learning model for quantum state characterization

  2025cites this paper
• Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignment

  2025cites this paper
• Clairvoyant performance bounds for adaptive beamforming in pulse-echo imaging.

  2025cites this paper
• Robust and early howling detection based on a sparsity measure

  2025cites this paper
• Blind sonar image quality assessment via machine learning: Leveraging micro- and macro-scale texture and contour features in the wavelet domain

  2025cites this paper
• Indoor Deterministic Simulations and Statistical Modeling at Sub-THz Frequencies for Future Wireless Networks

  2025cites this paper
• Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Models

  2025cites this paper
• Multi-Resolution Spectrums Analysis-Based DOA Estimation Using Energy and Sparsity Joint Mask

  2025cites this paper
• Interpreting the Linear Structure of Vision-language Model Embedding Spaces

  2025cites this paper
• Truncated Huber Penalty for Sparse Signal Recovery with Convergence Analysis

  2025cites this paper
• Forecasting Surgical Bed Utilization: Architectural Design of a Machine Learning Pipeline Incorporating Predicted Length of Stay and Surgical Volume

  2025cites this paper
• A Robust Deep Learning-Based DOA Super-Resolution Estimation Method With Single Snapshot

  2025cites this paper
• Weight-sparse transformers have interpretable circuits

  2025cites this paper
• On the Use of the Detectivity Parameter for the Condition Monitoring of Wind Turbines

  2025cites this paper
• Optimizing Multi-Modal Trackers via Sensitivity-aware Regularized Tuning

  2025cites this paper
• On Rank Graduation Metrics for High Dimensional Ordinal Data

  2025cites this paper
• Reference Microphone Selection for Guided Source Separation based on the Normalized L-p Norm

  2025cites this paper
• Towards Inference-time Scaling for Continuous Space Reasoning

  2025cites this paper
• A Multiclass ROC Curve

  2025cites this paper
• Toward Unifying Group Fairness Evaluation from a Sparsity Perspective

  2025influential citation
• Pursuit Regularity Index for Parkinson's Disease Detection via Videonystagmography

  2025cites this paper
• Dynamic metric aggregation tracker

  2025cites this paper
• Probability distributions and typical sparsity measures of Hilbert transform-based generalized envelopes and their application to machine condition monitoring

  2025cites this paper
• Pico: A Modular Framework for Hypothesis-Driven Small Language Model Research

  2025cites this paper
• Random Projections and Natural Sparsity in Time-Series Classification: A Theoretical Analysis

  2025cites this paper
• Instantaneous Angular Speed-Based Fault Detection for Planet and Cycloidal Gears in Rotary Vector Reducers

  2025cites this paper
• Optimizing Multi-Modality Trackers via Significance-Regularized Tuning

  2025cites this paper
• Maximum Correntropy Linear Prediction for Voice Inverse Filtering: Theoretical Framework and Practical Implementation

  2025cites this paper
• Sparse Channel Reconstruction: A Generative-Adversarial-Network-Based Approach

  2025cites this paper
• Channel Measurements and Multipath Characterization for Indoor Sub-Terahertz Communication

  2025cites this paper
• A Nonconvex Fractional Regularization Model in Robust Principal Component Analysis via the Symmetric Alternating Direction Method of Multipliers

  2025cites this paper
• Modeling and Analysis of Land-to-Ship Maritime Wireless Channels at 5.8 GHz

  2025cites this paper
• An ensemble learning model with dynamic sampling and feature fusion network for class sparsity in credit risk classification

  2025cites this paper
• Hippocampal Ripple Diversity organises Neuronal Reactivation Dynamics in the Offline Brain

  2025cites this paper
• Cycle-embedded sparsity measures as a generalized objective function of impulsive mode decomposition for impulsive fault component extraction

  2025cites this paper
• Experimental verification of threshold quantum state tomography on a fully-reconfigurable photonic integrated circuit

  2025cites this paper
• A Health Monitoring Index for Rotating Machinery Based on the Logarithmic Interval Quantile Root Mean Square Value Ratio

  2025cites this paper
• Enhanced diagnosis of bearing and gear faults using Hilbert-Huang transform, singular value decomposition, and supervised learning methods

  2025cites this paper
• Quaternion Nuclear Norms Over Frobenius Norms Minimization for Robust Matrix Completion

  2025cites this paper
• Hebb’s Vision: The Structural Underpinnings of Hebbian Assemblies

  2025cites this paper
• Empirical Evaluation of Progressive Coding for Sparse Autoencoders

  2025cites this paper
• Seismic Acoustic Impedance Inversion Using Combined Higher Order Lp-Pseudo-Norm With Overlapping Group Sparsity

  2025cites this paper
• Measuring inequality in society-oriented Lotka–Volterra-type kinetic equations

  2025cites this paper
• Time and Frequency Domain Blind Deconvolution Based on Generalized Lp/Lq Norm for Rolling Bearing Fault Diagnosis

  2025cites this paper
• A convolution and transformer-based method with effective stain normalization for breast cancer detection from whole slide images

  2025cites this paper
• Sparse Frequency Representation Using Autocorrelation of Variational Mode Functions to Detect Compound Fault in Rotating Machines

  2025influential citation
• A multi-delay extension of the Discrete/Random Separation method

  2025cites this paper
• From Kinetic Theory to AI: a Rediscovery of High-Dimensional Divergences and Their Properties

  2025cites this paper
• Stable recovery guarantees for ℓp/ℓq-minimization based on restricted isometry and orthogonality properties

  2025cites this paper
• Neural Ordinary Differential Equations with Differentiable Hidden State for Irregular Time Series

  2025cites this paper
• Unsupervised Deep Equilibrium Model Learning for Large-Scale Channel Estimation with Performance Guarantees

  2025influential citation
• A New Adaptive Stopping Criterion for Sparse Channel Estimation in MIMO-OFDM Underwater Acoustic Communications

  2025cites this paper
• Prunability of Multi-Layer Perceptrons Trained with the Forward-Forward Algorithm

  2025cites this paper
• Measuring inequality in high dimensions: a Gini-based approach

  2025influential citation
• Data-driven dryout prediction in helical-coiled once-through steam generator: A physics-informed approach leveraging the Buckingham Pi theorem

  2024cites this paper
• Multitask feature selection within structural datasets

  2024influential citation
• Adaptation to visual sparsity enhances responses to infrequent stimuli

  2024cites this paper
• Towards trustworthy remaining useful life prediction through multi-source information fusion and a novel LSTM-DAU model

  2024cites this paper
• Critical dimensions in the empirical measurement of common shareholding

  2024cites this paper
• Continuous Movement Quantification in Preterm Infants Using Fiber Mat: Advancing Long-Term Monitoring in Hospital Settings

  2024cites this paper
• Compressed sensing for rapid tabletop X-ray absorption spectroscopy

  2024cites this paper
• Channel Sparsity Variation and Model-Based Analysis on 6, 26, and 105 GHz Measurements

  2024cites this paper
• Evaluation of sparsity metrics and evolutionary algorithms applied for normalization of H&E histological images

  2024cites this paper
• Impulsive mode decomposition

  2024cites this paper
• Unsupervised Learning of Neural Semantic Mappings with the Hungarian Algorithm for Compositional Semantics

  2024cites this paper
• On Partial Smoothness, Activity Identification and Faster Algorithms of $L_{1}$ Over $L_{2}$ Minimization

  2024cites this paper
• Investigations on improved Box-Cox sparsity measures for machine condition monitoring.

  2024cites this paper
• Comparing Activation Typicality and Sparsity in a Deep CNN to Predict Facial Beauty

  2024cites this paper
• Quantifying Sparsity of Underwater Acoustic Channel: Field Measurement and Analysis

  2024cites this paper
• Adaptation to visual sparsity enhances responses to isolated stimuli.

  2024cites this paper
• Evaluating audio quality ratings and scene analysis performance of hearing-impaired listeners for multi-track music.

  2024cites this paper
• Organizing space through saccades and fixations between primate posterior parietal cortex and hippocampus

  2024cites this paper
• IF equation: A feature extractor for high-concentration time-frequency representation and application to mixed signals analysis

  2024cites this paper
• Measuring multidimensional inequality: a new proposal based on the Fourier transform

  2024cites this paper
• Nonlinear Blind Deconvolution Based on Generalized Normalized lp/lq Norm for Early Fault Detection

  2024cites this paper
• Local Rényi entropy-based Gini index for measuring and optimizing sparse time-frequency distributions

  2024cites this paper
• New health indicators for the monitoring of bearing failures under variable loads

  2024cites this paper
• Speech-Aware Binaural DOA Estimation Utilizing Periodicity and Spatial Features in Convolutional Neural Networks

  2024cites this paper
• Multivariate Gini-type discrepancies

  2024influential citation
• An efficient new adaptive variational mode decomposition algorithm for extracting adventitious lung sounds

  2024cites this paper
• Benchmarking XAI Explanations with Human-Aligned Evaluations

  2024cites this paper
• Anchor Attention, Small Cache: Code Generation With Large Language Models

  2024cites this paper
• Investigations on collision entropy and its variant for machine condition monitoring

  2024cites this paper
• Cyclic Impulse Spectrum Analysis of Time-Frequency Slices for Axle-Box Bearing Fault Identification

  2024cites this paper