Estimating entropy on m bins given fewer than m samples

No abstract is available for this paper.

• Publication year

  2004

• Venue

  IEEE Transactions on Information Theory

• Publication date

  2004-09-01

• Fields of study

  Mathematics, Computer Science

• Identifiers
  DOI 10.1109/TIT.2004.833360
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Estimation of Entropy and Mutual Information

  2003influential reference
• Convergence properties of functional estimates for discrete distributions

  2001cited by this paper
• A Probabilistic Theory of Pattern Recognition

  1996cited by this paper
• Geometrizing Rates of Convergence, III

  1991cited by this paper
• The Logarithmic Integral I: Frontmatter

  1988cited by this paper
• The Logarithmic Integral

  1986cited by this paper
• Asymptotic Methods In Statistical Decision Theory

  1986cited by this paper
• On Measures of Entropy and Information

  1961cited by this paper
• Note on the bias of information estimates

  1955cited by this paper
• A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations

  1952cited by this paper

• Estimating the Shannon Entropy Using the Pitman--Yor Process

  2026cites this paper
• Improved Sample Upper and Lower Bounds for Trace Estimation of Quantum State Powers

  2025cites this paper
• A Comprehensive Study on Non-Parametric Measures of Entropy and Their Mathematical Characteristics

  2025cites this paper
• Information-Theoretic Lower Bounds for Approximating Monomials via Optimal Quantum Tsallis Entropy Estimation

  2025cites this paper
• The Cramér-Rao approach and global quantum estimation of bosonic states

  2024cites this paper
• Entropy testing and its application to testing Bayesian networks

  2024cites this paper
• Memory Complexity of Estimating Entropy and Mutual Information

  2024cites this paper
• Collision Entropy Estimation in a One-Line Formula

  2023cites this paper
• Statistical Inference With Limited Memory: A Survey

  2023cites this paper
• Truly Bayesian Entropy Estimation

  2022cites this paper
• New Quantum Algorithms for Computing Quantum Entropies and Distances

  2022cites this paper
• Dimension-Free Empirical Entropy Estimation

  2021cites this paper
• Complete Problems for Multi-Pseudodeterministic Computations

  2021cites this paper
• Sliced Mutual Information: A Scalable Measure of Statistical Dependence

  2021cites this paper
• The Information Bottleneck Problem and its Applications in Machine Learning

  2020cites this paper
• A Survey on Distribution Testing: Your Data is Big. But is it Blue?

  2020cites this paper
• Minimax Estimation of Divergences Between Discrete Distributions

  2020cites this paper
• On the Efficient Estimation of Min-Entropy

  2020cites this paper
• Systematic errors in connectivity inferred from activity in strongly recurrent networks

  2020cites this paper
• On the High Accuracy Limitation of Adaptive Property Estimation

  2020cites this paper
• On the Competitive Analysis and High Accuracy Optimality of Profile Maximum Likelihood

  2020cites this paper
• The Optimality of Profile Maximum Likelihood in Estimating Sorted Discrete Distributions

  2020cites this paper
• Investigation on the high-order approximation of the entropy bias

  2020cites this paper
• Estimating Entropy of Distributions in Constant Space

  2019cites this paper
• Distribution Testing Lower Bounds via Reductions from Communication Complexity

  2019cites this paper
• Systematic errors in connectivity inferred from activity in strongly coupled recurrent circuits

  2019cites this paper
• Structural changes in large economic datasets: A nonparametric homogeneity test

  2019cites this paper
• HEDGE: Efficient Traffic Classification of Encrypted and Compressed Packets

  2019cites this paper
• Importance of Small Probability Events in Big Data: Information Measures, Applications, and Challenges

  2019cites this paper
• Distributional property testing in a quantum world

  2019cites this paper
• Demystifying Fixed $k$ -Nearest Neighbor Information Estimators

  2018cites this paper
• Learning from the past: A reverberation of past errors in the cerebellar climbing fiber signal

  2018cites this paper
• Differences of HLA Gene Regulatory Network in Human Myeloid and Lymphoid Leukemias

  2018cites this paper
• Minimax Optimal Additive Functional Estimation with Discrete Distribution: Slow Divergence Speed Case

  2018cites this paper
• Entropy Rate Estimation for Markov Chains with Large State Space

  2018cites this paper
• Direct Estimation of Density Functionals Using a Polynomial Basis

  2018cites this paper
• Proofs of Proximity for Distribution Testing

  2018cites this paper
• Distributed Simulation and Distributed Inference

  2018cites this paper
• L G ] 6 F eb 2 01 8 Recovering Structured Probability Matrices

  2018cites this paper
• Minimax optimal estimators for additive scalar functionals of discrete distributions

  2017cites this paper
• Shannon Entropy Estimation in ∞-Alphabets from Convergence Results: Studying Plug-In Estimators

  2017influential citation
• Quantum Query Complexity of Entropy Estimation

  2017cites this paper
• Bridging Mixture Model Estimation and Information Bounds Using I-MMSE

  2017cites this paper
• Optimal rates of entropy estimation over Lipschitz balls

  2017cites this paper
• Instance optimal learning of discrete distributions

  2016cites this paper
• Recovering Structured Probability Matrices

  2016cites this paper
• Sample complexity of the distinct elements problem

  2016cites this paper
• On the minimum probability of classification error through effective cardinality comparison

  2016cites this paper
• Isotonic Hawkes Processes

  2016cites this paper
• Minimax rate-optimal estimation of KL divergence between discrete distributions

  2016cites this paper
• Alice and Bob Show Distribution Testing Lower Bounds (They don't talk to each other anymore.)

  2016cites this paper
• Ensemble Estimation of Information Divergence

  2016cites this paper
• Sharp minimax bounds for testing discrete monotone distributions

  2016cites this paper
• AN AUTOMATIC INEQUALITY PROVER AND INSTANCE

  2016cites this paper
• Demystifying fixed k-nearest neighbor information estimators

  2016cites this paper
• Improving Accuracy and E � ciency of Mutual Information for Multi-modal Retinal Image Registration using Adaptive Probability Density Estimation

  2015cites this paper
• Testing Closeness With Unequal Sized Samples

  2015cites this paper
• Estimating Information-Theoretic Quantities

  2015cites this paper
• Quantum Spectrum Testing

  2015cites this paper
• Estimation and Compression over Large Alphabets Electrical Engineering (communication Theory and Systems) Efficient Compression of Distributions with a Few Modes . . . . 26

  2015cites this paper
• Does dirichlet prior smoothing solve the Shannon entropy estimation problem?

  2015influential citation
• Maximum Likelihood Estimation of information measures

  2015cites this paper
• Minimax estimation of information measures

  2015cites this paper
• Optimal Bounds for Estimating Entropy with PMF Queries

  2015cites this paper
• Instance Optimal Learning

  2015cites this paper
• Practical Analysis of Encrypted Network Traffic

  2015cites this paper
• Optimal entropy estimation on large alphabets via best polynomial approximation

  2015cites this paper
• Big Data on the Rise? - Testing Monotonicity of Distributions

  2015cites this paper
• Maximum Likelihood Estimation of Functionals of Discrete Distributions

  2014cites this paper
• Minimax Rates of Entropy Estimation on Large Alphabets via Best Polynomial Approximation

  2014cites this paper
• Improved Estimation of Entropy for Evaluation of Word Sense Induction

  2014cites this paper
• Must one learn the channel to communicate at capacity?

  2014cites this paper
• The Complexity of Estimating Rényi Entropy

  2014influential citation
• An Automatic Inequality Prover and Instance Optimal Identity Testing

  2014cites this paper
• Estimating Renyi Entropy of Discrete Distributions

  2014cites this paper
• Minimax Estimation of Functionals of Discrete Distributions

  2014influential citation
• Testing equivalence between distributions using conditional samples

  2014cites this paper
• QUADRATIC TSALLIS ENTROPY BIAS AND GENERALIZED MAXIMUM ENTROPY MODELS

  2014cites this paper
• Electrical Engineering (Communication Theory and Systems)

  2014cites this paper
• Estimation and Compression over Large Alphabets.

  2014cites this paper
• Improved Estimation of Entropy for Evaluation of Word Sense Induction

  2014cites this paper
• Temporal contingency - 1-s2.0-S0376635713001873-main

  2013cites this paper
• Shannon entropy estimation from convergence results in the countable alphabet case

  2013cites this paper
• Clear and Present Data: Opaque Traffic and its Security Implications for the Future

  2013cites this paper
• Maximum Entropy Based Multivariate Dependence Analysis with a Case Study for HLA Gene Regulatory Network in Human Leukemia

  2013cites this paper
• Instance-by-instance optimal identity testing

  2013cites this paper
• Dompter les Distributions de Probabilité Géantes

  2013cites this paper
• Some worst-case bounds for Bayesian estimators of discrete distributions

  2013cites this paper
• On the Efficiency of Mathematics in Intrusion Detection: The NetEntropy Case

  2013cites this paper
• Improving accuracy and efficiency of mutual information for multi-modal retinal image registration using adaptive probability density estimation

  2013cites this paper
• Testing Properties of Collections of Distributions

  2013cites this paper
• Testing k-wise independent distributions

  2012cites this paper
• Taming big probability distributions

  2012cites this paper
• Universal algorithms: Building a case for pointwise convergence

  2012cites this paper
• Large alphabets: Finite, infinite, and scaling models

  2012influential citation
• Large alphabets : finite , infinite , and scaling models ( Invited Paper )

  2012influential citation
• INFORMATION THEORY BASED ANALYSIS FOR UNDERSTANDING THE REGULATION OF HLA GENE EXPRESSION IN HUMAN LEUKEMIA

  2012cites this paper
• On the estimation of entropy for unknown support size

  2012cites this paper
• Testing probability distributions using conditional samples

  2012cites this paper
• Algorithmic Approaches to Statistical Questions

  2012influential citation