Estimating probabilities from experimental frequencies.

Estimating the probability distribution q governing the behavior of a certain variable by sampling its value a finite number of times most typically involves an error. Successive measurements allow the construction of a histogram, or frequency count f, of each of the possible outcomes. In this work, the probability that the true distribution be q, given that the frequency count f was sampled, is studied. Such a probability may be written as a Gibbs distribution. A thermodynamic potential, which allows an easy evaluation of the mean Kullback-Leibler divergence between the true and measured distribution, is defined. For a large number of samples, the expectation value of any function of q is expanded in powers of the inverse number of samples. As an example, the moments, the entropy, and the mutual information are analyzed.

• Publication year

  2002

• Venue

  Physical review. E, Statistical, nonlinear, and soft matter physics

• Publication date

  2002-01-28

• Fields of study

  Mathematics, Physics, Medicine

• Identifiers
  DOI 10.1103/PhysRevE.65.046124arXiv cond-mat/0201516PMID 12005943
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar, PubMed

• No claims are published for this paper.

• No concepts are published for this paper.

• Predictability, Complexity, and Learning

  2000cited by this paper
• Tables of integrals

  1998cited by this paper
• Analytical estimates of limited sampling biases in different information measures.

  1996cited by this paper
• Entropy estimation of symbol sequences.

  1996cited by this paper
• Tables of Integrals, Series, and Products

  1995cited by this paper
• Estimating functions of probability distributions from a finite set of samples.

  1994cited by this paper
• Statistical mechanics of learning from examples.

  1992cited by this paper
• Table of Integrals, Series, and Products

  1966cited by this paper
• Handbook of Mathematical Functions.

  1966cited by this paper
• Information Theory and Statistics

  1960cited by this paper

• Phosphoproteomics data-driven signalling network inference: Does it work?

  2022cites this paper
• Inferring a Property of a Large System from a Small Number of Samples

  2021cites this paper
• Estimating the Mutual Information between Two Discrete, Asymmetric Variables with Limited Samples

  2019cites this paper
• Information-Theoretical Analysis of the Neural Code in the Rodent Temporal Lobe

  2018cites this paper
• Assessing the Relevance of Specific Response Features in the Neural Code

  2018cites this paper
• Rhythmic activities of the brain: Quantifying the high complexity of beta and gamma oscillations during visuomotor tasks.

  2018cites this paper
• Disambiguating the role of noise correlations when decoding neural populations together

  2016cites this paper
• Chapter 1 Information Communication

  2016cites this paper
• Information-theoretical analysis of the statistical dependencies among three variables: Applications to written language

  2015cites this paper
• An Information Theoretic Criterion for Empirical Validation of Time Series Models

  2015cites this paper
• APER S ERIES An Information Theoretic Criterion for Empirical Validation of Time Series Models

  2015cites this paper
• Tackling Model Selection and Validation : an Information Theoretic Criterion

  2014cites this paper
• Shannon entropy: a rigorous notion at the crossroads between probability, information theory, dynamical systems and statistical physics

  2014cites this paper
• Modulus and direction of the neural current vector identify distinct functional connectivity modes between human MT+ areas.

  2010cites this paper
• Classification of skin tumours through the analysis of unconstrained images

  2009cites this paper
• Burst Firing is a Neural Code in an Insect Auditory System

  2008cites this paper
• An information-theoretic study of neuronal spike correlations in the mammalian cerebral cortex

  2008cites this paper
• Optimal instruments and models for noisy chaos.

  2007cites this paper
• Inferring Markov chains: Bayesian estimation, model comparison, entropy rate, and out-of-class modeling.

  2007cites this paper
• How Random is a Coin Toss? Bayesian Inference and the Symbolic Dynamics of Deterministic Chaos

  2006cites this paper
• Accuracy of joint entropy and mutual information estimates

  2004cites this paper
• Predictability, Complexity, and Learning

  2000cites this paper