SCIENCE@home

On Pinsker's and Vajda's Type Inequalities for Csiszár's $f$ -Divergences

Gustavo L. Gilardoni

Published 2006 in IEEE Transactions on Information Theory

ABSTRACT

Let <i>D</i> and <i>V</i> denote respectively Information Divergence and Total Variation Distance. Pinsker's and Vajda's inequalities are respectively <i>D</i> ≥ [ 1/ 2] <i>V</i><sup>2</sup> and <i>D</i> ≥ log[( 2+<i>V</i>)/( 2-<i>V</i>)] - [( 2<i>V</i>)/( 2+<i>V</i>)]. In this paper, several generalizations and improvements of these inequalities are established for wide classes of <;i>f<;/i>-divergences. First, conditions on <i>f</i> are determined under which an <i>f</i>-divergence <i>Df</i> will satisfy <i>Df</i> ≥ <i>cf V</i><sup>2</sup> or <i>Df</i> ≥ <i>c</i><sub>2,</sub><i>f V</i><sup>2</sup> + <i>c</i><sub>4,</sub><i>f V</i><sup>4</sup>, where the constants <i>cf</i>, <i>c</i><sub>2,</sub><i>f</i> and <i>c</i><sub>4,</sub><i>f</i> are best possible. As a consequence, lower bounds in terms of <i>V</i> are obtained for many well known distance and divergence measures, including the χ<sup>2</sup> and Hellinger's discrimination and the families of Tsallis' and Rényi's divergences. For instance, if <i>D</i><sub>(α)</sub> (<i>P</i>||<i>Q</i>) = [α(α-1)]<sup>-1</sup> [∫<i>p</i><sup>α</sup><i>q</i><sup>1-α</sup><i>d</i>μ-1] and ℑ<sub>α</sub> (<i>P</i>||<i>Q</i>) = (α-1)<sup>-1</sup> log[∫<i>p</i><sup>α</sup><i>q</i><sup>1-α</sup><i>d</i>μ] are respectively the relative information of type α and the Rényi's information gain of order α, it is shown that <i>D</i><sub>(α)</sub> ≥ [ 1/ 2] <i>V</i><sup>2</sup> + [ 1/ 72] (α+1)(2-α) <i>V</i><sup>4</sup> whenever -1 ≤ α ≤ 2, α ≠ 0,1 and that ℑ<sub>α</sub> ≥ [( α)/ 2] <i>V</i><sup>2</sup> + [ 1/ 36] α(1 + 5 α- 5 α<sup>2</sup> ) <i>V</i><sup>4</sup> for 0 <; α <; 1. In a somewhat different direction, and motivated by the fact that these Pinsker's type lower bounds are accurate only for small variation (<i>V</i> close to zero), lower bounds for <i>Df</i> which are accurate for both small and large variation (<i>V</i> close to two) are also obtained. In the special case of the information divergence they imply that <i>D</i> ≥ log[ 2/( 2-<i>V</i>)] - [( 2-<i>V</i>)/2] log[( 2+<i>V</i>)/2], which uniformly improves Vajda's inequality.

PUBLICATION RECORD

CITATION MAP

EXTRACTION MAP

CLAIMS

  • No claims are published for this paper.

CONCEPTS

  • No concepts are published for this paper.

REFERENCES

Showing 1-45 of 45 references · Page 1 of 1

CITED BY

Showing 1-100 of 107 citing papers · Page 1 of 2

12
Next