Information theoretic inequalities
IEEE Transactions on Information TheoryPublished 1 January 1991
Amir Dembo, Thomas M. Cover, J. Thomas
Citations757
SJR quartileQ1
SJR score1.46
SNIP1.76
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The authors focus on the entropy power inequality (including the related Brunn-Minkowski, Young's, and Fisher information inequalities) and address various uncertainty principles and their interrelations.
Abstract
The role of inequalities in information theory is reviewed, and the relationship of these inequalities to inequalities in other branches of mathematics is developed. The simple inequalities for differential entropy are applied to the standard multivariate normal to furnish new and simpler proofs of the major determinant inequalities in classical mathematics. The authors discuss differential entropy inequalities for random subsets of samples. These inequalities when specialized to multivariate normal variables provide the determinant inequalities that are presented. The authors focus on the entropy power inequality (including the related Brunn-Minkowski, Young's, and Fisher information inequalities) and address various uncertainty principles and their interrelations.>
Keywords
Decision SciencesMathematicsPhysics and Astronomy
Bell System Technical JournalA Mathematical Theory of Communication
80,675 Citations1948Claude E. Shannon
HAL (Le Centre pour la Communication Scientifique Directe)Lecture Notes in Mathematics
10,101 Citations2011Stéphane Crépey, Olivier Guéant +4 more
Journal of the American Statistical AssociationInequalities: Theory of Majorization and its Applications.
4,984 Citations1981Barry C. Arnold, Albert W. Marshall +1 more
Reviews of Modern PhysicsGeneral properties of entropy
1,978 Citations1978A. Wehrl
This paper discusses properties of entropy, as well as related concepts such as relative entropy, skew entropy, dynamical entropy, etc, in detail with reference to their implications in statistical mechanics, to get a glimpse of systems with infinitely many degrees of freedom.
SIAM Journal on Applied MathematicsUncertainty Principles and Signal Recovery
1,079 Citations1989David L. Donoho, Philip B. Stark
The uncertainty principle can easily be generalized to cases where the “sets of concentration” are not intervals, and for several measures of “concentration” (e.g., $L_2 $ and $L-1 $ measures).
On Extensions of the Brunn-Minkowski and Prékopa-Leindler Theorems, Including Inequalities for Log Concave Functions, and with an Application to the Diffusion Equation
740 Citations2002H. J. Brascamp, Élliott H. Lieb
Information and ControlSome inequalities satisfied by the quantities of information of Fisher and Shannon
700 Citations1959A. J. Stam
A certain analogy is found to exist between a special case of Fisher's quantity of information I and the inverse of the “entropy power” of Shannon and this constitutes a sharpening of the uncertainty relation of quantum mechanics for canonically conjugated variables.
Journal of Functional AnalysisOn extensions of the Brunn-Minkowski and Prékopa-Leindler theorems, including inequalities for log concave functions, and with an application to the diffusion equation
672 Citations1976H. J. Brascamp, Élliott H. Lieb
IEEE Transactions on Information TheoryA simple converse for broadcast channels with additive white Gaussian noise (Corresp.)
457 Citations1974P. Bergmans
A simple converse is established showing the optimality of sets of achievable rates for the additive white Gaussian noise broadcast channel, found by Cover and generalized by Bergmans.
Advances in MathematicsBest constants in Young's inequality, its converse, and its generalization to more than three functions
419 Citations1976H. J. Brascamp, Élliott H. Lieb
Cambridge University Press eBooksGeometric Measure Theory
344 Citations2023Janina Kotus, Mariusz Urbański
IEEE Transactions on Information TheoryThe convolution inequality for entropy powers
342 Citations1965Nelson M. Blachman
An improved version of Stam's proof of Shannon's convolution inequality for entropy power is presented, which is obtained by mathematical induction from the one-dimensional case.
Communications in Mathematical PhysicsProof of an entropy conjecture of Wehrl
321 Citations1978Élliott H. Lieb
Inventiones mathematicaeGaussian kernels have only Gaussian maximizers
302 Citations1990Élliott H. Lieb
Journal of Functional AnalysisSuperadditivity of Fisher's information and logarithmic Sobolev inequalities
280 Citations1991Eric A. Carlen
Information and ControlNonnegative entropy measures of multivariate symmetric correlations
225 Citations1978Te Sun Han
A “hierarchical structure” of probabilistic dependence relations is proposed where it is shown that any symmetric correlation associated with a nonnegative entropy is decomposed into pairwise conditional and/or nonconditional correlations.
IEEE Transactions on Information TheoryA new entropy power inequality
209 Citations1985Max H. M. Costa
A strengthened version of Shannon's entropy power inequality for the case where one of the random vectors involved is Gaussian is proved and it is shown that if independent Gaussian noise is added to an arbitrary, multivariate random variable, the entropy power of the resulting random variable is a concave function of the variance of the added noise.
Communications in Mathematical PhysicsEntropy production by block variable summation and central limit theorems
107 Citations1991Eric A. Carlen, Avy Soffer
Journal of Functional AnalysisSome integral identities and inequalities for entire functions and their application to the coherent state transform
88 Citations1991Eric A. Carlen
SIAM Journal on Matrix Analysis and ApplicationsDeterminant Inequalities via Information Theory
78 Citations1988Thomas M. Cover, Anne Thomas
IEEE Transactions on Information TheoryOn the similarity of the entropy power inequality and the Brunn- Minkowski inequality (Corresp.)
76 Citations1984Max H. M. Costa, Thomas M. Cover
IEEE Transactions on Information TheorySimple proof of the concavity of the entropy power with respect to added Gaussian noise
61 Citations1989Amir Dembo
The relationship between Fisher information and entropy is found to be useful for proving the central limit theorem and one who seeks new entropy inequalities should try to find new equalities about Fisher information, or at least to exploit the existing ones in new ways.
Mathematics in Science and Engineering/Mathematics in science and engineeringGeometric Inequalities
52 Citations1979
Mathematical Proceedings of the Cambridge Philosophical SocietySome inequalities concerning positive-definite Hermitian matrices
39 Citations1955Ky Fan
Information and ControlA class of definitions of “duration” (or “uncertainty”) and the associated uncertainty relations
21 Citations1960Moshe Zakai
A new class of definitions for “ time duration” and “bandwidth” (or “time uncertainty’ and ”frequency uncertainty”), in terms of norms of L p spaces, is suggested and some properties of the definitions and the associated uncertainty relations are derived.
IEEE Transactions on Information TheoryAn information - theoretic proof of Hadamard's inequality (Corresp.)
19 Citations1983Thomas M. Cover, Abbas El Gamal
Hadamard's inequality follows immediately from inspection of both sides of the entropy inequality h(X_{1}, X_{2},, X_{n})\leq h( X_{i}) when (X_{l}, X_2},\cdots, X_n}) is multivariate normal.
American Mathematical MonthlyA Convexity Proof of Hadamard's Inequality
18 Citations1982Albert W. Marshall, Ingram Olkin