On the Properties and Estimation of Pointwise Mutual Information Profiles
arxiv(2023)
摘要
The pointwise mutual information profile, or simply profile, is the
distribution of pointwise mutual information for a given pair of random
variables. One of its important properties is that its expected value is
precisely the mutual information between these random variables. In this paper,
we analytically describe the profiles of multivariate normal distributions and
introduce a novel family of distributions, Bend and Mix Models, for which the
profile can be accurately estimated using Monte Carlo methods. We then show how
Bend and Mix Models can be used to study the limitations of existing mutual
information estimators, investigate the behavior of neural critics used in
variational estimators, and understand the effect of experimental outliers on
mutual information estimation. Finally, we show how Bend and Mix Models can be
used to obtain model-based Bayesian estimates of mutual information, suitable
for problems with available domain expertise in which uncertainty
quantification is necessary.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要