The $χ$-Divergence for Approximate Inference.
arXiv: Machine Learning(2016)
摘要
Variational inference enables Bayesian analysis for complex probabilistic models with massive data sets. It works by positing a family of distributions and finding the member in the family that is closest to the posterior. While successful, variational methods can run into pathologies; for example, they typically underestimate posterior uncertainty. We propose CHI-VI, a complementary algorithm to traditional variational inference with KL($q$ || $p$) and an alternative algorithm to EP. CHI-VI is a black box algorithm that minimizes the $chi$-divergence from the posterior to the family of approximating distributions. In EP, only local minimization of the KL($p$ || $q$) objective is possible. In contrast, CHI-VI optimizes a well-defined global objective. It directly minimizes an upper bound to the model evidence that equivalently minimizes the $chi$-divergence. In experiments, we illustrate the utility of the upper bound for sandwich estimating the model evidence. We also compare several probabilistic models and a Cox process for basketball data. We find CHI-VI often yields better classification error rates and better posterior uncertainty.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络