Batch and match: black-box variational inference with a score-based divergence
CoRR(2024)
摘要
Most leading implementations of black-box variational inference (BBVI) are
based on optimizing a stochastic evidence lower bound (ELBO). But such
approaches to BBVI often converge slowly due to the high variance of their
gradient estimates. In this work, we propose batch and match (BaM), an
alternative approach to BBVI based on a score-based divergence. Notably, this
score-based divergence can be optimized by a closed-form proximal update for
Gaussian variational families with full covariance matrices. We analyze the
convergence of BaM when the target distribution is Gaussian, and we prove that
in the limit of infinite batch size the variational parameter updates converge
exponentially quickly to the target mean and covariance. We also evaluate the
performance of BaM on Gaussian and non-Gaussian target distributions that arise
from posterior inference in hierarchical and deep generative models. In these
experiments, we find that BaM typically converges in fewer (and sometimes
significantly fewer) gradient evaluations than leading implementations of BBVI
based on ELBO maximization.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要