Variational Wishart Approximation for Graphical Model Selection: Monoscale and Multiscale Models

Graphical models are powerful tools to describe high-dimensional data; they provide a compact graphical representation of the interactions between different variables and such representation enables efficient inference. In particular for Gaussian graphical models, such representation is encoded by the zero pattern of the precision matrix (i.e., inverse covariance). Existing approaches to learning Gaussian graphical models often leverage the framework of penalized likelihood, and therefore suffer from the issue of regularization selection. In this paper, we address the structure learning problem of Gaussian graphical models from a variational Bayesian perspective. Specifically, sparse promoting priors are imposed on the off-diagonal elements of the precision matrix. We then approximate the posterior distribution of the precision matrix by a Wishart distribution using the framework of variational Bayes, and derive efficient natural gradient based algorithms to learn the model. We consider both monoscale and multiscale graphical models. Numerical results show that the proposed method can learn sparse graphs that can reliably describe the data in an automated fashion.