Pairwise Decomposition of Directed Graphic Models for Performing Amortized Approximate Inference.

Exact inference for large directed graphic models (known as Bayesian networks (BN)) is difficult as the space complexity grows exponentially in the tree-width of the model. Approximate inference, such as generalized belief propagation (GBP), is used instead. GBP treats inference as an energy function optimization problem. The solution is found using iterative message passing, which is inefficient and convergent problematic. Recent progress on amortized technique for GBP is an attractive alternative solution that can optimize the energy function using (deep) neural networks, requiring no message passing. Despite being efficient, the amortized technique for GBP is so far only applied to undirected graphic models with specific structures and factors, with no guarantee of the approximation quality for general models. This is because the energy function to be amortized is defined by a region graph that is ad-hoc and difficult to construct. To ensure the amortized technique for GBP is applied to BN inference for practical use, we propose (i) A new pairwise conversion (PWC) algorithm converts all the conditional probability distributions in the BN into pairwise factors to facilitate neural network parameterization and the region graph construction. (ii) An improved loop structured region graph (LSRG) algorithm to generate a valid region graph satisfying desired region properties. (iii) The energy function defined by the PWC-LSRG region graph can be directly amortized using (deep) neural networks and yield sensible approximations. Experiments show that the proposed amortized PWC-LSRG algorithm improves significantly in accuracy and efficiency compared to conventional algorithms.