Performing Bayesian Network Inference Using Amortized Region Approximation with Graph Factorization

Exact inference for large, directed graphical models, also known as Bayesian networks (BNs), can be intractable as the space complexity grows exponentially in the tree-width of the model. Approximate inference, such as generalized belief propagation (GBP), is used instead. GBP treats inference as the Bethe/Kikuchi energy function optimization problem. The solution is found using iterative message passing, which is inefficient and convergent problematic. Recent progress on amortized technique for GBP is an attractive alternative solution that can optimize the Bethe/Kikuchi energy function using (deep) neural networks, requiring no message passing. Despite being efficient, the amortized technique for GBP is applied to undirected graphical models with specific structures and factors, with no guarantee of the approximation quality. This is because the energy function to be amortized is defined by a region (or factor) graph that is ad hoc and difficult to construct to ensure sensible approximations. This paper proposes a new amortized GBP algorithm applied to BN for efficient inference. The proposed algorithm is composed of the following: (i) a new pairwise conversion (PWC) algorithm that converts all the conditional probability distributions in the BN into pairwise factors to facilitate efficient region graph constructions; (ii) following PWC, an improved loop structured region graph (LSRG) algorithm was derived to generate a valid region graph satisfying desired regional properties; and (iii) the energy function defined by the proposed PWC-LSRG region graph can be directly amortized using (deep) neural networks to ensure sensible approximations. Empirical studies show that the proposed amortized PWC-LSRG algorithm is of practical use and significantly improves convergence and efficiency compared to conventional algorithms.