Parallelizable Sampling of Markov Random Fields

Markov Random Fields (MRFs) are an im- portant class of probabilistic models which are used for density estimation, classifica- tion, denoising, and for constructing Deep Belief Networks. Every application of an MRF requires addressing its inference prob- lem, which can be done using deterministic inference methods or using stochastic Markov Chain Monte Carlo methods. In this paper we introduce a new Markov Chain transition operator that updates all the variables of a pairwise MRF in parallel by using auxiliary Gaussian variables. The proposed MCMC operator is extremely simple to implement and to parallelize. This is achieved by a formal equivalence result between arbitrary pairwise MRFs and a particular type of Re- stricted Boltzmann Machine. This result also implies that the later can be learned in place of the former without any loss of modeling power, a possibility we explore in experi- ments.