Learning Doubly Intractable Latent Variable Models via Score Matching

Most current approaches for learning latent variable models, such as variational methods, require access to a normalized joint distribution. However, for models that do not belong to the standard, widely-studied parametric classes or that are derived from an undirected graphical model, the normalizer is often not readily available. Here we generalize the score matching approach [1] to learn a wide class of latent variable models based on joint exponential family (or maximum-entropy) distributions with arbitrary sufficient statistic vectors. We derive a stochastic gradient based optimization scheme that does not depend on the computation of normalizing constants for either of the joint or the posterior densities.