Entropic Regularization of Mixed-membership Network Models using Pseudo-observations

Mixed-membership network models permit a node in a graph to take on different latent roles in different interactions. However, while mixed-membership block models often do out-perform classical network models, the actual degree of mixed-membership in many graphs is small, with nodes usually taking on only a handful of many possible roles. We thus present a novel slightly mixed membership stochastic block model, in which the degree of mixed-membership can be controlled. This model is based on a novel regularization method, where the generative model is extended to include variables that measure aggregate statistics (e.g., the entropy of the distribution of latent roles assigned to nodes), as well as “noisy copies” of these aggregates. We then pseudo-observe a desired value for the noisy copies, which has the effect of penalizing models whose aggregates differ greatly from the desired value. Here we demonstrate two applications of this technique: one which encourages slightly-mixed membership, and one which encourages balanced clusters. Experiments with several networks from different domains show that the new models improve performance, as measured by link perplexity and cluster recovery.