Tripod: Three Complementary Inductive Biases for Disentangled Representation Learning
arxiv(2024)
摘要
Inductive biases are crucial in disentangled representation learning for
narrowing down an underspecified solution set. In this work, we consider
endowing a neural network autoencoder with three select inductive biases from
the literature: data compression into a grid-like latent space via
quantization, collective independence amongst latents, and minimal functional
influence of any latent on how other latents determine data generation. In
principle, these inductive biases are deeply complementary: they most directly
specify properties of the latent space, encoder, and decoder, respectively. In
practice, however, naively combining existing techniques instantiating these
inductive biases fails to yield significant benefits. To address this, we
propose adaptations to the three techniques that simplify the learning problem,
equip key regularization terms with stabilizing invariances, and quash
degenerate incentives. The resulting model, Tripod, achieves state-of-the-art
results on a suite of four image disentanglement benchmarks. We also verify
that Tripod significantly improves upon its naive incarnation and that all
three of its "legs" are necessary for best performance.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要