Transformer Neural Autoregressive Flows
CoRR(2024)
摘要
Density estimation, a central problem in machine learning, can be performed
using Normalizing Flows (NFs). NFs comprise a sequence of invertible
transformations, that turn a complex target distribution into a simple one, by
exploiting the change of variables theorem. Neural Autoregressive Flows (NAFs)
and Block Neural Autoregressive Flows (B-NAFs) are arguably the most perfomant
members of the NF family. However, they suffer scalability issues and training
instability due to the constraints imposed on the network structure. In this
paper, we propose a novel solution to these challenges by exploiting
transformers to define a new class of neural flows called Transformer Neural
Autoregressive Flows (T-NAFs). T-NAFs treat each dimension of a random variable
as a separate input token, using attention masking to enforce an autoregressive
constraint. We take an amortization-inspired approach where the transformer
outputs the parameters of an invertible transformation. The experimental
results demonstrate that T-NAFs consistently match or outperform NAFs and
B-NAFs across multiple datasets from the UCI benchmark. Remarkably, T-NAFs
achieve these results using an order of magnitude fewer parameters than
previous approaches, without composing multiple flows.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要