Representing Unordered Data Using Complex-Weighted Multiset Automata.
ICML(2020)
摘要
Unordered, variable-sized inputs arise in many settings acrossmultiple fields. The ability for set- and multiset-oriented neuralnetworks to handle this type of input has been the focus of muchwork in recent years. We propose to represent multisets usingcomplex-weighted multiset automata and show how themultiset representations of certain existing neural architecturescan be viewed as special cases of ours. Namely, (1) we provide a newtheoretical and intuitive justification for the Transformer model’srepresentation of positions using sinusoidal functions, and (2) weextend the DeepSets model to use complex numbers, enabling it tooutperform the existing model on an extension of one of their tasks.
更多查看译文
关键词
unordered data,complex-weighted
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络