Arithmetic Feature Interaction Is Necessary for Deep Tabular Learning
CoRR(2024)
摘要
Until recently, the question of the effective inductive bias of deep models
on tabular data has remained unanswered. This paper investigates the hypothesis
that arithmetic feature interaction is necessary for deep tabular learning. To
test this point, we create a synthetic tabular dataset with a mild feature
interaction assumption and examine a modified transformer architecture enabling
arithmetical feature interactions, referred to as AMFormer. Results show that
AMFormer outperforms strong counterparts in fine-grained tabular data modeling,
data efficiency in training, and generalization. This is attributed to its
parallel additive and multiplicative attention operators and prompt-based
optimization, which facilitate the separation of tabular samples in an extended
space with arithmetically-engineered features. Our extensive experiments on
real-world data also validate the consistent effectiveness, efficiency, and
rationale of AMFormer, suggesting it has established a strong inductive bias
for deep learning on tabular data. Code is available at
https://github.com/aigc-apps/AMFormer.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要