Feature Selection as Deep Sequential Generative Learning
arxiv(2024)
摘要
Feature selection aims to identify the most pattern-discriminative feature
subset. In prior literature, filter (e.g., backward elimination) and embedded
(e.g., Lasso) methods have hyperparameters (e.g., top-K, score thresholding)
and tie to specific models, thus, hard to generalize; wrapper methods search a
feature subset in a huge discrete space and is computationally costly. To
transform the way of feature selection, we regard a selected feature subset as
a selection decision token sequence and reformulate feature selection as a deep
sequential generative learning task that distills feature knowledge and
generates decision sequences. Our method includes three steps: (1) We develop a
deep variational transformer model over a joint of sequential reconstruction,
variational, and performance evaluator losses. Our model can distill feature
selection knowledge and learn a continuous embedding space to map feature
selection decision sequences into embedding vectors associated with utility
scores. (2) We leverage the trained feature subset utility evaluator as a
gradient provider to guide the identification of the optimal feature subset
embedding;(3) We decode the optimal feature subset embedding to
autoregressively generate the best feature selection decision sequence with
autostop. Extensive experimental results show this generative perspective is
effective and generic, without large discrete search space and expert-specific
hyperparameters.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要