Eliminating Information Leakage in Hard Concept Bottleneck Models with Supervised, Hierarchical Concept Learning
CoRR(2024)
摘要
Concept Bottleneck Models (CBMs) aim to deliver interpretable and
interventionable predictions by bridging features and labels with
human-understandable concepts. While recent CBMs show promising potential, they
suffer from information leakage, where unintended information beyond the
concepts (either when concepts are represented with probabilities or binary
states) are leaked to the subsequent label prediction. Consequently, distinct
classes are falsely classified via indistinguishable concepts, undermining the
interpretation and intervention of CBMs.
This paper alleviates the information leakage issue by introducing label
supervision in concept predication and constructing a hierarchical concept set.
Accordingly, we propose a new paradigm of CBMs, namely SupCBM, which achieves
label predication via predicted concepts and a deliberately-designed
intervention matrix. SupCBM focuses on concepts that are mostly relevant to the
predicted label and only distinguishes classes when different concepts are
presented. Our evaluations show that SupCBM outperforms SOTA CBMs over diverse
datasets. It also manifests better generality across different backbone models.
With proper quantification of information leakage in different CBMs, we
demonstrate that SupCBM significantly reduces the information leakage.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要