Push Quantization-Aware Training Toward Full Precision Performances via Consistency Regularization
CoRR(2024)
摘要
Existing Quantization-Aware Training (QAT) methods intensively depend on the
complete labeled dataset or knowledge distillation to guarantee the
performances toward Full Precision (FP) accuracies. However, empirical results
show that QAT still has inferior results compared to its FP counterpart. One
question is how to push QAT toward or even surpass FP performances. In this
paper, we address this issue from a new perspective by injecting the vicinal
data distribution information to improve the generalization performances of QAT
effectively. We present a simple, novel, yet powerful method introducing an
Consistency Regularization (CR) for QAT. Concretely, CR assumes that augmented
samples should be consistent in the latent feature space. Our method
generalizes well to different network architectures and various QAT methods.
Extensive experiments demonstrate that our approach significantly outperforms
the current state-of-the-art QAT methods and even FP counterparts.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要