PaPr: Training-Free One-Step Patch Pruning with Lightweight ConvNets for Faster Inference
arxiv(2024)
摘要
As deep neural networks evolve from convolutional neural networks (ConvNets)
to advanced vision transformers (ViTs), there is an increased need to eliminate
redundant data for faster processing without compromising accuracy. Previous
methods are often architecture-specific or necessitate re-training, restricting
their applicability with frequent model updates. To solve this, we first
introduce a novel property of lightweight ConvNets: their ability to identify
key discriminative patch regions in images, irrespective of model's final
accuracy or size. We demonstrate that fully-connected layers are the primary
bottleneck for ConvNets performance, and their suppression with simple weight
recalibration markedly enhances discriminative patch localization performance.
Using this insight, we introduce PaPr, a method for substantially pruning
redundant patches with minimal accuracy loss using lightweight ConvNets across
a variety of deep learning architectures, including ViTs, ConvNets, and hybrid
transformers, without any re-training. Moreover, the simple early-stage
one-step patch pruning with PaPr enhances existing patch reduction methods.
Through extensive testing on diverse architectures, PaPr achieves significantly
higher accuracy over state-of-the-art patch reduction methods with similar FLOP
count reduction. More specifically, PaPr reduces about 70
in videos with less than 0.8
which is a 15
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要