Comprehensive Survey of Model Compression and Speed up for Vision Transformers
arxiv(2024)
摘要
Vision Transformers (ViT) have marked a paradigm shift in computer vision,
outperforming state-of-the-art models across diverse tasks. However, their
practical deployment is hampered by high computational and memory demands. This
study addresses the challenge by evaluating four primary model compression
techniques: quantization, low-rank approximation, knowledge distillation, and
pruning. We methodically analyze and compare the efficacy of these techniques
and their combinations in optimizing ViTs for resource-constrained
environments. Our comprehensive experimental evaluation demonstrates that these
methods facilitate a balanced compromise between model accuracy and
computational efficiency, paving the way for wider application in edge
computing devices.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要