One-Shot Sensitivity-Aware Mixed Sparsity Pruning for Large Language Models
ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)(2023)
摘要
Various Large Language Models(LLMs) from the Generative Pretrained
Transformer(GPT) family have achieved outstanding performances in a wide range
of text generation tasks. However, the enormous model sizes have hindered their
practical use in real-world applications due to high inference latency.
Therefore, improving the efficiencies of LLMs through quantization, pruning,
and other means has been a key issue in LLM studies. In this work, we propose a
method based on Hessian sensitivity-aware mixed sparsity pruning to prune LLMs
to at least 50
sparsity adaptively based on sensitivity, allowing us to reduce pruning-induced
error while maintaining the overall sparsity level. The advantages of the
proposed method exhibit even more when the sparsity is extremely high.
Furthermore, our method is compatible with quantization, enabling further
compression of LLMs.
更多查看译文
关键词
model compression,sparsity pruning,large language models,mixed sparsity
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要