SparseCoder: Identifier-Aware Sparse Transformer for File-Level Code Summarization
CoRR(2024)
摘要
Code summarization aims to generate natural language descriptions of source
code, facilitating programmers to understand and maintain it rapidly. While
previous code summarization efforts have predominantly focused on method-level,
this paper studies file-level code summarization, which can assist programmers
in understanding and maintaining large source code projects. Unlike
method-level code summarization,file-level code summarization typically
involves long source code within a single file, which makes it challenging for
Transformer-based models to understand the code semantics for the maximum input
length of these models is difficult to set to a large number that can handle
long code input well, due to the quadratic scaling of computational complexity
with the input sequence length. To address this challenge, we propose
SparseCoder, an identifier-aware sparse transformer for effectively handling
long code sequences. Specifically, the SparseCoder employs a sliding window
mechanism for self-attention to model short-term dependencies and leverages the
structure message of code to capture long-term dependencies among source code
identifiers by introducing two types of sparse attention patterns named global
and identifier attention. To evaluate the performance of SparseCoder, we
construct a new dataset FILE-CS for file-level code summarization in Python.
Experimental results show that our SparseCoder model achieves state-of-the-art
performance compared with other pre-trained models, including full
self-attention and sparse models. Additionally, our model has low memory
overhead and achieves comparable performance with models using full
self-attention mechanism.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要