VulEval: Towards Repository-Level Evaluation of Software Vulnerability Detection
arxiv(2024)
摘要
Deep Learning (DL)-based methods have proven to be effective for software
vulnerability detection, with a potential for substantial productivity
enhancements for detecting vulnerabilities. Current methods mainly focus on
detecting single functions (i.e., intra-procedural vulnerabilities), ignoring
the more complex inter-procedural vulnerability detection scenarios in
practice. For example, developers routinely engage with program analysis to
detect vulnerabilities that span multiple functions within repositories. In
addition, the widely-used benchmark datasets generally contain only
intra-procedural vulnerabilities, leaving the assessment of inter-procedural
vulnerability detection capabilities unexplored.
To mitigate the issues, we propose a repository-level evaluation system,
named VulEval, aiming at evaluating the detection performance of
inter- and intra-procedural vulnerabilities simultaneously. Specifically,
VulEval consists of three interconnected evaluation tasks: (1)
Function-Level Vulnerability Detection, aiming at detecting intra-procedural
vulnerability given a code snippet; (2) Vulnerability-Related
Dependency Prediction, aiming at retrieving the most relevant dependencies
from call graphs for providing developers with explanations about the
vulnerabilities; and (3) Repository-Level Vulnerability Detection,
aiming at detecting inter-procedural vulnerabilities by combining with the
dependencies identified in the second task. VulEval also consists of a
large-scale dataset, with a total of 4,196 CVE entries, 232,239 functions, and
corresponding 4,699 repository-level source code in C/C++ programming
languages. Our analysis highlights the current progress and future directions
for software vulnerability detection.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要