DeVAIC: A Tool for Security Assessment of AI-generated Code
arxiv(2024)
摘要
Context: AI code generators are revolutionizing code writing and software
development, but their training on large datasets, including potentially
untrusted source code, raises security concerns. Furthermore, these generators
can produce incomplete code snippets that are challenging to evaluate using
current solutions. Objective: This research work introduces DeVAIC (Detection
of Vulnerabilities in AI-generated Code), a tool to evaluate the security of
AI-generated Python code, which overcomes the challenge of examining incomplete
code. Method: We followed a methodological approach that involved gathering
vulnerable samples, extracting implementation patterns, and creating regular
expressions to develop the proposed tool. The implementation of DeVAIC includes
a set of detection rules based on regular expressions that cover 35 Common
Weakness Enumerations (CWEs) falling under the OWASP Top 10 vulnerability
categories. Results: We utilized four popular AI models to generate Python
code, which we then used as a foundation to evaluate the effectiveness of our
tool. DeVAIC demonstrated a statistically significant difference in its ability
to detect security vulnerabilities compared to the state-of-the-art solutions,
showing an F1 Score and Accuracy of 94
cost of 0.14 seconds per code snippet, on average. Conclusions: The proposed
tool provides a lightweight and efficient solution for vulnerability detection
even on incomplete code.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要