iTES: Integrated Testing and Evaluation System for Software Vulnerability Detection Methods

To find software vulnerabilities using software vulnerability detection technology is an important way to ensure the system security. Existing software vulnerability detection methods have some limitations as they can only play a certain role in some specific situations. To accurately analyze and evaluate the existing vulnerability detection methods, an integrated testing and evaluation system (iTES) is designed and implemented in this paper. The main functions of the iTES are:(1) Vulnerability cases with source codes covering common vulnerability types are collected automatically to form a vulnerability cases library; (2) Fourteen methods including static and dynamic vulnerability detection are evaluated in iTES, involving the Windows and Linux platforms; (3) Furthermore, a set of evaluation metrics is designed, including accuracy, false positive rate, utilization efficiency, time cost and resource cost. The final evaluation and test results of iTES have a good guiding significance for the selection of appropriate software vulnerability detection methods or tools according to the actual situation in practice.