A Vulnerability Analysis and Prediction Framework

As the world approaches a state of greater dependence on technology, many products face increasing threats from malicious attackers who are attempting to take advantage of vulnerabilities in software design. Most of the known vulnerability information is already aggregated, stored in text format, and readily accessible to the public, making such an aggregated database a prime corpus for analysis using data mining methods. Multiple research efforts have been launched in which individual aspects of such cyber-security corpora were analyzed to create taxonomies, assess vulnerability impact, and improve vulnerability detection. However, minimal effort has been committed to analyze cyber-security corpora to explore correlations between vulnerabilities, to study the evolution of a vulnerability from its genesis, and to predict vulnerabilities using multi-faceted algorithms. In this paper, we propose an integrated data mining framework to automatically describe how vulnerabilities develop over time and detect the evolution of a specific vulnerability. Additionally, our framework has a predictive functionality that can be used to predict specific vulnerabilities or to estimate future appearance probabilities of vulnerability groups. In our framework, we use (1) a Topically Supervised Evolution Model (TSEM) that can discover temporal themes from a text corpus, (2) a diffusion-based storytelling technique that sifts through past vulnerability reports to describe how a current vulnerability threat evolved, and (3) several prediction models that use features from a cyber-security corpus to predict vulnerabilities. A series of experiments demonstrate that the proposed framework can not only discover evolutionary patterns in today's most pressing vulnerabilities with a high degree of precision, but it can also predict vulnerabilities with impressive accuracy. As case studies, we also explore the development of vulnerabilities in certain products, providing a unique insight into the correspondence between seemingly unrelated vulnerabilities and the impact of that correspondence on overall software security.