AutoCombo: Automatic Malware Signature Generation Through Combination Rule Mining

ABSTRACTMalware detection is an essential step in building trustworthy computer systems. Signature-based detection detects a sample as malware if the sample data match or contain a pre-stored malware signature. Among all detection methods that malware experts are constantly exploring, signature-based malware detection is indispensable, due to its simplicity, explainability and efficiency. Malware signatures could have various formats, for example, a substring, a subsequence, or a combination rule. A combination rule signature could be viewed as a fixed set of properties, each of which describes some characteristic of an analyzed sample. Although security experts have dedicated many efforts to extract meaningful features from samples, the step of signature generation from the features has been rather ad hoc and time-consuming. This paper focuses on the generation of combination rule signatures. We abstract and formally define the problem of combination rule malware signature generation, followed by a systematic study towards an effective and efficient implementation. Inspired by classic frequent itemsets mining solutions, the proposed AutoCombo approach is greedy but also complete. It generates higher quality signatures first, but is also able to traverse all possible property combinations for a complete generation. Further optimizations and future research potential are also discussed. The proposed approach is currently in use to assist the analysis for millions of files per day in a large security company. Our evaluation results using large-scale production data have also shown its efficacy. With the release of over 10 million real production records as well as our exploratory code, we hope this initial study could draw AI experts' attention and advance the research even further in this field.