Experimental Comparison of Machine Learning Models in Malware Packing Detection

Recently , malware is widely distributed by combining recent technologies such as packing, encoding and obfuscation to bypass anti-virus software. These kinds of technologies allow malware to survive longer, infect various computers and devices for longer periods of time, create a number of mutated malware, and make experts spend longer to analyze malware. Packers disrupt the reverse engineering process, making it difficult for security researchers to analyze new or unknown malware. Thus, we need to analyze as many malware as possible by first detecting the packed malware and analyzing not-packed malware, and then unpack the packed malware. Previously, the packing detection methods were based on mainly signature and entropy detection. However, these methods have increased the undetected rate with the appearance of custom packers. Due to these problems, there have been many research efforts on machine learning-based malware packing detection and classification. In this paper, we present an extensive experimental comparison of these machine learning-based algorithms. In particular, we extract a total of 13 important features and considers eight machine learning algorithms to detect the packing of malware. Experimental results show that we can also detect well malware packed by custom packers which did not studied in previous studies.