Register Automata for Malware Specification.

With the huge impact that internet is having in our daily life, it is becoming urgent to have efficient malware detection techniques. In this paper, we present a new approach to perform malware detection. We use register automata to describe malware specifications, and pushdown systems to model the program. This allows to keep track of both the program’s stack and the values of the registers. Indeed, both the stack and the registers are needed to have precise malware specifications. To check whether the program contains some malicious behavior, we perform a kind of product between the pushdown system and the register automaton describing the malicious behaviors. Whether the program is malicious or not is then reduced to reachability checking in pushdown systems. We implemented our techniques in a prototype and obtained encouraging preliminary results.