n-Grams exclusion and inclusion filter for intrusion detection in Internet of Energy big data systems

The advent of Internet of Energy (IoE) and the seamless integration of grid operators, power generators, distributors, sensors, and end users promise more efficient use of energy. However, the IoE will inherit the vulnerabilities from all of the integrated systems, and this raises concerns for trust and privacy. The evolving complexity and increased speed of network-based attacks emphasizes the need for an efficient intrusion detection system. Consequently, with the emergence of new attacks and the increasing number of signatures, traditional signature-based intrusion detection systems cannot both sift through big data and meet high network speeds. Detection performance severely deteriorates when matching hundreds of gigabits per second to the growing number of attack signatures. Given that pattern matching takes up to 60% of the overall intrusion detection time, this paper presents a new and fast software-based pattern matching system, Exscind. It proposes an exclusion-inclusion filter to preclude clean traffic before having to do expensive pattern matching. Additionally, if the traffic is malicious, the system only matches against a subset of signatures that have a high probability of being a match. We extensively evaluate the system's performance and conclude that using 6-grams signature prefix provides the best speedup and memory consumption with negligible false positives and linear scaling. We report a best-case speedup of 6.5 times for normal traffic and 1.53 times for the worst possible scenario. For best-case normal traffic, Exscind skips pattern matching for 98.36% of the packets.