Anomaly detection framework for highly scattered and dynamic data on large-scale networks using AWS

Network anomaly detection is crucial for securing computer networks and communications. However, handling highly scattered data in large-scale, dynamic networks poses challenges, leading to increased false positives and false negatives in anomaly identification. To address this, a new anomaly detection framework employing ensemble learning techniques was introduced. This framework uses decision trees for scattered data and gradient-boosting algorithms for accuracy in dynamic network behaviors. It successfully identifies known and unknown abnormalities while ensuring scalability and reducing false positives and negatives. The framework was tested on CICIDS2017, NSLKDD, and KDDCUP99 datasets, achieving outstanding accuracy rates of 100