Anomalous Traffic Detection System for Enterprise using Elastic stack with Machine Learning - R Biradar, Ruchita; Nagaraja, G.S (International Journal of Computer Sciences and Engineering, 2021)
The logs in a network are not bound to be perfect perpetually. The behavior of the network traffic is bound to deviate from the expected one sometimes and when that occurs, the traffic is said to be anomalous. Anomalous traffic can be problematic for various reasons, be it external attacks, or transfer of outdated data or even serving customers for networking companies. When the network size is at a large scale, this becomes an even bigger problem to tackle. The anomaly detection systems currently in place are either trained with aging datasets or are not able to handle large loads efficiently. Hence arises the need for a scalable solution which can provide security to a network by detecting anomalies in it and alerting with quick response when an anomaly occurs by learning from its past behavior. The paper offers an end-to- end solution for the introduction of an anomaly detection system using machine learning into an enterprise environment, right from the collection of logs to the generation of alerts, effectively. This is implemented with an infrastructure that includes Elasticsearch, Logstash and Kibana along with the added feature of Machine Learning