Hunting cyberattacks: experience from the real backbone network

Mikołaj Komisarek^1,2, Marek Pawlicki^1,2+, Mikołaj Kowalski³, Adrian Marzecki³,
Rafał Kozik^1,2 and Michał Choraś^2,4

¹ITTI Sp. z o.o., Poznań, Poland
{m.komisarek, mpawlicki}@itti.com.pl

²Bydgoszcz University of Science and Technology, Bydgoszcz, Poland

³Orange, Warsaw, Poland

⁴FernUniversität in Hagen, Germany

Abstract

Computer networks are exposed to attacks which have been increasingly more effective. To counter these emerging threats, researchers and security engineers work relentlessly to keep up with the arms race and offer improvements to intrusion detection systems as soon as possible. In the recent years, there has been an increase in the proliferation of systems employing deep learning and machine learning algorithms to detect suspicious patterns more effectively. To leverage AI effectively in a real-world scenario of intrusion detection, a scalable stream processing system to feed the detection algorithms with data samples in a timely and reliable manner has to be established. In this paper, two use cases of intrusion detection are presented. The first one shows a real-world example of data collected by one the largest telecom operators - ORANGE. The data was gathered for the SIMARGL project. The second use case presents the experiments and results of intrusion detection based on the Netflow scheme. The paper also proposes a scalable streaming architecture based on the Apache Spark and Apache Kafka technologies. The results of the evaluation of the effectiveness of detecting malicious behavior in network packets using several machine learning techniques in conjunction with the stream processing framework are presented.

Keywords: Machine learning, Stream processing, Intrusion detection, Network data

+: Corresponding author: Marek Pawlicki
ITTI Sp. z o.o., Rubież 46, 61-612 Poznań, Poland, Tel: +48 61/ 622 69 85, Email: mpawlicki@itti.com.pl,
Web: https://www.itti.com.pl

Journal of Wireless Mobile Networks, Ubiquitous Computing, and Dependable Applications (JoWUA)
Vol. 13, No. 2, pp. 128-146, June 2022 [pdf]

Received: November 22, 2021; Accepted: May 18, 2022; Published: June 30, 2022

DOI: 10.22667/JOWUA.2022.06.30.128