Parallel big data processing system for security monitoring in Internet of Things networks

Igor Kotenko
1, 2+, Igor Saenko1, 2, and Alexey Kushnerevich1, 2
 

1St. Petersburg Institute for Informatics and Automation (SPIIRAS)
39, 14-th Liniya, Saint-Petersburg, 199178, Russia

 

2St. Petersburg National Research University of Information Technologies, Mechanics and Optics (ITMO University), 49, Kronverkskiy prospekt, Saint-Petersburg, 197101, Russia
{ivkote, ibsaen, kushnerevich}@comsec.spb.ru

 

Abstract

Nowadays, the Internet of Things (IoT) networks are increasingly used in many areas. At the same time, the approach connected with the implementation of the network security monitoring system is of particular relevance for the protection of IoT networks from threats. Due to the peculiarities for construction and operation of IoT networks, the use of traditional protection systems for IoT is difficult or impossible. One of such features is the need to analyze very large amounts of data in real time and with minimal computational cost. Given the limited computing capabilities of IoT networks, we propose the architecture of a big data distributed parallel processing system based on Hadoop and Spark software platforms. The issues related to the implementation of this system and its main components are also considered. The results of an experimental evaluation of the system performance are discussed. They confirm the conclusion about its high efficiency. A comparative evaluation of the implemented systems on Hadoop and Spark platforms is conducted.

 

Keywords: Complex Event Processing, Hadoop, Spark, Security Monitoring

 

+: Corresponding author: Igor Kotenko
Laboratory of Computer Security Problems, St. Petersburg Institute for Informatics and Automation (SPIIRAS), 39, 14-th Liniya, Saint-Petersburg, 199178, Russia, Tel: +7(812) 328-71-81, E-Mail: ivkote@comsec.spb.ru,
Web: http://www.comsec.spb.ru/

 

Journal of Wireless Mobile Networks, Ubiquitous Computing, and Dependable Applications (JoWUA)

Vol. 8, No. 4, pp. 60-74, December 2017 [pdf]