Analysis of Anomalous Behavior in Network Systems Using Deep Reinforcement Learning with CNN Architecture

Abstract: In order to gain access to networks, different types of intrusion attacks have been designed, and the attackers are working on improving them. Computer networks have become increasingly important in daily life due to the increasing reliance on them. In light of this, it is quite evident that algorithms with high detection accuracy and reliability are needed for various types of attacks. The purpose of this paper is to develop an intrusion detection system that is based on deep reinforcement learning. Based on the Markov decision process, the proposed system can generate informative representations suitable for classification tasks based on vast data. Reinforcement learning is considered from two different perspectives, deep Q learning, and double deep Q learning. Different experiments have demonstrated that the proposed systems have an accuracy of $99.17\%$ over the UNSW-NB15 dataset in both approaches, an improvement over previous methods based on contrastive learning and LSTM-Autoencoders. The performance of the model trained on UNSW-NB15 has also been evaluated on BoT-IoT datasets, resulting in competitive performance.