Using AI/ML to gain situational understanding from passive network observations

Abstract: The data available in the network traffic fromany Government building contains a significant amount ofinformation. An analysis of the traffic can yield insightsand situational understanding about what is happening inthe building. However, the use of traditional network packet inspection, either deep or shallow, is useful for only a limited understanding of the environment, with applicability limited to some aspects of network and security management. If weuse AI/ML based techniques to understand the network traffic, we can gain significant insights which increase our situational awareness of what is happening in the environment.At IBM, we have created a system which uses a combination of network domain knowledge and machine learning techniques to convert network traffic into actionable insights about the on premise environment. These insights include characterization of the communicating devices, discovering unauthorized devices that may violate policy requirements, identifying hidden components and vulnerability points, detecting leakage of sensitive information, and identifying the presence of people and devices.In this paper, we will describe the overall design of this system, the major use-cases that have been identified for it, and the lessons learnt when deploying this system for some of those use-cases