Spectral Anomaly Detection in Very Large Graphs: Models, Noise, and Computational Complexity

Abstract: Anomaly detection in massive networks has numerous theoretical and computational challenges, especially as the behavior to be detected becomes small in comparison to the larger network. This presentation focuses on recent results in three key technical areas, specifically geared toward spectral methods for detection. We first discuss recent models for network behavior, and how their structure can be exploited for efficient computation of the principal eigenspace of the graph. In addition to the stochasticity of background activity, a graph of interest may be observed through a noisy or imperfect mechanism, which may hinder the detection process. A few simple noise models are discussed, and we demonstrate the ability to fuse multiple corrupted observations and recover detection performance. Finally, we discuss the challenges in scaling the spectral algorithms to large-scale high-performance computing systems, and present preliminary recommendations to achieve good performance with current parallel eigensolvers.