Time and Memory Efficient Parallel Algorithm for Structural Graph Summaries and two Extensions to Incremental Summarization and $k$-Bisimulation for Long $k$-Chaining

Abstract: We developed a flexible parallel algorithm for graph summarization based on vertex-centric programming and parameterized message passing. The base algorithm supports infinitely many structural graph summary models defined in a formal language. An extension of the parallel base algorithm allows incremental graph summarization. In this paper, we prove that the incremental algorithm is correct and show that updates are performed in time $\mathcal{O}(\Delta \cdot d^k)$, where $\Delta$ is the number of additions, deletions, and modifications to the input graph, $d$ the maximum degree, and $k$ is the maximum distance in the subgraphs considered. Although the iterative algorithm supports values of $k>1$, it requires nested data structures for the message passing that are memory-inefficient. Thus, we extended the base summarization algorithm by a hash-based messaging mechanism to support a scalable iterative computation of graph summarizations based on $k$-bisimulation for arbitrary $k$. We empirically evaluate the performance of our algorithms using benchmark and real-world datasets. The incremental algorithm almost always outperforms the batch computation. We observe in our experiments that the incremental algorithm is faster even in cases when $50\%$ of the graph database changes from one version to the next. The incremental computation requires a three-layered hash index, which has a low memory overhead of only $8\%$ ($\pm 1\%$). Finally, the incremental summarization algorithm outperforms the batch algorithm even with fewer cores. The iterative parallel $k$-bisimulation algorithm computes summaries on graphs with over $10$M edges within seconds. We show that the algorithm processes graphs of $100+\,$M edges within a few minutes while having a moderate memory consumption of $<150$ GB. For the largest BSBM1B dataset with 1 billion edges, it computes $k=10$ bisimulation in under an hour.