Accelerating Unstructured SpGEMM using Structured In-situ Computing

Abstract: Sparse matrix-matrix multiplication (SpGEMM) is a critical kernel widely employed in machine learning and graph algorithms. However, real-world matrices' high sparsity makes SpGEMM memory-intensive. In-situ computing offers the potential to accelerate memory-intensive applications through high bandwidth and parallelism. Nevertheless, the irregular distribution of non-zeros renders SpGEMM a typical unstructured software. In contrast, in-situ computing platforms follow a fixed calculation manner, making them structured hardware. The mismatch between unstructured software and structured hardware leads to sub-optimal performance of current solutions. In this paper, we propose SPLIM, a novel in-situ computing SpGEMM accelerator. SPLIM involves two innovations. First, we present a novel computation paradigm that converts SpGEMM into structured in-situ multiplication and unstructured accumulation. Second, we develop a unique coordinates alignment method utilizing in-situ search operations, effectively transforming unstructured accumulation into high parallel searching operations. Our experimental results demonstrate that SPLIM achieves 275.74$\times$ performance improvement and 687.19$\times$ energy saving compared to NVIDIA RTX A6000 GPU.