Sparsity-Aware Roofline Models for Sparse Matrix-Matrix Multiplication

Abstract: Sparse matrix-dense matrix multiplication (SpMM) is a critical kernel in scientific computing, graph analytics, and machine learning, whose performance is often constrained by memory bandwidth. In this work, we investigate the applicability and limitations of roofline modeling for SpMM by explicitly accounting for the impact of matrix sparsity structure on arithmetic intensity and attainable performance. We evaluate three SpMM implementations: Compressed Sparse Row (CSR), Compressed Sparse Blocks (CSB), and Intel's Math Kernel Library (MKL). Each implementation was tested using large-scale matrices from the SuiteSparse collection and grouped by sparsity pattern, including block-structured, banded (diagonal), scale-free, and uniformly random matrices. We derive sparsity-aware roofline models that incorporate memory traffic, cache locality, and blocking behavior, and demonstrate that a single model is insufficient to accurately predict performance across diverse structures. Experiments were conducted on an AMD-based Perlmutter compute node with a varying number of columns in the dense matrix. In particular, blocking and structured sparsity significantly alter effective arithmetic intensity. The results show that accurate roofline-based performance analysis of SpMM requires sparsity-aware modeling, and that data layout and blocking strategies must be evaluated in the context of matrix structure rather than through a single unified model.

• The paper introduces novel sparsity-aware roofline models that tailor arithmetic intensity formulations to specific matrix structures in SpMM.
• It develops distinct analytical models for random, diagonal/banded, blocked, and scale-free sparsity patterns, validated using SuiteSparse matrices and state-of-the-art implementations.
• The study demonstrates that performance optimization in SpMM requires aligning kernel selection with matrix structural characteristics to fully exploit memory bandwidth and cache locality.

Authoritative Summary of "Sparsity-Aware Roofline Models for Sparse Matrix-Matrix Multiplication" (2604.06637)

Introduction and Motivation

Sparse matrix–dense matrix multiplication (SpMM) is ubiquitous across scientific computing, graph analytics, and emerging deep learning workloads, especially graph neural networks (GNNs) and sparsified transformers. The performance of SpMM is fundamentally governed by both the structure of the sparse matrix and underlying hardware memory bandwidth. Classical roofline models provide a means to predict performance by linking arithmetic intensity (AI) and attainable throughput, yet fail to account for structural sparsity heterogeneity. This paper introduces rigorously derived sparsity-aware roofline models, incorporating explicit structural modeling of random, diagonal, block-structured, and scale-free sparsity patterns, validated against empirical SpMM results from SuiteSparse matrices and state-of-the-art implementations (CSR, CSB, Intel MKL).

Roofline Modeling and Matrix Structural Regimes

The work argues that a universal roofline model is inadequate for characterizing SpMM performance across diverse matrix structures. Instead, the authors propose distinct AI formulations tailored for four archetypal patterns:

• Random Sparsity: Assumes uniform distribution of nonzeros, minimal locality, maximal memory traffic per nonzero, yielding the lowest arithmetic intensity and a bandwidth-limited lower performance bound.
• Diagonal/Banded Sparsity: Concentration of nonzeros along diagonals or bands enables high locality and cache reuse in accessing $B$, producing elevated AI and tight theoretical upper bounds.
• Blocked Sparsity: Structural tiling and clustering, modeled via probabilistic estimation of block-column occupancy and cache retention, leads to intermediate, structure-dependent reuse and AI.
• Scale-Free Sparsity: Incorporates power-law degree distributions common in real-world graphs. High-degree (hub) rows enable disproportionate reuse, modeled analytically via edge-mass concentration, boosting attainable AI beyond random sparsity predictions.

These structural regimes fundamentally modulate effective memory traffic and attainable bandwidth-bound performance, as demonstrated by the distinct vertical lines in the Roofline plot marking each AI regime.

Figure 1: Bandwidth-bound Roofline comparison for representative matrices, quantifying how each sparsity-aware model predicts attainable SpMM performance.

Experimental Design and Implementation Analysis

The evaluation leverages large-scale matrices from SuiteSparse and synthetically generated random/diagonal structures, classified by the outlined sparsity patterns. Benchmarks span three SpMM implementations (CSR, CSB, MKL) on an AMD EPYC “Milan”–based Perlmutter compute node (64 threads, 122.6 GB/s measured DRAM bandwidth). Performance is measured across a sweep of dense matrix widths ($d = 1,4,16,64$).

Key technical findings include:

• Uniform Random Matrices: All implementations achieve lowest throughput; CSB approaches but does not reach the Roofline at higher $d$ due to minor cache locality.
• Diagonal/Banded Regimes: Real matrices underperform the theoretical roof due to deviations from ideal structure; banded roofline serves more as an upper bound.
• Block-Structured Sparsity: CSB achieves near-Roofline performance, confirming blocked locality assumptions.
• Scale-Free Sparsity: MKL and CSR outperform in low-$d$, owing to hub locality. CSB occasionally exceeds the bandwidth roof, signifying that its access patterns are serviced by higher effective bandwidth due to cache exploitation, not a violation of hardware limits.

The performance gap at large $d$ illustrates cache working set expansion: as $B$ and $C$ widen, cache locality deteriorates, and SpMM becomes strictly memory-bound for all formats.

Figure 1: Attained performance versus sparsity-aware bandwidth Roofline, showing measured and predicted limits across structural classes.

Implications and Limitations

The results demonstrate that structure-aware models are necessary for accurate SpMM performance prediction and optimization. Single roofline models systematically misrepresent attainable throughput on workloads with non-uniform sparsity. In practical terms, SpMM kernel selection and blocking/tile parameterization must be informed by the precise matrix structure to maximize memory subsystem exploitation.

Theoretical implications are significant: structural heterogeneity directly governs computational density and resource utilization. While the adopted models improve predictive capability, the approach still abstracts away detailed cache miss penalties and memory latency effects; further refinement is needed for cache-aware and hierarchical roofline extensions, and for modeling additional structural regimes (e.g., community matrices, irregular block sparsity).

The study also establishes that machine learning–based predictors (e.g., WISE [yesil23_wise], DDB [yesil22ddb]) can benefit from incorporation of explicit structural features in performance modeling, instead of relying solely on empirical kernel selection.

Future Directions

Prospective research trajectories include:

• Hierarchical Modeling: Integrating cache-level bandwidths and detailed miss modeling to capture locality-induced bandwidth amplification.
• Structure Identification Algorithms: Automated detection of matrix regimes and dynamic selection of SpMM strategies.
• GPU and Distributed Extensions: Adapting sparsity-aware modeling for GPU tensor cores [li22quantizedspmm] and scaling to distributed computation [block2024two].
• Deep Learning Applications: Exploiting structure-aware SpMM optimization in pruning and sparsification of LLMs [wang2025generalsparse].

Conclusion

The paper delivers a robust framework for sparsity-aware roofline modeling of SpMM, analytically and empirically substantiating the necessity for structural regime distinction. The approach sets a new standard for predicting attainable SpMM performance across application-diverse workloads and hardware platforms. Going forward, structure-aware performance models will be critical in developing adaptive, high-throughput SpMM kernels, especially as AI and scientific computing increasingly adopt variable sparsity in their foundational operators.