A unified sparse matrix data format for efficient general sparse matrix-vector multiply on modern processors with wide SIMD units

Abstract: Sparse matrix-vector multiplication (spMVM) is the most time-consuming kernel in many numerical algorithms and has been studied extensively on all modern processor and accelerator architectures. However, the optimal sparse matrix data storage format is highly hardware-specific, which could become an obstacle when using heterogeneous systems. Also, it is as yet unclear how the wide single instruction multiple data (SIMD) units in current multi- and many-core processors should be used most efficiently if there is no structure in the sparsity pattern of the matrix. We suggest SELL-C-sigma, a variant of Sliced ELLPACK, as a SIMD-friendly data format which combines long-standing ideas from General Purpose Graphics Processing Units (GPGPUs) and vector computer programming. We discuss the advantages of SELL-C-sigma compared to established formats like Compressed Row Storage (CRS) and ELLPACK and show its suitability on a variety of hardware platforms (Intel Sandy Bridge, Intel Xeon Phi and Nvidia Tesla K20) for a wide range of test matrices from different application areas. Using appropriate performance models we develop deep insight into the data transfer properties of the SELL-C-sigma spMVM kernel. SELL-C-sigma comes with two tuning parameters whose performance impact across the range of test matrices is studied and for which reasonable choices are proposed. This leads to a hardware-independent ("catch-all") sparse matrix format, which achieves very high efficiency for all test matrices across all hardware platforms.

• The paper presents the SELL-C-σ format that unifies several sparse storage techniques to efficiently utilize wide SIMD units.
• It details a tunable approach with chunk size and sorting scope parameters that enhance memory access and reduce zero-fill overhead.
• Performance evaluations on platforms like Intel Xeon Phi and Nvidia Tesla GPUs demonstrate significant improvements over traditional spMVM methods.

Evaluating a Unified Sparse Matrix Storage Format for Efficient Sparse Matrix-Vector Multiplication

Sparse matrix-vector multiplication (spMVM) is a critical component of many numerical algorithms in scientific computing, particularly in applications such as quantum physics, fluid dynamics, and structural mechanics. The paper authored by Kreutzer, Hager, Wellein, Fehske, and Bishop presents a comprehensive analysis of a unified sparse matrix storage format named SELL-$C$-$\sigma$, which aims to optimize spMVM performance across various modern processor architectures with wide SIMD units, including Intel Xeon Sandy Bridge, Intel Xeon Phi, and Nvidia Tesla GPUs.

The SELL-$C$-$\sigma$ format is a variant of Sliced ELLPACK, proposed as a SIMD-friendly format that merges long-standing ideas from vector computing and GPGPU programming. Unlike traditional formats such as Compressed Row Storage (CRS) and unmodified ELLPACK, the SELL-$C$-$\sigma$ format is structured to exploit wide SIMD units efficiently by organizing matrix data in chunks. Each chunk consists of rows padded to the same length, minimizing zero-fill overhead while maintaining high memory access efficiency within each SIMD unit. The format allows for a degree of tunability with parameters $C$ (chunk size) and $\sigma$ (sorting scope), enhancing adaptability across diverse hardware configurations.

Through extensive performance analysis on various architectures, the authors demonstrate that SELL-$C$-$\sigma$ achieves superior performance compared to CRS, notably on architectures with wide SIMD capabilities such as the Intel Xeon Phi. For instance, significant performance gains on matrices with small $N_\mathrm{nzr}$ validate the efficiency of SELL-$C$-$\sigma$ in handling SIMD vectorization overheads, which are particularly problematic in CRS format implementations. Additionally, the paper explores the impact of sorting scopes on performance, showing how local sorting enhances chunk occupancy while balancing trade-offs in RHS vector access performance.

The implications of this research are significant for the development of hardware-agnostic numerical software. By facilitating efficient spMVM operations across heterogeneous systems, SELL-$C$-$\sigma$ enables a single storage format that potentially reduces complexity in code maintenance and improves performance portability, paving the way for more robust scientific computing frameworks. As heterogeneous computing becomes more prevalent, the SELL-$C$-$\sigma$ format offers a viable strategy for future-proofing numerical libraries against evolving processor architectures.

Future developments in AI and computational hardware may further drive the adoption and refinement of SELL-$C$-$\sigma$. For instance, integrating this format with emerging AI accelerator technologies or hybrid computing models, such as MPI+X or OpenACC, could initiate new research trajectories, enhancing the flexibility and efficiency of scientific computations at exascale levels. Additionally, further optimization of matrix partitioning strategies, particularly for matrices with diverse structural properties, may yield additional performance benefits across varying computational workloads. By addressing these aspects, SELL-$C$-$\sigma$ maintains its relevance and potential for significant contributions to high-performance numerical computing paradigms.