A mixed precision Jacobi SVD algorithm

Abstract: We propose a mixed precision Jacobi algorithm for computing the singular value decomposition (SVD) of a dense matrix. After appropriate preconditioning, the proposed algorithm computes the SVD in a lower precision as an initial guess, and then performs one-sided Jacobi rotations in the working precision as iterative refinement. By carefully transforming a lower precision solution to a higher precision one, our algorithm achieves about 2 times speedup on the x86-64 architecture compared to the usual one-sided Jacobi SVD algorithm in LAPACK, without sacrificing the accuracy.

• The paper introduces a mixed precision approach combining initial low precision SVD with high precision refinement, achieving approximately 2x speedup on x86-64 systems.
• It leverages a rank-revealing QR factorization for preconditioning, ensuring a high-quality starting matrix and stable convergence in Jacobi iterations.
• Empirical tests confirm that the algorithm maintains accuracy comparable to traditional methods while significantly reducing computational time.

A Mixed Precision Jacobi SVD Algorithm

Introduction

This paper introduces a mixed precision Jacobi algorithm for computing the Singular Value Decomposition (SVD) of dense matrices. By leveraging low precision arithmetic for initial computation followed by high precision iterative refinements, the authors aim to enhance computational efficiency significantly. The approach achieves approximately a 2x speedup on x86-64 architectures compared to traditional high precision Jacobi SVD algorithms without compromising on accuracy.

One-Sided Jacobi Algorithm

The one-sided Jacobi SVD algorithm involves orthogonalizing matrix columns iteratively using Jacobi plane rotations to diagonalize $X^HX$. The convergence strategy leverages a cyclic pattern of column pairs, which ensures stability and efficiency.

Mixed Precision Algorithm

The algorithm comprises the following stages:

1. Preconditioning: A rank-revealing QR factorization determines an initial matrix $X$ that's conducive for iterative refinement.
2. Low Precision SVD: Compute an approximate SVD in lower precision (e.g., single precision), which provides a coarse solution quickly.
3. Switching Precision: Transition this approximate solution to higher precision, preparing it for refinement.
4. Iterative Refinement: Apply high precision Jacobi iterations, capitalizing on rapid convergence when starting from an approximately correct solution.

   Figure 1: The relative run time of the algorithm on x86-64 architecture, demonstrating significant speedup.

Performance Metrics

The proposed method achieves about two times the speed advantage without sacrificing relative accuracy, confirmed through extensive empirical testing. The mixed precision strategy is especially effective when high accuracy is necessary, leveraging single precision computation where possible.

Conclusions

Algorithmic improvements such as leveraging QR factorizations and careful preconditioning allow the mixed precision Jacobi SVD algorithm to maintain high accuracy at reduced computational cost. Future explorations could extend the approach to more complex systems, particularly benefiting from more capable modern hardware architectures. The mixed precision approach suggests promising directions in numerical linear algebra, offering a compelling trade-off between performance and precision.