Minimizing the Arithmetic and Communication Complexity of Jacobi's Method for Eigenvalues and Singular Values

Abstract: In this paper, we analyze several versions of Jacobi's method for the symmetric eigenvalue problem. Our goal throughout is to reduce the asymptotic cost of the algorithm as much as possible, as measured by the number of arithmetic operations performed and associated (sequential or parallel) communication, i.e., the amount of data moved between slow and fast memory or between processors in a network. In producing rigorous complexity bounds, we allow our algorithms to be built on both classic $O(n^3)$ matrix multiplication and fast, Strassen-like $O(n^{\omega_0})$ alternatives. In the classical setting, we show that a blocked implementation of Jacobi's method attains the communication lower bound for $O(n^3)$ matrix multiplication (and is therefore expected to be communication optimal among $O(n^3)$ methods). In the fast setting, we demonstrate that a recursive version of blocked Jacobi can go even further, reaching essentially optimal complexity in both measures. We also discuss Jacobi-based SVD algorithms and a parallel version of block Jacobi, showing that analogous complexity bounds apply.