A survey of scalar and vector extrapolation

Abstract: Scalar extrapolation and convergence acceleration methods are central tools in numerical analysis for improving the efficiency of iterative algorithms and the summation of slowly convergent series. These methods construct transformed sequences that converge more rapidly to the same limit without altering the underlying iterative process, thereby reducing computational cost and enhancing numerical accuracy. Historically, the origins of such techniques can be traced back to classical algebraic methods by AlKhwarizmi and early series acceleration techniques by Newton, while systematic approaches emerged in the 20th century with Aitken process and Richardson extrapolation. Later developments, including the Shanks transformation and Wynn epsilon algorithm, provided general frameworks capable of eliminating multiple dominant error components, with deep connections to Pade approximants and rational approximations of generating functions. This paper presents a comprehensive review of classical scalar extrapolation methods, including Richardson extrapolation, Aitken process, Shanks transformation, Wynn epsilon algorithm, and other algorithms. We examine their theoretical foundations, asymptotic error models, convergence properties, numerical stability, and practical implementation considerations. The second part of this work is dedicated to vector extrapolation methods: polynomial based ones and epsilon algorithm generalizations to vector sequences. Additionally, we highlight modern developments such as their applications to iterative solvers, Krylov subspace methods, and large-scale computational simulations. The aim of this review is to provide a unified perspective on scalar and vector extrapolation techniques, bridging historical origins, theoretical insights, and contemporary computational applications.