Acceleration via silver step-size on Riemannian manifolds with applications to Wasserstein space

Abstract: There is extensive literature on accelerating first-order optimization methods in a Euclidean setting. Under which conditions such acceleration is feasible in Riemannian optimization problems is an active area of research. Motivated by the recent success of varying step-size methods in the Euclidean setting, we undertake a study of such algorithms in the Riemannian setting. We show that varying step-size acceleration can be achieved in non-negatively curved Riemannian manifolds under geodesic smoothness and generalized geodesic convexity, a new notion of convexity that we introduce to aid our analysis. As a core application, we show that our method provides the first theoretically guaranteed accelerated optimization method in Wasserstein spaces. In addition, we numerically validate our method's applicability to other problems, such as optimization problems on the sphere.