Horospherically Convex Optimization on Hadamard Manifolds Part I: Analysis and Algorithms

Abstract: Geodesic convexity (g-convexity) is a natural generalization of convexity to Riemannian manifolds. However, g-convexity lacks many desirable properties satisfied by Euclidean convexity. For instance, the natural notions of half-spaces and affine functions are themselves not g-convex. Moreover, recent studies have shown that the oracle complexity of geodesically convex optimization necessarily depends on the curvature of the manifold (Criscitiello and Boumal, 2022; Criscitiello and Boumal, 2023; Hamilton and Moitra, 2021), a computational bottleneck for several problems, e.g., tensor scaling. Recently, Lewis et al. (2024) addressed this challenge by proving curvature-independent convergence of subgradient descent, assuming horospherical convexity of the objective's sublevel sets. Using a similar idea, we introduce a generalization of convex functions to Hadamard manifolds, utilizing horoballs and Busemann functions as building blocks (as proxies for half-spaces and affine functions). We refer to this new notion as horospherical convexity (h-convexity). We provide algorithms for both nonsmooth and smooth h-convex optimization, which have curvature-independent guarantees exactly matching those from Euclidean space; this includes generalizations of subgradient descent and Nesterov's accelerated method. Motivated by applications, we extend these algorithms and their convergence rates to minimizing a sum of horospherically convex functions, assuming access to a weighted-Fr\'echet-mean oracle.