Breaking a Logarithmic Barrier in the Stopping Time Convergence Rate of Stochastic First-order Methods

Abstract: This work provides a novel convergence analysis for stochastic optimization in terms of stopping times, addressing the practical reality that algorithms are often terminated adaptively based on observed progress. Unlike prior approaches, our analysis: 1. Directly characterizes convergence in terms of stopping times adapted to the underlying stochastic process. 2. Breaks a logarithmic barrier in existing results. Key to our results is the development of a Gr\"onwall-type argument tailored to such stochastic processes. This tool enables sharper bounds without restrictive assumptions.