Convergence of projected stochastic approximation algorithm

Abstract: We study the Robbins-Monro stochastic approximation algorithm with projections on a hyperrectangle and prove its convergence. This work fills a gap in the convergence proof of the classic book by Kushner and Yin. Using the ODE method, we show that the algorithm converges to stationary points of a related projected ODE. Our results provide a better theoretical foundation for stochastic optimization techniques, including stochastic gradient descent and its proximal version. These results extend the algorithm's applicability and relax some assumptions of previous research.