Global Convergence and Acceleration for Single Observation Gradient Free Optimization

Abstract: Simultaneous perturbation stochastic approximation (SPSA) is an approach to gradient-free optimization introduced by Spall as a simplification of the approach of Kiefer and Wolfowitz. In many cases the most attractive option is the single-sample version known as 1SPSA, which is the focus of the present paper, containing two major contributions: a modification of the algorithm designed to ensure convergence from arbitrary initial condition, and a new approach to exploration to dramatically accelerate the rate of convergence. Examples are provided to illustrate the theory, and to demonstrate that estimates from unmodified 1SPSA may diverge even for a quadratic objective function.