Zeroth-Order Non-smooth Non-convex Optimization via Gaussian Smoothing

Abstract: This paper addresses stochastic optimization of Lipschitz-continuous, nonsmooth and nonconvex objectives over compact convex sets, where only noisy function evaluations are available. While gradient-free methods have been developed for smooth nonconvex problems, extending these techniques to the nonsmooth setting remains challenging. The primary difficulty arises from the absence of a Taylor series expansion for Clarke subdifferentials, which limits the ability to approximate and analyze the behavior of the objective function in a neighborhood of a point. We propose a two time-scale zeroth-order projected stochastic subgradient method leveraging Gaussian smoothing to approximate Clarke subdifferentials. First, we establish that the expectation of the Gaussian-smoothed subgradient lies within an explicitly bounded error of the Clarke subdifferential, a result that extends prior analyses beyond convex/smooth settings. Second, we design a novel algorithm with coupled updates: a fast timescale tracks the subgradient approximation, while a slow timescale drives convergence. Using continuous-time dynamical systems theory and robust perturbation analysis, we prove that iterates converge almost surely to a neighborhood of the set of Clarke stationary points, with neighborhood size controlled by the smoothing parameter. To our knowledge, this is the first zeroth-order method achieving almost sure convergence for constrained nonsmooth nonconvex optimization problems.