An Augmented Lagrangian Approach to Composite Problems with a Random Linear Operator

Abstract: We consider the minimization of a sum of a smooth function with a nonsmooth composite function, where the composition is applied on a random linear mapping. This random composite model encompasses many problems, and can especially capture realistic scenarios in which the data is sampled during the optimization process. We propose and analyze a method that combines the classical Augmented Lagrangian framework with a sampling mechanism and adaptive update of the penalty parameter. We show that every accumulation point of the sequence produced by our algorithm is almost surely a critical point.