Softplus Penalty Functions for Constrained Optimization

Abstract: Penalty functions are widely used to enforce constraints in optimization problems and reinforcement leaning algorithms. Softplus and algebraic penalty functions are proposed to overcome the sensitivity of the Courant-Beltrami method to strong objective function gradients. These penalty functions are shown to converge in one fifth as many iterations on problems with fifty optimized parameters and be less sensitive to objective function gradient and scaling than the Courant-Beltrami penalty function. The softplus and algebraic penalty functions developed in this work allow optimized solutions to be found in fewer iterations with greater control over constraint accuracy.