Sparse Regularization by Smooth Non-separable Non-convex Penalty Function Based on Ultra-discretization Formula

Abstract: In sparse optimization, the $\ell_{1}$ norm is widely adopted for its convexity, yet it often yields solutions with smaller magnitudes than expected. To mitigate this drawback, various non-convex sparse penalties have been proposed. Some employ non-separability, with ordered weighting as an effective example, to retain large components while suppressing small ones. Motivated by these approaches, we propose ULPENS, a non-convex, non-separable sparsity-inducing penalty function that enables control over the suppression of elements. Derived from the ultra-discretization formula, ULPENS can continuously interpolate between the $\ell_{1}$ norm and a non-convex selective suppressing function by adjusting parameters inherent to the formula. With the formula, ULPENS is smooth, allowing the use of efficient gradient-based optimization algorithms. We establish key theoretical properties of ULPENS and demonstrate its practical effectiveness through numerical experiments.