Machine Learning Powered Feasible Path Framework with Adaptive Sampling for Black-box Optimization

Abstract: Black-box optimization (BBO) involves functions that are unknown, inexact and/or expensive-to-evaluate. Existing BBO algorithms face several challenges, including high computational cost from extensive evaluations, difficulty in handling complex constraints, lacking theoretical convergence guarantees and/or instability due to large solution quality variation. In this work, a machine learning-powered feasible path optimization framework (MLFP) is proposed for general BBO problems including complex constraints. An adaptive sampling strategy is first proposed to explore optimal regions and pre-filter potentially infeasible points to reduce evaluations. Machine learning algorithms are leveraged to develop surrogates of black-boxes. The feasible path algorithm is employed to accelerate theoretical convergence by updating independent variables rather than all. Computational studies demonstrate MLFP can rapidly and robustly converge around the KKT point, even training surrogates with small datasets. MLFP is superior to the state-of-the-art BBO algorithms, as it stably obtains the same or better solutions with fewer evaluations for benchmark examples.