Constrained Bayesian optimization with merit functions

Abstract: Bayesian optimization is a powerful optimization tool for problems where native first-order derivatives are unavailable. Recently, constrained Bayesian optimization (CBO) has been applied to many engineering applications where constraints are essential. However, several obstacles remain with current CBO algorithms that could prevent a wider adoption. We propose CBO algorithms using merit functions, such as the penalty merit function, in acquisition functions, inspired by nonlinear optimization methods, e.g., sequential quadratic programming. Merit functions measure the potential progress of both the objective and constraint functions, thus increasing algorithmic efficiency and allowing infeasible initial samples. The acquisition functions with merit functions are relaxed to have closed forms, making its implementation readily available wherever Bayesian optimization is. We further propose a unified CBO algorithm that can be seen as extension to the popular expected constrained improvement (ECI) approach. We demonstrate the effectiveness and efficiency of the proposed algorithms through numerical experiments on synthetic problems and a practical data-driven engineering design problem in the field of plasma physics.