Adaptive Sampling-Based Bi-Fidelity Stochastic Trust Region Method for Derivative-Free Stochastic Optimization

Abstract: Bi-fidelity stochastic optimization is increasingly favored for streamlining optimization processes by employing a cost-effective low-fidelity (LF) function, with the goal of optimizing a more expensive high-fidelity (HF) function. In this paper, we introduce ASTRO-BFDF, a new adaptive sampling trust region method specifically designed for solving unconstrained bi-fidelity stochastic derivative-free optimization problems. Within ASTRO-BFDF, the LF function serves two purposes: first, to identify better iterates for the HF function when a high correlation between them is indicated by the optimization process, and second, to reduce the variance of the HF function estimates by Bi-fidelity Monte Carlo (BFMC). In particular, the sample sizes are dynamically determined with the option of employing either crude Monte Carlo or BFMC, while balancing optimization error and sampling error. We demonstrate that the iterates generated by ASTRO-BFDF converge to the first-order stationary point almost surely. Additionally, we numerically demonstrate the superiority of our proposed algorithm by testing it on synthetic problems and simulation optimization problems with discrete event simulations.