Black-box Optimization with Simultaneous Statistical Inference for Optimal Performance

Abstract: Black-box optimization is often encountered for decision-making in complex systems management, where the knowledge of system is limited. Under these circumstances, it is essential to balance the utilization of new information with computational efficiency. In practice, decision-makers often face the dual tasks of optimization and statistical inference for the optimal performance, in order to achieve it with a high reliability. Our goal is to address the dual tasks in an online fashion. Wu et al (2022) [arXiv preprint: 2210.06737] point out that the sample average of performance estimates generated by the optimization algorithm needs not to admit a central limit theorem. We propose an algorithm that not only tackles this issue, but also provides an online consistent estimator for the variance of the performance. Furthermore, we characterize the convergence rate of the coverage probabilities of the asymptotic confidence intervals.

• The paper proposes a novel online SPSA-based algorithm that integrates optimization with simultaneous statistical inference.
• It employs constant step size and exponential smoothing to ensure convergence and robust variance estimation with a CLT guarantee.
• Numerical experiments validate improved convergence rates and scalability in high-dimensional, complex systems.

Black-box Optimization with Simultaneous Statistical Inference for Optimal Performance

The paper addresses a complex topic in the field of black-box optimization, focusing specifically on scenarios that require simultaneous statistical inference for optimal performance. Black-box optimization is critical in complex system management, where explicit system models may be unavailable or computational expensive to evaluate. This work introduces an algorithm designed to handle the dual tasks of optimization and statistical inference in an online fashion, offering practical solutions for decision-making in such scenarios.

Overview

Optimization combined with statistical inference is especially important in environments with streaming data, where decision-makers must repeatedly solve similar problems under uncertainty. The authors propose an algorithm based on Simultaneous Perturbation Stochastic Approximation (SPSA) to address this. The typical offline approach—characterized by a clear separation between optimization and statistical inference—typically incurs significant costs. The authors' method integrates these dual tasks within a streaming data context, allowing for the exploitation of new information to improve computational efficiency.

Key Contributions

1. Algorithm Development: The work develops an online algorithm leveraging SPSA for optimization, while employing exponential smoothing (constant step size) for statistical inference, effectively addressing the issue noted by preceding studies that typical sample averages do not adhere to the expected statistical properties.
2. Theoretical Insights: The paper provides theoretical guarantees, including a Central Limit Theorem (CLT) for the performance estimates. It also establishes the convergence for the variance estimator, ensuring robust statistical inference.
3. Simultaneous Optimization and Inference: By employing a decay rate of perturbation distinct from optimization, and integrating constant step size for estimation, the solution adeptly balances the representation issue noted in sample averages without resorting to extensive additional sampling, as in previous methodologies like the four-point method.
4. Variance Estimation: A novel contribution is the derivation and validation of a variance estimator within the algorithm, solving a critical need for online applications where variance estimation must adapt to incoming data without storing raw observations extensively.
5. Tightness and Convergence: The proof concerning the tightness of parameter estimates, and the implications for Ornstein-Uhlenbeck processes, showcase the sophistication of the statistical models underlying the proposed method.

Numerical Results

Numerical experiments demonstrate the algorithm's performance against established benchmarks in literature, showcasing improvements in both convergence rates and accuracy of statistical inference. Notably, the paper addresses the computational challenges and provides a scalable approach, demonstrating superior performance in high-dimensional and complex real-world scenarios.

Implications for Future Research

This study paves the way for further exploration into multi-timescale stochastic approximation methods and their applications in complex systems where analytical model paths are not feasible. The combination of simulation-based optimization and inference may spur advancements across domains like reinforcement learning, financial engineering, and operational system management.

Conclusion

The paper provides a robust framework for addressing simultaneous optimization and inference in black-box settings, overcoming previous limitations with a novel methodological approach. This contribution is vital for advancing real-time decision-making processes in complex environments, where adaptability and computational efficiency are paramount. The detailed theoretical underpinnings also open avenues for future research into more sophisticated approximation and modeling techniques.