A Review of Sequential Decision Making via Simulation

Abstract: Optimization via simulation has been well established to find optimal solutions and designs in complex systems. However, it still faces modeling and computational challenges when extended to the multi-stage setting. This survey reviews the models and methodologies of single-stage optimization via simulation and multi-stage stochastic programming. These are necessary theoretical components to push forward the development of sequential decision making via simulation. We identify the key challenge of sequential decision making via simulation as the appropriate modeling of the stage-wise value function, for which we survey the state-of-the-art meta-models and their potential solution algorithms.