Optimal Simulation Budget Allocation Under Unknown Sampling Variance

Abstract: The ranking and selection problem is a popular framework in the simulation literature for studying optimal information collection. We study a version of this problem in which the simulation output for each design is normally distributed with both its mean and variance being unknown. Using a Bayesian representation of the probability of correct selection, which allows us to explicitly model uncertainty about the variance, we provide a theoretical characterization of the optimal allocation of the simulation budget. Prior work on optimal budget allocation was unable to distinguish between known and unknown sampling variance. We show the impact of this type of uncertainty on the allocation, and design new sequential procedures that can be guaranteed to learn the optimal allocation asymptotically without the need for tuning or forced exploration.