Confidence set for mixture order selection

Abstract: A fundamental challenge in approximating an unknown density using finite Gaussian mixture models is selecting the number of mixture components, also known as order. Traditional approaches choose a single best model using information criteria. However, often models with different orders yield similar fits, leading to substantial model selection uncertainty and making it challenging to identify the optimal number of components. In this paper, we introduce the Model Selection Confidence Set (MSCS) for order selection in Gaussian mixtures - a set-valued estimator that, with a predefined confidence level, includes the true mixture order across repeated samples. Rather than selecting a single model, our MSCS identifies all plausible orders by determining whether each candidate model is at least as plausible as the best-selected one, using a screening based on a penalized likelihood ratio statistic. We provide theoretical guarantees for asymptotic coverage, and demonstrate its practical advantages through simulations and real data analysis.