Modifications of the BIC for order selection in finite mixture models

Abstract: Finite mixture models are ubiquitous tools in modern statistical modeling, and a frequently encountered problem that arises in their implementation is the choice of model order. In Kerebin (2000, Sankhya: The Indian Journal of Statistics, Series A, 62, pp. 49-66), the frequently used Bayesian information criterion (BIC) was proved to provide consistent order estimation in the mixture model setting. However, the result requires particularly strong model regularity, including the existence of higher moments and higher derivatives of the component density function. We introduce the $\nu$-BIC and $\epsilon$-BIC, which modifies the BIC by weighting the penalty by a negligibly small logarithmic factors that are immaterial in practice. We prove that the minor modification enables consistency guarantees under weaker conditions, particularly without differentiability and with minimal moment assumptions. We demonstrate how our theory apply to obtaining order selection consistency for Gaussian mixtures, non-differentiable Laplace mixtures, and mixtures of regression models.