Posterior Consistency in Parametric Models via a Tighter Notion of Identifiability

Abstract: We study Bayesian posterior consistency in parametric density models with proper priors, challenging the perception that the problem is settled. Classical results established consistency via MLE convergence under regularity and identifiability assumptions, with the latter taken for granted and rarely examined. We refocus attention on identifiability, showing that inconsistency arises only when the true distribution coincides with a weak limit of model densities in a way that violates identifiability. While such failures occur naturally in nonparametric settings, they are implausible and effectively self-inflicted in parametric models. Our analysis shows that classical regularity conditions are unnecessary: a mild strengthening of identifiability suffices to ensure consistency in parametric models, even when the MLE is inconsistent. We also demonstrate that parametric inconsistency requires carefully engineered, oscillatory model features aligned with the true distribution, which is unlikely to occur without adversarial design. Our findings also clarify the distinct mechanisms behind Bayesian and frequentist inconsistency and advocate for separate theoretical treatments.