Mean Field Variational Bayesian Inference and Statistical Mechanics of Gaussian Mixture Model

Abstract: One of the main modeling in many data science applications is the Gaussian Mixture Model (GMM), and Mean Field Variational Bayesian Inference (MFVBI) is classically used for approximate fast computation. In this paper, we provide a definitive answer to the fundamental inquiry about the uncertainty quantification of the MFVBI applied to the GMM. It turns out that GMM can be considered as a generalization of Curie--Weiss model in statistical mechanics. The standard quantities like partition function and free energy appear naturally in the process of our analysis.