Variational Bayes inference and Dirichlet process priors

Abstract: This paper shows how the variational Bayes method provides a computational efficient technique in the context of hierarchical modelling using Dirichlet process priors, in particular without requiring conjugate prior assumption. It shows, using the so called parameter separation parameterization, a simple criterion under which the variational method works well. Based on this framework, its provides a full variational solution for the Dirichlet process. The numerical results show that the method is very computationally efficient when compared to MCMC. Finally, we propose an empirical method to estimate the truncation level for the truncated Dirichlet process.