The iterated Dirichlet process and applications to Bayesian inference

Abstract: Consider an i.i.d. sequence of random variables, taking values in some space $S$, whose underlying distribution is unknown. In problems of Bayesian inference, one models this unknown distribution as a random measure, and the law of this random measure is the prior. When $S = {0, 1}$, a commonly used prior is the uniform distribution on $[0, 1]$, or more generally, the beta distribution. When $S$ is finite, the analogous choice is the Dirichlet distribution. For a general space $S$, we are led naturally to the Dirichlet process (see [Ferguson, 1973]). Here, we consider an array of random variables, and in so doing are led to what we call the iterated Dirichlet process (IDP). We define the IDP and then show how to compute the posterior distribution, given a finite set of observations, using the method of sequential imputation. Ordinarily, this method requires the existence of certain joint density functions, which the IDP lacks. We therefore present a new, more general proof of the validity of sequential imputation, and show that the hypotheses of our proof are satisfied by the IDP.