模式识别与机器学习.pdf

Technical Rep ort No Department of Statistics University of Toronto Markov Chain Sampling Metho ds for Dirichlet Pro cess Mixture Mo dels Radford M Neal Department of Statistics and Department of Computer Science University of Toronto Toronto Ontario Canada httpwwwcsutorontoca radford radfordstatutorontoca September Abstract Markov chain methods for sampling from the posterior distribution of a Dirichlet process mixture model are reviewed and two new classes of methods are pre sented One new approach is to make MetropolisHastings updates of the indicators specifying which mixture component is associated with each observation perhaps sup plemented with a partial form of Gibbs sampling The other new approach extends Gibbs sampling for these indicators by using a set of auxiliary parameters These meth odsaresimpletoimplementandaremoreecientthanpreviouswaysofhandlinggeneral Dirichlet process mixture models with nonconjugate priors Intro duction Modeling a distribution as a mixture of simpler distributions is useful both as a non parametric density estimation method and as a way of identifying latent classes that can explain the dependencies observed between variables Mixtures with a countably innite number of components can reasonably be handled in a Bayesian framework by employing a prior distribution for mixing proportions such as a Dirichlet process that leads to a few of these components dominating Use of countably innite mixtures by passes the need to determine the correct number of components in a nite mixture model a task which is fraught with technical diculties In many contexts a countably innite mixture is also a more realistic model than a mixture with a small numb