hierarchical dirichlet strongprocessesstrong - university at buffalo.pdf

Hierarchical Dirichlet Processes Yee Whye Teh tehyw@.sg Department of Computer Science, National University of Singapore, Singapore 117543 Michael I. Jordan jordan@ Computer Science Division and Department of Statistics, University of California at Berkeley, Berkeley CA 94720-1776, USA Matthew J. Beal mbeal@ Department of Computer Science Engineering, State University of New York at Buffalo, Buffalo NY 14260-2000, USA David M. Blei blei@ Department of Computer Science, Princeton University, Princeton, NJ 08544, USA November 15, 2005 Abstract We consider problems involving groups of data, where each observation within a group is a draw from a mixture model, and where it is desirable to share mixture components between groups. We assume that the number of mixture components is unknown a priori and is to be inferred from the data. In this setting it is natural to consider sets of Dirichlet processes, one for each group, where the well-known clustering property of the Dirichlet process provides a nonparametric prior for the number of mixture components within each group. Given our desire to tie the mixture models in the various groups, we consider a hierarchical model, specifically one in which the base measure for the child Dirichlet processes is itself distributed according to a Dirichlet process. Such a base measure being discrete, the child Dirichlet processes necessar- ily share atoms. Thus, as desired, the mixture models in the different groups necessarily share mixture components. We discuss representations of hierarchical Dirichlet processes in terms of a stick-breaking process, and a generalization of the Chinese restaurant process that we refer to a