hierarchical dirichlet strongprocessesstrong - soda hall.pdfVIP

Hierarchical Dirichlet Processes Yee Whye Teh
 , Michael I. Jordan
   , Matthew J. Beal  and David M. Blei
 
 Computer Science Div.,  Dept. of Statistics   Dept. of Computer Science University of California at Berkeley University of Toronto Berkeley CA 94720, USA Toronto M5S 3G4, Canada  ywteh,jordan,blei @ beal@ Abstract We propose the hierarchical Dirichlet process (HDP), a nonparametric Bayesian model for clustering problems involving multiple groups of data. Each group of data is modeled with a mixture, with the number of components being open-ended and inferred automatically by the model. Further, components can be shared across groups, allowing dependencies across groups to be modeled effectively as well as conferring generaliza- tion to new groups. Such grouped clustering problems occur often in practice, e.g. in the problem of topic discovery in document corpora. We report experimental results on three text corpora showing the effective and superior performance of the HDP over previous models. 1 Introduction One of the most significant conceptual and practical tools in the Bayesian paradigm is the notion of a hierarchical model. Building on the notion that a parameter is a random variable, hierarchical models have applications to a variety of forms of grouped or relational data and to general problems involving “multi-task learning” or “learning to learn.” A simple and classical example is the Gaussian means problem, in which a grand mean  is drawn from some distribution, a set of  means are then drawn independently from a Gaussian with mean   , and data are subsequently drawn independently from  Gauss