DecouplingSparsityandSmoothness方案.PDFVIP

Decoupling Sparsity and Smoothness in the Discrete Hierarchical Dirichlet Process Chong Wang David M. Blei Computer Science Department Computer Science Department Princeton University Princeton University chongw@ blei@ Abstract We present a nonparametric hierarchical Bayesian model of document collections that decouples sparsity and smoothness in the component distributions (i.e., the “topics”). In the sparse topic model (sparseTM), each topic is represented by a bank of selector variables that determine which terms appear in the topic. Thus each topic is associated with a subset of the vocabulary, and topic smoothness is modeled on this subset. We develop an efficient Gibbs sampler for the sparseTM that includes a general-purpose method for sampling from a Dirichlet mixture with a combinatorial number of components. We demonstrate the sparseTM on four real-world datasets. Compared to traditional approaches, the empirical results will show that sparseTMs give better predictive performance with simpler inferred models. 1 Introduction The hierarchical Dirichlet process (HDP) [ 1] has emerged as a powerful model for the unsupervised analysis of text. The HDP models documents as distributions over a collection of latent components, which are often called “topics” [2, 3]. Each word is assigned to a topic, and is drawn from a distribution over terms associated with that topic. The per-document distributions over topics represent systematic regularities of word use among the documents; the per-topic distributions over terms encode the randomness inherent in observations from the topics. The num