a generic approach to topic models - arbylon.pdfVIP

A Generic Approach to Topic Models Gregor Heinrich Fraunhofer IGD University of Leipzig Darmstadt, Germany Abstract. This article contributes a generic model of topic models. To define the problem space, general characteristics for this class of models are derived, which give rise to a representation of topic models as “mixture networks”, a domain- specific compact alternative to Bayesian networks. Besides illustrating the inter- connection of mixtures in topic models, the benefit of this representation is its straight-forward mapping to inference equations and algorithms, which is shown with the derivation and implementation of a generic Gibbs sampling algorithm. 1 Introduction Mixture models [1] are a powerful tool to model complex probabilistic distributions by convex sums of component densities, p (x) p (zk)p (x ), where z is an index k k variable that indicates which component k the observation x originates from. Among the large class of such models, mixture models with discrete component densities p (x ) k are of particular interest because in this case the component densities can serve as weighting functions for other mixtures, which themselves can have again discrete or non-discrete component densities. This fact makes it possible to construct models that consist of cascades or even networks of coupled discrete mixtures as generative struc- ture underlying one or more observable mixtures with arbitrary (e.g., discrete or Gaus- sian) component densities. Such a coupling of mixtures can be considered a defining chara