HIERARCHICAL LEARNING OF GRIDS OF MICROTOPICS.pdfVIP

Under review as a conference paper at ICLR 2016 HIERARCHICAL LEARNING OF GRIDS OF MICROTOPICS Nebojsa Jojic, Alessandro Perina and Dongwoo Kim Microsoft Corporation Redmond, WA - USA {jojic,alperina}@ 5 1 ABSTRACT 0 2 The counting grid is a grid of microtopics, sparse word/feature distributions. The v generative model associated with the grid does not use these microtopics indi- o vidually. Rather, it groups them in overlapping rectangular windows and uses N these grouped microtopics as either mixture or admixture components. This paper builds upon the basic counting grid model and it shows that hierarchical reasoning 3 helps avoid bad local minima, produces better classification accuracy and, most 1 interestingly, allows for extraction of large numbers of coherent microtopics even ] from small datasets. We evaluate this in terms of consistency, diversity and clarity L of the indexed content, as well as in a user study on word intrusion tasks. We M demonstrate that these models work well as a technique for embedding raw im- ages and discuss interesting parallels between hierarchical CG models and other . t deep architectures. a t s [ 1 INTRODUCTION 3 v 1 Machine learning has recently entered a renaissance both in terms of research breakthroughs and 0 wide spread of practical uses. Bo