基于概率主题的文本层次分类.pdfVIP

第39卷第8期 2009 年 8 月 非@精弩 4主革大注毫也 JOURNAL OF UNIVERSITY OF SCI剧CE AND TECHNOLOGY OF CHINA Article ID: 0253-2778(2009) 08-0875-05 Vo1. 39.No. 8 Aug. 2009 Hierarchical text categorization with probabilistic topics QUAN Xiao-jun , LIN Yang-gang , LUO Qi-ming , CHEN En-hong Wepaπmentof Com阳ter Science and Tech7Wlo口 ， University of Science and Tech7Wlogy of China. Hefei 230027.αina) Abstract: Probabilìstic topic model ìs a statistical generative model for automatically extracting a set of topics from a collection of documents and then representing these documents as mixtures of topics. Topics obtained by this method pick out significant semantic information of documents. and they have broad applications in many fields. A novel approach was proposed for hierarchical text categorization based on the probabilistic topic model. The approach first extracted a set of topics based on Gibbs sampling , then computed the similarities between test documents and each class based on the topics. Results of experiments on 20 NewsGroups dataset show that this approach is able to produce superior classification performance when ror叩ared to support vector machines. Key words: hierarchical categorization; probabilistic topic model; Gibbs sampling CLC number: TP311 胁cument code: A 基于概率主题的文本层次分类 权小军，林洋港，罗奇鸣，陈恩红 (中国科学技术大学计算机科学与技术系，安徽合肥 230027) 摘要:概率主题模型是一种统计生成模型，它从文档集合中抽取一系列主题，并将这些文档表示为不同主题 依照一定概率混合而成.通过这种模型发现的主题，能揭示丈档的语义信息，在很多领域都有着广泛的应用. 为此基于概率主题模型，提出了一种新的层次文本分类方法.该方法首先利用 Gibbs 抽样提取一系列主题， 然后计算测试文档和每个类的基于主题的相似度，在 20 NewsGroups 数据集上的实验结果表明，该方法的 分类性能明显超越支持向量机分类方法. 关键词:层次分类;概率主题模型;Gibbs 抽样 o Introduction With the rapid growth of information on the intemet and intranets , the amount of text documents on the web is becoming tremendous. This demands more convenient measures to organize and access Received: 2008-04-11 ; Revised: 20饨-05-20 them. 1n many application fields. taxonomies and hierarchies are efficient ways for arranging ∞ncepts ， subject headings. and categories. Tax