基于信息熵的特征选择精选.pdfVIP

140 2013 ，49 （10） Computer Engineering and Applications 计算机工程与应用 基于信息熵理论的特征权重算法研究 郭红钰 GUO Hongyu 华北计算技术研究所，北京 100083 North China Institute of Computer Technology, Beijing 100083, China GUO Hongyu. Research on term weighting algorithm based on information entropy theory. Computer Engineering and Applications, 2013, 49 （10）：140-146. Abstract ：Text representation is an important process to perform text categorization, and the method of text representation plays an important role in the final classification accuracy. This paper proposes a new term weighting algorithm ETFIDF （Entropy based TFIDF ）based on information entropy theory to overcome the limitations of the traditional term weighting algorithm TFIDF （Term Frequency and Inverted Document Frequency ）. ETFIDF not only considers the number of times a term occurs in a document and the number of documents in training set in which a term occurs, but also takes into account the distribution of documents in the training set in which the term occurs. Experimental results show that ETFIDF outperforms TFIDF in text categorization. Furthermore, detailed theoretical analysis and experimental study on the relationship between ETFIDF and feature selection have been done in this paper. Experimental results show that, it can represent the text more accurately if we take into account the distri- bution of documents in the training set in which the term occurs in the text representation stage. Moreover, it can achieve higher performance for the combination of ETFIDF and feature selection algorithm if we consider both the accuracy and efficiency. Key words ：information entropy; term weighting; feature selection; text categorization 摘 要：文本表示是使用分类算法处理文本时必不可少的环节，文本表示方法的选择对最终的分类精度起着至关重要的 作用。针对经典的特征权重计算方法TFIDF （Term Frequency and Inverted Document Frequency ）中存在的不足，提出了 一种基于信息熵理论的特征权重算法ETFIDF （Entropy based TFIDF ）。ETFIDF 不仅考虑特征项在文档中出现的频率及 该特征项在