基于文本降维和蚁群算法的文本聚类研究-计算机应用技术专业论文.docxVIP

安徽大学硕士学位论文Abs廿actAbstract 安徽大学硕士学位论文Abs廿act Abstract As techno logy branch of text mining，text clustering plays more and more important role．Text clustering technology classify similar text，it is convenient for people to dig out the potential and valuable information from the mass text iflformation． In this paper，using Fudan University Chinese text corpus for text clustering research，using similarity of the text combined with corresponding clustering algorithm similar text will be clustering．Because the structural characteristics of Chinese text itself,SO before text clustering we should text process，namely the text segmentation and remove useless stop words．This paper CAS segmentation system(ICTCLAS)for text segmentation and Harbin Institute of Technology stop list for filtering stop words．Then，in order to get the feature words which express the content of the text，the feature words selected．At last，the computer not directly deal with the information of the non structure the semi structure．In this paper，using vector space model(VSM)to represent text，term frequency-inverse document frequency(TF-IDF)value to represent the weight value of each feature WOrd． Through analyzing the whole process of text processing，it is found that the vector space model can have the problem of high dimension and sparse data elements if this model is directly composed of the processed keywords．In order to solve the problem，this paper first divided into two steps to select the feature words，the first step is to the chi square test to extract the feature words and get the set of feature words，the second step is clustering of feature words using Hierarchical clustering based on semantics，merge synonyms similar words in the set of feature words， Then，the TF-IDF value of each word in the set of feature words is calculated，and the vector space model is generated．However，the model still has the defects of high dimension and sparse elements．In this paper,we the method of singular value decompo