基于支持向量机的大规模文本分类研究与设计-计算机软件与理论专业论文.docxVIP

山东师范大学硕士学位论文Abstract 山东师范大学硕士学位论文 Abstract With the rapid development of communication，computering and of especially intemet，all kinds of information has grown geometrically．So does text as information carrier．In order to pick up valid information from the massive and complicated text， timely and accurately，text presentation and automatic text categorization technology have received widespread attention．Text categorization is very helpful for effectiveness and efficiency of information retrieval，which promotes personalized service and improves Information acquisition mode．So good Classification performance is the focus，and text categorization algorithm based on SVM is more more research foCUS． First，the dissertation analyzes the overall model for text classification，text representation and key Technology for text categorization．In feature extraction， several different methods of feature selection are compared such as document frequency，CHI distribution，information gain as well as mutual information．It is proved that the method of feature selection based on IG is better than other methods． In text representation，vector space model is implemented by using TF／IDF．And among multi-classification algorithms，one versus others algorithm is used and the results are quite satisfactory． The dissertation focuses on the statistical learning theory，probes in support vector machine algorithm based on it，and expounds on the current status of research and application of support vector machines，as well as the problems faced．Furthermore， the author analyzes mad disscuss on training and classification algorithm of SVM，as well as hot issues such as the algorithms for solving large problems．The dissertation proposes a parallel SVM classification algorithm——PCSMO—KNN coupled with SVMQP idea to cope with bottlenecks．namely the computation time and memory as for the massive and confused text classification problems．The algorithm assigns the massive text into many parallel processors