基于特征模板和SVM的中文微博观点句识别方法研究-计算机科学与技术专业论文.docxVIP

万方数据 万方数据 III III 摘要 随着 WEB2.0 技术及微博平台的迅猛发展，发布中文微博逐渐成为很多人的一 种生活方式和习惯。通过分析微博观点句，个体消费者可以参考其他购买了产品或 服务的用户的感受和观点做出他自己的购买决定，公司可以了解顾客的体验从而提 升产品或服务的质量、营销策略或竞争模式，国家可以根据大众言论更好地掌握民 生、分析舆情和服务大众。但由于海量的微博数据中包含无数冗余、包含噪声及无 观点的信息，所以识别并提取中文微博中的观点句已成为近年来的研究热点之一。 本文主要从以下几个方面开展研究工作：首先，总结和分析微博文本的独有特 征，根据这些特征对 Hownet 词库进行扩展，并对初始微博语料中的不规范化等现象 进行了相应的预处理。然后，利用信息增益对泛选的特征进行了特征筛选，并设计 一个评分函数来探索识别观点句的特征模板，然后将该特征模板与 SVM 分类器相结 合，使用 Java 程序设计语言对中文微博语料进行了观点句与非观点句的二分类实验。 最后，通过分析非观点句的特点，设计了一个否定模板，并进行了相应的实验来验 证其有效性。 本文的主要创新点为：1）从多方面对微博特征进行考虑，从而对 Hownet 词库 进行有效扩展，使得分词处理更加准确。2）结合信息增益的方法和评分函数以及相 关分析设计特征模板，从而达到更好的分类效果。3）设计了一个否定模板，提供了 探索观点句识别的逆向思路。 关键词：观点句， 特征模板， 支持向量机， 文本分类 I 万方数据 万方数据 Abstract With the rapid development of WEB2.0 technology and microblog platform, posting microblog have increasingly become a kind of living style and habit by public. Through these Chinese weibo sentences, individual and organization have chance to obtain public’s thoughts and views towards things, nation is possible to grasp, control and lead media and public’s opinion. Nevertheless, massive data from microblog platform contains redundant, noisy and non-opinionated information, we have no access to extract opinionated sentences from them by artificial way. Therefore, how to identify and extract the sentences with users’ opinions from big data has become a hot field of research. This paper mainly discusses as followed: To start with, summarize and analyze the special characteristics of weibo texts. According to these characteristics, we get down to expand the Hownet lexicon and for the phenomena such as informal language usage in weibo, the initial texts are preprocessed. Furthermore, information gain is employed to select features, and later we use a score function to further explore how to construct a feature rule template. Then, by virtue of SVM classifier, we use Java programming language to implement the binary classification experiment which identifies opinionated and non-opinionated sentences in Chinese weibo texts.