基于主题的短文本分类动态ppt作品.pptxVIP

基于主题的短文本分类Short Sparse Text Classification With Hidden Topics陈二二指导老师：XX2012年9月10日1342OUTLINEOUTLINE工作内容模型描述实验分析背景简介1长文本分类短分类器背景简介信息量少特征稀疏语义歧义训练集测试集2模型描述2信息量少找到隐含主题作为特征值短文本“长文本”?模型描述特征稀疏语义歧义3工作内容3Business ComputersEngineeringComputersHealth工作内容-目的CultureArtsEntertainmentPoliticsSocietyEducationScienceSports3原始数据 340.5MB， 31,273个文件数据处理 去除重复的文件，提取网页正文，过滤不必要信息， 移除停用词，单词词干化；修善爬虫工具等最终数据 67.1MB，8,957行，词汇量30,283 共8,307,727个单词部分关键词列表（2827）Business advertising, e-commerce, finance, investment, contract ...Computers hardware, software, database, multimedia, keyboard ...Culture-Arts-Entertainment book, music, movie, painting, piano ...Education-Science course, graduate, professor, university, research ...Engineering automobile, telecommunication, civil eng., engineer ...Health diet, therapy, healthcare, treatment, nutrition, hospital...Politics-Society government, legislation, party, regime, military, mayor ...Sports baseball, cricket, football, tennis, Olympic games, basketball ...Wikipedia工作内容-全局数据集获取URL=/wiki/footballFootball refers to a number of sports that involve, to varying degrees, kicking a ball with the foot to score a goal. The most popular of these sports worldwide is association football, more commonly known as just football or soccer“ …3原始数据 340.5MB， 31,273个文件数据处理 去除重复的文件，提取网页正文，过滤不必要信息， 移除停用词，单词词干化；修善爬虫工具等最终数据 67.1MB，8,957行，词汇量30,283 共8,307,727个单词TwitterWikipedia原始数据 200,000条数据处理 过滤不必要信息，分词，单词修正，去停用词，词干 化工作内容-文本数据集获取Coooooooool, what an amazing football match, you should watch it. http://bit.ly/H0fFA @chensj09 Coooooooool, what an amazing football match, you should watch it. http://bit.ly/H0fFA @chensj09 单词修正过滤信息过滤信息单词修正cool what an amazing football match you should watch it cool what an amazing football match you should watch it coooooooool what an amazing football match you should watch it coooooooool what an amazing football match you should watch it 去停用词去停用词cool amazing football match watchcool amazing football match watch词干化8 cool amaze football matc