分布式话题检测方法分析-电力系统及其自动化专业论文.docx

哈尔滨工业大学工学硕士学位论文 哈尔滨工业大学工学硕士学位论文 - - II - Abstract With the rapid prog of social media, people interact with the Internet more frequently and easily than before, and Internet resources grow explosively. The rich Internet resources bring the convenience of social resources and the trouble to find useful resources at the same time. To solve the problem above, this paper have a research on distributed topic detection. The research focus on optimizing the precision and speed of topic detection. The main research achievements are summarized as follows: First, Do the prophase survey and study the origin and correlation algorithms of topic detection. Research the topic detection algorithm, and implement original Single-Pass algorithm. Second, This paper propose the secondary topic detection method named Double-Pass base on the Single-Pass. The aim of this algorithm is to let the cluster information of the first detection guide the second one. To strengthen the c luster information, the CFC-DP algorithm is come up with by applying the CFC theory, which is used to classification in other paper, and it is used to topic detection in this paper. The experiment shows that Double-Pass topic detection is better than previous ones from the aspect of F value, and CFC-DP topic detection is better than Double-Pass. Third, To improve the speed of topic detection and meet the need of handling voluminous chunks of data, this paper propose a distributed topic detection method based on Hadoo, definite the operation of task decomposition and composition. The experiment shows that distributed topic detection can improve its speed greatly with good F value at the same time. Finally, based on the above study, the prototype system of distributed topic detection is designed and implemented. The prototype system includes the following Five modules: Data reading modules, Data preprocessing module, topic detection module, distributed procession module, topic saving and show module. Key words: topic