WEB日志过滤技术的研究与实现计算机科学与技术专业论文.docxVIP

哈尔滨工业大学工学硕士学位论文 哈尔滨工业大学工学硕士学位论文 - - II - Abstract With the rapid development of Internet, the resources of web log increase more and more. How to use these huge amounts of data to better serve for users becomes hot topic of research nowadays. Web Log Mining applies traditional data mining techniques to web logs to obtain the valuable model of using website applied to personalized service, website design and business decision-making. The data preprocessing in the process of Web Log Mining plays a vital role. The techniques of web logs filtering mentioned in this paper belong to it. Web log filter technology means filtering the logs left after users visiting the website. Its purpose is to filter the logs which are unable to reflect the interest of users, such as picture information, Web site navigation, Site Home and so on. In order to realize web log filter technology, this paper proposes two methods. One is based on the link rules and the other is based on the web content. The process of rules-based filtering technology is that first classify the web logs by the kind of links as particular as possible, and then check out each kind of links to make sure whether contained user interest by manual, at last generate the filtering rules. Rules-based filtering technology’s advantages are high precision and fast in the process of filtering, but its disadvantage is inefficient in statistic phrase and the rules are easy to be disabled with the change of the website. Content-based web log filtering is that to determine whether the log contains the user interest is by the result of text classification of the corresponding web page. This part is the most import method proposed in this paper. Content-based web log filtering mainly contains two parts. One is KNN classification based on the structure information of web page, the other is SVM classification based on the text information of web page, and the result is determined by the two classifiers. Content based log filter is not