基于网站特征的钓鱼网页检测研究与实现.docVIP

南京邮电大学 毕 业 设 计（论 文） 题 目 基于网站特征的钓鱼网页检测研究与实现 专 业 软件工程 学生姓名 贡亮 班级学号 B060314 指导教师 张卫丰 指导单位 计算机学院 日期：2010年 3月 15日至 2010年 6月 18日 摘 要 本文首先介绍了网络钓鱼的概念，以及研究目的和意义，并且提出了两种全新的钓鱼网页检测方法：1)基于网络拓扑的检测方法；2)基于空间数据索引的检测方法；其中方法1分析了钓鱼网站和被攻击网站以及普通网站在拓扑上的差异，进而首次提出了一种从网络拓扑角度着手的钓鱼页检测方法。使用网络爬虫根据可疑的网页网址抓取一系列相关的网页，然后根据抓取的网页集合分析目标网站的特性，抽取网站的拓扑特征，最后使用训练后的分类器判断目标网站是否为钓鱼网站。方法2是一种基于空间布局的钓鱼网页检测方法涉及了一种基于页面视觉布局特征结合空间数据库的设计方案，主要解决了基于网页视觉相似性角度进行快速钓鱼网页检测的问题，首先对整个网页抽取自创的网页布局特征，之后所有布局特征送入空间数据库进行统一索引，便于后期查询；最后使用我们开发的机器学习匹配模块接收特征数据进行训练，优化网页相似性阀值的参数。本文使用多种技术和算法验证了检测效果，经大量实验数据检验证明，两种方法均构建了一种高速高精度（99.1%）和召回率(99.1%)的钓鱼网页检测系统，在保证高准确率的同时，处理数据容量上有了重大提升，并显著减小网页检测时间。 关键词：钓鱼网页检测；网站分析；Web文档分析；网页抓取；拓扑分析； 空间数据索引；网页布局分析；布局特征块； ABSTRACT Phishing lure user into leaking sensitive information by imitating the legal websites. This kind of fraud sites has made a great financial lost to victims all around the world. In this paper, we proposed two different new phish detection methods. 1)Anti-phishing based on site’s topology 2) Anti-phising based on layout features and spatial database. In the first method, a web crawler is deployed to pre-fetch a series of web pages related to the suspicious URL, then 15 kinds of chosen topological features are extracted from pre-fetch web pages. At last, a classifier were trained and deployed to classify target website. Our method basically consists of three core modules: 1) Layout Feature Extractor; 2) Spatial database; 3) Machine learning module. Firstly, we proposed a kind of block feature to represent web page’s layout and designed a feature extractor using web browser. Then a Spatial database is redesigned and deployed for indexing all the features according to there spatial relationships. At last a all blocks which is similar to the feature extracted from suspicious page are analyzed by machine learning module designed by us. Various methods were selected to analysis the features and lots of data-mining algorithms were used to evaluate our methods. As shown by tests