批量图片下载器设计论文.docVIP

Mass picture downloader 图片批量下载器 摘 要 批量下载图片可以方便用户从一个网站下载大量图片，节省用户时间，给用户带来便利。本文设计和实现一个基于正则表达式的图片批量下载器。 本文首先在分析网上现有的图片下载器的工作原理及运行结果的基础之上，针对原有的爬虫搜索所有URL的下载方式所存在的不合理性与盲目性，基于正则表达式对URL进行判定和解析，设计了新的图片下载器的总体架构，且该架构主要包括爬虫模块和正则表达式自动产生模块两个部分。 本文对爬虫模块的设计和实现原理进行了详细阐述。爬虫模块设计新的下载方式，要求用户指定的正则表达式为图片URL的提取标准，使得程序可以更快搜索并下载下来图片。程序根据用户指定的正则表达式再结合爬虫程序来提取出一个网页中的地址，直到到达指定层数，最后执行下载。在分析URL的同时，程序亦建立网站层次树，以便用户查看那些未下载成功的图片在网站上的确切位置，从而可以选择重新下载。 本文详细介绍了正则表达式自动模块的设计和实现，该模块通过用户输入的ＵＲＬ，从而得到一个比较高效的正则表达式，以供图片批量下载器使用。并且程序对一个正规的图片网站执行下载，运行结果表明，该程序能够快速准确的完成用户指定的下载任务，且使用较为方便，大大节省了用户的时间。 论文最后对全文做出了总结，并对未来研究方向进行了展望。 关键词：图片下载，正则表达式，爬虫程序，动态规划算法，ＵＲＬ规范 ，JAVA Mass picture downloader Abstract Mass picture downloader is a software that can save users time by downloading thousands of pictures from a website automatically.This thesis is about the design and the realization of the mass picture downloader.A new architecture of the software has been built，which includes two parts called crawler and regular expression generator. Basing on the analysis of the working mechanism and the results of existent mass picture downloaders，This thesis describes a new way that is different from traditional blindly search on URLs using a Crawler.The new way is based on the analysis to the URLs by using regular expression. This thesis describes the design and the realization of the crawler in detail.A new way has been designed for the crawler part.In the new way，a regular expresssion appointed by the user is used to extract the URLs from a webpage，which，indeed，make the search and downloading much faster.The program extracts a URL from a webpage according to the regular expression，then gets the content of the webpage indexed by the extracted URL，and then analyzes the content of the webpage by using the regular expresssion next layer，until reaches the designated layer count.During the analysis of the URLs，a websiteTree is being built，in order to do the users a favor when they want to see the address of the downloading-unfinis