Domain-Specific Deep Web Sources Discovery .pdf

Ying Wang, Wanli Zuo, Tao Peng, Fengling HeCollege of Computer Science and Technology, Jilin University, Changchun 130012, Chinawangying810413@163.comAbstractThe web has been rapidly “deepened” with myriadsearchable databases online, where data are hidden behindquery interfaces. However, users often have difficulties infinding the right sources and then querying over them inmyriad useful databases online. For solving this problem,this paper presents a new method by importing focusedcrawling technology to automatically accomplish deep websources discovery. Firstly, locate web sites for Domain- Specific data sources based on focused crawling. Secondly,judge whether the web site exists deep web query interfacein the former three depths. Lastly, judge whether the deepweb query interface is relevant to a given topic. Importingfocused crawling technology makes the identification ofdeep web query interface locate in a specific domain andcapture relative pages to a given topic instead of pursuinghigh overlay ratios. This method has dramatically reducedthe quantity of pages for the crawler to identify deep webquery interfaces.1. IntroductionRecently, the web has been rapidly “deepened” with themyriad searchable databases online, where data are hiddenbehind query interfaces. As Figure 1 conceptuallyillustrates, on this so-called “deep web”, numerous onlinedatabases provide dynamic query-based data accessthrough their query interfaces, instead of static URL links.A study [1][2] in April 2004 estimated 450,000 onlinedatabases. The sources in deep web are estimated to besignificantly large and of higher quality than the “Surfaceweb” indexed by search engines.Figure 1. Deep web sources.However, users often have difficulties in finding the