在深度链接中查询结构化数据.pptVIP

Deep Web Integration:Querying Structured Data on the Deep Web Fangjiao Jiang Outline Background Access Deep Web MetaQuerier Metasearch engine vs. MetaQuerier Related research groups Conclusion … Some suggestions Background The previous Web: things are just on the surface The current Web: Getting “deeper” A great number of data is hidden behind query forms The Problem for access data from Deep Web Deep = not accessible through traditional search engines Why is it important? More than 10 million distinct forms Why is it important? Up to 5,000 billions dynamic result pages Why is it important?  ——Google’s Recent Survey [CIDR 2007] If there are 1 billion web pages 25 million potential Deep Web sources Challenge: How to enable effective access to the Deep Web? Access the Deep Web Three different manners Warehouse-like approach MetaQuerier Surfacing the Deep Web (1) Warehouse-like approach (2) MetaQuerier (3) Surfacing the Deep Web [VLDB’08] Viewpoint Many domains and many languages No human in the loop, no site-specific scripts Main idea predicting input values for text boxes predicting input combinations Google’s Deep-Web crawling system Affects more than 1000 queries per second Enables access to more than a million Deep-Web sites Spans 50+ languages and 100+ domains MetaQuerier A Survey on Deep Web [SIGMOD 2006] How many deep-Web sources are out there? 307,000 sites, 450,000 DBs, 1,258,000 interfaces. How structured in Deep Web? 348,000 (structured) : 102,000 (text) == 3 : 1 How do search engines cover them? covered 10% sources. What’s the subject distribution of Web databases? Across all areas How complex are they? “Amazon effects” Reported the “Amazon effect”… Technical Challenges How to discover the query interface? Which form is the query interface of a Web database? How to understand a query interface? Where is the first condition? What’s its attribute? How to match query interfaces? What does “author” on this source