MethodsforDomain-IndependentInformationExtractionfrom.pptVIP

Methods for Domain-Independent Information Extraction from the Web An Experimental Comparison Oren Etzioni et al. Prepared by Ang Sun asun@cs.nyu.edu 2009-03-01 1. Motivation Recall from previous papers such as Snow 04 about Hypernym/hyponym extraction that their systems’ recall are low. Motivation of this paper: How can we improve system’s recall and extraction rate without sacrificing precision? 2. KnowItAll: The Baseline System KnowItAll is an autonomous, domain-independent system that extracts facts, concepts, and relationships from the Web (Etzioni et al. 2004). Overview of KnowItAll’s architecture 1) Generate: KNOWITALL utilizes a set of eight domain-independent extraction patterns to generate candidate facts (cf. (Hearst 1992)). 2) Test: KNOWITALL automatically tests the plausibility of the candidate facts it extracts using pointwise mutual information (PMI) statistics computed by treating the Web as a massive corpus of text. 2. KnowItAll: The Baseline System(Cont’) Extractor: The Extractor automatically instantiates domain-independent extraction patterns to class-specific patterns. NP1 such as NPList2 head(NP1)= plural(Class1) properNoun(head(each(NPList2))) = instanceOf(Class1, head(each(NPList2))) keywords: plural(Class1) such as” For example, if Class1 is set to “City” then the rule spawns the search engine query “cities such as”, downloads the Web pages, and extracts every proper noun immediately following that phrase as a potential city. Search Engine Interface: (They actually used Google API) For the above example, it would lead KNOWITALL to 1) automatically formulates queries based on its extraction rules and in this example it issues the search-engine query “cities such as”; 2) downloads in parallel all pages named in the engine’s results; 3) applies the Extractor to the appropriate sentences on each downloaded page. 2. KnowItAll: The Baseline System(Cont’) Assessor: Uses pointwise mutual information (PMI) to assess the