IEbyCandidateClassificationJansenAbney,Cohenetal.ppt

IE by Candidate Classification:Califf Mooney William Cohen 1/21/03 Where this paper fits Example task for Rapier Rapier: the 3-slide version A bottom-up rule learner: initialize RULES to be one rule per example; repeat { randomly pick N pairs of rules (Ri,Rj); let {G1…,GN} be the consistent pairwise generalizations; let G* = Gi that optimizes “compression” let RULES = RULES + {G*} – {R’: covers(G*,R’)} } where compression(G,RULES) = size of RULES- {R’: covers(G,R’)} and “covers(G,R)” means every example matching G matches R Rapier: an alternative approach Combines top-down and bottom-up learning Bottom-up to find common restrictions on content Top-down greedy addition of restrictions on context Use of part-of-speech and semantic features (from WORDNET). Special “pattern-language” based on sequences of tokens, each of which satisfies one of a set of given constraints tok{‘ate’,’hit’},POS{‘vb’}, tok{‘the’}, POS{‘nn’ Rapier: the N slide version Rapier: IE with “rules” Rule consists of Pre-filler pattern Filler pattern Post-filler pattern Pattern composed of elements, and each pattern element matches a sequence of words that obeys constraints on value, POS, Wordnet class of words in sequence Total length of sequence Example: “IBM paid an undisclosed amount” matches PreFiller: pos=nn or nnp:1 anyword:2 Filler: word=‘undisclosed’:1 PostFiller: semanticClass=‘price’ A rule might match many times in a document Algorithm:1 Algorithm:2 An “obvious choice” Some sort of search based on this primitive step: Pick a PAIR of rules R1,R2 Form the most specific possible rule that is more general than either R1 or R2 But in Rapier’s language there are multiple generalizations of R1 and R2… Algorithm: 3 Algorithm 4: pairwise generalization of patterns Algorithm 4: pairwise generalization of patterns Algorithm 4: pairwise generalization of patterns Algorithm 4: pairwise generalization of patterns Algorithm: 5 Evaluation of Rules Results Extraction by Sliding Window A “N