Supporting Annotation Layers for Natural Language Processing支持自然语言处理的注释图层.pptVIP

April 16, 2004 Supporting Annotation Layers for Natural Language Processing Using the Web as an Implicit Training Set:Application to Structural Ambiguity Resolution Preslav Nakov and Marti HearstComputer Science Division and SIMSUniversity of California, Berkeley Motivation Huge datasets trump sophisticated algorithms. “Scaling to Very Very Large Corpora for Natural Language Disambiguation”, ACL 2001 (Banko Brill, 2001) Task: spelling correction Raw text as “training data” Log-linear improvement even to billion words Getting more data is better than fine-tuning algorithms. How to generalize to other problems? Web as a Baseline “Web as a baseline” (Lapata Keller 04;05): applied simple n-gram models to: machine translation candidate selection article generation noun compound interpretation noun compound bracketing adjective ordering spelling correction countability detection prepositional phrase attachment All unsupervised Findings: Sometimes rival best supervised approaches. = Web n-grams should be used as a baseline. Our Contribution Potential of these ideas is not yet fully realized We introduce new features paraphrases surface features Applied to structural ambiguity problems Data sparseness: need statistics for every possible word and for word combinations Problems (unsupervised): Noun compound bracketing PP attachment NP coordination Task 1: Prepositional Phrase Attachment PP attachment (a) Peter spent millions of dollars. (noun) (b) Peter spent time with his family. (verb) quadruple: (v, n1, p, n2) (a) (spent, millions, of, dollars) (b) (spent, time, with, family) Related Work Supervised (Brill Resnik, 94): transformation-based learning, WordNet classes, P=82% (Ratnaparkhi al., 94): ME, word classes (MI), P=81.6% (Collins Brooks, 95): back-off, P=84.5% (Stetina Makoto, 97): decision trees, WordNet, P=88.1% (Toutanova al., 04): morphology, syntax, WordNet, P=87.5% (Olteanu Moldovan, 05): in context, parser, FrameNet, We