Probabilistic Part-of-Speech Tagging Using Decision Trees-英文文献.pdfVIP

Probabilistic PartofSp eech Tagging Using Decision Trees  Helmut Schmid y IMSCL Institut f ur maschinelle Sprachverarb eitung Universitat Stuttgart Azenb ergstr D Stuttgart Germany email schmidimsunistuttgartde Abstract In this pap er a new probabilistic tagging metho d is presented which avoids problems that Markov Mo del based taggers face when they have to estimate transition probabilities from sparse data In this tagging metho d transition probabilities are estimated using a decision tree Based on this metho d a partofsp eech tagger called TreeTagger has b een implemented which achieves accuracy on PennTreebank data which is b etter than that of a trigram tagger on the same data Keywords Corpusbased NLP Statistical NLP PartofSp eech Tagging Intro duction Word forms are often ambiguous in their partofsp eech POS The English word form store for example can b e either a noun a nite verb or an innitive In an utterance this ambiguity is normally resolved by the context of a word eg in the sentence The PCs could store two pages of data store can only b e an innitive The predictability of the partofsp eech from the context is used by automatic partofsp eech taggers Several metho ds have b een prop osed to annotate words automatically with partofsp eech tags Some researchers used rulebased systems Greene and Rubin Brill Others imple mented probabilistic metho ds eg Bahl and Mercer Church Cutting et al DeRose Kemp e