汉蒙词法分析及其在统计机器翻译中的应用-模式识别与智能系统专业论文.docxVIP

AbstractABSTRACT Abstract ABSTRACT Lexical analysis is a fundamental research of Natural Language Processing (NLP)．Its accuracy has a direct effect on Natural Language Processing(such as machine translation)．This dissertation presents a two—level statistical model for Chinese lexical analysis and Mongolian morphological analysis．The model is evaluated in the training and testing sets which were segmented and tagged by language experts．The results show that the model sighificantly ouperforms previous methods．Moreover,the lexical and morphological information is added in Chinese-Mongolian statistical machine translation system，which is tested on Aligned bilingual corpus．The evaluation results show that the addition of lexical and morphological information improves the quality of the translation． This dissertation systematically introduces the definition of Conditional Random Fields(eRrs)，graphical structure of CRFs model，the potential function， feature functions，training and decoding algorithms．The further improvements of CRFs are given as follows：simplifying the graphical structure of CRFs，designing feature function，improving decoding algorithm；and applying CRFs to Chinese lexical analysis and Mongolian morphological analysis． This dissertation presents a model of Chinese word segmentation based on Local Ambiguity Word Grid and Conditional Random Fields．First，the model uses Local Ambiguity Word Grid algorithm to generate rough segmentation results in the lower level．Then，it segments the text again based on CRFs，and sets the rough results as one feature．The system is tested in the MSRA and PKU testing sets which are provided by the SIGHAN2005 Chinese Language Processing Bakeoff． F-measures of the system in the closed test are 97．1％and 95．1％respectively． Moreover,the dissertation constructs a statistical model for Chinese POS tagging， which could use more context information． Because of Mongolian language feature that achieves morphological changes through connecting