2016 Automatic Evaluation of Chinese Translation Output_Word-Level or Character-Level.pdfVIP

Automatic Evaluation of Chinese Translation Output: Word-Level or Character-Level Maoxi Li Chengqing Zong Hwee Tou Ng National Laboratory of Pattern Recognition Department of Computer Science Institute of Automation, Chinese Academy of National University of Singapore Sciences, Beijing, China, 100190 13 Computing Drive, Singapore 117417 mxli, cqzong @ nght@.sg machine translation SMT , it can happen that the Abstract most accurate word segmentation as judged by the human gold-standard segmentation may not Word is usually adopted as the smallest unit in most tasks of Chinese language processing. produce the best translation output Zhang et al., However, for automatic evaluation of the quali- 2008 . While state-of-the-art Chinese word ty of Chinese translation output when translat- segmenters achieve high accuracy, some errors still ing from other languages, either a word-level remain. approach or a character-level approach is possi- Instead of segmenting a Chinese sentence into ble. So far, there has been no detailed study to words, an alternative is to split a Chinese sentence compare the correlations of these two ap- into characters, which can be readily done with proaches with human assessment. In this paper, perfect accuracy. However, it has been reported we compare word-level metrics with character- that a Chinese-English phrase-based SMT system level metrics on the submitted output of Eng- Xu et al., 2004 that relied on characters without lish-to-Chinese translation systems in the IWSLT’08 CT-EC and NIST’08 EC tasks. Our CWS performed slightly worse than when it used experimental results reveal that character-level segmented words. It has been recognized that vary- metrics correlate with human assessment better ing segmentation granularities are needed for SMT than word-level metrics. Our analysis suggests Chang et al., 2008