《统计自然语言处理与信息检索》第10讲机器翻译.pptVIP

* 三个问题：翻译模型、语言模型和解码模型。 * 具体来讲又一个ｂａｙｅｓ公式的解释。 因此，要做ｓｍｔ必须要解决三个问题 语言模型的解释，那个更像句子？这就是语言模型在起作用。 惊呆的语言模型 * * the course contains the following topics: History, rule based 机器翻译的评价方法 * 稽核平均值 N元匹配个数。 r/c corpus level r/c sentence level Bilingual evaluation understudy 因此，译文句子的相对于参考译文的长短，对翻译的精确率有很大的影响。 由于长的句子，在计算bigram匹配的时候，因为匹配是一次性匹配，也就是匹配上一次，便拿掉。因此，句子越长，匹配不上的可能性越大，因此自然又惩罚。 因此主要要考虑句子短的句子。 IBM的公式是这样的。 首先他搞一个系数。当句子长度大于参考译文的长度时，惩罚系数伟1，意味着不惩罚。 当局资长度小雨参考译文的长度时，惩罚系数是， BLEU指就是指翻译的评价指， 是unigram, bigram, trigram, N-gram 平价值的几何平均值。然后加上一个根句长有关的惩罚系数。 BLEU detailed equation (Ming) * the course contains the following topics: History, rule based * Parallel corpora are critical resources for machine translation. However, for most language pairs and domains, the parallel corpora are not readily available. With the sharp increase of web data, web mining become a promising solution to this knowledge bottleneck. Google also make big effort in web mining parallel data for machine translation. This is also show the importance of this project. * * The mining procedure consists of 3 steps: given input sources, candidate URL pairs are first identified. Then parallel documents are mined from the candidate URLs. Finally, with sentence alignment, parallel sentences are extracted from parallel documents. Three sources are used: web directory, MSN search engine, 5 billion URLs provided by MSN search. For the web directory like what is provided by Yahoo, each website listed in the web directory are examined for bilingual websites by looking for some trigger words. For example, here is a English web page. The anchor text here, Chinese version, is a clear clue for a bilingual website. We can also search for bilingual websites by using MSN. The query consists of some trigger words together with some content keywords. For example, to search for bilingual websites in travel domain, the following query are used: … The following are MSN returns. The top returned URL actually refer to such a website.