Computational Linguistics 计算语言学.ppt

9/13/1999 Intro to NLP JHU CS 600.465/Jan Hajic Computational Linguistics计算语言学 史晓东 Xiamen University mandel@ 46/course_cl.html outline n元语言模型 平滑算法（1） 平滑算法（2） 高级话题 Statistical LM Toolkit 看几幅漫画 What is a Language Model？ A language model is a probability distribution over word sequences P(“And nothing but the truth”) ?? 0.001 P(“An nuts sing on de roof”) ? 0 语言模型对每个词串（句子）都赋予一个概率。对合法串，概率高；不合法的，概率小。 The sum of probabilities of all word sequences has to be 1. A bad language model 在听不清的情况，每个人试图猜测说话者说的话（专业术语叫解码），选取自己认为最合理的。 证人的语言模型很糟糕。说出来的话不像英语（DE）。估计听不大懂英语 我的汉语语言模型也很糟糕 记得小时候的歌：“走在乡间的小路上”，？？的老牛是我同伴 暮归 魔鬼 磨谷 ？？ 概率模型的好处 语言老师说，“只能这样说！”（prescriptive） 语言模型说，“这样说比较好！”，“大家都这样说！” 连续模型 － 离散模型（我们的讨论对象） 在许多应用中语言模型不可或缺 Speech recognition Handwriting recognition Optical character recognition（OCR） Spelling correction Information retrieval Machine translation 比如语音识别 如何计算概率：Chain rule P(“And nothing but the truth”) = P(“And”) ?P(“nothing|and”) ? P(“but|and nothing”) ? P(“the|and nothing but”) ? P(“truth|and nothing but the”) Markov approximation Assume each word depends only on the limited local context, e.g. on previous two words. This is called trigram models P(“the|… whole truth and nothing but”) ? P(“the|nothing but”) P(“truth|… whole truth and nothing but the”) ? P(“truth|but the”) With Markov Assumption Caveat The formulation P(Word| Some fixed prefix) is not really appropriate in many applications. It is only the case if we’re dealing with real time speech where we only have access to prefixes. But if we’re dealing with text we already have the right and left contexts. There’s no a priori reason to stick to left contexts. 对w1w2w3, 可以用P(w3|w1w2)，也可以用P(w2|w