中国人民大学图书馆培训课件一.pptVIP

龙星计划课程:信息检索?? Statistical Language Models for IR Outline More about statistical language models in general Systematic review of language models for IR The basic language modeling approach Advanced language models KL-divergence retrieval model and feedback Language models for special retrieval tasks More about statistical language models in general What is a Statistical LM? A probability distribution over word sequences p(“Today is Wednesday”) ? 0.001 p(“Today Wednesday is”) ? 0.0000000000001 p(“The eigenvalue is positive”) ? 0.00001 Context/topic dependent! Can also be regarded as a probabilistic mechanism for “generating” text, thus also called a “generative” model Why is a LM Useful? Provides a principled way to quantify the uncertainties associated with natural language Allows us to answer questions like: Given that we see “John” and “feels”, how likely will we see “happy” as opposed to “habit” as the next word? (speech recognition) Given that we observe “baseball” three times and “game” once in a news article, how likely is it about “sports”? (text categorization, information retrieval) Given that a user is interested in sports news, how likely would the user use “baseball” in a query? (information retrieval) Source-Channel Framework(Model of Communication System [Shannon 48] ) Basic Issues Define the probabilistic model Event, Random Variables, Joint/Conditional Prob’s P(w1 w2 ... wn)=f(?1, ?2 ,…, ?m) Estimate model parameters Tune the model to best fit the data and our prior knowledge ?i=? Apply the model to a particular task Many applications The Simplest Language Model(Unigram Model) Generate a piece of text by generating each word independently Thus, p(w1 w2 ... wn)=p(w1)p(w2)…p(wn) Parameters: {p(wi)} p(w1)+…+p(wN)=1 (N is voc. size) Essentially a multinomial distribution over words A piece of text can be regarded as a sample drawn according to this word distribution Text Generation with Unigram LM Estimation of Unigram LM Em