北大信息检索搜索引擎课件0 Introduction-IR.pptVIP

* Probabilistic model Given D, estimate P(R|D,Q) and P(NR|D,Q) P(R|D,Q)=P(D|R,Q)*P(R,Q)/P(D,Q) P(D,Q), P(R,Q) constant ? P(D|R,Q) D = {t1=x1, t2=x2, …} * Prob. model (cont’d) For document ranking * Prob. model (cont’d) How to estimate pi and qi? A set of N relevant and irrelevant samples: ri Rel. doc. with ti ni-ri Irrel.doc. with ti ni Doc. with ti Ri-ri Rel. doc. without ti N-Ri–n+ri Irrel.doc. without ti N-ni Doc. without ti Ri Rel. doc N-Ri Irrel.doc. N Samples * Prob. model (cont’d) Smoothing (Robertson-Sparck-Jones formula) When no sample is available: pi=0.5, qi=(ni+0.5)/(N+0.5)?ni/N May be implemented as VSM: dot product xi: presence / absence of a term * BM25 k1, k2, k3, d: parameters qtf: query term frequency dl: document length avdl: average document length Doc. length normalization TF factors * (Classic) Presentation of results Query evaluation result is a list of documents, sorted by their similarity to the query. E.g. doc1 0.67 doc2 0.65 doc3 0.54 … * System evaluation Efficiency: time, space Effectiveness: How is a system capable of retrieving relevant documents? Is a system better than another one? Metrics often used (together): Precision = retrieved relevant docs / retrieved docs Recall = retrieved relevant docs / relevant docs relevant retrieved retrieved relevant * General form of precision/recall Precision change w.r.t. Recall (not a fixed point) Systems cannot compare at one Precision/Recall point Average precision (on 11 points of recall: 0.0, 0.1, …, 1.0) * An illustration of P/R calculation List Rel? Doc1 Y Doc2 Doc3 Y Doc4 Y Doc5 … Assume: 5 relevant docs. * MAP (Mean Average Precision) rij = rank of the j-th relevant document for Qi |Ri| = #rel. doc. for Qi n = # test queries E.g. Rank: Q1 Q2 1 4 1st rel. doc. 5 8 2nd rel. doc. 10 3rd rel. doc. * Some other measures Noise = retrieved irrelevant docs / retrieved docs Silence = non-retrieved relevant docs / relevant docs