SMT200的7-3.pptVIP

Lecture 3 Word alignment and EM algorithm References Kevin Knight, 1999, A Statistical MT Tutorial Workbook, prepared in connection with the JHU summer workshop, April 30, 1999 Philipp Koehn, 2007, Empirical Methods in Natural Language Processing, Lecture 15, http://www.inf.ed.ac.uk/teaching/courses/ emnlp Michael Collins, 1997, The EM algorithm, in fulfillment of the Written Preliminary Exam II requirement, Sept. 1997, / 93356.html Outline of Lecture 3 3.1 Examples about word alignment 3.2 Descriptions on EM algorithm 3.3 Examples and a demo to implement EM algorithm 3.4 Relevant concepts for EM algorithm 3.5 Properties of EM algorithm 3.1 Examples about word alignment Examples about word alignment (1) Word alignment Example 1: German ? English Examples about word alignment (2) Formalizing alignment with an alignment function Mapping an English target word at position i to a German source word at position j with a function a : i ? j For Example 1 Examples about word alignment (3) Example 2: reordering Examples about word alignment (4) Example 3: one-to-many translation Examples about word alignment (5) Example 4: dropping words Examples about word alignment (6) Example 5: inserting words Examples about word alignment (7) Example 6: a real example Examples about word alignment (8) Collect statistics from parallel corpus Examples about word alignment (9) lexical translations from Model 1 3.2 Descriptions on EM algorithm Descriptions on EM algorithm (1) Lexical translation and word alignment: chicken and egg problem Descriptions on EM algorithm (2) EM’s two steps Descriptions on EM algorithm (3) E-step: calculate p(a|e,f) M-step: calculate c(e|f;e,f) re-compute t(e|f;e,f) over whole corpus Descriptions on EM algorithm (4) From Model 1: E-step Descriptions on EM algorithm (5) M-step: Collect counts Equation (17) of Model 1 After collecting these counts over a corpus, we can re-estimate the model, i.e. calculate t(e|f;e,f) again: Descriptions on EM algorithm (6)