More Probabilistic Models.ppt

More Probabilistic Models Introduction toArtificial Intelligence COS302 Michael L. Littman Fall 2001 Administration 2/3, 1/3 split for exams Last HW due Wednesday Wrap up Wednesday Sample exam questions later… Example analogies, share, etc. Topics Goal: Try to practice what we know about probabilistic models Segmentation: most likely sequence of words EM for segmentation Belief net representation EM for learning probabilities Segmentation Add spaces: bothearthandsaturnspin Applications: no spaces in speech no spaces in Chinese postscript or OCR to text So Many Choices… Bothearthandsaturnspin. B O T H E A R T H A N D S A T U R N S P I N. Bo-the-art hands at Urn’s Pin. Bot heart? Ha! N D S a turns pi N. Both Earth and Saturn spin. …so little time. How to choose? Probabilistic Approach Standard spiel: Choose a generative model Estimate parameters Find most likely sequence Generative Model Choices: unigram Pr(w) bigram Pr(w|w’) trigram Pr(w|w’,w’’) tag-based HMM Pr(t|t’,t’’), Pr(w|t) probabilistic context-free grammar Pr(X Y|Z), Pr(w|Z) Estimate Parameters For English, can count word frequencies in text sample: Pr(w) = count(w)/sumw count(w) For Chinese, could get someone to segment, or use EM (next). Search Algorithm gotothestore Compute the maximum probability sequence of words. p0 = 1 pj = maxij pj-i Pr(wi:j) p5 = max(p0 Pr(gotot), p1 Pr(otot), p2 Pr(tot), p3 Pr(ot), p4 Pr(t)) Get to point i, use one word to get to j. Unigrams Probs via EM g 0.01 go 0.78 got 0.21 goto 0.61 o 0.02 t 0.04 to 0.76 tot 0.74 o 0.02 t 0.04 the 0.83 thes 0.04 h 0.03 he 0.22 hes 0.16 hest 0.19 e 0.05 es 0.09 s 0.04 store 0.81 t 0.04 to 0.70 tore 0.07 o 0.02 or 0.65 ore 0.09 r 0.01 re 0.12 e 0.05 EM for Segmentation Pick unigram probabilities Repeat until probability doesn’t improve much Fractionally label (like forward-backward) Use fractional counts to reestimate unigram probabilities Probability Distribution Represent probability distribution on a bit sequence. A B Pr(