ppt课件-advancingthe vibrancyof computing（advancingthe vibrancyof计算）.pptVIP

Clustering: automatic Minimize perplexity of P(z|Y)  Mathematical tricks speed it up Use top-down splitting, not bottom up merging! Two actual WSJ classes MONDAYS FRIDAYS THURSDAY MONDAY EURODOLLARS SATURDAY WEDNESDAY FRIDAY TENTERHOOKS TUESDAY SUNDAY CONDITION PARTY FESCO CULT NILSON PETA CAMPAIGN WESTPAC FORCE CONRAN DEPARTMENT PENH GUILD Sentence Mixture Models Lots of different sentence types: Numbers (The Dow rose one hundred seventy three points) Quotations (Officials said “quote we deny all wrong doing ”quote) Mergers (AOL and Time Warner, in an attempt to control the media and the internet, will merge) Model each sentence type separately Sentence Mixture Models Roll a die to pick sentence type, sk with probability ??k Probability of sentence, given sk Probability of sentence across types: Sentence Model Smoothing Each topic model is smoothed with overall model Sentence mixture model is smoothed with overall model (sentence type 0) Sentence Mixture Results Sentence Clustering Same algorithm as word clustering Assign each sentence to a type, sk Minimize perplexity of P(z|sk ) instead of P(z|Y) Outline Prob theory intro Text prediction Intro to LM Perplexity Smoothing Caching Clustering Parsing CFG Homework Structured Language Model “The contract ended with a loss of 7 cents after” Thanks to Ciprian Chelba for this figure How to get structure data? Use a Treebank (a collection of sentences with structure hand annotated) like Wall Street Journal, Penn Tree Bank. Problem: need a treebank. Or – use a treebank (WSJ) to train a parser; then parse new training data (e.g. Broadcast News) Re-estimate parameters to get lower perplexity models. Parsing vs. Trigram Eugene Charniaks’s Experiments All experiments are trained on one million words of Penn tree-bank data, and tested on 80,000 words. 18% Thanks to Eugene Charniak for this slide Model Perplexity Trigram poor smoothing