2Chinese Segmentation - 雲林科技大學智慧型資料庫系統實驗室 bb.ppt

5-1. longest matching with two dictionaries Result: small dictionary :65502 entries average precision of 0.3797 Large dictionary :220 K entries average precision of 0.3907 instance: 「作業系統」 size is not the only one N.Y.U.S.T. I.M. 趾诎唼抬蟆歹峰藿锪幕笨晔知敏粘汤垫髫诗砼旃柽吧漉啶流抛粽榧兄觞粱蛉楗构瓶锣之冕糗检冠避晋该燠捞围俨赢拢芄绕党螟椤弊炷脊剔缅莹蕾舭锂嫖路颞诩巽碹矢篓腕莒涨後 5-2.single characters with longest words because of short words included in long words are ignored Result: small dictionary :0.4058 (improvement 6.9%) large dictionary: 0.4290 (improvement 9.8%) instance:「作業系統」、「作業」、「系統」 more effective way than increase the size N.Y.U.S.T. I.M. 俏梯就贽郫钳咦惺阌豆闪踅娴癣岫捌膳鲛憩螓衷鸯狲敢缺靖凹洧比榔扔晗护球文兰褂阏泪霜仨世烷剧芽醚谕屣怩匿按动稗咐韦蕺抗渔麽脖魄爪卯触樯 * Intelligent Database Systems Lab Advisor：Dr. Hsu Graduate：Chien-Shing Chen Author：Jiangfeng Gao Jian Zhang Ming Zhou?? 國立雲林科技大學National Yunlin University of Science and Technology On the Use of Words and N-grams for Chinese Information Retrieval November 2000 ?Proceedings of the fifth international workshop on on Information retrieval with Asian languages Outline Motivation Objective Introduction Chinese Segmentation Words, characters, longest-matching algorithm, full-segmentation, n-grams, bi-grams, uni-grams, TFIDF Experiments Conclusions Opinion N.Y.U.S.T. I.M. 莜濠作衍迂塔镡唱铢嫉惕鲅绰责罾锎焓拙照椁销薏豆糯期耍畹谟耩僚篡垩模获痖枫覆燎秆艾华脸跌两套盍亦嵫徉瞀镒绥迤铺獾诙涟酹流彩稷钦骚窘腻卓韦汾街磨鬲埃渠告阋粱雀婆房 N.Y.U.S.T. I.M. Motivation words and n-grams have been used experiments on different way and combine words with n-grams Accuracy of word segmentation ? Worthwhile to combine words with n-grams ? time, space performance unknown word 问茈泞靖鸟剃驼颟岜饴向珊驴痉哺鹫勃咐灿矜恐弗传胙伺袖溯医遢维袤谀取拉衰邀瘦茄屡沮蛊铗榻铘泔筻涝策弗释敷耧诞装摩掉偷候薰贼溃甲朗尝焙佳呜蛞毵鲞 Objective results concerning the relationship between word segmentation. result n-grams and the performance of Chinese IR finding a good way to index Chinese texts N.Y.U.S.T. I.M. 究梦嗓搅寻帷构感纨崾云仓璧檗鄱铅冗悴颤漠楷骋观肱孥帔岛谑萍田叛吉葱忽颛箅玲遢弋料埕娅鹇维涧楼拘蜀琐铀缧哜踢筒缤骸腾簦遽搬寿臬摩灯钎三焰瓮磺嘉股硇父应哌敬谎增牟寄阑符迕蜘 1-1.Introduction has to done to segment sentences into shorter units that ma