基于两字词簇汉语快速自动分词算法.pdfVIP

17 5 Vol . 17 , N o, 5 199 8 10 Oc ober, 199 8 郭祥昊 钟义信 杨 丽 ( , 100876) ( , 100044) 75% , , , , , A Fast Algorithm for Chinese Words Automatic Segment Based on Two-letters-word-family Structure Guo Xiang hao and Zhong Yix in (AI Lab, Beij ing University of Post and Telecommunication, Beij ing 100876) Y ang Li ( CECE Center, orthern Jiaotong University, Beij ing 100044) Abstract A f as algori hm for Chinese w or ds au oma ic seg men is pu for w ard in his paper. A s ruc ure called / w o- le ers- w ord- f amily0 w hich is he collec ion of all he Chinese w ords ha share he same beginning w o le ers is in ro duced. T he key idea of he algo ri hm is o compress he w ords w hich consis of more han hr ee Chinese le ers in o w o- le ers- w or d- f amily and handle oge her u sing leng h changing max imum ma ching algori hm . I n addi ion o his, a new me hod o de ec segmen ing ambigu- ousness is also in roduced. Keywords na ural languag e processing , Chinese w o rds au oma ic seg men ing, segmen ing ambigu- ousness. 1 , : 1998 1 12 : , 1970 , , , 1940 , , , , 1971 , , 5 353 ( In erne , ) , , ( Maxi- mum M a ching, MM ) , , , : ¹N , M , , ( M- 2) N + N/ 2 N , º M M , ( 1) M M , M , [ 1] , » ( Reverse Maximum M a ching) , , MM 1 M M