A new similarity computing method based on concept similarity in Chinese text processing.pdfVIP

ScencefnChnnS已rf8sF ，n厂0丌 fDn cfence wwW．sCiChina．COm infO．SCiChina．cOm @ 2008 SClENCEINCHINAPRESS www．springer¨nk．cOm Sp一ringer Anew simiIaritycOmputingmethOdbasedOn CcOOnnCCeeDDttSslimmliIIaarri1ttVVi1nnC乙hni1nneeSseetteeXxttDDrrOOCceeSsSsi1nngg PENGJjng’，2十YANGDongQing，TANGShiWei’WANGTengJiaO’＆GAOJun’ ， ， ’SchooIofEIectrDnicsEngineeringandComputerScience，PekingUniVersjty，Be_jing1O0871，China； 。Department0fScienceand。rechnoIogy ChengduMunicipaIPubIicSecurity，Bureau，Chengdu61O017 ， China ThepaperprOposesanew textsimiIarity computingmethodbasedon conCept sim llarityjnChinesetextprOcessing．Thenew methOdconvertstexttOwOrdsVec· tOrspacemOdeIatfirst，and then spIitswOrds into a setOfcOncepts．ThrOugh cOmputingtheinnerproductsbetweenconcepts，itObtainsthesim_lar-tybetween wOrds．The new methodcOmputesthesim_larityOftextbased OnthesimiIarity0f wordsatlast．ThecontributionsofthepaperincIude：1 proposeanewcomputing formulabetweenwords；2 proposeanew te】ctsim_laritycompUtingmethodbased onwordssim-larity；3 successfu¨yusethemethodintheapplicationofsimiIarity computingofWEBnews；and4 provethevaIidityofthemethodthroughextensiVe experiments． concep1similar ，simIlaritycompung，Vec1orspace，|nnerpr0ductspace 1 IntrOduCti0n Thetextinfom atiOnretrieValisanimpOrtantpartintheinfomlatiOntechnOlOgy’andtexts1mllar— itycomputingisOneofthehOt—spOtresearchfieldsintextinf0肌 atiOnsearches．ThepurpOse0f sirnjlaritycomputingpr0posedinthispaperistocOmparethesimilardegreebetweenanypairsof textstrings．ThiscOmputingmethOdcanbewidelyusedintheareaOfnaturallanguageprOcess， such astextautOmatica1classificatiOn，c1ustering，fuzzy query，machinetmnslation，andautO— matictab1oid． ThetypicalmOdelsintextsimilaritycomputinginc1udeBOOleanm0de1，VectOrspacemOde1 VSM ，andpr0babilisticmode1．ThesemodelsrespectiVe1yusedi仟erentmethodst0pr0cess characterweight，classmablestudy，andtheVSM isoneofthemostef-fectivemodels． ReceivedAugust28，2oo7；d|cceptedJanuarv1l，2008 d0i：l0．1OO7／sl1432一o08．0103—4 十corresponding