基于GPU计算连续分布式词向量的方法.pdfVIP

基于GPU计算连续分布式词向量的方法
1,2+ 1 1 1 1 吴振华 , 余雷, 马文静, 龙国平 ，李玉成 1(中国科学院 软件研究所 并行软件与计算科学实验室, 北京 100190) 2(中国科学院大学, 北京 100190) ComputingContinuousDistributedWordsRepresentationsonGPU* 1,2+ 1 1 1 1 WUZHENHUA , YULEI , MAWENJING , LONGGUOPING , LIYUCHENG 1(Laboratoryo ParallelSoftwareandComputationalScience,Instituteo Software,ChineseAcademyo Sciences,Beijing,100190, China) 2(Universityo ChineseAcademyo Sciences,Beijing, 100190,China) +Correspondingauthor:E-mail:wuzhenhua10@ Abstract: To solve the problem using machine learning methods in natural language processing field, it’s a fundamental work to expresswordsintovectors.TheWord2vecdevelopedbyGooglegroupisanopensourcetoolandbecomingthemainmethodtoget words representations. By using deep learning models, Word2vec can obtained the continuous distributed words representations, which can roughly compute the distance o two words, and the vector operations between words can capture many linguistic regularities.Thispaperpresents aimplementationtotrain and getthe distributedwordsrepresentations onGPU,including CBOW model and Skip-grammodel, andthere aretwo algorithms in eachmodel, calledhierarchical Softmaxandnegative sample.Inthe hierarchical Softmaxalgorithm,weuse afullbinary treeto insteadHu mantree, make itmore easytoimplement onGPU;Using someartfulwaystosetsomeparameter suchaslearningrateandthreadsconfiguration,improvedtheperformanceandthequalityo wordsrepresentations. In theNVIDIATesla C2050 GPU,theperformance has reached 14to 28times compareto singlethread in CPU,and2to7times comparetomultiplethreadsinCPU.Whatmore,theaccuracyorqualityo wordsrepresentations arealittle betterthantheresultgotfromCPU. Keywords:Distributedwordsrepresentations;Word2vec;Naturallanguageprocessing;DeepLearning;GPU 摘 要 在用机器学习方法解决自然语言处理领域的问题 将词或符号表示成向量是一项基本工作: ,