分布式随机方差消减梯度下降算法topkSVRG-计算机科学与探索.PDFVIP

ISSN 1673-9418 CODEN JKYTA8 E-mail: fcst@vip. 163.com Journal of Frontiers of Computer Science and Technology 1673-9418/2018/ 12(07)- 1047-08 Tel: +86-10 doi: 10.3778/j.issn. 1673-9418.1705044 分布式随机方差消减梯度下降算法topkSVRG* 1,2 1,2 1+ 1 王建飞 ，亢良伊 ，刘 杰 ，叶 丹 1. 中国科学院 软件研究所 软件工程技术研发中心，北京 100190 2. 中国科学院大学，北京 100190 Distributed Stochastic Variance Reduction Gradient Descent Algorithm topkSVRG WANG Jianfei1,2 , KANG Liangyi1,2 , LIU Jie1+ , YE Dan1 1. Technology Center of Software Engineering, Institute of Software, Chinese Academy of Sciences, Beijing 100190, China 2. University of Chinese Academy of Sciences, Beijing 100190, China + Corresponding author: E-mail: ljie@ WANG Jianfei, KANG Liangyi, LIU Jie, et al. Distributed stochastic variance reduction gradient descent algorithm topkSVRG. Journal of Frontiers of Computer Science and Technology, 2018, 12(7) ：1047-1054. Abstract: Machine learning problems are usually converted into an objective function to optimize, and the optimiza- tion algorithms are the important tool to solve the parameters in the objective function. Currently, stochastic gradient 索 descent (SGD) is one of the most widely used optimization methods, but it is susceptible to noise gradient so as to 探 get sub- linear convergence rate. The improved stochastic variance reduction gradient (SVRG) can achieve linear convergence rate, but SVRG is a serial algorithm. In order to deal with the distributed training problem of large-