MapReduce for Machine Learning on Multicore（MapReduce多核机器学习）.pdf

Map-Reduce for Machine Learning on Multicore Cheng-Tao Chu Sang Kyun Kim Yi-An Lin chengtao@ skkim38@ ianl@ YuanYuan Yu Gary Bradski Andrew Y. Ng yuanyuan@ garybradski@gmail ang@ Kunle Olukotun kunle@ . CS. Department, Stanford University 353 Serra Mall, Stanford University, Stanford CA 94305-9025. . Rexee Inc. Abstract We are at the beginning of the multicore era. Computers will have increasingly many cores (processors), but there is still no good programming framework for these architectures, and thus no simple and unified way for machine learning to take advantage of the potential speed up. In this paper, we develop a broadly ap- plicable parallel programming method, one that is easily applied to many different learning algorithms. Our work is in distinct contrast to the tradition in machine learning of designing (often ingenious) ways to speed up a single algorithm at a time. Specifically, we show that algorithms that fit the Statistical Query model [15] can be written in a certain “summation form,” which allows them to be easily par- allelized on multicore computers. We adapt Google’s map-reduce [7] paradigm to demonstrate this parallel speed up technique on a variety of learning algorithms including locally weighted linear regression (LWLR), k-means, logistic regres- sion (LR), naive Bayes (NB), SVM, ICA, PCA, gaussian discriminant analysis (GDA), EM,