深度学习并行优化算法for.pptxVIP

Parallelism in Deep LearningYueqing WangSupervisor: Prof. DouContentsBackgroundData ParallelismModel ParallelismParallel Optimization MethodsParallel FrameworkBackgroundWhy need parallelism in DL?Big data Imagenet: 1000 categories, 1.2 million images for training and 150,000 images for testing and validating.Too many parameters to trainGoogLeNet –Top1 of Large Scale Visual Recognition Challenge 2014 Iterative algorithmsBackgroundBasic steps of some optimization methodsConjugate Gradient (CG), L-BFGS, stochastic gradient descent (sgd) W means the parameters we want to train“Cal ΔW” is the most