基于开发标准OpenCL的深度学习研究与探索.pptxVIP

基于开放标准OpenCL的深度学习研究和探索谷俊丽AMD Research Collaborated with product teammailto:Junli.Gu@AMD.comJunlimailto:Junli.Gu@AMD.com.mailto:Junli.Gu@AMD.comGu@AMmailto:Junli.Gu@AMD.comDmailto:Junli.Gu@AMD.com.mailto:Junli.Gu@AMD.comcmailto:Junli.Gu@AMD.comomOutline深度学习及其发展状况深度学习对系统实现的挑战基于OpenCL的深度学习探索DNN 模型hidden1? What is a Deep Neural Network (DNN)?? 3~24 hidden layers, millions to billions of parameters? DNN + Big Data is leading recent direction in machine learning? Rich Varieties of DNN Structures? MLP (Multi-level Perceptron)/ AutoEncoder? CNN (Convolutional Neural Network)? DBN (Deep belief network)/RBM (Restricted Boltzmann Machine)? Deep Learning on DNN model? Random initialized parameters? Trained to converge by feeding large scale of data (Big Data)hidden2Inputhidden3OutputneuronsweightedconnectionStarting to get really hot after winning 2012 ILSVRC competition3 | IMPLEMENTING A LEADING LOADS PERFORMANCE PREDICTOR ON COMMODITY PROCESSORS | JUNE 19, 2014深度学习过程 (DEEP LEARNING)Deep Learning: DNN model + Big Data??Actually human defined features no longer work well for Big Data scenarios with noise.All features leaDrnt ebyetrpaininLgedaatar, wnitihnougt hu!man interference.DNN for Speech ! 10k hours of voice data! 10b training samples!Months on a GPU cluster!Voice!,Text!Image!ResultsDNN model4 | IMPLEMENTING A LEADING LOADS PERFORMANCE PREDICTOR ON COMMODITY PROCESSORS | JUNE 19, 2014深度学习为何强大？HIERARCHICAL FEATURE EXTRACTION? Extract features layer by layer from input data, to form hierarchical representation that is beyond human’sdefinition? Features have semantic meanings5 | AMD DNN深度学习正在引领潮流? Why internet companies purse DNN these days?? Original human defined algorithms don’t work well for Big Data? Competing in machine learning to understand Big Data? DNN (deep neural networks) is breaking through leading direction? Large scale of image classification/recognition/search, face recognition? Online recommendation for electronic business? Voice recognition, m