《Deep Learning简单介绍》.pdf

Deep Learning简单介绍 吴伟 新浪微博：tornadomeet 博客地址：/tornadomeet 2014.06.19 #1 Outline 1. Deep Learning的定义及历史: DL定义，DL历史，BP算法, pre-training, fine-tuning 2. 无监督学习模块: 1. Auto-encoder(sparse auto-encoder, denoising auto-encoder, contractive auto-encoder) 2. RBM(loss函数表达式，GRBM, CD算法,PCD算法等) 3. Sparse Coding( project gradient descent, block coordinate descent, ISTA, PSD) 3. Building a Deep Network: 1. DNN, Deep Auto-encoder 2. DBN 3. DBM 4. DSN 4. Convolutional Neural Network(CNN)： 1. BP in CNN 2. 预训练：PCANet, Convolutional Auto-encoder, Convolutional RBM, Convolutional Sparse Coding (Deconvolution NN),Convolutional PSD 3. CNN改进思路 #2 Outline 5. Recurrent Neural Network(RNN)： 基本结构图和BPTT算法。 6. Some Tricks in DL: 1. trick来源 2. dropout, Noise in Activations, maxout, dropout, dropconnect, ReLU, tiled CNN 7. 特征的可视化： 最大化激活值法、采样法、上层权值线性组合法、Deconvolution法 8. DL的应用： Acoustic Model, Object Recognition(有监督，无监督), NLP 9. DL的未来： 1. Why need deep? 2. What’s wrong with back-propagation? 3. A Theoreticians Nightmare? 4. What’s need to be solved in DL? 5. What’s the hot topic next in DL? 10. DL的一些资料 一、Deep Learning定义及历史 • DL的定义 • DL的历史 • BP算法 • Pre-training • Fine-tuning Deep Learning的定义 • Deep learning is a set of algorithms in machine learning that attempt to model high-level abstractions in data by using architectures composed of multiple non-linear transformations. ———wikipedia 的最新定义 1. DL是机器学习算法的一种 2. DL是用来提取数据的高层、抽象的特征 3. DL是由多重非线性变换复合而成(意味着DL并不 一定