《07.Deep Learning in Action_肖达》.ppt

深度学习科普和实战 肖达 Agenda A two minutes tour of DL fundamentals DL in Action with GPU/Theano/Pylearn2 Deep Learning in a Nutshell深度学习管窥 对黑盒子的限定 深度：多层非线性信息处理 Input = L1 = L2 = … = Ln = Output 学习：内部结构通过学习涌现 机器学习与特征表示 反向传播算法（Back-Prop） 计算每个样本的损失函数（实际输出与预期输出的差别）对各参数的梯度 应用链式求导法则 学习过程 1、前向传播激励响应 2、和目标比较得到损失 3、反向传播修正权重 Agenda A two minutes tour of DL fundamentals DL in Action with GPU/Theano/Pylearn2 What you need An off-the-shelf PC with 650w+ power supply A GPU (GTX 580/780/Titan) Get familiar with Linux, Python and Numpy Total cost 8k￥ “DL不再是高富帅的专利，GPU+Theano乃我等屌丝之福音。” — by Xingyuan Open souce libraries Theano by Bengios group @U. Montreal transparent use of GPU automatic gradient computation pylearn2 high-level DL library based on Theano contain most building blocks needed for DL experiments contain a cuda-convnet wrapper cuda-convnet by Hintons group @U. Toronto VERY fast C++/CUDA implementation of convolutional neural networks not a user-friendly library How to launch a DL experiment in 5 minutes Specify 3 things in my_exp.yaml dataset model training algorithm exec train.py my_exp.yaml An example yaml file dataset: train !obj:pylearn2.datasets.cifar10.CIFAR10 { toronto_prepro: True, which_set: train, one_hot: 1, axes: [c, 0, 1, b], start: 0, stop: 40000 }, model: !obj:pylearn2.models.mlp.MLP { batch_size: 128, input_space: !obj:pylearn2.space.Conv2DSpace { shape: [32, 32], num_channels: 3, axes: [c, 0, 1, b], }, layers: [ !obj:pylearn2.models.maxout.MaxoutConvC01B { layer_name: conv1, pad: 2, num_channels: 32, num_pieces: 1, kernel_shape: [5, 5], pool_shape: [3, 3], pool_stride: [2, 2], irange: .01, min_zero: True, tied_b: True, max_kernel_norm: 9.9, }, !obj:pylearn2.models.maxout.MaxoutConvC01B { layer_name: conv2, algorithm: !obj:pylearn2.training_algorithms.sgd.SGD { batch_size: 128, learning_rate: .01, init_momentum: