李宏毅机器学习-课件_7-DL (v2).pdfVIP

Deep Learning Deep learning attracts lots of attention. • I believe you have seen lots of exciting results before. Deep learning trends at Google. Source: SIGMOD/Jeff Dean Ups and downs of Deep Learning • 1958: Perceptron (linear model) • 1969: Perceptron has limitation • 1980s: Multi-layer perceptron • Do not have significant difference from DNN today • 1986: Backpropagation • Usually more than 3 hidden layers is not helpful • 1989: 1 hidden layer is “good enough”, why deep? • 2006: RBM initialization (breakthrough) • 2009: GPU • 2011: Start to be popular in speech recognition • 2012: win ILSVRC image competition Three Steps for Deep Learning Step 1: Step 2: Step 3: pick define a set goodness of the best Neural of function function function Network Deep Learning is so simple …… Neural Network     z      z   z      z “Neuron” Neural Network Different connection leads to different network structures Network parameter : all the weights and biases in the “neurons” Fully Connect Feedforward Network 1 4 0.98 1 -2 1 -1 -2 0.12 -1 1 0 Sigmoid Function    z 1    z