Neural Networks for Machine Learning Lecture （神经网络对机器学习讲座）.pdfVIP

Neural Networks for Machine Learning Lecture 2a An overview of the main types of neural network architecture Geoffrey Hinton with Nitish Srivastava Kevin Swersky Feed-forward neural networks •  These are the commonest type of neural network in practical applications. –  The first layer is the input and the last output units layer is the output. –  If there is more than one hidden layer, we call them “deep” neural networks. hidden units •  They compute a series of transformations that change the similarities between cases. input units –  The activities of the neurons in each layer are a non-linear function of the activities in the layer below. Recurrent networks •  These have directed cycles in their connection graph. –  That means you can sometimes get back to where you started by following the arrows. •  They can have complicated dynamics and this Recurrent nets with can make them very difficult to train. multiple hidden layers –  There is a lot of interest at present in finding are just a special case efficient ways of training recurrent nets. that has some of the hiddenàhidden •  They are more biologically realistic. connections missing. Recurrent neural networks for modeling sequences time à •  Recurrent neural networks are a very natural way to model sequential data: o o o u u u