6 MLP神经网络.pptVIP

Multi-layer Perceptrons Junying Zhang contents structure universal theorem MLP for classification mechanism of MLP for classification nonlinear mapping binary coding of the areas MLP for regression learning algorithm of the MLP back propagation learning algorithm heuristics in learning process XOR and Linear Separability Revisited Remember that it is not possible to find weights that enable Single Layer Perceptrons to deal with non-linearly separable problems like XOR: However, Multi-Layer Perceptrons (MLPs) are able to cope with non-linearly separable problems. Historically, the problem was that there were no learning algorithms for training MLPs. Actually, it is now quite straightforward. Structure of an MLP it is composed of several layers neurons within each layer are not connected ith layer is only fully connected to the (i+1)jth layer Signal is transmitted only in a feedforward manner Structure of an MLP Model of each neuron in the net includes A nonlinear activation function ? the net is nonlinear The function is smooth ? derivative Generally, sigmoidal function The network contains one or more layers of hidden neurons that are not part of input or output of the net ? enable the net to learn complex tasks Expressive power of an MLP Questions How many hidden layers are needed? How many units should be in a (the) hidden layer? Answers Komogorov’s mapping neural network existence theorem (universal theorem) Komogorov’s mapping neural network existence theorem (universal theorem) Any continuous function g(x) defined on the unit hypercube can be represented in the from For properly chosen functions and It is impractical the functions and are not the simple weighted sums passed through nonlinearities favored in neural networks It tells us very little about how to find the nonlinear functions based on data — the central problem in network based pattern recognition those functions can be extremely complex; they ar