neural networks for machine learning面向机器学习的神经网络.pptxVIP

Neural Networks for Machine LearningLecture 12aThe Boltzmann Machine learning algorithmThe goal of learningIt is also equivalent to maximizing the probability that we would obtain exactly the N training cases if we did the followingLet the network settle to its stationary distribution N different times with no external input.Sample the visible vector once each time.We want to maximize the product of the probabilities that the Boltzmann machine assigns to the binary vectors in the training set.This is equivalent to maximizing the sum of the log probabilities that the Boltzmann machine assigns to the training vectors.Why the learning could be difficult Consider a chain of units with visible units at the ends If the training set consists of (1,0) and (0,1) we want the product of all the weights to be negative. So to know how to change w1 or w5 we must know w3. w2 w3 w4hiddenvisiblew1w5A very surprising factEverything that one weight needs to know about the other weights and the data is contained in the difference of two correlations.Expected value of product of states at thermal equilibrium when v is clamped on the visible unitsExpected value of product of states at thermal equilibrium with no clampingDerivative of log probability of one training vector, v under the model.Why is the derivative so simple?The energy is a linear function of the weights and states, so:The process of settling to thermal equilibrium propagates information about the weights.We don’t need backprop.The probability of a global configuration at thermal equilibrium is an exponential function of its energy.So settling to equilibrium makes the log probability a linear function of the energy.Why do we need the negative phase? The positive phase finds hidden configurations that work well with v and lowers their energies. The negative phase finds the joint configurations that are the best competitors and raises their energies. An inefficient way to collect the stat