Cascadecorrlation级联相关.pptVIP

cascade-correlation Cascade-correlation By Kranthi sudhan Contents Motivation Recollecting back-prop Cascading architecture Learning Algorithm Example Comparision with other systems Motivation Curse of dimensionality Simple network Determines the structure Fast learner Recollect What`s Back-propagation? Problems with this alogorithm. How CC is going to solve these problems? Cascade-Correlation CC combines 2 key ideas. Cascade architecture. Learning alogorithm. Cascade Architecture Begins with some inputs and one or more outputs. Every input is connected to every output. Bias is permanently set to +1. Stage -1 Stage-2 Stage-3 Algorithm Train stage 1. If error is not acceptable, proceed. Train stage 2. If error is not acceptable proceed. Etc Algorithm 1. Train all the connections ending at an output unit with a usual learning algorithm until the error of the net no longer decreases. 2 CC starts with a minimal network consisting only of an input and an output layer. Both layers are fully connected Algorithm 3. Generate the so-called candidate units. Every candidate unit is connected with all input units and with all existing hidden units. Between the pool of candidate units and the output units there are no weights. Algorithm 4. Try to maximize the correlation between the activation of the candidate units and the residual error of the net by training all the links leading to a candidate unit. Learning takes place with an ordinary learning algorithm. The training is stopped when the correlation scores no longer improves. Algorithm In order to maximize this S,we compute partial derivative of S with respect to each candidate units incoming weight 5.Choose the candidate unit with the maximum correlation, freeze its incoming weights and add it to the net. Algorithm 6. To change the candidate unit into a hidden unit, generate links between the selected unit and all the output units. Since the weights leading to the