Chapter 15 Dynamically Driven Recurrent Networks 《神经网络与 及机器学习》 .pptVIP

Copyright ?2009 by Pearson Education, Inc. Upper Saddle River, New Jersey 07458 All rights reserved. Neural Networks and Learning Machines, Third Edition Simon Haykin Chapter 15 Dynamically Driven Recurrent Networks Figure 15.1 Nonlinear autoregressive with exogenous inputs (NARX) model; the feedback part of the network is shown in red. Figure 15.2 State-space model; the feedback part of the model is shown in blue. Figure 15.3 Simple recurrent network (SRN); the feedback part of the network is shown in blue. Figure 15.4 Recurrent multilayer perceptron; feedback paths in the network are printed in blue. Figure 15.7 Turing machine. Figure 15.8 Illustration of Theorems I and II, and corollary to them. Figure 15.9 (a) Architectural graph of a two-neuron recurrent network N. (b) Signal-flow graph of the network N. unfolded in time. Figure 15.10 Fully connected recurrent network for formulation of the RTRL algorithm; the feedback connections are all shown in blue. Table 15.1 Figure 15.11 Sensitivity graph of the fully recurrent network of Fig. 15.6. Note: The three nodes, labeled ξl,n are all to be viewed as a single input. Figure 15.12 Illustration of the vanishing-gradient problem: (a) State xn resides in the basin of attraction, β, but outside the reduced attraction set ?. (b) State xn resides inside the reduced attraction set ?. Figure 15.13 Nonlinear state-space model depicting the underlying dynamics of a recurrent network undergoing supervised training. Table 15.2 Figure 15.14 Closed-loop feedback system embodying the RMLP and the EKF: (a) The RMLP, with weight vector ?wn|n - 1, operates on the input vector un to produce the output vector yn. (b) The EKF, supplied with the prediction ?dn|n - 1 = yn, operates on the desired response dn to produce the filtered weight vector ?wn|n = ?wn+1|n, thereby preparing the closed-loop feedback system for the next iteration. Figure 15.15 Block-diagonal representation of the filtering-error covariance