Chapter 15 Dynamically Driven Recurrent Networks 《神经网络与机器学习》 教学课件.pptVIP

Chapter 15 Dynamically Driven Recurrent Networks Figure 15.1 Nonlinear autoregressive with exogenous inputs (NARX) model; the feedback part of the network is shown in red. Figure 15.2 State-space model; the feedback part of the model is shown in blue. Figure 15.3 Simple recurrent network (SRN); the feedback part of the network is shown in blue. Figure 15.4 Recurrent multilayer perceptron; feedback paths in the network are printed in blue. Figure 15.5 Second-order recurrent network; bias connections to the neurons are omitted to simplify the presentation. The network has 2 inputs and 3 state neurons, hence the need for 3 X 2 = 6 multipliers. The feedback links in the figure are printed in blue to emphasize their global role. Figure 15.6 Fully connected recurrent network with two inputs, two hidden neurons, and one output neuron. The feedback connections are shown in blue to emphasize their global role. Figure 15.7 Turing machine. Figure 15.8 Illustration of Theorems I and II, and corollary to them. Figure 15.9 (a) Architectural graph of a two-neuron recurrent network N. (b) Signal-flow graph of the network N. unfolded in time. Figure 15.10 Fully connected recurrent network for formulation of the RTRL algorithm; the feedback connections are all shown in blue. Table 15.1 Figure 15.11 Sensitivity graph of the fully recurrent network of Fig. 15.6. Note: The three nodes, labeled ξl,n are all to be viewed as a single input. Figure 15.12 Illustration of the vanishing-gradient problem: (a) State xn resides in the basin of attraction, β, but outside the reduced attraction set ?. (b) State xn resides inside the reduced attraction set ?. Figure 15.13 Nonlinear state-space model depicting the underlying dynamics of a recurrent network undergoing supervised training. Table 15.2 Figure 15.14 Closed-loop feedback system embodying the RMLP and the EKF: (a) The RMLP, with weight vector ?wn|n - 1, operates on the input vector un to produce the output vect