《Training Recurrent Neurocontrollers for Real-Time Applications》.pdf

IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 18, NO. 4, JULY 2007 1003 Training Recurrent Neurocontrollers for Real-Time Applications Danil V. Prokhorov, Senior Member, IEEE Abstract— In this paper, we introduce a new approach to train re- of the closed-loop system and effective training algorithms current neurocontrollers for real-time applications. We begin with such as nprKF [Kalman filter by Norgaard, Poulsen, and Ravn training a recurrent neurocontroller for robustness on high-fidelity (NPR)] [3], [4] to train RNN controllers offline. In the second models of physical systems. For training, we use a recently devel- oped derivative-free Kalman filter method which we enhance for stage, we deploy our RNN controller with fixed weights in a controller training. After training, we fix weights of our recurrent real-time environment in which only internal state variables neurocontroller and deploy it in an embedded environment. Then, of the networks are subject to efficient adaptation. Thus, we we carry out additional training of the neurocontroller by adapting utilize both quickness of adaptation of the short-term memory in real time its internal state (short-term memory), rather than its (controller state variables) and permanence of control strategies weights (long-term memory). Such real-time training is done with a new combination of simultaneous perturbation stochastic approx- captured in the long-term memory (controller weights) and imation (SPSA) and adaptive critic. Our critic is also a recurrent offer a practical way to combine these benefits in the same neural network (RNN), and it is trained by