Reinforcement Control via Heuristic Dynamic Programming.pdfVIP

Reinforcement Control via Heuristic Dynamic Programming K Wendy Tang and Govardhan Srikant wtangeesunysbedu and gsrikanteesunysbedu 1 Department of Electrical Engineering SUNY at Stony Bro ok Stony Bro ok NY ABSTRA CT Heuristic Dynamic Programming HDP is the simplest kind of Adaptive Critic which is a p owerful form of reinforcement control It can be used to maximize or minimize any utility function such as total energy or trajectory error of a system over time in a noisy environment Unlike supervised learning adaptive critic design does not require the desired control signals be known Instead feedback is obtained based on a critic network which learns the relationship between a set of control signals and the corresponding strategic utility function It is an approximation of dynamic programming A simple Heuristic Dynamic Programing HDP system involves two subnetworks the Action net work and the Critic network Each of these networks includes a feedforward and a feedback component A ow chart for the interaction of these components is included To further il lustrate the algorithm we use HDP for the control of a simple D planar r obot Intro duction Neuro control is dened as the use of neural networks to emit control signals for dynamic systems There are ve basic schemes the supervised control direct inverse control neural adaptive control Backpropagation of Utility and Adaptive Critic Designs Werb os provided a detailed summary of the ve schemes including the pros and cons of each metho d Of these techniques b oth Backpropagation of Utility and Adaptive Critic are capable of m