Control Task for Reinforcement Learning with Known Optimal Solution for Discrete and Continuous Actions英文文献资料.docVIP

J. Intelligent Learning Systems Applications, 2009, 1, 28-41 Published Online November 2009 in SciRes (www.SciRP.org/journal/jilsa) Control Task for Reinforcement Learning with Known Optimal Solution for Discrete and Continuous Actions Michael C. R?TTGER , Andreas W. LIEHR 1 2 Service-Group Scientific Information Processing, Freiburg Materials Research Center, Freiburg, Germany. 1 2 Email: roettm@gmx.de, Liehr@fmf.uni-freiburg.de Received November 10 ; 2008; revised January 29 , 2009; accepted January 30 , 2009. th th th ABSTRACT The overall research in Reinforcement Learning (RL) concentrates on discrete sets of actions, but for certain real-world problems it is important to have methods which are able to find good strategies using actions drawn from continuous sets. This paper describes a simple control task called direction finder and its known optimal solution for both discrete and continuous actions. It allows for comparison of RL solution methods based on their value functions. In order to solve the control task for continuous actions, a simple idea for generalising them by means of feature vectors is presented. The resulting algorithm is applied using different choices of feature calculations. For comparing their performance a simple measure is introduced. Keywords: comparison, continuous actions, example problem, reinforcement learning, performance 1. Introduction In Reinforcement Learning (RL), one solves optimal control problems without knowledge of the underlying system’s dynamics on the basis of the following perspec- tive: An agent which is aware of the current state of its environment, decides in favour of a particular action. The performance of the action