control task for reinforcement learning with known optimal solution for discrete and continuous actions控制任务,强化学习与已知最优解为离散和连续的行为.pdfVIP

J. Intelligent Learning Systems Applications, 2009, 1, 28-41 Published Online November 2009 in SciRes (www.SciRP.org/journal/jilsa) Control Task for Reinforcement Learning with Known Optimal Solution for Discrete and Continuous Actions 1 2 Michael C. RÖTTGER , Andreas W. LIEHR Service-Group Scientific Information Processing, Freiburg Materials Research Center, Freiburg, Germany. Email: 1roettm@gmx.de, 2Liehr@fmf.uni-freiburg.de Received November 10th ; 2008; revised January 29th, 2009; accepted January 30th, 2009. ABSTRACT The overall research in Reinforcement Learning (RL) concentrates on discrete sets of actions, but for certain real-world problems it is important to have methods which are able to find good strategies using actions drawn from continuous sets. This paper describes a simple control task called direction finder and its known optimal solution for both discrete and continuous actions. It allows for comparison of RL solution methods based on their value functions. In order to solve the control task for continuous actions, a simple idea for generalising them by means of feature vectors is presented. The resulting algorithm is applied using different choices of feature calculations. For comparing their performance a simple measure is introduced. Keywords : comparison, continuous actions, example problem, reinforcement learning, performance 1. Introduction In Reinforcement Learning (RL), one solves optimal is more likely to be usable in practice than a very good control problems without knowledge of the underlying but slow or complicated one. All these criteria allow for system’s dynamics on the basis of the following perspec- some kind of black-box testing and can give a good es- tive: An agent which is aware of the current state of its