基于概率模型的动态分层强化学习-控制理论与应用.pdfVIP

28 11 Vol. 28 No. 11 2011 11 Control Theory Applications Nov. 2011 :2011 , , , (, 410083) : , , . , , . , . : ; ; ; : TP273 : A Dynamic hierarchical reinforcement learning based on probability model DAI Zhao-hui, YUAN Jiao-hong, WU Min, CHEN Xin (School of Information Science and Engineering, Central South University, Changsha Hunan 410083, China) Abstract: To deal with the overwhelming dimensionality in the large-scale reinforcement-learning and the strong depen- dence on prior knowledge in existing learning algorithms, we propose the method of dynamic hierarchical reinforcement -- learning based on the probability model (DHRL model). This method identifies some key states automatically based on probability parameters of the state-transition probability model established based on Bayesian learning, then generates some state-subspaces dynamically by clustering, and learns the optimal policy based on hierarchical structure. Simulation results -- show that DHRL model algorithm improves the learning efficiency of the agent remarkably in the complex environment, and can be applied to learning in the unknown large-scale world. Key words: dynamic hierarchical reinforcement-learning; Bayesian learning; state-transition probability model; agent 1 (Introduction) . , , [1] (dynamic , . [2] hierarchical reinforcement learning, DHRL). . (Q--), , , . DHRL,