实时动态规划的最优行动判据及算法改进-软件学报.pdfVIP

ISSN 1000-9825, CODEN RUXUEW E-mail: jos@ Journal of Software , Vol.19, No.11, November 2008, pp.2869−2878 DOI: 10.3724/SP.J.1001.2008.02869 Tel/Fax: +86-10 © 2008 by Journal of Software . All rights reserved. ∗ 实时动态规划的最优行动判据及算法改进 + 范长杰, 陈小平 ( 中国科学技术大学 计算机科学与技术系,安徽 合肥 230027) Optimal Action Criterion and Algorithm Improvement of Real-Time Dynamic Programming FAN Chang-Jie, CHEN Xiao-Ping+ (Department of Computer Science and Technology, University of Science and Technology of China, Hefei 230027, China) + Corresponding author: E-mail: xpchen@ Fan CJ, Chen XP. Optimal action criterion and algorithm improvement of real-time dynamic programming. Journal of Software, 2008,19(11):2869−2878. /1000-9825/19/2869.htm Abstract : This paper is primarily to improve the efficiency of real-time dynamic programming (RTDP) algorithm for solving Markov decision problems. Several typical convergence criteria are compared and analyzed. A criterion called optimal action criterion and a corresponding branch strategy are proposed on the basis of the upper and lower bound theory. This criterion guarantees that the agent can act earlier in a real-time decision process while an optimal policy with sufficient precision still remains. It can be proved that under certain conditions one can obtain an optimal policy with arbitrary precision by using such an incremental method. With these new techniques, a bounded incremental real-time dynamic programming (BI-RTDP) algorithm is designed. In the experiments of two typical real-time simulation systems, BI-RTDP outperforms the other state-of-the-art RTDP algorithms tested. K