基于后悔值的多Agent冲突博弈强化学习模型-软件学报.PDFVIP

ISSN 1000-9825, CODEN RUXUEW E-mail: jos@ Journal of Software , Vol.19, No.11, November 2008, pp.2957−2967 DOI: 10.3724/SP.J.1001.2008.02957 Tel/Fax: +86-10 © 2008 by Journal of Software . All rights reserved. ∗ 基于后悔值的多Agent 冲突博弈强化学习模型 + 肖 正 , 张世永 (复旦大学 计算机与信息技术系,上海 200433) Reinforcement Learning Model Based on Regret for Multi-Agent Conflict Games + XIAO Zheng , ZHANG Shi-Yong (Department of Computer and Information Technology, Fudan University, Shanghai 200433, China) + Corresponding author: E-mail: xiaozheng206@163.com, Xiao Z, Zhang SY. Reinforcement learning model based on regret for multi-agent conflict games. Journal of Software, 2008,19(11):2957−2967. /1000-9825/19/2957.htm Abstract : For conflict game, a rational but conservative action selection method is investigated, namely, minimizing regret function in the worst case. By this method the loss incurred possibly in future is the lowest under this very policy, and Nash equilibrium mixed policy is obtained without information about other agents. Based on regret, a reinforcement learning model and its algorithm for conflict game under multi-agent complex environment are put forward. This model also builds agents’ belief updating process on the concept of cross entropy distance, which further optimizes action selection policy for conflict games. Based on Markov repeated game model, this paper demonstrates the convergence property of this algorithm, and analyzes the relationship between belief and optimal policy. Additionally, compared with extended Q-learning algorithm under MMDP (multi-agent markov deci