基于TD-error自适应校正深度Q学习主动采样方法.PDFVIP

计 算 机 研 究 与 发 展 DOI:10. 7544/issn1000-1239. 2018. ******** Journal of Computer Research and Development 卷(期):起止页,年 修改稿：2017-70812 所属类别：人工智能 基于TD-error 自适应校正的深度Q 学习主动采样方法 手机 白辰甲 刘鹏 赵巍 唐降龙 （哈尔滨工业大学计算机科学与技术学院模式识别与智能系统研究中心 哈尔滨 150001） （bai_chenjia@ ） Active Sampling for Deep Q-learning Based on TD-error Adaptive Correction Bai Chenjia, Liu Peng, Zhao Wei, and Tang Xianglong （Pattern Recognition and Intelligent System Research Center, School of Computer Science and Technology, Harbin Institute of Technology, Harbin 150001） Abstract Deep reinforcement learning(DRL) is one of research hotspots in artificial intelligence. Deep Q-learning is one of the representative achievements of DRL. In some fields, its performance has met or exceeded the level of human expert. It is necessary for training deep Q-learning to acquire lots of samples. These samples are obtained by the interaction between agent and environment. However, it is usually computationally intensive and sometimes impossible to keep away from interaction risk. We propose an active sampling method based on TD-error adaptive correction in order to solve sample efficiency problem in deep Q-learning. In various deep Q-learning methods, the updating of storage priority in experience memory lags behind the updating of Q-network parameters. It causes that a lot of samples are not selected to apply in Q-network training because the storage priority cannot reflect the true distribution of TD-error in experience memory. The TD-error adaptive correction active sampling method proposed in this paper uses the replay periods of