基于经验回放Q-Learning的最优控制算法.pdfVIP

M ay COM PU T ER ENGINEERING A ND DESIGN Vol No Learning Q QLearning ex perience replay ER QLearning QLearning ERQLearning QLearning Sarsa BLSPI ERQLearning Q T P A doi issn j Optimal control based on ex perience replay and Q Learning H U A N G Xiao an y Control Engineering School Chengdu U niversity of Information T echnology Chengdu China Abstract Aiming at the problem of high computation cost in on line optimal control strategy for real time system an optimal control algorithm based on ex perience replay and Q Learning w as proposed T he ex perience replaying technique w as adopted to reuse the samples to solve the problem that real time sy stem can not get enough samples T hrough QLearning algorithm and gradient descent method the parameter vector of value function w as updated T he algorithm based on ER and QLearning w as named ERQLearning and its computation cost w as analyzed Results of simulation show compared w ith Q Learning Sarsa and BLSPI ERQLearning can balance more time steps than the three methods w ith higher convergence rate Key words control strategy experience replaying QLearning realtime system samples 0 QLearning T D Q Sarsa ERQLearning Sarsa QLear