多步 R 学习算法(英文).pdfVIP

DOI : 10 . 15918/ j .j bi t1004 -0579 . 1999 . 03 . 004 Jour n al of Beijin g I n t itute of Technology , 1999 , V ol . 8, No . 3 In cr emental Mu lt i- St ep R- Learn ing Hu Guanghua 　Wu Cangpu ( Dep artment of Automat ic Control , Beijing I n titut e of T echnology , Beijin g 10008 1) Ab tr act 　Aim 　To inv e tigat e th e model- free mu lt i- t ep average reward reinforcement learning algorithm . Method 　By comb ining th e R- lear nin g algorithm with the t emp oral- difference lear nin g ( TD( λ) - learning) algorithm for av erage reward prob lem , a novel in cremental algo- rithm , called R( λ)- learn in g , wa propo ed . Re u lt and Conclu ion 　Th e prop o ed algorithm i a natural ext en ion of the Q ( λ)- learn in g , the multi- t ep di counted reward reinforcement lear nin g algorithm , t o th e average rew ard ca e . Simulat ion re u lt h ow that the R( λ) - learning with intermedia te λvalu e mak e ign ificant performance improvement over the imp le R- lear nin g . Key word 　rein forcement learning ; av erage reward ; R- learning ; Markov deci ion proce e ; t emporal differen ce learning [ 1] Recent ly , t he average reward rein forcement learnin g , originally propo ed by Schwart z in 1993 , h a been paid more and more at tention to . Mahadevan[ 2] gave a nice di cu ion about av- erage reward prob lem . He al o provided ome exten ive numerical experiment on R- learning