Fixed vs Dynamic Subtransfer in Reinforcement Learning.pdfVIP

Fixed vs Dynamic Sub-transfer in Reinforcement Learning Technical report James L. Carroll Machine Intelligence, Learning, and Decisions Laboratory Brigham Young University Provo Ut. 84601 USA james@jlcarroll.net December 27, 2001 Abstract Task transfer is the process whereby one task, known as the source task, is used to bias the learn- We survey various transfer methods in Q-learning, a ing of another task, known as the target task. Task type of reinforcement learning, and present a vari- transfer is important in the “life long learning” [7][3] ation on fixed sub-transfer which we call dynamic approach to AI. In the life long learning approach sub-transfer. We describe the pros and cons of dy- one agent encounters a variety of tasks over the life- namic sub-transfer as compared with the other trans- time of the learning agent. The agent applies in- fer methods, and we describe qualitatively the situa- formation from each task to speed the learning of tions where this method would be preferred over the each subsequent task. This approach can poten- fixed version of sub-transfer. tially reduce the training time and adaptability of a reinforcement-learning agent. Furthermore through a process known as shaping [cite] such a transfer tech- 1 Introduction nique can increase th