On Learning By Exchanging advice.pdfVIP

AISB Journal 1(3), c 2002 .uk On Learning by Exchanging Advice Lu??s Nunes  and Euge?nio Oliveira y ISCTE/LIACC-NIADR, FEUP Av. Dr. Roberto Frias 4200-465, Porto, Portugal, Luis.Nunes@iscte.pty FEUP/LIACC-NIADR, FEUP Av. Dr. Roberto Frias 4200-465, Porto, Portugal, eco@fe.up.pt Abstract One of the main questions concerning learning in Multi-Agent Systems is: ”(How) can agents benefit from mutual interaction during the learning process?”. This paper describes the study of an interactive advice-exchange mechanism as a possible way to improve agents’ learning performance. The advice-exchange technique, discussed here, uses supervised learning (backpropagation), where reinforcement is not directly coming from the environment but is based on advice given by peers with better per- formance score (higher confidence), to enhance the performance of a heterogeneous group of Learning Agents (LAs). The LAs are facing similar problems, in an environ- ment where only reinforcement information is available. Each LA applies a different, well known, learning technique: Random Walk (hill-climbing), Simulated Annealing, Evolutionary Algorithms and Q-Learning. The problem used for evaluation is a sim- plified traffic-control simulation. In the following text the reader can find a description of the traffic simulation and Learning Agents (focused on the advice-exchange mecha- nism), a discussion of the first results obtained and suggested techniques to overcome the problems that have been observed. Initial results indicate that advice-exchange can improve learning speed, although ”bad advice” and/or blind reliance can disturb the learning performance. The use of supervised learning to incorporate advice given from non-expert peers using different learning algorithms, in problems where no su- pervision information is available, is, to the best of the authors’ knowledge, a new concept in the area of Multi-Agent Systems Learning. 1 Introduction 1.1 Framework The objective of this work i