How to Explore Your Opponent’s Strategy (almost) Optimally.pdf

How to Explore Your Opponent’s Strategy (almost) Optimally David Carmel and Shaul Markovitch Computer Science Department, Technion, Haifa 32000, Israel {carmel,shaulm}@cs.technion.ac.il Abstract by adaptive agents in MAS for learning interaction strate- gies with other agents [13]. RL is based on the idea that This work presents a lookahead-based exploration strat- the tendency to produce an action should be reinforced if egy for a model-based learning agent that enables explo- it produces favorable results, and weakened if it produces ration of the opponent’s behavior during interaction in a unfavorable results. RL spends little computation resources multi-agent system. Instead of holding one model, the per example but requires large number of examples. A model-based agent maintains a mixed opponent model, a model-based approach for learning interaction strategies [5] distribution over a set of models that reflects its uncertainty reduces the number of examples needed for adaptation, by about the opponent’s strategy. Every action is evaluated investing more computational resources in deeper analysis according to its long run contribution to the expected utility of interaction experience. The learning agent makes use of and to the knowledge regarding the opponent’s strategy. We an explicit model of the other agent’s strategy to generate present an efficient algorithm that returns an almost opti- expectations about its behavior. At any stage of the game, mal expl