Coupled Replicator Equations for the Dynamics of Learning in Multiagent Systems.pdfVIP

Coupled Replicator Equations for the Dynamics of Learning in Multiagent Systems Yuzuru Sato1, 2, ∗ and James P. Crutchfield2, † 1Brain Science Institute, Institute of Physical and Chemical Research (RIKEN), 2-1 Hirosawa, Saitama 351-0198, Japan 2Santa Fe Institute, 1399 Hyde Park Road, Santa Fe, NM 87501 (Dated: February 8, 2008) Starting with a group of reinforcement-learning agents we derive coupled replicator equations that describe the dynamics of collective learning in multiagent systems. We show that, although agents model their environment in a self-interested way without sharing knowledge, a game dynamics 2 emerges naturally through environment-mediated interactions. An application to rock-scissors-paper 0 game interactions shows that the collective learning dynamics exhibits a diversity of competitive and 0 cooperative behaviors. These include quasiperiodicity, stable limit cycles, intermittency, and deter- 2 ministic chaos—behaviors that should be expected in heterogeneous multiagent systems described g by the general replicator equations we derive. u A PACS numbers: 05.45.-a, 02.50.Le, 87.23.-n Santa Fe Institute Working Paper 02-04-017 8 1 Adaptive behavior in multiagent systems is an impor- assume that there are two such agents X and Y that at tant interdisciplinary topic that appears in various guises each time step