图多智能体任务建模视角下的协作子任务行为发现.pdfVIP

计算机研究与发展DOI：10.7544/issn1000-1239.202440189

Journal

of

Computer

Research

and

Development61（8）：1904−1916，2024

图多智能体任务建模视角下的协作子任务行为发现

李超李文斌高阳

（计算机软件新技术全国重点实验室（南京大学）南京210023）

（chaoli1996@）

DiscoveringCoordinatedSubtaskPatternsfromaGraphicalMulti-AgentTask

ModelingPerspective

Li

Chao,

Li

Wenbin,

and

Gao

Yang

（StateKeyLaboratoryforNovelSoftwareTechnology(NanjingUniversity),Nanjing210023）

AbstractNumerous

multi-agent

tasks

exhibit

a

nearly

decomposable

structure,

wherein

interactions

among

agents

within

the

same

interaction

set

are

strong

while

interactions

between

different

sets

are

weak.

Efficiently

modeling

this

structure

and

leveraging

it

to

coordinate

agents

can

enhance

the

learning

efficiency

of

multi-agent

reinforcement

learning

algorithms

for

cooperative

multi-agent

tasks,

while

existing

work

typically

neglects

and

fails.

To

address

this

limitation,

we

model

the

nearly

decomposable

structure

using

a

dynamic

graph

and

accordingly

propose

a

novel

algorithm

named

coordinated

subtask

pattern

(CSP)

that

enhances

both

local

and

global

coordination

among

agents.

Specifically,

CSP

identifies

agents’

interaction

sets

as

subtasks

and

utilizes

a

bi-level

structure

to

periodically

distribute

agents

into

multiple

subtasks,

which

ensures

accurate

characterizations

regarding

their

interactions

on

the

dynamic

graph.

Based

on

the

subtask

assignment,

CSP

proposes

intra-subtask

and

inter-subtask

pattern

constraints

to

facilitate

both

local

and

global

coordination

among

agents.

These

two

constraints

ensure

that

partial

agents

within

the

same

subtask

are

aware

of

their

action

selections

and

all

agents

select

superior

joint

actions

that

maximize

the

overall

task

performance.

Experimentally,

we

e