基于MADDPG算法的多智能体协同控制研究.pdfVIP

摘 要 传统的工业机器人是建立在精确的数学模型的基础上，其控制方法通常是在固定 的环境中设定特定任务。然而这样的传统控制系统不具备适应性以及泛化性，当机器 人处于的环境发生细微的改变时，机器人则无法准确的完成任务，因此智能控制算法 逐渐成为机器控制的研究热点。随着强化学习以及深度学习的不断发展，将深度强化 学习算法应用到机器人控制受到了广大的研究人员的关注。 本文首先介绍了传统机械控制的控制原理和深度强化学习的发展，阐述了本课题 的研究背景与研究目的。其次，在基于物理引擎的MuJoCo环境中采用了确定性策略 梯度算法 （Deep Deterministic Policy Gradient，DDPG），重点研究DDPG算法在单个 智能体环境的鲁棒性与通用性。然而，随着智能体数量的增加而引起的环境不稳定， 导致一般的深度强化学习在联合行动空间中存在一定的困难。最后为了解决这个问 题，本次设计一个四个机械手臂通过协作控制到达目标位置的任务，并使用了多智能 体确定性策略梯度算法 （Multi-Agent Deep Deterministic Policy Gradient，MADDPG） 在MuJoCo 中进行训练。实验结果表明，每个机械手臂均可以通过自主学习获取信息 和积累经验来完成任务，同时模型的收敛效果良好，说明MADDPG算法在复杂环境下 具有良好的性能，并成功的学习了多智能体协作策略。 关键字：传统机械控制；深度强化学习算法；协同策略 研究类型：应用研究 Abstract Traditional industrial robots are based on precise mathematical models, and their control methodsusually set specific tasks in a fixed environment.However, such a traditional control system does not have adaptability and generalization. When the environment in which the robot is located changes slightly, the robot cannot complete the task accurately. Therefore, intelligent control algorithms have gradually become a research focus of machine control. With thecontinuousdevelopment ofreinforcement learning and deep learning,the application of deep reinforcement learning algorithms to robot control has attracted the attention of researchers. This paper first introduces the control principle of traditional mechanical control and the development of deep reinforcement learning, and elaborates the research background and research purpose of this topic. Secondly, in the MuJoCo environment based on the physics engine, a Deterministic Policy Gradient (DDPG) algorithm isused to focus on the robustness and versatility of the DDPG algorithm in a single agent environment. However, with the instability of the environment caused by the increase in the number of agents, general deep reinforcement learning has certain difficult