无人集群系统行为决策学习奖励机制 |
| |
引用本文: | 张婷婷,蓝羽石,宋爱国. 无人集群系统行为决策学习奖励机制[J]. 北京航空航天大学学报, 2021, 47(12): 2442-2451. DOI: 10.13700/j.bh.1001-5965.2020.0600 |
| |
作者姓名: | 张婷婷 蓝羽石 宋爱国 |
| |
作者单位: | 1.陆军工程大学 指挥控制工程学院, 南京 210017 |
| |
基金项目: | 国家自然科学基金61802428中国博士后科学基金2019M651991军委科技委国防科技基金2019-JCJQJJ-014 |
| |
摘 要: | 未来作战的发展方向是由多智能体系统构成的无人集群系统通过智能体之间自主协同来完成作战任务。由于每个智能体自主采取行为和改变状态,增加了智能群体行为策略训练的不稳定性。通过先验约束条件和智能体间的同构特性增强奖励信号的实时性,提高训练效率和学习的稳定性。采用动作空间边界碰撞惩罚、智能体间时空距离约束满足程度奖励;通过智能体在群体中的关系特性,增加智能体之间经验共享,进一步优化学习效率。在实验中,将先验增强的奖励机制和经验共享应用到多智能体深度确定性策略梯度(MADDPG)算法中验证其有效性。结果表明,学习收敛性和稳定性有大幅提高,从而提升了无人集群系统行为学习效率。
|
关 键 词: | 无人集群系统 MADDPG算法 对抗任务 行为决策 奖励机制 |
收稿时间: | 2020-10-23 |
Behavioral decision learning reward mechanism of unmanned swarm system |
| |
Affiliation: | 1.School of Command and Control Engineering, Army Engineering University of PLA, Nanjing 210017, China2.The 28th Research Institute of China Electronics Technology Group Corporation, Nanjing 210017, China3.School of Instrument Science and Engineering, Southeast University, Nanjing 210096, China |
| |
Abstract: | Unmanned swarm system is composed of a multi-agent system, which can meet task requirements through autonomous and cooperative behavior. The instability of agent training is increased because agents adopt behavior and change states autonomously. In this paper, the prior constraints and the isomorphism between agents are used to enhance the real-time performance of reward signals and improve the efficiency of training and the stability of learning. Specifically, it includes the punishment of action space boundary collision and the reward for the satisfaction degree of the space-time distance constraint between agents. At the same time, through the relationship characteristics of agents in the group, experience sharing among agents is increased to further optimize the learning efficiency. In the experiment, the prior enhanced reward mechanism and experience sharing are applied to the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm to verify its effectiveness. It is observed that the learning convergence and stability are greatly improved, and thus the behavior learning efficiency of unmanned swarm system is enhanced. |
| |
Keywords: | |
|
| 点击此处可从《北京航空航天大学学报》浏览原始摘要信息 |
|
点击此处可从《北京航空航天大学学报》下载全文 |