基于经验移植的自主空战对抗学习方法

现有的机器学习方法大多是交互式的学习模式，这类方法在训练过程非常依赖与环境之间的交互数据。空战对抗任务是一种奖励非常稀疏的训练任务，智能体在学习开始的很长一段时间内，都在探索能够获得奖励的动作。如果每一个新的任务都重新训练，是非常浪费计算资源的。因此，设计了一种基于经验移植的学习方法，使得经过训练的智能体能够将知识分享给新的智能体，提高其在新任务上的学习效率。首先，借鉴人类通过经验进行快速学习的现象，构建了基于经验移植的学习的模型；其次，兼顾知识分享和新任务的特征，明确了经验的内涵，建立了"知识+任务→经验"的融合认知方式；再次，设计了借鉴学习方法，将外部经验与任务相结合，进而转化为新个体的知识；最后，使用经验适用度作为筛选指标，分析了经验适用度对借鉴学习效率的影响，确定了执行借鉴学习的筛选边界。新个体通过借鉴学习后能够获得关于新任务的初步知识，在新任务中更快地找到能够获得奖励的动作策略，从而提升在新的任务中的学习速度。

Learning method for autonomous air combat based on experience transfer

Most of the existing machine learning methods are in interactive learning mode, whose training process relies heavily on the interactive data with the environment. Air combat is a training mission with sparse rewards, with the system usually exploring for a long period of time to find actions that can obtain rewards during the beginning stage of learning. Retraining for every new mission wastes the computing resources. Therefore, a learning method based on experience transfer is designed in this paper, enabling the trained agent to share knowledge with the new agent and thereby improving its learning efficiency in the new task. First of all, a learning model based on experience transfer is constructed by referring to the phenomenon that mankind can learn rapidly through experiences. Secondly, considering both the knowledge sharing and characteristics of the new task, the connotation of experience is defined, and a cognitive mode of "knowledge + task → experience" is established. Thirdly, a reference learning method is designed, combining external experience with the task to further transform it into knowledge of the new agent. Finally, using experience applicability as the screening index, we analyze the influence of experience applicability on the reference learning efficiency, determining the screening boundary of implementing the reference learning. The new agent can therefore obtain preliminary knowledge about the new mission by reference learning and find action policies that can obtain reward so as to improve the learning speed in the new learning mission.