基于强化学习的多智能体系统目标围捕控制

针对多智能体系统目标围捕问题，提出了基于强化学习的目标围捕控制方法。首先，对多智能体系统进行马尔可夫博弈建模，设计能够控制系统到期望围捕状态并满足避障要求的势能函数，将模型控制与强化学习原理结合，利用势能模型引导的改进多智能体强化学习算法进行围捕。其次，在已有势能模型的基础上建立跟踪围捕和环航围捕2种围捕策略。前者通过设计速度势能函数实现多智能体一致跟踪。后者加入虚拟环航点，设计虚拟环航点势能函数实现期望环航。最终，仿真验证了多智能体强化学习围捕控制策略的有效性。     

深度确定性策略梯度算法用于无人飞行器控制

对深度确定性策略梯度算法训练智能体学习小型无人飞行器的飞行控制策略进行了探索研究。以多数据帧的速度、位置和姿态角等信息作为智能体的观察状态,舵摆角和发动机推力指令作为智能体的输出动作,飞行器的非线性模型和飞行环境作为智能体的学习环境。智能体在与环境交互过程中除了获得包含误差信息的密集惩罚外,也有达成一定目标的稀疏奖励,该设计有效提高了飞行数据的样本多样性,增强了智能体的学习效率。最后智能体实现了从位置、速度和姿态角等信息到控制量的端到端飞行控制,并进行了变航迹点、模型参数拉偏、注入扰动和故障条件下的飞行控制仿真,结果表明智能体除了能有效完成训练任务外,还能应对多种训练时未学习的飞行任务,具有优秀的泛化能力和鲁棒性,该方法具有一定的研究价值和工程参考价值。     

基于多智能体的无人作战平台多机协同对抗多目标任务决策方法

建立了基于多智能体的多机协同作战任务决策方法结构模型,提出了基于神经网络与证据理论的敌我双方对抗态势分析方法和基于完全信息静态博弈模型的多机协同对抗多目标任务决策方法,并进行了基于典型作战想定的多机协同对抗多目标任务决策方法仿真研究.     

基于STDP奖励调节的类脑面向目标导航

动物具有优秀的空间自主定位导航能力,能够实现在无先验环境信息下的导航定位和导航决策过程。针对智能体在连续空间中面向目标导航问题,研究了一种基于生物学放电时间依赖可塑性学习规则的智能体面向目标导航算法。首先分析了动物面向目标导航决策过程中的生理学机理,在此基础上,构建了基于脉冲神经网络的位置细胞和动作细胞模型。动作细胞间权值采用横向竞争模型更新,通过环境奖励信号的更新,采用放电时间依赖可塑性学习规则对位置细胞前馈动作细胞模型的突触权重进行权值调节,利用动作细胞群的脉冲放电现象表征智能体运动方向和速度。最后,对所提算法进行了仿真实验验证。仿真结果表明,所提出的类脑面向目标导航算法能够在单障碍环境中实现30 ms左右的规划速度,相比传统强化学习Q学习方法平均路径规划长度缩短了15.9%。     

基于深度Q网络的多智能体逃逸算法设计CSCD

当前多智能体追逃博弈问题通常在二维平面下展开研究,且逃逸方智能体运动不受约束,同时传统方法在缺乏准确模型时存在设计控制策略困难的问题。针对三维空间中逃逸方智能体运动受约束的情况,提出了一种基于深度Q网络(DQN)的多智能体逃逸算法。该算法采用分布式学习的方法,逃逸方智能体通过对环境的探索学习得到满足期望的逃逸策略。为提高学习效率,根据任务的难易程度将智能体策略学习划分为两个阶段,并设计了相应的奖励函数引导智能体探索满足期望的逃逸策略。仿真结果表明,该算法所得逃逸策略效果稳定,并且具有泛化能力,在改变一定的初始位置条件后,逃逸方智能体也可成功逃逸。     

序言

多智能体的协同导航与控制作为人工智能及导航控制的重要分支,在当今受到广泛关注.“多智能体系统协同导航与控制技术”专栏,致力于探索自然界中生物个体间的信息交互和共享机制,发掘多智能体之间灵活有效的协同方法,解决导航、定位、环境感知等问题,以提高多智能体在复杂环境下的决策与工作能力.     

基于相对方位信息和单间距测量的多智能体编队协同控制

由多个载体组成的多智能体系统对复杂环境具有更高的适应性,能够完成传统单个载体无法完成的任务。针对多智能体编队集结与队形移动跟踪问题,提出了一种改进的多智能体编队协同控制新算法。首先,以拒止环境下跟随智能体仅能通过光学传感器测量相对方位信息为任务背景,针对"领导者——第一跟随者"结构的多智能体编队,提出了基于相对方位信息与单间距测量的控制器,使得第一跟随者智能体可以追随移动的领导者智能体,并且可以通过改变与领导者智能体的间距对编队整体队形进行缩放控制。其次,提出一种了改进的分布式控制律,使得其他跟随者智能体可以仅通过两个相对方位信息完成编队飞行。然后,根据Lyapunov第二方法,构建了系统的能量函数,验证了所提出算法的稳定性。最后,通过数值仿真实验对所提算法进行了验证。仿真结果表明,基于该控制律多智能体系统能够完成编队集结、队形缩放和编队飞行的任务。     

多舵面飞机电力作动系统协调控制策略研究

随着大型飞机舵面结构布局的变化,传统的集中式结构飞控系统难以满足舵面协调过程中准确性的要求。为此引入多智能体的概念,将单个舵面等效为一个智能体,构建分布式电力作动系统的多智能体系统结构。采用联盟式体系结构,分别对联盟内部分体式舵面智能体的同步联动控制、不同联盟间舵面协调偏转控制进行控制策略的设计,并建立仿真模型。仿真结果表明,舵面能够准确地收敛到给定的舵面协调偏转状态,并且该策略能有效抑制舵面负载干扰引起的协调偏转率波动,解决了传统集中式飞控系统多舵面协调控制准确性不好的问题。     

异构多智能体系统分组输出时变编队跟踪控制

空地协同控制是前沿的热点研究之一,以无人机、无人车为代表的空地智能体动力学模型的差异为研究带来了挑战。研究了高阶异构多智能体系统在有向拓扑条件下的分组输出时变编队跟踪控制问题,提出了虚拟领导者、分组领导者以及跟随者组成的三层协同控制架构。虚拟领导者用于规划整个多智能体系统的状态轨迹,分组领导者跟踪虚拟领导者所提供的轨迹信息,并相互协作以实现分组间的协同配合。跟随者跟踪分组领导者的输出并实现期望的输出编队。在有向通信拓扑结构条件下,基于局部邻居间的相对信息、观测器理论和滑模控制理论构造了控制协议,利用Lyapunov稳定性理论证明协议的有效性。数值仿真结果表明提出的方法能够实现无人机、无人车等异构智能体的空地协同,具有较好的工程应用价值。     

Q-learning强化学习协同拦截制导律CSCD

为实现多枚导弹协同拦截机动目标,提升拦截效能,提出了一种Q-learning强化学习协同拦截制导律。首先,基于逃逸域覆盖理论,建立了非线性多弹协同拦截模型。其次,以视线角速率为状态,依据脱靶量构造奖励函数,通过离线训练生成强化学习智能体,并结合传统比例制导控制方法,构建基于强化学习的变导引系数制导律,实时生成实现协同拦截的制导指令。最终,通过数值仿真验证了所提算法的有效性和优越性。     

Formation tracking control for time-delayed multi-agent systems with second-order dynamics

In this paper, formation tracking control problems for second-order multi-agent systems (MASs) with time-varying delays are studied, specifically those where the position and velocity of followers are designed to form a time-varying formation while tracking those of the leader. A neigh-boring relative state information based formation tracking protocol with an unknown gain matrix and time-varying delays is presented. The formation tracking problems are then transformed into asymptotically stable problems. Based on the Lyapunov-Krasovskii functional approach, condi-tions sufficient for second-order MASs with time-varying delays to realize formation tracking are examined. An approach to obtain the unknown gain matrix is given and, since neighboring relative velocity information is difficult to measure in practical applications, a formation tracking protocol with time-varying delays using only neighboring relative position information is introduced. The proposed results can be used on target enclosing problems for MASs with second-order dynamics and time-varying delays. An application for target enclosing by multiple unmanned aerial vehicles (UAVs) is given to demonstrate the feasibility of theoretical results.     

基于零化相对速度偏角的变结构末制导律设计

针对拦截问题,选取导弹和目标的相对速度矢量与导弹—目标视线之间的夹角（即相对速度偏角）作为滑模面,无须对导弹—目标相对运动学方程进行归一化,且不对弹目速度比进行限制,将有关目标运动的加速度和方位信息视为干扰量,应用变结构理论设计了一种变结构末制导律,并应用Lyapunov理论进行了稳定性分析。仿真结果表明,所设计的导引律对目标机动具有很强的鲁棒性。     

Three-dimensional analysis of relationship between relative orientation and motion modes

Target motion modes have a close relationship with the relative orientation of missile-totarget in three-dimensional highly maneuvering target interception. From the perspective of relationship between the sensor coordinate system and the target body coordinate system, a basic model of sensor is stated and the definition of relative angular velocity between the two coordinate systems is introduced firstly. Then, the three-dimensional analytic expressions of relative angular velocity for different motion modes are derived and simplified by analyzing the influences of target centroid motion, rotation around centroid and relative motion. Finally, the relationships of the relative angular velocity directions and values with motion modes are discussed. Simulation results validate the rationality of the theoretical analysis. It is demonstrated that there are significant differences of the relative orientation in different motion modes which include luxuriant information about motion modes. The conclusions are significant for the research of motion mode identification,maneuver detection, maneuvering target tracking and interception using target signatures.     

基于多Agent的舰载机弹射起飞仿真层次模型(英文)

With the aid of multi-agent based modeling approach to complex systems, the hierarchy simulation models of carrier-based aircraft catapult launch are developed. Ocean, carrier, aircraft, and atmosphere are treated as aggregation agents, the detailed components like catapult, landing gears, and disturbances are considered as meta-agents, which belong to their aggregation agent. Thus, the model with two layers is formed i.e. the aggregation agent layer and the meta-agent layer. The information communication among all agents is described. The meta-agents within one aggregation agent communicate with each other directly by information sharing, but the meta-agents, which belong to different aggregation agents exchange their information through the aggregation layer first, and then perceive it from the sharing environment, that is the aggregation agent. Thus, not only the hierarchy model is built, but also the environment perceived by each agent is specified. Meanwhile, the problem of balancing the independency of agent and the resource consumption brought by real-time communication within multi-agent system （MAS） is resolved. Each agent involved in carrier-based aircraft catapult launch is depicted, with considering the interaction within disturbed atmospheric environment and multiple motion bodies including carrier, aircraft, and landing gears. The models of reactive agents among them are derived based on tensors, and the perceived messages and inner frameworks of each agent are characterized. Finally, some results of a simulation instance are given. The simulation and modeling of dynamic system based on multi-agent system is of benefit to express physical concepts and logical hierarchy clearly and precisely. The system model can easily draw in kinds of other agents to achieve a precise simulation of more complex system. This modeling technique makes the complex integral dynamic equations of multibodies decompose into parallel operations of single agent, and it is convenient to expand, maintain, and reuse the pro     

筒式偏心在轨分离角速度抑制方法

筒式偏心在轨分离是一类特殊的在轨分离问题,小卫星偏心安装而产生的分离力矩将导致分离角速度,进而影响小卫星的分离指向精度,甚至导致释放平台姿态失稳。而常规的姿态大角速度机动、姿态快速稳定控制方法难以在小卫星出筒前的极短时间内完成分离角速度抑制。因此,进行了卫星筒式偏心在轨分离动力学分析,基于分离角速度的产生,提出了抑制分离姿态干扰的前馈控制力矩法和角速度预偏置法。在此基础上,推导了关键控制参数的近似计算公式,给出了控制量的优化求解方法,并分析了控制干扰因素对抑制结果的影响。最后,通过仿真算例分析,对比验证了两种抑制方法的有效性,并给出了其工程应用的建议。     

Distributed event-triggered adaptive control for second-order nonlinear uncertain multi-agent systems

In this paper, the event-triggered consensus control problem for nonlinear uncertain multi-agent systems subject to unknown parameters and external disturbances is considered. The dynamics of subsystems are second-order with similar structures, and the nodes are connected by undirected graphs. The event-triggered mechanisms are not only utilized in the transmission of information from the controllers to the actuators, and from the sensors to the controllers within each agent, but also in the communication between agents. Based on the adaptive backstepping method, extra estimators are introduced to handle the unknown parameters, and the measurement errors that occur during the event-triggered communication are well handled by designing compensating terms for the control signals. The presented distributed event-triggered adaptive control laws can guarantee the boundness of the consensus tracking errors and the Zeno behavior is avoided. Meanwhile, the update frequency of the controllers and the load of communication burden are vastly reduced. The obtained control protocol is further applied to a multi-input multi-output second-order nonlinear multi-agent system, and the simulation results show the effectiveness and advantages of our proposed method.     

Autonomous attitude coordinated control for spacecraft formation with input constraint,model uncertainties,and external disturbances

To synchronize the attitude of a spacecraft formation flying system, three novel autonomous control schemes are proposed to deal with the issue in this paper. The first one is an ideal autonomous attitude coordinated controller, which is applied to address the case with certain models and no disturbance. The second one is a robust adaptive attitude coordinated controller, which aims to tackle the case with external disturbances and model uncertainties. The last one is a filtered robust adaptive attitude coordinated controller, which is used to overcome the case with input con- straint, model uncertainties, and external disturbances. The above three controllers do not need any external tracking signal and only require angular velocity and relative orientation between a spacecraft and its neighbors. Besides, the relative information is represented in the body frame of each spacecraft. The controllers are proved to be able to result in asymptotical stability almost everywhere. Numerical simulation results show that the proposed three approaches are effective for attitude coordination in a spacecraft formation flying system.     

圆形轨迹导引下的无人机事件触发抗干扰环绕控制

针对内部不确定性以及外部环境摄动的目标环绕控制问题,在基于反步法的双层制导框架下,利用级联控制思想,提出了一种圆形轨迹导引下的四旋翼无人机事件触发抗扰环绕控制方法。在轨迹回路中,构建了可满足持续激励条件的目标位置估计器,保证仅通过视线方位角就能获取可满足最终一致有界条件的目标估计项。随后,基于目标的位置估计结果,设计了目标环绕控制律生成线速度指令,并通过方向向量场验证了该环绕制导律的有效性,消除了现有李雅普诺夫向量场制导(LVFG)对相对位置和目标速度的依赖。在姿态回路中,通过采用扩张状态观测器(ESO)补偿系统的集总不确定性,设计了基于相对阈值事件触发控制的姿态控制器,在有效降低控制器到执行机构之间信号传输频率的同时,实现了四旋翼无人机对静止/移动目标环绕。然后,借助输入状态稳定性定理证明了系统的稳定性。仿真结果表明,所提控制方案能够实现圆形轨迹导引下四旋翼无人机对静止/移动目标的环绕监视。     

Event-triggered adaptive control for attitude tracking of spacecraft

Plug-and-play technology is an important direction for future development of spacecraft and how to design controllers with less communication burden and satisfactory performance is of great importance for plug-and-play spacecraft. Considering attitude tracking of such spacecraft with unknown inertial parameters and unknown disturbances, an event-triggered adaptive backstepping controller is designed in this paper. Particularly, a switching threshold strategy is employed to design the event-triggering mechanism. By introducing a new linear time-varying model, a smooth function, an integrable auxiliary signal and a bound estimation approach, the impacts of the network-induced error and the disturbances are effectively compensated for and Zeno phenomenon is successfully avoided. It is shown that all signals of the closed-loop system are globally uniformly bounded and both the attitude tracking error and the angular velocity tracking error converge to zero. Compared with conventional control schemes, the proposed scheme significantly reduces the communication burden while providing stable and accurate response for attitude maneuvers. Simulation results are presented to illustrate the effectiveness of the proposed scheme.