Improving multi-target cooperative tracking guidance for UAV swarms using multi-agent reinforcement learning

Multi-Target Tracking Guidance(MTTG) in unknown environments has great potential values in applications for Unmanned Aerial Vehicle(UAV) swarms. Although Multi-Agent Deep Reinforcement Learning(MADRL) is a promising technique for learning cooperation, most of the existing methods cannot scale well to decentralized UAV swarms due to their computational complexity or global information requirement. This paper proposes a decentralized MADRL method using the maximum reciprocal reward to learn cooper...     

重访机制驱动的多无人机协同动目标搜索方法

为实现多无人机高效捕获灰色任务区域内的移动目标,考虑传感器探测概率与虚警概率,提出了重访机制驱动的协同搜索规划（RMD-CSP）方法,以降低目标遗漏与误判概率。考虑无人机飞行性能约束,以最大化任务执行效能为目标建立多无人机协同搜索模型。根据目标先验信息初始化环境搜索信息图（包括目标概率分布图、环境不确定度图与环境搜索状态图）,利用无人机实时探测信息,基于贝叶斯准则持续更新搜索信息图。定制基于环境不确定度更新的重访机制,通过增加长时间未被重访区域的环境不确定度,引导无人机搜索该区域,降低移动目标的遗漏概率;定制基于目标函数权重更新的重访机制,引导无人机快速重访发现新的疑似目标的区域,对疑似目标进行再次确认,减少由于传感器虚警概率造成的目标误判概率。采用滚动时域规划架构,将搜索规划问题分解为一系列短时域规划问题,提升了求解效率。在典型任务想定下,通过数值仿真试验验证了所提方法的有效性。仿真结果表明,RMD-CSP能够在秒级时间内生成每个时域的搜索航迹,相比于光栅式搜索方法与标准的概率启发式搜索方法,能够引导无人机捕获更多的移动目标,同时减少误判次数,有效提升了多无人机协同搜索的任务效能。     

Formation flight of fixed-wing UAV swarms: A group-based hierarchical approach

This paper investigates a formation control problem of fixed-wing Unmanned Aerial Vehicle (UAV) swarms. A group-based hierarchical architecture is established among the UAVs, which decomposes all the UAVs into several distinct and non-overlapping groups. In each group, the UAVs form hierarchies with one UAV selected as the group leader. All group leaders execute coordinated path following to cooperatively handle the mission process among different groups, and the remaining followers track their direct leaders to achieve the inner-group coordination. More specifically, for a group leader, a virtual target moving along its desired path is assigned for the UAV, and an updating law is proposed to coordinate all the group leaders’ virtual targets; for a follower UAV, the distributed leader-following formation control law is proposed to make the follower’s heading angle coincide with its direct leader, while keeping the desired relative position with respect to its direct leader. The proposed control law guarantees the globally asymptotic stability of the whole closed-loop swarm system under the control input constraints of fixed-wing UAVs. Theoretical proofs and numerical simulations are provided, which corroborate the effectiveness of the proposed method.     

编队无人机的高生存力协同航路规划方法

提出了一种基于多目标遗传算法的编队无人机高生存力协同航路规划方法。方法由备选航路生成和协同规划两个步骤组成。备选航路生成的目的是为编队中的每一个无人机生成多条航路,该步骤采用的算法是多目标遗传算法。协同规划的目的是为各个无人机从备选航路中选择航路,使得各个无人机同时到达目标区域,以增加任务突然性,提高整个编队的生存力。通过仿真算例,把方法与基于Voronoi图的方法作了对比,给出了方法的优缺点分析。     

Adaptive path planning for unmanned aerial vehicles based on bi-level programming and variable planning time interval

This paper presents an adaptive path planner for unmanned aerial vehicles (UAVs) to adapt a real-time path search procedure to variations and fluctuations of UAVs’ relevant performances, with respect to sensory capability, maneuverability, and flight velocity limit. On the basis of a novel adaptability-involved problem statement, bi-level programming (BLP) and variable planning step techniques are introduced to model the necessary path planning components and then an adaptive path planner is developed for the purpose of adaptation and optimization. Additionally, both probabilistic-risk-based obstacle avoidance and performance limits are described as path search constraints to guarantee path safety and navigability. A discrete-search-based path planning solution, embedded with four optimization strategies, is especially designed for the planner to efficiently generate optimal flight paths in complex operational spaces, within which different surface-to-air missiles (SAMs) are deployed. Simulation results in challenging and stochastic scenarios firstly demonstrate the effectiveness and efficiency of the proposed planner, and then verify its great adaptability and relative stability when planning optimal paths for a UAV with changing or fluctuating performances.     

基于分布式模型预测控制的多UAV协同区域搜索

针对多无人机(UAV)协同区域搜索问题展开研究。提出了一种基于分布式模型预测控制(DMPC)的多UAV分布式优化搜索方法。首先基于传统的搜索图模型,建立了多UAV协同搜索的问题描述和状态空间模型,然后在DMPC框架下,将集中式多UAV在线优化决策问题转化为各架UAV的小规模分布式优化问题,采用基于纳什最优和粒子群优化(PSO)相结合的算法实现对每个子系统优化问题的迭代求解。仿真结果表明:DMPC方法能够有效地降低多UAV协同搜索决策问题的求解规模,是一种可行的方法。     

未知动态环境下有人/无人机协同搜救研究

针对未知动态环境下搜救效率和智能化水平不高的问题，提出了一种基于变步长模型预测控制的有人机/无人机协同搜救方式。根据环境的动态变化性和探测的不确定性，建立了表示环境确定度和目标探测概率的动态搜救图模型，根据态势信息选择时间域步长，并根据有人机/无人机协同搜救任务特点，设计了搜救收益指标，并利用粒子群优化进行求解。仿真结果表明了本文方法的有效性和优越性。     

多无人机协同覆盖路径规划

多无人机协同覆盖路径规划(CPP)由于其并行性和容错能力,对于提高无人机完成侦察、监视、搜索等任务的效率具有重要意义。提出了一种基于无人机任务性能评价和任务区域划分的多无人机协同CPP算法。定量分析了无人机执行覆盖任务的能力,根据无人机及携带成像传感器的性能给出了计算无人机任务性能指数的数学公式;提出了一种基于任务性能和子区域宽度的任务区域划分算法,使无人机的总转弯次数达到最少。仿真结果表明,所提出的CPP算法能够规划出全局最优的多无人机协同覆盖路径。     

基于流水避石原理的无人机三维航路规划方法

借鉴自然界流水避石现象,提出一种基于流体计算的无人机(UAV)三维(3D)航路规划方法.首先介绍了球心位于坐标原点时,球形障碍三维绕流问题的解析解.之后采用旋转平移矩阵与流线数据叠加方法生成了任意位置多障碍同时存在的三维流线.为验证解析解的有效性同时给出该方法基于数值模拟的计算过程,对适合无人机三维航路规划的流体模型和数值求解方法进行了分析,并给出了通过数值模拟求解航路的方法.最后,根据无人机机动约束对流线进行处理得到可飞航路,将航路长度、纵向和横侧向机动次数作为子目标函数对航路进行综合评价.仿真结果表明:解析法航路规划中,圆球障碍的地形建模简单计算量小,航路集中在由起点至终点的航路带间;数值法航路规划适合障碍分布复杂的地形,航路分布于规划空间中.这两种方法的航路平滑,能够满足无人机飞行约束,航路具有绕流意义的最优性,可以避免势场法的局部极小问题,并且可以提供多条备选航路.     

基于Leader-Follower编队的无人机协同跟踪地面目标制导律设计

对地面目标的自动跟踪是无人机在任务应用阶段需要解决的重要问题之一,多无人机协同跟踪能够提高对目标运动状态的估计精度并降低目标丢失的概率,因而具有重要研究意义。本文提出了一种基于Leader-Follower编队的无人机协同跟踪制导方法,解决了传统Standoff跟踪模式对地面目标的速度范围限制问题。首先,通过控制无人机的航向不断趋近于地面目标牵连跟踪圆切线方向的方法设计了Leader无人机自动跟踪地面目标的制导律并完成了稳定性证明;其次,通过控制Follower无人机的速度和航向角逐渐趋近于Leader无人机速度和航向的协同跟踪策略,分别设计了Follower无人机自动跟踪Leader无人机的制导律和编队相位协同制导律并完成了稳定性证明;最后,分别针对静止目标、匀速直线运动目标和变速运动目标的跟踪问题进行了仿真验证,结果表明所提出的制导方法能够实现对不同运动状态地面目标的自动协同跟踪,并且跟踪性能优于基于李雅普诺夫向量法的制导方法。     

多太阳能无人机覆盖路径优化方法

为解决传统电动无人机在覆盖作业时存在的续航时间短的问题,提出应用多架太阳能无人机进行覆盖作业。首先,在建立了应用于覆盖作业的太阳能无人机的能量模型的基础上,提出了能量流动效率这一指标来评价太阳能无人机在作业过程中对能量的利用率。其次,针对边界存在障碍物的凹多边形区域和内部含障碍物的多边形区域,以总作业完成时间最短为优化目标,提出基于无向图搜索方法的覆盖路径优化模型,定义约束方程限制无人机按照一定规则访问无向图中的节点,通过混合整数线性规划的方法求解每架无人机的最优飞行路径。再次,考虑无人机转弯时的姿态变化对能量流动效率的影响,将总作业完成时间最短和总能量流动效率最高同时作为优化目标,建立双目标优化方程,在首先以作业时间最短为优化目标进行求解的基础上,通过有限遍历的方式选择使能流效率和作业时间相对最优的覆盖飞行方向及飞行路径。大量仿真实验表明,所提的优化模型选取不同的优化目标,应用于不同形状的待覆盖区域,适用性广,在工程上应用范围广、可行性强。     

基于改进一致性算法的无人机编队控制

编队飞行是指多架无人机保持以一定队形进行飞行的状态,相比于单架飞机执行任务,无人机编队能够增加搜索面积,提高飞机飞行性能,增大完成任务成功率。编队控制是实现编队安全高效完成指定任务的前提。本文以一致性理论为基础,针对无人机运动模型的特点与实际飞行要求,对基本的一致性算法进行改进,提出了改进一致性无人机编队控制算法。首先利用纵向和横侧向解耦的自动驾驶仪模型给出了无人机的三自由度运动方程,根据机动性与飞行性能要求定义了各方向上的加速度、速度与角速度约束。基于一致性理论,将编队控制分为平面与纵向2个方向进行,在状态控制的基础上,利用各状态变量间的几何关系对无人机运动自由度进行转换,加入编队队形信息,设计了编队控制算法。为了使算法生成的指令信号满足约束条件,提出了"最小调整"约束条件处理策略。依据粒子群算法对各无人机的爬升加速度进行优化,以避免机间碰撞。仿真结果表明：提出的编队控制算法具备编队成形与变换功能,能够使无人机编队状态快速收敛到指定值,且保持指定队形,无人机飞行状态满足所有约束条件。     

Target localization based on cross-view matching between UAV and satellite

Matching remote sensing images taken by an unmanned aerial vehicle (UAV) with satellite remote sensing images with geolocation information. Thus, the specific geographic location of the target object captured by the UAV is determined. Its main challenge is the considerable differences in the visual content of remote sensing images acquired by satellites and UAVs, such as dramatic changes in viewpoint, unknown orientations, etc. Much of the previous work has focused on image matching of homologous data. To overcome the difficulties caused by the difference between these two data modes and maintain robustness in visual positioning, a quality-aware template matching method based on scale-adaptive deep convolutional features is proposed by deeply mining their common features. The template size feature map and the reference image feature map are first obtained. The two feature maps obtained are used to measure the similarity. Finally, a heat map representing the probability of matching is generated to determine the best match in the reference image. The method is applied to the latest UAV-based geolocation dataset (University-1652 dataset) and the real-scene campus data we collected with UAVs. The experimental results demonstrate the effectiveness and superiority of the method.     

基于DDPG算法的无人机集群追击任务

无人机的集群化应用技术是近年来的研究热点，随着无人机自主智能的不断提高，无人机集群技术必将成为未来无人机发展的主要趋势之一。针对无人机集群协同执行对敌方来袭目标的追击任务，构建了典型的任务场景，基于深度确定性策略梯度网络（DDPG）算法，设计了一种引导型回报函数有效解决了深度强化学习在长周期任务下的稀疏回报问题，通过引入基于滑动平均值的软更新策略减少了DDPG算法中Eval网络和Target网络在训练过程中的参数震荡，提高了算法的训练效率。仿真结果表明，训练完成后的无人机集群能够较好地执行对敌方来袭目标的追击任务，任务成功率达到95%。可以说无人机集群技术作为一种全新概念的作战模式在军事领域具有潜在的应用价值，人工智能算法在无人机集群的自主决策智能化发展方向上具有一定的应用前景。     

Multi-UAV reconnaissance task allocation for heterogeneous targets using an opposition-based genetic algorithm with double-chromosome encoding

This paper presents a novel multiple Unmanned Aerial Vehicles (UAVs) reconnaissance task allocation model for heterogeneous targets and an effective genetic algorithm to optimize UAVs’ task sequence. Heterogeneous targets are classified into point targets, line targets and area targets according to features of target geometry and sensor’s field of view. Each UAV is regarded as a Dubins vehicle to consider the kinematic constraints. And the objective of task allocation is to minimize the task execution time and UAVs’ total consumptions. Then, multi-UAV reconnaissance task allocation is formulated as an extended Multiple Dubins Travelling Salesmen Problem (MDTSP), where visit paths to the heterogeneous targets must meet specific constraints due to the targets’ feature. As a complex combinatorial optimization problem, the dimensions of MDTSP are further increased due to the heterogeneity of targets. To efficiently solve this computationally expensive problem, the Opposition-based Genetic Algorithm using Double-chromosomes Encoding and Multiple Mutation Operators (OGA-DEMMO) is developed to improve the population variety for enhancing the global exploration capability. The simulation results demonstrate that OGA-DEMMO outperforms the ordinary genetic algorithm, ant colony optimization and random search in terms of optimality of the allocation results, especially for large scale reconnaissance task allocation problems.     

基于遗传模拟退火算法的无人机航迹规划

航迹规划技术是无人机任务规划系统中重要的核心技术之一,无人机飞行空间广阔,需要一种快速搜索最佳路径的方法.首先在飞行区域中建立数字地图模型和防空威胁区模型,在满足无人机飞行约束条件的情况下,为无人机航迹规划提供一种遗传模拟退火算法,充分利用模拟退化算法的概率突跳特性和遗传算法强大的快速搜索能力.仿真结果表明,使用该算法无人机能够自动避开模拟数字地图的威胁区,搜索出一条安全有效航迹,并保证航线的完整性和最优性.     

Flight safety measurements of UAVs in congested airspace

Describing spatial safety status is crucial for high-density air traffic involving multiple unmanned aerial vehicles (UAVs) in a complex environment. A probabilistic approach is proposed to measure safety situation in congested airspace. The occupancy distribution of the airspace is represented with conflict probability between spatial positions and UAV. The concept of a safety envelope related to flight performance and response time is presented first instead of the conventional fixed-size protected zones around aircraft. Consequently, the conflict probability is performance-dependent, and effects of various UAVs on safety can be distinguished. The uncer-tainty of a UAV future position is explicitly accounted for as Brownian motion. An analytic approximate algorithm for the conflict probability is developed to decrease the computational consumption. The relationship between safety and flight performance are discussed for different response times and prediction intervals. To illustrate the applications of the approach, an experi-ment of three UAVs in formation flight is performed. In addition, an example of trajectory planning is simulated for one UAV flying over airspace where five UAVs exist. The validation of the approach shows its potential in guaranteeing flight safety in highly dynamic environment.     

复杂环境下微小飞行器惯性/激光雷达Robust-SLAM方法

针对微小型飞行器在巡检、探测和地图构建等应用中关键的自主导航技术,提出了一种基于惯性辅助的激光雷达Robust-SLAM方法用于微小型飞行器自主导航。相对于传统的激光雷达SLAM方法,该方法在SLAM框架中引入了感知环境突变检测方法,并且加强了惯性与SLAM的组合程度,有效地解决了高程方向感知环境发生突变时激光雷达SLAM定位误差大的问题。室内车库实际飞行实验结果表明,该方法能够实现微小型飞行器在三维空间中实时可靠的自主导航,具有较好的工程应用价值。     

多无人机监控航路规划

为了提高监控任务的效率，在执行任务前必须规划设计出高效的无人机飞行航路。针对这一问题，考虑在任务区域内进行多无人机监控航路优化存在计算的复杂性和收敛性以及计算数据量限制等问题，采用分散式方法对监控航路进行了优化，并以无人机的监控任务为例提出了一种监控效率指标评估的计算方法，解决了航路规划中的监控效率量化问题。通过该方法得到的无人机监控任务的飞行航路可以有效地提高无人机的监控效率。     

Collision free 4D path planning for multiple UAVs based on spatial refined voting mechanism and PSO approach

In this paper, a four-dimensional coordinated path planning algorithm for multiple UAVs is proposed, in which time variable is taken into account for each UAV as well as collision free and obstacle avoidance. A Spatial Refined Voting Mechanism(SRVM) is designed for standard Particle Swarm Optimization(PSO) to overcome the defects of local optimal and slow convergence.For each generation candidate particle positions are recorded and an adaptive cube is formed with own adaptive side length to indicate occupied regions. Then space voting begins and is sorted based on voting results, whose centers with bigger voting counts are seen as sub-optimal positions. The average of all particles of corresponding dimensions are calculated as the refined solutions. A time coordination method is developed by generating specified candidate paths for every UAV, making them arrive the same destination with the same time consumption. A spatial-temporal collision avoidance technique is introduced to make collision free. Distance to destination is constructed to improve the searching accuracy and velocity of particles. In addition, the objective function is redesigned by considering the obstacle and threat avoidance, Estimated Time of Arrival(ETA), separation maintenance and UAV self-constraints. Experimental results prove the effectiveness and efficiency of the algorithm.