Imaginary filtered hindsight experience replay for UAV tracking dynamic targets in large-scale unknown environments

As an advanced combat weapon, Unmanned Aerial Vehicles (UAVs) have been widely used in military wars. In this paper, we formulated the Autonomous Navigation Control (ANC) problem of UAVs as a Markov Decision Process (MDP) and proposed a novel Deep Reinforcement Learning (DRL) method to allow UAVs to perform dynamic target tracking tasks in large-scale unknown environments. To solve the problem of limited training experience, the proposed Imaginary Filtered Hindsight Experience Replay (IFHER) generates successful episodes by reasonably imagining the target trajectory in the failed episode to augment the experiences. The well-designed goal, episode, and quality filtering strategies ensure that only high-quality augmented experiences can be stored, while the sampling filtering strategy of IFHER ensures that these stored augmented experiences can be fully learned according to their high priorities. By training in a complex environment constructed based on the parameters of a real UAV, the proposed IFHER algorithm improves the convergence speed by 28.99% and the convergence result by 11.57% compared to the state-of-the-art Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm. The testing experiments carried out in environments with different complexities demonstrate the strong robustness and generalization ability of the IFHER agent. Moreover, the flight trajectory of the IFHER agent shows the superiority of the learned policy and the practical application value of the algorithm.     

基于场景识别的惯性基类脑导航方法

针对无人机导航中惯性器件产生的漂移误差不断随时间累积进而影响导航精度的问题,以仿生类脑导航为研究背景,提出了一种新的导航方法。与传统导航策略不同的是,该方法从周期性校正累积位置误差的角度出发,采用设置位置细胞节点的思路,利用训练好的卷积神经网络模型在细胞节点处进行图像匹配,从而在位置细胞节点处实现惯导位置漂移误差校正。同时,建立节点之间的漂移误差模型,调整误差方程系数,以达到修正漂移误差的目的。最后,基于无人机飞行实验结果验证了该方法对自主导航的有效性和鲁棒性,该方法能够有效提高无人机导航的精度。     

一种用于仿生导航无人机航姿求解的混合滤波方法

针对现有组合导航系统易被干扰欺骗以及姿态求解精度不足的问题，设计了惯性测量单元(IMU)与偏振光传感器组成的航姿参考系统(AHRS)。同时，考虑到传统的姿态求解方法精度不高，提出了一种用于仿生导航无人机航姿求解的混合滤波方法。将Mahony滤波后的姿态值作为系统观测量，再结合扩展卡尔曼滤波(EKF)实现传感器数据的深层融合，以获得高精度的姿态角信息。实验结果表明：在静态环境下采用混合滤波方法求解的姿态值能有效滤除偏振光传感器和加速度计内部噪声干扰，其稳定性明显优于两种方法各自求解时的情况；在动态实验中该方法能有效抑制单独采用Mahony滤波时存在的超调问题，表现出更高的动态解算精度，从而为偏振光组合导航系统提供了更精确的姿态估计信息。     

Optimal video communication strategy for intelligent video analysis in unmanned aerial vehicle applications

For Unmanned Aerial Vehicles (UAV), the intelligent video analysis is a key technology in intelligent autonomous control, real-time navigation and surveillance. However, poor UAV wireless links would degrade the quality of video communication, leading to difficulties in video analysis. To meet the challenges of packet-loss and limited bandwidth in adverse UAV channel environments, this paper proposes a parameter optimization mechanism for UAV intelligent video analysis. In the proposed method, an Optimal Strategy Library (OSL) is designed to optimize the parameters for video encoding and forward error correction. Adapted to the packet-loss rate and bandwidth in practical UAV wireless network, the proposed OSL can facilitate the encoding of video sequences and the recovery of degraded videos with optimal performance. Experimental results demonstrate that the proposed solution can keep intelligent video analysis working efficiently with adverse UAV wireless links, and is capable of maximizing the inference accuracy of Multi-Object Tracking (MOT) algorithms in various scenarios.     

无人机空中加油自主会合的制导与控制

针对无人机空中加油的自主会合问题,进行了相应制导律和非线性控制器的设计。通过改进的带角度约束的三维比例制导律实现对航向角的控制,以协调转弯的方式将航迹角指令转化为姿态角指令。基于无人机六自由度的动力学模型,针对无人机的姿态控制,采用时标分离的方法设计了慢子系统和快子系统,并对这两个子系统分别进行动态逆控制设计。同时,基于滑模控制的方法设计了满足自主会合要求的速度控制律。在保证无人机飞行稳定的基础上,实现了对控制和制导指令的精确跟踪。仿真结果表明,所设计的制导律和控制律能够实现无人机空中加油的自主会合,具有良好的动态特性。     

Improving multi-target cooperative tracking guidance for UAV swarms using multi-agent reinforcement learning

Multi-Target Tracking Guidance(MTTG) in unknown environments has great potential values in applications for Unmanned Aerial Vehicle(UAV) swarms. Although Multi-Agent Deep Reinforcement Learning(MADRL) is a promising technique for learning cooperation, most of the existing methods cannot scale well to decentralized UAV swarms due to their computational complexity or global information requirement. This paper proposes a decentralized MADRL method using the maximum reciprocal reward to learn cooper...     

面向城市飞行安全的无人机离散型多路径规划方法

为了提高无人机（UAV）在城市环境中运行的安全性,且能生成多条备选路径,提出一种离散型城市环境下基于无人机飞行安全的多路径规划方法。根据定义的城市环境模型、无人机的飞行规则和安全性原则,建立无人机飞行安全性分析模型和离散型多路径规划问题的数学模型。为提高算法的收敛速度和解的优质性,以及使算法能够同时输出多条路径,针对蚁群（ACO）算法的运行机制,设计聚类算子,提出改进聚类蚁群（CIACO）算法。实验结果表明,所提方法能够快速的收敛输出多条风险值较低的飞行路径。     

基于导向强化Q学习的无人机路径规划

随着无人机的广泛应用,其飞行能耗和计算能力面临着瓶颈问题,因此无人机路径规划研究越来越重要。很多情况下,无人机并不能提前获得目标点的确切位置和环境信息,往往无法规划出一条有效的飞行路径。针对这一问题,提出了基于导向强化Q学习的无人机路径规划方法,该方法利用接收信号强度定义回报值,并通过Q学习算法不断优化路径;提出"导向强化"的原则,加快了学习算法的收敛速度。仿真结果表明,该方法能够实现无人机的自主导航和快速路径规划,与传统算法相比,大大减少了迭代次数,能够获得更短的规划路径。     

密林环境空地协同GNSS/UWB快速高精度定位技术

针对密林中卫星信号遮挡难以实现快速高精度定位的问题,提出了一种密林环境空地协同全球卫星导航系统/超宽带(GNSS/UWB)高精度定位方法。该方法综合考虑无人机平台在空旷环境快速运动与超宽带的强穿透性测量等特征,通过无人机携带GNSS/UWB集成化载荷,以移动单基站模拟多基站,配合密林中UWB标签组网测距,完成密林中UWB标签定位。对基站布设方案展开研究,为密林环境下空地协同GNSS/UWB快速高精度定位技术应用提供基站布设指导和依据,并且利用密林环境下实测测距实验+仿真定位实验对提出的方法进行验证。结果表明,所述方法可有效实现密林环境下的快速高精度定位,精度达到分米级,并且通过优选基站布设网型、范围及高度可有效提高定位精度,为密林环境下快速高精度定位方法提供了理论支撑。     

复杂环境下微小飞行器惯性/激光雷达Robust-SLAM方法

针对微小型飞行器在巡检、探测和地图构建等应用中关键的自主导航技术,提出了一种基于惯性辅助的激光雷达Robust-SLAM方法用于微小型飞行器自主导航。相对于传统的激光雷达SLAM方法,该方法在SLAM框架中引入了感知环境突变检测方法,并且加强了惯性与SLAM的组合程度,有效地解决了高程方向感知环境发生突变时激光雷达SLAM定位误差大的问题。室内车库实际飞行实验结果表明,该方法能够实现微小型飞行器在三维空间中实时可靠的自主导航,具有较好的工程应用价值。     

Deep learning enabled localization for UAV autolanding

This article concentrates on ground vision guided autonomous landing of a fixed-wing Unmanned Aerial Vehicle (UAV) within Global Navigation Satellite System (GNSS) denied environments. Cascaded deep learning models are developed and employed into image detection and its accuracy promoting for UAV autolanding, respectively. Firstly, we design a target bounding box detection network BboxLocate-Net to extract its image coordinate of the flying object. Secondly, the detected coordinate is fused into spatial localization with an extended Kalman filter estimator. Thirdly, a point regression network PointRefine-Net is developed for promoting detection accuracy once the flying vehicle’s motion continuity is checked unacceptable. The proposed approach definitely accomplishes the closed-loop mutual inspection of spatial positioning and image detection, and automatically improves the inaccurate coordinates within a certain range. Experimental results demonstrate and verify that our method outperforms the previous works in terms of accuracy, robustness and real-time criterions. Specifically, the newly developed BboxLocate-Net attaches over 500 fps, almost five times the published state-of-the-art in this field, with comparable localization accuracy.     

基于深度强化学习的多无人机协同进攻作战智能规划

无人机依靠作战效费比高、灵活自主等优势逐步替代了有生力量作战,多无人机协同作战任务规划成为热点研究问题。针对传统任务规划采用的智能优化算法存在的依赖静态、低维的简单场景、机上计算较慢等不足,提出一种基于深度强化学习（DRL）的端到端的多无人机协同进攻智能规划方法。将压制敌防空作战（SEAD）任务规划过程建模为马尔科夫决策过程,建立基于近端策略优化（PPO）算法的SEAD 智能规划模型,通过两组实验验证智能规划模型的有效性和鲁棒性。结果表明：基于DRL 的智能规划方法可以实现快速、精细规划,适应未知、连续高维的环境态势,智能规划模型具有战术协同规划能力。     

基于深度强化学习的无人机栖落机动控制策略设计

无人机栖落机动飞行是一种无需跑道的降落方法，能够提升无人机在复杂环境下执行任务的适应能力。针对具有高非线性、多约束特性的无人机栖落机动过程，提出了一种基于模仿深度强化学习的控制策略设计方法。首先，建立了固定翼无人机栖落机动的纵向非线性动力学模型，并设计了无人机栖落机动的强化学习环境。其次，针对栖落机动状态动作空间大的特点，为了提高探索效率，通过模仿专家经验的方法对系统进行预训练。然后，以模仿学习得到的权重为基础，采用近端策略优化方法学习构建无人机栖落机动的神经网络控制器。最后，通过仿真验证了上述控制策略设计方法的有效性。     

Relevant experience learning: A deep reinforcement learning method for UAV autonomous motion planning in complex unknown environments

Unmanned Aerial Vehicles (UAVs) play a vital role in military warfare. In a variety of battlefield mission scenarios, UAVs are required to safely fly to designated locations without human intervention. Therefore, finding a suitable method to solve the UAV Autonomous Motion Planning (AMP) problem can improve the success rate of UAV missions to a certain extent. In recent years, many studies have used Deep Reinforcement Learning (DRL) methods to address the AMP problem and have achieved good results. From the perspective of sampling, this paper designs a sampling method with double-screening, combines it with the Deep Deterministic Policy Gradient (DDPG) algorithm, and proposes the Relevant Experience Learning-DDPG (REL-DDPG) algorithm. The REL-DDPG algorithm uses a Prioritized Experience Replay (PER) mechanism to break the correlation of continuous experiences in the experience pool, finds the experiences most similar to the current state to learn according to the theory in human education, and expands the influence of the learning process on action selection at the current state. All experiments are applied in a complex unknown simulation environment constructed based on the parameters of a real UAV. The training experiments show that REL-DDPG improves the convergence speed and the convergence result compared to the state-of-the-art DDPG algorithm, while the testing experiments show the applicability of the algorithm and investigate the performance under different parameter conditions.     

带FDI算法的无人机组合导航姿态确定控制系统设计

无人机导航系统的作用是提供导航数据给飞控计算机作为制导和控制用,因为飞控计算机的性能在很大程度上依赖于导航数据,导航系统的某一个错误可能会导致整个无人机的失败,因此,导航系统应具有故障检测和隔离(FDI)算法,介绍了一种用于无人机导航的FDI和低成本的组合导航系统,硬件包括低成本商业用MEMSIMU、GPS接收器、磁力计和导航计算机,软件包括带FDI算法的卡尔曼滤波。     

A guidance law for UAV autonomous aerial refueling based on the iterative computation method

The rendezvous and formation problem is a significant part for the unmanned aerial vehicle（UAV） autonomous aerial refueling（AAR） technique. It can be divided into two major phases： the long-range guidance phase and the formation phase. In this paper, an iterative computation guidance law（ICGL） is proposed to compute a series of state variables to get the solution of a control variable for a UAV conducting rendezvous with a tanker in AAR. The proposed method can make the control variable converge to zero when the tanker and the UAV receiver come to a formation flight eventually. For the long-range guidance phase, the ICGL divides it into two sub-phases： the correction sub-phase and the guidance sub-phase. The two sub-phases share the same iterative process. As for the formation phase, a velocity coordinate system is created by which control accelerations are designed to make the speed of the UAV consistent with that of the tanker.The simulation results demonstrate that the proposed ICGL is effective and robust against wind disturbance.     

地磁辅助的无缝INS/GPS组合导航研究

INS/GPS组合导航已经成为当前无人机导航系统的主要实现形式,由于GPS信号容易受到干扰,在恶劣的电磁环境下信号易丢失,从而导致GPS卫星信号失锁而无法使用。地磁导航作为一种无源导航方法,其难以受到外界干扰且具有较强的自主性,从而为克服GPS在干扰情况下无法对INS误差实现持续无缝修正的不足提供了很好的途径。针对INS/GPS组合导航中GPS卫星信号失锁的情况,设计提出了使用地磁匹配导航进行辅助实现无人机无缝导航的实现方案,设计了基于地磁特征的地磁匹配算法和地磁匹配辅助的INS/GPS组合无缝自主导航算法,并通过仿真验证了采用地磁匹配辅助导航方法,可以在GPS无效的情况下,实现对INS导航误差的持续无缝修正,从而提高导航系统性能。     

包含乘性噪声自适应修正的非合作目标相对导航算法

无人机（UAV）态势感知的任务是利用机载传感器对未知环境进行目标识别和引导，针对无人机与非合作目标间中远距离的相对导航问题，提出了一种基于角度和距离量测的相对状态估计算法。在现有滤波算法的基础上，为了提高精度和稳定性，本文利用了列文伯格-马夸尔特（LM）优化的思想对迭代卡尔曼滤波（IEKF）算法进行改进，提出了一种LM-IEKF算法，并推导该算法在迭代过程中的状态更新方程及协方差阵的递推公式。在此基础上，考虑到距离传感器由于信号相关特性而引入的乘性噪声，现有的加性噪声模型难以适应，因此，进一步提出了基于量测噪声自适应修正的Modified LM-IEKF方法，通过在线实时更新噪声阵提高滤波的精度，并设置渐消记忆指数平滑估计结果。算法验证结果表明，与现有的EKF、IEKF算法相比，在仅含加性噪声的情况下，LM-IEKF算法具有更好的性能；在包含乘性噪声的情况下，Modified LM-IEKF可以有效地估计量测噪声，与目前广泛使用的EKF算法相比，在综合相对位置和相对速度精度上分别提高了10%和23%。     

区域导航无人机空中基准站的多源信息融合容错导航系统研究

作为卫星导航系统的补充和备份,区域导航服务系统近年来得到较大发展。在基于无人机的区域导航服务系统中,无人机自身的定位精度对区域导航服务系统的可靠运行有直接的影响。针对无人机导航传感器及系统的容错和可靠性问题,设计了具有针对性和自优化功能的多源信息融合容错导航方案,提出了一种优化的基于矢量分配形式的自适应联邦滤波算法。通过对每个状态量设计不同的信息分配系数,实现传感器量测噪声的动态优化调整,有效减小了传感器故障对融合导航系统的影响,提高了无人机导航系统的鲁棒性。验证分析表明,该方法可以减小子滤波器故障信息对融合导航系统联邦滤波全局估计的影响,避免了故障子滤波器在信息重置过程中对系统造成的污染,提高和保障了无人机空中基准站多源信息融合导航系统的稳定性和可靠性。     

基于比例导引角度线性化的无人机避障研究

基于无人机与障碍物之间的几何关系,在极坐标系下建立了无人机与障碍物之间的运动学方程,通过引入基于角度线性化的比例导引律,使无人机能够顺利避开障碍物。引入的基于角度线性化的比例导引律是关于无人机相对速度航向角与避障点视线角的函数,由比例函数及偏差函数组成。为解决初始时刻相对速度航向角不满足引入导引律的问题,在调节函数中加入了指数项。通过稳定性证明,得到了满足要求的导引律参数取值范围。最后对设计的避障算法进行了仿真,仿真结果验证了算法的有效性。