Relevant experience learning: A deep reinforcement learning method for UAV autonomous motion planning in complex unknown environments

Unmanned Aerial Vehicles (UAVs) play a vital role in military warfare. In a variety of battlefield mission scenarios, UAVs are required to safely fly to designated locations without human intervention. Therefore, finding a suitable method to solve the UAV Autonomous Motion Planning (AMP) problem can improve the success rate of UAV missions to a certain extent. In recent years, many studies have used Deep Reinforcement Learning (DRL) methods to address the AMP problem and have achieved good results. From the perspective of sampling, this paper designs a sampling method with double-screening, combines it with the Deep Deterministic Policy Gradient (DDPG) algorithm, and proposes the Relevant Experience Learning-DDPG (REL-DDPG) algorithm. The REL-DDPG algorithm uses a Prioritized Experience Replay (PER) mechanism to break the correlation of continuous experiences in the experience pool, finds the experiences most similar to the current state to learn according to the theory in human education, and expands the influence of the learning process on action selection at the current state. All experiments are applied in a complex unknown simulation environment constructed based on the parameters of a real UAV. The training experiments show that REL-DDPG improves the convergence speed and the convergence result compared to the state-of-the-art DDPG algorithm, while the testing experiments show the applicability of the algorithm and investigate the performance under different parameter conditions.     

Towards intelligent design optimization: Progress and challenge of design optimization theories and technologies for plastic forming

Plastic forming is one of enabling and fundamental technologies in advanced manufacturing chains. Design optimization is a critical way to improve the performance of the forming system, exploit the advantages of high productivity, high product quality, low production cost and short time to market and develop precise, accurate, green, and intelligent (smart) plastic forming technology. However, plastic forming is quite complicated, relating to multi-physics field coupling, multi-factor influence, multi-defect constraint, and triple nonlinear, etc., and the design optimization for plastic forming involves multi-objective, multi-parameter, multi-constraint, nonlinear, high-dimensionality, non-continuity, time-varying, and uncertainty, etc. Therefore, how to achieve accurate and efficient design optimization of products, equipment, tools/dies, and processing as well as materials characterization has always been the research frontier and focus in the field of engineering and manufacturing. In recent years, with the rapid development of computing science, data science and internet of things (IoT), the theories and technologies of design optimization have attracted more and more attention, and developed rapidly in forming process. Accordingly, this paper first introduced the framework of design optimization for plastic forming. Then, focusing on the key problems of design optimization, such as numerical model and optimization algorithm, this paper summarized the research progress on the development and application of the theories and technologies about design optimization in forming process, including deterministic and uncertain optimization. Moreover, the applicability of various modeling methods and optimization algorithms was elaborated in solving the design optimization problems of plastic forming. Finally, considering the development trends of forming technology, this paper discusses some challenges of design optimization that may need to be solved and faced in forming process.     

Computation of unsteady turbomachinery flows: Part 1—Progress and challenges

There are numerous unsteady flow influences in turbomachinery. These can potentially make a substantial total impact on efficiency, and hence the environment and operating costs over the life of a gas turbine engine. These unsteadiness sources are reviewed. Also, the turbomachinery zones where unsteady modeling is mandatory for meaningful solutions is outlined. The various unsteady modeling hierarchies are reviewed. These range from linear harmonic to Direct Numerical Simulations (DNS). Unsteady reduced order modeling encompassing deterministic stresses and body forces are reviewed. Hierarchies are presented for different modeling lineages and fidelity levels. Mixed fidelity methods are proposed, where low and high fidelity treatments are combined. For example, Large Eddy Simulation (LES) and Unsteady Reynolds Averaged Simulations (URANS) being combined with body forces to provide appropriate system boundary conditions.A daunting array of modeling and numerical methods and strategies are found for the user to select. Each has their own theoretical limitations. Clearly a user must be aware of these. Reported performances of the different approaches are found to vary considerably between relatively similar applications. The reviewed work suggests that Computational Fluid Dynamics (CFD), as ever, is an activity that needs strong reviewing of processes, tools and overseeing of modeling practices. With regard to LES, grid densities used for typical complex geometry simulations currently appear to be too coarse. This reflects the lack of current computational performance and hence the need for reduced order models.     

一种基于学习的高超声速飞行器智能控制方法

针对吸气式高超声速飞行器的飞行控制问题,提出一种基于学习的智能控制方法。为便于控制器设计,将飞行器动力学模型划分为速度子系统和高度子系统：为解决速度子系统控制输入受限的问题,提出一种基于强化学习的智能控制方案;对于考虑有限通信资源的高度子系统跟踪控制问题,提出一种基于事件触发的确定学习控制方案。该方案包含离线学习训练和在线触发控制两个阶段。首先在本地离线学习训练阶段获取并存储系统的未知动态知识,随后利用所获取的经验知识设计基于事件触发机制的在线触发控制器。本文所提方案基于学习的思想将离线学习训练获取的智能体和经验知识应用于在线控制,使得所提方案能够快速计算控制指令且通信资源占用少。仿真结果说明了所提出方法的有效性。     

基于态势评估及DDPG算法的一对一空战格斗控制方法

已有的空中格斗控制方法未综合考虑基于专家知识的态势评估及通过连续性速度变化控制空战格斗的问题。基于深度确定性策略梯度（DDPG）强化学习算法,在态势评估函数作为强化学习奖励函数的基础上,设计综合考虑飞行高度上下限、飞行过载以及飞行速度上下限的强化学习环境;通过全连接的载机速度控制网络与环境奖励网络,实现DDPG算法与学习环境的交互,并根据高度与速度异常、被导弹锁定时间以及格斗时间设计空战格斗结束条件;通过模拟一对一空战格斗,对该格斗控制方法在环境限制学习、态势评估得分以及格斗模式学习进行验证。结果表明：本文提出的空战格斗控制方法有效,能够为自主空战格斗进一步发展提供指导。     

跨传感器异步迁移学习的室内单目无人机避障

针对强化学习策略由仿真环境向实际迁移困难的问题,以提高无人机采用无深度信息单目视觉时的行人规避能力为目标,提出一种基于异步深度神经网络结构的跨传感器迁移学习方法。首先,在仿真环境中仅使用虚拟单线激光雷达作为传感器,通过基于确定性策略梯度(DDPG)的深度强化学习方法,训练得到一个稳定的初级避障策略。其次,用单目摄像头和激光雷达同步采集现实环境中的视觉和深度数据集并逐帧绑定,使用上述初级避障策略对现实数据集进行自动标注,进而训练得到无需激光雷达数据的单目视觉避障策略,实现从虚拟激光雷达到现实单目视觉的跨传感器迁移学习。最后,引入YOLO v3-tiny网络与Resnet18网络组成异步深度神经网络结构,有效提高了存在行人场景下的避障性能。     

组合压气机中尾迹与势流同频/异频干涉的叠加特性

为揭示多级压气机中上下游叶轮对中间叶片叠加气动影响特性,阐述不同叠加干涉情况下下游叶轮进气角度变化,采用数值方法模拟了一级轴流和一级离心组成的组合压气机非定常流场.详细讨论了上游动叶尾迹和下游动叶势流对中间导流叶栅段气流非定常流动的异频和同频叠加干涉特性,依据计算结果,直观地展示了静叶通道中两种干涉间相互激励和抑制作用的位置和时间,与数学公式的推导结果相互印证.研究结果表明:当上下游动叶对中间静叶段异频干涉时,干涉的激励、抑制区域的轴向位置随时间发生变化;当上下游动叶对中间静叶干涉频率相同时,干涉的相互激励、抑制区域的轴向位置不随时间发生变化,但干涉的激励、抑制区域的轴向位置受时序位置影响.另外,上游动叶尾迹与下游离心叶轮势流的不同叠加情况,决定着下游离心叶轮进口相对气流角的大小及波动幅值.     

航天器可应用实时以太网分析

在航天器上应用以太网的目的是借助以太网的灵活性, 获得方便 的通信接入及数据传输的高带宽, 并且能适应控制领域通信的实 时性和确定性. 实时以太网的确定性、通信调度机制和传输特征是其满足航天器网络通信特性需求的主要判定依据. 通过分析航天器数据通信特点, 对比实时以太网通信调度策略, 得出应用于航天器的实时以太网应具备时间同步、强实时性、确定性、高带宽、多数据类型、双通道冗余及兼容标准以太网等特征.      

空间机械臂的确定性学习控制

研究一类空间机械臂系统的基于模式的控制方法。首先在训练阶段,基于确定性学习理论设计自适应神经网络控制器使机械臂系统跟踪不同的任务模式,得到对应于不同任务模式的一系列空间机械臂闭环动态的局部准确神经网络建模,并利用这些模型构造对应不同任务模式的常值神经网络控制器。其次,在测试阶段,首先快速识别出任务模式,然后调用相应的常值神经网络控制器实现对空间机械臂系统基于模式的闭环控制。理论证明基于模式的控制方法可提高机械臂闭环系统的控制性能,并可避免频繁切换。理论结果最后在空间机械臂中得到了仿真校验。     

高超声速飞行器气动/隐身优化设计方法

针对高超声速飞行器气动布局设计中气动设计与隐身设计矛盾的问题,采用高精度气动和隐身计算方法,建立了基于直接全局优化算法、二次曲线参数化方法和Kriging代理模型的多学科优化设计平台,并对典型高超声速布局升力体外形开展气动/隐身一体化优化设计研究。结果表明：升力体布局典型状态下升阻比由3.13提高到3.69,考虑垂直极化和水平极化状态,俯仰±30°的雷达散热截面（RCS）均值下降60%以上,表明该平台具有良好的寻优能力,风洞试验结果验证了优化算法的可行性;高超声速飞行器的机身和翼/舵等部件具有显著的绕射特性,物理光学法等高频算法不能准确捕捉前后缘绕射,应当采用矩量法计算其RCS特性;高超声速飞行器的垂直极化和水平极化的RCS特性差异巨大,在设计中应当予以考虑。