基于强化学习的多无人机避碰计算制导方法

针对大量固定翼无人机在有限空域内的协同避碰问题,提出了一种基于多智能体深度强化学习的计算制导方法。首先,将避碰制导过程抽象为序列决策问题,通过马尔可夫博弈理论对其进行数学描述。然后提出了一种基于深度神经网络技术的自主避碰制导决策方法,该网络使用改进的Actor-Critic模型进行训练,设计了实现该方法的机器学习架构,并给出了相关神经网络结构和机间协调机制。最后建立了一个实体数量可变的飞行场景模拟器,在其中进行"集中训练"和"分布执行"。为了验证算法的性能,在高航路密度场景中进行了仿真实验。仿真结果表明,提出的在线计算制导方法能够有效地降低多无人机在飞行过程中的碰撞概率,且对高航路密度场景具有很好的适应性。     

一种基于学习的高超声速飞行器智能控制方法

针对吸气式高超声速飞行器的飞行控制问题，提出一种基于学习的智能控制方法。为便于控制器设计，将飞行器动力学模型划分为速度子系统和高度子系统：为解决速度子系统控制输入受限的问题，提出一种基于强化学习的智能控制方案；对于考虑有限通信资源的高度子系统跟踪控制问题，提出一种基于事件触发的确定学习控制方案。该方案包含离线学习训练和在线触发控制两个阶段。首先在本地离线学习训练阶段获取并存储系统的未知动态知识，随后利用所获取的经验知识设计基于事件触发机制的在线触发控制器。本文所提方案基于学习的思想将离线学习训练获取的智能体和经验知识应用于在线控制，使得所提方案能够快速计算控制指令且通信资源占用少。仿真结果说明了所提出方法的有效性。     

Improving multi-target cooperative tracking guidance for UAV swarms using multi-agent reinforcement learning

Multi-Target Tracking Guidance(MTTG) in unknown environments has great potential values in applications for Unmanned Aerial Vehicle(UAV) swarms. Although Multi-Agent Deep Reinforcement Learning(MADRL) is a promising technique for learning cooperation, most of the existing methods cannot scale well to decentralized UAV swarms due to their computational complexity or global information requirement. This paper proposes a decentralized MADRL method using the maximum reciprocal reward to learn cooper...     

CFRP蜂窝夹层结构板埋件集群区域的热应力失效模式分析及补强设计

航天结构中,蜂窝板常采用在内部镶嵌埋件的方式使其与其他部件连接.为了分析CFRP蜂窝夹层结构板埋件集群区域面板在温度应力作用下失效的原因,进行了失稳破坏应力理论计算和数值分析,由分析结果发现是面板失稳导致了结构破坏.提出了加厚面板和局部加强两种补强方法并分析了其可行性和效率,通过对比发现局部补强的方法更加可行和高效.最...     

基于深度强化学习的复杂地形适应机器人设计与实验

针对行星表面轻量化自主探测任务，基于仿生思想设计了一种仿海胆结构的十二足球形机器人，其具备自主改变构型以贴合复杂地形的能力，可实现无倾覆、高容错的全向运动；基于数据驱动方法，对该机器人设计了一种数据高效的无模型强化学习运动策略，可实现无先验知识的从0到1步态训练以及步态的实物样机快速部署。通过在平面地形和非结构化地形中对其进行仿真实验，验证了经过训练的机器人具备自主运动、适应非结构地形等能力；通过与常用基准策略进行对比，证实了本文提出的运动策略具有训练高效、鲁棒性好的优势；最后通过开发原理样机，开展实物实验验证了仿真环境中所生成的步态在真实物理环境中的动力学可行性。     

一种高速可变形飞行器智能变形决策方法

针对一类高速可变形飞行器(HMFV)的变形决策问题，提出一种基于深度确定性策略算法(DDPG)下考虑综合性能指标最优的智能变形决策方法。首先，以一类后掠角可连续变化的高速飞行器为研究对象，给出变形飞行器动力学模型，分析模型特性及变形量与关键气动参数之间的定性关系。其次，基于关键气动数据特征分析，考虑包含气动性能、控制误差在内的综合性能指标，设计一种基于DDPG算法的智能变形决策方案。再者，针对带有标称控制器的HMFV进行变形决策训练，实时获得滑翔过程中不同飞行状态下的最优构型。最后，仿真结果表明所设计的智能变形决策算法收敛效果好，且具备较好的泛化性能。相比于固定外形，可通过变形使得在不同状态下的升阻比保持最优，且与考虑单一决策指标相比，考虑综合指标最优的变形决策可进一步缩小姿态动态跟踪误差。     

Reinforcement learning based dynamic distributed routing scheme for mega LEO satellite networks

Recently, mega Low Earth Orbit (LEO) Satellite Network (LSN) systems have gained more and more attention due to low latency, broadband communications and global coverage for ground users. One of the primary challenges for LSN systems with inter-satellite links is the routing strategy calculation and maintenance, due to LSN constellation scale and dynamic network topology feature. In order to seek an efficient routing strategy, a Q-learning-based dynamic distributed Routing scheme for LSNs (QRLSN) is proposed in this paper. To achieve low end-to-end delay and low network traffic overhead load in LSNs, QRLSN adopts a multi-objective optimization method to find the optimal next hop for forwarding data packets. Experimental results demonstrate that the proposed scheme can effectively discover the initial routing strategy and provide long-term Quality of Service (QoS) optimization during the routing maintenance process. In addition, comparison results demonstrate that QRLSN is superior to the virtual-topology-based shortest path routing algorithm.     

A policy iteration method for improving robot assembly trajectory efficiency

Bolt assembly by robots is a vital and difficult task for replacing astronauts in extra-vehicular activities (EVA), but the trajectory efficiency still needs to be improved during the wrench insertion into hex hole of bolt. In this paper, a policy iteration method based on reinforcement learning (RL) is proposed, by which the problem of trajectory efficiency improvement is constructed as an issue of RL-based objective optimization. Firstly, the projection relation between raw data and state-action space is established, and then a policy iteration initialization method is designed based on the projection to provide the initialization policy for iteration. Policy iteration based on the protective policy is applied to continuously evaluating and optimizing the action-value function of all state-action pairs till the convergence is obtained. To verify the feasibility and effectiveness of the proposed method, a noncontact demonstration experiment with human supervision is performed. Experimental results show that the initialization policy and the generated policy can be obtained by the policy iteration method in a limited number of demonstrations. A comparison between the experiments with two different assembly tolerances shows that the convergent generated policy possesses higher trajectory efficiency than the conservative one. In addition, this method can ensure safety during the training process and improve utilization efficiency of demonstration data.