二次奖罚学习自动机

研究了奖罚型学习自动机的一种非线性强化算法。与线性的奖罚模型（ＬＲＰ）不同，新模型的行动选择概率的更新函数为二次的。这使得该模型的学习性能优于ＬＲＰ，且对不同的环境，其具有不同的行为和特点。     

Propulsionless planar phasing of multiple satellites using deep reinforcement learning

This work creates a framework for solving highly non-linear satellite formation control problems by using model-free policy optimisation deep reinforcement learning (DRL) methods. This work considers, believed to be for the first time, DRL methods, such as advantage actor-critic method (A2C) and proximal policy optimisation (PPO), to solve the example satellite formation problem of propellantless planar phasing of multiple satellites. Three degree-of-freedom simulations, including a novel surrogate propagation model, are used to train the deep reinforcement learning agents. During training, the agents actuated their motion through cross-sectional area changes which altered the environmental accelerations acting on them. The DRL framework designed in this work successfully coordinated three spacecraft to achieve a propellantless planar phasing manoeuvre. This work has created a DRL framework that can be used to solve complex satellite formation flying problems, such as planar phasing of multiple satellites and in doing so provides key insights into achieving optimal and robust formation control using reinforcement learning.     

基于强化学习的航空器机场智能静态路径规划

随着人工智能迅速发展以及“智慧机场”的提出,研究人工智能在机场如何有效地辅助机场管制人员,驾驶员指挥航空器在地面滑行具有重要意义。本文提出一种基于强化学习的滑行路径规划方法,构建航空器机场地面强化学习移动模型,并以海口美兰机场为案例采用 Python 内置工具包 Tkinter 进行场面仿真;在此基础上,考虑机场航空器滑行规则,采用 Off-Policy 中 Q-Learning 算法求解贝尔曼方程,实现航空器在 Model-based 环境中进行静态路径规划。结果表明：本文所提方法能够实现停机位到跑道出口智能静态路径规划     

Relevant experience learning: A deep reinforcement learning method for UAV autonomous motion planning in complex unknown environments

Unmanned Aerial Vehicles (UAVs) play a vital role in military warfare. In a variety of battlefield mission scenarios, UAVs are required to safely fly to designated locations without human intervention. Therefore, finding a suitable method to solve the UAV Autonomous Motion Planning (AMP) problem can improve the success rate of UAV missions to a certain extent. In recent years, many studies have used Deep Reinforcement Learning (DRL) methods to address the AMP problem and have achieved good results. From the perspective of sampling, this paper designs a sampling method with double-screening, combines it with the Deep Deterministic Policy Gradient (DDPG) algorithm, and proposes the Relevant Experience Learning-DDPG (REL-DDPG) algorithm. The REL-DDPG algorithm uses a Prioritized Experience Replay (PER) mechanism to break the correlation of continuous experiences in the experience pool, finds the experiences most similar to the current state to learn according to the theory in human education, and expands the influence of the learning process on action selection at the current state. All experiments are applied in a complex unknown simulation environment constructed based on the parameters of a real UAV. The training experiments show that REL-DDPG improves the convergence speed and the convergence result compared to the state-of-the-art DDPG algorithm, while the testing experiments show the applicability of the algorithm and investigate the performance under different parameter conditions.     

纳米隔热材料热辐射特性的理论计算

高温下纳米隔热材料内热辐射的影响将显著增强,其热辐射特性对热辐射传热有很大影响。为了认识高温纳米隔热材料的热辐射特性,采用Mie理论建立了掺杂纤维增韧剂和遮光剂的纳米隔热材料热辐射特性理论计算方法,编写了纳米隔热材料热辐射特性计算程序,对某纳米隔热材料的热辐射特性进行了理论研究,得到了光谱衰减系数、光谱散射反照率以及全光谱平均辐射特性参数及散射相函数。理论模拟结果表明:在3～9.5μm波长范围内,纳米隔热材料对热辐射具有强烈的衰减作用,对3～7μm的热辐射呈现强烈的散射特征,对7～9.5μm的热辐射,随波长增大散射特征逐渐减弱,对9.5μm的热辐射呈现较强的吸收特征。在300～1 300 K,该纳米隔热材料全光谱平均衰减系数>5×104 m-1,平均散射反照率>0.96,具有较强的前向散射特征,这些特征来源于遮光剂粒子,增韧剂影响很小。     

高性能与高功能纤维的发展

概述了用于复合材料增强体的高性能纤维和具有各种物理与化学功能用途的高功能纤维的进展现状并推测其发展趋势。     

粘钢加固在增强结构强度中的应用

本文结合工程实例介绍了采用粘钢加固法来增强结构强度的设计与施工 ,初步探讨了结构增层改建中采用粘钢加固的计算方法、施工工艺及取得的实际效果。     

基于Sarsa(λ)强化学习的空间机械臂路径规划研究

针对目标特性未知的在轨操作环境，研究了典型空间操作机械臂的路径规划策略。采用Sarsa(λ)强化学习方法实现目标跟踪及避障的自主路径规划与智能决策，该方法将机械臂系统的每节臂视为一个决策智能体，通过感知由目标偏差和障碍距离程度组成的二维状态，设计符合人工经验的拟合奖赏函数，进行各臂转动动作的强化训练，最终形成各智能体的状态-动作值函数表，即可作为机械臂在线路径规划的决策依据。将本方法应用于多自由度空间机械臂路径规划任务，仿真结果表明新算法能在有限训练次数内实现对移动目标的稳定跟踪与避障，同时各智能体通过学习所得的状态-动作值函数表，具备较强的后期在线自主调整能力，从而验证了算法较强的鲁棒性和智能性。     

强夯法加固湿陷性黄土地基实例分析

通过陕西渭南开发区某建筑地基的加固实例，对使用强夯法消除黄土湿陷性，提高黄土地基强度的施工技术与质检方法进行了分析研究，得出了在湿陷性黄土地区含水量适中，地下水位较深时采用强夯法效果显著，对实际施工具有一定的指导意义。     

UAV navigation in high dynamic environments: A deep reinforcement learning approach

Unmanned Aerial Vehicle (UAV) navigation is aimed at guiding a UAV to the desired destinations along a collision-free and efficient path without human interventions, and it plays a crucial role in autonomous missions in harsh environments. The recently emerging Deep Reinforcement Learning (DRL) methods have shown promise for addressing the UAV navigation problem, but most of these methods cannot converge due to the massive amounts of interactive data when a UAV is navigating in high dynamic environments, where there are numerous obstacles moving fast. In this work, we propose an improved DRL-based method to tackle these fundamental limitations. To be specific, we develop a distributed DRL framework to decompose the UAV navigation task into two simpler sub-tasks, each of which is solved through the designed Long Short-Term Memory (LSTM) based DRL network by using only part of the interactive data. Furthermore, a clipped DRL loss function is proposed to closely stack the two sub-solutions into one integral for the UAV navigation problem. Extensive simulation results are provided to corroborate the superiority of the proposed method in terms of the convergence and effectiveness compared with those of the state-of-the-art DRL methods.