Energy-optimal trajectory planning for solar-powered aircraft using soft actor-critic

High-Altitude Long-Endurance (HALE) solar-powered Unmanned Aircraft Vehicles (UAVs) can utilize solar energy as power source and maintain extremely long cruise endurance, which has attracted extensive attentions from researchers. Trajectory optimization is a promising way to achieve superior flight time because of the finite solar energy absorbed in a day. In this work, a method of trajectory optimization and guidance for HALE solar-powered aircraft based on a Reinforcement Learning (RL) framework is introduced. According to flight and environment information, a neural network controller outputs commands of thrust, attack angle, and bank angle to realize an autonomous flight based on energy maximization. The validity of the proposed method was evaluated in a 5-km radius area in simulation, and results have shown that after one day-night cycle, the battery energy of the RL-controller was improved by 31% and 17% compared with those of a Steady-State (SS) strategy with a constant speed and a constant altitude and a kind of state-machine strategy, respectively. In addition, results of an uninterrupted flight test have shown that the endurance of the RL controller was longer than those of the control cases.     

基于深度学习光流法的荧光油膜全局速度测量

针对基于先验的传统光流法存在前提条件苛刻的问题，提出使用基于深度学习的光流法进行荧光油膜全局速度测量。采用数值仿真试验对基于先验的改进HS光流法和基于深度学习的FlowNet2光流法进行对比，结果显示：在不外加干扰时，改进HS光流法和FlowNet2光流法的平均端点误差分别为0.458 7像素/s和0.381 7像素/s；在亮度变化、噪声干扰或不同的演化时间下，FlowNet2光流法的平均端点误差均明显低于改进HS光流法，平均端点误差差值最大可达5.19像素/s；风洞试验进一步证明，FlowNet2光流法能够获得正确、清晰、定量的荧光油膜全局速度场，较改进HS光流法鲁棒性更高，对风洞工程应用具有一定的参考价值。     

基于深度强化学习的四足机器人后空翻动作生成方法

四足机器人灵巧运动技能的生成一直受到机器人研究者们的广泛关注,其中空中翻滚运动既能展现四足机器人运动的灵活性又具有一定的实用价值.近年来,深度强化学习方法为四足机器人的灵巧运动提供了新的实现思路,利用该方法得到的闭环神经网络控制器具有适应性强、稳定性高等特点.本文在绝影Lite机器人上使用基于模仿专家经验的深度强化学习方法,实现了仿真环境中四足机器人的后空翻动作学习,并进一步证明了设计的后空翻闭环神经网络控制器相比于开环传统位置控制器具有适应性更高的特点.     

迭代学习方法在PID控制器参数自动调节中的应用及收敛性分析

针对PID控制器参数在具有不确定项或输出干扰项的非线性时变系统中的调节困难,提出了应用迭代学习方法整定PID控制器参数的方法,利用系统的跟踪误差信息自动整定控制器参数,通过理论分析得到此设计方法收敛的充分条件。此方法设计简单且具有普遍性,仿真结果验证了此设计方法的有效性。     

边缘网络下多无人机协同计算和资源分配联合优化策略

针对边缘网络环境下多人机之间存在计算负载不均，造成卸载任务失败的问题，提出了一种多无人机间协作的智能任务卸载方案。通过联合考虑多无人机任务分配、计算资源分配和无人机飞行轨迹，引入公平性指数建立了无人机公平负载最大化和能量消耗最小化问题。基于多智能体深度强化学习框架，提出了融合轨迹规划和任务卸载的分布式算法。仿真结果表明，所提出的多无人机协作方案可以显著提高任务完成率和负载公平度，并且有效适用于大规模用户设备场景。     

Multi-agent Q-Learning control of spacecraft formation flying reconfiguration trajectories

This paper presents a novel approach based on multi-agent reinforcement learning for spacecraft formation flying reconfiguration tracking problems. In this scheme, spacecrafts learn the control strategy via transfer learning. For this matter, a new generalized discounted value function is introduced for the tracking problems. Due to the digital nature of spacecraft computer systems, local optimal controllers are developed for the spacecrafts in discrete-time. The stability of the controller is proven. Two Q-learning algorithms are proposed, in each of which the optimal control solution is learned on-line without knowledge about the system dynamics. In the first algorithm, each agent learns the optimal control independently. In the second one, each agent shares the learned information with other agents. Next, the collision avoidance capability is provided. The effectiveness of the presented schemes is verified through simulations and compared with each other.     

基于LSTM-ResNet模型的时变结构损伤检测

面向火箭结构健康监测，提出了一种基于深度学习的损伤检测方法，直接将多个通道的振动数据作为输入，并基于由长短时记忆网络LSTM（Long Short-Term Memory Networks）和残差卷积神经网络ResNet（Residual Convolutional Neural Networks）组合而成的LSTM-ResNet网络进行损伤识别。其优点在于，首先利用LSTM提取信号的时间依赖特征，减轻了由某些通道信号缺失带来的影响，再利用ResNet在不损耗特征的情况下进一步提取空间特征，提高了训练效率和损伤辨识准确性。通过充液圆筒振动放水实验模拟火箭飞行状态下的燃料消耗，并基于自主构建的数据集和公用数据集对LSTM-ResNet、LSTM、ResNet以及ResNet-LSTM网络进行了训练，训练结果表明，LSTM-ResNet组合网络无论在传感器是否存在故障的情况下都具有更好的性能，损伤检测精度更高。     

Sea fog detection based on unsupervised domain adaptation

Sea fog detection with remote sensing images is a challenging task. Driven by the different image characteristics between fog and other types of clouds, such as textures and colors, it can be achieved by using image processing methods. Currently, most of the available methods are datadriven and relying on manual annotations. However, because few meteorological observations and buoys over the sea can be realized, obtaining visibility information to help the annotations is difficult. Considering t...     

抽油机故障诊断的分布驱动主动学习算法

抽油机示功图直观显示了抽油机工作情况，但实际工况情况呈现典型的长尾分布特性，类别严重不平衡。传统方法无法准确识别小类别工况，也无法获得井下工作状态准确识别。针对这一问题，提出一种基于分布驱动的多类别长尾数据代价敏感主动学习算法（Cost-sensitive active learning algorithm based on distribution -driven multi-class long-tailed data， CALA）。首先，考虑数据分布特性，以最小化代价为优化目标确定数据的最佳聚类簇数；其次，通过加入预分类误差代价来更新之前得到的最佳聚类簇数；然后，构建集成分类模型作为分类器；最后，通过迭代来平衡数据分布。采用某油田真实的示功图数据进行测试，显著性实验分析证明CALA在小类别工况诊断上具有更好的性能。     

Locally generalised multi-agent reinforcement learning for demand and capacity balancing with customised neural networks

Reinforcement Learning (RL) techniques are being studied to solve the Demand and Capacity Balancing (DCB) problems to fully exploit their computational performance. A locally generalised Multi-Agent Reinforcement Learning (MARL) for real-world DCB problems is proposed. The proposed method can deploy trained agents directly to unseen scenarios in a specific Air Traffic Flow Management (ATFM) region to quickly obtain a satisfactory solution. In this method, agents of all flights in a scenario form a multi-agent decision-making system based on partial observation. The trained agent with the customised neural network can be deployed directly on the corresponding flight, allowing it to solve the DCB problem jointly. A cooperation coefficient is introduced in the reward function, which is used to adjust the agent’s cooperation preference in a multi-agent system, thereby controlling the distribution of flight delay time allocation. A multi-iteration mechanism is designed for the DCB decision-making framework to deal with problems arising from non-stationarity in MARL and to ensure that all hotspots are eliminated. Experiments based on large-scale high-complexity real-world scenarios are conducted to verify the effectiveness and efficiency of the method. From a statistical point of view, it is proven that the proposed method is generalised within the scope of the flights and sectors of interest, and its optimisation performance outperforms the standard computer-assisted slot allocation and state-of-the-art RL-based DCB methods. The sensitivity analysis preliminarily reveals the effect of the cooperation coefficient on delay time allocation.