Locally generalised multi-agent reinforcement learning for demand and capacity balancing with customised neural networks

Reinforcement Learning (RL) techniques are being studied to solve the Demand and Capacity Balancing (DCB) problems to fully exploit their computational performance. A locally generalised Multi-Agent Reinforcement Learning (MARL) for real-world DCB problems is proposed. The proposed method can deploy trained agents directly to unseen scenarios in a specific Air Traffic Flow Management (ATFM) region to quickly obtain a satisfactory solution. In this method, agents of all flights in a scenario form a multi-agent decision-making system based on partial observation. The trained agent with the customised neural network can be deployed directly on the corresponding flight, allowing it to solve the DCB problem jointly. A cooperation coefficient is introduced in the reward function, which is used to adjust the agent’s cooperation preference in a multi-agent system, thereby controlling the distribution of flight delay time allocation. A multi-iteration mechanism is designed for the DCB decision-making framework to deal with problems arising from non-stationarity in MARL and to ensure that all hotspots are eliminated. Experiments based on large-scale high-complexity real-world scenarios are conducted to verify the effectiveness and efficiency of the method. From a statistical point of view, it is proven that the proposed method is generalised within the scope of the flights and sectors of interest, and its optimisation performance outperforms the standard computer-assisted slot allocation and state-of-the-art RL-based DCB methods. The sensitivity analysis preliminarily reveals the effect of the cooperation coefficient on delay time allocation.     

基于VBGMM-DCNN的列车卫星定位欺骗干扰检测方法

面向基于全球导航卫星系统的铁路列车定位实施欺骗干扰的主动检测，在卫星定位解算层次，运用深度学习建模学习方法的优势，提出一种基于变分贝叶斯高斯混合模型-深度卷积神经网络（variational Bayesian Gaussian mixture model-deep convolutional neural network, VBGMM-DCNN）的列车卫星定位欺骗干扰检测方法。该方法首先提取能够充分体现欺骗干扰对定位解算过程作用影响的卫星观测特征参数，构建干扰检测特征矢量；然后，采用VBGMM模型拟合经过预处理的特征向量的概率分布，得到二维概率密度图；最后，将概率密度图用于DCNN模型实施欺骗干扰的检测决策。结合现场实验所得运行场景数据，利用实验室搭建的欺骗干扰测试环境实施了干扰注入测试与检验，结果表明,欺骗干扰检测性能随着DCNN网络深度的增加而提升，相对于常规有监督决策方法F1值最高提升44.68%。基于VBGMM-DCNN的欺骗干扰检测能够适应测试验证中运用的列车运行特征及定位观测条件，所达到的检测性能优于对比算法。     

商用飞机持续适航事件经济性分析模型

我国某型民用飞机于2015 年开始陆续交付运行，积累了大量的持续适航事件和数据，但缺少量化的评价机制，用于持续适航事件的经济性分析。本文通过对民用飞机持续适航事件和数据的收集与分析，结合对主制造商和航空公司的成本和收益损失的计算，提出持续适航事件成本结构；在此基础上，建立单个事件的成本模型和持续适航事件经验曲线模型，进一步提出预测机型成熟期的持续适航事件成本预测模型并验证。结果表明：在机队持续适航成本预测模型提出的过程中，飞机停场天数、事件接收到确定措施时间和措施实施时间的取值敏感度高，在后续机队的运行过程中，应尽量减小这三个参数的取值，以降低持续适航事件成本。     

基于Q-Learning的深度神经网络自适应退避策略

针对无人机自组织网络,结合Q-Learning和深度神经网络,提出一种自适应退避策略,以提高基于竞争的MAC协议通信性能.以Matlab为仿真平台,仿真比较了自适应退避策略与二进制指数退避策略的性能.     

Blind identification of space–time block codes based on deep learning

Deep Learning(DL) has important applications to both commercial and military communications, such as software-defined radio, cognitive radio and spectrum surveillance. While DL has been intensively studied for modulation recognition, there are very few investigations for blind identification of Space-Time Block Codes(STBCs). This paper proposes a Residual Network(RN)-based model for identifying 6 kinds of STBC signals with a single receiving antenna, including the same length of coding matrix. I...     

基于深度强化学习的四足机器人后空翻动作生成方法

四足机器人灵巧运动技能的生成一直受到机器人研究者们的广泛关注,其中空中翻滚运动既能展现四足机器人运动的灵活性又具有一定的实用价值.近年来,深度强化学习方法为四足机器人的灵巧运动提供了新的实现思路,利用该方法得到的闭环神经网络控制器具有适应性强、稳定性高等特点.本文在绝影Lite机器人上使用基于模仿专家经验的深度强化学习方法,实现了仿真环境中四足机器人的后空翻动作学习,并进一步证明了设计的后空翻闭环神经网络控制器相比于开环传统位置控制器具有适应性更高的特点.     

Large-scale real-world radio signal recognition with deep learning

In the past ten years, many high-quality datasets have been released to support the rapid development of deep learning in the fields of computer vision, voice, and natural language processing. Nowadays, deep learning has become a key research component of the Sixth-Generation wireless systems (6G) with numerous regulatory and defense applications. In order to facilitate the application of deep learning in radio signal recognition, in this work, a large-scale real-world radio signal dataset is created based on a special aeronautical monitoring system - Automatic Dependent Surveillance-Broadcast (ADS-B). This paper makes two main contributions. First, an automatic data collection and labeling system is designed to capture over-the-air ADS-B signals in the open and real-world scenario without human participation. Through data cleaning and sorting, a high-quality dataset of ADS-B signals is created for radio signal recognition. Second, we conduct an in-depth study on the performance of deep learning models using the new dataset, as well as comparison with a recognition benchmark using machine learning and deep learning methods. Finally, we conclude this paper with a discussion of open problems in this area.     

利用无线通信链路进行基于深度学习的大雾天气监测

为了以低成本、高时空分辨率进行大雾天气监测，提出一种利用无线通信链路进行基于深度学习的大雾天气监测方法。由于信道中不同浓度的大雾天气在信号中留有的特征不同，采集了4种不同浓度大雾下的无线电信号，建立无线电大雾天气监测数据集；通过在传统ResNet50网络中引入注意力机制并进行特征融合，得到改进后的A-ResNet50模型。利用A-ResNet50网络提取接收信号中留有的不同浓度大雾天气的特征，对四类不同浓度大雾天气进行分类识别，达到监测大雾天气的目的。所提方法在建立的数据集上进行了验证，相较于其他传统分类算法，本方法性能最优，最终识别准确率达到86.18 %，结果证明了该方法的可行性和有效性。     

拦截大气层内机动目标的深度强化学习制导律

针对大气层内高速机动目标的拦截问题，提出了一种基于双延迟深度确定性策略梯度(TD3)算法的深度强化学习制导律，它直接将交战状态信息映射为拦截弹的指令加速度，是一种端到端、无模型的制导策略。首先，将攻防双方的交战运动学模型描述为适用于深度强化学习算法的马尔科夫决策过程，之后通过合理地设计算法训练所需的交战场景、动作空间、状态空间和网络结构，并引入奖励函数整形和状态随机初始化，构建了完整的深度强化学习制导算法。仿真结果表明：与比例导引和增强比例导引两种方案相比，深度强化学习制导策略在脱靶量更小的同时能够降低对中制导精度的要求；具有良好的鲁棒性和泛化能力，并且计算负担较小,具备在弹载计算机上运行的条件。     

基于强化学习的航天器姿态控制器设计

航天器在轨执行某些任务时，其质量参数会发生未知变化，传统控制方法在这种情况下控制效果不佳。本文提出基于强化学习的航天器姿态控制器设计方法，该方法在姿态控制器训练过程中不需要对航天器进行动力学建模，不依赖航天器的质量参数。当质量参数发生较大未知变化时，训练好的控制器仍然可以保持较好的控制效果。仿真测试表明:使用基于强化学习方法训练的控制器确实具有良好的鲁棒性。此外，回报函数的设计会明显影响姿态控制器的训练，因此对不同的回报函数设计进行了研究。