Locally generalised multi-agent reinforcement learning for demand and capacity balancing with customised neural networks

Reinforcement Learning (RL) techniques are being studied to solve the Demand and Capacity Balancing (DCB) problems to fully exploit their computational performance. A locally generalised Multi-Agent Reinforcement Learning (MARL) for real-world DCB problems is proposed. The proposed method can deploy trained agents directly to unseen scenarios in a specific Air Traffic Flow Management (ATFM) region to quickly obtain a satisfactory solution. In this method, agents of all flights in a scenario form a multi-agent decision-making system based on partial observation. The trained agent with the customised neural network can be deployed directly on the corresponding flight, allowing it to solve the DCB problem jointly. A cooperation coefficient is introduced in the reward function, which is used to adjust the agent’s cooperation preference in a multi-agent system, thereby controlling the distribution of flight delay time allocation. A multi-iteration mechanism is designed for the DCB decision-making framework to deal with problems arising from non-stationarity in MARL and to ensure that all hotspots are eliminated. Experiments based on large-scale high-complexity real-world scenarios are conducted to verify the effectiveness and efficiency of the method. From a statistical point of view, it is proven that the proposed method is generalised within the scope of the flights and sectors of interest, and its optimisation performance outperforms the standard computer-assisted slot allocation and state-of-the-art RL-based DCB methods. The sensitivity analysis preliminarily reveals the effect of the cooperation coefficient on delay time allocation.     

高校贫困生资格认定的实证分析

我国现阶段构建高校贫困生资助体系时,应采取与市场经济中"效益优先,兼顾公平"的指导思想,做到"教育公平优先".由于各方的强烈呼吁,国家从2007年下半年开始加大了对贫困学生的资助力度,而具体落实到实际操作方法时,以达到资助公平为目标,其前提是必须准确的界定贫困生身份,使真正的贫困生获得必要的资助,以最大限度的实现教育公平.     

基于极限学习机的修正当前统计模型跟踪算法

针对当前统计(Current Statistical,CS)模型由于先验知识的缺乏和其结构特点导致的跟踪精度降低的问题,通过使用极限学习机(Extreme Learning Machine,ELM)并根据目标历史状态信息,实时估计并修正CS模型的加速度估计值,提出了基于ELM的修正CS模型跟踪算法.ELM的训练结果表明...     

基于学习的图像智能适配显示技术

为适应不同的显示分辨率,出现了各式各样的图像适配显示(IR)的方法.提出了基于图像列的一种快速适配显示方法.在处理过程中,首先,计算一个原始图像的重要性图;其次,根据图像每列的重要性程度为其分配一个比例因子,对不同图像而言,应对比例因子设置不同的上限才可以得到较好的结果;最后,提出通过机器学习方法计算出不同图像的上限,从而可以高效率地得到理想的结果.根据每一列的比例因子采用像素融合的方式处理图片得到目标分辨率.本方法是基于列实现的,其复杂度低、便于计算;设置每列系数的上限控制了图像重要部分的宽度,从而减少了不连贯,处理结果更为自然.      

柔性铰接板振动测量及迭代学习主动控制

大型柔性板状结构广泛应用于航空航天领域,这种结构特性极易引起低频振动,针对这一问题以柔性铰接板为研究对象,探究了柔性铰接板在受外部激励作用下弯曲模态的振动特性.为此搭建了一套用于测控柔性铰接板振动特性的实验平台,采用了立体视觉和压电陶瓷传感器分别检测柔性铰接板的振动信息,以此信息作反馈,采用了变遗忘因子的迭代学习控制方法对柔性铰接板的振动进行主动控制,同时以PD控制的实验效果作为对比.实验结果表明,不论是大幅值的快速衰减还是小幅值的残余振动抑制,迭代学习控制效果都优于PD控制.     

Modeling M(3000)F2 based on extreme learning machine

This paper presents an novel extreme learning machine (ELM)-based prediction model for the ionospheric propagation factor M(3000)F2 at Darwin station (12.4°S, 131.5°E; −44.5°dip) in Australia. The proposed ELM model is trained with hourly daily values of M(3000)F2 from the period 1998–2014 except 2001 and 2009. The hourly daily values of 2001 (high solar activity) and 2009 (low solar activity) are used for validating the prediction accuracy. The proposed ELM for modeling M(3000)F2 can achieve faster training process and similar testing accuracy compared with backward propagation neural network (BPNN). In addition, the performance of the ELM is verified by comparing the predicted values of M(3000)F2 with observed values and the international reference ionosphere (IRI −2016) model predicted values. Based on the error differences (the root mean square error (RMSE) and the M(3000)F2 percentage improvement values M(3000)F2_IMP(%)), the result demonstrates the effectiveness of the ELM model compared with the IRI-2016 model at hourly, daily, monthly, and yearly in high (2001) and low (2009) solar activity years. The ELM also shows good agreement with observations compared with the IRI during disturbed magnetic activity.     

Chicks incubated in hypomagnetic field need more exogenous noradrenaline for memory consolidation

The geomagnetic field (GMF) is one of the essential characteristics of the terrestrial environment but does not apply in outer space. The elimination of GMF may interfere with the normal activities of life in many aspects. Previous behavioral experiments have found that long-term memory is impaired in chicks incubated in a near-zero magnetic environment (i.e. hypomagnetic field or HMF). The present study was designed to evaluate the possible involvement of noradrenergic change in the functional abnormality observed before. A HMF space was produced by nullifying the natural GMF with three pairs of Helmholtz coils. The one-trial passive avoidance learning paradigm was performed on day-old chicks incubated in either the HMF space or the natural GMF. Exogenous noradrenaline was administered by intracerebral injections and the effect on memory consolidation was compared between the two categories of subjects. In the behavioral paradigm, the HMF chicks had a higher elimination rate than the GMF chicks and displayed a significant reduction in overall responsiveness. The administration of moderate doses (0.1–0.5 nmol/hemisphere) of noradrenaline led to fairly good memory retention in GMF chicks but had little effect on HMF chicks. However, long-term memory of HMF chicks could be elevated to the normal level by much higher doses (1.0–1.75 nmol/hem) of the drug. These results suggest that prolonged exposure to HMF may induce disorders in the noradrenergic system in the brain and indicate a potentiality of counteracting the ill-effect of GMF deprivation with appropriate pharmacological manipulation.     

神经网络理论在微分对策中的应用

阐述了微分对策理论在实际应用中存在的问题,并从动态冲突的实际过程出发,将神经网络理论与分层式智能控制理论相结合,来模型化微分对策问题.将微分对策的双向优化问题转化成基于神经网络理论的目标机动性辨识和单向优化控制问题,将微分对策的定性与定量结合起来,为有效加入人的经验提供了一种技术途径.并采用自适应分布式学习速率,大大提高了神经网络的学习速度和精度.     

Multi-agent Q-Learning control of spacecraft formation flying reconfiguration trajectories

This paper presents a novel approach based on multi-agent reinforcement learning for spacecraft formation flying reconfiguration tracking problems. In this scheme, spacecrafts learn the control strategy via transfer learning. For this matter, a new generalized discounted value function is introduced for the tracking problems. Due to the digital nature of spacecraft computer systems, local optimal controllers are developed for the spacecrafts in discrete-time. The stability of the controller is proven. Two Q-learning algorithms are proposed, in each of which the optimal control solution is learned on-line without knowledge about the system dynamics. In the first algorithm, each agent learns the optimal control independently. In the second one, each agent shares the learned information with other agents. Next, the collision avoidance capability is provided. The effectiveness of the presented schemes is verified through simulations and compared with each other.     

基于分布式深度学习的多星计算卸载策略

在边缘计算增强的低轨卫星网络场景下,低轨卫星集群协同处理地面任务能有效降低用户响应时延。对卫星集群的联合卸载决策和资源分配优化问题进行研究,将其描述为一个混合整数规划问题,并采用了一种基于分布式深度学习算法的卫星边缘计算卸载算法（deep learning based offloading algorithm,DLOA）。该算法使用多个并行DNN用于生成卸载决策并采用经验回放存储新生成的卸载决策,当采用隐藏层结构不同的DNN,收敛速度比同构DNN提升18%,收敛值与最优值的比值基本为1,可以认为已收敛至最优。此外,探讨了DNN的数量对所使用的算法的影响,仿真结果表明采用少量DNN就可以获得近优的收敛效果。通过对不同任务规模下采用不同算法的任务完成率进行研究,结果表明DLOA算法可通过采用异构DNN和优化资源分配方案显著提升完成率,其较单星运算方案任务完成率提升1倍,较二进制粒子群算法方案提升20%。