边缘网络下多无人机协同计算和资源分配联合优化策略

针对边缘网络环境下多人机之间存在计算负载不均，造成卸载任务失败的问题，提出了一种多无人机间协作的智能任务卸载方案。通过联合考虑多无人机任务分配、计算资源分配和无人机飞行轨迹，引入公平性指数建立了无人机公平负载最大化和能量消耗最小化问题。基于多智能体深度强化学习框架，提出了融合轨迹规划和任务卸载的分布式算法。仿真结果表明，所提出的多无人机协作方案可以显著提高任务完成率和负载公平度，并且有效适用于大规模用户设备场景。     

Locally generalised multi-agent reinforcement learning for demand and capacity balancing with customised neural networks

Reinforcement Learning (RL) techniques are being studied to solve the Demand and Capacity Balancing (DCB) problems to fully exploit their computational performance. A locally generalised Multi-Agent Reinforcement Learning (MARL) for real-world DCB problems is proposed. The proposed method can deploy trained agents directly to unseen scenarios in a specific Air Traffic Flow Management (ATFM) region to quickly obtain a satisfactory solution. In this method, agents of all flights in a scenario form a multi-agent decision-making system based on partial observation. The trained agent with the customised neural network can be deployed directly on the corresponding flight, allowing it to solve the DCB problem jointly. A cooperation coefficient is introduced in the reward function, which is used to adjust the agent’s cooperation preference in a multi-agent system, thereby controlling the distribution of flight delay time allocation. A multi-iteration mechanism is designed for the DCB decision-making framework to deal with problems arising from non-stationarity in MARL and to ensure that all hotspots are eliminated. Experiments based on large-scale high-complexity real-world scenarios are conducted to verify the effectiveness and efficiency of the method. From a statistical point of view, it is proven that the proposed method is generalised within the scope of the flights and sectors of interest, and its optimisation performance outperforms the standard computer-assisted slot allocation and state-of-the-art RL-based DCB methods. The sensitivity analysis preliminarily reveals the effect of the cooperation coefficient on delay time allocation.     

拦截大气层内机动目标的深度强化学习制导律

针对大气层内高速机动目标的拦截问题，提出了一种基于双延迟深度确定性策略梯度(TD3)算法的深度强化学习制导律，它直接将交战状态信息映射为拦截弹的指令加速度，是一种端到端、无模型的制导策略。首先，将攻防双方的交战运动学模型描述为适用于深度强化学习算法的马尔科夫决策过程，之后通过合理地设计算法训练所需的交战场景、动作空间、状态空间和网络结构，并引入奖励函数整形和状态随机初始化，构建了完整的深度强化学习制导算法。仿真结果表明：与比例导引和增强比例导引两种方案相比，深度强化学习制导策略在脱靶量更小的同时能够降低对中制导精度的要求；具有良好的鲁棒性和泛化能力，并且计算负担较小,具备在弹载计算机上运行的条件。     

基于强化学习的航天器姿态控制器设计

航天器在轨执行某些任务时，其质量参数会发生未知变化，传统控制方法在这种情况下控制效果不佳。本文提出基于强化学习的航天器姿态控制器设计方法，该方法在姿态控制器训练过程中不需要对航天器进行动力学建模，不依赖航天器的质量参数。当质量参数发生较大未知变化时，训练好的控制器仍然可以保持较好的控制效果。仿真测试表明:使用基于强化学习方法训练的控制器确实具有良好的鲁棒性。此外，回报函数的设计会明显影响姿态控制器的训练，因此对不同的回报函数设计进行了研究。     

Sequential dynamic resource allocation in multi-beam satellite systems:A learning-based optimization method

Multi-beam antenna and beam hopping technologies are an effective solution for scarce satellite frequency resources. One of the primary challenges accompanying with Multi-Beam Satellites(MBS) is an efficient Dynamic Resource Allocation(DRA) strategy. This paper presents a learning-based Hybrid-Action Deep Q-Network(HADQN) algorithm to address the sequential decision-making optimization problem in DRA. By using a parameterized hybrid action space,HADQN makes it possible to schedule the beam patte...     

航天器反应式碎片规避动作规划方法

提出一种航天器反应式碎片规避动作规划方法，首先以扰动流体动态系统(IFDS)算法作为动作规划的基础算法，通过其中的总和扰动矩阵对航天器的轨道速度矢量进行修正，实现轨道机动规避；然后，建立基于双延迟深度确定性策略梯度(TD3)深度强化学习算法的反应式动作规划方法，通过TD3在线优化IFDS规划参数，实现对碎片群的“状态-动作”最优、快速规避决策。在此基础上，将优先级经验回放和渐进式学习策略引入该方法中，提升训练效率。最后，仿真结果表明，所提方法可使航天器安全规避多发、突发、动态且形状各异的空间碎片群，且具有较好的实时性。     

SiO2气凝胶骨架增强改性研究进展

分析SiO_2气凝胶骨架增强改性的特点,阐述了共前驱体、化学添加剂、老化、表面改性和热处理工艺在SiO_2气凝胶骨架增强方面的国内外研究动态和未来发展趋势,以期为设计并优化SiO_2气凝胶的微结构及性能提供理论依据和方法。     

集群多机器人系统建模研究的发展与展望

凭借高效、鲁棒、应用广泛的特性,集群多机器人系统已经成为当今研究的热点课题之一,具有重要的实用价值。首先简述了自顶而下和自底而上两种多机器人系统研究思路的当前研究概况。然后从拟生物集群系统模型引入,进而引出一致性系统模型,针对低阶、高阶、异质、时延等一致性模型进行分析总结,单独阐述了多智能体强化学习系统模型的研究情况,并分别讨论了三种集群多机器人系统自组织建模方法的研究现状与各自存在的问题,总结与分析了集群多机器人系统运动的发生机理。最后,分析了现有集群多机器人系统模型尚待解决的关键问题和面临的挑战,并对其未来发展进行了展望。