分布式卫星系统自主运行体系结构模型研究

在分析现有分布式卫星系统(DSS)体系结构不足的基础之上,针对DSS自主运行特征,提出了一种适应动态、复杂不确定环境的DSS两级系统层次的混合体系结构模型.同时根据多星协作和星内自治的需求,给出一种较为通用的面向自主协作的卫星控制结构Agent模型,并讨论了该结构中各Agent模块的组成、功能及相互联系.      

Multi-agent Q-Learning control of spacecraft formation flying reconfiguration trajectories

This paper presents a novel approach based on multi-agent reinforcement learning for spacecraft formation flying reconfiguration tracking problems. In this scheme, spacecrafts learn the control strategy via transfer learning. For this matter, a new generalized discounted value function is introduced for the tracking problems. Due to the digital nature of spacecraft computer systems, local optimal controllers are developed for the spacecrafts in discrete-time. The stability of the controller is proven. Two Q-learning algorithms are proposed, in each of which the optimal control solution is learned on-line without knowledge about the system dynamics. In the first algorithm, each agent learns the optimal control independently. In the second one, each agent shares the learned information with other agents. Next, the collision avoidance capability is provided. The effectiveness of the presented schemes is verified through simulations and compared with each other.     

Locally generalised multi-agent reinforcement learning for demand and capacity balancing with customised neural networks

Reinforcement Learning (RL) techniques are being studied to solve the Demand and Capacity Balancing (DCB) problems to fully exploit their computational performance. A locally generalised Multi-Agent Reinforcement Learning (MARL) for real-world DCB problems is proposed. The proposed method can deploy trained agents directly to unseen scenarios in a specific Air Traffic Flow Management (ATFM) region to quickly obtain a satisfactory solution. In this method, agents of all flights in a scenario form a multi-agent decision-making system based on partial observation. The trained agent with the customised neural network can be deployed directly on the corresponding flight, allowing it to solve the DCB problem jointly. A cooperation coefficient is introduced in the reward function, which is used to adjust the agent’s cooperation preference in a multi-agent system, thereby controlling the distribution of flight delay time allocation. A multi-iteration mechanism is designed for the DCB decision-making framework to deal with problems arising from non-stationarity in MARL and to ensure that all hotspots are eliminated. Experiments based on large-scale high-complexity real-world scenarios are conducted to verify the effectiveness and efficiency of the method. From a statistical point of view, it is proven that the proposed method is generalised within the scope of the flights and sectors of interest, and its optimisation performance outperforms the standard computer-assisted slot allocation and state-of-the-art RL-based DCB methods. The sensitivity analysis preliminarily reveals the effect of the cooperation coefficient on delay time allocation.     

基于冲突分类与消解的多智能体路径规划算法设计CSCD

多智能体路径规划应用广泛但求解困难。为更好地处理多智能体路径规划中的路径冲突问题,提高求解效率,将冲突进一步分类为相向顶点冲突和交叉顶点冲突,并提出了对应的消解方式。相向顶点冲突的消解方法采用提前添加约束的方式,避免在消解其冲突的过程中产生另一个可预见的冲突;交叉顶点冲突的消解方法采用寻找最佳等待时间的方式,在消解其冲突的同时消解其他存在的冲突。两种冲突消解方法均可减小约束树的规模,在一定程度上减少算法的计算量。并提出了基于冲突搜索算法的高层节点冲突搜索算法。实验结果表明,所提出的冲突分类及消解方式有效地减小了算法高层中约束树的规模,降低了算法计算量,并在智能体密集的环境下表现出更大的优势。     

Adaptive event-triggered distributed optimal guidance design via adaptive dynamic programming

In this paper, the multi-missile cooperative guidance system is formulated as a general nonlinear multi-agent system. To save the limited communication resources, an adaptive eventtriggered optimal guidance law is proposed by designing a synchronization-error-driven triggering condition, which brings together the consensus control with Adaptive Dynamic Programming(ADP) technique. Then, the developed event-triggered distributed control law can be employed by finding an approximate solution of eve...     

具有动态领导者的多智能体系统分组一致性控制

对一类具有动态领导者的非线性多智能体系统的一致性进行研究，解决了领导-跟随一致性控制问题。首先，考虑到多目标任务中广泛存在的合作-竞争机制和领导者的控制输入非零的情况，基于邻居智能体的相对状态信息，设计了一类分布式的分组一致性控制器。通过在所设计的控制器中增加补偿项，解决了由领导者非零控制输入引出的问题。引入了一类Lipschitz-like条件，解决了非线性项在实现分组一致时所带来的困难。其次，利用图论和构造Lyapunov函数，通过求解代数Riccati方程得出了系统实现分组一致性的充分条件，并基于所设计的控制器实现了多智能体系统动态领导-跟随的分组一致性控制。最后，通过数值仿真和结果分析验证了所提出控制方案的有效性和可行性。     

Improving multi-target cooperative tracking guidance for UAV swarms using multi-agent reinforcement learning

Multi-Target Tracking Guidance(MTTG) in unknown environments has great potential values in applications for Unmanned Aerial Vehicle(UAV) swarms. Although Multi-Agent Deep Reinforcement Learning(MADRL) is a promising technique for learning cooperation, most of the existing methods cannot scale well to decentralized UAV swarms due to their computational complexity or global information requirement. This paper proposes a decentralized MADRL method using the maximum reciprocal reward to learn cooper...     

集群多机器人系统建模研究的发展与展望

凭借高效、鲁棒、应用广泛的特性,集群多机器人系统已经成为当今研究的热点课题之一,具有重要的实用价值.首先简述了自顶而下和自底而上两种多机器人系统研究思路的当前研究概况.然后从拟生物集群系统模型引入,进而引出一致性系统模型,针对低阶、高阶、异质、时延等一致性模型进行分析总结,单独阐述了多智能体强化学习系统模型的研究情况,...