基于DDQN的片上网络混合关键性消息调度方法

对片上网络（NoC）承载的混合关键性消息进行实时调度是其应用于航空电子系统片上多核通信的关键。为解决可满足性模理论（SMT）法求解效率低、低优先级消息等待延迟大的问题，提出了一种基于双深度Q网络（DDQN）的混合关键性消息调度方法。将虫孔交换机制下的消息调度问题建模为马尔可夫决策过程，建立包含环境、动作、状态、奖励的多层感知调度模型；随机生成多组分布不同的混合关键性消息作为训练样本，采用DDQN算法求解该调度模型；在此基础上，提出并实现了带孔隙DDQN算法，在保证时间触发（TT）消息可调度前提下为速率约束（RC）消息预留用于虫孔交换的时隙。算例研究表明:所提方法的求解时长及TT消息确定性端到端延迟的平均值均低于SMT法；带孔隙DDQN算法的RC消息延迟较不带孔隙DDQN算法和SMT法显著降低。 

A DDQN-based mixed-criticality messages scheduling method for network-on-chip

Real-time scheduling of mixed-criticality messages carried by network on chip (NoC) is the key to its application to on-chip muli-core communication in avionics system. A double deep Q-network(DDQN) method was proposed to solve the problem of satisfiability modulo theories (SMT) to low efficiency and high delay of low-priority messages. The message scheduling problem under wormhole switch mechanism was modeled as a Markov decision process, and a scheduling model including environment, action, state and reward was established. Then, DDQN was applied to interact with environment in different message distribution generated randomly, and the empirical sequence obtained through interaction was regarded as the training sample of the neural network. In addition, a scheduling method named pore-DDQN was implemented, that is, a time slot was reserved for rate-constrained (RC) messages on the condition that time-triggered (TT) messages can be scheduled. The case study shows that the solution time and the average end-to-end delay of TT messages of DDQN are lower than that of SMT, and the delay of RC messages with pore-DDQN is significantly lower than that of DDQN and SMT.