A policy iteration method for improving robot assembly trajectory efficiency

Bolt assembly by robots is a vital and difficult task for replacing astronauts in extra-vehicular activities (EVA), but the trajectory efficiency still needs to be improved during the wrench insertion into hex hole of bolt. In this paper, a policy iteration method based on reinforcement learning (RL) is proposed, by which the problem of trajectory efficiency improvement is constructed as an issue of RL-based objective optimization. Firstly, the projection relation between raw data and state-action space is established, and then a policy iteration initialization method is designed based on the projection to provide the initialization policy for iteration. Policy iteration based on the protective policy is applied to continuously evaluating and optimizing the action-value function of all state-action pairs till the convergence is obtained. To verify the feasibility and effectiveness of the proposed method, a noncontact demonstration experiment with human supervision is performed. Experimental results show that the initialization policy and the generated policy can be obtained by the policy iteration method in a limited number of demonstrations. A comparison between the experiments with two different assembly tolerances shows that the convergent generated policy possesses higher trajectory efficiency than the conservative one. In addition, this method can ensure safety during the training process and improve utilization efficiency of demonstration data.     

加工装配误差影响下偏转射流压力伺服阀静态特性分析

电液压力伺服阀是电液压力伺服控制系统的核心控制元件，广泛应用于航空、航天、军事等领域。区别于流量伺服阀，压力伺服阀在滑阀放大器的设计上多采用带有压力控制容腔的三通阀结构，不同的滑阀结构使得现有的偏转射流流量伺服阀仿真模型难以满足压力伺服阀性能预测的需求。本文基于AMESim平台建立了偏转射流压力伺服阀的仿真模型，并通过实验对仿真模型进行了验证，验证结果表明仿真模型能够准确地描述压力伺服阀的静态特性。最后，通过仿真模型分析了加工装配误差影响下压力伺服阀输出性能差异，仿真结果可为偏转射流压力伺服阀的性能预测和结构设计提供参考。