Knowledge-Based Reinforcement Learning for Industrial Robotic Assembly.

Although reinforcement learning has shown promise in solving industrial assembly tasks, it still faces challenges such as poor sample efficiency and sparse rewards that limit its learning capability. To address these challenges, we split multi-step assembly tasks into modular sub-tasks and use CAD-based prior knowledge to facilitate sub-task learning. The geometric information from the knowledge extraction module accelerates the learning of grasping and placing. Instead of considering multiple possible placement errors in a jig-free environment during training, which would significantly increase training time, our method uses a compensation module with a spatial transformer network to deal with errors. We evaluated our method on two 3D-printed models with different materials and achieved a completion success rate of 96% for the plastic model and 94% for the metal model. Our results indicate that our model can be robustly applied to products with similar geometry without requiring additional model updates.