Pose estimation in robotic electric vehicle plug-in charging tasks using auto-annotation and deep learning-based keypoint detector

The rapid growth of electric vehicles and advancements in self-driving technologies necessitate the development of specialized infrastructure to support autonomy. Accurate pose estimation of electric vehicles sockets is crucial for efficient and reliable plug-in charging operations. This process is inherently complex due to several factors, including the textureless black color of the socket, environment-dependent lighting conditions, and the presence of small geometrical features. To address these challenges, we propose a comprehensive method that combines Deep Neural Network-based pose estimation and auto-annotation methodology. Auto-annotation facilitates the generation of diverse training data, enhancing the accuracy, robustness, and generalization capabilities of the deep learning model. The advantages of the proposed method were evaluated through a comprehensive series of experiments conducted in both simulation and controlled laboratory settings. In our evaluation, we implemented a robotic charging system consisting of a manipulator with a charging plug and a hand-eye monocular camera and conducted plug-in testing in three scenarios: (A) uniform lighting conditions, (B) dark, and (C) highly uneven illumination of the socket surface. The experimental results show that our method can achieve precise and reliable pose estimation with mean absolute errors less than 0.73 mm and 0.82 deg, and an average Insertion Success Rate of over 97.5%.