Multi-label Classification for Metal Defects from SEM Images using Deep Learning

Significant concerns have focused on how to discern different categories of metal pipes failure during routine industrial production inspection. Traditional failure analysis techniques based on scanning electron microscopy (SEM) are inefficient and rely on the expertise of experts. In order to reduce the high labor cost and accuracy associated with manual judgment, we developed a classification system using two deep learning methods, Vision Transformer (ViT) and Convolutional neural network (CNN), to examine metal pipe defects. In this study, the multi-label classification dataset containing 2,075 SEM images of four subcategories was first created. To identify the speed and effect, eight models containing three resolution were trained, validated, and tested to obtain the optimal. Finally,inference speed of these models were tested to match the real-time criteria on x86 and arm architectures computing platforms. Results showed that models based on EfficinetNet and Vision Transformer can accurately identify metal flaws on SEM images in real-time, which are similar as the results from manual judgement. In particular, EfficientNetB4 had higher accuracy of 0.969 in cracking, 0.933 in fatigue, and 0.938 in inclusion at 600x450 resolution, while ViT performed better in hole images (0.974 accuracy). EfficientNetB0 had the fastest detection speed at the same resolution, which can be used on devices with low computing performance.