Physics Informed Synthetic Image Generation for Deep Learning-Based Detection of Wrinkles and Folds

Abstract Deep learning-based image segmentation methods have showcased tremendous potential in defect detection applications for several manufacturing processes. Currently, majority of deep learning research for defect detection focuses on manufacturing processes where the defects have well-defined features and there is tremendous amount of image data available to learn such a data-dense model. This makes deep learning unsuitable for defect detection in high-mix low volume manufacturing applications where data are scarce and the features of defects are not well defined due to the nature of the process. Recently, there has been an increased impetus towards automation of high-performance manufacturing processes such as composite prepreg layup. Composite prepreg layup is high-mix low volume in nature and involves manipulation of a sheet-like material. In this work, we propose a deep learning framework to detect wrinkle-like defects during the composite prepreg layup process. Our work focuses on three main technological contributions: (1) generation of physics aware photo-realistic synthetic images with the combination of a thin-shell finite element-based sheet simulation and advanced graphics techniques for texture generation, (2) an open-source annotated dataset of 10,000 synthetic images and 1000 real process images of carbon fiber sheets with wrinkle-like defects, and (3) an efficient two-stage methodology for training the deep learning network on this hybrid dataset. Our method can achieve a mean average precision (mAP) of 0.98 on actual production data for detecting defects.