Cracks Segmentation of Engineering Structures in Complex Backgrounds Using a Concatenation of Transformer and CNN Models Driven by Scene Understanding Information

Routine visual inspection of cracks is essential to monitoring the safety conditions of engineering structures. Structural crack detection and segmentation based on machine vision have been extensively studied to automatically extract crack information in recent years. However, the complex background images in diverse fieldinspection scenarios often severely degrade the performance of crack segmentation algorithms. A novel crack segmentation method guided by image scene information understanding is proposed to improve the segmentation accuracy of surface cracks in engineering structures. The information in the structural crack images taken during inspection is summarized as cracks, auxiliary objects for crack detection and other background objects. Firstly, a cascaded model including Transformer and CNN is used to extract and separate the semantic information of cracks and complex backgrounds from patch scale to pixel scale in the image scenes. Then, objects such as manually annotated handwriting that are beneficial for cracks confirmation and segmentation are identified from the background. Finally, an adaptive crack information enhancement technique is presented to further improve the segmentation accuracy of cracks by capitalizing on the comprehension of the self and related information of handwriting and cracks. The detection results in actual steel bridges and concrete buildings show that utilizing handwriting information and excluding background objects based on understanding of scene information can significantly improve the accuracy of crack segmentation. Compared to a single Transformer network (SegFormer), the average accuracy and IOU have been improved by 0.3 similar to 0.4 and 0.2 similar to 0.3, respectively.