Performance and limitations of deep learning semantic segmentation of multiple defects in transmission electron micrographs

Transmission electron microscopy (TEM) is a popular method for characterizing and quantifying defects in materials. Analyzing digitized TEM images is typically done manually, which is a time-consuming and potentially error-prone task that is not scalable to large dataset sizes, motivating development of automated methods for quantifying and analyzing defects in TEM images. In this work, we perform semantic segmentation of multiple defect types in electron microscopy images of irradiated FeCrAl alloys using a deep learning mask regional convolutional neural network (Mask R-CNN) model. We evaluate the performance of the model based on distributions of defect shapes, sizes, and areal densities relevant to informing physical modeling and understanding irradiated Fe-based materials properties. To better understand the performance and present limitations of the model, we provide examples of useful evaluation tests, which include a suite of random splits and dataset-size-dependent and domain-targeted cross-validation tests, exposing potential weak points in the model applicability domain. Our model predicts the expected irradiation-induced material hardening to within 10–20 MPa (about 10% of total hardening), on par with experimental error. Finally, we discuss the first phase of an effort to provide an easy-to-use, open-source object detection tool to the broader community for identifying defects in new images.