Breast Imaging by Cascaded CNN from Joint Microwave and Ultrasonic Data

In the context of early breast tumor characterization, combining electromagnetic (EM) and ultrasound (US) modalities is of interest, since both non-ionizing and low-cost, and harboring complementary resolution features. Here, a new Convolutional Neural Network (CNN) structure is proposed, denoted as Structurally-Aware Complex Cascaded Neural Network (SACC-CNN). It consists of two parts, the Structurally-Aware Reconstruction Net (SARNet) and the Structurally-Aware Classification Net (SACNet). SACNet outputs the tissue type map which is then fed to the SARNet, which reconstructs the EM and US parameters. These two parts can be seen as two independent modules. A physics-guided loss function is implemented in the SARNet network to enhance structural similarity. Main features of the approaches, illustrated by simulation, are described.