Coupling Model- and Data-Driven Networks for CT Metal Artifact Reduction

Computed tomography (CT) images are often corrupted by undesirable artifacts due to the presence of metallic implants, creating the problem of metal artifact reduction (MAR). Existing deep learning-based efforts of tackling this problem share two main common limitations, limiting their practical applications. Firstly, single domain knowledge is insufficient for MAR task, since image domain networks ignore the consistency constraint, while sinogram (X-ray projection) domain networks inevitably result in severe secondary artifacts. Secondly, dual domain networks which overcome this issue by jointly training subnetworks in both domains occupy more GPU memory during training due to joint training nature of their subnetworks. To address these issues, we propose a novel dual domain MAR framework that couples model- and data-driven networks. The proposed coupling strategy can help save GPU memory during training and enables the utilization of complementary network architectures. Specifically, we build a tight frame-based model-driven image domain subnetwork via encoding the non-local repetitive streaking patterns of metal artifacts as an explicit tight frame sparse representation model, while elaborate a transformer-based data-driven sinogram domain subnetwork. These two constructed subnetworks are coupled and trained separately to save GPU memory. To link the sinogram and image domains, we elaborate a prior artifact-attention threshold generating (PATG) module, which passes sinogram domain information to the image domain. The PATG module adaptively rescales multiple prior metal artifacts by taking into account the interdependencies among these artifacts. Extensive experiments on synthesized and clinical datasets demonstrate that the proposed method outperforms the state-of-the-art MAR methods by a large margin in terms of artifact reduction accuracy.