MedNet: A Dual-Copy Mechanism for Medical Report Generation from Images

Generating medical reports from images is a complex task in the healthcare domain. Existing approaches often rely on predefined templates to retrieve sentences and do not take into account the hierarchical structure of medical reports. Additionally, they overlook the selective copying of input sequences to output sequences. To address these challenges, we propose MedNet, a generation-based model that employs a dual-copy mechanism to automatically generate medical reports from images. Our methodology involves first extracting features from images using an image encoder. Next, we use a dual-copy mechanism as the sequence encoder for retrieved reports to combine word generation in the decoder with bi-copying subsequences from the input sequence and placing them appropriately. Finally, a language decoder generates coherent and informative reports. We evaluate our MedNet on two public datasets, Open-I and MIMIC-CXR, and demonstrate that it outperforms current state-of-the-art methods. Our approach not only improves the quality of the generated reports but also allows for flexible generation, making it well-suited for a variety of healthcare applications. The proposed dual-copy mechanism in our approach enables the utilization of both integral tokens and sub-tokens, enhancing the accuracy and relevance of generated reports. Our work represents a significant step forward in the field of automated medical report generation from images.