Transformer based Multitask Learning for Image Captioning and Object Detection
Advances in Knowledge Discovery and Data Mining Lecture Notes in Computer Science(2024)
摘要
In several real-world scenarios like autonomous navigation and mobility, to
obtain a better visual understanding of the surroundings, image captioning and
object detection play a crucial role. This work introduces a novel multitask
learning framework that combines image captioning and object detection into a
joint model. We propose TICOD, Transformer-based Image Captioning and Object
detection model for jointly training both tasks by combining the losses
obtained from image captioning and object detection networks. By leveraging
joint training, the model benefits from the complementary information shared
between the two tasks, leading to improved performance for image captioning.
Our approach utilizes a transformer-based architecture that enables end-to-end
network integration for image captioning and object detection and performs both
tasks jointly. We evaluate the effectiveness of our approach through
comprehensive experiments on the MS-COCO dataset. Our model outperforms the
baselines from image captioning literature by achieving a 3.65
BERTScore.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要