MM-RCNN: Toward Few-Shot Object Detection in Remote Sensing Images With Meta Memory

In the area of optical remote sensing image processing, object detection is among the utmost essential and difficult tasks. By virtue of the excellent feature representation abilities of deep convolutional neural networks (DCNNs), the performance of remote sensing object detection has recently increased dramatically. However, DCNN-based methods necessitate sufficient labeled training samples and tend to experience a considerable performance fall when training examples are insufficient. Many of the recently developed few-shot object detection (FSOD) methods attempt to solve the issue by using the idea of meta-learning, which intends to extract knowledge that can be generalized across various tasks. Despite its success, the negligence of the knowledge learned in the past, and inter-class correlations hinder the detection ability of novel classes. In this article, we propose a new meta-memory based FSOD approach named MM-RCNN. Specifically, MM-RCNN adopts a memory moduel to store each category’s knowledge learned in the training stage and the memory-based external attention (MEA) to aggregate all categories’ information simultaneously. Based on MEA, we design two feature enhancement modules for the region proposal network (RPN) and detection head to boost the performance. Experiments over two remote sensing benchmarks, i.e., DIOR and NWPU VHR10, verify the capability of our method (0.239 and 0.557 average mAP across all settings).