MeMu: Metric Correlation Siamese Network and Multi-class Negative Sampling for Visual Tracking

•A multi-class negative sampling method is proposed for Siamese network, which contains three kinds of negative samples with different amount of background information to better distinguish object and background.•A metric correlation filter module is proposed to learn the filter that can better distinguish object and background through the multi-class negative samples.•We present a Siamese network-based architecture to embed the metric correlation filter module mentioned above.•The proposed tracking algorithm performs favorably against the state-of- the-art methods.