Meta-metric for saliency detection evaluation metrics based on application preference
Multimedia Tools Appl.(2018)
摘要
Existing saliency detection evaluation metrics often produce inconsistent evaluation results. Because of the widespread application of image saliency detection, we propose a meta-metric to evaluate the performance of these metrics based on the preference of an application that uses saliency maps as weighting maps. This study uses content-based image retrieval (CBIR) as the representative application. First, we perform CBIR using image features extracted from deep convolutional layers of convolutional neural networks as well as saliency maps computed by various saliency detection algorithms as the weighting maps over queries. Second, we establish the preference order of the saliency detection algorithms in the CBIR application by sorting the mean average precision. Third, we determine the preference order of these algorithms using existing saliency detection evaluation metrics. Finally, our meta-metric evaluates these metrics by correlating the preference order in the CBIR application with that determined by each evaluation metric. Experiments on three publicly available datasets show that, of 24 evaluation metrics, the traditional metric: area under receiver operating characteristic curve is the best metric for a CBIR application.
更多查看译文
关键词
Saliency detection,Traditional evaluation metrics,Image quality assessment,Content-based image retrieval
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络