What Do Datasets Say About Saliency Models?

Given the amount and variety of saliency models, the knowledge of their pros and cons, the applications they are more suitable for, or which are the more challenging scenes for each of them, would be very useful for the progress in the field. This assessment can be done based on the link between algorithms and public datasets. In one hand, performance scores of algorithms can be used to cluster video samples according to the pattern of difficulties they pose to models. In the other hand, cluster labels can be combined with video annotations to select discriminant attributes for each cluster. In this work we seek this link and try to describe each cluster of videos in a few words.