Learning representation via indirect feature decorrelation with bi-vector-based contrastive learning for clustering.

Clustering is an essential task in machine learning, and learning clustering-friendly representation is crucial for clustering performance. Recently, methods that combine contrastive learning with feature decorrelation have demonstrated their efficacy in learning representation for clustering. However, the feature decorrelation is usually performed in an additional feature space, making the optimization more complex. Moreover, the dimension of this extra feature space is linked to the batch size, but this essential correlation that affects feature decorrelation has been less studied. This paper explores the connection between batch size and feature decorrelation in contrastive learning, then proposes to utilize the linear independence created by contrastive learning with small batch sizes to promote feature decorrelation indirectly. Further, this work introduces bi-vector-based contrastive learning to enable contrastive learning with small batch sizes, which is also of interest to researchers with limited resources. Besides, spectral clustering is utilized to explain the advantage of using linear independence for feature decorrelation and clarify its condition. Experiment results on five benchmark datasets demonstrate the efficacy of the proposed method against state-of-the-art methods. As for the CIFAR-10 and CIFAR-100, our method achieves the clustering accuracy of 88.9% and 52.1%, which are 7.4% and 9.6% higher than the non-prior SOTA method.