Long-tailed Diffusion Models with Oriented Calibration

Diffusion models have the ability to produce high-quality images with remarkable realism and diversity. Their effectiveness heavily relies on massive training on large-scale datasets, which, however, can be considerably impaired in the presence of real-world long-tail data. For long tail diffusion model generation, current works focus on the calibration and enhancement of the tail generation with head-tail knowledge transfer. The transfer process relies on the abundant diversity derived from the head class and, more significantly, the condition capacity of the model prediction. However, it is worth noting that the dependency on the conditional model prediction to realize the knowledge transfer might exhibit bias during training, leading to unsatisfactory generation results and lack of robustness. To address the issue, we directly establish the knowledge transfer from head data samples, based on the multi-objective characteristics of the score function in the diffusion process. To this end, a directional calibration for the estimation of noisy tail sample score is performed towards the clean head samples~(T2H), leveraging the similarity within the data distribution from head to tail classes. This augmentation for the tail score estimation encourages better diversity in generating the samples of tail categories. We extensively evaluate our approach with experiments on multiple benchmark datasets, demonstrating its effectiveness and superior performance compared to existing methods.