Imbalanced Dataset

[millanp95]

Hi,

Thank you for this implementation. It is my understanding that some contrastive frameworks build upon entropy maximization, which leads to inapplicability in the contexts of imbalanced datasets. From the paper, I could see that you are also maximizing the entropy in your loss function. Can the instance-level term mitigate the entropy maximization issue and make the method suitable for imbalanced datasets?

Thanks

[Yunfan-Li]

Yes, the instance-level contrastive learning is not sensitive to imbalance datasets. In fact we have tested our method on some imbalanced datasets by simply removing the entropy maximization term on the cluster-level contrastive head and it gives reasonable results instead of trivial solution. Perhaps you could try to add a smaller weights on the entropy maximization term and strengthen the instance-level term just like you said.