mmpretrain icon indicating copy to clipboard operation
mmpretrain copied to clipboard

[Feature] Validate SSL embeddings using RCDM

Open crypdick opened this issue 11 months ago • 0 comments

Describe the feature

Representation Conditional Diffusion Model (RCDM) is a method to visualize the representations learned by self-supervised models. This can be used to analyze self-supervised models, and diagnose issues such as why the model is overfitting to the training dataset (Appendix D in the paper). It accomplishes this by training a diffusion model conditioned on the SSL embeddings.

Paper: https://arxiv.org/abs/2112.09164 Reference implementation: https://github.com/facebookresearch/RCDM

Will you implement it?

  • [ ] I would like to implement this feature and create a PR!

crypdick avatar Aug 03 '23 17:08 crypdick