mmpretrain
mmpretrain copied to clipboard
[Feature] Validate SSL embeddings using RCDM
Describe the feature
Representation Conditional Diffusion Model (RCDM) is a method to visualize the representations learned by self-supervised models. This can be used to analyze self-supervised models, and diagnose issues such as why the model is overfitting to the training dataset (Appendix D in the paper). It accomplishes this by training a diffusion model conditioned on the SSL embeddings.
Paper: https://arxiv.org/abs/2112.09164 Reference implementation: https://github.com/facebookresearch/RCDM
Will you implement it?
- [ ] I would like to implement this feature and create a PR!