How to use with SSL methods like DINOv2?

[josephcappadona]

For a SSL model like a ViT with DINO and iBOT heads, where should the MuReadout be placed? At the output of the backbone, or the output of the projection heads, or both?

Has anyone used muP with DINOv2 before? I did some searching but found nothing.