How to use self supervised learning of the ViTMSN model in image_classification.ipynb?

[JensSettelmeier]

Dear Niels and huggingface team,

thanks for the notebook about fine-tuning ViTs from huggingface. Similar to ViTMAE, ViTMSN should be trainable in self-supervised fashion. That means it should be possible to load a pre-trained model with: pretrained_ViT = AutoModel.from_pretrained("facebook/vit-msn-base-4") and then finetune it on a costume unlabeled big dataset with the masked siamese approach, right? I thought there should be a function like "ViTMSNForPreTraining," similar to "ViTMAEForPreTraining," but unfortunately, I couldn't find it. Is there a workaround, or is someone from huggingface currently working on it by any chance?

I would appreciate your support!

Cheers, Jens