FACIL icon indicating copy to clipboard operation
FACIL copied to clipboard

How to integrate ViT in Networks?

Open Snimm opened this issue 1 month ago • 1 comments

I want to use Huggingface's ViTForImageClassification. How do I integrate it in FACIL? I want to load the pretrained model 'google/vit-base-patch16-224'. I have read the instructions to add networks in readme of networks. However I am still not sure how to implement it. How do I set "self.head_var = 'fc'" when head is changed by "model.classifier = nn.Linear(768, num_classes)"? How exactly will a class even be created in this case?

Snimm avatar May 17 '24 18:05 Snimm