Pretrained weights

[FBehrad]

Hello,

Thank you for preparing the code. I used the following code to create FlexiViT and then I fine-tuned it on my task. As the results are not good I was wondering whether I need to pretrain the network or if the code is already using pre trained models.

from flexivit_pytorch import (flexivit_base, flexivit_huge, flexivit_large,
                              flexivit_small, flexivit_tiny)

net = flexivit_tiny()
net = flexivit_small()
net = flexivit_base()
net = flexivit_large()
net = flexivit_huge()

[bwconrad]

flexivit_tiny() and the other functions initialize the model with random weights, so yes, you should pretrain before fine-tuning on a small dataset.