Ben Conrad
Results
2
comments of
Ben Conrad
The original authors do most of their experiments on ImageNet-21k, which you can get from the [ImageNet site](https://www.image-net.org/index.php). But regardless of the dataset, all you need to do is set...
`flexivit_tiny()` and the other functions initialize the model with random weights, so yes, you should pretrain before fine-tuning on a small dataset.