How to Fine-tune ImageNet-1K by ImageNet-21K pretrained

[tiancity-NJU]

trafficstars

Thanks for you great work. I want to fine-tune on ImageNet-1K by ImageNet-21K pretrained bacause it's more faster to reproduce your result. So how to do it? I try to resume the backbone and change the head to 1K. Do I need to freeze the backbone? or just train the whole model by lr=1e-5? The max batchsize is 64 for swin-B, How to reach the bs=1024 you mentioned in your paper?

[ancientmooner]

You can try accumulate gradients to enlarge your batch size from 64 to 1024.

Will provide configs for finetuning from ImageNet-21K to ImageNet-1K.

[zeliu98]

Hi @tiancity-NJU, you can find instructions and configs here: https://github.com/microsoft/Swin-Transformer/blob/main/get_started.md#fine-tuning-from-a-imagenet-22k21k-pre-trained-model