How to finetune

[chenlindu]

Great job! I’m wondering if there are any scripts available that I can use to fine-tune this model for my specific task.

[shenyunhang]

Hi @CDchenlin, Our training script should be able to fine-tune your task with some preparation.

1. register new data to detectron2
2. maybe need to modify the data mapper
3. add the data to model configs
4. maybe need to modify the loss function

[skylning]

微调对设备有什么要求吗？一张rtx4090，只有少量数据可以不？

[shenyunhang]

我们是在V100上训练。 RTX4090的显存可能不太够，可以考虑冻结部分层，或者语言模型离线提取prompt，可以节省些内存。 我们也在训练ViT-Ti的小模型。