What's the different between Instruction Tuning to Align with Natural Language Instructions and Finetune Using Your Own Data?

[JustinZou1]

What's the different between Instruction Tuning to Align with Natural Language Instructions and Finetune Using Your Own Data?

[yuewang-cuhk]

Hi there, these two are very similar and you can view the instruction tuning as a specific use case of finetuning on your own data. In the instruction tuning example, we showcase how to finetune CodeT5+ 16B on A100-40G with deepspeed and mix precision training, while the finetuning on your own data example is more for smaller models (220M/770M). So if you want to finetune CodeT5+ 2B/6B/16B, the instruction tuning example would be better example to follow.