Finetuning config file

[joellliu]

❓ The question

Hi, I am wondering if you can provide your config file for finetuning on the Tulu V2 dataset? It would be helpful for reproducing the finetuning results. In addition, have you tried SFT on the 1B model? How's the result? Thank you!

[2015aroras]

Have you looked at https://github.com/allenai/open-instruct? It might have what you're looking for.

[joellliu]

Have you looked at https://github.com/allenai/open-instruct? It might have what you're looking for.

@2015aroras Thanks for your reply! It seems the open-instruct repo is mainly for DPO. I wonder if you can provide the supervised finetuning config for training on the Tuluv2 dataset. Thank you!

[2015aroras]

@hamishivi You probably know more about this?

[ethanlshen]

Hi! I have the same question and am wondering if there have been any updates on this? Thanks!

[hamishivi]

Hi, you can find the OLMo 2 SFT and DPO configs here: https://github.com/allenai/open-instruct/tree/main/configs/train_configs/olmo2

We do SFT and DPO in open-instruct (and RLVR).