Does xtuner support DPO for InternVL?

Open fabriceyhc opened this issue 1 year ago • 1 comments

I am trying to do a custom DPO fine-tuning for internvl_v2_internlm2_2b_lora_finetune, but the default config is oriented towards vanilla supervised fine-tuning with images. I tried to compare / incorporate changes from internlm2_chat_1_8b_dpo_full but am running into some issues with the dataset formats supported.

Is this something that xtuner actually supports at the moment?

Oct 07 '24 19:10 fabriceyhc

https://github.com/hhaAndroid/xtuner/blob/hha_0919/my_llava/README.md#dpo

Oct 09 '24 04:10 hhaAndroid