SiLLM icon indicating copy to clipboard operation
SiLLM copied to clipboard

[Feature request] Add ORPO finetuning

Open s-kostyaev opened this issue 2 months ago • 2 comments

Hi @armbues, thank you for this great project.

Please add ORPO finetuning to do SFT and DPO in one step https://arxiv.org/abs/2403.07691

s-kostyaev avatar Apr 15 '24 13:04 s-kostyaev