[Feature] DPO training for Wan2.1/Wan2.2 models

[vibe-viscot]

Motivation

Could you please add support for DPO training for Wan2.1 and Wan2.2 models?

Thank you!

Related resources

No response

[zhisbug]

could you describe your use case to provide context? e.g. what kind of dataset you use, which DPO training algorithm for video?

[vibe-viscot]

I read the Qwen-Image paper and noticed it uses DPO training. While there are some open-source frameworks that can do SFT training for Wan2.1/Wan2.2, none of them seem to support DPO training. I'm hoping FastVideo can implement offline DPO training - basically preparing the chosen and rejected data pairs in advance, then doing offline DPO training with them.