Support Several MLLM Models

[BUAADreamer]

What does this PR do?

This PR is working!!

If you are interested, you can use my branch https://github.com/BUAADreamer/LLaMA-Factory for now.

Support models:

• [x] Video-LLaVA/LLaVA-Video-Next with videos and images as inputs
  • sft with only video/image inputs (mllm_demo/video_demo)
  • sft with video/image mix inputs (visual_mix_demo)
• [x] LLaVA-Next
• [x] Idefics-2

Features:

• [x] fine-tuning: sft/ppo/dpo/ktp/orpo/simpo
• [x] inference: add video inference

Before submitting

• [x] Did you read the contributor guideline?