DiffSynth-Studio icon indicating copy to clipboard operation
DiffSynth-Studio copied to clipboard
verified publisher icon modelscope

feat: support I2V training

Open qiwang1996 opened this issue 10 months ago • 4 comments

qiwang1996 avatar Feb 26 '25 11:02 qiwang1996

Wow! Thanks! We will test and review the code. By the way, could you tell me how much VRAM is required to train the I2V model?

Artiprocher avatar Feb 26 '25 12:02 Artiprocher

your code is so clear that it's very easy to understand and further develop. It just costs me no more than 1.5hrs to write this so i am not certainly sure these codes are right. image image I use single H800 and it is still training now. I will also further test my codes after training finished. The Peak VRAM use is about 42868MiB as the above picture shows.

qiwang1996 avatar Feb 26 '25 13:02 qiwang1996

there maybe some bugs in my code, i trained and infer i2v lora through it but get wrong results. Can anybody point out how to revise code for bug free.

qiwang1996 avatar Feb 27 '25 03:02 qiwang1996

i have not enough time to debug these days....

qiwang1996 avatar Feb 27 '25 03:02 qiwang1996

there maybe some bugs in my code, i trained and infer i2v lora through it but get wrong results. Can anybody point out how to revise code for bug free.

The data processing is not aligned. There is a crop operation in the video, but not in the image.

shapera-lab avatar Mar 10 '25 06:03 shapera-lab

@qiwang1996 Thank you for your contribution. Is this working correctly now?

xiaocaijiayou avatar Mar 11 '25 08:03 xiaocaijiayou