Problems for Motion LoRA training and inference

[aulaywang]

I use ~100k videos selected from WebVid-2M to train the MotionLoRA, and I used the checkpoint of 40k steps (outputs/train_motion_training_xxx_2024-06-13T04-09-10/checkpoints/checkpoint-step-40000.ckpt) to replace the mm_sd_v15_v2.ckpt in configs/prompts/v2/v2-2-RealisticVision-MotionLoRA.yaml. However, the results are not so good: there are flickers around the image and the images almost remain stable (not animation). I am just curious about whether I loaded the training checkpoints at a wrong place? Or did any other people face the similar problems? How to use the checkpoint of the Motion LoRA?

[Ke-Hao]

Hello! Have you solved your problem? I could not find the training code of MotionLoRA. How do you trained the MotionLoRA? Thanks!