ali-vilab
When can I expect to have the code of DreamVideo? Thanks.
@lmm077 Hello, thank you for your attention to DreamVideo. Ideally, we plan to open source the code in about one to two months. We need to go through the internal approval process and are busy with other ongoing work. Please understand and be patient, thank you.
@lmm077 Hello, thank you for your attention to DreamVideo. Ideally, we plan to open source the code in about one to two months. We need to go through the internal approval process and are busy with other ongoing work. Please understand and be patient, thank you.
Hi, two months have passed, and what is the progress of the open-source of dreamvideo?
@lmm077 Hello, thank you for your attention to DreamVideo. Ideally, we plan to open source the code in about one to two months. We need to go through the internal approval process and are busy with other ongoing work. Please understand and be patient, thank you.
Hi, two months have passed, and what is the progress of the open-source of dreamvideo?
Hello, we will open source the code before the end of this month. We have already cleaned up the code and are about to submit it for internal review.
Hi guys, the code is released.
Hello, thanks for releasing the code of Dreamvideo, but I am not sure about the following questions:
- Can I generate videos with any resolutions? like 512x512 or 576x320
- could you plz release more pretrained models?
- Given some training videos with motion "bow", the inference videos cannot produce correct motion, especially not aligned with the given prompt. (1) a panda is bowing under the tree (2) an alien is bowing on the Mars
https://github.com/ali-vilab/VGen/assets/25433111/8155eae8-e76e-4128-8ade-d40dbe5a84cd
https://github.com/ali-vilab/VGen/assets/25433111/42d92209-9c09-4111-adf2-9d38709116db
Thanks again for your nice work.
Hi guys, the code is released.
Hello, thanks for releasing the code of Dreamvideo, but I am not sure about the following questions:
- Can I generate videos with any resolutions? like 512x512 or 576x320
- could you plz release more pretrained models?
- Given some training videos with motion "bow", the inference videos cannot produce correct motion, especially not aligned with the given prompt. (1) a panda is bowing under the tree (2) an alien is bowing on the Mars
A_panda_is_bowing_under_the_tree_8888_5.mp4 An_alien_is_bowing_on_Mars_8888_1.mp4 Thanks again for your nice work.
Hello, thanks for your interest.
- We recommend using the same resolution as the base model. In our experiments, the base model was trained with a resolution of 256x256.
- Yes, we will release models in a few days.
- (a) Have you put the reference images of panda and alien in the motionReferenceImgs folder and edited the config file? (b) What about using more training steps or a larger learning rate? (c) What about trying different seeds? If none of the above solves the problem, you can send the training videos and configuration file to my email "[email protected]" and I will gladly work this out.
Hi guys, the code is released.
Hello, thanks for releasing the code of Dreamvideo, but I am not sure about the following questions:
- Can I generate videos with any resolutions? like 512x512 or 576x320
- could you plz release more pretrained models?
- Given some training videos with motion "bow", the inference videos cannot produce correct motion, especially not aligned with the given prompt. (1) a panda is bowing under the tree (2) an alien is bowing on the Mars
A_panda_is_bowing_under_the_tree_8888_5.mp4 An_alien_is_bowing_on_Mars_8888_1.mp4 Thanks again for your nice work.
Hello, thanks for your interest.
- We recommend using the same resolution as the base model. In our experiments, the base model was trained with a resolution of 256x256.
- Yes, we will release models in a few days.
- (a) Have you put the reference images of panda and alien in the motionReferenceImgs folder and edited the config file? (b) What about using more training steps or a larger learning rate? (c) What about trying different seeds? If none of the above solves the problem, you can send the training videos and configuration file to my email "[email protected]" and I will gladly work this out.
Thank you for replying so quickly.
- I am not sure what "the base model" refers to. Does it denote modelscope or sd2.1?
- Waiting for your nice models.
- (a) I did not use reference images, during inference, I set "use_clip_adapter_condition" to False. Just use motion customization without subject customization. (b) Yep. I need to use more training steps, I just train the model for only 600 steps before. (c) changing seeds may help a lot. Thx again.
Hi guys, the code is released.
Hello, thanks for releasing the code of Dreamvideo, but I am not sure about the following questions:
- Can I generate videos with any resolutions? like 512x512 or 576x320
- could you plz release more pretrained models?
- Given some training videos with motion "bow", the inference videos cannot produce correct motion, especially not aligned with the given prompt. (1) a panda is bowing under the tree (2) an alien is bowing on the Mars
A_panda_is_bowing_under_the_tree_8888_5.mp4 An_alien_is_bowing_on_Mars_8888_1.mp4 Thanks again for your nice work.
Hello, thanks for your interest.
- We recommend using the same resolution as the base model. In our experiments, the base model was trained with a resolution of 256x256.
- Yes, we will release models in a few days.
- (a) Have you put the reference images of panda and alien in the motionReferenceImgs folder and edited the config file? (b) What about using more training steps or a larger learning rate? (c) What about trying different seeds? If none of the above solves the problem, you can send the training videos and configuration file to my email "[email protected]" and I will gladly work this out.
Thank you for replying so quickly.
- I am not sure what "the base model" refers to. Does it denote modelscope or sd2.1?
- Waiting for your nice models.
- (a) I did not use reference images, during inference, I set "use_clip_adapter_condition" to False. Just use motion customization without subject customization. (b) Yep. I need to use more training steps, I just train the model for only 600 steps before. (c) changing seeds may help a lot. Thx again.
The base model refers to the video diffusion model (ModelScopeT2V) and we do not use SD. Thanks for your patience.
@XiaominLi1997 We have released the model weights and provided some examples, please refer to the latest code.