CogVideo icon indicating copy to clipboard operation
CogVideo copied to clipboard

Published 20 hours ago • verified publisher icon THUDM

Video Reconstruction based on Inversion

Open quanhaol opened this issue 9 months ago • 3 comments

Feature request / 功能建议

Video Reconstruction based on Inversion

Motivation / 动机

I want to use CogVideoX-5b-I2V to reconstruct a video based on DDIM Inversion. But, the result seems to be really poor. Is this normal? Or is there something wrong.

https://github.com/user-attachments/assets/b3c54e00-6ee6-4444-b31b-9cea00062bf1

Your contribution / 您的贡献

.

quanhaol avatar Jan 27 '25 10:01 quanhaol

We indeed have not carried out similar operations, but it should be noted that CogVideoX-5b-I2V uses ddpm instead of ddim.

zRzRzRzRzRzRzR avatar Jan 28 '25 12:01 zRzRzRzRzRzRzR

Specifically, you can see the configuration we used to infer CogVideoX5B in the CLI demo and the readme. About DDIM reverse reconstruction video. We indeed have not conducted any related tests.

zRzRzRzRzRzRzR avatar Jan 28 '25 12:01 zRzRzRzRzRzRzR

Thanks for your response! I tried to inverse a boat video based on DDIM Inversion with Cogvideox-2b(which is suggested in CLI demo), it also has very poor result.

Inversed Video: https://github.com/user-attachments/assets/5ab0e144-2dd4-46c1-9be3-3951b1626e42 Original Video: https://github.com/user-attachments/assets/86a3716f-ad2e-44d3-bc97-b2ebb878886f

quanhaol avatar Jan 29 '25 10:01 quanhaol