CogVideo icon indicating copy to clipboard operation
CogVideo copied to clipboard

Is it possible to implement a sliding temporal window to the CogVideoX model?

Open jpgallegoar opened this issue 1 year ago • 1 comments

Feature request / 功能建议

Would it be possible to create a sliding window sampler for ComfyUI that would take the previous x samples (latents) and generate a new one based on that, making it possible to extend videos further than 48 samples?

I gave it a go with OpenAI o1, Claude and Gemini 1.5 Pro but keep getting the same errors (spent probably 10h+ on this). I'm not technical enough to be able to do it myself.

Motivation / 动机

It would be possible to generate longer, yet consistent videos. The current method uses only the last frame of the video, which is not enough to generate consitency in movement.

Your contribution / 您的贡献

I can share some of the code I used, but it's incorrect. Better to start from scratch.

jpgallegoar avatar Oct 05 '24 13:10 jpgallegoar

Is it related to this work https://github.com/huggingface/diffusers/pull/9389 ?

rodjjo avatar Oct 05 '24 18:10 rodjjo