ComfyUI-CogVideoXWrapper icon indicating copy to clipboard operation
ComfyUI-CogVideoXWrapper copied to clipboard

feature request - CogVideoX1.5-5B-SAT, master kijai!

Open whmc76 opened this issue 1 year ago • 11 comments

https://huggingface.co/THUDM/CogVideoX1.5-5B-SAT/tree/main

whmc76 avatar Nov 08 '24 07:11 whmc76

go go go!

Minamiyama avatar Nov 08 '24 07:11 Minamiyama

Nothing to do until they release diffusers version

jpgallegoar avatar Nov 08 '24 08:11 jpgallegoar

Diffusers implementation of the new rotary embeds is not done, so the model doesn't run yet. Gotta wait a bit.

kijai avatar Nov 08 '24 10:11 kijai

https://huggingface.co/Kijai/CogVideoX-5b-1.5 can we use this right now?

ruSauron avatar Nov 09 '24 17:11 ruSauron

https://huggingface.co/Kijai/CogVideoX-5b-1.5 can we use this right now?

If you want to test, sure, many things still not working though, the I2V works okayish at least. But you have to use this branch: https://github.com/kijai/ComfyUI-CogVideoXWrapper/tree/1.5_test

kijai avatar Nov 09 '24 17:11 kijai

Sorry, I'm trying after node update to use the 1.5 models, but they both (T2V, I2V) don't work. With I2V I get the following error:

  • Exception Message: The size of tensor a (4050) must match the size of tensor b (2700) at non-singleton dimension 2

I've used ComfyUI-CogVideoXWrapper\examples\cogvideox_I2V_example_01.json as workflow, changed model to "kijai/CogVideoX-5b-1.5-I2V" and CLIP to "t5xxl_fp8_e4m3fn.safetensors"

I've also tried using other workflows and creating workflows from scratch.

git checkout
Your branch is up to date with 'origin/1.5_test'.
git rev-list --min-parents=2 --max-count=1 HEAD
21025c4742e16f43f33c99db9193d53f818797c9

Log: 15_error.txt

ruSauron avatar Nov 09 '24 19:11 ruSauron

Sorry, I'm trying after node update to use the 1.5 models, but they both (T2V, I2V) don't work. With I2V I get the following error:

  • Exception Message: The size of tensor a (4050) must match the size of tensor b (2700) at non-singleton dimension 2

I've used ComfyUI-CogVideoXWrapper\examples\cogvideox_I2V_example_01.json as workflow, changed model to "kijai/CogVideoX-5b-1.5-I2V" and CLIP to "t5xxl_fp8_e4m3fn.safetensors"

I've also tried using other workflows and creating workflows from scratch.

git checkout
Your branch is up to date with 'origin/1.5_test'.
git rev-list --min-parents=2 --max-count=1 HEAD
21025c4742e16f43f33c99db9193d53f818797c9

Log: 15_error.txt

How many frames did you try with?

Seems odd, I followed those exact same steps now and it works for me.

kijai avatar Nov 09 '24 20:11 kijai

16 frames, now tried with 17+ frames. Thanks, the cause of the problem was not obvious

ruSauron avatar Nov 09 '24 20:11 ruSauron

16 frames, now tried with 17+ frames. Thanks, the cause of the problem was not obvious

Yeah it's weird with that, will make it more obvious once I'm certain what frame lengths work. Another thing I've noticed that the old default resolution (720x480) looks terrible, but for example 768x768 is nice. I think there's still something wrong in the current diffusers implementation.

kijai avatar Nov 09 '24 20:11 kijai

https://huggingface.co/Kijai/CogVideoX-5b-1.5 can we use this right now?

If you want to test, sure, many things still not working though, the I2V works okayish at least. But you have to use this branch: https://github.com/kijai/ComfyUI-CogVideoXWrapper/tree/1.5_test

How many VRAM can be use for this 5b-1.5 Model?

littleyeson avatar Nov 12 '24 06:11 littleyeson

https://huggingface.co/Kijai/CogVideoX-5b-1.5 can we use this right now?

If you want to test, sure, many things still not working though, the I2V works okayish at least. But you have to use this branch: https://github.com/kijai/ComfyUI-CogVideoXWrapper/tree/1.5_test

How many VRAM can be use for this 5b-1.5 Model?

Depends on frame count and resolution used, it's no different from before.

kijai avatar Nov 12 '24 06:11 kijai