leejet
[Bug] causvid Lora for wan 2.1 1.3B doesn’t work
Git commit
I’m trying to use causvid Lora with wan 2.1 1.3B version the Lora loaded but it’s not working so I found another causvid Lora that’s not for ComfyUI but it won’t load please fix wan 2.1 Lora’s support causvid is the only way I can generate videos on my 8gb ram I can’t even do the required resolution so I’m trying to get a video at 320x256 resolution but all I get is deformed burnt videos txt 2 Img with wan works fine tho
Operating System & Version
Android Samsung s20+ sd865 adreno 650
GGML backends
CPU
Command-line arguments used
./sd -M vid_gen --diffusion-model /root/stable-diffusion.cpp/build/models/Phantom_Wan_1.3B-Q3_K_S.gguf --vae /root/stable-diffusion.cpp/build/models/wan_2.1_vae_fp8_e4m3fn.safetensors --t5xxl /root/stable-diffusion.cpp/build/models/umt5xxl-encoder-q2_k.gguf --lora-model-dir /root/stable-diffusion.cpp/build/models/ -s -1 --steps 8 --sampling-method euler --scheduler simple -W 384 -H 216 --fps 16 --video-frames 17 --clip-on-cpu -p "A sensual cyber-geisha with reflective armor, A dark fantasy concept art, matte painting, submerged in clear shallow water on a white sand beach, a single softbox studio light, high-key exposure, complex gear details, smoke and particle effects, crisp focus lora:wan-causvid-1.3b:1.0" -n "anime, cartoon, drawing, 3d render, cgi render, ai generated, ugly face" -o /root/stable-diffusion.cpp/build/videos/wan.avi --diffusion-fa --offload-to-cpu --threads 8 --cfg-scale 5.0 --flow-shift 5.0 && ffmpeg -i /root/stable-diffusion.cpp/build/videos/wan.avi video.mp4
Steps to reproduce
Put Lora in a custom folder then put Lora path into the commands then paste commands into termux everything loads fine but after looking at video causvid isn’t working just deformed burnt video
What you expected to happen
Get quality video
What actually happened
Deformed video
Logs / error messages / stack trace
No response
Additional context / environment details
No response
That's because causVid is not just like most LoRAs that add new knowledge to the model, it fundamentally changes the way the model is supposed to be sampled, kinda like the LCM and DMD LoRAs that require using different sampling methods to work properly, but with an even more drastic change: the model becomes causal, meaning it can't process all the video at once anymore.
Supporting CausVid would mean adding support for a new kind of models, which makes this not a bug, but a missing feature.
That's because causVid is not just like most LoRAs that add new knowledge to the model, it fundamentally changes the way the model is supposed to be sampled, kinda like the LCM and DMD LoRAs that require using different sampling methods to work properly, but with an even more drastic change: the model becomes causal, meaning it can't process all the video at once anymore.
Ohhh so which sampler scheduler is supported with causvid lora in sd.cpp?
Ohhh so which sampler scheduler is supported with causvid lora in sd.cpp?
I think LCM would be the best if autoregressive video generation was supported, but since it's not implemented yet, you're out of luck.
Wan generates videos all at once, and CausVid is trained to generate images frames by frame (kinda). So it won't work for now.