AnimateLCM icon indicating copy to clipboard operation
AnimateLCM copied to clipboard

No position encoder in your huggingface sd15 model weights

Open continue-revolution opened this issue 7 months ago • 1 comments

Someone asked me to support your work in AUTOMATIC1111 SD WebUI, and I have some questions.

There is no position encoder found in https://huggingface.co/wangfuyun/AnimateLCM/resolve/main/AnimateLCM_sd15_t2v.ckpt?download=true

However, in your diffusers weights https://huggingface.co/wangfuyun/AnimateLCM/resolve/main/diffusion_pytorch_model.fp16.safetensors?download=true, I do find position encoder, but it's incomplete. Typically in mid_blocks there are 2 position encoders, but in your diffusers model, there is only one.

It seems to me that you did require position encoding in your model because I saw https://github.com/G-U-N/AnimateLCM/blob/master/animatelcm_sd15/configs/inference-t2v.yaml#L23

I would appreciate if anyone can explain to me the difference between your model and the original AnimateDiff model architecture, or anyone can fix this issue by updating your weights on huggingface.

continue-revolution avatar Jul 11 '24 10:07 continue-revolution