A problem about diffusers

[zeezou-isee]

Hi, @Wangt-CN thanks for your great job and open source!

Recently I focus on Video Generation and I try to reproduce your DisCo project, actually, I find an error when loading the sd-image-variations-diffusers at

vae = AutoencoderKL.from_pretrained(args.pretrained_model_path, subfolder="vae" ),

and it gives me a hint to change it into

vae = AutoencoderKL.from_pretrained(args.pretrained_model_path, subfolder="vae" , low_cpu_mem_usage=False , device_map= None)

then it works. However, this hint tells that it will init the missing weights randomly, I think that's why I got a terrible result(Much worse than experiment in your paper) when I run bash moretiktok_cfg.sh.

• Debug info like below:

ValueError: Cannot load <class 'diffusers.models.autoencoder_kl.AutoencoderKL'> from /homeB/Public/zouxingze/DisCo-main/output/diffusers/sd-image-variations-diffusers because the following keys are missing: encoder.mid_block.attentions.0.proj_attn.weight, decoder.mid_block.attentions.0.query.weight, decoder.mid_block.attentions.0.query.bias, decoder.mid_block.attentions.0.value.bias, encoder.mid_block.attentions.0.value.bias, decoder.mid_block.attentions.0.proj_attn.weight, decoder.mid_block.attentions.0.value.weight, encoder.mid_block.attentions.0.key.weight, decoder.mid_block.attentions.0.proj_attn.bias, encoder.mid_block.attentions.0.value.weight, decoder.mid_block.attentions.0.key.bias, decoder.mid_block.attentions.0.key.weight, encoder.mid_block.attentions.0.proj_attn.bias, encoder.mid_block.attentions.0.query.bias, encoder.mid_block.attentions.0.query.weight, encoder.mid_block.attentions.0.key.bias. Please make sure to pass low_cpu_mem_usage=False and device_map=None, if you want to randomely initialize those weights or else make sure your checkpoint file is correct.

BTW, I download diffusers on early-July at sd-image-variations-diffusers.

I wonder whether it's diffusers's version conflict causing these keys missing or not? Or how can I load the right weights for sd-image-variations-diffusers? I appreciate it if you can help me to solve this question.

What's more, I found that when I try to use pip install xxx to install packages you provided in your DisCo page Getting Started, the packages installed by default meet some dependencies conflict [on my device Ubantu, conda version : 24.1.2, conda-build version 24.1.2 ,python =3.8]. Firstly if someone meet the same question ,maybe you can try : pip install ninja==1.10.2 , pip install xformers=0.0.13 .