Yuxuan Zhang

显存不够，内存也不够。。 需要28G显存和内存运行BF16

啊，那个模型我们也不太了解，我们现在维护的是GLM-4 不过，正常来说，你这个实现方法应该是对的，同时建议把seed换成一样的，top p 和temperature改成 0

did you use pipe.enable_model_cpu_offload()? if not will use 36GB and will cause this problem if you want to use multi GPUS just remove pipe.enable_model_cpu_offload()

PYTORCH_CUDA_ALLOC_CONF=exandable_segments:True Try this. And what is your nvidia driver and GPU? V100 I guess. It should work( Although we only test in 3090 and A100)

you can use torch. 2.2 2.3 2.4 both not work right?

must one or more gpus >= 20GB beacuse of VAE.

Not now, we will try tied vae, we tested balance in 3 GPU in 20GB. you can try if you can run at 3 * 16G GPU

> > Not now, we will try tied vae, we tested balance in 3 GPU in 20GB. you can try if you can run at 3 * 16G GPU >...

did you follow with cli_demo.py code and using NVIDIA Ampere or higher GPU like 3090 4090

you can try reinstall the diffusers and accelerate libs from source, and A100 must work with using infersence/cli_demo.py in this github repos