Can the RTX 2080 Ti with 11GB RAM perform inference tasks ？

[pily1]

System Info / 系統信息

I used the master version,but when it inferenced with 2b model or 5b model,it reported a error,cuda out of memory my question is "Can the RTX 2080 Ti with 11GB RAM perform inference tasks ？“

Information / 问题信息

• [X] The official example scripts / 官方的示例脚本
• [ ] My own modified scripts / 我自己修改的脚本和任务

Reproduction / 复现过程

I use the command "python cli_demo.py --prompt "A girl ridding a bike." --model_path THUDM/CogVideoX-2b torch.OutofMemoryError:CUDA out of memory.Tried to allocate 35.31 GiB

Expected behavior / 期待表现

output a mp4 video

[jjhaggar]

They added this to the README.md, maybe it can help you make it work:

🔥🔥 News: 2024/8/29: By adding pipe.enable_sequential_cpu_offload() and pipe.vae.enable_slicing() to the inference code of CogVideoX-5B, VRAM usage can be reduced to 5GB. Please check the updated cli_demo.

[pily1]

I followed the README.md instruction,then it reported error "torch.OutofMemoryError:CUDA out of memory.Tried to allocate 35.31 GiB"

[zRzRzRzRzRzRzR]

same as #204

[wayne931121]

They added this to the README.md, maybe it can help you make it work:

🔥🔥 News: 2024/8/29: By adding pipe.enable_sequential_cpu_offload() and pipe.vae.enable_slicing() to the inference code of CogVideoX-5B, VRAM usage can be reduced to 5GB. Please check the updated cli_demo.

Thank you, after runing

pipe.enable_sequential_cpu_offload()
pipe.vae.enable_slicing()
pipe.vae.enable_tiling()

I can run in my 6GB GPU laptop !

Reference:
https://github.com/zai-org/CogVideo?tab=readme-ov-file#cogvideox-5b