'RuntimeError: CUDA out of memory.' when use RTX3080

[YeautyYE]

My GPU is RTX3080, but when I use the command sudo sh ./scripts/inference_cogvideo_pipeline.sh, the following error occurs

RuntimeError: CUDA out of memory. Tried to allocate 54.00 MiB (GPU 0; 9.78 GiB total capacity; 9.53 GiB already allocated; 28.31 MiB free; 9.54 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

[wenyihong]

Hi, It takes around 25GB GPU memory to inference with batchsize=1 (on our A100).