cuda out of memory

[8600862]

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 880.00 MiB (GPU 0; 23.70 GiB total capacity; 20.08 GiB already allocated; 602.56 MiB free; 21.64 GiB reserved in total by PyTorch)

I using 3090 with 24g memory, so how many memory did it need

[Steven-SWZhang]

Currently, we have only developed and validated on A100. At the moment, we are preparing machines with V100 and A10, and we hope to eventually be compatible with these three types of GPU cards.

[ryanmafrty]

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 880.00 MiB (GPU 0; 23.70 GiB total capacity; 20.08 GiB already allocated; 602.56 MiB free; 21.64 GiB reserved in total by PyTorch)

I using 3090 with 24g memory, so how many memory did it need

try model.half() before load the model to gpu, especially unet, worked for me(took aboout 17G).

[JS-an]

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 880.00 MiB (GPU 0; 23.70 GiB total capacity; 20.08 GiB already allocated; 602.56 MiB free; 21.64 GiB reserved in total by PyTorch) I using 3090 with 24g memory, so how many memory did it need

try model.half() before load the model to gpu, especially unet, worked for me(took aboout 17G).

Where should I write this code?

[ryanmafrty]

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 880.00 MiB (GPU 0; 23.70 GiB total capacity; 20.08 GiB already allocated; 602.56 MiB free; 21.64 GiB reserved in total by PyTorch) I using 3090 with 24g memory, so how many memory did it need

try model.half() before load the model to gpu, especially unet, worked for me(took aboout 17G).

Where should I write this code?

in my version I modified tools/inferences/inference_i2vgen_entrance.py line 138:

• model = model.to(gpu)

• model = model.half().to(gpu)

[frankchieng]

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 880.00 MiB (GPU 0; 23.70 GiB total capacity; 20.08 GiB already allocated; 602.56 MiB free; 21.64 GiB reserved in total by PyTorch) I using 3090 with 24g memory, so how many memory did it need

try model.half() before load the model to gpu, especially unet, worked for me(took aboout 17G).

Where should I write this code?

in my version I modified tools/inferences/inference_i2vgen_entrance.py line 138:

• model = model.to(gpu)

• model = model.half().to(gpu)

have u tried the new gradio_app.py,how can we save the GPU VRAM?

[ryanmafrty]

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 880.00 MiB (GPU 0; 23.70 GiB total capacity; 20.08 GiB already allocated; 602.56 MiB free; 21.64 GiB reserved in total by PyTorch) I using 3090 with 24g memory, so how many memory did it need

try model.half() before load the model to gpu, especially unet, worked for me(took aboout 17G).

Where should I write this code?

in my version I modified tools/inferences/inference_i2vgen_entrance.py line 138:

• model = model.to(gpu)
• model = model.half().to(gpu)

have u tried the new gradio_app.py,how can we save the GPU VRAM?

not following this anymore, but should be the same, just find the real model and half it befor load to gpu. hornestly it should be fixed by ali long ago.

[frankchieng]

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 880.00 MiB (GPU 0; 23.70 GiB total capacity; 20.08 GiB already allocated; 602.56 MiB free; 21.64 GiB reserved in total by PyTorch) I using 3090 with 24g memory, so how many memory did it need

try model.half() before load the model to gpu, especially unet, worked for me(took aboout 17G).

Where should I write this code?

in my version I modified tools/inferences/inference_i2vgen_entrance.py line 138:

• model = model.to(gpu)
• model = model.half().to(gpu)

have u tried the new gradio_app.py,how can we save the GPU VRAM?

not following this anymore, but should be the same, just find the real model and half it befor load to gpu. hornestly it should be fixed by ali long ago.

thx,problem solved,set up cog environment like Docker,then in gradio run cog predict,i can successfully run this project on 4090 locally with gradio interface

[DoubleCake]

I copied the code from inference and replaced modelscope's model loading in gradio.py. So simply adjust it yourself.