Unable to run on aws g3s.xlarge

[darganad]

I am trying to run inference using aws g3s.xlarge

GPU Memory (GiB) of 8 GB Memory is 30.5 GB

Exception: Failed to invoke function <function inference_i2vgen_entrance at 0x7fb7d397aca0>, with CUDA out of memory. Tried to allocate 258.00 MiB (GPU 0; 7.44 GiB total capacity; 6.72 GiB already allocated; 75.06 MiB free; 6.89 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

Please advise thank you

[Steven-SWZhang]

I'm sorry that we do not support inference on an 8G GPU at the moment.