CUDA error: no kernel image is available for execution on the device

[liuxz-cs]

We run the scripts for inference a video by the command python run_net.py \ --cfg configs/exp01_vidcomposer_full.yaml \ --input_video "demo_video/blackswan.mp4" \ --input_text_desc "A black swan swam in the water" \ --seed 9999 . We get the error as follows, File "/root/paddlejob/workspace/lxz/miniconda3/envs/VideoComposer/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "/root/paddlejob/workspace/lxz/videocomposer/tools/videocomposer/unet_sd.py", line 238, in forward out = xformers.ops.memory_efficient_attention(q, k, v, attn_bias=None, op=self.attention_op) File "/root/paddlejob/workspace/lxz/miniconda3/envs/VideoComposer/lib/python3.8/site-packages/xformers/ops.py", line 574, in memory_efficient_attention return op.forward_no_grad( File "/root/paddlejob/workspace/lxz/miniconda3/envs/VideoComposer/lib/python3.8/site-packages/xformers/ops.py", line 189, in forward_no_grad return cls.FORWARD_OPERATOR( File "/root/paddlejob/workspace/lxz/miniconda3/envs/VideoComposer/lib/python3.8/site-packages/torch/_ops.py", line 143, in __call__ return self._op(*args, **kwargs or {}) RuntimeError: CUDA error: no kernel image is available for execution on the device

The version of torch is the same as yours. The version of cuda is 11.3, and torch==1.12.0+cu113, torchvision==0.13.0+cu113. We use a V100, and when we execute nvidia-smi the cuda version shown on V100 is 11.4. We think the version of our machine is compatible, and we do not know where the problem is.

[liaogulou]

同问，一样的错误

[liuxz-cs]

同问，一样的错误

可能是机器支持的cuda版本或者算力不够

[Steven-SWZhang]

I guess there may be two possible reasons: 1) Your GPU memory may not be sufficient. The current model inference requires 28G of GPU memory. Please check your machine; 2) If the torch version is correct, please try recompiling xformers from the source. Other researchers have solved this problem in this way before.

[Zeldalina]

我也遇到同样的问题，我的经历过程是，先是单张3090部署推理，运行报错。考虑显存不足的问题，使用2张3090部署推理，运行显示线程错误。之后使用V100-SXM2-32G部署推理同样的报上述错误，直到我使用A100-PCIE-40GB部署推理，问题解决。