EAT_code icon indicating copy to clipboard operation
EAT_code copied to clipboard

RuntimeError: CUDA error: CUBLAS_STATUS_INVALID_VALUE when calling `cublasSgemm(..)

Open guo-king666 opened this issue 10 months ago • 4 comments

(eat) XX@1104:~/wav/EAT_code-main$ python demo.py --root_wav '/XX/wav/EAT_code-main/demo/video_processed/W015_neu_1_002' --emo hap deepprompt_eam3d_all_final_313 cuda is available /home/password123456/anaconda3/envs/eat/lib/python3.7/site-packages/torch/functional.py:478: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at /opt/conda/conda-bld/pytorch_1659484809535/work/aten/src/ATen/native/TensorShape.cpp:2894.) return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined] 0%| | 0/20 [00:00<?, ?it/s] 0%| | 0/1 [00:00<?, ?it/s] Traceback (most recent call last): File "demo.py", line 467, in test(f'./ckpt/{name}.pth.tar', args.emo, save_dir=f'./demo/output/{name}/') File "demo.py", line 360, in test emoprompt, deepprompt = emotionprompt(x) File "/home/password123456/anaconda3/envs/eat/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "/home/password123456/wav/EAT_code-main/modules/prompt.py", line 46, in forward s_trg = self.mappingnet(z_trg, y_org).reshape(bs, -1, 128) File "/home/password123456/anaconda3/envs/eat/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "/home/password123456/wav/EAT_code-main/modules/audioencoder.py", line 327, in forward h = self.shared(z) File "/home/password123456/anaconda3/envs/eat/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "/home/password123456/anaconda3/envs/eat/lib/python3.7/site-packages/torch/nn/modules/container.py", line 139, in forward input = module(input) File "/home/password123456/anaconda3/envs/eat/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, **kwargs) File "/home/password123456/anaconda3/envs/eat/lib/python3.7/site-packages/torch/nn/modules/linear.py", line 114, in forward return F.linear(input, self.weight, self.bias) RuntimeError: CUDA error: CUBLAS_STATUS_INVALID_VALUE when calling cublasSgemm( handle, opa, opb, m, n, k, &alpha, a, lda, b, ldb, &beta, c, ldc) 请问一下进行推理的时候,显示这个错误怎么解决呢

guo-king666 avatar Feb 26 '25 07:02 guo-king666

你好,感觉像CUDA版本问题,会不会是显卡太新了,用的CUDA版本比较高?

yuangan avatar Feb 26 '25 09:02 yuangan

你好,感觉像CUDA版本问题,会不会是显卡太新了,用的CUDA版本比较高?

CUDA版本11.8 pytorch版本1.12

guo-king666 avatar Feb 27 '25 06:02 guo-king666

cuda版本有点高,我之前是在3090上跑的,用的这个配置: Image 你可以对齐一下看看,具体的环境配置在environment.yml里面。 如果机器不支持11.3,可以试试用colab跑跑看哈。

yuangan avatar Feb 27 '25 06:02 yuangan

cuda版本有点高,我之前是在3090上跑的,用的这个配置: 大家可以看一下,具体的环境配置在environment.yml里面。 如果机器不支持11.3,可以试试用colab跑跑看哈。 Image

好的,非常感谢!我试着去修改下CUDA和pytorch版本,再运行看看

guo-king666 avatar Feb 27 '25 07:02 guo-king666