CUDA ERROR

[lukeewin]

trafficstars

Environment

• OS (e.g., Linux): Ubuntu Server 22.04
• FunASR Version (e.g., 1.0.0): 1.1.12
• ModelScope Version (e.g., 1.11.0): 1.19.2
• PyTorch Version (e.g., 2.0.0): 2.5
• How you installed funasr (pip, source): pip install funasr
• Python version: 3.9
• GPU (e.g., V100M32) 2080Ti
• CUDA/cuDNN version (e.g., cuda11.7): cuda 12.8
• Docker version (e.g., funasr-runtime-sdk-cpu-0.4.1)
• Any other relevant information:

Additional context

在ubuntu中运行funasr进行模型推理，报下面错误。

CUDA error: an illegal memory access was encountered\nCUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.\nFor debugging consider passing CUDA_LAUNCH_BLOCKING=1\nCompile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.\n')
Traceback (most recent call last):
  File "/root/miniconda3/envs/asr/lib/python3.9/site-packages/celery/app/trace.py", line 453, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/root/miniconda3/envs/asr/lib/python3.9/site-packages/celery/app/trace.py", line 736, in __protected_call__
    return self.run(*args, **kwargs)
  File "/usr/local/src/new_asr_spk_api/top/lukeewin/asr/service/asr_service.py", line 56, in asr_task
    raise self.retry(exc=e, countdown=5, max_retries=2)
  File "/root/miniconda3/envs/asr/lib/python3.9/site-packages/celery/app/task.py", line 743, in retry
    raise_with_context(exc)
  File "/usr/local/src/new_asr_spk_api/top/lukeewin/asr/service/asr_service.py", line 32, in asr_task
    result = asr.model.generate(audio_path, is_final=True, batch_size_s=300)
  File "/root/miniconda3/envs/asr/lib/python3.9/site-packages/funasr/auto/auto_model.py", line 303, in generate
    return self.inference_with_vad(input, input_len=input_len, **cfg)
  File "/root/miniconda3/envs/asr/lib/python3.9/site-packages/funasr/auto/auto_model.py", line 376, in inference_with_vad
    res = self.inference(
  File "/root/miniconda3/envs/asr/lib/python3.9/site-packages/funasr/auto/auto_model.py", line 342, in inference
    res = model.inference(**batch, **kwargs)
  File "/root/miniconda3/envs/asr/lib/python3.9/site-packages/funasr/models/fsmn_vad_streaming/model.py", line 712, in inference
    speech = speech.to(device=kwargs["device"])
RuntimeError: CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

请问有人遇到过吗？如何解决？看报错好像出自vad模型推理。