fairseq icon indicating copy to clipboard operation
fairseq copied to clipboard

SM60 Nvidia P100 is supported or not? Hubert Failing with missing kernel image in forward pass.

Open Ph0rk0z opened this issue 2 years ago • 0 comments

I have an issue when using hubert and fairseq on a P100 GPU. Same code runs fine on P40 and RTX 3090. Is compute 6.0 not supported?



Traceback (most recent call last):
  File "/home/supermicro/ai/TavernAI-extras/modules/voice_conversion/rvc/rvc.py", line 233, in vc_single
    audio_opt = vc.pipeline(
  File "/home/supermicro/ai/TavernAI-extras/modules/voice_conversion/rvc/vc_infer_pipeline.py", line 355, in pipeline
    self.vc(
  File "/home/supermicro/ai/TavernAI-extras/modules/voice_conversion/rvc/vc_infer_pipeline.py", line 154, in vc
    logits = model.extract_features(**inputs)
  File "/home/supermicro/ai/TavernAI-extras/modules/voice_conversion/fairseq/models/hubert/hubert.py", line 535, in extract_features
    res = self.forward(
  File "/home/supermicro/ai/TavernAI-extras/modules/voice_conversion/fairseq/models/hubert/hubert.py", line 445, in forward
    unmasked_features = features.clone()
RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

Is this a missing image from fairseq or torch? I have built fairseq from source and it still errors.

Ph0rk0z avatar Dec 01 '23 13:12 Ph0rk0z