parallelformers icon indicating copy to clipboard operation
parallelformers copied to clipboard

RuntimeError: CUDA error: peer access is not supported between these two devices

Open Dorcoh4 opened this issue 2 years ago • 1 comments

I tried running the example from the readme but received the above error. Does that mean that my hardware is not supported?

Environment

  • OS : Ubuntu
  • Python version : 3.7.11
  • Transformers version : 4.23.1
  • Whether to use Docker: no
  • GPUs: NVIDIA GeForce RTX 2080 Ti

Dorcoh4 avatar Nov 14 '22 13:11 Dorcoh4

@Dorcoh4 hello. I am not involved with this project, however the error you see indicates that (iirc) the PCI bus has been blocked from using root PCI bridges. This is common in virtualized cloud providers.

Where are you running the workload? I may have workarounds, depending on provider!

sdake avatar Oct 06 '23 09:10 sdake