POCO icon indicating copy to clipboard operation
POCO copied to clipboard

RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when running the code

Open CUHKWilliam opened this issue 2 years ago • 7 comments

Hi! Thanks for your interesting and enlightening work on point cloud reconstruction tasks, and we are trying to reproduce your work. However, we encounter an error when running your code: Capture I am wondering how to fix this bug? Thanks for your timely response.

CUHKWilliam avatar May 25 '22 14:05 CUHKWilliam

when i use uperhead_transformer net, I have also the problem, and look for the problem in the internet, other people said a lot of method, but one of all can't solve the problem, I guess the problem about the memory of cuda, when the memory is big and sufficient.

funny000 avatar Jun 14 '22 01:06 funny000

What's your CUDA version ? If it is < 11, you should consider upgrade it, try and see if it fixes the error.

ListIndexOutOfRange avatar Jun 24 '22 08:06 ListIndexOutOfRange

I have the same problem with CUDA 11.1 and Pytorch 1.8.1 (these are the versions mentioned in the repo) on a machine with P100. Trying to figure out the solution now.

vaheta avatar Nov 23 '22 20:11 vaheta

did you get any clues, I'm still running into the same issues using the same poco environement specified in requirements.txt

cyberkarim avatar Jun 15 '23 11:06 cyberkarim

I have the same problem with CUDA 11.1 and Pytorch 1.8.1 (these are the versions mentioned in the repo) on a machine with P100. Trying to figure out the solution now.

Did you get any clues to solve the issue

I'm using redhat linux distro. My loaded cuda packages are :

hap3

cyberkarim avatar Jun 15 '23 11:06 cyberkarim

did you get any clues, I'm still running into the same issues using the same poco environement specified in requirements.txt

To be honest, I don't remember - I ended up reinstalling the CUDA and Pytorch a bunch of times, and at some point, it worked.

vaheta avatar Jun 15 '23 11:06 vaheta

Issue still pending, I was able to make it work once by reinstalling all the dependencies. Once I switched to another node server, reinstalled all dependencies following the same guidlines, it got me back to this error. @aboulch there's still no clear way to solve this issue.

cyberkarim avatar Jun 20 '23 10:06 cyberkarim