gaussian-splatting icon indicating copy to clipboard operation
gaussian-splatting copied to clipboard

Cuda out of memory when calling the distCUDA2

Open M1sak1-Mei opened this issue 1 year ago • 7 comments
trafficstars

At the same env,code could run with 3090, but when using Tesla V100 32g File "***/gaussian-splatting/scene/gaussian_model.py", line 134, in create_from_pcd dist2 = torch.clamp_min(distCUDA2(torch.from_numpy(np.asarray(pcd.points)).float().cuda()), 0.0000001) MemoryError: std::bad_alloc: cudaErrorMemoryAllocation: out of memory python-BaseException

3090 is using ubuntu22.04, V100 is ubuntu20.04 I have tried reinstall simple-knn,it does not work

M1sak1-Mei avatar Apr 07 '24 09:04 M1sak1-Mei

same situation here, simple-knn doesn't work with a800 but works fine with 3090

wb014 avatar Apr 14 '24 11:04 wb014

我把cudasdk,conda什么的都重装,环境重新配置之后就好了 My issus fixed after I reinstalled everything

M1sak1-Mei avatar Apr 17 '24 08:04 M1sak1-Mei

Hey!

I had the same problem and it somehow got fixed on its own. I couldn't figure out why that was happening but I think it has something to do with CUDA versions and the specific GPU you're using. My problem thankfully fixed itself magically but I have a couple of suggestions.

Maybe you can try deleting the build directory that gets created in your submodules/simple-knn directory and try reinstalling via pip. I guess some executables created on a different GPU when you first compile it might cause issues on other GPUs.

Also according to this issue, the only place distCUDA2 and simple-knn are used is to calculate the initial scale values of the gaussians. The answers to the issue have code examples on how to do the exact same thing with scipy without using simple-knn at all so you can basically remove that line of code.

I hope this helps, good luck!

altaykacan avatar Apr 27 '24 19:04 altaykacan