votenet icon indicating copy to clipboard operation
votenet copied to clipboard

Training time on SUN RGBD Dataset

Open jytime opened this issue 5 years ago • 2 comments

Hi all,

I was wondering what's the training time of votenet on the sun rgbd dataset. According to my experience, using one 1080 Ti GPU on an Ubuntu 16.4 server, it would take around a day. In the paper of votenet, it's about 10 hours on a Volta Quadro GP100 GPU.

Is this difference caused by GPU version, or did I set anything wrong? I have closed the evaluation process during training. Any comment is appreciated.

Best

jytime avatar Nov 04 '19 07:11 jytime

@jytime I think your training time is reasonable. If you are using SSD as well, then I believe the GPU costs the difference.

charlesq34 avatar Nov 07 '19 07:11 charlesq34

@jytime Hello, Recently i am also working on reproducig Votenet. I use a Windows server with one GPU 1080ti and 16GB RAM. However, when i train the net on sunrgbd dataset, I am caught with a MemoryError problem in ...\lib\multiprocessing\reduction.py and the bug line is cls(buf, protocol).dump(obj).

Since your device is simliar to mine and you have trained it successfully, i am wondering if you have done any code optimization or any other parameter modifications.

Thank you very much!!!

StevenZzz07 avatar Dec 17 '19 11:12 StevenZzz07