Sathya Patel
Sathya Patel
Hi Julian @juliandewit He ran it on Nvidia Tesla K10 GPU's and obviously it takes hours to complete per epoch. @wojiushishen : why don't you try Nvidia GPU cloud High...
I'd commented line no 172 in tools/train.py if hasattr(cfg, 'convert_to_lite_model'): model = convert_to_lite_model(model, cfg) Working fine now!! Training has started successfully. I encountered some bugs in model_surgery.py , Can you...
Thanks! it worked. Another quick question, Is it possible to train QAT with pretrained weights ? I'd trained Centernet with customized model and datasets. I'm trying to do QAT on...
How much GPU memory do we need for this training ? I tried with single GPU instance of 16 GB throws CUDA out of memory. I'm passing a Model, dummy...
i reduced to batch size 16,8,4,2,1. facing same memory issue with batch size 1. If I comment xnn.quantize.QuantTrainModule in the code. Training has started without quantization module.
I'm using Centernet model and pretrained weights Input Image size is 512 X 320 total sample of 1062 images ERROR: Loaded train 929 samples /home/ubuntu/anaconda3/envs/edge-ai/lib/python3.7/site-packages/torch/utils/data/dataloader.py:481: UserWarning: This DataLoader will create...
issue is insufficient GPU memory. Changing input size of an images doesn't work. still thros same run time error. I removed loading pretrained weights section in the code Training has...
Hi Petr, I am also facing the same issue on AD_Invert.py.. 1 epoch is still running more than 15 hours kdd99_test dataset on GPU server. Did you find any solution...