Acceleration Training

[ZouhairBALLOUCH]

Hi Sir ; Is there a possibility to speed up the training, because for me each epoch needs more than 3 hours
Thank you very much in advance

[R1CH4RDL]

Hi, did you solve your problem? I met a similar problem where I could only run the code on 1 gpu and the gpu memory used was only 161MB. It took 4 full days, almost 100 hours, to train 100 epochs in total.

[ZouhairBALLOUCH]

Hi; not yet , could you please share with me the pre-trained model that you have obtained Best regards

[xianyu-wang]

i had the same problem at one time. The cuda library was not loaded but it didn't show any warning or error. The program continued with CPU only. However, this issue didn't show again the day after. I have no idea what caused this issue

[helloCZZ]

Hi, did you solve your problem? I met a similar problem where I could only run the code on 1 gpu and the gpu memory used was only 161MB. It took 4 full days, almost 100 hours, to train 100 epochs in total.

hello . I met a similar problem. could you please share with me the pre-trained model that you have obtained Best regards

[AlberTgarY]

Hi, I had the same issue, I used the colab pro+ which has tesla V100, but it doesn't use GPU to train as well, so every epoch also took me 3 hours. Then I tried GPUSHARE which is similar to AWS, my VM has 3090 and AMD EPYC 7601, every epoch took me 1.5 hours, which is quite decent so far. tho it is still using CPU to train.