MapTR icon indicating copy to clipboard operation
MapTR copied to clipboard

Single card(RTX 3090) training results

Open sjhfdl opened this issue 1 year ago • 12 comments

Thanks for the great work and sharing the code!

I trained through a single RTX3090 graphics card, the configuration file is maptr_tiny_r50_24e.py, the results after training are shown in the figure, the results are not ideal, and there is a big gap between the results in the paper, I would like to ask if you have tried the single card training, or what should be paid attention to in the training Figure_1

sjhfdl avatar Jun 08 '23 07:06 sjhfdl

I have also encounter the same problem

adasfag avatar Jun 20 '23 15:06 adasfag

We trained it on the two A100 GPUS, and the Map result is about 0.35 in the epoch 24

adasfag avatar Jun 20 '23 15:06 adasfag

We trained it on the two A100 GPUS, and the Map result is about 0.35 in the epoch 24

Hello, I solved this problem, the reason is that the paper used 8 Gpus for training, and I trained on a single card, so I reduced the initial learning rate lr and weight_decay by 8 times, changed to lr=0.75e-4, weight_decay=0.00125, and then decreased the initial learning rate LR and weight_decay by 8 times. Also, enlarge the warmup_iters in lr_config by a factor of eight, to 4000

sjhfdl avatar Jun 20 '23 15:06 sjhfdl

Thanks

adasfag avatar Jun 20 '23 15:06 adasfag

Hello .I train the model again as your advice,but the Map is about 45.7 in the epoch 24. Could you provicd your single GPU result?Thank you very much

adasfag avatar Jun 23 '23 14:06 adasfag

Hello .I train the model again as your advice,but the Map is about 45.7 in the epoch 24. Could you provicd your single GPU result?Thank you very much

First of all, I would like to apologize to you. Due to the computing power of my graphics card, when I adjusted the learning rate, I only trained the author's code for two epochs, and I felt that the accuracy of the second epoch had reached 0.15, so I did not continue the training. Then I went to verify my method, and the accuracy of the training was similar to the results given by the author. My idea is that the results of the multi-card run will be slightly lower than those of the single card, and then I assume that the method of the author can also run on my own computer and produce similar results as in the paper.

sjhfdl avatar Jun 23 '23 15:06 sjhfdl

Hello .I train the model again as your advice,but the Map is about 45.7 in the epoch 24. Could you provicd your single GPU result?Thank you very much

First of all, I would like to apologize to you. Due to the computing power of my graphics card, when I adjusted the learning rate, I only trained the author's code for two epochs, and I felt that the accuracy of the second epoch had reached 0.15, so I did not continue the training. Then I went to verify my method, and the accuracy of the training was similar to the results given by the author. My idea is that the results of the multi-card run will be slightly lower than those of the single card, and then I assume that the method of the author can also run on my own computer and produce similar results as in the paper.

It seems that the single result is lower than those of the multi-card, maybe it needs a more suitable lr and It confused me.

adasfag avatar Jun 23 '23 15:06 adasfag

It seems that the single result is lower than those of the multi-card, maybe it needs a more suitable lr and It confused me.

Yes, you need a good learning rate configuration, you can try it a few times, maybe because our graphics card models are different

sjhfdl avatar Jun 23 '23 15:06 sjhfdl

Hello .I train the model again as your advice,but the Map is about 45.7 in the epoch 24. Could you provicd your single GPU result?Thank you very much

Could you show me the test results of the training?

sjhfdl avatar Jun 23 '23 15:06 sjhfdl

0 214-0 5 0 498-1 0 0 659-1 0

adasfag avatar Jun 23 '23 15:06 adasfag

I have met this problem as well

lrx02 avatar Jul 01 '23 01:07 lrx02