loss drops slowly

[zgq91]

Hi: Thank you for your open source project. When I trained mb1-ssd, loss decreased very slowly after loss 3.1. test Average Precision Per-class:0.296

[dmatos2012]

Were you able to solve the issue? I do have a custom dataset, but i run into the same issue that after some time the loss plateaus and my AP is not so good either.

[sunwell1994]

I train the coco but the minimum loss is still 3.3 while mAP is terrible. Do you guys have some ideas?

[dmatos2012]

@sunwell1994 train with the default lr = 1e-3 for 160k iterations, and then continue training lr = 1e-4 for 40k iterations and finally 40k with lr = 1e-5. Since you are training COCO, objects tend to be smaller so try that.

This is all according to the SSD paper.

[heimayouxia]

@sunwell1994 @zgq91 Same issue here on my custom dataset. I have tried the training strategy as @dmatos2012 's suggested. It barely helps. Loss still drops dead slow while mAP is still ridiculously low. My TF-ssd project uses the same dataset and the model shapes up in half of the training length I spent on this project. Does anyone have a clue?