coco trainning

[vsooda]

@chuanqi305 thanks for your great job.

I saw you update the README about you had ever trained on coco. I am training coco, but after 50000 iteration, the loss is still 8.0. Is that right? How many iterations you train coco? Could you please check the coco 81 setup.

ps: I had add force_color: true in the MobileNetSSD_train.prototxt, or it will coredump.

I0802 10:37:42.755049  4918 solver.cpp:243] Iteration 55210, loss = 7.94175
I0802 10:37:42.755126  4918 solver.cpp:259]     Train net output #0: mbox_loss = 7.04632 (* 1 = 7.04632 loss)
I0802 10:37:42.788318  4918 sgd_solver.cpp:138] Iteration 55210, lr = 0.000125
I0802 10:38:08.430493  4918 solver.cpp:243] Iteration 55220, loss = 7.96467
I0802 10:38:08.430521  4918 solver.cpp:259]     Train net output #0: mbox_loss = 7.99792 (* 1 = 7.99792 loss)
I0802 10:38:08.608438  4918 sgd_solver.cpp:138] Iteration 55220, lr = 0.000125
I0802 10:38:35.221209  4918 solver.cpp:243] Iteration 55230, loss = 7.33587
I0802 10:38:35.221287  4918 solver.cpp:259]     Train net output #0: mbox_loss = 7.33598 (* 1 = 7.33598 loss)
I0802 10:38:35.338119  4918 sgd_solver.cpp:138] Iteration 55230, lr = 0.000125

[chuanqi305]

@vsooda thanks for your solution of coredump，I converted the whole dataset to avoid this. The final loss of coco is about 4.0 - 5.0, I fixed the base MobileNet weights for the first 10000 iterations and it converged faster.

[vsooda]

@chuanqi305 thank you

[suzhenghang]

Hi @chuanqi305 , Did you train the mobilenet-ssd first on cocotrainval35k, then finetuning it on VOC07+12? By the way, Could you share some details about the coco training? Many thanks in advance.

[chuanqi305]

Yes，just like you said, I trained it on coco and finetuned it on VOC0712. I fixed the base MobileNet weights for the first 10000 iterations. Not so much tricky I used except tuning the learning rate.

[didulle]

Hi. Could you share the annotation files that have been used. It's easy to get in the internet the annotations for image classification. However, I didn't find how to learn a model from annotation that contains multiple objects in one single image. Thanks!!!

[liangshuang1993]

Hi, I've also tried on COCO dataset, the final loss is between 4- 5, but mAP is between 0.3-0.4, Could you tell me your mAP, thanks!

[ryusaeba]

@liangshuang1993 Hi, I also got the same result. Do you have any solution so far?

[liangshuang1993]

@ryusaeba I gave up :sob:

[jtdutta1]

@liangshuang1993 can you think as to why this is happening? Even after 50k iterations, the loss oscillates between 4 and 5 and it even sometimes reaches 3.8. But the mAP still remains 0.33.