Reproducing mAP with one gpu

[qwenqw]

Issue summary

hi, I just trained the pretrained VGG model on VOC 07+12 dataset. However, the mAP is just 74.3% on VOC2007test. I used the file "ssd_pascal.py" and trained with one GPU(gtx 1080Ti). I just changed the batch_size =8. I wonder how to reproduce the mAP 77.2% in paper. So what should I fix now? Or there are some strategies to improve the detection performance? Could you give me any suggestions?
Thank you !

The data augmentation strategies are as default. And more configurations are as follows: batch_size = 8 accum_batch_size = 8 iter_size = accum_batch_size / batch_size solver_mode = P.Solver.GPU ...... solver_param = { # Train parameters 'base_lr': base_lr, 'weight_decay': 0.0005, 'lr_policy': "multistep", 'stepvalue': [320000, 400000,480000], 'gamma': 0.1, 'momentum': 0.9, 'iter_size': iter_size, 'max_iter': 480000, 'snapshot': 80000, 'display': 10, 'average_loss': 10, 'type': "SGD", 'solver_mode': solver_mode, 'device_id': device_id, 'debug_info': False, 'snapshot_after_train': True, # Test parameters 'test_iter': [test_iter], 'test_interval': 10000, 'eval_type': "detection", 'ap_version': "11point", 'test_initialization': False, }

Your system configuration

Operating system: ubuntu16.04 Compiler: CUDA version (if applicable): CUDNN version (if applicable): BLAS: Python or MATLAB version (for pycaffe and matcaffe respectively):

[qwenqw]

Ah, I forgot to mention that I uesd the pretrained VGG model (300*300). Please tell me if someone else has the same problem with me or run it properly. Thank you very much.

[CrazySssst]

forget to add expand_param in train_transform_param ? train_transform_param = { 'mirror': True, 'mean_value': [127.5, 127.5, 127.5], 'scale':0.007843, 'resize_param': { ... }, 'distort_param': { ... }, 'expand_param': { 'prob': 0.5, 'max_expand_ratio': 4.0, }, 'emit_constraint': { 'emit_type': caffe_pb2.EmitConstraint.CENTER, } }

[qwenqw]

Thanks for your reply. I check it again. The expand_param is as default in ssd_pascal.py. So I am very confused.

[CrazySssst]

why you set accum_batch_size=8 ? 8 is too small.

batch_size = 8 accum_batch_size = 32

this setting work well for me

[qwenqw]

@CrazySssst Thank you very much ! Now I train it again with accum_batch_size = 32. Hope to have a better result.

[qwenqw]

@CrazySssst Sorry to bother you again. I trained it with batch_size = 8 , accum_batch_size = 32. After 12w iterations, the mAP on VOC2007test is only 68%. And if I set accum_batch_size = 8, the mAP is 74.3%.Other configurations are as default. It's a little weird. And I really don't know how to solve it. Can you help me ? Thank you.

[liuguiyangnwpu]

@qwenqw I have trained the model on my own dataset. After I have finished the train step, but I want to calculate the mAP, can you tell me where is the source code to calculate the mAP ?

thanks !

[fanqinmin]

Hi. I have the same problem, have you solved it?

[liuguiyangnwpu]

@fanqinmin

solver_param = {
    # Train parameters
    'base_lr': base_lr,
    'weight_decay': 0.0005,
    'lr_policy': "multistep",
    'stepvalue': [80000, 100000, 120000],
    'gamma': 0.1,
    'momentum': 0.9,
    'iter_size': iter_size,
    'max_iter': 0,
    'snapshot': 0,
    'display': 10,
    'average_loss': 10,
    'type': "SGD",
    'solver_mode': solver_mode,
    'device_id': device_id,
    'debug_info': False,
    'snapshot_after_train': False,
    'show_per_class_result': True, # should switch on this status
    # Test parameters
    'test_iter': [test_iter],
    'test_interval': 5000,
    'eval_type': "detection",
    'ap_version': "11point",
    'test_initialization': True,
    }

[fanqinmin]

@weiliu89 hi, I also trained the pretrained VGG model on VOC 07+12 dataset. However, the mAP is just 68.3% on VOC2007test. I used the file "ssd_pascal.py" and trained with one GPU. So what should I fix now? Or there are some strategies to improve the detection performance? Could you give me any suggestions? Thank you !

[Xueyan0920]

@fanqinmin Referring to the Table1 of Wei Liu's paper, when training on VOC 07+12（union of VOC 2007 and VOC 2012 trainval）,SSD300 achieves 74.3% mAP on VOC2007 test. He also tells ,they achieved even better results using an improved data augementation scheme in folow-on experiments:77.2%mAP for 300*300 input.

[ujsyehao]

@fanqinmin How do you get 68.3% mAP?