Can not reproduce the mAP of D0 with the weight provided

[yukitsuji]

Describe the bug When I ran the ./scripts/test_coco.sh with the provided weight, I can not reproduce the mAP you @thuyngch reported. Could you verify that you can reproduce the mAP with it in your environment? I just modified the path to coco2017 dataset to run the code

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.314
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.498
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.331
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.146
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.357
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.458
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.273
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.413
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.433
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.205
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.491
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.611

Reproduction

1. What command or script did you run?

./scripts/test_coco.sh

2. Did you make any modifications on the code or config? Did you understand what you have modified?

I just modified the path to coco2017 dataset to run the code

3. What dataset did you use?

coco 2017 dataset

Environment

ubuntu18.04
TorchVision: 0.5.0
OpenCV: 4.2.0
MMCV: 0.4.4
MMDetection CUDA Compiler: 10.1
PyTorch: 1.4.0
timm: 0.1.20
Python: 3.6.9
NVCC: Cuda compilation tools, release 10.1, V10.1.243

[yukitsuji]

@thuyngch Could you please tell me your training environment like above?

Training with atss_effdet_d0.py did not also reach 33.8 mAP (just 29.5 mAP). I think these results are caused by the environment differences.

[thuyngch]

Hi @yukitsuji ,

I added environment information here: https://github.com/thuyngch/ATSS-EfficientDet-PyTorch/blob/master/environment.yml

Also, I fixed eps in SyncBN in order to get mAP=33.8 with the released weight.

Please re-run it and let me know if it still has bugs. Thanks,

[yukitsuji]

Thanks! The fixed config works well for inference (now 33.8). Could you please share the training log of D0? I would like to confirm the training config and loss curve are same as you.