Yet-Another-EfficientDet-Pytorch
Yet-Another-EfficientDet-Pytorch copied to clipboard
zylo117
Low mAP and poor detection of cones dataset
Hi there,
I used an annotated cones dataset. It was annotated using supervisly. Then I used this repo to convert the annotations to COCO format. I trained effienctdet-d0 using this command python train.py -c 0 -p cones --data_path ../cones_dataset/good/ --batch_size 64 --lr 1e-5 --num_epochs 100 --load_weights weights/efficientdet-d0.pth knowing that I have 800 training and 340 validation images. While training I got a very high classification loss. I run COCO_eval.py and got the following results:
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.243 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.351 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.295 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.014 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.323 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.669 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.114 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.287 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.293 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.017 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.422 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.758
mAP are quit low, so I run efficientdet_test.py to visualise the results, and got the following image, which shows that most of the cones were not detected, and some cones have two different labels:
Does anyone have any advice on how to have better mAP and better inference results?
@IzzeddinTeeti Hi, have you tried head only true? I have got decent results on D0, D1, D2, not D3 yet:)> You should do 2 steps training 1st: head only true, 2nd: head only False. Please see the tutorials.
Probably Formula Student right? You should definitly do what @Ronald-Kray said. Additionally, try scaling the anchors down in params.yaml if you did not already. Had a major improvement with it. Despite I was using the official EfficientDet implementation on Tensorflow. In this repo the smalles anchor is 32 pixels regarding the input size, which can be too big for cones. Also I would suggest to try without any augmentation, because obviously the objects are heavily dependent on color. The "double detections" could vanish when training more epochs.
@IzzeddinTeeti Hi, have you tried head only true? I have got decent results on D0, D1, D2, not D3 yet:)> You should do 2 steps training 1st: head only true, 2nd: head only False. Please see the tutorials.
Thank you, I tried doing this, got better results but only slightly better:
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.285 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.381 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.342 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.009 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.397 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.799 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.125 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.303 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.303 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.008 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.435 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.84
Probably Formula Student right? You should definitly do what @Ronald-Kray said. Additionally, try scaling the anchors down in params.yaml if you did not already. Had a major improvement with it. Despite I was using the official EfficientDet implementation on Tensorflow. In this repo the smalles anchor is 32 pixels regarding the input size, which can be too big for cones. Also I would suggest to try without any augmentation, because obviously the objects are heavily dependent on color. The "double detections" could vanish when training more epochs.
Yes, I am a formula student. You mean anchors rations right? knowing that my input size is 640, how did you calculate the number 32?
@IzzeddinTeeti I thought you were the standard for D0 which would be 512. The smallest anchor is actually 8 but in backbone.py the anchors gets multiplied by 4.
So you could try anchors_scales: '[0.5, 0,75, 1 ]' maybe you could even add more scales than 3.
@SiBensberg, thank you so much, I have way better results after modifying the anchors_scales. I am also trying to apply your second advise which is suppressing the augmentation, but I am stuck at that.
Nice! I am happy to hear that.
Transforms for the train dataset would be here: https://github.com/zylo117/Yet-Another-EfficientDet-Pytorch/blob/15403b5371a64defb2a7c74e162c6e880a7f462c/train.py#L114-L116
as far as I understand there is just a flip augmentation implemented in this repo which you can leave the way it is. But you can obviously try implement others as they could improve your performance. I just made some bad experiences with color hue or jitters.
head_only = False can lead to better performance but you must have more data to train or it overfits
