I do coco_eval.py but the accuracy is all 0.000

[leesangjoon1]

My train command is "python train.py -c 2 -p coco159 --batch_size 8 --lr 1e-3 --num_epochs 10 --load_weights weights/efficientdet-d2.pth --head_only True" and My eval command is "python coco_eval.py -p coco159 -c 2 -w logs/coco159/efficientdet-d2_9_40.pth"

and command screen like this

running coco-style evaluation on project coco159, weights logs/coco159/efficientdet-d2_9_40.pth... loading annotations into memory... Done (t=0.00s) creating index... index created! 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 16/16 [00:25<00:00, 1.62s/it] Loading and preparing results... DONE (t=2.05s) creating index... index created! BBox Running per image evaluation... Evaluate annotation type bbox DONE (t=1.48s). Accumulating evaluation results... DONE (t=0.03s). Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.002 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.010 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.010

How to evaluate ??

please tell me

[zylo117]

You should make sure your model is well-trained.

[leesangjoon1]

You should make sure your model is well-trained.

I think I trained well I got the result.json file and did coco_eval.py

and this is my bbox_result.json file format

[ { "image_id": 29, "category_id": 3, "score": 0.8791428804397583, "bbox": [ 2258.295166015625, 4561.7685546875, 3215.476318359375, 5422.7294921875 ] }, { "image_id": 29, "category_id": 3, "score": 0.8781173825263977, "bbox": [ 2029.437744140625, 4560.53173828125, 2988.215087890625, 5420.59130859375 ] },

Is is anything problem??

[leesangjoon1]

You should make sure your model is well-trained. instances_white_test.zip

I attach my json file using training for inspecting difference with your json file .

[zylo117]

But you trained only for 40 steps? That's not enough

[leesangjoon1]

But you trained only for 40 steps? That's not enough

but I think although I trained 40 steps, at least the accuracy is more than 0%. but all the accuracy is 0%. I don't know why

[JasenWangLab]

But you trained only for 40 steps? That's not enough

but I think although I trained 40 steps, at least the accuracy is more than 0%. but all the accuracy is 0%. I don't know why

Have you solved this problem? I also encountered it. I trained 300 epoch and eval_ Coco map is 0

[zylo117]

But you trained only for 40 steps? That's not enough

but I think although I trained 40 steps, at least the accuracy is more than 0%. but all the accuracy is 0%. I don't know why

40 steps is almost the same as 0 steps, considering it takes at least dozens of epochs to make it work

[zylo117]

But you trained only for 40 steps? That's not enough

but I think although I trained 40 steps, at least the accuracy is more than 0%. but all the accuracy is 0%. I don't know why

Have you solved this problem? I also encountered it. I trained 300 epoch and eval_ Coco map is 0

it should has something to do with the dataset or the lr

[JasenWangLab]

But you trained only for 40 steps? That's not enough

but I think although I trained 40 steps, at least the accuracy is more than 0%. but all the accuracy is 0%. I don't know why

Have you solved this problem? I also encountered it. I trained 300 epoch and eval_ Coco map is 0

it should has something to do with the dataset or the lr

thank you，I will check my dataset

[auniquesun]

@zylo117 nice work! I encountered the same problem!

My dataset contains tiny birds, only 1 object class and about 66,000 images. I think the dataset is quite challenging since the size of a bird is usually <10 pixels and the resolution of an image is 1920x1080.

I trained the effidet-d8 from scratch for 10 epochs.

However, the results on my test set are also 0, shown below.

At the same time, I also train other detection models, such as faster rcnn, yolov5. Although those models show poor results, the mAP is greater than 0, at least.

I'm not sure whether it is the problem of efficientdet model or my tough dataset.

[zylo117]

don't use d7x for training!!!! you should modify the resolution of d0 or d1 so that they run on 2048x2048. Because when training d7x, you don't have enough memory, not unless you are very rich, you have a small batchsize and the training will be slow. Thus the poor result.