AP and mAP is 0

[Alemolet]

Even though I have correctly converted ground-truth .xml files and predicted .json files in .txt files. I'm running python path/to/main.py --no-plot (because it gives me an 'unpacking error' I cannot solve). Any ideas why?

[Cartucho]

can you send me a sample of the files you are using? [email protected]

[Alemolet]

I just had to customize the path-to-predicted/gt directories in your code. Now it works fine but the AP is still pretty low (0.08%) for the only class I'm using. Don't know if it is your algorithm or my training ahah

[Cartucho]

Great! ahah I'm not sure which of them is the case. However, you can have a look at the images for every single prediction and you will understand which of the cases is it (if you installed OpenCV).

[Alemolet]

Yes, I did installed OpenCV yet because of DarkFlow. What is the command to show the images? I'm running python C:\Users\Desktop\mAP-master\main.py and it shows me only the AP graph

[Cartucho]

You should have a new folder called results in the same directory

[Alemolet]

Found it. In there there is just a results.txt file with a bunch of precision and recall info. In these lists, can you tell me how to interpret the numbers? e.g. Precision=[0.50, 0.2, 0.13...] what does it mean?

[Cartucho]

Are you sure you have OpenCV and matplotlib installed? In the readme: prerequisites

Open the terminal and run the following commands:

    python -mpip install -U pip

    python -mpip install -U matplotlib

    python -mpip install -U opencv-python

[VaniOFX]

Could you share where the problem was because I am facing the same issue? Thanks!

[matteobarbieri]

@Cartucho I'm having a similar problem, I noticed a very low mAP value despite knowing (by inspecting results visually) that results seemed good.

I tested with a very simple case of two image, I'm attaching an archive of the input folder I'm using. (and of the results I'm getting)

input.zip results.zip

Thanks in advance for your assistance!

[Cartucho]

@matteobarbieri I checked and the results seem correct (if add the images there it will a lot easier for you to understand why).

For example, let's use the contact in your input. Why is it evaluated as a false positive?

ground-truth:

• contact 472 1334 513 1406

detection-results: <class_name> <confidence> <left> <top> <right> <bottom>

• contact 0.99 471 1298 508 1372

So let's calculate the IoU. First we can calculate the area of each rectangle: a_rect_gt = (513 - 472) * (1406 - 1334) = 2952 a_rect_dr = (508 - 471) * (1372 - 1298) = 2738

and their intersection is: intersection = (508 - 472) * (1372 - 1334) = 1368

IoU = intersection / (a_rect_gt + a_rect_dr - intersection) = 0.316... < 0.5

So it is indeed a false positive since the overlap is not significant.

[matteobarbieri]

@matteobarbieri I checked and the results seem correct (if add the images there it will a lot easier for you to understand why).

For example, let's use the contact in your input. Why is it evaluated as a false positive?

ground-truth:

• contact 472 1334 513 1406

detection-results: <class_name> <confidence> <left> <top> <right> <bottom>

• contact 0.99 471 1298 508 1372

So let's calculate the IoU. First we can calculate the area of each rectangle: a_rect_gt = (513 - 472) * (1406 - 1334) = 2952 a_rect_dr = (508 - 471) * (1372 - 1298) = 2738

and their intersection is: intersection = (508 - 472) * (1372 - 1334) = 1368

IoU = intersection / (a_rect_gt + a_rect_dr - intersection) = 0.316... < 0.5

So it is indeed a false positive since the overlap is not significant.

You are absolutely right, my mistake. I have to say that from the images it looked like they were much more overlapping, but math does not lie. Thanks and sorry for the waste of time!

[Cartucho]

No problem! I am trying to address all the issues but there are too many people and I am the only one coding haha, let me know if you have any suggestion

[pinks97]

i am getting the same problem of AP 0 as well as mAP

[pinks97]

could you help me please